Safer Internet Chatbot

Virtual assistants or conversational agents - widely known as chatbots - are becomingan increasingly pervasive part of our modern society, and are already widely used to take on tasks where permanent accessibility is beneficial. In our particular use case,an already operational German language chatbot is used to answer children’s questionsregarding chain letters. It is not a conversational chatbot, but serves a particular goal.In a typical scenario, a child asks a question and sends the received message, for which itwants to know whether it should be taken seriously or whether the message can be safely ignored. The chatbot’s task is to recognise the intent of the question, detect whether the received message represents a chain letter, and to respond appropriately and steer the conversation such that potential fears are alleviated and advice is given on how to proceed further. Throughout this work, we have improved the current German language chatbotby (i) providing an implementation based on open-source technologies, and conducteda (ii) quantitative evaluation of 120 different approaches based on machine learning,where we have combined a variety of algorithms, neural network architectures and textembedding methods. By applying transfer learning based on the BERT language model,we were able to achieve a classification performance of 0.9 for both metrics F-Score and accuracy. We have also examined (iii) the influence of emojis on the overall classification performance, where to our surprise we could not identify any clear effect. Furthermore,we have conducted (iv) a qualitative evaluation of our implementation and have therefo recompiled a questionnaire regarding the expected and actual results concerning the system behavior and performance. The feedback received through the questionnaire has beenvery positive, and it showed that we were able to increase the perceived quality of (v)intent recognition, (vi) chain letter detection and (vii) response generation.