Software Engineering for Chatbots/Conversational Agents: Building a Testing System for Chatbots

This master's thesis presents an innovative methodology for evaluating the performance of chatbots, focusing primarily on the accuracy and response time of the bot when presented with both original and paraphrased questions. The basis of our research is the development of a robust testing system. We have introduced a unique twist to the testing process by employing paraphrased questions alongside the original ones. This distinctive approach allows us to assess the chatbot's ability to comprehend and respond accurately to questions that convey the same intent but are structured differently. The system works by feeding a set of original and paraphrased questions to the chatbot and subsequently capturing and analyzing the responses. Through this research, we aim to contribute to the broader discussion on AI behavior, particularly as it pertains to the nuances of language comprehension and the implications of these systems in diverse real-world scenarios. We believe this work lays a solid foundation for further exploration in the field of chatbot evaluation and performance optimization.