Options
BoTest: a Framework to Test the Quality of Conversational Agents Using Divergent Input Examples
Date Issued
2018-03-11
Date Available
2018-04-09T09:37:35Z
Abstract
Quality of conversational agents is important as users have high expectations. Consequently, poor interactions may lead to the user abandoning the system. In this paper, we propose a framework to test the quality of conversational agents. Our solution transforms working input that the conversational agent accurately recognises to generate divergent input examples that introduce complexity and stress the agent. As the divergent inputs are based on known utterances for which we have the 'normal' outputs, we can assess how robust the conversational agent is to variations in the input. To demonstrate our framework we built ChitChatBot, a simple conversational agent capable of making casual conversation.
Sponsorship
Science Foundation Ireland
Other Sponsorship
Lero
Type of Material
Conference Publication
Publisher
ACM
Copyright (Published Version)
2018 ACM
Language
English
Status of Item
Peer reviewed
Journal
IUI'18 Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion
Conference Details
ACM IUI (Intelligent User Interfaces), Tokyo, Japan, 07-11 March 2018
ISBN
978-1-4503-5571-1/18/03
This item is made available under a Creative Commons License
File(s)
Loading...
Name
botest-framework-test.pdf
Size
246.18 KB
Format
Adobe PDF
Checksum (MD5)
dc75c730e6a6475b8a1c634bcc03bf15
Owning collection
Mapped collections