BoTest: a Framework to Test the Quality of Conversational Agents Using Divergent Input Examples

Files in This Item:
File Description SizeFormat 
botest-framework-test.pdf246.18 kBAdobe PDFDownload
Title: BoTest: a Framework to Test the Quality of Conversational Agents Using Divergent Input Examples
Authors: Ruane, ElayneFaure, ThéoSmith, RossBean, DanCarson-Berndsen, JulieVentresque, Anthony
Permanent link:
Date: 11-Mar-2018
Online since: 2018-04-09T09:37:35Z
Abstract: Quality of conversational agents is important as users have high expectations. Consequently, poor interactions may lead to the user abandoning the system. In this paper, we propose a framework to test the quality of conversational agents. Our solution transforms working input that the conversational agent accurately recognises to generate divergent input examples that introduce complexity and stress the agent. As the divergent inputs are based on known utterances for which we have the 'normal' outputs, we can assess how robust the conversational agent is to variations in the input. To demonstrate our framework we built ChitChatBot, a simple conversational agent capable of making casual conversation.
Funding Details: Science Foundation Ireland
metadata.dc.description.othersponsorship: Lero
Type of material: Conference Publication
Publisher: ACM
Copyright (published version): 2018 ACM
Keywords: Conversational agent testingConversational agent quality assessmentChatbot
DOI: 10.1145/3180308.3180373
Language: en
Status of Item: Peer reviewed
Is part of: IUI'18 Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion
Conference Details: ACM IUI (Intelligent User Interfaces), Tokyo, Japan, 07-11 March 2018
ISBN: 978-1-4503-5571-1/18/03
Appears in Collections:Computer Science Research Collection
PEL Research Collection

Show full item record

Citations 50

Last Week
Last month
checked on Sep 11, 2020

Page view(s)

Last Week
Last month
checked on Oct 25, 2020

Download(s) 50

checked on Oct 25, 2020

Google ScholarTM



This item is available under the Attribution-NonCommercial-NoDerivs 3.0 Ireland. No item may be reproduced for commercial purposes. For other possible restrictions on use please refer to the publisher's URL where this is made available, or to notes contained in the item itself. Other terms may apply.