BoTest: a Framework to Test the Quality of Conversational Agents Using Divergent Input Examples

DC FieldValueLanguage
dc.contributor.authorRuane, Elayne-
dc.contributor.authorFaure, Théo-
dc.contributor.authorSmith, Ross-
dc.contributor.authorBean, Dan-
dc.contributor.authorCarson-Berndsen, Julie-
dc.contributor.authorVentresque, Anthony-
dc.date.accessioned2018-04-09T09:37:35Z-
dc.date.available2018-04-09T09:37:35Z-
dc.date.copyright2018 ACMen_US
dc.date.issued2018-03-11-
dc.identifier.isbn978-1-4503-5571-1/18/03-
dc.identifier.urihttp://hdl.handle.net/10197/9305-
dc.descriptionACM IUI (Intelligent User Interfaces), Tokyo, Japan, 07-11 March 2018en_US
dc.description.abstractQuality of conversational agents is important as users have high expectations. Consequently, poor interactions may lead to the user abandoning the system. In this paper, we propose a framework to test the quality of conversational agents. Our solution transforms working input that the conversational agent accurately recognises to generate divergent input examples that introduce complexity and stress the agent. As the divergent inputs are based on known utterances for which we have the 'normal' outputs, we can assess how robust the conversational agent is to variations in the input. To demonstrate our framework we built ChitChatBot, a simple conversational agent capable of making casual conversation.en_US
dc.description.sponsorshipScience Foundation Irelanden_US
dc.language.isoenen_US
dc.publisherACMen_US
dc.relation.ispartofIUI'18 Proceedings of the 23rd International Conference on Intelligent User Interfaces Companionen_US
dc.rights© ACM, 2018. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in IUI'18 Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion, (2018) http://doi.acm.org/10.1145/3180308.3180373en_US
dc.subjectConversational agent testingen_US
dc.subjectConversational agent quality assessmenten_US
dc.subjectChatboten_US
dc.titleBoTest: a Framework to Test the Quality of Conversational Agents Using Divergent Input Examplesen_US
dc.typeConference Publicationen_US
dc.internal.authorcontactotheranthony.ventresque@ucd.ieen_US
dc.statusPeer revieweden_US
dc.identifier.doi10.1145/3180308.3180373-
dc.neeo.contributorRuane|Elayne|aut|-
dc.neeo.contributorFaure|Théo|aut|-
dc.neeo.contributorSmith|Ross|aut|-
dc.neeo.contributorBean|Dan|aut|-
dc.neeo.contributorCarson-Berndsen|Julie|aut|-
dc.neeo.contributorVentresque|Anthony|aut|-
dc.description.othersponsorshipLeroen_US
dc.internal.rmsid894296985-
dc.date.updated2018-03-01T22:21:01Z-
item.fulltextWith Fulltext-
item.grantfulltextopen-
Appears in Collections:Computer Science Research Collection
PEL Research Collection
Files in This Item:
File Description SizeFormat 
botest-framework-test.pdf246.18 kBAdobe PDFDownload
Show simple item record

SCOPUSTM   
Citations 50

6
Last Week
0
Last month
checked on Sep 11, 2020

Page view(s)

859
Last Week
7
Last month
checked on Nov 24, 2020

Download(s) 50

320
checked on Nov 24, 2020

Google ScholarTM

Check

Altmetric


This item is available under the Attribution-NonCommercial-NoDerivs 3.0 Ireland. No item may be reproduced for commercial purposes. For other possible restrictions on use please refer to the publisher's URL where this is made available, or to notes contained in the item itself. Other terms may apply.