Using Speech to Interrupt Complex Tasks

Interacting with computers using speech promises the benefit of multitasking while one’s hands and eyes are occupied by another task. Users of spoken dialogue sys- tems have not seen this technology as living up to its potential however, in part be- cause speech agents interactions largely behave like traditional interface interactions, initiated by the user. In order to harness the full multitasking benefit of speech as an interaction modality, speech agents must interactive proactively, but doing so means that agents will need to interrupt users engaged in other tasks. While general guide- lines have been proposed for the design of proactive agents in general, the design of proactive speech which interrupts has not been explored in specific. Interrupting speech from proactive agents can take design inspiration from the ways people use speech to interrupt other people, but human speech interruptions are likewise not well understood. The first study of this thesis uses a mixed methods approach to investi- gate the effect of the urgency on people’s timing and strategies for interrupting with speech. The second study complements that data by comparing the effect of urgency on interruption timings and strategies to the effect of the difficulty of the task which is interrupted. The third study uses a data-driven approach to classify the interruptible moments of a complex task in order to analyse the extent to which participants from the prior studies utilised these dynamic characteristics of the ongoing task to shape their interruptions. Finally, the fourth study applies findings from human speech inter- ruption to the design of a proactive agent to investigate the effects of human-inspired adaptivity to context on people’s perceptions of a proactive speech agent. Findings suggest that human speech interruptions are highly diverse and adaptive to context, but such adaptivity may be seen as inappropriate and inconsistent when applied to a speech agent. The implications of this research and its limitations are discussed in the closing chapter.