Conversational Agents as Reading Partners
In this project, we have designed a Conversational Agent (CA)-based audio story using Google’s voice-driven interface (Google Assistant). The CA pauses at particular points in the story and prompts children to answer an open-ended question. The CA gives feedback on the children’s responses, explaining why the answer is correct or incorrect. In cases where children fail to produce comprehensible answers (due to fuzzy pronunciation or a lack of comprehension of the prompt), the CA rephrases the original question in a multiple-choice format.
A two-month-long field study was conducted to iteratively revise the conversational design of the agent. A two-by-two factor experiment (N = 120) was then carried out to examine the effectiveness of the revised agent, with the two factors being the narrator (the CA or a human researcher) and the inclusion of dialogue.
Findings
A two-month-long field study was conducted to iteratively revise the conversational design of the agent. A two-by-two factor experiment (N = 120) was then carried out to examine the effectiveness of the revised agent, with the two factors being the narrator (the CA or a human researcher) and the inclusion of dialogue.
Findings
- Dialogue with an agent reading partner promoted children's story comprehension and engagement equally as well as a human reading partner did
- Children's communication patterns differed when they interacted with a human vs. agent:
- The agent partner promoted the audible intelligibility of children's responses
- Reading with a human partner elicited longer, more lexically diverse, and more relevant responses from children
- Children had varying patterns of communicating with the agents based on their age and language proficiency
- Children as young as 3 were able to understand the conversational agent and make themselves understood by it. In fact, they even spoke more clearly to the conversational agent, correctly perceiving the need to do so. Children also felt very positively about interacting with the conversational agent and wanted to continue learning that way.
- This series of studies has produced practical design heuristics that inform the development of conversational technologies for young users who are still developing their language skills and cognitive functioning
Future Work
To extend this project, we have initiated a collaboration with Sesame Workshop to develop conversational e-books featuring a popular Sesame character as children's dialogic partners. This work is innovative in three important ways:
To extend this project, we have initiated a collaboration with Sesame Workshop to develop conversational e-books featuring a popular Sesame character as children's dialogic partners. This work is innovative in three important ways:
- The conversational agents are designed to promote parent-children interaction during shared story reading, allowing me to examine the benefits of "trialogue" among an agent, parent, and the child for language development.
- The conversational e-books are intended to be culturally relevant to Latino communities. I will engage in a series of a co-design processes with families in a low-income Spanish-speaking Latino community.
- The conversational agents will be capable of code-switching, comprehending and responding to multilingual children's mixed language use. This will provide groundbreaking findings for conversational systems to move beyond monolingual interactions.
Funders
