Technology Review and Redesign – Hello Barbie
I am reviewing Hello Barbie because we have had some conversations about her in class and I already had some insider knowledge about it having interned at Toy Talk last quarter. I thought it might be useful to share this information because it’s a good example of how the limits of technology inform design, sometimes in ways that seem baffling to consumers/learners who might not be aware of those limitations. Apologies for the length of this–I had so much to say!
In its simplest description, Hello Barbie is a Barbie doll that children can have conversations with. More specifically, it is an electronic doll with a hidden microphone, speaker, and wireless capability. As far as I understand, when Barbie is turned on and connected to wi-fi, she also connects to Toy Talk’s speech recognition technology (ASR) and the program Toy Talk uses to “code” the conversations. When a child talks to Barbie, the ASR makes a hypothesis about what was said, then analyzes it in the coded conversation, and Barbie responds as written.
In terms of logic, Barbie’s conversations follow a game-like structure in order to better predict what a child might be saying. For example, rather than saying, “Hello! What do you want to talk about?” Barbie defaults to taking control of the conversation and suggesting topics, such as going on a pretend horse ride or playing news reporter. She asks guiding questions in each scenario like, “What’s your horse’s name?” or “Should we go right or left?” or “Who’s your best friend?” If Barbie did not do this and let the child take charge of the conversation, she would not really work.
Part of the problem is that ASR technology is not sophisticated enough to understand children’s voices very well. The technology, developed by Microsoft, “learned” to listen largely from male gamers on XBox. Because of this, and perhaps just because children aren’t as precise with their speech, the ASR hypothesis is probably only right about 30% of the time. When Barbie does not understand what the child said, she will give a kind of non-response like, “Huh. So anyway,…” and steer the conversation back on the track she originally laid out.
That said, children can direct the conversation to some extent. The program accounts for something Toy Talk refers to as “interjections,” which are certain words or phrases that the toy is always listening for. For example, if Barbie is talking about horses and you suddenly ask her, “How old are you?” she will sometimes answer. I say sometimes because these interjections don’t work very well. They are difficult to code given that after each thing Barbie says, she will be “listening” for very specific inputs that take priority over the interjections. For example, she might interpret that you simply said your horse’s name is, “How old are you?” and reply by saying something like, “Cool! What a great name.”
Finally, as we’ve discussed, whatever the child says to Barbie is recorded and kept on Toy Talk’s server. When parents get the toy, the agree to this pretty explicitly, but they have to agree in order to use the toy. The company collects this audio data in order to analyze how the toy is working and where to make improvements. (They also showcase funny audio clips on their website, with permission.) Because the toy is essentially “live,” fixes can be made to the conversation logic on the fly and immediately pushed out to the server. I do not know if Toy Talk has other plans for this audio data. It may help the ASR “learn” to hear children’s voices better. In any case, there is so much data that hardly anyone at Toy Talk (except the intern!) is listening to all of it, or even much of it. Still, I think it does present an ethical issue, both in terms of privacy (COPPA, anyone?) and responsibility. For example, I wonder what Toy Talk would do if they overhead a child telling Barbie she was being abused, or something else that would require intervention in any other setting.
In terms of child development, I think this toy does offer some advantages, even though it is not marketed as an educational toy. First of all, the writers were careful to make Hello Barbie a better female role model than Barbie has been in the past. Hello Barbie talks about enjoying school and math class, being kind to others, trying your best, and aspiring to be a scientist. Talking with Hello Barbie might also expose children to new vocabulary and storytelling structures to support language development. While there is not any overt curriculum in Barbie’s conversations, she will occasionally ask questions that take a certain amount of thought or social-emotional intelligence to answer, like what to do if you’re nervous for a test, or have trouble finishing your homework. Barbie also often expresses happiness to talk to the child, which may give children a greater sense of worth.
That said, Hello Barbie could be redesigned to better support children’s development. At the very least, Barbie’s conversations could be coded with more curriculum. I don’t mean that Hello Barbie should ask the child, “What is 2 + 2?” and listen for “four.” Rather, her conversations could be designed to really encourage children to think deeply or make new connections about things that impact their lives. The conversations could even be scaffolded in a way to build upon previous knowledge. From the audio data that I did listen to, I noticed that children were eager to talk when the toy or character needed help. Hello Barbie was always fairly competent, and at times it felt like she didn’t need the child at all, even if she expressed being the child’s “best friend.” Perhaps if Barbie regarded the child as the expert, children might feel more “listened to.” I would also be curious to study how much it matters to children whether or not they feel like Barbie heard exactly what they said, or if children simply want to talk to her and have her support.
Another small way to re-design the toy would be to think about the toy object itself. There is no real reason that Hello Barbie is a doll because her conversations have nothing to do with her being a moveable “body.” I think the designers could have taken better advantage of using the actual doll body in concert with the conversational feature. For example, Barbie could ask the child to make her walk, sense movement, and then ask the child where they are going. Right now, Hello Barbie’s body is little more than a telephone receiver. Encouraging the child to play with the doll like a traditional doll might put the child more at the helm of their own play and learning, rather than waiting on Barbie to find out what happens next.
I am not sure how to solve or redesign the problem of the audio data itself. I don’t know enough about ASR technology to understand whether or not there is a way around this issue, but I don’t know that there is one. Perhaps simply making it more explicit to parents that the audio is being recorded and that they can go into their account and erase it at any time would be the best approach to protect children.
A more radical re-design of Hello Barbie is one that I am actually thinking about doing some version of for my Master’s project. In this re-design, children would have access to the same software that the Toy Talk writers use to code the toy. Of course, the program would be more kid-facing so that young children could use it on their own. This would largely impact the complexity of the toy as a conversational tool, but it would give children the power to voice the toy themselves, rather than the other way around. Of course, there are limitations with this as well since the child would need to either choose only from pre-recorded lines, or else record the dialogue in their own voice. Still, I think this kind of play activity would require more thought and imagination from the child than simply answering Hello Barbie’s “yes or no” questions.
Overall, I think Hello Barbie and Toy Talk’s technology have a lot of potential as a educational tool, but it is not fully being realized. An offshoot of Pixar Animation Studios, perhaps Toy Talk is still thinking more in terms of entertaining children with its writing, rather than thinking from a play-first approach. I think Toy Talk will need to solve its privacy issues and conduct more in-depth research with how children interact with the technology to make sure its designs are responsible.
