You're standing at a crowded bar and the bartender asks you what you'd like to drink. Seems obvious right? You're in a drinking establishment after all. But why did he ask you that? Was he responding to something you said? Were you trying to catch his eye or waving some bills? If you're staring into space or talking to someone, he might leave you alone.
Now imagine that there was a robot behind the bar. How hard or easy would it be for the robot to read those same cues?
It's actually harder than it looks. Researchers at Austria's Bielefeld University developed a robotic bartender called James to try and recognize if a customer wants to place an order. Is the most important thing the angle the customer is standing, or how close she is to the bar? Or is it when she speaks? Each detail was fed to the robot so it can make a better judgment.
The study actually has nothing to do with eventually developing robotic bartenders — who could be worth their weight in gold if they could speed up bar service at a crowded holiday party.
Nope, the researchers used a robotic bartender to learn more about human communication. The study collected its data using a new technique dubbed the “Ghost-in-the-Machine” (GiM) paradigm, which helped the researchers determine how the human/robot used limited data and “recognizer modalities” like speech and body position of the customer to figure out how to interact and respond appropriately.
“The idea is that we put human participants ‘in the head' of the robot, provide them with the same type of information that a robot has access to, and then look at what the humans do with that information,” explains Dr. Jan “J.P.” de Ruiter in an email interview.
Thirty-one participants consulted a computer screen that held all of the relevant data (no video!), like the customer's position at the bar, visibility of the customer, angle of body and face to the bartender. The “customers” were actually recordings of customer behavior, not real-time participants. The researchers gathered the data during a trial session with James the robot at a faux bar in Munich.
The participants used the data presented in a step-by-step manner to figure out how to respond appropriately (do nothing, turn head toward customer, ask if they needed a drink) as if they were the robotic bartender. The “robot” continued to follow such prompts until a drink was successfully “served,” or the interaction ended.
“The ‘Ghost' is actually a human participant, and we can learn from them what information they need and whether the information for the robot is sufficient — if a human can't figure out what is going on, a robot certainly isn't going to,” says de Ruiter.
So, if the end game isn't a stellar martini with a metallic smile, what's the point of the study? “We are developing formal theories of social interaction. Nothing is more challenging for a theory than being implemented in a robot — one really is forced to understand what one is doing,” says de Ruiter. “And the bartender scenario was the perfect compromise: not entirely impossible, but complex enough to be interesting.”
The findings are already changing how science comprehends robot “brains.” “Contrary to what is often suggested in robotics, more information (modalities) is not always better. The humans focused on a limited number of channels, and ignored others,” explains fellow researcher Dr. Sebastian Loth via email. “In the initiation of the interaction, they primarily focused on nonverbal (body movement) information, but during the actual ordering, the speech became the dominant channel.”
Try not to be too dejected over the apparent lack of interest in robotic drink-mixing skills. The data the researchers gleaned from this study could be helpful in more meaningful ways.
“There is absolutely no need for robot bartenders as far as we know, nor is the state of the art in robotics advanced enough to make it possible to replace human bartenders,” de Ruiter says. “But the social algorithms and research methods that we develop in this project could significantly facilitate the development of other service robots.”