How Pleo Works

In the words of John Sosoka, chief technical officer of Ugobe, many of today's consumer robots have "really celebrated being a robot...if you look at Roboraptor or Sony AIBO, the cool thing about them [is] that they [are] a robot." Then there's Pleo, Ugobe's new robotic life form. Pleo celebrates being a dinosaur.

Robot Image Gallery

Instead of having a boxy body and limbs with visible joints, Pleo resembles a baby Camarasaurus. Camarasaurus was a sauropod, or a large, plant-eating, four-legged dinosaur. Camarasaurus lived about 150 million years ago in what is now North America and Europe. They could grow to more than 60 feet (18 meters) in length and weigh about 40,000 pounds (18 metric tons). But babies of the species, which hatched from eggs, were small -- about the size of Pleo.

Unlike real dinosaurs, Pleo doesn't really have a gender. "It's not necessarily a he, [but "he"] is just easy for me," says Sosoka. For consistency, we'll assume Pleo is male, too. For the first five to 10 minutes of his life, Pleo acts as a hatchling, slowly opening his eyes and getting used to the light. Then, for 30 to 45 minutes, he's an infant, slowly and tentatively exploring and getting used to the world around him. After that, he's a juvenile. He grazes, walks, sniffs, plays and wiggles. He explores the world, responds to people and other Pleos, and interacts with his environment.

It takes more than just a realistic layer of artificial skin to make this process seem lifelike instead of robotic. And while Pleo has many of the same components that other robots do, they work together with a slightly different purpose -- to create the illusion of life. Instead of just creating a functional robot, they create a convincing representation of a baby dinosaur.

In this article, we'll explore the technology behind Pleo. We'll start with a look at how Pleo came to be and why he looks like a Camarasaurus instead of a T. Rex or Utahraptor.

A Baby Camarasaurus

The idea for a baby dinosaur came from Caleb Chung, co-inventor of the Furby and one of the inventors behind Pleo. "[Chung had] created tons and tons of ... other kinds of products, and he had always wanted to do a dinosaur," Sosoka says. The next step was to determine the dinosaur's species. "We wanted to do a quadruped because they are more stable," Sosoka explains.

In addition to being stable, Pleo needed to be expressive. Sosoka continues:

The ability to express and interact emotionally was one of the key things we wanted to do. In order to do that, we needed to have some features in the dinosaur ... that would make it easy for people to see emotion. The one great one is a tail ... If a dog [has] a tail, you can just see, the tail goes up when they are excited, it goes down between their legs when they are scared ... [if] they are not sure then it wags really slowly, and then they decide, "Oh yeah, it's okay!" then it wags.

The tail wasn't the only reason the designers chose to create a Camarasaurus. "It has a relatively long neck. A lot of dinosaurs don't have real long necks," says Sosoka. "And then it had a fairly large head. Pleo's head is bigger ... and it's not perfectly just scaled because there is a lot of stuff to put in there ... but the combination of wanting to have the tail, have a good-sized head, a long neck, the quadruped, led us to that particular dinosaur."

There also was plenty of research material on the Camarasaurus. They were abundant in North America, so there are lots of Camarasaurus fossils, including infant and juvenile fossils. "We could see that this would be the actual size the animal would be at somewhere around two weeks old," Sosoka says. "So they come out of this big, round egg ... then of course they grow to be huge, this gigantic thing. So by choosing this and picking this size, it's the real size that one of these would be if you could have one hatch and run around your table or living room."

Once the developers decided on a species, the next step was developing a prototype. We'll look at how Pleo went from idea to reality in the next section.

Prehistoric Consumer Robots

Robots for home use may seem novel or even futuristic, but store-bought, programmable robots have been around for a while. For example, in the 1980s, there were robots known as Omnibot and Robie Sr. You could program these robots using a remote control -- the robots stored the programs on cassette tapes. Some of today's robots, including Robosapien and Robosapien V2, are also programmable using a remote control, but they store information on digital memory.

The Pleo Prototype

It takes a lot to get from the idea of a dinosaur to a walking model, especially when the goal is for the model to be particularly lifelike. "The first idea ... was an idea on how we could create much more natural and character-based animation," Sosoka says. "We were working on developing a prototype to see if this idea would really be effective ... Over the course of a couple of months, [that] resulted in a little walker. We had a blue walker and red walker, and they were little, almost like stick-figure robots made out of normal hobby servos and things like that. But when you saw them move, it was very expressive." You can see one of these walkers in our Pleo video.

The next step was to take that basic idea of a walking robot and transform it into a walking dinosaur. Doing this involved a team of experts, and not just experts in robotics. "A lot of the things we need are very, very specialized," Sosoka says. "We have a kind of extended family of contractors ... we have to fly different people [in] from different places to do that." The team working on Pleo could change from day to day. "We have special AI talents and sensor talents and all kinds of things ... You don't know exactly which kind of people you are going to run into when you are out there."

A lot of these experts worked to make Pleo look and sound real. Sosoka explains:

Depending on when you show up at the lab, you might see a recoding session when we are doing voices for Pleo ... then we have our recording engineer that works with them ... then there are people like the sculptor who when we got the basic idea of [which] dinosaur ... Once we have the idea of what dinosaur we wanted to represent, we had a sculptor that works on all kinds of dinosaur sculptures at the La Brea tar pits and historical recreations ... he was looking for ways to make this more realistic.

The sculptor created the physical texture and shape of Pleo's thermoplastic skin. "He made all these little patterns to bring out the little bumps on the skin, he made all these different pattern pieces," Sosoka says. A different expert created the patterns of color on Pleo's body. "Another person we had is a special-effects paint wizard," Sosoka explains.

Pleo's lifelike qualities also come from the way he moves. We'll look at the inspiration for Pleo's behavior next.

Pleo Behavior and Movement

Pleo has lots of programming that controls his movements. He can also combine programmed movements to create new movements based on his environment. That way, he can appear to have an intent or a purpose for his actions and to express emotions. But these actions don't necessarily reflect what a real Camarasaurus would do. "Our goal with Pleo is not to be a model ... of that Camarasaurus, but we wanted to be as scientifically accurate as we could," says Sosoka.

So the Ugobe team worked with paleontologists, zoologists and other specialists to create a realistic foundation for Pleo's behavior while also making the dinosaur more personable and playful. "[Caleb Chung] worked with some scientists to try to understand more about this creature and how they lived and how they ... moved and ideas about what they might have sounded like and things like that." To that research, the team added the qualities people might like in a pet. "Sometimes the character is more like a dog than a dinosaur," says Sosoka, "We sprinkled in a little more character there, but wherever we could we tried to go with science and our understanding of that animal."

Whether it's acting like a dinosaur or a dog, Pleo's behavior comes from artificial intelligence. Sosoka explains. "We have a little memory, kind of like in classic AI, like a perceptron or a little neuron. And based on what happens in Pleo's world [the values in his memory] change."

These changing values allow different Pleos to behave differently, depending on their environment. Sosoka continues:

The selection of what Pleo is going to do is based on animal models taken from the field. [We can] model some of those processes digitally, and that's how we get some of the antonymous and simple learning behavior that Pleo can exhibit. So depending on what Pleo experiences, that Pleo will make different choices in how he responds to stimuli in the future. So if the Pleo ... is always being abused or if he is an environment that a lot of scary things happen, then he will become less active, it will be harder to get him excited and things like that. On the other hand, if you have a Pleo that gets played with a lot then you know when you play with him he will be much more expressive and happy ... [He might] howl or sing.

These changes to Pleo's behavior aren't necessarily permanent. As Sosoka explains, "In our design, Pleo doesn't really get to be fully grown up. He starts really young, where he can barely know where you are touching him, and he starts to understand his body and then move. Pleo doesn't get past the late teenage age, so he doesn't really get that set in his ways ... he is still plastic in that way. He is still able to change."

Along with Pleo's sounds, which are played through two speakers in his body, these movements help Pleo convey the appearance of emotions, from hunger to joy. Next, we'll look at some of the specifics of getting Pleo to move expressively.

Pleo's Sensory System

It takes 14 motors to control Pleo's movements. But Pleo has to do more than just walk around aimlessly. His movements have to be purposeful, and he has to respond to people and objects. For this to happen, Pleo needs sensors -- lots of sensors.

Some of these sensors are touch sensors. "We didn't want you to feel like you were finding a sensor or pushing a button or something," Sosoka explains, "so one of the things that we did is we put eight capacitive sensors in ... capacitive touch sensors like sometimes [are] on lights ... or on your iPod." Finding a way to use these sensors was a challenge. "If you're snowboarding you go to press your iPod through your gloves, it doesn't work because it's not pressure that does it. It's actually the electrical coupling with the water in your body."

This meant that Pleo's skin -- while completely necessary to make him look real -- could get in the way of the sensors. Sosoka says:

The idea was to try to tune them so that you could be a hair's breadth off of Pleo's skin, and he would detect your touch almost like he had tiny little hairs or something ... Well it turns out that it's really easy to run capacitive sensors on something like an iPod or a lamp, but it's very difficult to do capacitive sensors on something under a skin that's moving and buckling as he moves, [and] with the motors everywhere, with all these electrical interference ... so it actually ended up taking us a long time to sort out [the] interference problems.

These capacitive sensors look like thin, metal strips. They're located on Pleo's legs, back, shoulder, head and under his chin. In addition to these sensors are:

An infrared (IR) transmitter/receiver
Two microphones, located where the dinosaur's real ears would be
A color camera
An infrared interrupter, which lets him detect opaque objects in his mouth
Tilt/shake sensors, which detect changes in his position
Ground sensors on the bottoms of his feet
Force feedback sensors in the motors in his legs

Interpreting and responding to all this input requires a network of processors and circuits. On the next page, we'll look at how Pleo handles all this data.

Pleo's Processor

Getting Pleo to make purposeful movements and sounds starts with processing all of the input he receives. The eight touch sensors are a big source of information. Sosoka explains, "Each of [the touch sensors] has dedicated processing that they can do inside of a dedicated chip."

A separate processor in Pleo's head handles input from the area around that part of his body. Sosoka describes the processing power located there:

It's an ARM7 32-bit microprocessor, and it's very powerful and it's very fast ... That processor is responsible for managing the sound input through Pleo's two microphones for binaural hearing. [It also handles] infrared communication and infrared object detection and for the CMOS sensor, the camera sensing ... there is a photo interrupter in his mouth, so when we put something in its mouth ... that's opaque [to infrared light] Pleo can sense that it's there. That processor also manages the two touch sensors that are right there under his chin and on top of his head. Pleo's Head Pleo's slightly oversized head isn't just for the sake of appearance. In addition to all the sensors and processing power stored there, the head contains a motor and a gearbox to control his eyes and mouth.

Other processors help turn all this input into movement. Sosoka explains:

There are four small, 8-bit microprocessors, and those ... are used to control the motors. So they take the high-level motion description, kind of like a vector format for motion, and then they render it into the actual motion that Pleo has. They do that through the use of a servo motor system ... they do a few other housekeeping chores, but they are primarily running the 14 motors.

Pleo's final microprocessor, another ARM7 23-bit processor, governs Pleo's connections to computer data from the outside world. "It handles the SD card, it handles the USB, it manages the flash data," Sosoka says. This processor also acts as Pleo's overall brain. "It manages the talking to the head, talking to the motor controllers, it deals with kind of all the other stuff that's going on in there. That's where Pleo's personality and behavior exist."

On the next page, we'll look at how Pleo's body carries out these instructions, and we'll explore some of the challenges the development team discovered when working on Pleo's physical structure.

Pleo's Motors, Tails and Spines

Like most consumer robots, Pleo runs on batteries. These nickel metal hydride (NiMH) batteries are inflexible, and they take up a lot of room, so they have to go in the largest part of Pleo's body -- his abdomen. This made realistic movement a challenge. Sosoka explains:

When you are walking behind your dog, they are making like this S-curve. They are swaying back and forth, and it's really compelling to be able to move like that. When you put a pivot right in the middle [of a robot], you lose this wonderful big space where you could put all your boards, your batteries and everything.

The result was a removable battery pack that didn't interfere with the pivot in the middle of Pleo's back. The battery pack uses nickel metal hydride (NiMH) batteries and has a four-hour charge time for an hour of battery life.

Another challenge was Pleo's tail, which is central to the appearance of emotions. The trick was to make the tail movable while making it sturdy. Sosoka describes the tail's movement:

So there is strong steel wires, like a little marionette. They control the tail. [There are] four wires, and the horizontal ones oppose each other and the vertical ones. You can do whatever combination and curl the tail up. So those wires are pretty strong ... Picture a piece of nylon running down inside of all the vertebrae. We captured it at both the ends so that it could absorb some of the tension.

Motors move these wires in response to instructions from his processors. The motors also move Pleo's head, neck and legs. Clutches and force-feedback sensors in his legs help protect his components and provide responses to obstacles he encounters. "Pleo has clutches in all of his motors so that if he lands on his feet, his feet can give a little. They don't transmit all of the pressure. Those clutches also allow Pleo to know that you are messing with [his leg], and then he can cry out or limp," Sosoka says.

Out of the box, a Pleo has the ability to explore and learn from his environment. But Pleo also has the ability to be programmed and customized. We'll look at how this works on the next page.

Customizing Pleo

Pleo's artificial intelligence works within the context of his operating system, Life OS. Because of the capabilities of Life OS -- some of which will be implemented in a Pleo software update -- people have the opportunity to change the way Pleo sounds and behaves. The overall goal, according to Sosoka, is "to allow lots of people to implement their vision and their dreams ... one of the things we spend a lot of time on is just finding ways to make it easy for people to express their idea of how a creature like Pleo should act or create their own experiences around it."

The basic idea is that educators, performers and hobbyists could customize Pleo using a digital memory device known as an SD card. With an SD card and a card reader, you could create programs and sounds and transfer them from a computer to a Pleo. Sosoka says:

Pleo has lots of sounds that he can produce in response to different activities ... They're sound commands that can turn into lots of different sounds. [These commands have] IDs, and so one of the things that we do is we let you just record a WAV file on your computer and name it the right thing ... [and save it as] the right kind of file. And then if you put that file on an SD card and you plug it into Pleo, your Pleo will behave just the way it did before except that whenever you would normally [hear] that Pleo's original sound, Pleo will play whatever that WAV file is. So when Pleo is doing his (makes dinosaur sound) to get your attention or try to get you get to feed him or something, you can record something completely different ... [whatever] you wanted to do. "I want a burger," or "get me a croissant."

Another forthcoming customization method for Pleo involves writing a simple computer program and saving it to an SD card. Using the Pawn scripting language, which is like a simpler version of the C programming language, people will be able to create their own Pleo controls. These abilities will be part of the Pleo developer's kit (PDK).

Many of these abilities will come as firmware upgrades to existing Pleos. Users will be able to download the upgrades from the Pleo site and transfer them to their Pleo using a USB connection. These upgrades will be transferred directly to Pleo's memory, but user modifications, like new programs and sounds, will be played from SD cards.

These are all part of the future of Pleo -- and the future of Ugobe. The company hasn't revealed its next robotics project, but according to Sosoka, "It will be a lot different from Pleo ... Pleo has a big space that we can explore and end users can explore, and so we can kind of let that space mature. We'll do something with the next life form will be in a very different space."

Lots More Information

Sources

Pleo's World: Discover Pleo. http://www.pleoworld.com/discover
Sosoka, John. Personal interview. Conducted 10/3/2007.
Stemmerman, Julia. Personal communication. 10/12/2007.