Miscellaneous helper robots have been whirring, rolling, and bumping round for years. They often take the type of one thing lovely, with huge doe eyes and a chunky white plastic physique that’s pleasant and fully non-threatening. Or they’ll be the other, like Boston Dynamics’ agile robots that transfer in an uncanny valley-like approach however lack issues like faces. A robotic referred to as Misty falls squarely within the former class; even the identify is cute. Misty Robotics designed it to be extensible and versatile, with applied sciences like laptop imaginative and prescient on board the robotic itself, and ripe for iteration by enterprising builders. At Microsoft Construct 2019, we obtained an prolonged demonstration of Misty and an extended chat with the corporate behind it.
A person is standing in a room
Ben Edwards, Misty Robotics’ head of developer engagement, plopped Misty on a espresso desk in entrance of me. At simply a few ft tall or so, Misty wants the additional top. To reveal Microsoft’s laptop vision-enabled scene description, the robotic rolled over to me and instructed me to say “cheese,” which I did. Misty blinked its robotic doe eyes, and I heard the digitized sound of a digital camera shutter. Click on. “A person is standing in entrance of a mirror,” mentioned Misty.
That wasn’t fairly proper. I wasn’t standing, and there was no mirror in entrance of nor behind me; possibly the glass-covered portray over my shoulder was particularly reflective and precipitated the confusion. “A few of that’s a little bit bit on Microsoft, and a few of it’s on us,” Edwards chuckled. “We’re sending form of a small picture proper now, however she typically will get that it’s an individual [who is] doing one thing, or what they’re holding.”
We tried it once more. Click on. “A person is standing in a room,” mentioned Misty. Once more, technically not fairly true, however getting nearer.
“‘A person standing is a room’ is form of her go-to with Microsoft’s [scene description],” mentioned Edwards. The Misty Robotics crew has additionally hooked it as much as different cognitive providers from Amazon and IBM. The outcomes fluctuate, Edwards mentioned, however the notable piece is that Misty can connect with any comparable type of third-party service.
However Misty closely makes use of Microsoft applied sciences. Its working system is Home windows IoT Core — it makes use of Android eight (Oreo) for navigation and laptop imaginative and prescient — and C# code the place all the abilities are written. “We’re transferring in direction of .NET Core, which isn’t far off from C# anyhow. However we expect that’s going to offer us true cross-platform capabilities on Linux, Home windows, and macOS. Individuals can simply use it and get into it,” mentioned Edwards. They’ve additionally built-in with Azure Cognitive Companies and the text-to-speech engine, and there’s a Misty SDK extension for Visible Studio.
Components and items
Misty is laden with sensors, cameras, and miscellaneous laptop imaginative and prescient applied sciences, with Qualcomm Snapdragon 820 and 410 chips and the Qualcomm Snapdragon Neural Processing Engine. The opposite hardware bits are designed to be extraordinarily extensible.
Edwards and an assistant confirmed me extra of Misty’s skills, together with the way it listens once you name it and turns to face whoever’s speakeing because of its three Qualcomm Fluence Professional-powered far-field mics. When it appears at you, you’ll be able to see a line of sensors throughout its brow. The sensors are literally all one unit, made by Occipital, that features a 166° wide-angle digital camera, a 4K digital camera, and IR depth sensors, and it employs SLAM sensor fusion and Occipital’s personal Bridge Engine for spatial computing.
One of many benefits of this sensor bar is that it helps Misty navigate again to its charging station, even in the dead of night. In a pinch, although, Misty may activate the flashlight that’s mounted close to its proper “ear.” The top has capacitive contact sensors for added controls, and there’s a Nebula-style panel that you could swap out for a special kind of digital camera, or a laser pointer, or different instrument you’ll be able to suppose up. The (patent pending) three levels of freedom (3DoF) neck lets Misty emote a little bit by letting it cock its head to the aspect like a curious pet.
There are audio system within the robotic’s chest. All 4 corners of Misty’s base have time-of-flight bump sensors so it doesn’t run into objects and obstacles, nor fall off of issues like espresso tables. The arms don’t do something, though once more, they’re meant to be extensible, so one may dream up some helpful concepts with specifically designed appendages. Essentially the most helpful arm on this explicit Misty provided a cupholder.
There’s a backpack module that’s primed for the likes of a Raspberry Pi or Arduino that may add bespoke performance to Misty. “We did a demo at CES the place we had a temperature sensor on the again,” Edwards mentioned. If Misty was someplace that obtained too sizzling, it may very well be programmed to ship an alert to a dashboard. “Think about — you possibly can get readings throughout in a warehouse,” he mentioned.
A robotic’s goal
Misty has a number of neat gadgetry on board, however the usefulness of such a robotic shouldn’t be instantly clear. It’s not precisely agile, and it’s small. Nobody would mistake it for a robotic that’s designed to stack warehouse packing containers, or tow autos, for instance.
The corporate is starting to see a few of these makes use of developments for Misty, Edwards mentioned. In well being care — elder care particularly — they’re discovering that Misty could be useful for people who find themselves growing older in place. Along with easy helps like reminding somebody to take their treatment, there’s a sure stage of companionship one can glean, even from a pleasant robotic that interacts with you in primary methods, he mentioned. Extra importantly, distant members of the family can work together with a neighborhood member of the family to an extent by a robotic. Possibly you will get a notification on a dashboard in your cellphone when your beloved takes their medication, as an illustration. Edwards mentioned plenty of firms are engaged on purposes in that space.
Misty may also be helpful for constructing administration. (That is an space by which Microsoft has expressed curiosity, Edwards mentioned.) You may deploy a robotic like Misty across the store or warehouse to carry out standing reporting, or different comparable duties which might be time-consuming and thereby pricey for people to carry out.
It might appear system of IoT sensors may work simply as nicely, and arguably extra gracefully, than a little bit robotic buzzing round on treads. Edwards famous that there could be gaps in these programs, although. Additional the robotic could alleviate some issues for folks about feeling overly surveilled, like in a room that’s consistently monitored by cameras. “However a robotic may roam round and solely be current a sure period of time after which take an image [for reporting].”
There are potential safety purposes, too, like performing easy anomaly detection, and even trying to see if a given individual is meant to be in a given place at a given time. Edwards and a companion demonstrated this explicit use case for me. Misty checked out me and hollered, “Intruder! Intruder! You higher depart now. I’ve already recorded you.” Apparently, one Misty Robotics engineer has already used Misty’s anomaly detection to thwart coworkers who had been surreptitiously borrowing his instruments.
Edwards talked about sound technician had the concept to make use of Misty to roll round and assist him examine sound ranges in music venues, which may save him possibly half an hour of setup time. “We’re not ever going to consider these situations,” Edwards mentioned. “But when we get this in somebody’s palms, possibly they suppose up a killer app for some share of the inhabitants.”
Misty is at the moment fairly expensive at $2,400. It’s a tricky promote and not using a killer utility, so to talk, and Edwards is aware of this. “We predict that robots must have extra actually beneficial use circumstances in houses and places of work earlier than individuals are going to be like, ‘Oh yeah, I’ll purchase a $1,000, [or] $2,000 robotic [for] my home,” he mentioned. However the concept is that these will come as soon as builders begin dreaming up methods to make use of Misty.