Facebook’s Smart Home Robots: The Next Step in IoT?

Smart homes are getting more helpful with the addition of new technologies and constantly improving AI assistants. We can now use our voice to control our lights, music, car charging, door locks, and much more. We can even have a set type of APIs connected together to form a voice-enabled assistant that informs us about the weather, Internet searches, news, and more.

But these capabilities are rudimentary compared to the questions that most of us have to figure out on our own every day: “Where did I leave my phone?”, “How much cilantro is left in the fridge?”, “When did Scott leave the house for school? Or has he not left yet?”, and even “What was our electricity bill last month?”

San Francisco-headquartered social network giant Facebook is working on its own artificial intelligence (AI) algorithms that will allow us to use home assistants in a new way: By giving them a body and enabling them to tackle more complex questions like the ones mentioned above. Called “embodied AI” by the company, the smart robots will be able to remember things, help you track down lost items, and even complete tasks for you.

A New Ecosystem of Devices

Can you imagine smart home assistants working in tandem with robots, all while you lounge around on the couch or get actual things done instead of spending hours cleaning, organizing, and planning? Smart homes are the product of connected devices and sensors working together with the help of software; this Internet of Things (IoT) application hasn’t taken off in full force yet, but the potential is nearly endless for the types of features and inputs we can build into this ecosystem. We could potentially create a solution for almost any question or thought we’ve ever had in our homes.

Facebook’s announcement about the advancement of their embodied AI mentioned that the AI navigates using audio-visual cues. This means smart assistants are a necessary part of the company’s embodied AI technology. Facebook points out that most robots are “deaf” in today’s age. They only use their sight and other visual indicators (like peripheral radar) to guide their movement.

But Facebook is training their embodied AI with audio and visual data, allowing the robots to detect and follow things that make noise and use sound to map out a physical space. The company is using the SoundSpaces dataset to train its AI. The dataset simulates noise and sounds you would hear in an indoor environment, like running water, a TV playing, doors opening and closing, and phones ringing.

Sounds obviously vary in volume and vibration depending on where the listener is located and where the source of the sound is located. Sound can also range even more when it’s dampened by an object on top of it. This training data helps the embodied AI make a geometrically-detailed map of its surrounding space and the rest of the house. According to Facebook, this means that the AI can act upon natural commands like “go find the ringing phone”, rather than “go find the ringing phone, it’s located 25 feet north of the door.”

Mapping Out Home Life

Facebook also announced the development of a new tool called SemanticMapnet, which creates pixel-level maps of the indoor space. The tool is being used by the robots to understand and navigate indoor spaces in relation to each other. The goal for this tool is to create a map for the robots that can answer questions like “Which living room wall is the TV against?” and “How many pieces of furniture are in the bedroom?” With this knowledge in hand, robots can also create an inventory of the objects in each room, as well as each object’s position for later retrieval.

The company released similar tools named Replica and Habitat Simulator in mid-2019. Eventually, Facebook wants these technologies to be integrated into augmented reality experiences, allowing the user to use mental energy in places other than where objects are located in their home. The company’s CTO, Mike Schroepfer, says, “If you can build these systems, they can help you remember the important parts of your life.”

The Implications of an Assistant That’s Too Smart

It’s important to remember the context of these tools: they’re created by a social media company wanting access to your home, implementing robots that upload your home’s personal details and layout into their cloud-based system. It’s not clear what the data privacy implications for this technology would be. Also, would the robot be Facebook’s property or the property of the person who purchased it? Trust is a major topic of contention here, especially with Facebook’s track record.

Consider also that our brains will easily forget where things are in our homes if we let them. Don’t we want to remember important objects and information about our home? What if something happened to the robot and it became useless? What would we do then?

It’s imperative that, although the company wants us to forget, we don’t forget the most important things in and around our homes. It could be detrimental to fully turn over everything to a robot. Even though the technology aims to make our lives easier, there are other ways we could see an improvement in our quality of life, and a memorizing robot may not be a part of that plan — at least, not right now.

AI applications (and most other technologies) often ultimately uncover unintended and unforeseen consequences, and this type of tool, as obtrusive and personal as it is, would be no exception. Suffice it to say that there would be quite a few news articles about the good and bad surrounding Facebook’s robots.

Keeping One Eye Open

Technology isn’t here to make our lives more difficult, but we should seriously ponder what we actually need to make our lives easier. For example, Facebook robots may be great for elderly people and those with dementia to augment their memory and help them with tasks. But the majority of us will have to consider if we want a Facebook robot hovering over us while we cook, reminding us that we forgot to add the salt. As humans, we need learning experiences to inform our lives and actions, and the Facebook robots risk taking those valuable experiences in our own homes away from us.

What do you think about Facebook’s embodied AI idea? Is it controversial to you, or is it something you’re interested in getting as soon as possible? As always, let us know your thoughts in the comments below!