MIT CSAIL Researchers Training Home Robots with iPhone Scans

Some researchers at MIT CSAIL have been using simulations based on iPhone scans to train home robots to navigate different spaces.

There are numerous reasons why there is a need for more non-vacuum robots in residential settings. The issue of unstructured and semi-structured environments is at the forefront of the list. Each residence is unique, from the layout, lighting, surfaces, and the presence of humans and pets. Even though a robot can accurately map each residence, the spaces are perpetually in a state of upheaval.

This week, researchers at MIT CSAIL are introducing a novel approach to training home robotics in simulation. An iPhone can scan a portion of one’s residence, which can be uploaded into a simulation.

Recent decades have seen the integration of simulation as a fundamental component of robot training. It enables robots to attempt and fail at tasks thousands, or even millions, of times in the same amount of time as it would take to complete them once in the actual world.

Also, the repercussions of failure in simulation are considerably less severe than in real life. Consider the possibility that instructing a robot to place a mug in a dishwasher necessitated it to damage 100 actual mugs.

“The robot can practice millions and millions of times, which is why training in the virtual world in simulation is so powerful,” explains researcher Pulkit Agrawal in a video associated with the research. “It may have caused a thousand dishes to break, but it is inconsequential, as everything was in the virtual world.”

Nevertheless, simulation is limited in accurately representing dynamic environments such as the household, much like the robots themselves. The robot’s adaptability to various environments can be significantly improved by making simulations as accessible as an iPhone scan.

The system is ultimately more adaptable when something is unavoidably out of place, whether it be a dish left on the kitchen counter or a piece of furniture being moved, as a result of creating a robust enough database of environments such as these.