Robot memory is often less reliable than it looks. Robots in homes and factories need to remember which objects are nearby and where they are. However, the AI that describes those objects is not always right. For example, a robot might label a defibrillator as a fire extinguisher. Then it could not find it in an emergency. Now MIT researchers have built a system that lets robots notice doubtful memories and fix them.

Who is working on robot memory?

Graduate students Harry Zhang and Nicolas Gorlo co-led the research. The senior author is Luca Carlone. He is an associate professor in MIT’s Department of Aeronautics and Astronautics. He also directs the MIT SPARK Lab and is a principal investigator in the Laboratory for Information and Decision Systems. The team is presenting the paper at NeurIPS, one of the largest AI conferences.

What is the robot memory project?

The new system is called UQ-DAAAM. The name stands for Uncertainty Quantification for Describe Anything, Anywhere, at Any Moment. It builds on the team’s earlier DAAAM system. That system writes detailed notes about objects and saves them in a 3D map. The new version adds one key feature. Specifically, it measures how much the robot should trust each note. This connects to earlier work on teaching robots to map large environments.

How does it fix robot memory?

First, UQ-DAAAM checks how confident the AI is about each description. Next, it compares that description with views of the same object from other angles. It then combines these checks into one uncertainty score. If the score is too high, the system picks extra camera views to clear up the confusion. Importantly, the team proved it can choose these views without rerunning the AI. That saves a lot of computing power.

After that, a language model merges the notes into one description. It gives more weight to newer and clearer notes. The whole process runs in real time on a moving robot. As a result, it cleared up confusion for about 92 percent of objects. It also improved accuracy by 5 percent over the previous best method.

MIT’s Impact

Strong robot memory is key for multistep jobs, like clearing a table or helping workers build machines. Next, the team plans to bring UQ-DAAAM to mobile robots with arms. They also want robots to revisit uncertain areas to improve their maps. For a broader look at this field, see our piece on physical AI. The full story is on MIT News.