Researchers have developed an AI system designed to tackle a critical obstacle to the growing deployment of robots across North America: the risk of autonomous machines making costly or dangerous decisions based on inaccurate memories of their surroundings.
MIT researchers have developed a system that enables robots to identify unreliable memories and correct mistaken descriptions of objects, addressing a critical weakness in machines expected to operate safely alongside people.
The technology, called UQ-DAAAM, evaluates how confident a robot should be about what its cameras have recorded. By comparing descriptions of the same object from different viewpoints, it can detect inconsistencies, select clearer observations and update its stored understanding of the environment.
The implications extend beyond improving a robot’s ability to answer questions. A machine that mistakes a defibrillator for a fire extinguisher, for example, could fail to retrieve essential equipment during an emergency. Similar errors could disrupt industrial assembly, warehouse operations or household tasks requiring reliable object recognition.
Researchers at the Massachusetts Institute of Technology say their approach offers a way to identify and correct these errors while a robot is moving, rather than leaving inaccurate information embedded in its memory.
UQ-DAAAM stands for Uncertainty Quantification for Describe Anything, Anywhere, at Any Moment. It builds on an earlier MIT system, DAAAM, which uses a vision-language model (VLM) to describe objects appearing in images captured by a robot’s onboard cameras.
Those descriptions are stored as captions within a three-dimensional map of the surrounding environment. The robot can subsequently consult the map to answer questions about objects, their locations and when they were observed.
However, image-based memory presents an important limitation. Objects may appear only partially, be viewed from a distance or be captured in poor lighting. Blurry images can also produce misleading descriptions that remain uncorrected as the robot continues working.
The researchers illustrate the problem with a gray chimney incorrectly identified as a squirrel on a rooftop. Although such a mistake might appear trivial, the underlying failure could become significant when a robot is asked to locate tools, navigate cluttered spaces or assist people.
With no way to correct itself, this wrong memory would persist as the robot proceeds in its environment – which could impact downstream applications.
The new framework addresses this problem by incorporating uncertainty quantification into the robot’s existing memory system. Instead of treating every stored caption as equally trustworthy, it assesses the reliability of individual descriptions and revisits those that appear questionable.
Amending memory
As a robot travels through a building or other environment, its cameras continually capture images. Many of these show the same objects from slightly different angles, creating opportunities to compare observations and identify discrepancies.
UQ-DAAAM first examines the confidence of the vision-language model in the words used to describe an object. It then evaluates whether captions generated from alternative viewpoints agree with the original description.
“We fuse both sources of uncertainty into a single number that captures how certain the VLM is in its overall understanding of the object,” said Luca Carlone, an Associate Professor in MIT’s Department of Aeronautics and Astronautics (AeroAstro), Principal Investigator in the Laboratory for Information and Decision Systems (LIDS), Director of the MIT SPARK Laboratory and the senior author of the paper presenting UQ-DAAAM.
When uncertainty crosses a predefined threshold, the system seeks additional observations that could resolve the ambiguity. Rather than indiscriminately processing every available frame, it uses a heuristic to identify the images most likely to improve its understanding.
In the chimney example, a later frame might show smoke rising from the structure, providing evidence that the original squirrel description was incorrect. The robot can then use that additional information to revise its memory.
The researchers mathematically proved that their selection technique can identify useful frames without repeatedly running the vision-language model. This reduces the computational burden, an important requirement for robots operating with limited onboard processing resources.
Carlone worked with MIT graduate students Harry Zhang and Nicolas Gorlo, the paper’s co-lead authors.
The research was supported in part by the US Army Research Laboratory and the US Office of Naval Research.
“The key challenge Nico and Harry had to overcome is how to combine VLMs with uncertainty quantification into something that is running in a fraction of a second on a robot that is moving,” Carlone said.
The ability to process uncertainty quickly is particularly relevant to mobile robots. They must interpret changing surroundings while navigating, avoiding obstacles and carrying out tasks. A system that requires lengthy analysis could interfere with those activities.
Once the framework identifies additional captions likely to reduce uncertainty, a large language model combines them into a revised description. The process gives greater weight to newer captions and those considered less ambiguous.
Remembering with confidence
In tests, this refinement process resolved ambiguities for approximately 92% of objects in the robot’s environment, requiring only two additional captions per object. The researchers also reported a 5% improvement in object-description accuracy compared with the state of the art.
The system produced more accurate answers than baseline methods when asked about objects, their locations and the last time they had been observed. These results suggest that correcting stored descriptions can improve a robot’s ability to use its surroundings as a source of reliable information.
Performing this uncertainty quantification procedure across the entire map the robot generates allows it to improve its understanding of what it has observed.
The research highlights a distinction between conversational AI and machines acting in physical environments. A chatbot can acknowledge uncertainty in a written response, but a robot may need to make an immediate decision that affects equipment, infrastructure or human safety.
“Contrary to ChatGPT, when we think about physical AI, it is not enough to simply put a disclaimer cautioning that sometimes the robot might be wrong. Allowing a robot to know what it doesn’t know is going to be essential for this technology to be deployed safely and have a positive impact on society,” said Carlone.
The team is now investigating how the framework could guide other kinds of robots, including mobile manipulators that move through spaces and physically interact with objects. Such systems must combine navigation with grasping, handling and other actions where mistaken object identification could have practical consequences.
Another direction involves identifying parts of an environment that remain poorly understood. Rather than waiting for a clearer image to appear incidentally, a robot could deliberately return to uncertain locations to gather better evidence.
The researchers are also exploring whether similar techniques can improve systems that predict how a robot’s environment may change over time. Extending uncertainty estimation beyond remembered observations could support more dependable planning and decision-making.
For organizations considering robots in workplaces, the work addresses a problem that may be less visible than advances in movement or manipulation: whether an autonomous system can recognize that its own representation of the world is incomplete or wrong.
