Abstract
Anomaly detection plays an important role in small mobile robotic applications, such as inspection and surveillance. For anomaly detection, various approaches have been proposed based on statistical models, machine learning, symbolic reasoning, and pre-defined rule-based logic. While these approaches have been demonstrated to be effective, they do not generalize well to new observations in open-world settings, which can contain unseen anomalies that are not included in the training datasets for detection models or the pre-defined rules. In this thesis, we propose a new context-aware anomaly detection solution for small mobile robots, which leverages an adaptive strategy to integrate the powerful reasoning capability of large language models (LLMs).Specifically, our solution uses a vision-language model with defined rules to enable onboard anomaly detection for small mobile robots. When the local system has low confidence, our solution connects with LLM services to receive enhanced reasoning for further anomaly analysis. We performed experiments in 6 environments using the Yahboom ROSMASTER X3 ROS2 Robot. Our experiment results show that our LLM-enhanced solution enables the system to recognize anomalies beyond the scope of predefined scenarios or rules.