Abstract
Unmanned aerial vehicles or UAVs are becoming frequently used in the military and in law enforcement applications. For the military, they not only provide additional surveillance but aid in recon, combat, and protection (Marino, 2024). Drones in law enforcement are considered force multipliers by organizations like the Federal Law Enforcement Training Center, and that gives officers multifunctional tools that can assist in daily duties. However, it should be noted that when UAVs fly, the environment they are in can be unpredictable. UAVs are vulnerable to the environment and to autonomous path determination attacks which can lead to deviation from its path or crashing. While there are detection models already, many rely on raw data, ignoring real-world physics and the relationships between physics and sensors. The method presented in this paper is an anomaly detection framework that uses a reinforcement learning (RL) deep Q-network (DQN) to learn from real flight data to find normal and anomalous behaviors. In this paper, we will compare the effectiveness of using raw data and sensor fused data to train the RL. The contribution that this research adds to the existing research is the various sensor fusions created to detect malfunctions and anomalies of physical sensors. SF data involves cross-verifying data through various checks. Unlike many existing machine learning (ML) solutions which rely on raw datasets, the solution presented compares this method to normalized sensor fused data based on drone-specific aerodynamics before evaluation. After running evaluations, we found that the sensor fused and normalized model consistently achieved higher rewards during training runs compared to the raw data. The sensor fused model was also superior when it came to anomaly detection. for the DQN rewards, the total reward for the sensor fused data was over two times more than the raw data and combo fused. What was significant, however, was seeing that the combo fused data performed poorly in comparison to the sensor fused. This research has applications in military defense, law enforcement, and commercial uses. Its main purpose is malfunction detection, so it is useful for anyone who needs highly secure, tamper-proof autonomous navigation. The goal for this research is the eventual integration of this framework into UAVs so it can be used in real-time. The main goal is to integrate multi-UAV communication networks such as blockchain smart contracts where drones can monitor each other and tell operators about potential malfunctions before the drone crashes.