Sandia Researchers Apply Reinforcement Learning to Autonomous Drone Pursuit

Researchers at Sandia National Laboratories have successfully utilized reinforcement learning to teach autonomous drones complex, cooperative maneuvers in a simulated game of tag. This project, which was presented at the 2026 IEEE International Conference on Robotics and Automation, aims to refine algorithms for systems that must navigate, pursue, or evade targets in dynamic environments.

The research team focused on a scenario involving two pursuer drones and two evader drones, with the latter tasked with reaching a protected base. While the pursuers utilized a traditional proportional navigation algorithm to intercept targets, the evading drones relied on policies developed through reinforcement learning to coordinate their movements. This approach allowed the evaders to learn behaviors that forced the pursuers into collisions, effectively clearing a path to the objective.

Spencer Jensen, a computer scientist at Sandia and the project lead, noted that reinforcement learning offers a flexible alternative to rigid, hand-coded control logic. By treating the guidance module as a reinforcement learning problem, the team could bypass the need for exhaustive mathematical modeling of every possible interaction. The methodology functions similarly to training a pet, where virtual rewards guide the algorithm toward optimal, goal-oriented behaviors.

Christian Llanes, a doctoral student at the Georgia Institute of Technology and a Sandia intern, emphasized the complexity of reward shaping during the training phase. The team had to carefully calibrate the penalties and incentives to ensure the drones developed the desired cooperative tactics. This simulation-based training provided a foundation for the drones to achieve complex objectives without explicit programming for every potential encounter.

The researchers specifically implemented a reward function that penalized the evaders for being tagged while providing positive reinforcement for maintaining distance from pursuers and reaching the designated base. This reward shaping process required multiple iterations to ensure the agents prioritized survival and cooperation over erratic, high-speed movement. By adjusting the weight of these rewards, the team successfully encouraged the drones to utilize the pursuers’ own momentum against them.

Transitioning these algorithms from computer simulations to physical hardware presented significant challenges, primarily due to the simulation-to-reality gap. The researchers found that unmodeled dynamics—such as battery-induced rotor speed fluctuations and aerodynamic interference between drones—often degraded performance in the real world. To mitigate these discrepancies, the team conducted extensive characterization of their hardware to improve the fidelity of their simulation models.

The experimental setup utilized small, inexpensive quadrotor drones, each measuring approximately 3.5 inches in width. These units allowed for rapid iterative testing, as the physical impacts inherent in the game of tag rarely resulted in significant damage. The testing took place at the Center for Advanced Manufacturing and Innovation, which features an infrared motion-capture system to provide high-precision tracking during flight.

This work forms part of the broader AutonomyNM initiative, which seeks to advance autonomous navigation capabilities in national security applications. By refining these guidance modules on low-cost hardware, the researchers intend to reduce the performance variance between simulated training and physical deployment on high-consequence systems. Future efforts will likely focus on addressing communication latency and environmental variables that remain difficult to replicate in controlled simulations.

The reliance on reinforcement learning suggests a shift toward more adaptive autonomous systems capable of handling uncertainty. As these algorithms mature, the ability to maintain performance across the simulation-to-reality transition will remain a primary metric for success in the field. The project highlights the necessity of robust hardware-in-the-loop testing to ensure that learned behaviors hold up under physical constraints.

Ongoing research at the Center for Advanced Manufacturing and Innovation continues to prioritize these challenges. The team aims to scale these findings to more complex swarms, where the coordination requirements increase exponentially. These developments provide a framework for future autonomous defense systems that require high levels of agility and strategic cooperation in rapidly changing scenarios.

The technical implications of this research extend beyond simple pursuit-evasion games. By demonstrating that reinforcement learning can effectively manage multi-agent coordination, the team has established a baseline for more sophisticated autonomous operations. The ability to train agents in simulation and successfully deploy them on physical hardware remains a critical milestone for the broader robotics community.