Our project produces a toolkit called “Automated Stress Testing for Autonomy Architectures” (ASTAA) for testing autonomy software and other software systems with similar interfaces. It feeds potentially abnormal inputs into a system until it exhibits unsafe behavior. These inputs could include out-of-range sensor and vehicle data, inputs that cause divide by zero errors, and other bad data.

The ASTAA tool then generates a report of the problems it’s found. Tracking them down and fixing them improves the robustness and safety of autonomy systems.

The ASTAA tool can be used to test both entire systems and individual components at any point during the development process. It also supports benchmarking and regression testing, which are virtually impossible under traditional testing methods. This gives engineers and programmers a chance to address potential safety problems before overall system integration, field testing, and deployment. The ASTAA tool can uncover safety problems that are unlikely to arise during typical field testing, but may occur when the system is actually put to use.

Distribution Statement A – Approved for public release; distribution is unlimited. NAVAIR Public Affairs Office tracking number 2013-74, NREC internal case number STAA-2012-10-23