scholarly journals Verifying the Safety of Autonomous Systems with Neural Network Controllers

2021 ◽  
Vol 20 (1) ◽  
pp. 1-26
Author(s):  
Radoslav Ivanov ◽  
Taylor J. Carpenter ◽  
James Weimer ◽  
Rajeev Alur ◽  
George J. Pappas ◽  
...  
Keyword(s):  
Neural Network ◽  
Autonomous Systems ◽  
Neural Network Controllers

Compositional Learning and Verification of Neural Network Controllers

2021 ◽  
Vol 20 (5s) ◽  
pp. 1-26
Author(s):  
Radoslav Ivanov ◽  
Kishor Jothimurugan ◽  
Steve Hsu ◽  
Shaan Vaidya ◽  
Rajeev Alur ◽  
...  
Keyword(s):  
Neural Network ◽  
Program Verification ◽  
Controller Design ◽  
Autonomous Systems ◽  
Arbitrary Sequence ◽  
Verification Tools ◽  
Compositional Learning ◽  
Lidar Observations ◽  
Neural Network Controllers

Recent advances in deep learning have enabled data-driven controller design for autonomous systems. However, verifying safety of such controllers, which are often hard-to-analyze neural networks, remains a challenge. Inspired by compositional strategies for program verification, we propose a framework for compositional learning and verification of neural network controllers. Our approach is to decompose the task (e.g., car navigation) into a sequence of subtasks (e.g., segments of the track), each corresponding to a different mode of the system (e.g., go straight or turn). Then, we learn a separate controller for each mode, and verify correctness by proving that (i) each controller is correct within its mode, and (ii) transitions between modes are correct. This compositional strategy not only improves scalability of both learning and verification, but also enables our approach to verify correctness for arbitrary compositions of the subtasks. To handle partial observability (e.g., LiDAR), we additionally learn and verify a mode predictor that predicts which controller to use. Finally, our framework also incorporates an algorithm that, given a set of controllers, automatically synthesizes the pre- and postconditions required by our verification procedure. We validate our approach in a case study on a simulation model of the F1/10 autonomous car, a system that poses challenges for existing verification tools due to both its reliance on LiDAR observations, as well as the need to prove safety for complex track geometries. We leverage our framework to learn and verify a controller that safely completes any track consisting of an arbitrary sequence of five kinds of track segments.

Sign in / Sign up

Export Citation Format

Share Document