Marcel Vinzent PHD at Saarland University in the AI Group has presented a paper on Neural Policy at the prestigious AAAI Conference. The 37th edition took place in Washington DC.
The paper is “Neural Policy Safety Verification via Predicate Abstraction: CEGAR” and is written by Marcel Vinzent, Siddhant Sharma, Joerg Hoffmann.
Neural networks (NN) are an increasingly important representation of action policies π. Recent work has extended predicate abstraction to prove safety of such π, through policy predicate abstraction (PPA) which over-approximates the state space subgraph induced by π. The advantage of PPA is that reasoning about the NN – calls to SMT solvers – is required only locally, at individual abstract state transitions, in contrast to bounded model checking (BMC) where SMT must reason globally about sequences of NN decisions.
Indeed, it has been shown that PPA can outperform a simple BMC implementation. However, the abstractions underlying these results (i.e., the abstraction predicates) were supplied manually.
In this work the authors automate this step. They extend counterexample guided abstraction refinement (CEGAR) to PPA. This involves dealing with a new source of spuriousness in abstract unsafe paths, pertaining not to transition behavior but to the decisions of the neural network π.
They introduce two methods tackling this issue based on the states involved, and they show that global SMT calls deciding spuriousness exactly can be avoided. They devise algorithmic enhancements leveraging incremental computation and heuristic search. They show empirically that the resulting verification tool has significant advantages over an encoding into the state-of-the-art model checker nuXmv. In particular, the approach used is the only in our experiments that succeeds in proving policies safe.
The work contributes to TUPLES’s goal of developing verification methods capable of reasoning about AI systems, specifically represented by neural networks. Saarland University is partner in the TUPLES project (link to the Saarland page in Tuples).
You can read the paper here.
Online collaboration is working fine but sometimes meeting face to face is really a good thing. So we did it for the third time in KU Leuven.
We are part of a broader community of research and development efforts dedicated to advancing AI in a responsible and human-centered manner.
FAI Group, part of the Saarland Informatics Campus, contribution to the TUPLES project and ongoing research efforts in the foundations of AI are significant steps forward in advancing our understanding of this technology and ensuring its safe and responsible use in the future.
We are very pleased to announce that our coordinator, Sylvie Thiebaux, is at the prestigious IJCAI23 conference both as Area Chair and as presenter of the paper "Formal Explanations of Neural Network Policies for Planning".