Formal verification of neural agents in non-deterministic environments

We introduce a model for agent-environment systems where the agents are implemented via feed-forward ReLU neural networks and the environment is non-deterministic. We study the verification problem of such systems against CTL properties. We show that verifying these systems against reachability properties is undecidable. We introduce a bounded fragment of CTL, show its usefulness in identifying shallow bugs in the system, and prove that the verification problem against specifications in bounded CTL is in coNExpTime and PSpace-hard. We introduce sequential and parallel algorithms for MILP-based verification of agent-environment systems, present an implementation, and report the experimental results obtained against a variant of the VerticalCAS use-case and the frozen lake scenario.

THE BOUNDED FRAGMENT AND HYBRID LOGIC WITH POLYADIC MODALITIES

We show that the bounded fragment of first-order logic and the hybrid language with ‘downarrow’ and ‘at’ operators are equally expressive even with polyadic modalities, but that their ‘positive’ fragments are equally expressive only for unary modalities.

Σ2-collection and the infinite injury priority method

We show that the existence of a recursively enumerable set whose Turing degree is neither low nor complete cannot be proven from the basic axioms of first order arithmetic (P−) together with Σ2-collection (BΣ2). In contrast, a high (hence, not low) incomplete recursively enumerable set can be assembled by a standard application of the infinite injury priority method. Similarly, for each n, the existence of an incomplete recursively enumerable set that is neither lown nor highn-1, while true, cannot be established in P− + BΣn+1. Consequently, no bounded fragment of first order arithmetic establishes the facts that the highn and lown jump hierarchies are proper on the recursively enumerable degrees.