TUPLES’ members papers at the 2023 ICAPS Conference

Automatic Metamorphic Test Oracles for Action-Policy Testing AUTHORS: Jan Eisenhut, Álvaro Torralba, Maria Christakis and Joerg Hoffmann

Learned action policies π can be used to make real-time decisions in dynamic environments; one simply evaluates the policy on the current state in order to obtain the next action. Yet, this raises obvious concerns regarding potential policy “bugs”, that is, undesirable or even fatal policy behavior in particular situations. Testing – searching for bugs – is a natural paradigm to address these concerns.

A central component of action-policy testing are test oracles, which are responsible for recognizing states to be bugs. In the context of this paper, this means that, given a query state t, a test oracle attempts to establish that the behavior of π on t is sub-optimal. Recent work introduced metamorphic oracles realizing this by comparing the behavior of π on state pairs where one of the states is known to be easier to solve, i.e., if π performs better on a more difficult state s than on a simpler state t, then its behavior on t must be sub-optimal so that t must be a bug.

This paper shows how to automatically design such oracles based on simulation relations between states. It introduces two oracle families of this kind: first, morphing query states t to obtain suitable s; second, maintaining and comparing upper bounds on the cost of optimal plans for the states encountered during testing. Experiments show that these new oracles can find bugs much more quickly than the existing (search-based) alternatives and that the combination of the new oracles with search-based ones almost consistently dominates all other oracles.

You can view the paper here


The International Conference on Automated Planning and Scheduling (ICAPS) is the premier forum for exchanging news and research results on the theory and applications of intelligent and automated planning and scheduling technology. ICAPS 2023 is part of the ICAPS conference series. After three years of virtual events, ICAPS 2023 will be a physical conference again. ICAPS 2023 will be held July 8-13, 2023, in Prague, Czech Republic.


Latest News

We are very pleased to announce that our coordinator, Sylvie Thiebaux, is at the prestigious IJCAI23 conference both as Area Chair and as presenter of the paper "Formal Explanations of Neural Network Policies for Planning".

The workshop for Hierarchical Planning (HPlan) was founded in 2018 and is carried out since then every year at the International Conference of Automated Planning and Scheduling (ICAPS). Among the Organizing Committee of this 2023 edition is Daniel Höller from Saarland University, member of the TUPLES project.

The 2023 ICAPS (International Conference on Automated Planning and Scheduling) featured an array of captivating workshops, and one that involved many TUPLES’ members was the Workshop on Robust Decision Making for Planning and Scheduling (RDDPS).

Rebecca Eifler and Jörg Hoffmann from TUPLES are members of the organizing and steering committees respectively. Daniel Holler, member of the TUPLES Consortium, will give an invited talk.