where data driven and algorithmic solutions meet

Trustworthy AI

Our long term objective is to help building World-class transparent, explainable, accountable and trustworthy AI, based on smarter, safer, secure, resilient, accurate, robust, reliable and dependable solutions

Decision Support Systems provide algorithmic solutions to help a human decision maker deal with complex problems. Most DSS used in real-world contexts rely on symbolic approaches: these allow a decision maker to specify, in a formal language, which decisions or actions are available, how they affect the world, and which constraints must be satisfied. Automated solvers can then search for a set of decisions (i.e. a solution) that satisfies all constraints and optimise a given objective. In contrast to this approach, pure data-driven approaches have recently emerged. They compute action policies by learning from historical decision data or autonomously experimenting with a simulator.

Symbolic approaches are more interpretable, allow the user better control, and typically enable guarantees on the level of constraint satisfaction and other desirable properties of the solution. However, symbolic approaches have trouble dealing with uncertainty, cannot deal with implicit knowledge, typically rely on (possibly inaccurate) hand-crafted models, and have limited scalability.
Pure data-driven approaches have complementary properties: they handle uncertainty well and (once learning is over) can cope with large scale problems. However, they are more opaque, they are less controllable, and guarantees on properties of the solution are typically not given.

Recently, hybrid approaches have emerged as a promising avenue to address some of the main limitations of “pure” solutions.

Unfortunately, the actual implementation of hybrid methods is challenging:

If we are adding ML models to a symbolic system, hybrid DSS may inherit some of the same drawbacks that ML models have. In particular, if a black-box ML model is employed (e.g. a Neural Network), the resulting DSS will be less interpretable, and its decisions will be ultimately harder to explain; bias in the original dataset may translate into bias in the decisions suggested by the system; vulnerabilities in the ML model may lead to constraint violations.
If we are adding symbolic components to an ML system, hybrid DSSs may inherit some of the same drawbacks the symbolic components have, including lack of scalability and high manual costs for model design.

Furthermore, for the complex models of many practical applications, explanations of their behaviour may be understandable only by someone with a strong expertise in the underlying technology; and in contrast, if hand-crafted symbolic models are not sufficiently accurate, any result based on them will have limited reliability.

The TUPLES project sets out to investigate suitable combinations of symbolic and data-driven methods in planning and scheduling that accentuate their strengths while compensating for their weaknesses.

We will establish novel technical contributions including algorithms and quantitative metrics, and define controlled environments for reproducible research, all in tight interaction with representative use cases.

The project structure is oriented by a set of target properties and stages, which will guide the development of new techniques and provide the basis of a process for stakeholder engagement and validation.

Safe AI

No harm to people or the environment

An action policy is safe if executing it does not lead the environment to a fatal or undesirable state.

In many applications, safety is paramount. For instance, an aircraft pilot assistance system cannot prescribe a course of action which would lead to run out of fuel before landing. In the presence of uncertainty, it may not be possible to completely eliminate the possibility of unsafe behavior, but the probability of this occurring must be upper-bounded by a very small threshold.

We need to be able to generate safe solutions, and, especially in the presence of uncertainty, to be able to verify and quantify the extent to which a solution is safe.

We will combine data-driven and symbolic approaches to represent, synthesize and learn action policies that are safe.

Here, data-driven approaches enable better decisions in complex environments, whereas symbolic approaches are crucial to reason about and provide guarantees about safety. We will also develop methods to verify and quantify the degree to which a given policy is safe.

All of these approaches will be demonstrated on relevant use cases provided by TUPLES partners.

The production of safe policies for planning under uncertainty, especially taking into account constraints on the probability of unsafe behavior, is an active research area in P&S but also in formal methods and control.

Yet safety for data-driven methods, in particular learned action policies represented as neural nets, remains a wide open question, and is highly challenging as it ultimately combines the state explosion problem with the complexity of analysing neural nets.

First safety verification methods have been proposed, but these are still in their infancy.

Robust AI

Stability and sensitivity in all situations

In the ML literature, robustness is typically understood as the stability of the (correct) decision under irrelevant changes in the input.

Additionally, in planning and scheduling, robustness includes sensitivity to relevant changes in the environment, and generalisation to a wide range of situations.

For all of these interpretations, it is important to produce robust solutions, to be able to measure and quantify robustness, and to formally verify that a solution is robust in a given range of situations.

We will combine ML predictive methods (e.g. deep learning, decision trees) with symbolic P&S methods (constraint programming, mathematical programming, state space search, abstraction). These hybrid methods will enable us to learn and synthesise action policies represented by neural networks or decision tree ensembles, which are robust to uncertainty and generalise well.

We will leverage the symbolic nature of planning and scheduling models and approaches to verify the robustness of solutions, and will also provide efficient methods to quantify and test policy robustness.

We will demonstrate and evaluate the robustness of our solutions on several use cases provided by TUPLES partners (WP6) where robustness is critical.

These are challenging objectives, at the forefront of current AI research. For instance, verifying robustness of deep neural nets or tree ensembles for simple “point” decisions (e.g. in image classification) is currently a hot area of research.

Verifying plans, schedules and policies represented as neural nets is largely unexplored and much harder, as one must reason about arbitrarily long chains of decisions and their effect on the state of the environment.

First methods have been proposed for safety verification, but these methods are still in their infancy and robustness has not been addressed yet at all.

Scalable AI

Crucial for real-life applications

Real-world P&S problems feature large numbers of actions, state variables, resources, and contingencies, as well as complex constraints and objectives. The ability of a P&S system to scale to such large and complex problems is critical to its applicability.

Given the computational complexity of P&S (in their simplest form, scheduling is NP-hard and planning PSPACE-hard), scalability is a major issue.
Data-driven approaches often deliver superior scalability, but they:

often require large amounts of data
suffer from the aforementioned lack of transparency, safety, and robustness.

Hybrid approaches are suited to address both.

We will pursue more scalable hybrid approaches, including:

a) decision-focused learning which combines prediction with optimisation;
b) novel neural network architectures dedicated to learning policies and cost estimators from a small number of examples;
c) self-supervised learning approaches capable of bootstrapping to solve increasingly more complex and larger problems.

We will also learn compact descriptions of P&S problems that are amenable to more efficient resolution.

We will also address scalable algorithms for testing or verifying the robustness and safety of solutions.

For explaining solutions we will reduce complexity via interactive and approximate approaches that combine statistical and symbolic techniques.

Learning to produce complex plans and schedules, learning with fewer samples and/or trials, and learning to represent the world, are regularly mentioned as some of the main challenges for deep learning. Efficiently producing sufficiently accurate explanations and the scalable verification of policies are also widely acknowledged as open problems.

These challenges are actively pursued by a number of scientific communities, including planning, reinforcement learning, knowledge representation and reasoning, constraint programming, and automated verification. Recently, the PIs have independently developed some of the most promising avenues of research to solve some of these problems in their respective communities.

TUPLES will actively cross-fertilise these approaches to yield substantive improvements in scalability for trustworthy P&S.

While not directly tied to trustworthiness, the ability of an AI system to scale up to large and complex decision making problems, and to verify and explain DSS behavior in such problems is critical for its practical applicability.

Hybrid DSS systems provide challenges to scalability, but also opportunities to solve them if the integration is carefully managed.

Improving scalability will be a major goal within the project, and will necessitate advancement of the state-of-the-art.

Explainable AI

Understand the complexity to have confidence

Understanding the “what” and the “why” behind the solution returned by an AI system is a prerequisite to trusting the system, and it is a fundamental step for the trust calibration process.

In planning and scheduling in particular, users need to understand why a solution was chosen over others amongst a very large, potentially infinite, space of possible solutions.

The presence of uncertainty, and the use of data-driven approaches to handle this uncertainty adds considerably to these challenges.

Finally, ethical and legal dimensions are also critical to trustworthy P&S systems: an explainee is morally and may be legally entitled to an explanation of any bias or injurious consequence of a plan or schedule.

We will tackle these challenges by providing effective methods to compute the weakest sufficient conditions justifying the solution proposed by a P&S system involving both data-driven and symbolic components, thereby significantly extending the current landscape of explainable AI approaches.

We will further extend these approaches to provide interactive, approximate, and relevant explanations, with the aim to reduce the complexity of the problem, to address the explainee’s beliefs and intentions, as well as to direct the trust calibration process, favoring the involvement of the explainee in the process.

Leveraging symbolic search as well as model distillation approaches, we will investigate how to explain decisions taken by the neural action policies.

Finally, we will investigate the legal aspects of what constitutes an adequate explanation in the context of hybrid P&S systems.

The bulk of the current work on explainable machine learning concerns “point” decisions rather than a series of interacting decisions as required by planning and scheduling problems.

Members of the consortium have been particularly active in explainable ML and have also been amongst the first to investigate explanations of conventional P&S methods and systems of constraints in the new XAI era.

However, both these areas still pose significant challenges, and as far as we know, their integration to explain hybrid planning and scheduling systems has not yet been explored.

Trustworthy AI

Safe AI

Robust AI

Scalable AI

Explainable AI

TRUSTWORTHY AI

Keep in touch