ARIA: Safeguarded AI - Applications

The UK Advanced Research and Innovation Agency (ARIA)’s second funding call for Safeguarded AI solicits potential individuals or organisations interested in using their gatekeeper AI to build safeguarded products for domain-specific applications, such as optimising energy networks, clinical trials, or telecommunications networks.

Safeguarded AI’s success will depend on showing that the gatekeeper AI actually works in a safety-critical domain. The research teams selected for TA3 will work with other programme teams, global AI experts, academics, and entrepreneurs, in setting the groundwork to deploy Safeguarded AI in one or more areas.

In this first phase of funding, ARIA will allocate an initial £5.4M aimed at eliciting requirements, sourcing datasets, and establishing evaluation benchmarks for relevant cyber-physical domains. A second phase (£8.4m) will launch in approximately 1 year.

The application deadline is 2 October 2024. Successful/unsuccessful applicants will be notified on 18 November 2024. 

Watch the solicitation presentation on Vimeo.

Download the full funding call.

About the programme

The Safeguarded AI programme is a £59m-backed R&D effort to build out a general-purpose AI workflow for producing domain-specific AI agents or decision-support tools for managing cyber-physical systems with quantitative guarantees which improve upon both performance and robustness compared to existing operations.

To do this, we will employ a system that includes both state of the art, “frontier AI”, as well as human expertise to construct a gatekeeper system which monitors and ensures safe behaviour of other AI agents. This gatekeeper will consist of a formal world model and safety specifications about the application domain, and several ML components responsible for proposing effective task policies and generating verifiable safety guarantees, among others. The resulting Safeguarded AI system will unlock the raw potential of state of the art machine learning models in a wide array of business-critical or safety-critical applications domains where reliability is key.

The programme will develop the toolkit for building such a Safeguarded AI workflow, and demonstrate it in a range of applications domains such as energy, transport, telecommunication, healthcare, and more. This would, first, act as a proof of concept, proving that it’s possible to realise the benefits of AI in safety critical applications through quantitative safety guarantees; and second, catalyse further R&D to replicate and scale the results across application areas in the world.

The Safeguarded AI programme is divided into three main Technical Areas (TAs). This call for proposals is in Technical Area 3 (TA3), where we will fund teams to develop and prototype domain-specific applications of the Safeguarded AI workflow.

Who can apply?

TA3 is seeking two types of Creators (i.e. individuals and teams that ARIA will fund and support). Track 1 Creators are product developers, i.e. individuals, entrepreneurs and existing organisations (including startups, SMEs, large companies, and nonprofit R&D organisations) interested in leveraging near-future Safeguarded AI technology to develop products tailored to the needs of customers/end users in a variety of specific application domains (see section 4 for more details). As part of track 1, we are also open to funding operators who are looking to build a small team to do this R&D work in-house. Track 2 Creators are customers/end users, i.e. organisations who already want to participate in the co-creation of solutions—not by building an in-house solution R&D team, but by actively participating in the customer discovery processes run by Track 1 Creators whom we may fund in the same sector/domain.

We welcome applications from across the R&D ecosystem, including individual entrepreneurs, startups, and established companies. Our primary focus is on funding those who are based in the UK. For the vast majority of applicants, we therefore require the majority of the project work to be conducted in the UK (i.e. >50% of project costs and personnel time).

If you have any questions relating to the call, please submit your question to clarifications@aria.org.uk.

ARIA are also looking for expressions of interest from individuals or organisations to get involved in the development of an organisation spearheading the research + engineering comprising Technical Area 2 of this programme, which will develop the ML elements which harness frontier AI techniques into a general-purpose Safeguarded AI workflow. More information can be found on ARIA’s site at the link above.