Safeguarded AI: TA1.1 Theory
Backed by £59M, this ARIA programme aims to develop the safety standards we need for transformational AI.
Opportunity Details
When
Registration Opens
11/04/2024
Registration Closes
28/05/2024
Award
ARIA are looking to fund this Technical Area 1.1 with up to £3.5M in total for the first year, and expect to make 10 to 16 awards in this Technical Area.
Organisation
ARIA
As AI becomes more capable, it has the potential to power scientific breakthroughs, enhance global prosperity, and safeguard us from disasters. But only if it’s deployed wisely.
Current techniques working to mitigate the risk of advanced AI systems have serious limitations, and can’t be relied upon empirically to ensure safety. To date, very little R&D effort has gone into approaches that provide quantitative safety guarantees for AI systems, because they’re considered impossible or impractical.
By combining scientific world models and mathematical proofs we will aim to construct a ‘gatekeeper’, an AI system tasked with understanding and reducing the risks of other AI agents.
In doing so we’ll develop quantitative safety guarantees for AI in the way we have come to expect for nuclear power and passenger aviation.
Our goal: to usher in a new era for AI safety, allowing us to unlock the full economic and social benefits of advanced AI systems while minimising risks.
The first solicitation for this programme focuses on TA1.1 Theory. We are looking for R&D Creators, individuals and teams that ARIA will fund and support, to research and construct computationally practicable mathematical representations and formal semantics to support world-models, specifications about state-trajectories, neural systems, proofs that neural outputs validate specifications, and “version control” (incremental updates or “patches”) thereof. Further detail is available at ARIA’s website, linked below.
Eligibility
We welcome applications from across the R&D ecosystem, including individuals,
universities, research institutions, small, medium and large companies, charities and public
sector research organisations.
Our primary focus is on funding those who are based in the UK. However, funding will be
awarded to organisations outside the UK if we believe it can boost the net impact of a
programme in the UK. If you are a non-UK applicant, you must therefore outline any
proposed plans or commitments that will contribute to the programme in the UK within the
project’s duration.