AI Standards Lab publishes research on AI safety engineering, AI governance, and standards development. This list includes both lab publications and papers co-authored by our team members.
-
Principles and Guidelines for Randomized Controlled Trials in AI Evaluation
We have published a paper setting out five principles and 33 guidelines for randomised controlled trials measuring how AI access affects human behaviour, performance, or wellbeing, filling a gap in shared methodological standards.
-
The Case for ESM3 as a General-Purpose AI Model with Systemic Risk Under the EU AI Act
We examine whether biological foundation models like ESM3 meet the EU AI Act’s criteria for systemic risk. Despite exceeding relevant compute thresholds, ESM3 falls outside current classification guidance because it does not generate human language. We propose four remedies to close this gap.
-
Scorecard of AI Benchmark Quality
We introduce the “Scorecard of AI Benchmark Quality” to grade benchmarks across a set of dimensions reflecting its quality. We then use those scores to map the benchmark to appropriate use-cases. We presented our paper at the Evaluating Evaluations (EvalEval) Workshop of ACL 2026 in San Diego.
-
Open Problems in AI Incident Governance
We examine AI incident governance across regulatory and independent frameworks, identify open problems in how incidents are defined, classified, monitored, and reported, and propose monitoring and reporting principles, guidelines, and a reporting template.
-
Recommendations for the EU AI Act Digital Omnibus Trilogue
We have published a report analysing the Council and European Parliament positions going into the EU AI Act Omnibus trilogue. We make several recommendations for the parties engaged in the trilogue. Our main concern in making these recommendations is that, while it is positive if the Omnibus can remove some administrative burdens, it should not…
-
Defining AI Models and AI Systems: A Framework to Resolve the Boundary Problem
AI regulation assigns distinct obligations to providers of AI models and AI systems, but the lack of clear, consistent definitions for “AI model” and “AI system” creates ambiguity across the value chain. This paper surveys the definitions used in academic literature and regulatory documents, and proposes conceptual and operational definitions for drawing a principled boundary…
-
Recommendations on the European Parliament Amendments to the EU AI Act in the Digital Omnibus
We have published a report analysing some of the 750+ AI Act amendments that were proposed by the European Parliament in the context of the EU AI Act Omnibus. We provide a first analysis of these amendments, highlighting specific ones that we either welcome or oppose, based on our area of expertise.
-
A Scorecard for the Quality of AI Evaluations
We have published a working draft of a Quality Scorecard for AI Evaluations, a standards-based framework for assessing the reliability, validity, and rigour of AI evaluations. The scorecard provides structured scoring across five dimensions and a classification system to match evaluations to appropriate governance and deployment contexts.
-
Recommendations on the Digital Omnibus Amendments to the EU AI Act
We analysed the Commission’s Digital Omnibus proposals for the AI Act, highlighting concerns with Article 6(4) database deletion, Article 75(1) enforcement centralisation, and Article 4a data processing rules, whilst proposing targeted amendments to address critical regulatory gaps.