Safety Critical Labs / The Framework

Thirteen requirement
areas. Every AI
failure mode covered.

The SCL AI Requirements Framework establishes verifiable, pass/fail requirements for each AI specific failure mode not addressed by existing safety critical software standards. Requirements are defined by the failures they prevent, not the capabilities they enable. Algorithm agnostic. Domain applicable. Openly published under a citable DOI.

[ Framework v3.4 ]

AI-1 through AI-13

Safety Critical
Mission Critical
Operational Support
Click a tier to filter · Esc to clear
AI-1
Operational and data foundations
A declared Operational Design Domain (ODD) bounding the certification claim, training/validation/test separation with documented provenance, and CUI and ITAR classification inheritance through AI output channels.
AI-2
Addressing AI bias
Bias risk assessment across user classes and operational contexts, training data bias evaluation, continuous bias monitoring, and bias threshold alerting and response.
AI-3
ML test coverage
A defined test matrix serving as the ML equivalent of code coverage. Covers nominal performance, edge cases, failure mode injection, and distributional boundary testing.
AI-4
Continuous validation
Post deployment data drift monitoring, performance threshold maintenance, model maintenance criteria, and periodic model validation. Addresses a gap not covered by existing safety critical software standards.
AI-5
Hallucination prevention
Hallucination criteria definition, detection, response, and logging, with independent output validation for safety critical decisions. Graceful degradation to human override below threshold.
AI-6
Out of distribution detection
Training distribution characterization, runtime OOD detection, defined OOD response, and event logging. This is the condition under which AI behavior is least predictable.
AI-7
Adversarial robustness
Data poisoning protection, adversarial input protection, model inversion and extraction protection, model integrity, adversarial event logging, and AI supply chain security.
AI-8
Explainability
Operator accessible decision reasoning with traceable decision basis, confidence indication, reasoning inspection capability, and public disclosure support. Required for any AI system where a traditional safety case would demand logical inspection.
AI-9
Human and AI teaming
Human decision authority, operational situational awareness, trust calibration, graceful degradation, interaction logging, operator qualification, workload management, and training program requirements.
AI-10
Privacy and data protection
Personal data identification and minimization, lawful basis and consent management, data subject rights, privacy enhancing techniques, and cross border transfer controls. Applies where the system processes personal data; otherwise documented Not Applicable with rationale.
AI-11
Multi-model systems
Multi-model architecture documentation, cascading failure mitigation, ensemble coordination, combined confidence representation, and multi-model audit trail. Applies when the system coordinates multiple AI models.
Conditional
AI-12
Neural network requirements
Architecture documentation, confidence calibration, neural network explainability methods, training integrity, OOD detection extensions, adversarial vulnerability mitigation, generative model hallucination constraints, and deployment transformation validation.
Conditional
AI-13
Continuous learning and adaptation
Continuous learning permission criteria, runtime learning controls, learning data validation, catastrophic forgetting mitigation, learning validation, rollback procedures, ODD evolvability declaration, and continuous learning audit trail.
Conditional
[ Requirements Defined by Failure ]

The failures these
requirements target.

Each requirement area exists because a class of AI failure has already happened in the field. These are documented public cases, each mapped to the area written to catch that failure mode. The mapping names the target. It does not claim any outcome would have changed.

Uber ATG · Tempe, 2018
An automated vehicle did not classify a pedestrian in time
The NTSB investigation found the automated driving system alternated between classifications for a pedestrian crossing outside a crosswalk, and did not brake in time to avoid a fatal collision.
AI-6The failure mode out-of-distribution detection targets.
Air Canada · Tribunal, 2024
A support chatbot stated a refund policy that did not exist
A civil resolution tribunal held the airline liable after its website assistant described a bereavement refund it did not in fact offer, and ruled the company responsible for what its bot said.
AI-5The failure mode hallucination prevention targets.
Zillow Offers · 2021
A home-pricing model drifted out of its market
Zillow wound down its algorithmic home-buying unit after model price forecasts diverged from a fast-moving housing market, taking an inventory writedown reported above 300 million dollars.
AI-4The failure mode continuous validation targets.
Amazon · Recruiting, 2018
A hiring model learned to penalize women
Amazon scrapped an internal recruiting model, reported in 2018, after finding it downgraded resumes that referenced women's activities, a bias absorbed from a decade of historical hiring data.
AI-2The failure mode addressing AI bias targets.
Cruise · San Francisco, 2023
A robotaxi mishandled the moment after a collision
California regulators suspended Cruise's driverless permits after one of its vehicles, following an initial impact from another car, executed a pull-over maneuver that dragged a pedestrian who was in its path.
AI-9The failure mode human and AI teaming targets.
Microsoft Tay · 2016
A live-learning bot was corrupted within a day
Microsoft withdrew its conversational bot Tay less than a day after release, when users exploited its habit of learning from live interactions to steer it into offensive output.
AI-13The failure mode continuous learning and adaptation targets.
See these failure modes measured

The same classes of failure, run as pass and fail measurements on public data.

[ Classification Levels ]

Three tiers. Scaled to consequence.

The framework applies all thirteen requirement areas at different depths depending on the classification of your AI system. Core requirements (AI-1 through AI-10) apply based on classification tier. Architecture and paradigm requirements (AI-11 through AI-13) apply conditionally based on system design. Classification is determined during Phase 1 of the assessment.

Tier 1
Safety Critical AI
AI outputs directly affect human safety or system survivability. All AI-1 through AI-10 apply. No tailoring without Project Safety Review Board approval. AI-11, AI-12, AI-13 apply conditionally per system design.
AI-1 AI-2 AI-3 AI-4 AI-5 AI-6 AI-7 AI-8 AI-9 AI-10 AI-11 · conditional AI-12 · conditional AI-13 · conditional
Tier 2
Mission Critical AI
AI outputs affect operational success but not human safety. AI-1 through AI-6, AI-8, AI-9, and AI-10 apply; AI-7 is tailored with Chief Engineer approval and documented rationale. AI-11, AI-12, AI-13 apply conditionally per system design.
AI-1 AI-2 AI-3 AI-4 AI-5 AI-6 AI-7 · tailored AI-8 AI-9 AI-10 AI-11 · conditional AI-12 · conditional AI-13 · conditional
Tier 3
Operational Support AI
AI supports operations but does not drive critical decisions. AI-1 through AI-6 and AI-10 apply at minimum. AI-7, AI-8, AI-9 as tailored. Tailoring permitted with Software Assurance authority approval. AI-11, AI-12, AI-13 apply conditionally per system design.
AI-1 AI-2 AI-3 AI-4 AI-5 AI-6 AI-7 · tailored AI-8 · tailored AI-9 · tailored AI-10 AI-11 · conditional AI-12 · conditional AI-13 · conditional
[ What Assessment Produces ]

A binary determination.
A documented risk score.

Assessment against this framework produces one of two outcomes, accompanied by a quantified risk score. Each determination is documented against a specific version of the framework, at a defined classification level, with every finding on record. There is no maturity index and no subjective rating.

Certified All applicable requirements met. Certificate issued with a validity period and a scheduled surveillance audit.
Not Certified One or more requirements not met. Findings documented. Remediation and re-assessment required. A Determination Document is issued regardless of outcome.
[ Published Standard ]

Read every requirement
before you engage.

The framework is openly published under a citable DOI. Every requirement, verification method, and evidence standard is available for review before any assessment begins. Verification is co-located: every requirement carries its verification, and every verification traces to a requirement.

If you believe a requirement is technically incorrect, insufficiently grounded, or missing coverage for a known AI failure mode, SCL welcomes that challenge. The standard improves through scrutiny.

Document Requirements and Verification Standards for Artificial Intelligence in Safety-Critical Applications. Version 3.4
DOI 10.5281/zenodo.20692967
All versions 10.5281/zenodo.19024420 (cite this)
License CC BY-SA 4.0. Free to read, cite, and reproduce with attribution
Download the framework Read the white paper on SSRN

Every determination follows the same four phase process.

See the process