Risk Management in AI-DLC: An ISTQB-Based Approach

1. AI-DLC Framework Context

The AI-Driven Development Life Cycle (AI-DLC) framework published by AWS (Raja SP, 2025) structures software delivery into three phases: Inception, Construction, and Operations. Large Language Models (LLMs) and multi-agent systems generate source code, draft test suites, and perform code reviews alongside human engineers.

AI-DLC is a vendor-specific methodology definition rather than an industry standard such as ISO 31000 or the ISTQB syllabus. Under this model, risk tracking extends beyond human software engineering errors and requirement gaps.

2. ISTQB Risk Measurement Standard

Section 5.2 of the ISTQB Foundation Level Syllabus defines risk as a potential event or condition that produces an adverse outcome. Risk level is calculated using two variables:

Risk Level = Likelihood * Impact

ISTQB divides risks into product risks (defects affecting functionality, performance, security, or usability) and project risks (schedule slippage, resource constraints, scope changes). Every identified risk requires a designated owner and a mitigation action to ensure accountability.

3. AI-DLC Specific Risk Categories

AI-DLC incorporates risk assessment into the Inception phase alongside non-functional requirements, with continuous evaluation throughout Construction and Operations. Five specific risk items must be tracked:

• Business-logic correctness: AI implements rules incorrectly while producing valid syntax. Automated tests written against flawed understanding fail to detect these errors, requiring domain-expert review.
• Edge-case handling correctness: Missed null checks and incorrect error paths. Automated regression and mutation testing catch these defects without domain expertise.
• Review capacity constraints: Generated code volume exceeds human review throughput, reducing code approvals to an informal sign-off.
• Model drift: LLM updates or prompt context changes alter component behavior over time without code commits. Risk scores require re-assessment triggers tied to model and prompt version updates.
• Governance artifact errors: Architectural constraints defined in steering files propagate defects across all agent-generated code when incorrectly specified.

4. Operational Risk-Based Testing Execution

Testing strategy maps directly to the three delivery phases:

• Inception: Build the risk register using ISTQB likelihood and impact metrics. Add the AI-specific risk categories as explicit items. Assign owners to high-impact items: domain experts for business logic, platform leads for review capacity, ML owners for drift, and maintainers for steering files. Link high-impact risks directly to non-functional requirements.
• Construction: Set code review depth using risk scores. High-impact business-logic changes require full manual review by domain experts prior to merging. Low-impact components run through automated regression tests combined with periodic human sampling. Track defect rates by risk category to adjust review depth based on empirical AI performance metrics.
• Operations: Monitor model drift against the baseline established during Construction. Execute re-assessments weekly for high-impact components and immediately upon model or prompt version updates. Update steering files with confirmed production defects to prevent recurring errors.

5. Risk Matrix Template

Risk Level	Likelihood	Impact	Owner	Review Cadence	Action
High	Elevated AI error rate in complex business logic	Business-critical flow	Domain expert	Every release, and on model or prompt version updates	Full human review without automated merging
Medium	Intermittent, pattern-identifiable errors	Secondary functionality	Platform or QA lead	Bi-weekly sampling	Sampled review combined with automated regression tests
Low	Stable output across prior validation cycles	Isolated component with rapid rollback	AI review owner	Monthly, or on drift alert	AI-driven review with minimal oversight

6. Operational Changes in Quality Engineering

ISTQB assumes deterministic system behavior between risk evaluations. AI-DLC introduces probabilistic output distributions that change without source code commits. Quality engineering responsibilities focus on defining risk thresholds, assigning ownership, and establishing review cadences to determine when code requires human inspection.