A software house builds an assistant that summarizes medical discharge letters. The team is experienced, tests thoroughly for accuracy and delivers on time. In the pilot, the summaries turn out to be markedly worse for rare diseases. Nobody had set equal quality across patient groups as a development objective, so nobody had tested for it. A.6.1 asks the organization to decide, before building anything, what responsible development means to it, and to carry those objectives into every stage through defined processes.
A.6.1 is the management layer of the life cycle. The individual development steps are covered by A.6.2; A.6.1 sets the frame that every project runs in: objectives, stages, tests, approvals and the involvement of the people the result affects.
What does the standard require?
Annex A.6.1 of ISO/IEC 42001 contains two controls. The implementation guidance sits in Annex B.6.1.
- Objectives for responsible development (A.6.1.2). The organization identifies and documents objectives that steer the design and development of its AI systems, and builds measures for reaching them into the development cycle. The objectives connect to the AI objectives from clause 6.2. Annex C lists candidates such as accountability, AI expertise, availability and quality of training and test data, environmental impact, fairness, maintainability, privacy, robustness, safety, security, and transparency and explainability.
- Objectives apply in every stage. Annex B.6.1.2 illustrates this with fairness: such an objective shapes requirements, data acquisition, data conditioning, training, and verification and validation. To make that happen, the organization provides requirements and guidelines, for example a specific test method against unwanted bias.
- Defined development processes (A.6.1.3). The organization describes and documents the concrete processes it uses to design and develop AI systems responsibly. Annex B.6.1.3 lists points to consider: life cycle stages, testing requirements and planned test means, human oversight requirements with processes and tools (especially where natural persons can be affected), when impact assessments take place, rules for training data (permitted data, approved data suppliers, labelling), the expertise and training developers need, release criteria, approvals at the various stages, change control, usability and controllability, and the engagement of interested parties.
- Tailored to technology and function. Which processes apply in detail depends on the intended functionality and the AI techniques chosen. An organization may define its own stage model; ISO/IEC 22989 offers a generic one as a starting point.
In practice
One development frame for all projects. Bring objectives, stages, checkpoints and approvals together in a short document that applies to every AI project. A table with stage, entry criterion, exit criterion and responsible role has proven useful. The AI provider policy in the Starter Kit contains such a stage model, from conception to decommissioning.
Tie checkpoints to approvals. Decide where an impact assessment is due, for instance before the design is finalized and before deployment, and which results must be in place before the next stage starts. Assigning verification and approval to different people has proven useful.
Settle training data rules before they are needed. Write down which data sources are permitted, which data suppliers are approved and how data is labelled. This prevents a team under time pressure from using data of unclear origin whose rights or quality cannot be shown later.
Design human oversight in from the start. If the system affects people, the requirements should include ways to intervene and switch off, displays of limitations, and safeguards against blind trust in the output. Oversight added as an afterthought costs more and often works less well. Involve users and affected people early, for example through tests with real use cases.
Typical audit evidence
For A.6.1, auditors typically expect:
- Documented development objectives with metric, threshold and assigned stage (→ AI policy in the AI Compliance Starter Kit, section 5)
- A described stage model with entry and exit criteria (→ AI provider policy, section 8)
- Rules for training data and a list of approved data suppliers
- Test plan per project with test types, test data and acceptance thresholds (→ AI provider policy, section 13)
- Approval records per stage with the deciding person and date
- Change procedure for systems in development and operation (→ Change log)
- Training records for developers on AI-specific risks (→ Training register)
KPI
Share of AI development projects that passed every defined approval point with a documented decision
This figure shows whether the stage model is applied in daily work. The target is 100%. Gaps usually appear in rush projects and in small changes that nobody perceives as development.
Supporting KPIs:
- Share of development objectives with a measurable metric and threshold
- Number of approvals granted without complete test results (target: 0)
- Share of developers who have completed training on AI risks
Link to the AI Act
For providers of high-risk systems, A.6.1 is closely linked to the quality management system under Article 17 of the AI Act (Regulation (EU) 2024/1689). Article 17(1) requires, among other things, procedures for design, design control and design verification (point b), for development, quality control and quality assurance (point c), and for examination, test and validation before, during and after development (point d). The requirements of Articles 8 to 15, including data governance (Article 10), human oversight (Article 14) and accuracy, robustness and cybersecurity (Article 15), thereby become development requirements.
Article 14(3) expects oversight measures to be built into the system by the provider wherever feasible before it is placed on the market. The harmonized standard EN 18286 describes the quality management system for the purposes of the AI Act and builds on the same idea (→ EU AI Act).
Related controls
A.6.1 sets the frame that other controls fill:
- A.6.2 AI system life cycle: implements the frame stage by stage, from requirements to event logs.
- A.5 Assessing impacts of AI systems: delivers the assessments whose timing A.6.1.3 defines.
- A.7 Data for AI systems: goes deeper into the rules for training data.
- A.8.25 Secure Development Life Cycle: the ISO 27001 counterpart for information security in development.
Sources
- ISO/IEC 42001:2023, Annex A.6.1 and Annex B.6.1: Management guidance for AI system development; Annex C; clause 6.2
- Regulation (EU) 2024/1689 (AI Act), Articles 10, 14, 15, 17 and 25
- ISO/IEC 22989:2022, generic life cycle model, named in Annex B.6.1.3
- ISO/IEC 42001 in the Cenedril wiki: structure of the standard and its relation to the AI Act