Zum Hauptinhalt springen
ISO/IEC 42001 · Annex A · AI control

A.6.1 Management guidance for AI system development

Updated on 6 min Reviewed by: Cenedril Editorial
ISO/IEC 42001EU AI Act

A software house builds an assistant that summarizes medical discharge letters. The team is experienced, tests thoroughly for accuracy and delivers on time. In the pilot, the summaries turn out to be markedly worse for rare diseases. Nobody had set equal quality across patient groups as a development objective, so nobody had tested for it. A.6.1 asks the organization to decide, before building anything, what responsible development means to it, and to carry those objectives into every stage through defined processes.

A.6.1 is the management layer of the life cycle. The individual development steps are covered by A.6.2; A.6.1 sets the frame that every project runs in: objectives, stages, tests, approvals and the involvement of the people the result affects.

What does the standard require?

Annex A.6.1 of ISO/IEC 42001 contains two controls. The implementation guidance sits in Annex B.6.1.

  • Objectives for responsible development (A.6.1.2). The organization identifies and documents objectives that steer the design and development of its AI systems, and builds measures for reaching them into the development cycle. The objectives connect to the AI objectives from clause 6.2. Annex C lists candidates such as accountability, AI expertise, availability and quality of training and test data, environmental impact, fairness, maintainability, privacy, robustness, safety, security, and transparency and explainability.
  • Objectives apply in every stage. Annex B.6.1.2 illustrates this with fairness: such an objective shapes requirements, data acquisition, data conditioning, training, and verification and validation. To make that happen, the organization provides requirements and guidelines, for example a specific test method against unwanted bias.
  • Defined development processes (A.6.1.3). The organization describes and documents the concrete processes it uses to design and develop AI systems responsibly. Annex B.6.1.3 lists points to consider: life cycle stages, testing requirements and planned test means, human oversight requirements with processes and tools (especially where natural persons can be affected), when impact assessments take place, rules for training data (permitted data, approved data suppliers, labelling), the expertise and training developers need, release criteria, approvals at the various stages, change control, usability and controllability, and the engagement of interested parties.
  • Tailored to technology and function. Which processes apply in detail depends on the intended functionality and the AI techniques chosen. An organization may define its own stage model; ISO/IEC 22989 offers a generic one as a starting point.

In practice

One development frame for all projects. Bring objectives, stages, checkpoints and approvals together in a short document that applies to every AI project. A table with stage, entry criterion, exit criterion and responsible role has proven useful. The AI provider policy in the Starter Kit contains such a stage model, from conception to decommissioning.

Tie checkpoints to approvals. Decide where an impact assessment is due, for instance before the design is finalized and before deployment, and which results must be in place before the next stage starts. Assigning verification and approval to different people has proven useful.

Settle training data rules before they are needed. Write down which data sources are permitted, which data suppliers are approved and how data is labelled. This prevents a team under time pressure from using data of unclear origin whose rights or quality cannot be shown later.

Design human oversight in from the start. If the system affects people, the requirements should include ways to intervene and switch off, displays of limitations, and safeguards against blind trust in the output. Oversight added as an afterthought costs more and often works less well. Involve users and affected people early, for example through tests with real use cases.

Typical audit evidence

For A.6.1, auditors typically expect:

KPI

Share of AI development projects that passed every defined approval point with a documented decision

This figure shows whether the stage model is applied in daily work. The target is 100%. Gaps usually appear in rush projects and in small changes that nobody perceives as development.

Supporting KPIs:

  • Share of development objectives with a measurable metric and threshold
  • Number of approvals granted without complete test results (target: 0)
  • Share of developers who have completed training on AI risks

For providers of high-risk systems, A.6.1 is closely linked to the quality management system under Article 17 of the AI Act (Regulation (EU) 2024/1689). Article 17(1) requires, among other things, procedures for design, design control and design verification (point b), for development, quality control and quality assurance (point c), and for examination, test and validation before, during and after development (point d). The requirements of Articles 8 to 15, including data governance (Article 10), human oversight (Article 14) and accuracy, robustness and cybersecurity (Article 15), thereby become development requirements.

Article 14(3) expects oversight measures to be built into the system by the provider wherever feasible before it is placed on the market. The harmonized standard EN 18286 describes the quality management system for the purposes of the AI Act and builds on the same idea (→ EU AI Act).

A.6.1 sets the frame that other controls fill:

Sources

Frequently asked questions

Does A.6.1 apply if we only buy in AI?

The control is aimed at organizations that develop AI systems. If you only use ready-made systems, you can exclude A.6.1 in the statement of applicability with a justification. Retraining a model on your own data or rebuilding a system for a new purpose is development, though, and for high-risk systems it can make you the provider under Article 25 of the AI Act.

Where do I find suitable development objectives?

Annex C of the standard lists possible objectives such as fairness, robustness, maintainability, privacy, security, and transparency and explainability. Pick the ones that fit your systems and risks, and give each a metric. The AI objectives under clause 6.2 form the frame from which the development objectives are derived.

Do I have to adopt the life cycle model from ISO/IEC 22989?

That is optional. Annex B.6.1.3 names it as a generic model and explicitly allows organizations to define their own stages. What matters is that every stage has clear entry and exit criteria and that the checkpoints for testing, impact assessment and approval are placed within them.

No Cookies!

This wiki collects nothing, bakes nothing and leaves nothing behind. There's nothing to consent to. Privacy doesn't get better than this.