An energy supplier launches an AI chatbot for tariff enquiries. The project team disbands after go-live, and responsibility ends up “somewhere in IT”. A year later the tariffs change, the chatbot keeps quoting the old prices, and nobody feels in charge of updating its knowledge base. Go-live marks the start of the longest stretch in an AI system’s life, and that stretch is often the one without any plan.
ISO/IEC 42001 states in Annex C that risk sources can arise across the entire life cycle of an AI system. As examples, the standard names flaws in design, inadequate deployment, missing maintenance and problems at decommissioning.
What is behind it?
An AI system passes through several phases: idea and design, development, verification and validation, deployment, operation with monitoring and maintenance, and finally decommissioning. Each phase can create risks, and many only show up phases later. A design flaw becomes visible in operation, a gap at decommissioning possibly years after the fact.
Typical gaps by phase
- Design. The problem is framed wrongly, requirements are incomplete, operating limits are not described, questions of safety and fairness are never asked.
- Development and testing. Design decisions are undocumented, tests run without defined criteria, the business side never signs off the result.
- Deployment. There is no deployment plan, no release criteria and no fallback, and users have not been briefed.
- Operation and maintenance. Nobody is responsible, nobody monitors. Updates and retraining happen without testing or documentation, dependencies go stale.
- Decommissioning. Data and models are left lying around or vanish despite retention duties, dependent processes break away, interfaces point to nothing.
Why the life cycle breaks
Projects have a budget and a team until go-live. Operation afterwards is a permanent task for which neither money nor ownership is often planned. Several parties are involved as well: a service provider develops, IT runs the system, a business unit uses it. Knowledge can get lost at every handover. And machine learning changes faster than classic software, because models are retrained regularly. Documentation rarely keeps pace.
Impact
The consequences are as varied as the phases. A system degrades unnoticed, an untested change goes live, a model that was switched off keeps influencing decisions through a forgotten interface. Training data is kept longer than data protection allows, or has been deleted when an authority wants to see it. Life cycle gaps stay harmless for a long time and then become expensive all at once, usually in an audit, during an incident or when a key person leaves.
Practical examples
Automatic retraining without approval. An online retailer retrains its churn prediction model automatically every month. One month, a fault in the data pipeline feeds in duplicate records. The new model goes live without release criteria, and marketing courts the wrong customers for weeks.
Forgotten decommissioning. A financial services firm replaces an old scoring model with a new one. The old model’s nightly calculation keeps running and writes values into a table that a downstream process still reads. For months, two departments decide on the basis of different scores.
Handover without documentation. An agency develops a system that classifies support requests and hands it over at the end of the project. The documentation consists of the source code. When performance declines, nobody in-house can explain how the model was trained, and the contract with the agency has expired.
Effective controls
The following ISO/IEC 42001 controls counter this threat.
Prevention:
- A.6.1 Management guidance for AI system development: The organization sets objectives for responsible development, anchors them in the development life cycle and describes its processes for design and development, so that no project has to reinvent them.
- A.6.2 AI system life cycle: Each phase has criteria: documented requirements that cover the whole life cycle, a traceable design, verification and validation and a deployment plan with release criteria.
- A.10 Third-party and customer relationships: Responsibilities across the life cycle are divided between the organization, partners, suppliers and customers and documented.
Detection:
- A.6.2 AI system life cycle: System and performance monitoring in operation, together with event logs, shows when the system moves away from its target state.
Response:
- A.6.2 AI system life cycle: Defined procedures for repairs, updates and support, up-to-date technical documentation including all changes in operation and a plan for handling failures.
- A.10 Third-party and customer relationships: Suppliers deliver the necessary documentation and are required to correct their system when it does not work as intended.
Link to the AI Act
The AI Act (Regulation (EU) 2024/1689) consistently thinks of high-risk systems in terms of their life cycle. Risk management under Article 9 is a continuous process across the entire life cycle that is reviewed and updated regularly. Under Article 17, providers run a quality management system that covers design, development, testing and changes, among other things, and they monitor their system after placing it on the market under Article 72. Article 18 obliges them to keep the documentation available for ten years. A substantial modification triggers a new conformity assessment under Article 43(4). Deployers monitor operation under Article 26(5) on the basis of the instructions for use and inform the provider of risks (→ EU AI Act).
Related threats
- T07 Immature technology / model drift: Without monitoring in operation, drift goes undetected.
- T05 System hardware failures: Missing maintenance often shows first in sensors and compute hardware.
- MU-08 Continued use despite a known malfunction: Where nobody owns operation, known faults stay unfixed.
Sources
- ISO/IEC 42001:2023, Annex C.3.6: System life cycle issues, together with Annex B.6
- Regulation (EU) 2024/1689 (AI Act), Articles 9, 17, 18, 26, 43 and 72
- ISO/IEC 42001 in the Cenedril wiki: Annex C and the risk sources for AI systems