Zum Hauptinhalt springen
QMS risk · EN 18286

QMS-02 Undesired effects of the QMS

Updated on 6 min Reviewed by: Cenedril Editorial
EN 18286EU AI ActISO/IEC 42001

A provider with fifteen staff develops an AI system that pre-grades written exams at vocational schools. For its QMS it adopts the manual of a large corporation: forty procedures, six signatures for every change. A flaw in the grading logic is found after two days. By the time the fix has passed every station, seven weeks have gone by, and the faulty version keeps grading the whole time. At the end of every sprint, the development team fills in the required checklists in one go. The QMS ties up so much effort that it weakens the very safety it is meant to create.

EN 18286 names two directions in clause 6.1.1 in which risks to the quality management system (QMS) work. The first is missing the outcomes (QMS-01). The second, covered here, concerns the undesired effects that arise when the QMS is applied. The provider is expected to prevent them or at least reduce them.

What is behind it?

Every process in a QMS costs time, attention and staff. As long as it catches an error that would otherwise slip through, the price is justified. When the balance tips, side effects appear that undermine the purpose of the QMS. The product groups them into two forms.

Processes that slow things down

Release chains with many stations, forms for every detail, boards that meet only once a month: all of this slows development and operation. Corrections are the most delicate case. An urgent bug fix that has to take the same route as a new feature waits in line while the faulty version stays in use. Staff suffer as well: someone who spends half their time on evidence is missing from testing and monitoring.

Processes that run on paper only

The second form is quieter. The steps are performed, the fields filled in, the signatures given, and nobody really checks anything. Typical signs are risk questionnaires in which every question is answered “yes”, reviews without a single comment, or approvals granted minutes after the request. The QMS then produces paper that suggests a safety it does not deliver.

Both forms share a common root: processes designed or adopted without regard to the size, systems and risks of the provider. Duplicate structures add to this, for example when a medical device manufacturer runs its existing QMS and a second one for the AI Act side by side.

Impact

Delayed corrections extend the time during which a faulty system affects people. Article 20 requires the necessary corrective actions without delay once a system does not conform; a QMS that slows down exactly these actions stands in the way of that duty. Processes met on paper only lull the provider into a false sense of security. The damage shows up late, in an audit, in a request from an authority or after an incident, and then with records that confirm the problem more than they rebut it. On top of that comes a creeping damage: staff who experience the QMS as an obstacle look for ways around it.

Practical examples

Duplicate change management. A manufacturer of diagnostic software runs a QMS for medical devices and a separate one for the AI Act. Every model change needs two requests in two tools. After a year, the two records contradict each other: a change counts as released in one system and as open in the other.

Risk questionnaire as a formality. A provider of an AI system that plans shifts in care homes has a questionnaire with thirty questions completed for every sprint. The answers have been identical for months. When staff complain about an uneven distribution of night shifts, no questionnaire contains any hint of it, although the data would have shown it.

Blocked safety fix. A provider of a maintenance assistant for lifts discovers an error in the prediction of brake wear. The fix is small, yet like any change it has to go through the monthly release board. Three weeks pass, during which operators keep working with wrong predictions.

Effective measures

The AI Provider Policy from the AI Compliance Starter Kit addresses this risk in these places:

  • Section 2 (Purpose and scope): The extent of each aspect follows the size of the organization, and no aspect is dropped. The policy maps the existing AI management system to the requirements of Article 17, so no second management system comes into being.
  • Section 3 (Processes of the quality management system): Each process has its own monitoring that measures its results. This shows whether it has an effect or merely runs.
  • Section 12 (Development and quality assurance): Changes to the QMS are evaluated for their impact on the QMS and on every provider system before release.
  • Section 16 (Risk management): The risks of the QMS are kept in the AI Risk Register under a category of their own, and their actions are evaluated for effectiveness in the management review.
  • Section 17 (Post-market monitoring): The urgency of a correction follows the severity of the possible harm, and response times are recorded per system.
  • Section 26 (Review and maintenance): The policy is reviewed at least once a year, which is the fixed date for removing superfluous steps.

These ISO/IEC 42001 controls contribute as well:

Article 17(2) of Regulation (EU) 2024/1689 provides that the implementation of the QMS aspects is proportionate to the size of the provider’s organization. Whatever the size, the provider keeps the strictness and the protection that the conformity of its high-risk systems demands. Article 17(3) allows providers that already run a QMS under sectoral Union law to include the aspects of paragraph 1 in that system. Both provisions work directly against undesired effects. For the presumption of conformity, note that it covers Article 17(1), to the extent of Annex ZA of EN 18286; paragraphs 2 to 4 lie outside the standard (→ EN 18286: Quality Management System for AI Providers).

Sources

Frequently asked questions

May a small provider keep its QMS leaner?

Yes. Article 17(2) of the AI Act requires an implementation proportionate to the size of the organization. The strictness and the protection that the conformity of the high-risk systems demands stay in place. Each of the thirteen aspects of Article 17(1) may be lean, and none may be missing.

How do I recognize a process that is only met on paper?

By uniform entries, by checklists filled in all at once at the end of a period and by approvals granted a few minutes after the request. Another sign: the process has never produced a finding. Then it is either unnecessary or nobody performs it seriously.

How is QMS-02 connected to QMS-01?

Closely. Overloaded processes get bypassed, and then the QMS misses its outcomes. Conversely, treating QMS-01 with ever more control steps can create exactly the burden that QMS-02 describes. That is why it pays to assess both risks together.

No Cookies!

This wiki collects nothing, bakes nothing and leaves nothing behind. There's nothing to consent to. Privacy doesn't get better than this.