Zum Hauptinhalt springen
AI threat · ISO/IEC 42001 Annex C

T02 Lack of transparency and explainability

Updated on 6 min Reviewed by: Cenedril Editorial
ISO/IEC 42001EU AI Act

A regional bank uses a scoring model from an external service provider for consumer loans. A customer is turned down and asks why. The clerk sees nothing on her screen beyond a score and a red traffic light, and the provider’s documents describe the model in two paragraphs. The customer complains to the data protection authority. What the bank needs now is something it should have settled before the first use: which information it can give to whom about the system.

ISO/IEC 42001 lists a lack of transparency and explainability as a risk source in Annex C. The idea behind it is simple. An organization that cannot give interested parties adequate information about its AI system loses trustworthiness and struggles to account for the system’s results. This concerns customers as much as staff, auditors and authorities.

What is behind it?

Transparency and explainability are related, yet they mean different things. Transparency describes what is openly known about a system and its use. Explainability describes whether a single result can be justified in a way people understand. Either can be missing, and each gap has its own causes. A further question is who the explanation is for: a data scientist needs different details from a rejected applicant or an inspector from the supervisory authority.

Where transparency and explainability break down

  • Opaque models. Deep neural networks and large language models compute with millions to billions of parameters. Which features carried a given result can only be reconstructed approximately.
  • Missing documentation. Purpose, data basis, limits and known weaknesses are written down nowhere or known only to the development team. When a key person leaves, the knowledge leaves too.
  • Purchased systems. The supplier withholds details about data and model, and the contract contains no duty to provide information.
  • Wrong audience. The available information is written for specialists. Case handlers, affected people or senior management can do little with it.
  • Hidden use. People do not know that they are dealing with an AI system or that a decision rests on its output.

Impact

The first damage is to trust. Affected people find it harder to accept a decision that nobody can justify, and they push back more often. On top of that comes a problem of accountability: without an explanation, nobody can check whether a result was reached correctly, fairly and lawfully. Opacity therefore amplifies almost every other AI threat, because errors, bias and manipulation stay hidden for longer. The legal consequences range from complaints to regulatory proceedings and disputes over access rights, and inside the company uncertainty and resistance to the system grow.

Practical examples

Pre-screening of job applications. A company has an AI system sort incoming applications. A rejected candidate asks for the reasons. All HR can tell her is that the system rated her profile as a “weak match”. Whether gaps in her CV, her university or particular wording tipped the balance, nobody knows.

Diagnostic support in radiology. Software marks suspicious areas on X-ray images. The physicians see the marking, yet they learn neither how confident the system is nor for which types of images it performs worse. They therefore find it hard to weigh its hints and, depending on temperament, rely on them too much or too little.

Customer chat without labelling. An insurer answers enquiries through a chat on its website. Nowhere does it say that the answers come from a language model. When a customer relies on a wrong promise, it is unclear which documents the system drew on and who should have checked the answer.

Effective controls

The following ISO/IEC 42001 controls counter this threat.

Prevention:

  • A.6.2 AI system life cycle: The organization determines which technical documentation each group of interested parties needs and records design and development so that decisions can be traced later.
  • A.7 Data for AI systems: Provenance and preparation of the data are recorded. Without these details it is hard to explain why a model decides the way it does.
  • A.10 Third-party and customer relationships: Suppliers of AI systems deliver adequate documentation, and customers learn which domain of use a system is valid for.

Detection:

Response:

  • A.8 Information for interested parties: The organization decides which information users receive and knows its reporting obligations towards customers and authorities. When questions come in, it can answer with prepared and reviewed content.

The AI Act (Regulation (EU) 2024/1689) addresses transparency at several levels. Under Article 13, high-risk systems must be designed so that deployers can interpret their outputs and use them appropriately, and their instructions for use describe capabilities and limitations. Article 11 together with Annex IV governs technical documentation, Article 12 automatic logging. Article 50 requires for certain systems, regardless of risk class, that people can recognize when they interact with AI or are looking at AI-generated content. Article 86 gives people who are significantly affected by a decision based on a high-risk system from Annex III the right to obtain from the deployer a clear explanation of the system’s role and of the main reasons for the decision. For automated individual decisions, the GDPR additionally requires meaningful information about the logic involved (Article 13(2)(f), Article 14(2)(g), Article 15(1)(h)) (→ EU AI Act).

Sources

Frequently asked questions

Does every AI system have to be equally explainable?

The need depends on how strongly a result affects people and who has to understand it. A product recommendation in an online shop needs less justification than a system that helps decide on loans or job applications. ISO/IEC 42001 asks you to determine, for each group of interested parties, which information it needs and to provide it.

What can I do if the supplier will not disclose details about the model?

Set out in the contract which documentation you need as a minimum: purpose, limits, accuracy, a rough outline of the data basis and guidance on human oversight. For high-risk systems, the provider has to supply instructions for use under Article 13 of the AI Act anyway. If both are missing, the gap goes into the risk register and management makes a conscious decision about the use.

Is a general description of how the system works enough?

For transparency it is a good start. Affected people usually want to know why their own case turned out the way it did. For that you need a case-level explanation that names the main influencing factors, and a person who can talk it through.

No Cookies!

This wiki collects nothing, bakes nothing and leaves nothing behind. There's nothing to consent to. Privacy doesn't get better than this.