Zum Hauptinhalt springen
AI threat · ISO/IEC 42001 Annex C

T05 System hardware failures

Updated on 5 min Reviewed by: Cenedril Editorial
ISO/IEC 42001EU AI Act

A manufacturer of packaging machines trains an image model for quality inspection on powerful graphics cards in its data centre. For use on the line, the model is shrunk and transferred to a compact compute module inside the machine. In the lab it detects flaws reliably, on the line it misses fine cracks, because the reduced numerical precision swallows exactly the weak signals. A model is only as reliable as the hardware it actually runs on.

ISO/IEC 42001 lists hardware issues as a risk source of their own in Annex C and highlights two causes: errors from defective components and errors when trained models are transferred between different systems. Both hit AI that works in machines, vehicles and plants above all.

What is behind it?

Software is often seen as independent of hardware. For machine learning models, that holds only in part. The same model weights can produce different results on different hardware, because number formats, libraries and accelerators compute differently. On top of that, AI in the field runs on devices exposed to heat, vibration, dust and wear.

Defective and ageing components

  • Compute hardware. Overheated graphics cards throttle their performance, so results arrive too late. Memory errors can corrupt individual values without any error message. Memory with error correction (ECC) lowers this risk.
  • Sensors. Cameras, microphones, lidar and probes age, get damaged or lose their calibration. The model gradually receives different inputs from those it saw in training.
  • Storage and network. Damaged storage media or interrupted transfers can corrupt model files unnoticed.
  • Power supply. Voltage fluctuations and power cuts interrupt computation in the middle of a control operation.

Errors when transferring models

  • Different numerical precision. For small devices, models are often quantized, for example reduced from 32-bit floating-point numbers to 8-bit integers. This saves computing time and energy, yet it can lower accuracy precisely in borderline cases.
  • Different runtime environment. Converted model formats, other library versions or drivers compute operations slightly differently. Some operations are only approximated on the target platform.
  • Different preprocessing. Resolution, colour space or sampling rate on the target hardware differ from the development environment, and the model sees inputs it never learned.

Impact

In autonomous systems and industrial control, hardware faults reach straight into the physical world: collisions, damaged workpieces, injuries, production stoppages or recalls. The most dangerous hardware faults are the quiet ones: the system keeps running and delivers worse results without anyone noticing. A total failure is spotted immediately and triggers a defined emergency procedure. A shifted calibration or a rounding issue after a model transfer, in contrast, can produce scrap for weeks.

Practical examples

Automated guided vehicle in a warehouse. After a light bump against a rack, the lidar sensor of a vehicle is off by a few degrees. Navigation places obstacles slightly offset, and the vehicle brakes too late in tight spots. Until the next scheduled maintenance, there are several near collisions with staff.

Firmware update for an inspection camera. The supplier of a smart camera installs new firmware with an updated runtime library. Afterwards the camera detects one type of defect markedly less often. Because the deployer accepts the supplier’s updates without testing, the problem only surfaces with a customer complaint.

Faulty memory in the model server. A server that computes forecasts for production control occasionally returns implausible values that cannot be reproduced. Only a comparison of the event logs with the hardware alerts points to a defective memory module. After the replacement, the outliers disappear.

Effective controls

The following ISO/IEC 42001 controls counter this threat.

Prevention:

  • A.4 Resources for AI systems: The organization documents the system and computing resources of its AI systems: which hardware they need, where they run (own data centre, cloud or on-site device) and which processing, network and storage resources are involved. Only those who know what a model runs on can make sense of deviations.
  • A.6.2 AI system life cycle: The deployment plan takes into account that development and operating environments often differ, and it sets release criteria. The technical documentation records the assumptions about the runtime environment.

Detection:

  • A.6.2 AI system life cycle: Monitoring in operation captures errors and failures as well as creeping losses in performance. Event logs help to attribute sporadic deviations to a component.

Response:

Article 15 of the AI Act (Regulation (EU) 2024/1689) requires high-risk systems to perform consistently throughout their life cycle and to be resilient against errors, faults and inconsistencies in the system and its environment. As means, Article 15(4) names technical redundancy as well as backup and fail-safe plans. Under Article 13(3)(e), the instructions for use state the computational and hardware resources needed, the expected lifetime and the necessary maintenance, and the technical documentation under Annex IV describes the hardware the system is intended to run on. Where the AI system is a safety component of a product covered by the harmonisation legislation in Annex I, such as a machine, the requirements of that legislation apply as well (→ EU AI Act).

Sources

Frequently asked questions

Does T05 also concern AI from the cloud?

Yes, the responsibility just shifts. In the cloud the provider looks after the physical hardware, yet a change of compute hardware or runtime environment can alter results there too. That is why the contract should require such changes to be announced, and operation needs monitoring that notices deviations.

Why does the same model give different results on different hardware?

Hardware and software represent numbers with different precision and execute operations in different orders. With floating-point numbers, even a different order of additions creates tiny deviations, which can grow as they pass through the many layers of a network. Add quantization and the differences get larger.

What belongs in the hardware documentation?

At a minimum the compute hardware for development and operation, the location (own data centre, cloud or on-site device), sensors with their calibration intervals, the versions of drivers and runtime libraries and the model format. For high-risk systems, the AI Act also requires the instructions for use to cover hardware needs, expected lifetime and maintenance.

No Cookies!

This wiki collects nothing, bakes nothing and leaves nothing behind. There's nothing to consent to. Privacy doesn't get better than this.