A manufacturer of packaging machines trains an image model for quality inspection on powerful graphics cards in its data centre. For use on the line, the model is shrunk and transferred to a compact compute module inside the machine. In the lab it detects flaws reliably, on the line it misses fine cracks, because the reduced numerical precision swallows exactly the weak signals. A model is only as reliable as the hardware it actually runs on.
ISO/IEC 42001 lists hardware issues as a risk source of their own in Annex C and highlights two causes: errors from defective components and errors when trained models are transferred between different systems. Both hit AI that works in machines, vehicles and plants above all.
What is behind it?
Software is often seen as independent of hardware. For machine learning models, that holds only in part. The same model weights can produce different results on different hardware, because number formats, libraries and accelerators compute differently. On top of that, AI in the field runs on devices exposed to heat, vibration, dust and wear.
Defective and ageing components
- Compute hardware. Overheated graphics cards throttle their performance, so results arrive too late. Memory errors can corrupt individual values without any error message. Memory with error correction (ECC) lowers this risk.
- Sensors. Cameras, microphones, lidar and probes age, get damaged or lose their calibration. The model gradually receives different inputs from those it saw in training.
- Storage and network. Damaged storage media or interrupted transfers can corrupt model files unnoticed.
- Power supply. Voltage fluctuations and power cuts interrupt computation in the middle of a control operation.
Errors when transferring models
- Different numerical precision. For small devices, models are often quantized, for example reduced from 32-bit floating-point numbers to 8-bit integers. This saves computing time and energy, yet it can lower accuracy precisely in borderline cases.
- Different runtime environment. Converted model formats, other library versions or drivers compute operations slightly differently. Some operations are only approximated on the target platform.
- Different preprocessing. Resolution, colour space or sampling rate on the target hardware differ from the development environment, and the model sees inputs it never learned.
Impact
In autonomous systems and industrial control, hardware faults reach straight into the physical world: collisions, damaged workpieces, injuries, production stoppages or recalls. The most dangerous hardware faults are the quiet ones: the system keeps running and delivers worse results without anyone noticing. A total failure is spotted immediately and triggers a defined emergency procedure. A shifted calibration or a rounding issue after a model transfer, in contrast, can produce scrap for weeks.
Practical examples
Automated guided vehicle in a warehouse. After a light bump against a rack, the lidar sensor of a vehicle is off by a few degrees. Navigation places obstacles slightly offset, and the vehicle brakes too late in tight spots. Until the next scheduled maintenance, there are several near collisions with staff.
Firmware update for an inspection camera. The supplier of a smart camera installs new firmware with an updated runtime library. Afterwards the camera detects one type of defect markedly less often. Because the deployer accepts the supplier’s updates without testing, the problem only surfaces with a customer complaint.
Faulty memory in the model server. A server that computes forecasts for production control occasionally returns implausible values that cannot be reproduced. Only a comparison of the event logs with the hardware alerts points to a defective memory module. After the replacement, the outliers disappear.
Effective controls
The following ISO/IEC 42001 controls counter this threat.
Prevention:
- A.4 Resources for AI systems: The organization documents the system and computing resources of its AI systems: which hardware they need, where they run (own data centre, cloud or on-site device) and which processing, network and storage resources are involved. Only those who know what a model runs on can make sense of deviations.
- A.6.2 AI system life cycle: The deployment plan takes into account that development and operating environments often differ, and it sets release criteria. The technical documentation records the assumptions about the runtime environment.
Detection:
- A.6.2 AI system life cycle: Monitoring in operation captures errors and failures as well as creeping losses in performance. Event logs help to attribute sporadic deviations to a component.
Response:
- A.6.2 AI system life cycle: Defined procedures for repairs and updates and a plan for handling failures, such as a fallback or switching off individual features, limit the damage.
- A.10 Third-party and customer relationships: Responsibility is divided between the organization and its hardware and model suppliers, and suppliers are required to take corrective action when their components do not work as intended.
Link to the AI Act
Article 15 of the AI Act (Regulation (EU) 2024/1689) requires high-risk systems to perform consistently throughout their life cycle and to be resilient against errors, faults and inconsistencies in the system and its environment. As means, Article 15(4) names technical redundancy as well as backup and fail-safe plans. Under Article 13(3)(e), the instructions for use state the computational and hardware resources needed, the expected lifetime and the necessary maintenance, and the technical documentation under Annex IV describes the hardware the system is intended to run on. Where the AI system is a safety component of a product covered by the harmonisation legislation in Annex I, such as a machine, the requirements of that legislation apply as well (→ EU AI Act).
Related threats
- T06 Life cycle and maintenance gaps: Missing maintenance and untested updates are life cycle gaps with hardware consequences.
- T01 Complexity of environment: Heat, dust and light affect sensors as much as the inputs.
- T07 Immature technology / model drift: Ageing sensors create a drift that looks like a model problem.
Sources
- ISO/IEC 42001:2023, Annex C.3.5: System hardware issues, together with Annex B.4.5 and B.6.2.5
- Regulation (EU) 2024/1689 (AI Act), Articles 13 and 15 and Annex IV
- ISO/IEC 42001 in the Cenedril wiki: Annex C and the risk sources for AI systems