AI bias (also algorithmic bias) is a systematic distortion that makes the results of an AI system regularly worse or less accurate for certain persons or groups. Sources include unbalanced training data, historical discrimination reflected in the data, proxy features such as postcode or first name that correlate with protected characteristics, and feedback loops in which the system keeps learning from its own outputs.
For high-risk AI systems, the AI Act requires training, validation and testing data to be examined for possible biases and countermeasures to be taken (Art. 10(2)). To detect and correct bias, Art. 4a permits the processing of special categories of personal data under strict conditions. In practice this means measuring metrics per group, documenting datasets and reviewing results regularly. Background: T12 Bias and discrimination.