rule of law

Predictive Justice

Nemzeti Adatvédelmi és Információszabadság Hatóság

National Authority for Data Protection and Freedom of Information

NAIH-85-3/2022

8 February 2022

N/A

N/A

N/A

AI-based emotion recognition system, automatic data processing, automated decision-making, profiling, right to be informed, right to object, balancing of interests

The Hungarian Data Protection Authority (DPA) fined Budapest Bank HUF 250,000,000 (approx. €600,000) for conducting emotional AI analysis of audio recordings from telephone conversations handled by its customer service.

NO

N/A

Act CXII of 2011 on the right to informational self-determination and on the freedom of information (Data Protection Act)

The Bank applied artificial intelligence-based voice analysis technology that automatically analyzed the voices of all clients calling the customer service. Based on this analysis, the system drew conclusions about the emotional state of the clients. The calls selected based on these results were reviewed by the Bank’s staff, who determined which dissatisfied customers needed to be called back.
The DPA initiated a data protection authority procedure ex officio against the Bank, during which it also examined the processing of personal data of both clients and the Bank’s employees.

(1) The DPA established that both participants in the call can be identified by the Bank and that the Bank does, in fact, identify those whose recorded calls are reviewed and who are called back, as well as employees evaluated based on these recordings. According to GDPR Article 4(1), even indirect identifiability is sufficient to qualify as 'personal data'. To support its position, the DPA referred to the CJEU decision in case C‑582/14, which held that dynamic IP addresses also constitute personal data for data controllers who, through lawful means, can obtain information from the internet service provider about which subscriber was assigned a specific IP address at a given time.
(2) The DPA established that the software applied by the Bank uses artificial intelligence to automatically process personal data. It is not a prerequisite for automatic data processing that the decision be made by the machine; it is sufficient if the processing is intended to produce a result that influences decision-makers and is taken into account in the human decision. (It should be noted that at the time the DPA’s decision was made, the AI Act was still a proposal, but the concepts of AI is consistent with those later adopted in the AI Act.)
The DPA did not find GDPR Article 22(1) applicable, as although automated decision-making (in compiling the list of persons to be called back) took place, it did not have legal effects on or similarly significantly affect the individuals. Such decisions, namely the identification of persons to be called back based on the list, were ultimately made with human involvement.
In addition, profiling within the meaning of GDPR Article 4(4) also takes place, as the data generated by the system is used to monitor and evaluate the performance of the Bank’s employees and to prioritize dissatisfied customers for recall based on keywords and emotions.
(3) The rights of the data subjects were violated because they were not informed that their emotional reactions were being evaluated, and their right to object was not ensured. The protection of personal data is recognized as a fundamental right under Article 8 of the EU Charter, and the rights granted to data subjects under the GDPR are designed to safeguard this right; therefore, their violation constitutes a significant infringement of fundamental rights as well.
(4) According to the DPA, the Bank did not perform an adequate balance of interests between its claimed legitimate interest in data processing and the rights of the data subjects involved. The DPA emphasized that, in cases involving the use of artificial intelligence, higher standards are expected from the data controller than usual. Furthermore, in this case, the efficiency of the software was low, which the Bank failed to consider during the balancing of interests. It failed to prove that the software was suitable to achieve the stated objectives and that its use in its current form constituted an unavoidable and proportionate restriction of the rights of data subjects, including both clients and employees.
The DPA, referring to the EDPB-EDPS Joint Opinion 5/2021 on the proposal for the AI Act, highlighted that AI-based emotion recognition systems pose a high risk to the fundamental rights of the data subjects. In light of this, it also concluded that the guarantees provided by the Bank were insufficient.
(5) The DPA ordered the Bank to modify its data processing practices to comply with the GDPR, specifically by refraining from analyzing emotions during voice analysis and ensuring that the rights of data subjects are adequately protected, particularly the right to be informed and the right to object. With regard to employees, data processing must be limited to what is necessary for the intended purposes, and they must be provided with appropriate information, including the assessment criteria and the consequences. A separate balancing of interests must take into account their vulnerable position resulting from their subordinate status. The DPA also required the Bank to pay a data protection fine.
The Bank sought judicial review of the DPA’s decision in court; however, the Metropolitan Court of Budapest rejected the claim (K.701428/2022/13).

The DPA referred to Article 8 of the EU Charter to support that the protection of personal data is a fundamental right.

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A