Data protection conference sets GDPR guidelines for AI applications

Companies, organizations and authorities want to use artificial intelligence. This usually refers to generative AI, which has experienced a boom since the emergence of ChatGPT and the like. However, the use of large language models (LLMs) raises legal questions. Among other things, this concerns GDPR-compliant use. The German Data Protection Conference has therefore published a guide for companies. According to the publishers, this is also suitable for other AI models, but relates specifically to LLMs.

The guide is divided into three sections: the selected fields of application and purposes of AI applications, implementation and use. The Data Protection Conference is also preparing to take over the monitoring of the AI Act: "The data protection supervisory authorities are ready to take on the task of national market surveillance for AI systems", it says in a second paper. The expertise in digital fundamental rights protection could be expanded to include this competence.

When selecting an AI application in a company, it is essential to define a purpose, according to the data protection conference. Applications should only use personal data when it is actually needed. Not every AI application automatically uses personal data, but with language models, it is almost impossible to avoid it. As an example, the publishers cite the evaluation of raw material mapping, for which no personal data is used. However, the situation is different if an AI model already contains personal data as a result of the training material.

The data protection conference warns that in such cases, the people who use the applications based on it may not have any influence. Users must then ensure that no errors occur in their systems as a result. In other words, if an AI application is mischievous or incorrectly outputs names and data, it may no longer be allowed to be used. ChatGPT and co. sometimes reply for no apparent reason with names or answers from forum posts that were used in training.

Data processing only with a legal basis

Every AI application that processes personal data requires a corresponding legal basis – i.e. permission. If an AI application has an impact on a person, a human must decide. An AI must therefore not exclude an applicant from a process or reject them as unsuitable for a project. "Insufficient personnel resources, time pressure and a lack of transparency regarding the decision-making process of AI-supported preliminary work must not lead to results being accepted without review," the paper states.

In addition: Technically closed systems are preferable from a data protection perspective – compared to open systems in which information and data migrate to the cloud and to the provider. Providers must comply with their transparency obligations. Users, i.e. companies, can demand this and must also make informed decisions based on this. Documentation obligations are also regulated by the AI Act, for example, which stipulates that general purpose AI (GPAI) must be separately inspected, tested and certified.

As is well known, the GDPR also stipulates that people have a right to rectification and erasure. The guidance states: "Organizational and technical procedures must be designed for both rights so that they can be exercised effectively." However, it has so far been extremely difficult or even impossible to enforce this. No information can be deleted from LLMs. The most that can be done is to impose rules on the models, and therefore also on the applications, that they are no longer allowed to say something. However, this also means that it must be possible to fine-tune the AI applications of the companies themselves - not just on the provider side of the AI models.

Safety instead of unauthorized employees

According to the data protection conference, when implementing AI applications, it should be regulated whether the provider or user is legally responsible. Employees should also be clearly told what they may and may not do with which AI applications. The editors assume that numerous employees are currently using AI applications on their own authority - and violating data protection regulations. A data protection impact assessment is also necessary, which is already regulated in the GDPR - i.e. a risk assessment.

As many data protection aspects as possible should already be considered during implementation; the publishers refer to "data protection by design" and "by default". In addition to data protection, all requirements that generally apply to IT systems must also be met - from cybersecurity to usability.

The creation of an employment reference can also be traced back to a person without a name, for example, by date and creator. AI can establish such correlations particularly well. Additional care should be taken with particularly sensitive data, such as religious affiliation, health, sexuality and more. This also includes whether someone regularly visits a certain place. The processing of this data is only permitted under very specific conditions, but is generally prohibited. Bias is also problematic. If an AI application violates the General Equal Treatment Act, it is not permitted.

(emw)