Privacy Risks of AI and How To Avoid Them
25th January 2026
AI can be a helpful tool for organisations, but what are the risks to privacy and compliance? The introduction of AI to the general population in 2022 revolutionised many industries, and this emergent technology continues to develop and advance on an almost daily basis. It would be unreasonable to expect businesses not to use it, but we must be aware of the challenges it presents to data protection and privacy.
AI systems can process unprecedented amounts of personal data, creating unique privacy risks beyond those of traditional software. The growing use of AI for applications like credit scoring, hiring decisions, law enforcement, and healthcare raises many concerns about data protection, privacy and security, as well as the need for human oversight.
What AI Systems Handle Personal Data?
Generative learning models use patterns from massive datasets to generate new content, such as text, images, videos, and more. This is how large language models or LLMs like ChatGPT work. Machine learning AI systems use collected data to self-improve, enhance their own capabilities and automate actions, e.g. Amazon’s Alexa. AI systems in robotics perceive objects and images in the real world, e.g. surveillance cameras and self-driving cars. AI personalisation systems use data from user behaviours and preferences to personalise and tailor content, e.g. Instagram Ads.
Here are some areas where the use of AI can lead to problems with data privacy and regulatory compliance.
AI for Data Collection and Processing
AI can be used to collect large amounts of personal data, extract insights and produce new output. Using AI to collect and process personal data can raise issues with overcollection and lack of anonymisation. There is also the issue of the collected data being at risk of leaks or breaches, which is more significant than usual due to the vast amounts of data involved. It could also be possible that several organisations use the same AI model to collect and analyse data for their purposes, and the AI model then shares their customer data with other organisations that happen to use the same model. Very few organisations have their own in-house AI model, instead relying on third parties with whom they may not have a contractual relationship.
Unauthorised Data Usage and Consent
Users may not even be aware that their data is being collected by an AI model, or may not fully understand how much data is collected and how it will be used, raising concerns around consent. After all, much of the data held on us was not given by us with consent and knowledge, but inferred from what we watch or do. If personal data is collected without consent, individuals could be targeted using misleading and unwanted profiling. AI models can also scrape public and private data from the internet and use it for virtually any purpose without asking for consent. This data could then be used to infer other information about users, even sensitive information like political affiliation, health risks and more. It could also be used for automated decision-making, which has its own problems with ethics and consent, or even identity theft and fraud.
Malware
Apart from any concerns you may have regarding data leakage from reputable AI software providers, some criminal actors will be using AI to deliberately steal data or gain access to your systems. We have already seen cases of AI ingesting infected documents, with networks becoming infected as a result.
Ensure your cybersecurity or IT teams accompany you on this AI journey, as the dangers are real and many.
Surveillance, Facial Recognition and Algorithmic Bias
AI models are being used by law enforcement for surveillance and facial recognition, which, whilst they have the potential for great good, could be problematic, as biases and prejudice can be present in AI models, even subconsciously injected by the AI’s programmers. AI models can perpetuate harmful stereotypes and discriminate against certain groups of people. For example, a tutoring company faced legal consequences for using an AI-powered application software that automatically rejected older job candidates. AI models often have problems differentiating between people with darker skin, leading to potential racial bias. In 2020, Detroit auto shop worker Robert Williams was wrongfully arrested due to a flawed facial recognition algorithm that falsely identified him as a robbery suspect.
Regulations
Article 5(1)(c) of the GDPR details the principle of data minimisation, stating that organisations should only be collecting and processing personal data that is “adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed”. Many AI models overcollect data, which violates this principle.
Another core principle of the GDPR, detailed in Article 5(1)(b), is purpose limitation, meaning that organisations can only collect personal data for “specific, explicit and legitimate purposes”. You cannot use the same personal data for a different purpose without identifying a new lawful basis. Vague purposes like ‘improving the AI model’ are not specific enough for the principle of purpose limitation.
Article 22(1) of the GDPR also restricts automated decision-making and profiling that could significantly impact individuals, e.g. loan applications, job applications and more. Article 22 also grants individuals the right to appeal automated decisions and request that a human being review the decision. The DUAA 2025 amended this by permitting automated decision-making using standard category data in most circumstances unless the processing would have a significant or legal effect on data subjects. Data subjects still need to give permission for ADM to use special category data.
Best Practices for Organisations
Here are some ways that organisations can use AI tools while complying with the GDPR and other data protection regulations.
Transparency & Accountability
As with all data collection and processing activities, individuals should be made aware that their data is being collected and used, and for what purposes. Keeping users informed about how you use their data helps organisations to comply with data protection laws, whether or not they are using AI models to process personal data.
Conduct a DPIA
Conducting a specific AI risk assessment before using AI models to collect and process personal data can help you to identify and minimise any risks that could occur. This complies with Article 35(3), which indicates that any ‘high risk’ processing requires a DPIA. In this case, the ICO and EDPB both advise that large-scale processing of data or that using new or innovative technologies requires a DPIA.
Anonymisation and Pseudonymisation
Anonymising data before it is processed by AI models can help organisations to comply with data protection laws, as once data has been fully anonymised, it is no longer considered personal data and data protection law does not apply to it. Pseudonymisation, while a risk reduction technique, does not have the same effect and the data is still considered personal data. It is also worth noting that anonymous data can still be reidentified when combined with external information, e.g. a dataset stripped of individual identifiers such as names and addresses may appear anonymous, but when cross-referenced with another non-anonymous dataset, individuals may be reidentified.
If you use AI models, or are considering doing so, and want some support and advice on using them in a way that complies with data protection law, you can get in touch with us here at the Griffin House Consultancy for professional guidance.
Author: Mike Martin LLM
Mike is an information governance specialist and one of the founding Directors of the Griffin House Consultancy, a leading specialist data protection and information governance consultancy firm that supports hundreds of clients annually with their training, consulting and auditing requirements.
