Shadow AI poses new risk to companies
Four years into the AI revolution, it has become clear that almost everyone in knowledge-based businesses is using the technology in some way, but not every business knows exactly how.
This lack of oversight is contributing to a looming corporate security crisis, with ‘shadow AI’ – the use of AI by employees inside a business but beyond corporate reach – thriving.
Shadow AI refers to the use of AI tools by employees without the knowledge or approval of their employers.
A recent study of 6,000 employees globally found that 70% were using AI at least once a week, and companies were unaware of a third of this activity.
A research project led by MIT says a “shadow AI economy” has already developed because staff lack trust in official AI initiatives or sanctioned, ringfenced corporate tools and feel more comfortable turning to the LLMs they use in their personal lives to help them create documents or organise their workflows.
Shadow AI use has already been blamed for a range of security breaches.
Cloud provider Vercel, for example, said it may have suffered a loss of customer data after cybercriminals effectively took over an agentic AI tool used by an employee and through it gained access to the employee’s Google Workspace account.
Samsung reported three instances of confidential information being accessed within 20 days, all of them attributable to employees using ChatGPT.
Dr Leanne Allen, UK head of AI at KPMG, says that once employees use external tools, it becomes harder to know what data has been shared, what outputs are being relied upon, and whether any records exist for audit or accountability.
Risks include IP or copyrighted material being exposed, regulatory breaches, and poor quality outputs comprising work.
Related: Paramount Sues Warner Bros. in Takeover Battle
Almost half of employees surveyed by KPMG in conjunction with the University of Melbourne said they knew their workplace AI use contravened official policy.
Imperial College London research earlier this year found it was technically possible for LLMs to piece together parts of older versions of documents and create a new whole despite ‘de-duplication’ processes that should act as safeguards.
Igor Shilov, who worked on the project, says tech firms care about private information leaking but clearly aren’t on top of the problem.
IBM, which is running a public information campaign about the risks of shadow AI, was scathing about LLM security measures in recent research.
Jerry Cuomo, IBM Fellow and serial entrepreneur, stated that even though LLMs may not be intentionally storing data, that doesn’t mean that they’re not storing data unintentionally.
One response to shadow AI’s gradual infiltration would be to lean into repression, banning the use of unauthorised tools and improving the way AI use is monitored centrally.
However, this risks stifling positive behaviours and frustrating employees who want to use technology to improve their work.
Dr Leanne Allen suggests that organisations should think about how to make safe, effective AI genuinely accessible, with clear intended use cases and guardrails built into everyday workflows.
This means understanding more about which tools are being used and how, and providing a thoughtful and well-curated set of officially sanctioned AI products that are secure but also genuinely useful and tailored to the needs of staff in different roles.
That cultural change is likely to take time, however, and the fear is that with experts predicting cybercriminals are themselves being empowered and emboldened by access to AI, that may be a commodity many businesses do not possess.
