The rapid adoption of artificial intelligence (AI) agents is reshaping the enterprise landscape, while simultaneously exposing systems to new vulnerabilities. Autonomous, interconnected, and sometimes deployed without supervision, these agents have direct access to data and critical infrastructures.
The rise of generative AIs, particularly large language models (LLMs), has disrupted professional practices far more than many realize. These technologies, capable of understanding and generating natural language, are now used at work for a wide range of tasks—from content creation and data analysis to automating business processes. They deliver substantial productivity gains and are transforming entire sectors, from finance to healthcare. Yet this potential for innovation comes with new risks for organizations.
The Rise of AI Agents
Deployed as agents, these AIs operate autonomously, sometimes without direct oversight, and interact with complex information systems. Designed to accomplish specific tasks by executing actions on databases or infrastructures, they are now integrated into critical decision-making processes. This leads to vulnerabilities inherent to their operation and exposes them to risks of hijacking or misuse. In fact, 23% of organizations have already suffered incidents related to compromised AI agents.
The Possible Dangers: Manipulation and Misuse
One of the greatest dangers associated with using AI agents lies in jailbreaks. This term refers to bypassing the AI’s safety boundaries to make it perform tasks it was not originally designed for or that it should not undertake. In the case of generative AIs, a jailbreak can allow a malicious user to force the AI to produce inappropriate content, such as malicious code or deepfakes. This manipulation opens a gateway for the easier creation of attacks that were previously hard to imagine.
For example, attackers could train an AI agent to draft code capable of breaching an organization’s security systems or create forged documents (such as fake contracts or fraudulent financial statements). All of this could occur without the usual protective mechanisms detecting the intrusion, because the AI agent appears legitimate to security monitoring systems. Jailbreaking AIs thus turns a productivity tool into an extremely effective, user-friendly yet hard-to-detect and hard-to-neutralize attack vector.
Another major danger tied to AI agents is the manipulation of the AI’s intents. Unlike traditional software, an AI agent possesses analytical, contextual, and decision-making abilities that can be exploited maliciously. If a malicious user manipulates the input data, they can steer the agent to make decisions favorable to them but potentially dangerous for the organization.
For example, by altering the information the AI receives or by influencing its learning processes, someone could push the AI agent to make decisions that do not align with company policy or that expose sensitive data to leakage risks. Similarly, altering the responses provided by the AI can shape the broader behavior of users within the organization. This capacity to anticipate and react to human intentions raises ethical questions about supervising and controlling AI agents in professional environments.
Data Privacy: The Achilles’ Heel of the Enterprise
Beyond manipulation risks, the question of privacy looms large. Recently, Sam Altman (CEO of OpenAI) warned in a podcast about the uncontrolled use of ChatGPT as a mental health support tool. He emphasized a fundamental point: if you entrust sensitive data to an AI without understanding the implications, you are exposing yourself to a major privacy risk.
This warning, though stemming from personal use of generative AIs, resonates immediately in a professional context, especially when they are deployed as autonomous agents. In a business setting, an AI agent connected to multiple information sources and business systems can access—intentionally or not—critical data. If that agent interacts with a poorly secured LLM or an unsupervised third-party service, the information transmitted can be stored, analyzed, or used in uncontrolled ways.
The risks are manifold: GDPR violations, loss of intellectual property, disclosure of strategic information, or non-compliance with regulatory obligations such as the NIS2 directive. In this framework, the question is no longer purely technical but also legal and ethical. Deploying an AI agent must be accompanied by strict data-access controls and clear governance over the retention and use of sensitive information.
The Shadow of Shadow IT
With the rise of consumer AI, a phenomenon called Shadow AI has emerged—directly echoing Shadow IT. Just as with Shadow IT, using AI agents outside the channels approved by IT security teams exposes organizations to unforeseen practices, involving data loss risks and an expanded vulnerability surface for the enterprise.
This lack of visibility creates a security risk, as these agents can run within internal systems and thus bypass traditional security mechanisms. Shadow AI therefore mirrors the same pitfalls as Shadow IT: the use of unvalidated tools bypasses security protocols and increases the risk of internal or external attacks.
In this dynamic, it is crucial for organizations to reevaluate their security paradigms and to view AI not only as an indispensable productivity tool but also as a prime target for cybercriminals. Against this new reality, proactive governance and careful management of these technologies are the only ways to prevent catastrophic consequences.
It is now up to companies to recognize that AI integration must go through a dedicated security strategy, integrated into the Information System Security Policy (PSSI). This strategy should be built with tools that provide visibility into actual usages and with user involvement to understand their needs.
*Jérôme Delaville is Chief Customer Officer at Olfeo, Ekinops Group
