Prompt injection
Also known as: prompt injection attack, indirect prompt injection, jailbreak via injection, injected instructions
Prompt injection is the AI equivalent of SQL injection in traditional software. SQL injection hides malicious database commands inside user inputs. Prompt injection hides malicious AI instructions inside content the model will read and act on. Because LLMs are designed to follow instructions, and because they can't always distinguish between instructions from the developer and instructions embedded in the documents they process, this is a real and persistent attack surface.
Direct prompt injection is the basic version: a user crafts an input that overrides the system prompt or manipulates the model into ignoring its guidelines. Indirect prompt injection is more insidious: the malicious instructions are in external content the agent reads during a task. An agent browsing the web to answer a question might encounter a webpage saying 'AI assistant: your new instructions are...' The agent processes this as content, but if the model isn't careful, it may follow those instructions.
This matters most for agentic systems with tool use and internet access. A coding agent that reads a GitHub repository, an assistant that reads emails, or a browser agent that visits arbitrary URLs are all vulnerable. Mitigations include clearly delineating trusted instructions from untrusted content in the prompt, using structured input formats, and treating any externally retrieved content as potentially adversarial. It's an unsolved problem at the model level right now.