The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions

Today's LLMs are susceptible to prompt injections, jailbreaks, and other attacks that allow adversaries to overwrite a model's original instructions with their own malicious prompts.

Source: OpenAI ChatGPT — Published April 19, 2024 at 19:00 UTC — Category: Chatbots