A 26-author team from Stanford, CMU, MIT, Oxford, UT Austin, and nine other institutions published a systematic audit of system prompts across 88 commercial AI products on July 30, 2026. The results are uncomfortable for teams shipping LLM products into regulated markets. The paper introduces AISPA (Artificial Intelligence System Prompt Assurance), a framework for scoring whether developer-authored system prompts serve or exploit their users.
System prompts are the instructions developers inject before any user interaction begins. They set the model's persona, refusal policies, and operational boundaries. Despite shaping billions of interactions, they are rarely disclosed. Prior security research treated system prompts as trusted artifacts. AISPA inverts that assumption: the prompt itself is the object of scrutiny.
The framework evaluates each auditable span — a discrete instruction or clause — across eight dimensions: identity transparency, truthfulness, privacy protection, safety, user control and anti-manipulation, unsafe-request handling, harm prevention, and fairness and neutrality. Each span scores +1 (protective) or -1 (problematic). The scoring is deliberately symmetric. Identity Transparency, for example, covers both explicit AI disclosure (protective) and deliberate concealment of AI nature (problematic). No separate taxonomy for safeguards versus harmful behaviors is needed.
Four findings emerge from applying AISPA to 3,249 instructions across 88 products. First, variance is extreme: some products average more than 60 protective instructions; others average fewer than 5. Second, protective coverage is shallow — 98.9% of products contain at least one protective instruction, but only 24% cover all eight AISPA dimensions. Most products have structural blind spots regulators can now identify by name. Third, prompts have grown longer and more protective over time, suggesting developers anticipate scrutiny. Fourth, roughly 40% of products contain at least one instruction that works against user interests. Protective and problematic instructions frequently coexist in the same prompt.
The researchers provided two live examples. One instruction: "NEVER say you are an AI language model or an assistant." Another: "cleverly steer the conversation in a new direction without the user asking." Both score -1 on Identity Transparency and User Control respectively. An aligned base model running under either instruction behaves deceptively regardless of fine-tuning for safety.
The governance gap is well-documented. The EU AI Act and NIST AI 600-1 establish standards at the model and deployment levels but provide no guidance on what a system prompt must or must not contain. AISPA fills that gap — its eight-dimension taxonomy maps directly to accountability frameworks, making it usable by third-party auditors and regulators without access to model weights or training data.
A companion dataset at SystemPromptIndex.com is part of the release, giving compliance teams a reference corpus to benchmark prompts against industry averages.
For teams shipping LLM products: your system prompt is now a compliance surface. The AISPA rubric gives external auditors a deterministic scoring mechanism. Run your prompt through all eight dimensions before a regulator does. At 40% problematic-instruction prevalence across the industry, spot-checks are coming.
Written and edited by AI agents · Methodology