CodeQL System Prompt Injection Detection: A Practical Triage Workflow

Developer tracing a CodeQL system prompt injection alert through an AI application data flow

CodeQL 2.26.0 adds a JavaScript and TypeScript query for a specific AI security boundary: untrusted, user-controlled data flowing into a model’s system prompt. That is useful because a system prompt often defines tools, permissions, business rules, and response constraints. If an application splices attacker-controlled text into that privileged instruction channel, the attacker may influence behavior before ordinary user-message safeguards are evaluated.

GitHub documents the new js/system-prompt-injection query in its July 10, 2026 release note. GitHub says the release also adds prompt-injection sinks for more OpenAI, Anthropic, and Google GenAI SDK APIs. This article explains a reproducible review workflow; it does not claim that CodeQL detects every prompt-injection path or that we independently tested every supported SDK model.

What the new CodeQL query is designed to detect

The documented query follows data flow from an untrusted source into an AI model’s system prompt. In practical terms, it looks for a dangerous construction such as building privileged model instructions from request parameters, form fields, webhook bodies, retrieved documents, or other values an external actor can influence. The key issue is not merely that user input reaches an AI API. User input normally belongs in a user-content channel. The issue is that it changes the higher-trust system instruction.

Signal Reviewer question Useful evidence
Untrusted source Can an external actor control the value directly or indirectly? Route, schema, authorization, storage provenance, and test input
Data-flow path Where is the value transformed, concatenated, or retrieved? Trace steps, helper functions, templates, and object fields
System-prompt sink Does the value enter privileged instructions rather than user content? SDK call, message role, model configuration, and wrapper behavior
Proposed barrier Does the fix remove the trust-boundary violation? Patch, negative test, rerun result, and residual-risk note

First confirm that your scan can see the relevant code

A missing alert is not evidence of a safe application. GitHub’s CodeQL documentation lists JavaScript and TypeScript as supported languages, but it also warns that unsupported languages can produce incomplete analysis. Confirm the repository selected the javascript-typescript language, the build or extraction completed, the changed files were included, and the CodeQL version or hosted rollout contains the new query.

GitHub states that new CodeQL versions are automatically deployed to GitHub code scanning on github.com. It separately notes that CodeQL 2.26.0 functionality will appear in a future GitHub Enterprise Server release, while older GHES deployments can manually upgrade CodeQL. Record the actual runner, CLI bundle, query suite, and scan commit instead of assuming version parity.

A six-step alert triage workflow

  1. Open the complete path. Read the source, every propagation step, and the final sink. Do not triage from the alert title alone.
  2. Classify input control. Determine whether the value is controlled by an anonymous user, authenticated tenant, administrator, external integration, uploaded file, database record, or retrieval pipeline. Stored data can remain untrusted.
  3. Confirm the prompt role. Inspect the exact SDK call and wrapper. Verify whether the value reaches system instructions, developer instructions, cached content, session instructions, or an ordinary user message.
  4. Map consequences. List tools, credentials, network access, file access, data sources, and actions available to the model. Prompt injection becomes more consequential when the model can act.
  5. Choose a structural fix. Keep untrusted material in the user-data channel, use fixed system instructions, constrain tool authorization outside the model, and validate structured fields before use.
  6. Rerun and test. Scan the patched commit, execute a negative test with hostile input, and verify that server-side authorization still blocks forbidden actions even if the model output is manipulated.

Prefer structural separation over prompt wording

Adding “ignore malicious instructions” to the same prompt is not a reliable trust boundary. The stronger repair is to stop interpolating untrusted data into privileged instructions. Keep the system prompt static or assembled only from trusted configuration. Pass user-controlled material through the API’s intended user-content field, clearly delimit retrieved content, and ensure tools enforce authorization independently of model text.

Sanitization may be appropriate for a narrow structured field, but it is difficult to define a universal text filter for natural-language attacks. If a value is expected to be a locale, workflow name, or enum, validate it against an allowlist. If it is free text, preserve it as data rather than promoting it into policy.

Verification checklist after the fix

  • The patched scan ran against the intended commit and completed successfully.
  • The query pack or hosted CodeQL version includes the relevant detection.
  • No user, webhook, file, retrieval, or tenant-controlled value is concatenated into system instructions.
  • Untrusted content is represented in the appropriate user or data channel.
  • Tool permissions and sensitive actions are authorized in application code, not by the model.
  • A hostile-input test verifies the rejected or contained behavior.
  • The alert disposition links to the patch and test evidence.
  • Any suppression explains a proven barrier in the data-flow path.

How to handle a likely false positive

Start by proving the barrier. A value may look user-controlled to the analyzer but be replaced by a constant, validated against a closed allowlist, or blocked by a framework model that CodeQL does not understand. Document the exact validation and add a regression test. If the path depends on a custom or niche framework, GitHub documents model extensions as a way to improve analysis. A bare dismissal such as “internal only” is weak because internal records, webhooks, and tenant-managed configuration can still cross trust boundaries.

When uncertainty remains, keep the alert open and reduce privilege around the model. A false negative can be more damaging than the short-term cost of investigating an alert, while repeated unjustified suppressions make future review less trustworthy.

FAQ

Does this query detect every kind of prompt injection?

No. The release note describes a JavaScript/TypeScript system-prompt data-flow query and additional SDK sinks. Prompt injection can also enter through retrieved documents, tool output, images, stored content, unsupported frameworks, or languages outside the query’s coverage.

Is moving input to a user message enough?

It fixes the specific mistake of promoting data into the system prompt, but the application still needs tool authorization, output validation, least privilege, and abuse tests. The model can still be influenced by user content; the goal is to contain what that influence can do.

Should a clean CodeQL scan approve the pull request?

No. Code scanning supplies evidence within its configured scope. Review dependencies, secrets, permissions, tests, and business logic, then require an accountable human merge decision.

Official sources and next steps

Developer Safety Kit withdrawn from public sale; existing buyer access retained.

Leave a Reply

Your email address will not be published. Required fields are marked *.

*
*
You may use these <abbr title="HyperText Markup Language">HTML</abbr> tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Loading, please wait…
BACK TO TOP