AI coding agents are fast, creative, and sometimes dangerously helpful. One of the most overlooked risks when reviewing AI-generated code is secret leakage — hardcoded API keys, tokens, passwords, and credentials that the agent embeds directly into your source files, config files, or environment variables.
This is not a theoretical concern. Every week, developers push AI-generated diffs that contain real credentials to public repositories. Automated secret scanners catch some of these, but many slip through — especially in private repositories, internal tools, and configuration files that never reach public CI.
This article explains why AI coding agents leak secrets, what kinds of secrets appear most often, how to scan for them before merging, and how to build a lightweight secret-scanning gate that works for solo developers and small teams.
Why AI coding agents leak secrets
AI coding agents do not „know” that a string is sensitive. They pattern-match from training data, which includes millions of real configuration files, environment variable assignments, and connection strings. When you ask an agent to „set up a database connection” or „add authentication,” it frequently produces code like this:
# What the agent generates
DATABASE_URL = "postgres://admin:MyS3cretP@ss@db.example.com:5432/prod"
API_KEY = "sk-proj-abc123..."
STRIPE_SECRET = "sk_live_51N..."These are not fake placeholders. They are real patterns that agents reproduce from training data, sometimes with actual keys the agent picked up from your project context, environment files, or previous conversations.
Here are the most common scenarios:
- Copying from .env files — agents read your local environment and reproduce values into source files
- Generating realistic-looking credentials — the agent creates plausible API keys, tokens, and passwords that follow real format patterns
- Hardcoding connection strings — database URLs, Redis endpoints, and S3 paths with embedded credentials
- Creating config files with real secrets — YAML, JSON, TOML configs that contain tokens, private keys, and passwords
- Mirroring test fixtures — test data that contains „example” keys that happen to follow valid formats
The six types of secrets AI agents leak
Based on analysis of thousands of AI-generated diffs, here are the categories you should scan for:
1. API keys and tokens
The most common leak. AI agents frequently generate code that includes:
- OpenAI, Anthropic, and other AI provider API keys
- Cloud provider access keys (AWS, GCP, Azure)
- Service-to-service tokens (Stripe, SendGrid, Twilio)
- OAuth client secrets
Pattern example: sk-proj-..., sk_live_..., AKIA...
2. Database connection strings
Agents love to include complete connection URLs with embedded credentials:
postgres://user:password@host:5432/db
mysql://root:password123@localhost/mydb
mongodb+srv://admin:pass@cluster.mongodb.net/db3. Private keys and certificates
RSA, SSH, and PEM keys generated or referenced by agents in config files, deployment scripts, or test fixtures.
4. OAuth and JWT secrets
Client secrets, signing keys, and JWT secrets that agents embed in authentication configuration blocks.
5. Internal URLs and endpoints
Not secrets per se, but internal API endpoints, admin panels, and staging URLs that should not be in source control.
6. Credential patterns in configuration
YAML, JSON, and TOML files with password fields, token fields, and secret fields that the agent fills with realistic-looking values.
How to scan AI diffs for secrets before merge
You need a systematic approach — not just „I’ll be careful.” Here is a practical five-step secret-scanning workflow:
Step 1: Always review the diff, not just the final file
Secrets often appear in the diff even when the final file looks clean — because the agent adds a credential in one iteration and removes it in the next, or because the diff shows environment variable assignments that the agent later refactors. Always scan the unified diff.
Step 2: Scan for the six secret categories
Run a pattern-based scan on every AI-generated diff before merging. At minimum, check for:
- API key patterns (OpenAI, AWS, Stripe, generic key-value pairs)
- Database connection strings with embedded credentials
- Private key blocks (BEGIN RSA, BEGIN PRIVATE KEY)
- OAuth/JWT secret patterns
- Password fields in config files (YAML, JSON, TOML, .env)
- Internal URL patterns
Tools like gitleaks, truffleHog, and Snyk can scan repositories, but they typically run on the full repo — not on the specific AI-generated diff. For AI code review, you want a diff-focused scanner that checks only what changed.
Step 3: Flag config file changes separately
AI agents modify configuration files more often than business logic files. Any diff that touches .env, config.yml, docker-compose.yml, settings.json, or similar files deserves extra scrutiny — even if no secret patterns are detected, because agents sometimes encode credentials in non-obvious ways.
Step 4: Check for „example” secrets that follow real patterns
AI agents often generate „placeholder” secrets that follow real format patterns:
# "Fake" but realistic-looking
API_KEY = "sk-proj-1234567890abcdef1234567890abcdef"
DB_PASSWORD = "MyPassword123!"These are risky because:
- They follow valid formats and can be confused with real keys
- Secret scanners will flag them, creating noise in your security pipeline
- Other developers may assume they are real and use them
- Some „example” keys are actually valid keys from training data
Best practice: replace all agent-generated credentials with proper secret references (os.environ["API_KEY"], ${DB_PASSWORD}, vault references) before committing.
Step 5: Integrate secret scanning into your review workflow
Make secret scanning a required step in your AI code review process:
- Before merge: Run a diff-focused secret scanner on every AI-generated pull request
- In CI: Add a secret-scanning gate that fails the build if secrets are detected
- Pre-commit: Use a pre-commit hook that scans staged changes for secret patterns
- Periodic audit: Run a full-repository scan weekly to catch secrets that slipped through
Building a CI secret-scanning gate
Here is a minimal GitHub Actions workflow that scans AI-generated diffs for secrets:
name: Secret Scan on AI-Generated Changes
on:
pull_request:
types: [opened, synchronize]
jobs:
secret-scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Get diff
run: |
git diff origin/main...HEAD > /tmp/ai-diff.txt
- name: Scan diff for secrets
run: |
python3 scripts/scan_diff_secrets.py /tmp/ai-diff.txt
# Exit code 1 if secrets found
- name: Upload scan report
if: always()
uses: actions/upload-artifact@v4
with:
name: secret-scan-report
path: secret-scan-report.jsonThis pattern works with any diff-focused secret scanner. The key is scanning the diff (what changed) rather than the entire repository.
What to do when a secret is found in an AI diff
If your scan finds a secret in an AI-generated change:
- Do not merge the PR — even if the secret looks like a placeholder
- If the secret is real: rotate it immediately, scan the agent’s context for other leaks, and check git history for accidental commits
- If the secret is a realistic placeholder: replace it with a proper secret reference before merging
- Add an allowlist entry only if you are certain the pattern is a test fixture that will never be real
- Document the finding in your review notes so the team learns what the agent tends to leak
Common anti-patterns in AI secret handling
Anti-pattern 1: „It is just a test fixture”
AI-generated test fixtures frequently contain realistic-looking secrets. The problem: these fixtures end up in public repos, and automated scanners flag them. Worse, some „test” keys are actually valid keys from training data.
Fix: Use clearly fake patterns (test-key-not-real) or environment variable references in test fixtures.
Anti-pattern 2: „I will clean it up later”
The agent adds a real API key „temporarily” and you plan to replace it. Later never comes. The key stays in git history forever.
Fix: Never commit secrets, even temporarily. Use os.environ references from the start.
Anti-pattern 3: „The scanner found nothing”
Secret scanners have false negatives. They miss encoded secrets, custom formats, and multi-line keys. A clean scan does not mean the diff is safe.
Fix: Combine automated scanning with manual review of config file changes and credential-related code.
Anti-pattern 4: „It is in .env.example, not .env”
Agents sometimes put real values in .env.example or .env.template files, which are typically not gitignored. These files end up in your repository with real credentials.
Fix: Only put placeholder references (DATABASE_URL=your_db_url_here) in example files. Never include real values.
Anti-pattern 5: „The agent refactored it out”
An agent may add a secret in one change and refactor it to an environment variable in the next. The secret still exists in the diff and git history.
Fix: Always scan the full diff history of an AI-assisted branch, not just the final state. If a secret was ever committed, rotate it.
How CodeRiskTools helps with secret scanning
If you want a structured, diff-focused approach to catching secrets in AI-generated code:
- Basic kit includes a security check that flags hardcoded secrets, credential patterns, and unsafe config changes in your pre-merge review
- Pro kit adds expanded security prompts, risk scoring, and client-ready documentation for security findings
- Secret/Config Diff Scanner is a dedicated CLI tool that scans unified diffs for 20+ secret patterns and 5 config risk categories, with severity scoring, JSON/Markdown/HTML output, and CI exit codes
Each tool focuses on a specific part of the secret-scanning problem — from review checklists to automated diff scanning to CI integration.
Secret scanning checklist for AI-generated code
Use this checklist before merging any AI-generated pull request:
- ☐ Have I reviewed the diff (not just the final files)?
- ☐ Have I scanned for API key patterns (OpenAI, AWS, Stripe, generic)?
- ☐ Have I checked for database connection strings with embedded credentials?
- ☐ Have I searched for private key blocks (RSA, SSH, PEM)?
- ☐ Have I reviewed all config file changes (.env, YAML, JSON, TOML)?
- ☐ Have I replaced all agent-generated credentials with environment variable references?
- ☐ Have I flagged any „example” secrets that follow real patterns?
- ☐ If a real secret was found: have I rotated it and checked git history?
- ☐ Is my CI pipeline configured to scan diffs for secrets automatically?
- ☐ Have I documented the finding for the team?
FAQ
Is AI-generated code more likely to contain secrets than human-written code?
Yes, for two reasons. First, AI agents generate realistic-looking credentials from training data patterns. Second, agents have no concept of „sensitive” — they treat API keys and passwords the same as variable names. Human developers at least have an instinct to avoid hardcoding secrets, even if they sometimes fail.
Can I rely on GitHub’s built-in secret scanning?
GitHub’s secret scanning is a good safety net, but it has limitations: it only scans public repositories on free plans, it has a fixed set of patterns (no custom patterns without Advanced Security), and it does not catch secrets in private repos on Team plans. It also does not scan diffs specifically — it scans the full repository state. For AI code review, a diff-focused scanner catches changes earlier and more precisely.
What about pre-commit hooks?
Pre-commit hooks are excellent for catching secrets before they enter git history. Tools like gitleaks and detect-secrets have pre-commit integrations. The limitation: pre-commit hooks only catch what you stage — they do not review the AI agent’s full change scope. Use both pre-commit hooks and diff-focused scanning for full coverage.
Should I scan for secrets in test fixtures?
Yes. AI-generated test fixtures frequently contain realistic-looking secrets that follow real format patterns. Even if they are not real credentials, they trigger false positives in automated scanners and can confuse other developers. Replace them with clearly fake values or environment variable references.
How do I handle secrets that the agent generates and then removes?
If a secret appears in any commit in your branch’s history, treat it as compromised. The agent may have „refactored” the secret into an environment variable, but the original value is still in the diff and git history. Rotate the secret and scan the full branch history.
What is the difference between full-repo scanning and diff scanning?
Full-repo scanning checks all files in your repository for secrets. Diff scanning checks only what changed in a specific commit or pull request. For AI code review, diff scanning is more precise and faster — it focuses on exactly what the agent changed and ignores unchanged files that you have already reviewed.
Bottom line
Secret leakage is one of the most concrete, measurable risks in AI-generated code. Unlike vague „code quality” concerns, a leaked API key has immediate, quantifiable impact. Every AI-assisted pull request should be scanned for secrets before merge — not as a one-time check, but as a systematic part of your review workflow.
Start with a diff-focused scanner, add CI integration, and build the habit of reviewing config file changes with extra scrutiny. The checklist above takes under 5 minutes per PR and catches the most common patterns.
For a structured review workflow that includes secret scanning as a built-in check, see the Basic review kit or the Secret/Config Diff Scanner for automated diff scanning.
