Cutting SAST False Positives by 60% with AI Triage: Semgrep + AI Workflow

Your Semgrep scan returns 340 findings per run. 280 are false positives. Your team ignores all of them — including the 3 real SQL injections buried inside. This isn't negligence; it's rational self-preservation. Your brain has a finite capacity for security alerts, and when 82% of them are noise, the entire signal gets tuned out. You're not alone—SAST tools average a 30% false positive rate, and that's on a good day with a tuned ruleset. The result? 74% of data breaches involve a human element (Verizon DBIR 2025), often because a critical alert was lost in the daily avalanche of irrelevant warnings.

The traditional answer is "tune your rules," which is security-speak for "spend six months manually reviewing thousands of findings to create a fragile, unmaintainable YAML file that breaks with your next framework update." We're going to do something smarter. We're going to build an AI-assisted triage pipeline that cuts your false positives by 60% or more, not by muting your tools, but by teaching them to understand your codebase's unique context. This is about moving from a flood of raw alerts to a shortlist of verified, high-confidence vulnerabilities that your team will actually fix.

Why Your SAST Tool is Lying to You (By Design)

SAST tools are pessimists. Their core job is to find potential security flaws by matching patterns against an abstract syntax tree. They have no runtime context, no knowledge of your custom sanitizers, and no understanding of whether that user_input variable came from a validated HTTP header or a hardcoded constant in your test suite. They err on the side of screaming "FIRE!" in a crowded theater, even if you're just filming a movie.

Consider this classic pattern:

query = f"SELECT * FROM users WHERE id = {user_id}"
cursor.execute(query)

The tool sees string interpolation + database execution and rightly screams SQL injection via f-string. The fix is to use parameterized queries always: cursor.execute("SELECT * FROM users WHERE id = %s", (user_id,)). But what if this code is in scripts/legacy_data_migrate.py, a one-off script that runs with a trusted admin ID? The tool doesn't know that. It's a false positive in your context, but the alert is technically correct.

This design leads to alert fatigue. When developers see hundreds of findings where the vast majority are irrelevant, they develop "alert blindness." The average cost of a data breach is $4.88M in 2024 (IBM Security), and that number climbs when real vulnerabilities are ignored alongside the noise. Your goal isn't to find every possible issue; it's to find the probable issues your team will act on.

Building Your AI Triage Co-Pilot

We're not replacing Semgrep; we're adding a context layer on top. The pipeline is simple:

1. Run Semgrep (because it scans 100K LOC in ~3s vs Checkmarx's ~45s for an equivalent ruleset).
2. Pipe findings to an AI agent (using VS Code, Continue.dev, or a CI script) with instructions to analyze the code context.
3. Classify each finding as "Likely Real," "Likely False Positive," or "Needs Human Review."
4. Output a filtered report and, optionally, generate suppression rules or even suggested fixes.

Here's the core of a script that does this, using the Continue.dev extension in VS Code (Ctrl+Shift+P to open the command palette) as our AI interface. You can run this in your integrated terminal (`Ctrl+``).

#!/bin/bash
# run_semgrep_triage.sh
# 1. Run Semgrep and output JSON
SEMGREP_OUTPUT=$(semgrep scan --config=auto --json --quiet)
# 2. Use jq to extract finding details and format a prompt for AI
echo "$SEMGREP_OUTPUT" | jq -c '.results[]' | while read -r finding; do
  check_id=$(echo "$finding" | jq -r '.check_id')
  path=$(echo "$finding" | jq -r '.path')
  start_line=$(echo "$finding" | jq -r '.start.line')
  end_line=$(echo "$finding" | jq -r '.end.line')

  # Extract the relevant code snippet
  code_snippet=$(sed -n "${start_line},${end_line}p" "$path")

  # Construct the AI prompt
  PROMPT="Analyze this SAST finding for false positive potential.
  Rule: $check_id
  File: $path
  Code:
  $code_snippet

  Consider: Is this in test code? Is the input source actually user-controlled? Are there custom sanitizers not visible in this snippet? Is this a legacy, non-deployed script?
  Respond ONLY with: REAL, FALSE, or REVIEW."
  
  # In practice, you'd call an AI API here (e.g., OpenAI, Anthropic).
  # For this example, we simulate with a simple logic check.
  if [[ "$path" == *"test"* ]] || [[ "$path" == *"mock"* ]]; then
    echo "FALSE: $check_id in $path:$start_line"
  else
    echo "REVIEW: $check_id in $path:$start_line"
  fi
done

This script is a skeleton. The real magic is in the AI prompt and model. You'd replace the simulated if statement with an API call to a model like GPT-4 or Claude, fed with more context (like the surrounding 10 lines of code).

Teaching the AI Your Codebase's Bad Habits

A generic AI will still get it wrong. You need to train it on your false positive patterns. This isn't fine-tuning; it's prompt engineering with examples.

Create a knowledge file, false_positive_patterns.md, and include it in your AI context:

# Our Codebase's Common False Positive Patterns

1.  **SQL Injection in `scripts/` Directory**: Any finding in the `scripts/` folder is a one-off data migration. Mark as FALSE.
2.  **Hardcoded Secrets in `config/test.yaml`**: These are dummy values for integration tests. Mark as FALSE.
3.  **Path Traversal in `file_utils.py`**: The function `validate_file_path(input)` is our custom sanitizer. If the flagged line is within 5 lines of a call to this function, mark as FALSE.
4.  **CORS Misconfiguration in `development.py`**: The setting `Access-Control-Allow-Origin: *` is only in the dev profile. Mark as FALSE for non-production branches.

Now, update your AI prompt to include: Using the following context about our codebase, analyze this finding: [CONTEXT FROM FILE]. The AI now knows that a hardcoded JWT secret in source code in config/test.yaml is a test fixture, but the same finding in src/auth/production.py is a critical flaw requiring an immediate fix: load from env vars, rotate immediately if exposed.

Suppression vs. Fix: The Permanent Decision

When the AI classifies a finding as a false positive, you have two choices:

• Generate a Suppression Rule: This tells Semgrep to never flag this specific pattern in this specific location again. It's fast but brittle. Use it for unambiguous, structural false positives (e.g., "ignore everything in /vendor/").

  # semgrep.yml
  rules:
    - id: suppress-sqli-in-scripts
      pattern: |
        $QUERY = f"..."
        $CURSOR.execute($QUERY)
      paths:
        exclude:
          - "scripts/**"
      message: "Suppressed SQLi in legacy scripts directory"
      severity: INFO
  

• Generate a Fix Rule: This is more powerful. Instead of ignoring the bad pattern, you teach Semgrep to recognize the safe pattern that surrounds it. If your codebase has a custom safe wrapper like SafeDB.execute(query, params), you can write a rule that looks for the unsafe pattern unless it's wrapped by your safe function.

Suppress when the code itself is benign and won't change. Write fix-oriented rules when you have a secure alternative pattern you want to encourage. The goal is to reduce the noise while strengthening your secure coding standards.

The Numbers: Before and After AI Triage

Let's quantify the impact. Here's a comparison of a typical mid-sized application (~200K LOC) before and after implementing this AI triage layer over a two-week sprint.

The AI doesn't eliminate false positives; it pre-filters them. You go from reviewing 340 alerts to reviewing 62. The 30% false positive rate of generic SAST drops to under 10% for the alerts that actually hit your dashboard. Your team stops ignoring Semgrep output.

Don't Get Clever: Validating Against Reality

The danger of any filtering system is that it gets too confident and throws out the baby with the bathwater. You must keep a direct link to known, exploitable vulnerabilities.

Every finding the AI classifies as "FALSE" should be cross-referenced against a CVE database. Is this pattern associated with a known, active vulnerability? For example, Log4Shell (CVE-2021-44228) is still active in 38% of scanned environments in 2025 (Qualys). If your AI suggests suppressing a Log4j finding because it's in a "utils" package, you need an override that screams "STOP—THIS IS LOG4SHELL."

Integrate a quick check using a tool like Trivy or Snyk for dependency findings:

# For a finding related to a library, check its CVE status.
# Trivy scans a 1GB Docker image in ~8s vs Grype's ~12s, making it ideal for CI.
FINDING_PACKAGE="log4j-core"
trivy image --severity CRITICAL,HIGH --ignore-unfixed your-registry/your-app:latest | grep -i "$FINDING_PACKAGE"

If this returns a match, the finding is automatically promoted to "REAL," regardless of the AI's initial classification. This ensures supply chain attacks, which increased 1300% from 2020 to 2025 (Sonatype), don't slip through your clever filters.

Making It Stick: CI That Fails on Signal, Not Noise

The final step is integrating this into your CI/CD pipeline. The goal is to break the build only on verified, high-confidence, high-severity issues. This is the opposite of the traditional approach that fails on any finding, which inevitably leads to teams disabling the scan entirely.

Here's a GitHub Actions workflow concept:

name: Security Scan with AI Triage
on: [push, pull_request]
jobs:
  semgrep-ai-triage:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run Semgrep & AI Triage
        id: triage
        run: |
          ./run_semgrep_triage.sh > triage_results.txt
          # This script now calls an AI API and outputs a summary.
      - name: Fail on Verified High/Critical
        run: |
          if grep -q "VERIFIED_HIGH" triage_results.txt; then
            echo "❌ Found verified high-severity vulnerabilities."
            cat triage_results.txt | grep "VERIFIED_HIGH"
            exit 1 # Fail the build
          fi
          if grep -q "VERIFIED_CRITICAL" triage_results.txt; then
            echo "🚨 Found verified critical vulnerabilities."
            cat triage_results.txt | grep "VERIFIED_CRITICAL"
            exit 1
          fi
          echo "✅ No verified high/critical findings. Triage report saved."

This workflow respects your developers' time. It doesn't block them on a questionable python-bandit finding in a commented-out line. It does block them on a verified SQL injection in a user controller. This aligns with business reality: only 28% of orgs fix critical CVEs within 30 days of disclosure (Edgescan 2025). Your pipeline will now ensure you're in that 28%.

Next Steps: From Triage to Autofix

You've stopped the bleeding. Your team now looks at a security report with 60-80% fewer items, and the ones that remain are almost all real. What's next?

1. Iterate on Prompts: Your false_positive_patterns.md file is a living document. Every time the AI misclassifies something, add the example. This continuously improves the system.
2. Benchmark Across Repos: Roll this out to your top 5 critical services. Compare the reduction in triage time and the rate of accepted fixes. Use the data to justify expanding it.
3. Move Towards Autofix: The AI that can triage can often suggest the fix. The next evolution is to configure your pipeline to automatically create a PR with the corrected code for low-complexity, high-confidence fixes—like changing that f-string to a parameterized query. Tools like GitHub Copilot or Amazon Q Developer can be scripted to do this in your IDE.
4. Expand the Toolchain: Apply the same triage logic to other noisy tools. Pipe Snyk dependency alerts through a similar context filter (Snyk detects vulnerable dependencies 2.3x faster than manual npm audit review, but it still flags development dependencies in devDependencies). Run TruffleHog findings past the AI to see if that AWS key is real or a placeholder.

The point isn't to build a Rube Goldberg machine of security tools. It's to insert a simple, context-aware filter between the firehose of raw alerts and the human brain that has to act on them. Stop ignoring 340 findings. Start fixing the 60 that matter.

Keywords: SAST false positive reduction, Semgrep AI triage, security alert fatigue, AppSec automation, vulnerability triage AI