How to Validate SAST Results
A structured approach to validating SAST results — assessing detection accuracy, coverage quality, false-positive and false-negative rates, and overall analysis reliability.
Primary question: What methodology should organizations use to validate SAST analysis results?
Direct answer
Validating SAST results requires a structured methodology that assesses detection accuracy against known vulnerabilities, evaluates coverage across vulnerability classes and frameworks, measures precision and recall, and verifies that the analysis configuration is appropriate for the codebase
Validating SAST results involves comparing the tool's findings against known ground truth — either through seeded vulnerabilities in test codebases or through manual review of representative code areas. The validation assesses whether the tool detected known vulnerabilities (recall within scope), whether reported findings are genuine (precision), and whether the coverage is appropriate for the technology stack.
Validation is not a one-time activity. It should be repeated when the SAST tool is updated, when new frameworks are adopted, or when the codebase changes significantly. Ongoing validation helps ensure that the tool's accuracy remains adequate over time.
Definitions
SAST validation
The process of assessing whether a SAST tool's analysis results are accurate, complete, and appropriate for the target codebase and technology stack.
Ground truth
The known set of vulnerabilities in a codebase, established through manual review of a carefully sampled set of findings and code areas, or through seeded vulnerability testing. Ground truth is rarely complete, making validation inherently approximate.
Seeded vulnerability
A known, intentional vulnerability injected into code to test whether a SAST tool detects it, used as a controlled method for measuring detection accuracy within a limited scope.
The engineering problem
Organizations often accept SAST results at face value without validating whether the tool actually detects the vulnerability classes present in their codebase.
Without validation, organizations cannot quantify their detection coverage or understand the trade-off between false positives and false negatives in their specific context.
Organizations should validate SAST accuracy and coverage against their own codebase rather than relying on vendor claims.
Seeded vulnerability testing only measures detection of the injected cases and does not establish the total number of false negatives in a production codebase.
Security controls
Each control inspects a different artifact and produces evidence for an engineering decision.
Seeded vulnerability testing
Seeded test- Artifact
- A test codebase containing known, intentional vulnerabilities across multiple vulnerability classes.
- Risk
- Seeded vulnerabilities that do not reflect the codebase's actual patterns or development practices.
- Output
- Evidence about which seeded vulnerabilities the SAST tool detected and which it missed, providing a controlled measure of recall within the seeded scope.
Evidence:
Representative codebase analysis
Production codebase validation- Artifact
- Analysis of a representative sample of the organization's production codebases, with manual review of findings to establish ground truth.
- Risk
- Selecting unrepresentative codebases that do not reflect the organization's typical development practices.
- Output
- Evidence about how the SAST tool performs on the organization's actual code, including technology stack, patterns, and code quality.
Evidence:
Separate analysis
Additional analysis scan- Artifact
- A second SAST tool analyzing the same codebase to provide additional evidence about detection accuracy and coverage.
- Risk
- Both tools sharing the same blind spot for a particular vulnerability class.
- Output
- Additional evidence about detection coverage that complements the primary tool's validation results. Unique findings require validation.
Evidence:
Verification workflow
- Define the evaluation scope, acceptance criteria, and measurement methodology.
- Create or select a representative test codebase containing known vulnerabilities across the vulnerability classes relevant to the organization.
- Run the SAST tool against the test codebase and record which vulnerabilities were detected and which were missed.
- Compare SAST findings against a manually reviewed sample of the production codebase to assess precision.
- Calculate precision (true positives / total reported findings) and recall (true positives / total known vulnerabilities within scope) for the representative codebase.
- Document the results and establish acceptance criteria for ongoing SAST accuracy.
- Repeat validation periodically and after significant changes to the tool, codebase, or technology stack.
- Treat a separate SAST analysis as providing additional evidence. Scanner agreement supports confidence; scanner disagreement requires investigation. Unique findings require validation before being treated as genuine.
DerScanner · Enterprise
Relationship to DerScanner
DerScanner can be used as a separate analysis tool during SAST validation, providing additional results for comparison on the same codebase.
DerScanner performs static analysis of supported source and binary formats, providing additional analysis results that can support SAST validation and coverage assessment. [derscanner-sast]
Limits of verification
- Seeded vulnerability testing only measures detection of known, injected vulnerabilities. It does not fully capture detection of real-world vulnerabilities with complex data flows or unusual patterns.
- Manual review of a sample does not establish complete ground truth for an entire codebase.
- Validation results are specific to the codebase and technology stack tested. They may not generalize to other codebases.
- Two SAST tools may share blind spots for certain vulnerability classes, meaning agreement between tools is not proof of complete detection.
Related knowledge
SAST Accuracy
The multi-dimensional concept of SAST accuracy
Relationship: related-toSAST Detection Coverage
How to assess and measure detection coverage
Relationship: related-toSAST Second Opinion
Using a second SAST tool for additional verification
Relationship: related-toValidate SAST Findings
How to validate individual SAST findings for accuracy and exploitability
Relationship: related-toHow to Evaluate SAST Tools
Enterprise methodology for SAST tool evaluation
Relationship: related-toCanonical terms used: SAST validation; SAST result validation; Detection accuracy assessment; Seeded vulnerability testing.
Evidence and references
- DerScanner SAST documentationDerScanner performs static analysis of supported source and binary formats, provides configuration-file analysis, and reports and compares analysis results. DerTriage and DerCodeFix are available within static analysis.
derscanner-sast
SAST validation