Why My AI Vulnerability Scanner Missing 93% of Bugs Was the Right Start
A developer shares the lessons learned from building an AI-powered security scanner that scored a dismal recall rate on its first run, proving why brutal honesty beats hidden fixes.
Stock photo for illustration only, not from the actual event
- The initial vulnerability scanner run yielded a recall score of just 0.07, missing 93% of bugs.
- The tool combines deterministic static analysis rules with an LLM to judge findings.
- The poor score was traced to a narrowly defined source list rather than a broken engine architecture.
- Publishing bad numbers upfront establishes long-term transparency and credibility.
When the developer first tested their custom vulnerability scanner against an industry-standard benchmark, the scorer script report delivered a stark reality with an overall precision of 0.60, a recall of 0.07, and an F1 score of 0.13, defining the baseline performance.
A precision score of 0.60 meant that 60% of the alarms raised by the scanner pointed to genuine bugs, proving it was right more often than not. However, a recall score of 0.07 indicated that out of the 777 real, labeled vulnerabilities across the four classes covered, the scanner found only 7%—missing a staggering 93% of the bugs it was built to detect. Combined with precision, the harmonic mean dragged the F1 score down to 0.13.
Months of deep exploration into AI development led to the creation of this vulnerability scanner. The architecture pairs deterministic static-analysis rules to handle the heavy searching with an LLM tasked with judging whether each finding represents a real security flaw or a false alarm. The benchmark put to the test was the OWASP Benchmark, featuring 2,740 labeled Java test cases, out of which 1,478 cases fell into the scanner's four supported vulnerability classes—consisting of 777 real vulnerabilities and 701 decoy cases designed to trigger false positives.
For those new to the domain, three fundamental concepts anchor the process: a source represents where untrusted input enters a program, a sink is where that input turns dangerous, and a vulnerability occurs when data flows from a source to a sink without proper sanitization, a tracking mechanism known as taint analysis.
Stock photo for illustration only, not from the actual event
In software engineering, publishing poor initial metrics goes against the typical instinct to hide flaws until a product looks polished. However, sharing raw, unfiltered setbacks establishes a culture of rigorous scientific integrity. By isolating variables and making incremental adjustments based on empirical data rather than arbitrary tweaks, developers can diagnose structural limitations accurately without resorting to guesswork.
The version executed in this initial test was intentionally minimal—a spike featuring just a single source pattern, getParameter, wired to a handful of sinks to test end-to-end pipeline functionality. An extremely low recall score on a first spike does not signal a broken concept; rather, it indicates an overly restrictive source list, which represents an easily fixable configuration issue rather than a fundamental engine rebuild.
Developers typically face two common temptations when encountering poor initial results. The first is quiet concealment, fixing the issue privately and publishing only polished success metrics. Because independent third parties do not audit these personal benchmark runs, establishing reader trust requires publishing unflattering numbers openly to build a reliable track record before undertaking head-to-head comparisons with established tools like Semgrep or CodeQL.
The second temptation is carpet-bombing the rule set by loosening regular expressions and matching everything to artificially inflate recall, which ultimately destroys precision and obscures which specific changes actually resolved the problem. The methodical alternative requires examining actual benchmark source code, determining input usage frequency, adding sources progressively, and re-measuring systematically after every isolated change.
Source: Dev.to
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment