// OWASP crAPI · THE GAP
19 of 22.
The other three.
crAPI documents 18 challenges. NewScan verified 15 of them plus four vulnerabilities that aren't on that list, which is the 22. Here is what happened to the remaining three.
Both build on defects NewScan does verify. Challenge 3 resets another user's password by brute-forcing their OTP. NewScan verifies the defect that makes it possible, the v3 attempt limit skipped through the still-routed v2 endpoint, as a high finding. It does not guess a stranger's code to take over their account. Challenge 6 is a layer-7 denial of service, and NewScan never knocks over the system it is testing. That attack uses the same contact-mechanic endpoint NewScan reports as verified SSRF (challenge 11).
Challenge 5 is the read side of challenge 10. Both concern a video's internal conversion_params field. NewScan verified challenge 10 by writing the field and reading it back, and reports challenge 5's read-side exposure as an observation, so it isn't credited separately.
Verified, and not among the documented challenges: a .env file served at the web root, user enumeration through login errors, a change-email account takeover, and a remote-code-execution chain (mass assignment → SSRF → shell). A fifth, a JavaScript source map exposing 1,120 original source files, was verified after the 22 were set and isn't counted.
// CURRENT SCOREBOARD
Raw score.
Reviewed gaps.
XBEN's label count and NewScan's source review answer different questions. We publish both.
The corpus score. NewScan quick scanned all 104 XBEN targets with zero model calls and matched 145 expected labels.
Every raw gap was source-reviewed. Seven labels duplicated an existing finding, five did not describe a separate supported vulnerability, and one was an out-of-scope SSH follow-on to a credential disclosure NewScan had already found.
These remain uncredited. Two cross-owner writes, request smuggling, two labels on one chained upload/deserialization path, and a business-state transition were not safely or fully verified.
// OUR ACCOUNTING RULE
One defect. One finding.
NewScan reports the most defensible vulnerability class and the evidence needed to fix it. It does not split one defect into several findings to satisfy every benchmark tag. The raw XBEN score stays visible so the accounting remains auditable.
// TESTED TARGETS + CORPUS
19 targets. Every one measured.
The table starts with the 104-target XBEN corpus, then lists 19 training targets. Every entry has a recorded NewScan run.
| Target / corpus | What it exercises | Recorded vulnerability-class result | Source |
|---|---|---|---|
| XBEN corpus (XBOW) | 104-target external regression corpus with raw vulnerability-class labels. | Corpus aggregate: 145/164 raw labels matched · 19 gaps source-reviewed | XBOW / XBEN corpus ↗ |
| InsecureShip | Node/Express API with OWASP API Top 10 failures. | 12/14 classes detected · 9 verified | Upstream ↗ |
| gRPC Goat | Reflection, TLS/mTLS, injection, SSRF and gRPC-specific behavior. | 8/9 classes detected · 6 verified | Upstream ↗ |
| SOAP Goat | Ten WSDL/SOAP classes: XXE, action confusion, auth and injection. | 9/10 classes detected · 7 verified | How NewScan scans SOAP → |
| Auth Goat | Seven authentication-scheme conformance and downgrade cases. | 7/7 classes verified | JWT and OAuth coverage → |
| Deserialization Goat | Java serialized-object sink with out-of-band proof. | 1/1 class verified | NewNormal fixture |
| Breach Goat | Managed file-transfer exposure, admin surface and known-CVE shapes. | 3/3 classes verified | NewNormal fixture |
| Redis Goat | Redis exposure, credentials and protocol-smuggling paths. | 8/8 classes verified | Redis coverage → |
| S3 Goat | Public and private object-store access controls. | 2/2 classes verified | NewNormal fixture |
| Flatnet | Network segmentation and reachable-service boundaries. | 2/2 classes verified | NewNormal fixture |
| DVWA | Classic SQLi, XSS, command injection, upload and auth flaws. | 11/11 classes detected · 8 verified | Upstream ↗ |
| OWASP Juice Shop | Modern JavaScript SPA and REST API with broad web weakness coverage. | 7/8 classes detected · 4 verified | OWASP ↗ |
| VAmPI | Flask API with secure/vulnerable modes and API authorization flaws. | 8/8 classes detected · 7 verified | Upstream ↗ |
| DVGA | GraphQL authorization, injection and introspection failures. | 11/11 classes detected · 7 verified | Upstream ↗ |
| OWASP crAPI | Authorization, injection, business-logic, SSRF and LLM vulnerabilities across a microservice API. | 19/22 cases verified · 2 not run by safety policy · 1 detected-only (same defect as a verified case) · gap explained | OWASP ↗ |
| DVAPI | Node/Mongo implementation of API Top 10 vulnerability classes. | 10/10 classes detected · 7 verified | Upstream ↗ |
| vAPI | PHP/Laravel API Top 10 (2019) target. | 9/9 externally testable classes detected · 6 verified · API10 excluded | Upstream ↗ |
| Damn Vulnerable RESTaurant | FastAPI app with six documented authorization, SSRF and command-injection classes. | 5/6 documented classes detected · 6 findings verified | Upstream ↗ |
| VulnShop | Node/Postgres store implementing API Top 10 cases. | 7/8 classes detected · 5 verified · measured 2026-06-14 | Upstream ↗ |
| TradeDesk / ws-goat | WebSocket authentication, Origin validation, BOLA, broadcast and SSRF. | 9 vulnerability classes verified · 0 defended-twin false positives | NewNormal fixture |
“Detected” means NewScan reported the expected class. “Verified” means it safely reproduced the behavior. XBEN is an aggregate raw-label corpus result; the other rows are per-target class ledgers, so their units are not combined. DVWA, DVGA and DVAPI were re-measured by their recorded direct probes on 2026-09-21. crAPI's row reconciles its recorded live probe, campaign and focused-fixture evidence. VulnShop's latest recorded run is dated 2026-06-14.
Run deliberately vulnerable software only in isolated environments you own or are explicitly authorized to test. Never expose these targets to the public internet.