PILOT WAITLIST · COHORT 1 · Q1 2027
Get your AI coding tool scored on real metrics — before your CFO asks how you justified the spend.
An independent, pre-registered audit of how Claude Code, Cursor Agent, Replit Agent and Google Antigravity perform against your team's actual spec — across five quality dimensions, ten replications, with statistical confidence intervals.
The headline from the public benchmark
The auditor's main study scored four AI coding agents against three task specifications. 600 measurements. Pre-registered protocol. Live dashboard. Some findings:
CLAUDE CODE
0.00
off-spec features per run (30 runs). Tightest discipline.
REPLIT AGENT
1.00
hallucinated CLI commands per run. Ships its own architectural defaults regardless of spec.
CURSOR AGENT
0.17
off-spec endpoints per run. Occasional helpful overreach (/health, /metrics).
ANTIGRAVITY (GEMINI)
0.33
off-spec endpoints per run. Concentrated in web-app specs.
→ Browse the live dashboard
What the pilot audit gives you
01
Five-dimension score (security, complexity, duplication, hallucinations, effort)
02
Four-vendor comparison on YOUR spec, not a generic benchmark
03
Statistical tests (ANOVA + Tukey) with 95% confidence intervals
04
Per-spec breakdown so you see condition × task interactions
05
Written PDF report (~10 pages) + private dashboard URL (12 months)
06
Raw CSV + provenance JSON for your own analysis
How a pilot engagement runs
Day 0
You hand me a YAML or Markdown spec of the feature set your team is asking AI tools to build.
Day 0–3
Spec ingestion + a 30-min clarification call.
Day 3–7
Capture runs: 10 replications × 4 vendors × your spec, scored under the pre-registered protocol.
Day 7–9
Statistical analysis + report drafting.
Day 10
Delivery — PDF report, private dashboard, CSV. 30-min walkthrough call included.
Important — pilot timing.
I am the sole operator of this instrument, currently completing my MSc dissertation at Aston University under student visa. Commercial pilot engagements begin Q1 2027, once the relevant work-rights transition completes. Joining the waitlist now reserves your slot at the cohort-1 launch price (£2,000 per pilot, no payment until delivery).