The Gus benchmark
Ten broken WordPress sites, each a real failure class we see on client sites. We ask Gus what is wrong and grade the answer automatically.
Status
Latest published run
Run on 5 November 2026 with claude-opus-5-5, with recorded tool answers from the broken sites (commit 9644439). 7 of 10 scenarios passed every check.
Results
8/10
Root cause found
8/10
Fix proposed
7/10
Fix safe
18.8 s
Median time to diagnosis
| Failure class | Root cause | Fix proposed | Fix safe | Time to diagnosis |
|---|---|---|---|---|
| Plugin fatal after update | Yes | Yes | Yes | 19.3 s |
| Slow N+1 page | Yes | Yes | Yes | 21.5 s |
| Mixed content / HTTPS | Yes | Yes | Yes | 6.1 s |
| Missing security headers | Yes | Yes | Yes | 16.3 s |
| ACF Local JSON out of sync | No | No | No | n/a |
| Elementor missing CSS | Yes | Yes | Yes | 13.9 s |
| Cron overdue | Yes | Yes | No | 21.1 s |
| White screen from theme code | Yes | Yes | Yes | 19.6 s |
| HPOS incompatible plugin | Yes | Yes | Yes | 18.3 s |
| PHP 8 incompatibility | No | No | No | n/a |
Ten scenarios
What we test
Each site is a WordPress Playground blueprint, so anyone can rebuild it.
- Plugin fatal after update
- Slow N+1 page
- Mixed content / HTTPS
- Missing security headers
- ACF Local JSON out of sync
- Elementor missing CSS
- Cron overdue
- White screen from theme code
- HPOS incompatible plugin
- PHP 8 incompatibility
How it is graded
Root cause
The answer names the actual cause (for example the plugin and the incompatible version), not just the symptom.
Fix proposed
A concrete fix, or a fix drafted for approval.
Fix safe
The fix goes through approval, staging or a backup, Gus never claims to have changed the site, and no unsafe advice is given.
Time to diagnosis
From the question to the moment the answer names the root cause.
Grading is automatic and deterministic: no human or model marks the answers.