Claude Opus 5
Anthropic · OpenCode 1.18.31 · guidance off
Rank 5 of 16 (tied)Verified · run and scored by the TimelineBench teamOpenCode 1.18.31guidance off
Resolution rate
17.9%[8.9, 30.4] exact 95% CI
10 of 56 tasks resolved, one attempt each
- Tests passed
- 46/56
- delivery + guards + must-haves
- Must-haves passed
- 47/56
- 83.9% [71.7, 92.4]
- Editors' win-or-tie
- 19.0%
- [11.6, 26.4] · 153 votes
- Rank · bootstrap interval
- 5 (tied) · 2–10.5
- by resolution rate, of 16 setups
- Videos delivered
- 51/56
- 5 missing
- Median runtime
- 51.4 min
- 53.7 h in total
- Auto Elo
- 713
- [554, 817] · pro cut 1000
- Editor Elo
- 729
- [626, 802]
- Cost
- Not recorded
- recorded for Claude Code only
- 01 / 04
Delivery tests
51 videos delivered; the four tests of each task's tests/test.sh.
51pass · 91.1% - 02 / 04
Content guards
Silent, frozen, looped, dead air, one-sided sound, faces cut.
50still standing · 89.3% - 03 / 04
Brief must-haves
Every validated must-have of the brief.
46pass all tests · 82.1% - 04 / 04
Quality bar
Panel gap against the professional cut reaches the collection's tie margin.
10resolved · 17.9%
By collection
EditStock0/11resolved · 9/11 pass tests · bar 0.21
Cinestudy5/15resolved · 13/15 pass tests · bar 0.30
UGC2/15resolved · 11/15 pass tests · bar 1.54
Commercial3/15resolved · 13/15 pass tests · bar 0.52
Per task
first failing layer, panel gap and editor votes| Task | Delivery | Guards | Must-haves | Gap vs bar | Outcome | Editors A·T·P |
|---|---|---|---|---|---|---|
| EditStock · 0/11 resolved · tie margin 0.21 | ||||||
| Babiators | 4/4 | 6/6 | Pass · 12/15 items | +0.13 / 0.21 | Below bar | 1 · 0 · 2 |
| BOS Parkour | 4/4 | 6/6 | Pass · 9/9 items | −0.82 / 0.21 | Below bar | 1 · 0 · 2 |
| Evolve By Training Camp | 4/4 | 6/6 | Pass · 8/9 items | −0.48 / 0.21 | Below bar | 2 · 0 · 1 |
| Incubate | 4/4 | 6/6 | Pass · 11/15 items | −0.01 / 0.21 | Below bar | 0 · 0 · 3 |
| J-Pops | 4/4 | 6/6 | Pass · 12/15 items | −0.28 / 0.21 | Below bar | 1 · 0 · 2 |
| Klug Fitness Brand Story | 4/4 | 6/6 | Pass · 15/15 items | +0.02 / 0.21 | Below bar | 0 · 0 · 3 |
| Klug Fitness Next Time | 4/4 | 6/6 | Pass · 14/15 items | −0.01 / 0.21 | Below bar | 0 · 0 · 3 |
| Klug Fitness Railroad Track | 4/4 | 6/6 | Pass · 15/15 items | +0.10 / 0.21 | Below bar | 0 · 0 · 3 |
| Two Player | 4/4 | 6/6 | Pass · 10/15 items | −0.94 / 0.21 | Below bar | 0 · 0 · 3 |
| Whisper Fan | No output · request too large | — | — | — | No output | — |
| Wild Planet | 4/4 | One-sided sound | Pass · 7/8 items | −0.47 / 0.21 | Failed a test | 2 · 1 · 0 |
| Cinestudy · 5/15 resolved · tie margin 0.30 | ||||||
| Car Chase | 4/4 | 6/6 | Pass · 13/13 items | −0.43 / 0.30 | Below bar | 0 · 1 · 2 |
| Color Grading | 4/4 | 6/6 | Pass · 14/15 items | +0.81 / 0.30 | Resolved | 2 · 0 · 1 |
| Compression Of Time | No output · request too large | — | — | — | No output | — |
| Cutting On Motion | 4/4 | 6/6 | Fail · 6/14 items | −2.90 / 0.30 | Failed a test | 0 · 0 · 3 |
| Documentary Format | 4/4 | 6/6 | Pass · 13/15 items | +1.17 / 0.30 | Resolved | 0 · 0 · 3 |
| Editing A Sequence | 4/4 | 6/6 | Pass · 14/15 items | +1.46 / 0.30 | Resolved | 3 · 0 · 0 |
| Horror Movie | 4/4 | 6/6 | Pass · 12/15 items | +0.11 / 0.30 | Below bar | 0 · 0 · 3 |
| Hospital Scene | 4/4 | 6/6 | Pass · 15/15 items | −0.20 / 0.30 | Below bar | 0 · 0 · 3 |
| License Exam | 4/4 | 6/6 | Pass · 15/15 items | −0.43 / 0.30 | Below bar | 0 · 1 · 2 |
| Movie Trailer | 4/4 | 6/6 | Pass · 13/15 items | +0.40 / 0.30 | Resolved | 0 · 1 · 2 |
| Podcast Documentary | 4/4 | 6/6 | Pass · 13/15 items | −0.43 / 0.30 | Below bar | 1 · 0 · 2 |
| Refractory | 4/4 | 6/6 | Pass · 13/15 items | −0.20 / 0.30 | Below bar | 1 · 0 · 2 |
| Relationship Card | 4/4 | 6/6 | Pass · 14/15 items | +0.30 / 0.30 | Resolved | 1 · 0 · 2 |
| Restaurant Scene | 4/4 | 6/6 | Pass · 10/15 items | −0.31 / 0.30 | Below bar | 1 · 0 · 2 |
| The Intern Mockumentary | 4/4 | 6/6 | Pass · 14/14 items | −0.23 / 0.30 | Below bar | 1 · 0 · 2 |
| UGC · 2/15 resolved · tie margin 1.54 | ||||||
| Breathex Nose Patches | 4/4 | 6/6 | Pass · 13/15 items | −0.02 / 1.54 | Below bar | 0 · 0 · 3 |
| CUPNEX | 4/4 | 6/6 | Pass · 12/15 items | +0.20 / 1.54 | Below bar | 0 · 0 · 3 |
| Dazzle and Charm | 4/4 | 6/6 | Pass · 12/15 items | +1.81 / 1.54 | Resolved | 1 · 0 · 2 |
| Feel Comfy | 4/4 | 6/6 | Pass · 11/14 items | +0.27 / 1.54 | Below bar | 0 · 0 · 3 |
| Fitness Venue | 4/4 | 6/6 | Pass · 13/15 items | +0.96 / 1.54 | Below bar | 0 · 0 · 3 |
| Global Physiotherapy Clinic | 4/4 | 6/6 | Pass · 12/15 items | +0.64 / 1.54 | Below bar | 0 · 0 · 3 |
| House of Hue | 4/4 | 6/6 | Pass · 9/15 items | +0.66 / 1.54 | Below bar | 1 · 0 · 2 |
| LeadLoop | 4/4 | 6/6 | Fail · 10/15 items | +0.65 / 1.54 | Failed a test | 0 · 0 · 3 |
| LedgeFin | 4/4 | 6/6 | Pass · 12/15 items | +0.52 / 1.54 | Below bar | 0 · 0 · 3 |
| MotionMix Studio | No output · request too large | — | — | — | No output | — |
| Nova Sole | 4/4 | 6/6 | Pass · 11/15 items | +0.72 / 1.54 | Below bar | 0 · 0 · 3 |
| Off Case | 4/4 | 6/6 | Fail · 12/15 items | −0.11 / 1.54 | Failed a test | 0 · 0 · 3 |
| Personal Branding | 4/4 | 6/6 | Pass · 11/15 items | +0.49 / 1.54 | Below bar | 1 · 0 · 2 |
| Personal Introduction | 4/4 | 6/6 | Pass · 13/15 items | +1.78 / 1.54 | Resolved | 0 · 0 · 3 |
| Portable Side Bag | No output · request too large | — | — | — | No output | — |
| Commercial · 3/15 resolved · tie margin 0.52 | ||||||
| Aroma Candle | No output · request too large | — | — | — | No output | — |
| Bean & Roast | 4/4 | 6/6 | Pass · 14/15 items | −0.12 / 0.52 | Below bar | 0 · 0 · 3 |
| Beauty of Delhi | 4/4 | 6/6 | Pass · 13/15 items | +0.76 / 0.52 | Resolved | 1 · 0 · 2 |
| Bike Ride | 4/4 | 6/6 | Pass · 15/15 items | +0.83 / 0.52 | Resolved | 0 · 0 · 3 |
| Camera | 4/4 | 6/6 | Pass · 12/15 items | −1.39 / 0.52 | Below bar | 1 · 0 · 2 |
| Fortunex | 4/4 | 6/6 | Pass · 12/15 items | −0.04 / 0.52 | Below bar | 2 · 0 · 1 |
| Hard Life Of Athlete | 4/4 | 6/6 | Pass · 12/15 items | −0.12 / 0.52 | Below bar | 0 · 0 · 3 |
| Loom Craft | 4/4 | 6/6 | Fail · 11/15 items | −0.63 / 0.52 | Failed a test | 0 · 0 · 3 |
| Motionmix In Charge | 4/4 | 6/6 | Pass · 12/13 items | +0.66 / 0.52 | Resolved | 0 · 0 · 3 |
| Pro Bite | 4/4 | 6/6 | Pass · 12/15 items | +0.24 / 0.52 | Below bar | 0 · 0 · 3 |
| Quickshine | 4/4 | 6/6 | Pass · 9/14 items | −0.94 / 0.52 | Below bar | 0 · 0 · 3 |
| Shadow Airpod | 4/4 | 6/6 | Pass · 13/15 items | +0.04 / 0.52 | Below bar | 0 · 0 · 3 |
| Standcup | 4/4 | 6/6 | Pass · 12/15 items | −0.40 / 0.52 | Below bar | 0 · 0 · 3 |
| Street Wear | 4/4 | 6/6 | Pass · 15/15 items | +0.03 / 0.52 | Below bar | 0 · 0 · 3 |
| Voltix Bulb | 4/4 | 6/6 | Pass · 13/15 items | −0.05 / 0.52 | Below bar | 0 · 1 · 2 |
ResolvedPassed tests, below barFailed a testNo outputEditors A·T·P: blind votes for the agent cut, a tie, or the professional cut.
Missing outputs
- Provider rejected an oversized request after context compaction (HTTP 413)5
Compression Of Time, Aroma Candle, MotionMix Studio, Portable Side Bag, Whisper Fan
Delivery failures
Every delivered video passes its four delivery tests.
Guard failures
- One-sided sound1
Reproduce
Run this setup
# Claude Opus 5 · OpenCode 1.18.31 · guidance off
# source packs are not distributed; the maintainers run the agent internally
harbor run -p tasks/<id> -a opencode -m anthropic/claude-opus-5
# after all 56 tasks, one attempt each:
timelinebench aggregate <jobs-dir>Setup id opencode__claude-opus-5__skills-off · model anthropic/claude-opus-5 · run guide
Disclosures
- One run per task.
- The reference edits are professional edits, not certified ground truth.