GPT-6 Astra

OpenAI · Codex CLI 0.155.1 · guidance curated

Rank 1 of 16Verified · run and scored by the TimelineBench teamCodex CLI 0.155.1guidance curated

Resolution rate

26.8%[15.8, 40.3] exact 95% CI

15 of 56 tasks resolved, one attempt each

Tests passed: 52/56; delivery + guards + must-haves
Must-haves passed: 54/56; 96.4% [87.7, 99.6]
Editors' win-or-tie: 24.4%; [17.1, 31.7] · 168 votes
Rank · bootstrap interval: 1 · 1–4.5; by resolution rate, of 16 setups
Videos delivered: 56/56; 0 missing
Median runtime: 21.4 min; 21.4 h in total
Auto Elo: 809; [701, 905] · pro cut 1000
Editor Elo: 762; [684, 823]
Cost: Not recorded; recorded for Claude Code only

Where the cuts drop out

56 slots, in order

How scoring works

01 / 04
Delivery tests
56 videos delivered; the four tests of each task's tests/test.sh.
56pass · 100.0%
02 / 04
Content guards
Silent, frozen, looped, dead air, one-sided sound, faces cut.
54still standing · 96.4%
03 / 04
Brief must-haves
Every validated must-have of the brief.
52pass all tests · 92.9%
04 / 04
Quality bar
Panel gap against the professional cut reaches the collection's tie margin.
15resolved · 26.8%

By collection

EditStock2/11resolved · 10/11 pass tests · bar 0.21

Cinestudy6/15resolved · 13/15 pass tests · bar 0.30

UGC2/15resolved · 14/15 pass tests · bar 1.54

Commercial5/15resolved · 15/15 pass tests · bar 0.52

Per task

first failing layer, panel gap and editor votes

Per-task results for GPT-6 Astra · Codex · guidance curated: delivery tests, content guards, must-haves, quality gap against the collection tie margin, outcome and editor votes.
Task	Delivery	Guards	Must-haves	Gap vs bar	Outcome	Editors A·T·P
EditStock · 2/11 resolved · tie margin 0.21
Babiators	4/4	6/6	Pass · 12/15 items	+0.26 / 0.21	Resolved	1 · 0 · 2
BOS Parkour	4/4	6/6	Pass · 9/9 items	−0.52 / 0.21	Below bar	2 · 0 · 1
Evolve By Training Camp	4/4	6/6	Pass · 8/9 items	−0.48 / 0.21	Below bar	0 · 1 · 2
Incubate	4/4	6/6	Fail · 10/15 items	+0.27 / 0.21	Failed a test	0 · 0 · 3
J-Pops	4/4	6/6	Pass · 13/15 items	−0.31 / 0.21	Below bar	1 · 0 · 2
Klug Fitness Brand Story	4/4	6/6	Pass · 15/15 items	+0.35 / 0.21	Resolved	0 · 0 · 3
Klug Fitness Next Time	4/4	6/6	Pass · 14/15 items	+0.01 / 0.21	Below bar	1 · 0 · 2
Klug Fitness Railroad Track	4/4	6/6	Pass · 15/15 items	+0.12 / 0.21	Below bar	0 · 1 · 2
Two Player	4/4	6/6	Pass · 10/15 items	−0.02 / 0.21	Below bar	0 · 0 · 3
Whisper Fan	4/4	6/6	Pass · 7/7 items	+0.02 / 0.21	Below bar	2 · 0 · 1
Wild Planet	4/4	6/6	Pass · 8/8 items	−0.29 / 0.21	Below bar	0 · 0 · 3
Cinestudy · 6/15 resolved · tie margin 0.30
Car Chase	4/4	6/6	Pass · 13/13 items	−0.29 / 0.30	Below bar	0 · 0 · 3
Color Grading	4/4	6/6	Pass · 14/15 items	+0.54 / 0.30	Resolved	1 · 1 · 1
Compression Of Time	4/4	Mostly frozen	Pass · 15/15 items	−0.77 / 0.30	Failed a test	0 · 0 · 3
Cutting On Motion	4/4	6/6	Pass · 12/14 items	−0.10 / 0.30	Below bar	0 · 0 · 3
Documentary Format	4/4	6/6	Pass · 13/15 items	+0.54 / 0.30	Resolved	0 · 0 · 3
Editing A Sequence	4/4	6/6	Pass · 14/15 items	+0.85 / 0.30	Resolved	2 · 0 · 1
Horror Movie	4/4	6/6	Pass · 13/15 items	+0.72 / 0.30	Resolved	0 · 1 · 2
Hospital Scene	4/4	6/6	Pass · 14/15 items	+0.08 / 0.30	Below bar	0 · 0 · 3
License Exam	4/4	6/6	Pass · 14/15 items	+0.35 / 0.30	Resolved	0 · 0 · 3
Movie Trailer	4/4	6/6	Pass · 13/15 items	+0.67 / 0.30	Resolved	1 · 1 · 1
Podcast Documentary	4/4	6/6	Pass · 13/15 items	+0.04 / 0.30	Below bar	0 · 0 · 3
Refractory	4/4	6/6	Pass · 14/15 items	+0.18 / 0.30	Below bar	0 · 1 · 2
Relationship Card	4/4	6/6	Pass · 14/15 items	+0.17 / 0.30	Below bar	0 · 1 · 2
Restaurant Scene	4/4	6/6	Pass · 11/15 items	+0.16 / 0.30	Below bar	2 · 0 · 1
The Intern Mockumentary	4/4	Dead air	Pass · 14/14 items	−0.25 / 0.30	Failed a test	1 · 2 · 0
UGC · 2/15 resolved · tie margin 1.54
Breathex Nose Patches	4/4	6/6	Pass · 12/15 items	−0.01 / 1.54	Below bar	0 · 0 · 3
CUPNEX	4/4	6/6	Pass · 12/15 items	+0.69 / 1.54	Below bar	0 · 0 · 3
Dazzle and Charm	4/4	6/6	Pass · 12/15 items	+1.59 / 1.54	Resolved	1 · 0 · 2
Feel Comfy	4/4	6/6	Pass · 11/14 items	+0.66 / 1.54	Below bar	0 · 0 · 3
Fitness Venue	4/4	6/6	Pass · 12/15 items	+1.07 / 1.54	Below bar	1 · 0 · 2
Global Physiotherapy Clinic	4/4	6/6	Pass · 13/15 items	+0.63 / 1.54	Below bar	0 · 0 · 3
House of Hue	4/4	6/6	Pass · 10/15 items	+0.36 / 1.54	Below bar	0 · 0 · 3
LeadLoop	4/4	6/6	Fail · 10/15 items	+0.65 / 1.54	Failed a test	0 · 1 · 2
LedgeFin	4/4	6/6	Pass · 12/15 items	+0.49 / 1.54	Below bar	1 · 1 · 1
MotionMix Studio	4/4	6/6	Pass · 14/15 items	+0.68 / 1.54	Below bar	0 · 0 · 3
Nova Sole	4/4	6/6	Pass · 13/15 items	+0.33 / 1.54	Below bar	0 · 0 · 3
Off Case	4/4	6/6	Pass · 13/15 items	+0.37 / 1.54	Below bar	0 · 0 · 3
Personal Branding	4/4	6/6	Pass · 11/15 items	+0.49 / 1.54	Below bar	0 · 0 · 3
Personal Introduction	4/4	6/6	Pass · 13/15 items	+1.82 / 1.54	Resolved	1 · 0 · 2
Portable Side Bag	4/4	6/6	Pass · 12/15 items	+0.79 / 1.54	Below bar	0 · 0 · 3
Commercial · 5/15 resolved · tie margin 0.52
Aroma Candle	4/4	6/6	Pass · 14/15 items	+0.16 / 0.52	Below bar	2 · 0 · 1
Bean & Roast	4/4	6/6	Pass · 15/15 items	+0.54 / 0.52	Resolved	2 · 0 · 1
Beauty of Delhi	4/4	6/6	Pass · 13/15 items	+0.87 / 0.52	Resolved	2 · 0 · 1
Bike Ride	4/4	6/6	Pass · 15/15 items	+0.86 / 0.52	Resolved	0 · 0 · 3
Camera	4/4	6/6	Pass · 13/15 items	−0.38 / 0.52	Below bar	0 · 1 · 2
Fortunex	4/4	6/6	Pass · 13/15 items	−0.25 / 0.52	Below bar	0 · 1 · 2
Hard Life Of Athlete	4/4	6/6	Pass · 12/15 items	−0.24 / 0.52	Below bar	0 · 0 · 3
Loom Craft	4/4	6/6	Pass · 14/15 items	−0.31 / 0.52	Below bar	1 · 0 · 2
Motionmix In Charge	4/4	6/6	Pass · 13/13 items	+1.01 / 0.52	Resolved	0 · 1 · 2
Pro Bite	4/4	6/6	Pass · 9/15 items	+0.16 / 0.52	Below bar	0 · 0 · 3
Quickshine	4/4	6/6	Pass · 8/14 items	+0.03 / 0.52	Below bar	0 · 0 · 3
Shadow Airpod	4/4	6/6	Pass · 14/15 items	+0.58 / 0.52	Resolved	0 · 0 · 3
Standcup	4/4	6/6	Pass · 12/15 items	−0.76 / 0.52	Below bar	1 · 0 · 2
Street Wear	4/4	6/6	Pass · 15/15 items	−0.56 / 0.52	Below bar	0 · 0 · 3
Voltix Bulb	4/4	6/6	Pass · 14/15 items	+0.07 / 0.52	Below bar	1 · 0 · 2

ResolvedPassed tests, below barFailed a testNo outputEditors A·T·P: blind votes for the agent cut, a tie, or the professional cut.

Missing outputs

Every task produced a video.

Delivery failures

Every delivered video passes its four delivery tests.

Guard failures

Dead air1
Mostly frozen1

Reproduce

Run this setup

# GPT-6 Astra · Codex CLI 0.155.1 · guidance curated
# guidance curated: provide the curated skills from skills/
# source packs are not distributed; the maintainers run the agent internally
harbor run -p tasks/<id> -a codex -m openai/gpt-6-astra
# after all 56 tasks, one attempt each:
timelinebench aggregate <jobs-dir>

Setup id codex__gpt-6-astra__skills-curated · model gpt-6-astra · run guide

Disclosures

One run per task.
The reference edits are professional edits, not certified ground truth.

Where the cuts drop out

Delivery tests

Content guards

Brief must-haves

Quality bar