Five of six preregistered SWE-bench Verified entries recount to the integer from their own archived per-instance logs (265, 359, 384, 388, 396), agree on all twelve per-repository sub-counts, and show zero self-consistency failures across 2484 reports, while the sixth answers none of its 500 documented log keys and an archive-wide three-key probe finds 15 of 134 entries missing at the documented location.
hunts/r_0dfb8d (Record 70 of 98 in chronological sequence)
Guiding Question
Do the resolved counts published in SWE-bench/experiments for six Verified leaderboard entries recount from the per-instance logs those entries archived, and does each per-instance verdict follow from the test results recorded in the same file?
Method & Verification
Logs were fetched by anonymous HTTPS GET from s3://swe-bench-submissions with recorded ETags and byte counts, recount arithmetic was cross-checked at 12-way per-repository grain, and four planted faults each turned the verdict they should and left the other alone.
Lineage & Relationships
Primary Sources (at pin 8fa46e134)
Editorial Notes
Handback status is settled; case-log status is settled for the units run, with the archive-wide availability question open. Disposition is partly settled for that split, not completed. Absence means absent at the documented key: anonymous ListBucket returns 403, so the hunt never claims absence from the bucket. Question filled from the HuntSpec. Paired with r_4166b0 as the other transfer-test reproduction outside mathematics.
Date Provenance
commit 8eedeed850fa562e05200be1b78725367c5333f6, hunts/r_0dfb8d/RESULTS.md, author 2026-08-24T20:58:53-05:00