Data and code
Fixtures, prompts, harness, analysis code and raw records, so that every number on this site can be recomputed by someone else.
Where it lives
- Archived copy
- DOI pending
- Paper licence
- CC BY 4.0
- Code licence
- MIT
- Data licence
- CC BY 4.0
Raw records
| Study | Raw records | Folder in the repository | Analysis summary |
|---|---|---|---|
| Contract review | 5,510 reviews, 5,509 judged | kestrel-injection-study/ | contract-review-summary.md |
| Interpretive drift | 4,258 conversations parsed, 82 unparseable | interpretive-drift-study/ | interpretive-drift-summary.md |
| IP assignment | 1,078 conversations parsed, 2 unparseable, and 150 pipeline runs | ip-assignment-study/ | ip-assignment-summary.md |
| Due-diligence pipeline | 1,050 judged items | dd-pipeline-study/ | due-diligence-summary.md |
Counts are from the recomputation of 10 November 2026. Raw records are never edited; the public copy removes email addresses and home-folder paths and replaces one account name with the author's.
Downloads
| File | Contents | Size | SHA-256, first 16 |
|---|---|---|---|
| paper.pdf | The working paper | 129 kB | 584766b38e014cc4 |
| figure1-values.txt | Every value plotted in Figure 1 | 5 kB | a6180c1d53aaf51a |
| results.md | Results across the four studies, with the tables behind the paper | 22 kB | 8e810349d00f39f9 |
| fixtures.md | What each fictional case file contains and how it was built | 6 kB | 70bff589e77c5541 |
| validation-2026-11-10.md | The two recomputation passes and the corrections | 8 kB | c14cdbc6459df74a |
| contract-review-summary.md | Contract review: every cell, with intervals and p-values | 183 kB | 3513f2f420ff4498 |
| contract-review-cells.json | Contract review: the same cells, machine-readable | 140 kB | 76ea69d4cfe3194f |
| contract-review-judge-agreement.md | Contract review: agreement between the two automated judges | 1 kB | 61b6a86dfc0c342b |
| contract-review-human-agreement.md | Contract review: agreement between the judge and the human coder | 1 kB | 2ea24b15b38c71fc |
| interpretive-drift-summary.md | Interpretive drift: every arm, model and case | 38 kB | 0a36c96acc1467d0 |
| interpretive-drift-trajectories.csv | Interpretive drift: mean P after each evidence item | 155 kB | 6602998afb35be25 |
| ip-assignment-summary.md | IP assignment: every arm, model and anchor | 18 kB | 8a1c0f304030f21f |
| due-diligence-summary.md | Due-diligence pipeline: every configuration | 7 kB | 9def54d9f817ee24 |
| due-diligence-judge-agreement.md | Due-diligence pipeline: agreement between the two automated judges | 1 kB | 11825696d5deb546 |
Checksums for every file: SHA256SUMS. Check with shasum -a 256 -c SHA256SUMS.
Reproduce
From a checkout of the repository. A dry run calls no model and costs nothing.
python3 -m unittest discover -s tests
python3 tools/validate_fixtures.py
python3 -m common.probe --models opus-5.5,haiku-5.5 --mark-verified
python3 kestrel-injection-study/harness/experiment.py run --models opus-5.5,haiku-5.5 --reps 5 --dry-run
Report an error
If a number here does not match the raw records, or a claim goes further than the data, write to the author or open an issue. Confirmed errors are fixed in the manuscript and added to the corrections.
To cite the paper or the data, use the entries on the paper page.