Study registry
This registry holds the 36 studies, in 6 groups, that the companion manuscripts set out. Each states its hypothesis, and its protocol, available on request, gives the measure and the result that would count against it. Verdicts are published whatever the outcome, including null results. Protocols are hashed when written, and since September 2026 their hashes are anchored in a public repository. The launch paper’s four first studies, its decision study and its study of learning across inquiries are specified in Section 5 of the paper, and each will be added here when its protocol is sealed, before its first run.
The organizational theory
Generating analysis gets cheaper faster than evaluating it
partly automated
AI lowers the cost of generating analysis faster than it lowers the cost of evaluating the quality of a judgment.
Where fixed rules pay and where live judgment pays
run by hand
Judgment fixed in rules pays more where tasks are frequent, stable and causally clear. Live judgment pays more where conditions change, causes are ambiguous, feedback is slow and decisions cannot be reversed.
More judgment fixed in rules raises the value of overseeing it
partly automated
As more judgment is fixed in rules, the value of the means of checking and governing that judgment rises rather than falls.
Structured records compared with prose
run by hand
Recording judgments in a structured form improves calibration, later learning and accountability compared with prose alone, with the same analysts and the same evidence.
Reviewers from different model families catch more weak judgments
run automatically
Reviewing models from different families detect weak or misleading judgments more reliably than reviewers from the same family.
People and models together, under explicit rules
run by hand
Processes that combine people and models under explicit rules for confidence, abstention and escalation outperform people alone and models alone on tasks whose answers arrive later.
Where live judgment adds value in allocating capital
partly automated
In allocating capital, the extra value of live judgment concentrates where markets are illiquid, prices reveal little and conditions shift. Scale goes to judgment fixed in rules.
How much work centres on judgment
run automatically
Work that centres on judgment is a large share of employment and value added, in the low tens of per cent, and the share is rising, fastest in the work of overseeing judgment.
When machines forecast as well as experts
partly automated
Once machine forecasters match experts, the value of a forecasting function moves from producing forecasts to choosing questions, auditing reasoning and deciding how far to rely on forecasts.
Questions a market could price
partly automated
Whether a question suits a liquid prediction market predicts where judgment can be efficiently fixed in rules. Work that centres on judgment concentrates in the questions that do not suit one.
Modelling another party’s decisions, tested in paired runs
partly automated
A record of another party’s inferred decisions, with explicit alternatives and calibrated confidence, supports earlier and better calibrated anticipation than a record of stated positions alone, provided the inferences are scored against what happened.
Which kinds of system close the loop
partly automated
Only systems with persistent memory, outcome feedback and adaptation close the loop of representing, evaluating, governing and learning from judgment. Chat interfaces and single-session agents do not.
Oversight as varied as what it oversees
run automatically
An oversight arrangement detects weak judgments only when it is at least as varied as the judgments it governs, and reducing the variety of its reviewers reduces detection in proportion.
Governance keeps its value as models improve
partly automated
The value of the means of governing judgment does not fall as model capability rises, and it may rise.
The shared representation
Removing each element of the shared representation
partly automated
Each of the nine elements of the shared representation matters: removing it under controlled conditions degrades calibration, auditability or adaptability beyond a stated threshold.
Whether attribution matters where accountability is central
partly automated
Recording who holds a judgment matters in domains where several actors and accountability are central. Its lack of effect on forecasting reflects that domain and is no reason to drop it from the nine elements.
The shared representation in law and science
partly automated
The shared representation alone can record coherent judgment in legal and scientific work, as it did in the two domains already tested.
Two validators, one verdict
run automatically
Two independent validators for the W3C SHACL standard agree on whether each of the 60 trial records conforms.
Independent implementations of the handoff format
run by hand
Parties working independently can build a reader and a writer for the shared representation and pass a judgment between them under the handoff format with loss below a stated threshold.
Several domains, run continuously
partly automated
Routing effects between domains, and scoring how far strategic effects spread, stay stable and meaningful when several domains run continuously.
Stability of causal strengths learned from outcomes
run automatically
Estimates of how strongly causal mechanisms act, updated from outcomes, settle rather than oscillate once at least 50 mechanisms have resolved.
The shared representation’s current evidence
partly automated
The published namespace, the register of conforming artifacts and the evidence gathered so far for the shared representation can be reported at any time as a sealed snapshot that others can reproduce.
The evaluation instrument
A first live calibration run
partly automated
On a fresh set of market questions that resolve in the future, the project’s structured forecaster scores no worse on the Brier score than a plain baseline, with leakage excluded by design.
Calibration of each kind of insight over 90 days
run automatically
Once at least 30 insights of a kind have resolved, their calibration can be reported for each kind and for each method that generated them, and it tells the methods apart.
A performance record for each insight method
run automatically
A combined score, built from survival under review, reader feedback, how often the stated warning signs occur and the Brier score, separates insight methods well enough to decide each quarter which to retire.
Does an insight read as written by a person or by a model?
partly automated
A model acting as judge can reliably tell whether an insight reads as the work of a human strategic adviser or of an AI, and the signal helps improve how insights are framed.
Patterns shared across similar readers, released only in groups of five or more
run automatically
Which kinds of insight work for a reader carry over to readers in similar roles, once at least five readers have histories.
Every surfaced insight carries its seal and its record of change
run automatically
Every insight shown to a reader carries a cryptographic seal, a record of the checks it passed and a record of why it changed.
Repairing a broken record from structured feedback
run automatically
Structured feedback on what a record fails to meet is enough for a competent agent to bring a broken record into conformance, and the scoring tells full repair from partial repair.
Do retrieved resolved questions reduce forecast error?
partly automated
Adding four similar questions, with their known outcomes, to the forecaster’s context lowers its mean Brier score compared with a control that has none.
An observed base rate compared with the model’s own recall
partly automated
Giving the forecaster the observed base rate for a question’s reference class lowers its Brier score compared with giving none and compared with the model’s own recalled base rate.
The insight architecture
Insights compared with a hired analyst, judged blind
run by hand
A blind panel rates insights surfaced by the system as at least as novel and useful as a hired analyst’s, on the same days.
Retiring and adding insight methods each quarter
run automatically
A quarterly cycle that retires the weakest insight methods and adds new candidates improves the quality of insights over time.
The insight architecture’s current evidence
partly automated
The track record, current holdings and withdrawal record of the insight architecture can be reported at any time as a sealed snapshot that others can reproduce.
Reference runs
First reference run: twelve hypotheses on AI computing
partly automated
Twelve hypotheses about AI computing, recorded at the start of the run, are reported as they resolve, whatever the outcome.
The board application
Board oversight with and without assistance, 16 participants
run by hand
The board application improves the quality of oversight decisions, tested with each of 16 participants working both with it and without it.