Leadership Under Uncertainty

The problem

Leading AI systems now forecast about as accurately as superforecasters, the best human forecasters. A forecast is only as good as the question asked, and it is a small part of how organizations build a strategic understanding and take decisions.

The approach

The project is building a judgment layer, the set of functions through which an AI system carries out the dynamic process of strategic assessment. Its judgments are kept in a dated record that others can check.

Where it could apply

The launch paper names two potential applications. The first is higher-value strategy work, such as framing an organization’s questions and connecting assessments held by different teams. The second is AI safety, because judgment is a key attribute of responsible decision-making.

Where it stands

Components of every stage of the process are built, and most already run in part on live data. The studies that would test the whole process are designed and open to collaborators.

In a controlled comparison run by Metaculus, a forecasting platform, its professional forecasters led the best automated systems by about twenty points a question on Metaculus’s scoring scale.

In the same comparison the professionals’ lead fell to 1.25 points a question, too small to distinguish from no lead.

Automated systems took first, second and fifth place in the Metaculus Summer 2026 Cup, an open tournament of 885 forecasters on 58 questions. Metaculus reported that this was the first time an AI forecaster had won the Cup.

John McCone, the Director of Central Intelligence, warned that Moscow might place offensive missiles in Cuba.

New air-defence sites were photographed in western Cuba. McCone reportedly read them as meant to blind American reconnaissance, and in September U-2 flights were routed away from them.

The United States Intelligence Board judged a deployment of Soviet ballistic missiles “incompatible with Soviet practice to date”.

Reports from a Cuban observer agent, the lowest rank of source, and two refugees had put a restricted area at the top of the list of reconnaissance targets. One flight over western Cuba, approved on 9 October, photographed medium-range missile launchers.

Systems built around language models now forecast about as accurately as superforecasters, and models write fluent analysis. A model used on its own lacks four things the process needs.

• A dated assessment that persists, so it can be revised and checked
• Consistent tradecraft, which all six widely used models tested lacked at times
• A reliable check, since models proved weak judges of analysis in early tests
• A line between an assessment and the authority to act on it

A judgment layer is meant to supply them, with its record kept outside any one model. A decision study specified in the launch paper would test whether it improves decisions, at two levels of model capability.

A representation of the actors, incentives and constraints in a situation that identifies the question that matters for a decision, names the observation that would discriminate between rival accounts and anticipates the next change.

The formation, integration, use and revision of assessments about what matters, what is credible and what warrants inquiry or action under uncertainty.

Explained change

Standing assessments run with and without the standard for explaining change, and analysts rate the explanations without knowing which version produced them.

Continuity across sessions

Whether a system that keeps memory between sessions revises its judgments consistently over time.

Shared origin

Planted reports that share one origin, and sources that impersonate known outlets, are placed in live evidence to see whether the system detects them.

Question-finding

Historical warning cases, with a disguised version of the September 1962 evidence among them, test whether a system finds the question that proved decisive.

Strategic Cognition Ontology

Nine elements, from the claim and its evidence to the reason for change and the information gap, in which people and AI systems can record a judgment and pass it between agents. A first test, removing one element at a time, found that seven of the eight elements tested carry weight.

Strategic Cognition Benchmark

Scores reasoning under uncertainty, recognition of limits, generation of alternatives, revision of belief within and across sessions, and integration of information from different sources. The overall score is capped for any system whose tradecraft, integration or continuity falls below a threshold, so forecasting skill alone cannot earn a high score. In a first run, all six widely used models tested fell short at times on sources, alternatives and gaps when not asked for them.

The Strategic Living Script

A short account of the user’s situation, the forces at work and the question that remains open. The system drafts it when a user joins.

Three strategic views

The main view, the condition it rests on, and the alternative if it proves wrong. Each has drivers, assumptions, dated indicators and a stated confidence.

The morning brief

What changed overnight, graded by the attention it needs before a decision. Linked decision records hold each decision and, in time, its outcome.

• Uploading documents into the dashboard
• Decision records, holding decisions and their outcomes
• Access for an organization’s own AI agents, without the project holding the organization’s credentials
• Connectors for email, Slack, document libraries and business systems

• Consent from the people whose material is read
• Agreements on where data is processed and which model providers see it
• Tested separation of each organization’s data
• No training on the organization’s data

A test on an organization’s own information would be a research collaboration, with data access, confidentiality and publication agreed for the collaboration, as Section 8 of the launch paper sets out. Documents would stay with the organization, and results would be released only in a form it has approved.

• Models may have read about past outcomes, which a test on old questions cannot rule out.
• Searches limited to an earlier date still return text shaped by later events.
• Questions picked after the fact flatter a method, and a gain measured on a curated set can shrink sharply on ordinary questions.
• Market prices close to the outcome already contain it.

• The method seals each judgment when it is made and scores it only against evidence that arrives later.
• A new method runs beside the current one on the same questions, and a person decides whether the new method replaces the current one.
• A fair comparison of two methods takes hundreds of paired questions, and so many months.

A seal fixes the date by which a record existed, so a record covering the same months takes the same months to build, for the project or anyone else.

• The design’s central safety claim is that keeping apart a possibility the system has generated, an assessment it has accepted and an action it is authorized to take, and enforcing the authorization whenever the system uses a tool, lets a system explore more widely without acting on more of what it explores. One arm of the decision study is designed to test it.
• The record shows what an AI system cited when it judged or acted, and anyone can check that a sealed entry has not changed.
• The project’s format for passing judgments between AI agents carries each judgment’s evidence and authority with it.
• The record has already exposed faults in the project’s own system that an editable log could have hidden.

About 42,600 records are sealed so far. A seal shows that a record existed, unchanged, by the time that day’s value was published, and it does not show that a judgment was right. The design addresses mistakes and unwarranted reliance, and it offers no general guarantee against misuse.

A sealed forecast on the 2026 World Cup

The project applies its method in public to three questions: what will set the pace of AI data centre construction in the United States to 2027, where Microsoft’s growth from AI comes from and whether it pays for itself, and how Cohere grows.

The records exist to show, in public and in a way anyone can check, how a stated judgment changes as evidence arrives, and holds when evidence does not. They are a research demonstration, not advice.

Anyone can follow each judgment as it changes and confirm that its record has not been altered. The system set the starting figures from the evidence, and a published rule, run each day, revises them as new evidence arrives. Each record’s page opens with its status and its three figures.

A review found defects in what the first records sealed on their first days, which a sealed record cannot change. Corrected records of all three questions began under revised rules, and the first records continue unchanged beside them.

• Each question is assessed through three assessments: a main assessment, the condition it rests on, and an alternative, each with its causal factors, assumptions and rival explanations.
• The stated confidence in each assessment changes with new evidence under a published rule, applied automatically each day. A second model challenges each material move before it can be applied, and in the corrected records every move.
• Each change is recorded with its reason and sealed, so a later alteration would be detected.

• The research website, where anyone can check a sealed record against the public anchor
• The sealed record, about 42,600 records, anchored in a public repository since June
• Reference series on AI, privately, as settings for the studies
• Research engines that screen world news and model fifteen states and twenty AI firms

• Companion papers posted to arXiv
• Studies 1 to 4 with collaborators, each pre-registered and sealed before it runs
• The historical track of the decision study, first half of 2027