Independent research on strategic cognition, forecasting quality, and decision-making under uncertainty.
Publications forthcoming. For inquiries: chris@leadershipunderuncertainty.org
An independent research project investigating how to measure and improve strategic cognition — the reasoning that analysts and decision-makers use when working under uncertainty. We are developing open benchmarks and formal standards for evaluating this capability in both humans and AI systems.
SCB is a two-layer evaluation framework for strategic-analytic reasoning. Layer 1 measures forecasting accuracy using proper scoring rules (Brier scores, calibration curves). Layer 2 measures analytic tradecraft quality against ICD 203 — the Intelligence Community Directive that defines how professional analysts should reason under uncertainty. Most existing AI benchmarks measure only whether the answer is right; SCB also measures whether the reasoning process is sound.
Intelligence Community Directive 203 ("Analytic Standards") is the U.S. Intelligence Community's authoritative standard for analytic quality. It requires analysts to express uncertainty with calibrated confidence levels, identify assumptions, consider alternatives, and distinguish between evidence and judgment. These are exactly the reasoning skills that matter for strategic decision-making — and that current AI evaluations largely ignore. SCB operationalises ICD 203 into a measurable scoring rubric.
SCO is a formal reference ontology that defines the concepts, relationships, and constraints involved in strategic cognition. It is aligned with the Basic Formal Ontology (BFO), which is now mandated as the top-level ontology across U.S. Department of Defense and Intelligence Community systems (per the January 2024 ODNI/CDAO memorandum). SCO provides the structured vocabulary that SCB uses to describe what it measures.
Yes. An interactive SCO demo is available that lets you explore example ontology instances and see how strategic cognition concepts are structured.
Try the SCO Demo →Both papers are in final preparation. The SCB paper will be released first, followed by the SCO paper approximately three weeks later. Both will be submitted to arXiv (cs.AI, with cs.CL cross-list) and published in dedicated public GitHub repositories under an open license.
The papers are released under CC-BY 4.0. All code, including the SCB validator and conformance test suite, is released under the Apache 2.0 license. We want these standards to be adopted, not gated.
Leadership Under Uncertainty is an independent research project by Christopher Berzins. The work draws on analytic tradecraft standards, forecasting science, and formal ontology to address a gap in how we evaluate strategic reasoning capability.
If you work in AI evaluation, intelligence analysis, forecasting, or formal ontology and are interested in reviewing, testing, or collaborating on these standards, please reach out by email. We are particularly interested in hearing from researchers who would like to run the SCB validator against their own systems.