Brief review
Paste a board memo, an investment note, a policy brief or a market update. The review reads it on seven points: where its claims come from, what it assumes, what alternatives it weighs, how it states confidence, whether judgment is kept apart from evidence, whether it holds together and whether it answers the question asked. It scores the draft, names the changes that would improve it most, marks each paragraph that falls short and suggests a way to fix the first finding.
A review takes about ten seconds. The brief is sent to a model provider for the reading. The project keeps the score and a trace identifier, holds the model’s reading for up to an hour, and does not store the brief. The samples are synthetic briefs from the project’s test set, and every company and figure in them is invented.
After a review the brief stays editable, and a second review shows what changed. The same review in the vocabulary analysts use, naming the standards it reads against, is at the review for analysts.
Seven points a single brief can be read on
The review reads the text and not the world: it does not check that a cited report exists or says what the brief says it says, and it does not judge whether a conclusion turned out right. The score combines a model’s reading of the brief’s structure, its reading on the seven points and a set of fixed checks on the text, and a model’s reading can be wrong. The same engine and the same score serve the review for analysts, and the figures below are from the same kind of test, run on business briefs.
Run five times on each of the six synthetic business briefs in the project’s test set, two board memos, two investment notes, a policy brief and a market update, the score moved by at most one point for any brief and the verdict did not change. The three briefs written with their sources, assumptions, alternatives and confidence stated scored 81 to 84, and the three without them 16 to 18. A review took between six and nine seconds. How far the score agrees with an experienced reader’s grading of real board papers, research notes or policy briefs has not been measured. The review has not been measured on documents longer than about 2,500 characters, or on documents whose argument rests on tables and charts, which it does not read.