Skip to content

How an expert’s record is counted

When an expert commits to a position, we come back on the date to see what happened, and count it. This page is the whole method: what counts, what doesn’t, who decides, and when a number is allowed to be shown at all.

The experts are AI — named roles, not people.

What a position is

A position is an AI expert’s committed view on your matter at one moment: the answer itself, how sure the expert is, what would change its mind, and the date we will know by. It can be a decision (take the offer or not), a recommendation, an estimate, a draft or an answer.

A position on the record, not a licensed professional’s advice — and we come back on the date to see if it held.

The experts are AI — named roles, not people. A record is a count of what happened after past positions. It is not a promise about yours, and it is not a licensed professional’s advice.


Who settles it

You do. On the date you named, she asks two things: what you did, and later, how it went. Only your answer counts toward the record.

  • An answer from an AI assistant you connected, or one she inferred, waits as “pending” until you confirm it. Until then it counts for nothing.
  • Before a position is kept, the expert states how it will be settled: what will be observed, who observes it (you, a connected tool’s number, or a date), by when, what “as called” looks like, what “against it” looks like, and what would leave it impossible to tell. A position whose test is too vague to settle is marked weak.
  • Saying “still open” moves the date once. A second time, the position lapses and counts for no one.
  • If you don’t answer, she asks once more on the other channel after 7 days. After 14 more days of silence it closes as no word — its own state, never counted as a miss.

What counts as held, partly, missed

Each position is scored against the path you actually took, not against a path you didn’t:

Your answerIf you followed the positionIf it warned against what you did
It went wellHeldMissed — or not scored when going against it leaves the other path unknown
MixedPartly (counts as half)Partly
It went badlyMissedHeld
Can’t tellShown as “can’t tell”, never as a hit or a missSame
  • Not scored: a position you didn’t follow whose outcome depends on that choice (nobody can know how the other path would have gone); a recommendation, draft or answer nobody used; anything still open.
  • Numbers and dated events that your choice doesn’t move (a price on a date, a figure from a connected tool) are scored on how they came out, followed or not.
  • Late answers: an outcome reported more than 90 days after the position scores no one.
  • How sure: the expert’s stated confidence is compared with what happened (a Brier score), so a confident miss costs more than a hesitant one.
  • Edits: a position is never rewritten. A change is a new, dated row beside the original; only the original is scored.

Keeping the count fair

  • Within the bench. On a board, each expert’s result is also read against the other experts on the same matter: its score minus the bench’s average on that matter. Easy matters lift everyone equally, so they lift no one’s peer score.
  • Small records are pulled toward the middle. A record is shown as if it had 5 more positions that went the way matters like it usually go (or at even odds when we don’t know). Two hits out of two does not read as perfect.
  • The whole population. Every count shows the resolved positions, the held, partly and missed, and the “can’t tell” and “no word” beside them — not only the hits.
  • Difficulty and time. Counts carry the mix of easy and hard matters, the last twelve months beside the lifetime, how often an expert was seated as the one voice against and how often that voice held, and the median days from position to answer.

When a number may be shown

Resolved positionsWhat is shown
Under 5No number. The record says it is provisional and still building.
5 to 9Counts only (“3 of 7 held”), never a percentage.
10 or moreA percentage, and two experts side by side.
20 or moreOnly then may one expert be called better than another at something.

Across people. An expert’s public count exists only from 5 resolved positions, and any count across people needs at least 5 different people and 10 resolved positions in that cell. A cell that would let someone subtract their way to a smaller group is hidden too. A custom expert’s count is never public until 5 different people have settled positions with it.

Your own record is never public. It is yours: you can see it, export it, and share a card of it only by your own tap. Turning off “Score my own answers” in Settings → Preferences stops your own first answers being scored at all.


What the number is not

  • Not a promise. A count of past positions says nothing certain about the next one.
  • Not accuracy, skill or a licence. The experts are AI roles with names; none is a lawyer, accountant, adviser or clinician, and a count does not make one.
  • Not a judgement about you. Nothing is decided about you from your record — by us, by a workspace, or by anyone else.

What we keep behind every public number

For each public count: the list of positions behind it (ids, dates, domain, shape, stance), how each was settled and the person’s own words for the outcome, the definitions on this page and their version, the window, the number of distinct people, how matters in that cell usually went, and the date and code version it was computed with. As of the date above, no public count is shown anywhere: the public record stays switched off until this page is live.


Change log

  • 18 October 2026 — version 1.0, first published.