Cast of Thousands

Document ranking · early access

Too many to read? Rank the pile.

Upload the submissions. Panels of AI judges compare them in pairs and hand back a ranked shortlist that shows which gaps are real and which are ties, with a reason for every comparison. You still decide. This tells you where to look.

Illustration · invented proposals and numbers

A community fund has 14 proposals and money for 5.3 are clearly in. 6 are competing for the last 2 places. 5 are clearly out.

  1. Clearly in
  2. On the line: read these closely
  3. Clearly out
Each bar is a proposal's likely range after many pairwise comparisons; the dot is its best estimate, the thin line is the funding line. Where bars cross the line, the evidence can't separate them, so the board says so. Tap a proposal to see its closest call.

What's in your pile?

Pick the closest. We're deciding what to build first, and this is how we find out.

Your pick is filled into the signup form at the bottom of the page, so you don't need to type it twice.

Scores pretend. Comparisons admit.

The usual way is to mark each submission alone against a rubric. Six proposals from the board above might come back like this.

Scored one at a time

  1. Repair café7.4
  2. Rooftop bees7.3
  3. Seed bank7.3
  4. Peer tutoring7.2
  5. Bike couriers7.2
  6. Oral histories7.1

Who gets the last two places? The decimals decide, and the rooftop bees are in.

Compared with each other

  1. Seed bank
  2. Peer tutoring
  3. Repair café
  4. Oral histories
  5. Bike couriers
  6. Rooftop bees

Head to head, the rooftop bees lost to every one of the others. The other five overlap the line: a tie, so your panel reads all five.

Illustration · invented numbers

Judged in pairs, by a panel

Each comparison is A or B?, asked in both orders so position can't sway it, by several AI judges. That's closer to how a real panel argues than marking each item alone.

Honest about ties

Every submission gets a range, not just a rank. If the evidence can't separate two, you see a tie. A similar idea is how the Swiss National Science Foundation ranks proposals with uncertainty and draws lots among those whose ranges cross its funding line.

A reason for every call

Each pairwise judgment comes with the judge's reason, so you can see why a submission rose or fell, and disagree with it.

Checked against your past rounds

Before trusting it, run it on a round you've already decided and see how far it agrees with your panel. If it doesn't, you'll know before it matters.

What we haven't shown yet

We have a working prototype, not a product, and no customers. We have not yet published evidence that our rankings match human panels. Our first public test will rank past conference submissions and compare them with the real accept and reject decisions. We'll publish the result, including where it falls short.

Why ranges matter: in a study of 2,267 UK research council grant applications, the best AI model's scores correlated with average reviewer scores at about 0.26, while individual reviewers agreed with each other at about 0.24 (Thelwall et al.). Neither number is high. Careful readers disagree, so a ranking should show where they would.

  • People decide. Nothing is rejected automatically. The ranking orders the reading; it doesn't replace it.
  • Your documents stay yours. We won't train models on them. Today the prototype sends texts only to the AI model you pick, through OpenRouter or Anthropic. Before anyone uploads a real round, we'll publish which providers see documents, how long anything is kept, and how to delete it.
  • Run it yourself. The engine will be open source, so you can rank locally with your own model keys.
  • Price not set. Tell us what a round is worth to you.

For developers

Rank anything from code, or let your agent do it.

Model outputs, generated ideas, candidate copy, search results: if your pipeline generates many and needs the best few, it needs a judge. We're building an API, an MCP server so agents can set up rankings themselves, and an open-source engine you can run locally.

Only about this project. Privacy.

Our working prototype (codename simlab), today. The hosted API will differ.
# attach a folder of texts, then run a judge tournament
$ simlab datasets add ./proposals --name round-7
$ simlab run --dataset round-7 --rubric fit \
    --model <judge> --rounds 8 --estimate
$ simlab runs results <run> --json
# → rank, score, 90% interval, judge checks

Bring us your pile.

Join the waitlist and tell us what you'd rank. We plan to help early users run their first round, starting with a back-test on a round they've already decided.

We'll only email you about this project. Privacy.