Document ranking · early access
Too many to read? Rank the pile.
Upload the submissions. Panels of AI judges compare them in pairs and hand back a ranked shortlist that shows which gaps are real and which are ties, with a reason for every comparison. You still decide. This tells you where to look.
Illustration · invented proposals and numbers
A community fund has 14 proposals and money for 5.3 are clearly in. 6 are competing for the last 2 places. 5 are clearly out.
- Clearly in
Closest call Beat the tool library 4–2. “Both are useful; this one names its first 40 users and what they lose to spoilage now.”
Closest call Beat the clinic translations 5–1. “Running costs are spelled out; the other proposal leaves them to year two.”
Closest call Beat the night-bus audit 4–2. “Both are needed; this one already has the clinics signed up.”
- On the line: read these closely
Closest call Lost to the clinic translations 2–4, but the judges kept changing their minds. “Strong method, but the budget is a single line.”
Closest call Beat peer tutoring 4–2. “Clear plan for who keeps it going after the grant ends.”
Closest call Split 3–3 with the repair café. Neither says how they would know it worked.
Closest call Split 3–3 with peer tutoring. “Lovely idea; the site isn't secured yet.”
Closest call Lost to the repair café 2–4 and beat the courier scheme 4–2. Firmly mid-table, firmly unsure.
Closest call Lost to the oral histories 2–4. “Needs a pharmacy partner that isn't named.”
- Clearly out
Closest call Lost to the courier scheme 1–5. “Charming, but most of the budget is equipment with no plan for it.”
Closest call Lost to the rooftop bees 2–4. “Several such apps already exist; the proposal doesn't mention them.”
Closest call Lost to the toilet finder 2–4. “No sign anyone in the area was asked.”
Closest call Lost to the murals 1–5. “The audience is described as ‘everyone’.”
Closest call Lost every comparison it was in. “Off-brief for a community fund.”
What's in your pile?
Pick the closest. We're deciding what to build first, and this is how we find out.
Your pick is filled into the signup form at the bottom of the page, so you don't need to type it twice.
Scores pretend. Comparisons admit.
The usual way is to mark each submission alone against a rubric. Six proposals from the board above might come back like this.
Scored one at a time
- Repair café7.4
- Rooftop bees7.3
- Seed bank7.3
- Peer tutoring7.2
- Bike couriers7.2
- Oral histories7.1
Who gets the last two places? The decimals decide, and the rooftop bees are in.
Compared with each other
- Seed bank
- Peer tutoring
- Repair café
- Oral histories
- Bike couriers
- Rooftop bees
Head to head, the rooftop bees lost to every one of the others. The other five overlap the line: a tie, so your panel reads all five.
Illustration · invented numbers
Judged in pairs, by a panel
Each comparison is A or B?, asked in both orders so position can't sway it, by several AI judges. That's closer to how a real panel argues than marking each item alone.
Honest about ties
Every submission gets a range, not just a rank. If the evidence can't separate two, you see a tie. A similar idea is how the Swiss National Science Foundation ranks proposals with uncertainty and draws lots among those whose ranges cross its funding line.
A reason for every call
Each pairwise judgment comes with the judge's reason, so you can see why a submission rose or fell, and disagree with it.
Checked against your past rounds
Before trusting it, run it on a round you've already decided and see how far it agrees with your panel. If it doesn't, you'll know before it matters.
What we haven't shown yet
We have a working prototype, not a product, and no customers. We have not yet published evidence that our rankings match human panels. Our first public test will rank past conference submissions and compare them with the real accept and reject decisions. We'll publish the result, including where it falls short.
Why ranges matter: in a study of 2,267 UK research council grant applications, the best AI model's scores correlated with average reviewer scores at about 0.26, while individual reviewers agreed with each other at about 0.24 (Thelwall et al.). Neither number is high. Careful readers disagree, so a ranking should show where they would.
- People decide. Nothing is rejected automatically. The ranking orders the reading; it doesn't replace it.
- Your documents stay yours. We won't train models on them. Today the prototype sends texts only to the AI model you pick, through OpenRouter or Anthropic. Before anyone uploads a real round, we'll publish which providers see documents, how long anything is kept, and how to delete it.
- Run it yourself. The engine will be open source, so you can rank locally with your own model keys.
- Price not set. Tell us what a round is worth to you.
For developers
Rank anything from code, or let your agent do it.
Model outputs, generated ideas, candidate copy, search results: if your pipeline generates many and needs the best few, it needs a judge. We're building an API, an MCP server so agents can set up rankings themselves, and an open-source engine you can run locally.
# attach a folder of texts, then run a judge tournament
$ simlab datasets add ./proposals --name round-7
$ simlab run --dataset round-7 --rubric fit \
--model <judge> --rounds 8 --estimate
$ simlab runs results <run> --json
# → rank, score, 90% interval, judge checksBring us your pile.
Join the waitlist and tell us what you'd rank. We plan to help early users run their first round, starting with a back-test on a round they've already decided.