Compass

Mapping a field

A field is defined by its seed papers, search phrases, arXiv categories and the date it is defined as of. Serre follows the seeds' citations and references, adds recent papers close to the field, and ranks what it finds into a core, read in full, and a periphery, read by abstract.

Each paper in the core is read from its LaTeX source, or its PDF when there is none, and split into passages. Its results, methods, assumptions, limitations, open questions, negative results, theorems and definitions are recorded with the exact quote each rests on. A quote that cannot be found in the passage is rejected.

Planned: topics that keep their identity from one run to the next, and directions generated from the stated open problems and disagreements in a field.

The gauntlet

A concept is judged by a sequence of separate steps, each a model call of its own, never by whatever proposed it. Each step's answer is stored as an assessment, linked to the model call that produced it, and a rerun of the same input costs nothing.

Shape

Writes the concept down precisely: its thesis, its hypotheses, its formal claims, the smallest experiment that would decide it, the results that would kill it, the kind of contribution it makes, its research areas and the weeks of work it needs.

Novelty

A search of the literature, with every query kept. It names the nearest prior papers and how the concept differs from each, and returns open, partial or scooped. Novelty is a search with a trace, not a score.

Feasibility

What the work needs and whether it is available, the risks with their severity, and a verdict: feasible, feasible with risks, or infeasible.

Proof audit

For a concept with formal claims: restates each claim and checks it against known results, marking it likely true, unclear or likely false. It can raise doubt about a claim. It cannot prove one.

Venue review

A review at each candidate venue, written from the venue's policies, its profile and public reviews of the papers there most like the concept. It returns the objections most likely to sink the paper, how well the work fits the venue, and where its reception would fall among the venue's submissions.

Compare

Concepts that survive are compared two at a time, with all of their gauntlet evidence in front of the judge. The comparisons become the ranking.

Ranking

Comparing two concepts is a judgment models make reliably; scoring one on its own is not. So Compass ranks by pairwise comparison and fits a rating to the results, with an uncertainty that shrinks as a concept is compared more. Each round compares the concepts whose ratings are closest and least certain.

The submission plan

The plan answers where and when each judged concept should be submitted. It places concepts at venue deadlines over the next twelve months, a few papers in progress at a time, to maximise the papers expected to be accepted. Each placement shows its deadline chain and a fallback venue if the paper is rejected.

Named venues first

Venues a concept names are matched as people write them, tracks and workshops included. Every name that is not reviewed is reported with the reason.

Venues that fit

Other venues are added only when they share the concept's research areas, take its kind of contribution, and have a deadline far enough away to do the work. Calls that are not for papers, such as tutorials or workshop proposals, are never reviewed.

Venue rules

One archival venue per paper at a time. Concepts declared to overlap never share a venue cycle. A non-archival workshop can run alongside an archival submission, and the plan says to check that workshop's dual-submission rules.

Rolling journals

A journal such as TMLR has no deadline. The plan treats it as always open and says so in place of a table of dates.

Dossiers

Each concept has a dossier: its plan, every assessment with the evidence behind it, its standing in the ranking, its place in the submission plan with the venue's dates, and a recommendation. It is what a person reads before deciding at a gate.

Calibration

A venue review predicts where a paper would fall among a venue's submissions. Before that prediction may become an acceptance estimate, it is tested against the venue's real history: held-out submissions with public reviews, each reviewed exactly as a concept would be, without seeing its own reviews.

A run passes only when it scores at least 50 submissions, its ranking agrees with the real ratings (a rank correlation of at least 0.2), its probabilities are calibrated (an expected calibration error of at most 0.10), and it beats two simple baselines.

An estimated acceptance percentage appears only for a venue whose latest run passed. Everywhere else the plan orders venues by fit and labels the number uncalibrated, and the dossier says it is not a probability. A venue without public reviews is never calibrated.

Decisions on the record

Compass recommends; a person decides. At a gate the choices are the ones valid there, such as commit, park or kill, and each decision is logged with its reason and the evidence the concept had at that moment. A decision made on a page that has since gone stale is refused, not applied.

Try it

Compass is in closed testing as part of the Serre platform. Request access