By Model VS · Guide text updated . The pairs below are filled from the saved Arena snapshot and the saved LiveBench snapshot when the page is built. Model VS did not run the votes or the tasks. Report a correction with the two source ids and the file you expected.
What is each board counting?
Arena, in the overall slice we save, is a preference rating. People see two replies and pick one. The file stores a point rating, a lower bound, an upper bound, a vote count, and a raw rank. The raw rank is a sort of that rating. Overlapping bounds are a reason to keep both names on a conversational shortlist. The interval essay explains that reading in detail, including why a -high or -thinking suffix is a different record. This page assumes that rule and asks a narrower question: what to do when a task board orders the same string differently.
LiveBench, on the release named in our file, is a set of task columns grouped into categories. A category average is the unweighted mean of its task columns. Overall exists only when every category is complete, and it is the unweighted mean of those category averages. A Coding rank is a place among those complete rows. It is not a percent of your repository’s issues, and it is not a vote. The category guide is the place to match a job to a column. This page uses Coding and Overall only as two different LiveBench orderings that can disagree with Arena.
Those units cannot confirm each other. A person can prefer a reply that fails a checkable task. A model can lead a task column and still lose a side-by-side reading because the answer is curt, long, or in the wrong register. When that happens inside one product, the honest note is a disagreement, with the date of each file beside it. Methodology keeps the surfaces separate for this reason. The homepage can sort each surface. Sorting is a display. It does not elect a champion across surfaces.
A family nickname is a third way to invent a disagreement that the files do not contain. Arena may list claude-opus-4-6 while LiveBench lists a longer id that adds a thinking budget and an effort token. Those strings are different saved records. This page refuses them. The build keeps a pair only when the Arena model_name and the LiveBench model field are the same characters. If your procurement name is shorter than either id, you do not yet have a pair. Copy the id from the table you are citing.
Which exact ids reverse order here?
The build looks at overall Arena rows and at LiveBench rows that have a complete Overall. It then searches pairs of exact ids. One id must have the better Arena raw rank, meaning the smaller rank number. The other must have the better LiveBench rank on the scale being compared, again the smaller rank number. The pair with the largest combined gap is printed. A second search repeats the rule for LiveBench Overall and skips the pair already used for Coding, so the page shows two reversals when the files contain two. The gap is a selection key. It is not printed as a score, and it is not something you should paste into a memo.
LiveBench ranks in these tables are places among saved rows with a complete Overall, using the same category math as the rankings table. Arena ranks are the source raw ranks on the overall slice, not a rank we recompute from the rating. A Coding rank of 40 and an Arena rank of 10 are not “thirty places apart” in any shared league. They are places on different lists. Read the rating and the category average in their own columns. Read the interval before you treat the Arena order as settled. Read the vote count before you treat a wide interval as a mandate. The vote total and the band width are different questions, spelled out in Vote count versus interval width.
Exact source-id matches with a complete LiveBench Overall on this build: 18. A shared family name, or a suffix that differs by even one token, is not in this count.
| Role on this pair | Source model id | Arena organization | Arena raw rank | Arena rating | Published interval | Votes | LiveBench Coding rank | Coding average | LiveBench Overall rank | Overall |
|---|---|---|---|---|---|---|---|---|---|---|
| Better Arena raw rank | gemini-3.8-flash-high | 8 | 1494.77 | 1489.73–1499.82 | 26,298 | 54 | 72.49 | 34 | 75.83 | |
| Better LiveBench Coding rank | gpt-6-luna-max | openai | 90 | 1442.88 | 1436.34–1449.42 | 10,014 | 28 | 78.95 | 50 | 72.03 |
Reading the Arena pair in the order printed above, higher raw rank first. The first rating minus the second, before display rounding, is 51.89 points. The published intervals do not overlap. Separation still describes this snapshot's preference ratings, not the probability that one model will complete your task. LiveBench Coding ranks the same two ids in the opposite order on this snapshot.
| Role on this pair | Source model id | Arena organization | Arena raw rank | Arena rating | Published interval | Votes | LiveBench Overall rank | Overall average | LiveBench Overall rank | Overall |
|---|---|---|---|---|---|---|---|---|---|---|
| Better Arena raw rank | gemini-3.8-flash-high | 8 | 1494.77 | 1489.73–1499.82 | 26,298 | 34 | 75.83 | 34 | 75.83 | |
| Better LiveBench Overall rank | grok-4.7-xhigh | xai | 91 | 1442.46 | 1434.69–1450.23 | 6,405 | 25 | 77.40 | 25 | 77.40 |
Reading the Arena pair in the order printed above, higher raw rank first. The first rating minus the second, before display rounding, is 52.32 points. The published intervals do not overlap. Separation still describes this snapshot's preference ratings, not the probability that one model will complete your task. LiveBench Overall ranks the same two ids in the opposite order on this snapshot.
If a later sync removes one of these ids, the build will choose a different pair or say that no reversal remains. The rule stays: exact string, opposite order, both units kept. A pair whose intervals overlap is still a reversal on the point ratings and the raw ranks. Overlap means you are not owed a one-name preference winner. It does not erase the LiveBench order, and the LiveBench order does not erase the overlap. Write both facts down. The October changelog shows how a fixed list moves when ids enter or leave. Check the publish date in the caption before you quote a rank in a document that will outlive the week.
Absence is also a result. An id that sits on only one board is not a silent loss on the other. It was not in that file. Do not impute a rank for it, and do not average it with a cousin whose suffix differs. If you need the missing board, open the upstream site, record the id you actually saw, and then see whether our file still matches. We do not promise minute-level agreement with boards that can update after a save.
Which board matches the job?
Name the decision in one sentence before you look back at the pair. The sentence should say what the model must produce and what would make the output unacceptable. A preference question sounds like “which reply would a reader choose, given these two.” A task question sounds like “does the answer satisfy this check,” with the check named: a unit test, a schema, a citation that must appear, a math result. A submission question sounds like “what percent of a named issue set did this scaffold report.” Those are three decisions. The pair of tables above can inform the first two. It cannot answer the third. Coding-agent submissions live on the specialist snapshot and are read with their flags in the SWE-bench flag essay.
| Decision you are making | Read | Leave untouched |
|---|---|---|
| Which public reply people preferred | Arena rating, published interval, vote count, and the full source id | LiveBench category averages |
| Whether a release passed a named task family | That LiveBench category and the task columns inside it | Arena raw rank |
| What a coding-agent submission reported | The specialist row, its flag, and its card link | Both boards above |
| You need chat tone and a tested pipeline | Two shortlists, each labeled with its source and date | Any sum, mean, or single “winner” |
Use the first row of that table when the product is a public assistant and the risk is that people dislike the reply. Start with Arena intervals. If the two candidates overlap, keep both until a prompt of yours separates them. Use the second row when the pipeline has a check you can fail. Start with the LiveBench category that matches the check, and ignore the chat ordering until the check passes. Use the fourth row when both matter. A model can be on one shortlist and off the other. That split is a successful reading of a reversal, not a failed attempt to crown a winner.
Hard constraints come before either board. Data residency, a license you can accept, a context length, a tool API, and a price ceiling can remove an id that leads both tables. The boards do not know your contract. Provider documentation does, and it changes. If a constraint removes the id with the better Arena rank, the reversal is over for your shortlist. You do not compensate by borrowing the other id’s LiveBench score under the removed name.
What does an average destroy?
An average of an Arena rating and a LiveBench average destroys the unit, the date, the configuration, and the question. The result is a number nobody published. It will move when either file moves, and the movement will be unreadable, because a reader cannot tell whether people changed their votes or a task column changed. Model VS will not compute it, will not sort by it, and will not call the midpoint a better model. There is no model at the midpoint.
Rank arithmetic is the same mistake in disguise. “Arena rank plus Coding rank” treats a place on a preference list and a place on a task list as counters of the same kind. They are not. Subtracting them to see “how far the boards diverge” is useful only as the selection key this build already used, and only inside one snapshot, with the ids held fixed. The moment you show that key to a buyer as if it were quality, you have invented a third benchmark. We do not run that benchmark.
What you can publish, cleanly, is a sentence with both measurements and both dates. Shape: “On the Arena overall file published on the caption date, id A had the better raw rank and the printed interval. On LiveBench release named in the caption, id B had the better Coding rank. We did not combine them. The job we are hiring for is [the sentence].” That sentence survives a sync. A blended score does not.
Then leave the boards. How we rate is the protocol for a small check on your own cases: freeze the endpoint and the suffix, keep the failures in the denominator, and allow an inconclusive result. Three cases will not estimate a company-wide rate. They will tell you whether the reversal mattered for the sentence you wrote. If the preference id fails the check and the task id fails the tone, you may ship neither, or you may ship them in different parts of the product. Both outcomes are allowed. The boards have done their job when they forced that choice into the open.
Questions about a reversed ranking
Which board wins when the order flips?
Use the board that matches the job. Arena answers which reply people preferred in the saved vote. The named LiveBench category answers a scored task on that release. Keep both numbers when the job needs both.
Can two different suffixes count as one disagreement?
No. This page pairs a row only when the Arena model id and the LiveBench model id are the same string. A high suffix and an xhigh suffix are different records, even when the family name matches.
Does a reversal mean the higher Arena rank is wrong?
It means the two published orderings disagree for those exact ids on this snapshot. Arena still has an interval, and the LiveBench rank is a place among complete rows on one release. Neither ordering is your test suite.
Is there a combined score that settles the pair?
This site will not publish one. Adding a preference rating to a task average invents a unit that neither source defined. How we rate tells you to run your own cases instead.