In Lesson 8.1 you learned to assign each job to a model category. Now you need candidates — actual model names to shortlist. The internet will hand you a ranked list and call it the answer. A leaderboard is useful evidence. It is not a verdict.

Why leaderboards exist (and what they actually measure)

A leaderboard is a scoreboard for a specific game. Change the game — web UI building versus data analysis versus medical questions — and the rankings change. That is the first durable idea: a ranking is always "best at this measured thing, by this method, at this time."

The board we will use in this class is Arena, specifically its WebDev and Data & Analytics views. It ranks models on those tasks, shows what they cost, how much context they accept, and how much voting evidence sits behind each rank. That combination — capability signal plus price plus context plus vote depth — is what makes it useful for a builder. A bare "top 10 models" screenshot with none of that context is entertainment, not evidence.

A second durable idea: crowd preference is not the same as task success. A model can win votes for fluent, confident answers and still fail your acceptance test. Treat the leaderboard as a way to find candidates, then let your own test decide.

How to read the Arena board, column by column

Open Arena's WebDev / Data & Analytics leaderboard. Before looking at rank 1, read the controls and columns the way a builder reads a spec sheet.

Leaderboard row, decoded:

  Rank (± spread) + votes   →  how strong is the evidence?
  Category filter           →  is this even my game?
  Input / output price      →  what does real use cost?
  Context length            →  does my job fit inside it?
  Preliminary flag          →  should I wait before deciding?

Walk a row slowly. Suppose a model sits at #1 on WebDev with a narrow spread and thousands of votes — strong evidence *for WebDev*. Now check its output price and context length. If your task is "refactor one small component on a $2 budget," a model at #6 that costs one-tenth as much and still fits your repo may be the better assignment. The #1 model did not get worse. Your job just does not need what it charges for.

Three classic traps, stated plainly:

1. Overall #1 worship. The top overall model may be slow, expensive, or oversized for routine work. Rank measures capability on the board's game, not value for your job. 2. Ignoring the spread and votes. A three-place gap means nothing if the spreads overlap or the higher model has a tenth of the votes. That is noise, not superiority. 3. Price and context blindness. A leaderboard sorted by score hides the two columns that decide real projects: what it costs per million tokens and whether your files fit. Always read those columns before shortlisting.

The ByeBuy model-selection card

To force yourself past "it is ranked highest," fill in this card before you pick. It takes two minutes and prevents most bad model decisions.

Now the shortlist logic writes itself:

• Filter Arena to the matching category (WebDev for this example, Data & Analytics for a dashboard query task).
• Discard anything whose context length cannot hold your files, or whose price breaks your budget.
• Among the survivors, prefer narrow spreads and healthy vote counts.
• Shortlist three candidates: a strong default, a cheaper/faster alternative, and one backup from a different provider family (useful if one provider is slow or down — a preview of Class 10's routing ideas).

Example of a defended shortlist (names will age — the reasoning is the point):

Your test beats their test

Here is the hierarchy to memorize:

Leaderboard: finds candidatesShortlist of 3Your fixed task + your files + your acceptance testDecision: default model for this job

A leaderboard measures *their* tasks with *their* judges. Your project measures *your* task with *your* tests. So once you have three candidates, run the same bounded task through each — same files, same prompt, same acceptance criteria — and record what happened. Keep the brief identical; otherwise you cannot tell whether a better result came from the model or from the task quietly changing. (Class 06's "keep the brief fixed" rule and Class 11's comparison scorecard return to this method.)

This is also why you write down which model produced material output. When a result needs review next month, "Model A, WebDev-filtered shortlist, tests passed on this date" is provenance. "The AI did it" is not.

Practical exercise: build a real shortlist

1. Pick one defined task. Example: "Build a small ByeBuy product-grid component from this spec" (WebDev) or "Answer three questions from this sales CSV and show the queries" (Data & Analytics). 2. Fill in a model-selection card: task, what matters most, budget, context needed, verification. 3. Open Arena, set the matching category filter, and read rank, spread, votes, input/output price, context length, and any preliminary flag for at least five rows. 4. Shortlist three candidates with one sentence of defense each. No sentence may be only "it is ranked highest."

Finish line: a written card plus a three-model shortlist with price, context, and evidence notes — saved in your notes. Example: "Model X: WebDev rank 3, narrow spread, 12k votes, $3/$15 per M, 200k context — fits my repo and budget as default."

Quick verification: show your shortlist to a colleague (or yourself tomorrow) and ask: could they reproduce your reasoning from the card alone? If the answer depends on remembering a rank number, add the missing tradeoff.

Common failure mode: screenshotting the top of the board and stopping. Rankings decay within weeks; a card that records *why* each candidate fit survives every leaderboard refresh.

Check your understanding

1. What does a rank on Arena's WebDev board actually claim — and what does it not claim? 2. Why can the #1 model be the wrong choice for a small, cheap, routine task? 3. What do rank spread and vote count tell you that the rank number alone hides? 4. Which two columns decide whether a high-ranked model fits your real project? 5. Why is your own fixed test stronger evidence than any public leaderboard position?

Next, in Lesson 8.3, the words "open," "open-weight," and "local" stop being blurry: you will learn what you can actually download, host, and run — and how to read a model card on Hugging Face before trusting it.