September 12, 2026
A DATASET, A PROVIDER, AN API, AND A SCRAPED WEBSITE ARE DIFFERENT THINGS

Class 18 gave you the map: the ByeBuy Data directory lists thousands of possible sources across government, finance, weather, research, media, commerce, and your own organization. Now comes the harder skill. Finding a source is not the same as understanding what you found.
Beginners treat every page with numbers on it as "the data." Builders do not. A number on a random page tells you nothing about who produced it, how it arrives in your app, whether you may use it, or whether it will still be there next month. This lesson gives you the five words that prevent that mistake.
The five words
These sound similar. They are not interchangeable.
- Dataset — a collection of records. A CSV of historical unemployment rates. A table of company filings. A set of weather observations. A dataset answers: *what records exist?*
- Provider — the organization that owns, licenses, or delivers the information. The U.S. Securities and Exchange Commission. The Federal Reserve Bank of St. Louis. The World Bank. A dataset answers *what*; a provider answers *who stands behind it?*
- API — the supported software interface for retrieving data. Your app sends a structured request; the provider returns structured data, usually JSON. An API answers: *how does my program get it, reliably and repeatedly?*
- Website — a human-facing presentation layer. Designed for eyes, not programs. A website answers: *how does a person read it?*
- Scraping — extracting information from a rendered website with code that parses the page. Scraping answers: *how do I grab it without a supported interface?* — and that answer comes with two warnings: it is fragile (a redesign breaks your code) and it may be prohibited (the site's terms may forbid automated collection).
One provider can offer all of these at once. The Federal Reserve Bank of St. Louis is the provider behind FRED; the FRED dataset family contains thousands of economic series; the FRED API is the supported interface; the FRED website is the human presentation; and scraping the FRED website instead of using the API would be slower, more fragile, and entirely unnecessary.
Three sentences that save projects
Memorize these before you choose anything:
1. Authoritative does not mean convenient. The primary source is the most trustworthy — and often the least polished. SEC filing data is authoritative and read-only, but you must learn its structure and fair-access rules. 2. Free does not mean suitable. A free endpoint can have the wrong coverage, wrong update schedule, or wrong license for your product. 3. Scrapable does not mean production-ready. Just because your script *can* extract numbers from a page does not mean you *may* do so legally, or that the page will look the same tomorrow.
A model can help you explore — "what datasets does this provider publish? what does this field mean?" — but the builder must read four things personally: the terms of use, the documentation, the update schedule, and a sample response. No agent can accept legal terms on your behalf.
Four contrasting examples
This is why Part VI keeps returning to the same four sources. Each teaches a different shape:
| Source | Provider | What you get | Delivery method | Lesson it teaches |
|---|---|---|---|---|
| SEC EDGAR | U.S. Securities and Exchange Commission | Company filing history + XBRL financial facts in JSON | Public, read-only API; use a server or CLI, respect fair-access guidance | The authoritative source is sometimes deliberately plain — and that plainness *is* the reliability |
| FRED API | Federal Reserve Bank of St. Louis | Macroeconomic series with definitions and release timing | API key required; keep the series ID, units, and vintage with every fact | A source and an *interpretation* of that source are different things |
| World Bank Indicators API | World Bank | Country/development indicators | Readable public API, no key for basic use | Even "simple" public data needs exact indicator, geography, date, and unit |
| Open-Meteo API | Open-Meteo | Weather forecast + historical time series | Simple live request, rate limits apply | Forecast vs. observation, location, and timezone change the meaning of every row |
Notice the pattern: EDGAR gives you *filings, not prices*. FRED gives you *series with definitions, not answers*. The World Bank gives you *indicators that only mean something with their metadata*. Open-Meteo gives you *a forecast that expires*. None of them is "the data" in the abstract. Each is a specific dataset, from a specific provider, through a specific interface, with specific boundaries.
The ByeBuy Data directory is where discovery starts. It is not where evaluation ends. A directory listing tells you a source exists and points at the owner. It cannot tell you whether the license fits, whether the fields match your question, or whether the update rhythm works. That is Class 19's job, starting in the next lesson.
Visual: the source ladder
ORIGINAL OWNER (who vouches for the fact?)
e.g. SEC · statistical agency · World Bank · weather model operator
│
▼
PROVIDER / API (supported machine interface)
e.g. EDGAR APIs · FRED API · Indicators API · Open-Meteo API
│
▼
YOUR APPLICATION (retrieval → normalization → product rule)
│
▼
USER (decision, alert, report, recommendation)
═══════ separate route ═══════
CONSUMER WEBSITE (human presentation)
reading it ≠ permission to automate it
scraping it = fragile + possibly prohibited
Read the ladder top-down when choosing: owner first, interface second, your app third. Read a consumer website sideways: useful for *understanding*, dangerous as a *dependency*.
Practical exercise — 10-minute hands-on: Belgium unemployment
Pick one question: "What was the unemployment rate in Belgium?" Set a timer. You will touch three layers and record which one you can actually build on.
Minute 0–2 — The scraped page (what fails). 1. Search "Belgium unemployment rate" and open any top aggregator result (chart site, news page, search snippet). 2. Look for four fields: owner who vouches, indicator ID, geography code, unit + date. Write what you find. 3. What you will see: a number like "5.8%" with no indicator code, no BE geography code, no vintage, no terms page for automated reuse. That is a website — human presentation. Scraping it gives you fragile selectors plus unknown permission. Reject it for production here.
Minute 2–5 — The wrapper with provenance (what improves). 1. Open the human page: https://data.worldbank.org/indicator/SL.UEM.TOTL.NE.ZS?locations=BE 2. Confirm four fields on the page: indicator SL.UEM.TOTL.NE.ZS (Unemployment, total, % of total labor force), geography BE (Belgium), date (year), unit (%). Click the "Source" / "Methodology" note — it points to the International Labour Organization model, via national labor surveys. 3. Verdict so far: this is a provider + dataset (World Bank Indicators) wrapping an original owner (ILO / Statbel Belgium). Better than a random page, because owner, ID, geo, and unit are written down.
Minute 5–8 — The documented interface (what you build on). 1. Open the API URL directly in your browser: https://api.worldbank.org/v2/country/BE/indicator/SL.UEM.TOTL.NE.ZS?format=json&per_page=5 2. Confirm the JSON carries the same four fields: "indicator":{"id":"SL.UEM.TOTL.NE.ZS"}, "country":{"id":"BE"}, "date":"2023", "value":5.5, plus "unit" / source metadata on the indicator page. 3. If any of geography, date, or unit is missing, you have a page, not a source. Here all three are present — this is an API delivery method you can call repeatedly.
Minute 8–10 — Trace the primary org (who actually vouches?). 1. From the World Bank page, follow the source note to the origin: ILO modelled estimates, fed by Belgium's Statbel Labour Force Survey (https://statbel.fgov.be/en/themes/work-training/labour-market — check at build time). 2. Decide: for a prototype, the World Bank API is the pragmatic provider/interface (documented, stable, attributable). For a legal-grade claim about Belgian labor, the primary org is Statbel/ILO — cite them as owner on your card. 3. Run the ladder as a checklist: Owner named? Yes (World Bank wrapping ILO/Statbel). Interface documented? Yes (Indicators API URL above). App path clear? Yes (store indicator + BE + date + unit + retrieval time). Consumer page avoided as dependency? Yes.
Finish line: four lines — primary org + dataset/indicator + method/URL + one reason a random page fails (stale? unsourced? scraped? no license? no update schedule?).
Verify: open the method URL yourself and confirm it returns the indicator with geography (Belgium), date, and unit attached. If any of those three is missing, you have a page, not a source.
Common failure mode: choosing the prettiest chart from search results. Pretty is presentation. Your app needs provenance.
Check your understanding
1. Your script parses numbers off a company's investor-relations page. Which of the five words describes what you are doing — and what are its two risks? 2. Why might the authoritative provider be *harder* to use than a third-party wrapper — and why use it anyway? 3. What four things must the builder (not the model) read before committing to a source?
Transition
You now have the vocabulary. Next, you need the method: the five questions that turn any candidate source into an approved-or-rejected decision, recorded on a source card you — and your AI agent — can trust.
ARTICLE DISCUSSION
JOIN THE
CONVERSATION.
Got a question, a take, or a better way to do this? Log in and leave a comment.
