ByeBuy.ai
BUILD YOUR ESCAPE ROUTE · ✦ CURSOR · HOST IT · ◫ SUPABASE · CONNECT IT · ↯ RELAY · BUILD YOUR ESCAPE ROUTE · ✦ CURSOR · HOST IT · ◫ SUPABASE · CONNECT IT · ↯ RELAY ·
← BYEBUY NOTES

September 12, 2026

OBJECT STORAGE: WHERE FILES ACTUALLY LIVE

Object Storage: Where Files Actually Live

Lesson 25.4 gave every job an owner — and one row deserves its own lesson, because beginners misplace it more than any other. User uploads do not belong in the database next to user names. They do not belong inside a disposable container either. They belong in object storage, with the database holding only the facts *about* each file.

The database holds facts about the file, not the bytes

A database row for an uploaded PDF should look like this: owner, title, file type, size, permission level, storage path, processing status, upload date. Notice what is missing: the PDF itself. Large PDFs, images, video, audio, and exports do not belong as raw bytes in Postgres. Bloating database rows with megabytes makes backups slow, queries heavy, and restores painful — and databases cannot serve file downloads efficiently anyway.

The split is clean: Postgres remembers; storage holds. The database row is the card catalog entry. The file waits on a different shelf, fetched only when someone authorized asks for it.

What object storage is

Object storage is a durable cloud filing system that stores files as objects inside named containers called buckets. Each object has three parts: the file body (the bytes), a key (its path-like name inside the bucket, e.g. uploads/user-42/report-2026-09.pdf), and metadata (content type, size, upload time, custom tags).

Two focused examples: Supabase Storage, which pairs buckets with access policies and signed URLs inside the Supabase ecosystem, and Cloudflare R2, S3-compatible storage inside the Cloudflare ecosystem. You will also meet Amazon S3 everywhere in documentation — treat it as the widely used category reference for object storage concepts, not as a setup tutorial for this course.

The real product flow

Follow one PDF through a ByeBuy-style research product:

user uploads PDF
  → private object storage holds the PDF bytes
    → Postgres records owner / path / status
      → worker extracts text, creates chunks and embeddings
        → authorized user receives a temporary download link

Each handoff has an owner from Lesson 25.4's table. The app accepts the upload and writes bytes to a private bucket. Postgres records *who, what, where, and whether processed*. A background worker — not the request — does the slow extraction and embedding work (Class 26 owns that pattern). When the owner later clicks download, the app hands them a link that works briefly, for that file, for them.

Public versus private, and the signed URL

Two policies cover nearly everything. Public assets — site logos, marketing images, open sample datasets — may be delivered openly to anyone. Private user documents — uploads, invoices, research files, internal media — need explicit access rules, and the default is deny.

The mechanism that makes private files shareable without opening the bucket is the signed URL: a time-limited link the app generates so one user can upload or download one private object without ever seeing storage credentials and without the bucket going public. It says, in effect: "whoever holds this exact link may read this exact file until Friday at noon." The app checks permission first ("does this user own this file?"), generates the link, and the link expires on its own. No credential ever reaches the browser; no bucket policy was loosened.

Direct connection to RAG

If Part VI felt distant, here it docks. In a RAG product the four pieces sit in four places:

  • Original source file → object storage (the PDF itself).
  • Document and chunk metadata → Postgres (title, owner, section, status).
  • Embeddings → alongside the chunk records or the vector layer (Class 22's design).
  • Access checks → enforced *before* retrieval or download, every time.

Retrieval reads chunks the user is allowed to see; download serves files the user is allowed to hold. Skip the check at either gate and the system leaks private material with citations attached.

The five file rules

1. Private by default. New buckets deny public access until a deliberate decision opens a specific path. 2. Generated paths. Never let the uploader's filename become the key directly — build keys from IDs and timestamps (uploads/{user-id}/{doc-id}.pdf) to avoid collisions and path tricks. 3. Type and size checks. Validate content type and size at upload; a "PDF-only, 25 MB max" rule enforced in code beats any policy document. 4. Retention and deletion plan. State how long each file type lives and what deletes it — user action, expiry job, or legal hold. 5. Tested restore path. Periodically download a file from a fresh session and confirm it opens. An untested backup, as Class 26 will insist, is a hopeful file.

Practical exercise: write FILE-LIFECYCLE.md

Create FILE-LIFECYCLE.md for one file type your product accepts (upload PDF, avatar image, exported report — pick one). Record:

  • Bucket: which bucket holds it, public or private.
  • Database fields: owner, path/key, type, size, permission, status — name the actual columns.
  • Policy: who can upload, who can download, what the app checks first.
  • Processing job: which worker acts on it and what it writes back (text, chunks, embeddings, thumbnail).
  • Retention period and deletion path: how long it lives, what triggers deletion, and what deletes the database row versus the bytes.

Finish line: a file from which you can say where the actual bytes live, where the metadata lives, and how an authorized person gets the file back.

Verify: pick a file and trace it aloud: bytes where, metadata where, who may download, which link mechanism, what deletes it. Any hesitation names the gap to fill.

Common failure mode: storing raw bytes in a database column "temporarily" and building retrieval directly on them. It works for three files and collapses at three thousand — slow queries, giant backups, no signed-URL story. Split bytes from facts on day one.

Check your understanding

1. What three parts make up a stored object, and what identifies it inside a bucket? 2. Why must the app check permission before generating a signed URL? 3. In a RAG product, where do the source file, the chunk metadata, and the embeddings each live? 4. What is the difference between deleting a database row and deleting the file?

ARTICLE DISCUSSION

JOIN THE
CONVERSATION.

0 COMMENTS

BYEBUY ACCOUNT ACCESS

Sign in

Use your account to save routes and make the catalogue yours.

Enter your email and we’ll send a secure sign-in link and code.

NEW ROUTES ADDED WEEKLY · 9,235 CATALOGUE ENTRIES · BUILD · DEPLOY · QUERY · STACK · SAY BYE TO BUY · NEW ROUTES ADDED WEEKLY · 9,235 CATALOGUE ENTRIES · BUILD · DEPLOY · QUERY · STACK · SAY BYE TO BUY ·