ByeBuy.ai
BUILD YOUR ESCAPE ROUTE · ✦ CURSOR · HOST IT · ◫ SUPABASE · CONNECT IT · ↯ RELAY · BUILD YOUR ESCAPE ROUTE · ✦ CURSOR · HOST IT · ◫ SUPABASE · CONNECT IT · ↯ RELAY ·
CURRICULUM
← BYEBUY NOTES

September 13, 2026

MEASURE AUTOMATION VALUE BEFORE EXPANDING IT

ByeBuy.ai artwork for Measure Automation Value Before Expanding It

Lessons 88.1–88.4 built a careful workflow: verified, housed, permissioned, and failure-proofed. Now the temptation — automate five more things because the first one works. Resist it for thirty days. An automation that nobody measures becomes permanent whether or not it helps. This lesson gives it a trial period with a verdict.

The vocabulary, in plain language

  • Baseline: how the job performed before automation — minutes per run, errors per hundred, customer wait. Without it, "faster" means nothing.
  • Automation rate: share of runs completing with no human touch. High is nice; 100% is not the goal (some runs should touch a human).
  • Exception rate: share landing in the error or review queue. The honesty metric — a low rate with a hidden trash folder is lying.
  • Rework: human fixes after the automation "completed" — corrected CRM entries, resent emails, apologetic follow-ups. Invisible rework is where fake savings hide.
  • Time saved: (baseline minutes − review minutes) × runs, minus queue-review and fix time. Net, not gross.
  • Error cost: what mistakes cost — refunds, credits, lost trust, support load. One expensive error can erase a hundred fast runs.
  • Customer impact: did the customer feel anything better — faster confirmation, fewer errors, clearer messages? The only metric that ultimately matters.

The mental model: a probation period for a new hire. You set expectations (baseline), watch for thirty days (rates and rework), tally the true cost (errors), ask the customers (impact), then decide: promote, coach, reassign, or let go.

The 30-day review and its four verdicts

Schedule the review before launch — owner, date, data sources (logs from 88.1, queue from 88.4, scorecard from 84.5). At day 30, one verdict:

  • Improve — metrics beat baseline on time and quality, exceptions low and handled. Expand volume or adjacent steps cautiously.
  • Revise — helps sometimes but exceptions or rework are high. Narrow the scope, fix the top failure case, re-run 30 days. Most first automations land here.
  • Pause — vendor unreliable, volumes too low to justify upkeep, or owner overloaded. Park it; keep the contract and logs for later.
  • Retire — error cost exceeds time saved, or customers prefer the human path. Shutting down a bad automation is a success, not a failure.

Cost connects here: per-task platform fees, API and model spend, and infra all count against time saved — Lesson 89.2 turns this into standing guardrails. Platform usage pages (Make help, Zapier help, n8n docs) show where the numbers come from.

Exercise: write the review

Create AUTOMATION-REVIEW.md:

# AUTOMATION-REVIEW.md — [Workflow] — [Date]

## Baseline (pre-automation)
- Minutes/run ___ / errors per 100 ___ / customer wait ___ / monthly runs ___

## 30-day results
- Automation rate ___% / exception rate ___% / rework ___ cases
- Time saved (net of review+fix): ___ hrs
- Error cost: ___ (refunds/credits/incidents)
- Customer impact (evidence, not vibes): ___
- Platform/API cost: ___ (source: ___)

## Verdict: IMPROVE / REVISE / PAUSE / RETIRE
- Reason in two sentences: ___
- One next action + owner + date: ___

Worked mini-example — Sonariq intake triage: baseline 20 min per request, 60/month. After 30 days: automation rate 70%, exception rate 12% (mostly ambiguous briefs, handled in queue), rework 3 cases, net saved 11 hours, one misrouted brief cost an apology call. Verdict: REVISE — tighten the ambiguity detector, re-run 30 days before expanding to auto-assignment. The honest 12% exception rate is what makes the 11 saved hours believable.

Finish line: a dated AUTOMATION-REVIEW.md with baseline, both rates, net time saved, error cost, customer evidence, and one of the four verdicts.

Verify quickly: ask "could a skeptic reproduce these numbers from the logs?" If time saved ignores queue-review hours, or customer impact has no quote or ticket data, redo it.

Common failure mode: the vanity dashboard — automation rate 98% celebrated while rework, error cost, and platform fees sit off-screen. Measure the whole job or don't claim the win.

Check your understanding

1. Why must time saved be net of queue-review and fix time? 2. Which verdict fits "fast but customers complain more" — and what happens next? 3. What evidence counts for customer impact, and what doesn't?

Next

Class 88 complete: you connect work carefully — verified events, fitting homes, narrow keys, honest failures, measured value. Class 89 scales what earned it: quality systems, cost controls, runbooks, deliberate human boundaries, and a weekly operating review. Lesson 89.1 starts with quality control as part of the system.

ARTICLE DISCUSSION

JOIN THE
CONVERSATION.

0 COMMENTS

BYEBUY ACCOUNT ACCESS

Sign in

Use your account to save routes and make the catalogue yours.

Enter your email and we’ll send a secure sign-in link and code.

NEW ROUTES ADDED WEEKLY · 9,235 CATALOGUE ENTRIES · BUILD · DEPLOY · QUERY · STACK · SAY BYE TO BUY · NEW ROUTES ADDED WEEKLY · 9,235 CATALOGUE ENTRIES · BUILD · DEPLOY · QUERY · STACK · SAY BYE TO BUY ·