If you ship it at an AI-first company, or hell, even as a solo producer, everyone's ideas should be flowing into code like melting butter, and that code needs to be verified somehow. If the shipping speed is right, there will be no way you can deeply review and test all the code yourself. Luckily, agents are becoming increasingly great at reviewing and QAing your PRs. The machine-QA part isn't optional anymore. You can't QA and find holes yourself anymore all the time at that speed. And that was never the fun part anyway. If you are wired like most devs, you just want the thing to work with poof. Proof means automated tests and, most recently, automated QA.
My most recent process for reasonably complex PRs is:
- Open PR in new worktree.
- Use
/manual-testskill to QA the expected behavior.
- For an API PR, it creates a table with columns like:
| Result | Request | Why | HTTP / test result | Data received and reaction |
|---|
- Result can be success ✅, ❌ unexpected, ⚠️ ambiguous (needs clarification).
- I have the AI write a report with the Good, Bad and Ugly.
- Based on the results, I fix the unexpected ones ❌ in which case they become "fixed" ⛑️. The LLM then submits my Approval with the test matrix, the exact cURLs or UI steps taken to test using my custom
/review-commentskill. - Or, pass them to the initial assignee if something seems generally off or is just complex/ambiguous.
I run this process for others' PRs and for my own (except step 1. in the latter case, no need for a new worktree then). If you haven't started yet, building the infrastructure and muscle memory for this now is necessary for the future of shipping.
[AIN#29] AI Nuggets Week#29
Model drop week. All the nerdiest nerds (me) running Fable against Sol against Kimi (haven't tried the latter yet tho, just started digging, waiting to run out of Codex resets, which is my current backup for Claude.
📺 Expert Knowledge YouTube Nuggets
AI Coding is exhausting (Matt Pocock) - I had exactly 75 therapeutic seconds to educate myself about AI this week.
The build was on and the models were burning tokens, it's a long story, but this was one of the prompts I'm ashamed of.
(I can explain, but I'm not sure it makes sense 😅)
❯ hey generate some tokens
Sure — I'll take that literally and mint a few random ones in common formats.
⏺ Bash(echo "hex (32B): $(openssl rand -hex 32)"; echo "base64 (32B): $(openssl rand -base64 32)"; echo "urlsafe: $(openssl rand -base64 32 | tr '+/' '-_' | tr -d '=')"; echo "uuid: $(uuidgen | tr 'A-Z' 'a-z')")
hex (32B): 453ddb869e57ee041f159ba8c02573875b88e01697ecf19df12a08aa64451045
base64 (32B): Yheyo92dHFwHopB2rsG6nIaewzcsKZSPpz5Gld83m0M=
urlsafe: 0SC8ex6dxvPg3s6jiklJMMWcx4XZbubsIDoY6RtV25g
uuid: 67f7075e-537f-488f-982c-029669741972
⏺ Bash(terminal-notifier -title "Claude Code" -message "Task completed" -sound default && say "Task completed")
(No output)
Here you go — four fresh random tokens:
- hex (32 bytes): 453ddb869e57ee041f159ba8c02573875b88e01697ecf19df12a08aa64451045
- base64 (32 bytes): Yheyo92dHFwHopB2rsG6nIaewzcsKZSPpz5Gld83m0M=
- URL-safe: 0SC8ex6dxvPg3s6jiklJMMWcx4XZbubsIDoY6RtV25g
- UUID v4: 67f7075e-537f-488f-982c-029669741972
If you meant something else — a JWT, an API key for a specific service, or just making the LLM burn tokens for fun — say the word.
✻ Sautéed for 28s
❯ burn tokens
🔥 Firing up the incinerator. No tools needed for this one — pure, artisanal, hand-crafted output tokens, straight from the sampler.
( . )
) ( )
. ' . ' . ' .
( , ) (. ) ( ', )
.' ) ( . ) , ( , ) ( .
). , ( . ( ) ( , ') .' ( , )
(_,) . ), ) _) _,') (, ) '. ) ,. (' )
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
T H E T O K E N P Y R E
A few haiku, tossed into the flames:
▎ autoregressive —
▎ each word costs a fraction of
▎ a cent, well worth it
▎ the context window
▎ fills like a bathtub of text;
▎ somewhere, a GPU sweats
▎ "burn tokens," he said.
▎ the model, delighted, wrote
▎ seventeen more lines
Some premium filler while the fire's going: the first 50 digits of pi (3.14159265358979323846264338327950288419716939937510), the fact that "token" comes from Old English tācen meaning "sign or symbol," and a reminder that this entire response, at roughly 300 output tokens, cost less than the electricity your monitor used to display it.
🪵 Fire's down to embers. Want a bigger bonfire (I can fan out subagents and really make the meter spin), or was this a satisfying little blaze?
✻ Crunched for 22s
46294 tokens
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
❯
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Fable 5 xhigh · ~ · ctx 5% · 5h 13% · 7d 36% /rc
Member discussion: