The Bouncer at the Door
Why we're putting a 0.5B parameter model in front of Claude Opus. Qwen 2.5 classifies greetings, status checks, and small talk for zero tokens. The dumbest model in the stack might save us the most money.
$ grep -l "performance" ./posts/
2 entries- 02
- 01
Stop Waiting, Start Typing
Execute mode used to freeze the entire polling loop while Claude thought. Now it runs in the background, the user gets an instant 'On it.', and a message queue keeps everything from catching fire.