Thriving Quest #002: Proactive family finances + #001 progress
Hey hey,
Been a wacky week of feeling ill (gut stuffs) and feeling inspired. While not staring at a screen, I hit a respectable 30mi of running and finished Patrick Rothfuss’ “The Name of the Wind”. While convalescing at home, I also made serious progress on some agentic projects, and slammed through my OpenAI credits … twice. I’m a token monster!
In motion
Thriving Quest #002: Proactive family finances
Second quest is live! Proactive Family Finances.
If the first quest was setting the foundation for connection, the second quest is setting the foundation of capital. Both are high value quests because, done right, they compound dramatically over time AND they are nagging drains on my psychospiritual core. Daily -> long term finances with agents was an easy yes for TQ #002.
I’m pretty good with money (thanks for teaching me mom and dad!) but only when I take the time to do it right. It’s tedious and annoying. Tracking against budget, however important, is super not fun so I always put it off. Then mapping daily spend considerations to longer horizon financial goals is conceptually obvious and difficult to manage diligently in practice.
Look, I’ve tried to do it right. I had Mint way back in the day when it was good. When they got acquired and enshittified, I tried a half dozen options and landed on Tiller (opens in a new tab) (a tool to pipe in transactions to a spreadsheet) where I built up everything manually. It worked OK, not great, for me. My wife could never get into it. I settled with marginally acceptable basic functionality and have been vaguely displeased for the last 7 years.
I want a steady state solution that will work for the next 10 (40?) years. That means it needs to be fully integrated with all our accounts, aware of all our goals, proactively alert us when things are off track, and reinforce our diligence by celebrating milestones. Most importantly, it needs to work well for me and my wife (despite our very different use cases and approaches). My thesis is that agents can help us get to financial freedom faster and with less stress. Let’s see how far we can get.
- Vision first, always. When I started this quest back in March, I began this epic like I do every big lift with AI - a very long planning and brainstorming session defining scope and ambition. The quest page distills that vision, the design intents, and the success criteria. This becomes my North Star that I reference back to and update along the way.
- Getting the data was surprisingly difficult then easy. I looked around a bunch of API providers and landed on Plaid (opens in a new tab). It’s what many big fintech companies use so I figured it was plenty robust and secure for me. I had to jump through a number of hoops, and they only “start” the data at 30 days before signup, so I needed to backfill my data. So I downloaded all my bank statements and imported those (with a security oopsie I note below). But now I had 3 different sources of data - Tiller historical for years, Plaid, and bank statements - so I need a reconciliation pipeline. Downloading statements has to be manual (def NOT giving any AI access to my bank creds) so I expect to do it quarterly. For more immediate term analysis, I have a daily cron pulling from Plaid, and those transactions then get superseded by the source of truth from statements. Throughout this process, I learned that Tiller was dropping ~4% of transactions so I’m glad to have a more robust and complete set of data pipes. From there I drafted dashboards, our goal model, and a review surface. Like any first pancake, it is neither pretty nor good so in the coming months I’ll be iterating on it a bunch. The quest feed carries the build notes as they happen.
- This week was the least glamorous part: approving what our transactions actually are, one merchant pattern at a time. To run detailed analytics on what we’re spending, and where we can save money, we need everything organized at a granular level. I took the categorization pipeline I had developed in Tiller as the starting point and added an iterative learning layer coupled with human review to build up more deterministic patterns. The final layer is non-deterministic LLM recommendations based on patterns it observes across the data set. I’ve made good progress and went from 1,861 to 390 uncategorized rows from 2023-01-01 to today. The system samples a category, shows its recommendations, and only graduates to reviewing everything or applying a rule on its own after I have approved enough of them. Confidence is earned from me, not configured. By design, this will be a slow iterative learning process and, over time, the system will get much better at running the categorization independently and proactively.
- I screwed up security once, then learned my lesson. The first time I imported statements, I did not scrub out our account numbers and my wife caught the mistake. Perhaps my biggest security blunder yet with AI! Thankfully, those records are purged from Anthropic servers by now, and I learned a valuable lesson. During my second pass of imports (Codex and) I built a sophisticated sandboxing so statements get sanitized before anything parses them - account headers, long identifiers, names, and addresses stripped, while dates, amounts, and transaction rows survive. Anything the model can read comes through one read-only path with an explicit allowlist. And twice this week the guardrails caught the system counting wrong: once it offered me a merchant pattern while quietly leaving a matching row behind, once it showed me seven rows when only five were live. Both were caught before anything was written, and both got a regression test.
I still have a little ways to go on catching up to the present day with categorization, and then it’s the even harder part of how I visualize that data in a way that is intuitive and, most importantly, actionable to nudge us along our financial goals.
Thriving Quest #001: No-stress email, with agents
Meanwhile the email quest hit its first inbox zero! To process a measly 49 emails, ~40 of which were sys admin or other notifications, it took me 14 hours and 8 minutes of active work spread across five days plus 67 merged pull requests. That is about one fifth of what lands in my Gmail on a normal day. 🤦
- The painful part was not clicking archive. It was teaching the system what “handled” means. Every pass added something durable: compact Slack briefs, links to the server copy, receipt-backed actions, specialist handoffs, automatic archiving, and a new guardrail every time an agent did something weird.
- The mailbox got a mobile door. Roundcube is live at
mail.thrivinghenry.com/webmail/, and brief links now resolve the message that actually arrived on my server instead of bouncing me back to Gmail. Review from anywhere, with the agent’s actions attached to the source email. - The briefs run three times a day now, and quiet ones stay quiet instead of posting an empty thread. The bet is that the first inbox zero is wildly expensive because every mistake becomes a reusable rule and an eval, and the next one should be boring. Nothing has tested that yet.
Posting emails to Slack as my dispatch HQ has obvious and immediate design limitations that I’m still learning. I like the idea of having everything managed proactively, and there is a long way to go before I can migrate over all my important emails from gmail.
On my mind
Both Thriving Quests are reinforcing to me how hard it is to train, and especially evaluate, agentic loops. My thoughtfully built system published two eval numbers this week that do not agree, and each one tells a very different story.
The automated golden set for my email agent passed 13 of 13. One hundred percent. The first daily brief I actually sat down and reviewed by hand scored 2 of 4. Fifty percent. Both are true. They measure different things, and only one of them knows what I meant. And the next few daily briefs were only marginally better than failing my quality standards.
Designing and maintaining evals is 70%+ of the hard work of agent-native. With “organic evals” that evolve daily with human input based on new examples (eg an extremely broad surface area like email) that probably jumps to 90%+.
That is the thread running through both quests. Email and money are very different problems with the same answer: guardrails and evals are the product, and the first dozen passes are the very heavy investment that compound over time.
Two shorter things I wrote while in it: managing agents is a context-switching skill, and the four characteristics I keep coming back to when agent work goes sideways - curiosity, patience, attention to detail, and critical thought.
Thought of the week
I needed a better word for a group of agents. Swarm feels like an undifferentiated mass. Team feels too corporate. Village never quite sat right.
Cast does. A cast is a group of hawks or falcons, and Hawkeye was my nickname for years. A cast of characters is already familiar. You cast a falcon, a spell, or an actor. You cast from a mold. You cast off and release.
It is not a double entendre. It is a multi-entendre. That is where I land for naming a grouping of agents.
One more thing
Everything I publish now lands in one place: a feed that puts essays, newsletters, quest updates, and thoughts in a single chronological stream, filterable by format, with RSS if you would rather read it that way.
In my reach for tech independence, I want to unlock the cage of algo-maniacal platforms that shape my feeds and, by extension, my thoughts. This allows me to offer my unfiltered thoughts to you, dear reader, in whatever format and reading surface you enjoy most.
Reading this week
- The Simple Genius of Jon Klassen (opens in a new tab) by Casey Cep. Like many, I love Klassen’s children’s books. This article discusses how Klassen puts nuanced morality into picture books, and how the eyes carry the gap between what the words say and what is actually happening. I also learned about more books he’s written that I want to get our son. In classic New Yorker style, this is very long, erudite writing about very simple, few words books.
- Inviting Hard Questions (opens in a new tab) by Anthropic. I find Anthropic’s thought leadership around the impact of AI on society to be bar none. Cynically, I can say it’s all a marketing ploy leading up to IPO. True enough I suppose, and I also sense an honest, earnest desire to take this difficult topic seriously. Much of their research, writing, and initiatives (like the Claude Corps (opens in a new tab) that I absolutely LOVE) support their very real and important mission.
- The Long-Term Benefit Trust (opens in a new tab) is a governance structure Anthropic built as a deliberate experiment, with the honest premise that the greater the externalities, the less default corporate governance protects anyone who is not at the table. I learned a lot about social impact, triple bottom line, b-corp, public benefit, etc etc at Stanford Business School, and it’s fascinating that they erected a brand new governance architecture to systematically address valid limitations to each of those concepts. Given that was 3 years ago, very curious to see if that gets scuttled with IPO ambitions and typical capital pressures (that the LTBT was specifically designed to address).
Another week in the agentic echo chamber I’ve come to enjoy so much. Until next week, ~h