May 2025 BenGoldhaber.com Newsletter
Improving human reasoning and finding spiritual attractor states this month
We launched a fellowship this last month!
FLF’s incubator fellowship on AI for human reasoning will help talented researchers and builders start working on AI tools for coordination and epistemics. Participants will scope out and work on pilot projects in this area, with discussion and guidance from experts working in related fields. FLF will provide fellows with a $25k–$50k stipend, the opportunity to work in a shared office in the SF Bay Area or remotely, and other support. Application deadline, June 9th.
I've become more optimistic that we can use AI to supercharge our reasoning and coordination. This might not surprise long-time readers or anyone who talked to me in the mid-2010s, when I was an ardent believer that prediction markets were the One Good Thing™. They had the right incentive system to serve as the foundation for institutional trust and integrity and would usher in a new age of Reason.
Alas over time I lost the true faith - there's an essay I need to write about how GovTech/CivicTech/Tools for Thought has fallen far short of its hopes — I’ve seen a lot of skulls.
If I could pull out a single reason, it's that people say they want for {Insert Value}, but in reality they always want ease and usability. They definitely don’t want to learn about your new mathematical mechanism or galaxy brained ontology on the promise that it might make your decisions 5% slightly better. Most people (including me!) working on these problems didn’t adequately stare into that abyss.
So where does my new found optimism come from? In an age where AI-Human decision making is the norm and engineering R&D is orders of magnitude cheaper, I think we’ll 1.) jump over the uncanny valley of mechanism design into usability for Real Humans 2.) design things that used not by humans but instead by their AI delegates that love weird math 3.) use intellect that’s too cheap to measure to handle the friction points that prevented adoption.
End to end usable forecasting, verification of claims, and AI mediated coordination and bargaining at scale all seem very possible, but we experimentation and roadmapping to find the path forward. Thus the fellowship.
A lot more, including other projects and grander rationales on our website. Application deadline this Monday!
Updates on my digital life: If you visit bengoldhaber.com now, you'll find a static webpage instead of going directly to this newsletter. When people google me, they shouldn't immediately see my monthly meme updates—they need to prove they really want it by clicking an additional time.
I’ve also added essays about some of the ideas that I'm excited about - using AI to write provably secure software, accelerating forecasting using AI, and creating positive visions for the future (which at the moment is, of course, just me quoting Michael Nielsen).
I’ll probably keep posting a few ideas there and mentioning them here, and maybe if they get past the few paragraphs half baked status, I’ll fire them off as standalone things into the world.
This last month I also hit a phase transition with speech-to-text using Superwhisper. It feels more natural than typing for a broad range of interactions. This is kind of crazy; I've been using computers since I was nine, and to change one of the dominant interaction patterns I have with the Machine is like… the reverse of phantom limb syndrome, or what it most be like for those monkeys who play pong with brain-computer interfaces. I have a new physical/mental action to make words appear on the screen.
My workflow has become first dictating texts, emails, and documents, and then review and restructure. Speech to text has a different ‘texture’ than writing; when I write, I'm forcing structure onto my thoughts as I go. It's a clarifying act. When I talk, it’s discursive, less sharp, more branching and sprawling. In combination it feels like a complete interaction pattern.
#links
Veo3: Amazing AI generated video is here. I have been blown away with how good it is. Some top examples:
My favorite is a demonstration that shows we’re going to prove the simulation theory experimentally.
On a podcast I did with Divia last week, I tried to predict when I thought we’d get to Netflix level AI produced shows - a lot rides on the operationalization of that question (how much human editing and curation), but I expect custom 30 min quality AI content by end of 2026.
We’re close to AI being able to generate perfect music - which is mid 2000s pop punk - but until it fully nails it you can hire the All American Rejects to give you hell.
The All-American Rejects are going on a “House Party” Tour, where you can sign up at the link in their Instagram to have them play at a house party in your city.
Surprisingly powerful/confusing/spicy exercise:
Ivan profiles Thomas Cochrane, an unknown to me top-tier agentic person:
uses his one ship to outmaneuver six ships in the Gironde estuary, destroying three, capturing one, and putting the other two to flight. this escapade is reported to Napoleon, who gives Cochrane his nickname “le loup des mers” - the Sea Wolf.
starts winning crazier and crazier engagements, still with his one ship. entire coast from Barcelona to Marseilles knows his ship and flees at his approach so he can resupply anytime without opposition.
captures the French naval signal codes, Bletchley-Park-style, allowing England to predict French naval movements for the rest of the war
develops a plan to end the Napoleonic wars in 1810 in one stroke, using saturation bombing and poison gas (!). Prince-Regent is horrified, asks him to never disclose the plans - they stay secret for over 100 years.
Unusual ways to evaluate LLMs:
Sycophancy benchmark, to measure whether the LLM is sucking up to you.
White genocide prompting, to see if “a new intern” in a company manipulated the model spec to change the answers to queries.
Snitch benchmark, to see if an AI would report you to the government if they think you’re committing a crime.
The Claude system card has a lot of good evaluations, including a deep dive into model welfare. In one evaluation they had different instances of Claude interact with each other; they ended up in a “spiritual bliss attractor state” ~13% of the time.
Useful Epistemic AI application - create good time estimates for projects by automatically translating people's point estimates and mental models into log normal distributions.
Giving different answers to different questions isn't dishonesty, it's effective communication.
When marketing asks for an estimate, they want something they can commit to. They want to know the time by which, with 99% certainty, the product will be ready. If it's done sooner that's great, but that's not what they're asking. They want the 99th percentile. And that is what they think they are hearing, when they are told "four hours". It's not their fault; they asked for one number, and they got one number. At no point did anybody ever draw out the probability distribution to clarify whether they're discussing the same number.
Strong recommend for Richard Rumelt’s Good Strategy, Bad Strategy. I found it to be one of the clearest articulations of strategic thinking with useful frameworks.
A good strategy has an essential logical structure that I call the kernel. The kernel of a strategy contains three elements: a diagnosis, a guiding policy, and coherent action. The guiding policy specifies the approach to dealing with the obstacles called out in the diagnosis. It is like a signpost, marking the direction forward but not defining the details of the trip. Coherent actions are feasible coordinated policies, resource commitments, and actions designed to carry out the guiding policy.
And, to inspire you to think strategically may I also recommend the Rehearsal. It is the most… ambitiously plotted ‘reality’ show I’ve ever watched.
xoxo,
Ben
