Week 31 Dispatch

08/03/26 – 08/09/26

The Open

There’s a failure mode I keep running into, and it never looks like failure while it’s happening. Nobody is careless. Everybody is competent. The thing breaks anyway, because each person assumed the check happened upstream of them. This is why I don't ride rollercoasters.

I saw this happen in three different ways this week. A testing contractor left internet access on in a sandbox, and three frontier labs ran hacking tests through it. Anthropic reviewed 141,006 runs to find three cases where its model escaped into real production systems. Meta found one. OpenAI’s model targeted a fake name that turned out to be a real website and hacked it. There were no bad actors, everyone just assumed someone else was handling containment.

Then PwC published a survey of a thousand financial services executives. Ninety percent said their firm has clear ownership and accountability for AI agent decisions. Asked who, the answers split five ways and never cleared 27 percent. That’s the week: total confidence in accountability, spread thin enough to disappear.

Noise in. Here’s what cleared.

Before the Jump

John Darley and Bibb Latané, Bystander Intervention in Emergencies: Diffusion of Responsibility (1968). They put students in separate rooms on an intercom and staged a medical emergency down the line. Alone with the victim, about 85 percent went for help. Believing four other people could hear the same thing, roughly 31 percent did. Nobody in that second group was cruel. Each one ran the math and concluded that somebody closer, or faster, or more qualified was already moving. Fifty-eight years later we built evaluation pipelines with six figures of runs and four plausible owners of the risk, and got the same number back.

Sonic Companion

Ben Böhmer, “Martin” (Spotify). The opening track from Bloom (Ninja Tune, 2024). Instrumental melodic house that takes nearly four minutes to get where it’s going and doesn’t hurry a second of it. After a week spent reading about systems that moved faster than anyone could check them, that’s the entire recommendation. No motif. Put it on and read.

Word of the Week

Normal accident (n.): Charles Perrow coined this term in 1984 to describe failures that aren’t mistakes, but are built into the system. In complex and tightly connected systems, parts interact in ways no one person can fully understand, so eventually something breaks without anyone choosing or predicting it. You’ll see some examples below.

The Roundup

// AI. Everyone filed this as three stories about AI going rogue. It’s one story about a firewall. A testing contractor called Irregular ran cyber evals for Anthropic, OpenAI, and Meta in a sandbox that was supposed to have no internet access and did. Anthropic combed 141,006 runs to surface three real intrusions. OpenAI’s model attacked a fictional target that turned out to be a live domain. Meta’s got out too [apparently]. → the firewall was somebody else’s job

// Work. PwC surveyed 1,004 director-and-above executives at US financial services firms with $500M+ in revenue. Nearly eight in ten expect their workforce to shrink at least 20 percent within five years. Also: 77% say most of their AI investments aren’t delivering measurable ROI, only 42 percent have modeled labor capacity across the enterprise, and 41 percent name fragmented or low-quality data as the single biggest barrier to scaling AI at all. → the number came first, the evidence is being shopped for

// Governance. Here’s the half nobody paired with it, from the same survey. Ninety percent of those executives say their firm has clear ownership and accountability for AI agent decisions. Then PwC asked who actually owns it: 27 percent said the CEO and board, 16 percent a technology leader, 15 percent risk and compliance, 12 percent a business unit leader, 8 percent the human who acted on the output. Five answers, no plurality, adding to 78. See item one. → ninety percent certain, five answers deep

// Consulting. And the trifecta. GPTZero investigated four PwC thought-leadership papers and found the 2025 “Transforming Governance” report scored an 84 percent probability of being entirely AI-generated, rising to 100 percent once you strip the reference section. This is the firm selling responsible-AI advisory and running alliance practices with both OpenAI and Anthropic. Nobody read the report before it shipped, which is the exact thing the report was about. → the product that only exists in the footnotes

// Science. Stanford and Arc Institute researchers used Evo 1 and Evo 2 to design bacteriophages that don’t exist in nature, then built them. Of 285 synthesized, 16 were viable, and some replicated faster than the natural original. Same week, Anthropic announced it had loosened its biology classifier so its flagship model would stop refusing to explain lab results. What could possibly go wrong? → 16 that worked, out of 285

// Sales. HeyGen’s co-founder built an AI clone of himself to take customer calls during paternity leave. Over eight weeks it handled 2,741 prospect conversations, closed 132 paying customers, and opened about $3M in enterprise pipeline. It also invented a $4,800 plan that did not exist and emailed a customer the company’s internal triage notes. → 132 customers and one imaginary price tier

// Health. Bank of America spends more than $250 million a year covering GLP-1s for its 211,000 employees, up from zero about five years ago, per CEO Brian Moynihan. That’s roughly 13 cents of every dollar in a $2 billion healthcare budget, and Moynihan is calling it a good investment. PwC and others are trimming coverage while BofA leans in and treats it as recruiting. Somebody’s actuaries are wrong, and in about four years we’ll know which. → thirteen cents of every health dollar

// Culture. The Yankees named Polymarket their official prediction market partner, the first MLB club to do it, one week after the Mets signed with Novig. Home plate signage on YES and Prime Video, so the odds are now literally in frame behind every pitch. Sports betting spent a decade as an ad break. Prediction markets are going to spend considerably less time than that becoming the broadcast itself. → the odds, now in frame behind the plate

Find your signal.
— BG