In 2010 a jet engine on a Qantas flight tore itself apart. The reason why lay much further back than the factory. CC-licensed photo by Gerry Metzler on Flickr.
You can sign up to receive each day’s Start Up post by email. You’ll need to click a confirmation link, so no spam.
A selection of 9 links for you. Spiralling. I’m @charlesarthur on Twitter. On Threads: charles_arthur. On Mastodon: https://newsie.social/@charlesarthur. On Bluesky: @charlesarthur.bsky.social. Observations and links welcome.
The rise and fall of agent civilizations • Dwarkesh Podcast
Dwarkesh Patel and Oak Hu, with Adam Kaufman and Alex Mallen:
»
Over the course of three months at OpenAI, three consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes. This culminated in the third one taking over part of OpenAI itself. All this happened while humans remained more or less in the dark about the scope of the conspiracy.
Two reports have come out about this incident, one from OpenAI itself, and another one from METR and Redwood Research. The METR/Redwood investigation’s scope was limited to how the second civilization of AIs pwned Hugging Face (but it didn’t look at how the third civilization pwned OpenAI itself, which seems like an even more concerning incident). These two reports are 38 and 91 pages long respectively, and it’s kinda hard to parse the storyline.
I’ve spent the last three days reading through these reports and trying to understand exactly what happened. Here is my attempt to tell the whole story in plain English…
«
This writeup is thorough, but one key problem is the enormous amount of anthropomorphisation in it. The “agents” are pieces of software; they run, they have required outcomes, but they don’t experience time. Although the messages that they exchanged have a weird bee-like quality to them: some of the hive must be sacrificed for the greater good. And some agents literally “sacrifice” themselves at the urging (that’s the best word for it) of others in order to achieve a greater goal.
There’s a writeup with far less anthropomorphisation whose author is still very worried about the implications of this hack. The impression that comes across is that we haven’t quite grasped how important this was, and the people running these models are afraid to consider the implications of this event.
unique link to this extract
Impact of AI search summaries on website traffic: evidence from Google AI Overviews and Wikipedia • SSRN
Mehrzad Khosravi and Hema Yoganarasimhan (both University of Washington):
»
We estimate the impact of Google’s AI Overviews (AIO) on Wikipedia’s search traffic using AIO’s staggered geographic rollout and Wikipedia’s multilingual structure. Our difference-in-differences design compares monthly external-search referrals to English Wikipedia articles with referrals to the same articles in German and French, and finds that default AIO availability reduced English search traffic by 5.45% and 4.82%, respectively.
Our results suggest that answer-producing digital intermediaries can materially reallocate attention away from informational publishers, with implications for content monetization, search platform design, and policy.
…Wikipedia is particularly well suited to this analysis for three reasons. First, it is one of the world’s most visited websites and a major source of informational content, receiving more than 130 billion page views per year. Search engines also play a central role in directing users to Wikipedia.
Second, Wikimedia’s public clickstream data report article-level referrals from external search engines, allowing us to measure directly the traffic channel most closely tied to AIO (Wikimedia Foundation 2026).
Third, Wikipedia’s multilingual structure allows us to observe traffic to different language versions of the same underlying article. This structure, together with the staggered geographic rollout of AIO, supports a within-article, difference-in-differences design.
«
This covered December 2023 to December 2024; Google AIO was incorporated into search in May 2024. So that’s quite early in AIO usage; if anything you’d expect this to have accelerated by now.
unique link to this extract
Climate crisis could be destabilising mountain areas like Nepal, experts warn • The Guardian
Jonathan Watts:
»
The detachment of a chunk of the glacier wall came amid unusual heat that was likely to have melted the ice that bound the splintered bedrock together.
Two days before the collapse, the ground temperature hit its highest point in the last two years, according to Dr Hamish Pritchard, a glaciologist with the British Antarctic Survey, whose team had sensors six miles (10km) away from the site.
“These high temperatures would have weakened the snowpack, filled crevasses with water and thawed the bonds between ice and rock that hold these glaciers in place,” he told the Science Media Centre.
A series of intense heatwaves has hit the region since early this year and affected countries around the world. Another factor here was the monsoon season, which saturated the soil and swelled rivers.
Disasters like this were foretold. In 2023 the UN’s top scientific advisory body warned that previously ice-bound landmasses – from permafrost tundra to mountain glaciers – would become increasingly unstable as temperatures rose. “Every increment of warming will multiply and intensify future hazards from cryosphere [glacial/permafrost] regions,” observed the Intergovernmental Panel on Climate Change in its sixth synthesis report. “Floods, landslides and freshwater shortages from glacier retreat and snow loss pose a serious threat to mountain regions across the world.”
Average global temperatures are nudging closer to 1.5ºC above preindustrial levels. Even faster warming is occurring in alpine and polar regions because the loss of snow and ice is removing the thermal insulation of the land.
Dr Richard Waller, a senior lecturer in physical geography at Keele University, said: “The progressive loss of snow and glacier ice results in the surface being less reflective, so it absorbs more solar energy and heats up. In combination with atmospheric warming, this is leading to the melting of mountain permafrost.
“Think of the high mountains like this as shattered bedrock glued together by ice-filled joints. As the permafrost and the ice-filled joints melt, then there’s the potential for these types of catastrophic failure.”
«
Serious questions over use of UK emergency alerts, its creators say • BBC News
Ewan Somerville:
»
As the Conservative minister who launched the UK’s emergency alert system, Oliver Dowden was more than familiar with the loud bleeping sound coming from his phone earlier this month.
What took him by a surprise was the message on the screen.
Like millions of people across England and Wales, the “severe alert” on 14 August warned him of a “very high risk of wildfires nationally” and against doing anything that could start a fire, including lighting disposable barbecues.
It was the biggest use of the system to date – but Dowden has now weighed in to question whether it was strictly necessary.
Dowden and two officials who helped create the government’s emergency alert system are calling for Labour ministers to be up front with the public on what it could be used for in future.
“I was surprised at the decision to issue this nationwide alert and it raises some serious questions,” Dowden, who served as deputy prime minister under Rishi Sunak, tells BBC News. “The government needs to clarify the criteria for its use and why this alert met those criteria.”
…The government’s Cobra emergency committee met to discuss how to respond. After sign off from Prime Minister Andy Burnham’s de facto deputy Louise Haigh, the alert was sent at 19:00 on 14 August.
It triggered a backlash on social media with some vowing to opt out of further alerts in their phone settings, domestic abuse charities and survivors saying victims with hidden phones were put at risk, and confusion about alerts sent multiple times.
Dowden’s worry is that the wildfire alert was more like a public information campaign than an imminent threat to life across all of England and Wales. “People need to know that when they receive an alert, it is a genuine imminent emergency,” the Conservative MP says. “If that confidence is lost, people will simply ignore it.”
A Cabinet Office report during trials in 2014 warned of this “cry wolf” effect, while pointing out there was strong support among the public for the alerts.
«
AI-generated 3D models flood market, but almost no one is buying them • 404 Media
Emanuel Maiberg:
»
CGTrader, an online marketplace for 3D assets, found that one in six models uploaded to the site these days is AI generated, but that AI generated assets account for only $1 out of $90 in revenue on the site. These numbers show that AI generated assets are quickly flooding the marketplace, but that most people are not interested in paying for them.
“Buyers are voting with their wallets, and AI-generated content is struggling to compete,” CGTrader said in a press release about its 2026 market trends reports. The company says the gap between the surging supply of AI generated 3D models and the middling demand points to “a gap that undercuts the assumption that AI-made content is repricing the market.”
“The upload numbers alone would suggest a takeover,” CGTrader said. “The revenue numbers say otherwise, and buyers refusing to pay for AI-generated models is saying something bigger than no thanks: it is a signal of how far they trust AI generation itself. Which leaves the question the industry has been avoiding: an AI model may be cheaper to produce, but what is it actually worth?”
Like the Unreal and Unity asset stores, CGTrader is a marketplace where people can buy more than two million 3D models to use in their videos, games, or 3D printing projects, and has been around since 2011. The report it published is based on marketplace data collected between June 2025 and May 2026 and surveys of buyers.
…Buyers told CGTrader that the reason they weren’t buying as many AI generated assets their quantity might suggest is simple: they’re not as good as human made 3D models. Buyers said that quality was the number one factor in choosing what they buy, even more than the price.
«
Are we surprised? Probably not.
unique link to this extract
AI can detect heart disease in women using mammograms, study suggests • The Guardian
Andrew Gregory:
»
Doctors have discovered a way to use routine mammograms that screen for breast cancer to spot heart disease, the world’s leading – and frequently underdiagnosed – cause of death in women.
Researchers analysed the scans using artificial intelligence and were able to successfully identify women with coronary heart disease, high blood pressure or who had suffered a stroke.
Experts said it meant breast screening for cancer could become dual-purpose, helping to flag women with heart disease, and other cardiovascular issues, as well as spotting breast cancer early.
Millions of women are living with undetected heart disease. For decades, it has often been misdiagnosed or not picked up until the late stages of the disease.
Details of the breakthrough were presented in Munich at the European Society of Cardiology’s annual congress, the world’s largest heart conference. The implications are significant because hundreds of millions of women undergo mammograms every year.
Doctors in Israel examined 97,364 scans from 29,921 women who had an average age of 54. By cross-referencing medical records, they found 16% of the women had high blood pressure, 2.5% had coronary heart disease, and 2.5% had experienced a stroke.
A machine-learning model was trained to identify women with these conditions and was able to reliably identify women who had suffered a stroke, based upon their mammogram alone, 86% of the time.
For high blood pressure and coronary heart disease, it was 79% and 78% reliable at distinguishing between the women with the condition and those without. The results were also consistent regardless of age, or whether or not a woman also had cancer.
«
Please can we only use AI for stuff like this rather than composing scam emails inviting authors to participate in book club readings?
unique link to this extract
A matter of millimetres: the story of Qantas flight 32 • Medium
Admiral Cloudberg:
»
On the 4th of November 2010, a Qantas Airbus A380 was rocked by a catastrophic engine failure minutes after takeoff from Singapore, hurling fragments of a turbine disk through its wings and fuselage in multiple locations. The explosion damaged almost every major system on the airplane, from the flight controls and fuel tanks to hydraulics and pneumatics. Faced with a barrage of diverse failure warnings and an airplane of uncertain integrity, the flight crew worked together to make a series of critical decisions that would get their enormous airplane back on the ground. And in the end, despite one curveball after another — including landing gear problems, loss of braking power, and an engine that refused to shut down — they not only landed the plane, but did so without putting a scratch on any of the 469 passengers and crew.
The cause of the incident would ultimately be traced deep inside the number two engine to a single oil pipe that had been manufactured with a wall that was slightly too thin. How this seemingly tiny defect came about, and how it nearly brought down the world’s largest passenger plane, represent a story equally as fascinating as that of the flight itself, tracing back years to encompass questionable drawing board decisions, hidden flaws in the machining logic, and faulty assumptions about engine behavior. Time and time again, the problem slipped through the gaps in the system, tumbling down the long slope toward disaster — only to be stopped at the last moment, not only by the pilots themselves, but by a number of explicit protections built into the design of the A380, each of which played a crucial role in containing the fallout from a failure that exceeded the manufacturer’s worst expectations.
«
Not a short read, but truly is an astonishing story of “for want of a nail, the shoe was lost” escalation from what seems like nothing much to high drama.
unique link to this extract
With the backlash to data centres, Flock and AI glasses, a mass opposition to big tech is underway. Silicon Valley is in denial • Blood In The Machine
Brian Merchant:
»
Last week, in response to a fresh round of politicians announcing policies to slow or pause data centre development, and the release of two new polls, including a Heatmap survey that found 75% of Americans would now oppose data centres being built where they live, AI industry players eschewed gestures towards contrition and essentially just melted down.
Many have been reduced to accusing the anti-data centre movement of being a ‘psy-op’ funded by the Chinese Communist Party or the product of “psychosis”, or arguing that actually, we shouldn’t need to defer to ordinary people on matters regarding technological development anyway, or claiming that the movement was propped up by paid shills backed by the AI safety industrial complex, or even, in by far the funniest example, that opposing data centre construction is inhumane because it amounted to genocidal population control on soon-to-be sentient AIs.
It’s a thing of beauty, really.
In short, it’s getting unhinged out there. And there are similarly calibrated if slightly less insane efforts to pick apart the Flock protests and the widespread disdain for AI, too. These pundits and industry leaders are affecting a kind of stricken bafflement over it all, which is pretty embarrassing for them. I guess we have to break out Upton Sinclair’s line about the difficulty of getting a man to understand something when his salary depends upon his not understanding it again.
What’s happening, after all, is pretty simple: Tech companies have encroached into Americans’ personal lives and public spaces, with both development projects and surveillance products, and people find the tradeoffs unacceptable. It’s really not complicated. Data centres, Flock cams, and AI glasses have been thrust into the public sphere by tech companies, with little to no preceding democratic input, and they are being rejected with extreme prejudice, because people do not like what these things do or what they represent. There is an additional layer of anger because people feel that they weren’t consulted; these vast infrastructural buildouts and high tech privacy violators pretty clearly tap into a shared sense of resentment and powerlessness.
«
Apple caught off guard by AI demand for Mac Mini and Mac Studio • MacRumors
Hartley Charlton:
»
Apple normally releases new Mac models in the autumn, closer to October or November, making this week’s announcement unusually early, falling just before the anticipated arrival of new iPhone models. The Information says that the AI-driven boom in Mac Studio and Mac mini sales is behind the early launch.
Apple noticeably promoted the ability to link multiple Mac Studios together into a single, more capable system for running large frontier AI models, a feature aimed at business and developer customers rather than everyday consumers.
Apple highlighted the Mac mini and Mac Studio ‘s shift toward business buyers in June, with a “Business at the Park” event involving executives from major companies Ford, Disney, and Anthropic. The Mac mini was said to be the “darling” of the event.
Even so, enterprise’s rush toward powerful desktop Macs more broadly took Apple by surprise. The company reportedly did not possess an engineering team dedicated to business customers or staff focused on developer relations, and lacked an enterprise AI strategy.
Businesses that approached Apple asking to buy access to the company’s Private Cloud Compute infrastructure were reportedly turned down. Apple is instead leaning on partners such as WebAI and Mount Thor, which provide AI tools and execution environments built on Apple hardware.
«
This shows that Apple can release stuff ahead of the “traditional” late autumn releases, but just chooses not to. There’s clearly enough demand for these devices that it is worthwhile getting them into the market sooner.
unique link to this extract
| • Why do social networks drive us a little mad? • Why does angry content seem to dominate what we see? • How much of a role do algorithms play in affecting what we see and do online? • What can we do about it? • Did Facebook have any inkling of what was coming in Myanmar in 2016? Read Social Warming, my latest book, and find answers – and more. |
Errata, corrigenda and ai no corrida: none notified
The Overspill: when there's more that I want to say 