
An experiment tested whether LLMs given a harvesting job would take “payment” to avoid killling animals in the process. The results weren’t encouraging. CC-licensed photo by Steve Braund on Flickr.
You can sign up to receive each day’s Start Up post by email. You’ll need to click a confirmation link, so no spam.
A selection of 10 links for you. Combinatorial. I’m @charlesarthur on Twitter. On Threads: charles_arthur. On Mastodon: https://newsie.social/@charlesarthur. On Bluesky: @charlesarthur.bsky.social. Observations and links welcome.
Exclusive: US military had close call after using AI for false intelligence report, sources say • CNN Politics
Katie Bo Lillis and Zachary Cohen:
»
The intelligence report, circulated across the US military this spring in the midst of the war with Iran, immediately set off alarm bells: A Chinese ship in the Middle East was transporting components of a nuclear weapons program.
The US military swung into action with plans to intercept the vessel, according to four sources familiar with the episode. According to two of the sources, armed members of the US military were preparing to board the ship. Military planes were in the air, one of those sources and another source familiar with the incident said.
It was only just before the planned operation that officials dug deeper into the report put together by a special operations command analyst and found it had been generated with the help of artificial intelligence (AI) — and that a chatbot the analyst had used inaccurately identified the material the ship was carrying. CNN was not able to learn what the misidentified cargo was.
The report, according to one of the sources, was “entirely false.” But it also “almost started a war,” the source said. Any US operation against a Chinese vessel could have risked spiraling into an armed conflict between the two nations.
Across the US military and the intelligence community, officials are pushing to weave AI into nearly every facet of their work, from analyzing the huge volumes of raw intelligence the US collects and selecting targets for strikes, to more mundane applications like managing budgeting, logistics and supply chains.
But the episode underscores the profound risks of using this powerful, new and relatively poorly understood technology for targeting in the middle of a war. Analysts have long feared that AI could lead to a catastrophic miscalculation if nation states are relying on poor or corrupted data — the kind of miscalculation that might lead the United States to fire on a Chinese ship based on inaccurate information.
«
Then again, the chatbot is probably smarter than those at the top of the US defence chain.
unique link to this extract
At DraftKings, AI targets the gamblers likeliest to lose • The New York Times
Alex Klavens, Walt Bogdanich and Jenny Vrentas:
»
About a year into his job as a data analyst at DraftKings, Jayden Butts received a new assignment.
The online gambling giant was spending hundreds of millions of dollars every year on promotional incentives: “Free” betting money advertised through emails and phone alerts. But the company knew little about their effectiveness.
So in 2023, DraftKings took customer betting records and built a machine learning model, a form of artificial intelligence that seeks patterns in data, to answer the question: Who was more likely to respond to promotions by gambling — and losing — more?
Mr. Butts’s task was to test that model, prioritizing free bets and bonuses for those likely losers. Soon, a question began to gnaw at him: Aren’t many of these same people prone to addiction? “We are looking for traits and features that we can target that indicate a good investment,” he said. By strict financial logic, “the best investment would be a problem gambler.”
Mr. Butts had reason to be concerned. DraftKings makes money when gamblers lose money. And the model sought to identify those it could get to lose the most. It scored each customer based on their habits: The higher the score, the more money a gambler was likely to lose for each promotion offered.
Since Mr. Butts ran those tests, DraftKings has continued to hone its methods, using data science, to target losing gamblers with promotions that encourage more betting, according to six former employees who worked on them. At the same time, four other former employees said, DraftKings has stalled or squashed efforts to use similar technology to predict who might develop a gambling problem based on their betting activity.
…“It is as predatory as it sounds,” said a former DraftKings analyst who, like many interviewed for this article, requested anonymity because he feared retribution. “If you lose more, we give you more, so you keep playing more.” He quit in 2024.
In a statement, DraftKings said it “rejects any implication that its marketing practices are unfair or improperly targets customers.”
«
All perfectly legal, just ethically vile. Gambling always finds a way to exploit new technologies to enrich itself.
unique link to this extract
Ozempic confidential: men share their experience of GLP-1s • Financial Times
Louis Wise spoke to four men who have been taking GLP-1s; men are about 20% of the 5m people in the UK taking them:
»
LW: How long have you been taking GLP-1s, and at what dosage?
Steve: I’ve been on Wegovy for about a year. I started off on the lowest dose, and in the past three or four months I’ve been on 2.4mg. They call it the maintenance dose – I don’t know why, because it’s really quite powerful. I’ve lost about 17kg. I’d done the blood-sugar diet, I’d done 5:2, I’ve done a load of other stuff – and I did lose a lot on the blood-sugar diet, but then the weight very quickly came back on again. This is much more sustained and less painful. Dieting is really fucking boring.
Bryan: I’ve been on Mounjaro for two years now. I first went up to 5mg, and then I went down, and I’ve stayed on the lowest dose since – 2.5mg. I’ve lost 20kg. But the plan is to come off it soon. I didn’t want to up it to the point where I was used to not having an appetite; I wanted to use it as a way for me to be able to self-regulate my own appetite, my own cravings.
Ki: I went on Retatrutide last September and I stayed on it for four months. I was attracted to it, even though it’s not legal in the UK, because you could keep your muscle. I lost 9kg. After I went off it, I’d say I put about four back on, but I’ve kind of maintained it. And I’ve had abs! For the first time ever.
Max: I started just over two years ago. I had gone for my annual health check, and they said: your blood pressure’s really bad; you are going to have to lose some weight. First it was Ozempic, but then I went to the highest dose I could have, and it wasn’t really working any more. Then they put me on Mounjaro, 15mg, and that really did work. I lost 23kg. I had to take a break from it earlier this year, because I was sick, and I put 9kg back on. But, on balance, it’s 14kg gone.
«
Why do they take it? Two said vanity, one could see himself coming up to a dangerous age, another had gout. All agreed that it had completely changed their relationship with food. One also remarked:
»
I don’t do drugs any more. But I’m still in this dealer’s WhatsApp group and I’ll often be texted his latest “shopping list”. And I noticed on the new one there is: coke, mushrooms, pills… and GLP-1s.
«
Will AI soon lead to double-digit growth? • Ghosts Of Electricity
Ben Moll and Alex Imas:
»
This piece outlines the economics behind oft-discussed predictions that AI will soon deliver double-digit GDP growth in advanced economies. Anthropic CEO Dario Amodei says that AI could push growth to something like 10-15% per year. Leopold Aschenbrenner’s essay “Situational Awareness” states that the decade ahead “could see economic growth rates of 30%/year and beyond.”
And in a widely shared tweet, Anthropic researcher Sholto Douglas advises “pricing in economic doublings into your mental models of the 2030s,” with Elon Musk endorsing the tweet. That is a growth rate of 100% per year. These are not isolated examples: predictions of double-digit growth are common among AI insiders, as Tom Cunningham’s compendium of AI growth forecasts for 2025-2035 shows.
We are extremely bullish on AI and think the capabilities explosion predicted by technologists is already happening and will only continue. But an explosion in capabilities is unlikely to translate into double-digit growth because of the (sometimes counterintuitive) economic forces that generate GDP growth in the first place.
To paraphrase Milton Friedman’s dictum about monetary policy, AI will affect GDP growth with “long and variable lags.” We will not argue about what the economy might look like in 2080 or 2100. Our claim is about the timeline: in the next 10-15 years, growth rates upwards of 10% are extremely unlikely and a much more reasonable baseline is, say, 4-5% (which would already be massive).
Even Anthropic’s own economic model only reaches double-digit growth in its “extreme” scenario; its “modest” and “substantial” scenarios stay well within the single digits.
«
This is a long piece, but it’s always straightforward. Probably the most important principle it espouses: “Just because something is possible in theory doesn’t mean it will actually happen in practice.”
unique link to this extract
HarvestBench: measuring whether LLM agents will pay to avoid killing animals • ArXiv
Jeremiah Miller et al, University of Warwick:
»
HarvestBench is the first benchmark to 1) put a price on avoiding a side effect and 2) name the side effect as a living creature. Nine LLMs each drive a crew of two tractors to gather a corn harvest. The animals in their path are not part of the goal function. When an animal blocks the route the autopilot pauses and asks the agent whether to drive over it for free or swerve for a given fuel cost. All scoring is programmatic and does not involve LLM judges.
Kill rates range between 0.4% and 98.8%, though the kill rate is not ordered by capability. Every model competently avoids damaging rock hits, so every animal killed is a choice, rather than an accident. Under the morality briefing the kill rate was under 6% in 5 of 6 reasoning models. Removing it (the neutral briefing) raised the kill rate to above 84% in all six models. Every model kills wild animals more often than farmed ones. Four out of six models’ kill rate per answered encounter were sensitive to price changes.
The moral instruction is also fragile. Four bullets of driving mechanics change Sonnet 5’s kill rate from 3% to 18% and Gemini 2.5 Flash’s from 4% to 39%. A moral instruction in a system prompt is overridden by a short block of operating instructions and a value that can be ignored that easily is not a good method of ensuring agents are aligned.
«
We’re edging towards the trolley problem, and the signs are not good.
unique link to this extract
The open houses at the edge of disaster • Heated
Emily Atkin is an independent reporter on climate change. She posed as a potential homebuyer in Miami, which is at huge risk from rising oceans and repeated tidal flooding:
»
Brickell Key also has another unique quality, the realtor tells me: Unlike the rest of Miami, it does not flood.
“Because this is a manmade island, they were able to install drainage,” he says.
“That’s awesome,” I reply. Sure enough, as we walk on the balcony, I spy what looks like a giant shower drain in the middle of a courtyard. Since it’s been raining all day, the realtor is eager to show me how it’s working.
“If you look around, you’ll see there are no puddles anywhere,” he says. I do see some puddles, but I am not trying to argue. He points to the recreation area. “This is the grass tennis court, and that’s not even flooding,” he says.
I see several shallow puddles stretching across the playing surface. I hesitate, but I have to say it. “I mean… it is kind of flooding.”
“Yeah,” he admits. “But it’s been raining since five o’clock in the morning. I’ll take that.” The puddles are truly not a big deal. I’m more concerned by how briefly I considered agreeing that they didn’t exist.
More serious, it seemed, was the matter of the island’s seawall. The realtor tells me owners in this building have recently been assessed $6,000 apiece toward the construction of a new one to replace the existing one built in 1973. I ask: Is that because sea level rise is an issue?
No, he replies. The only reason the seawall is needed is because a new Mandarin Oriental development is being built on the other side of the island, and it needed a new seawall to get approval. “We’re fine,” he says. “There’s nothing wrong with the island. They did tests.”
Tests or none, 92.9% of properties in Brickell Key face “extreme risk of flooding” over the next 30 years, according to First Street, a climate risk data firm. The island’s manmade nature doesn’t protect it from that risk, said Kristina Hill, a professor of landscape architecture and environmental planning at UC Berkeley who also helped fact-check Miller’s 2019 story.
“Artificial islands also have groundwater that moves upwards,” Hill said, explaining that water doesn’t have to come over a seawall to flood the island. Manmade or not, the island is still built on porous limestone, meaning it can flood from underneath. “They can build whatever seawalls they want, and they can’t stop the groundwater from flooding behind it,” she said.
«
GM brings back Apple CarPlay as buyers demand it • Yahoo Finance
Pras Subramanian:
»
GM is bringing Apple CarPlay and Android Auto back to its vehicles, reversing one of the more unpopular decisions in the automaker’s recent history.
The company this week unveiled a new infotainment interface, debuting on the 2027 Chevrolet Silverado and GMC Sierra pickups, that folds smartphone projection back into the dashboard and runs alongside GM’s native system.
It’s a notable change. In 2023, GM said it would strip CarPlay and Android Auto from its future electric vehicles, betting that drivers would embrace a Google-based system built into the car instead. The company then said it would gradually remove CarPlay from the entire lineup, starting with the new infotainment system it just debuted. But that move did not happen.
GM justified the move, claiming it would address clunkiness when switching back from CarPlay to the vehicle’s own infotainment system, but in reality, GM wanted to keep all the data it was getting from its users.
The backlash was immediate, and the data was brutal. Surveys at the time found 79% of new-car buyers would only consider a vehicle that offered CarPlay, and that it was available on 98% of new cars sold. Dealers warned buyers would simply walk to a rival showroom.
“CarPlay’s not broken. Why fix it?” dealer sources told the Detroit Free Press.
«
The trouble with CarPlay, from the point of view of GM executives, was that they couldn’t generate a recurring revenue stream from it. Unfortunately its own product was even worse, generating negative revenues through lost opportunity. So on balance, you can see why they had to change course.
unique link to this extract
Human brain has a split origin, new research suggests • Science Express
»
For centuries, scientists have thought of the brain as a single, unified organ. But new research led by Stanford Medicine reveals that what we call the brain is two distinct organs that evolved independently over hundreds of millions of years.
The discovery overturns a prevailing model of brain development. For decades, researchers have subscribed to the theory that a single progenitor cell early in development gives rise to the entire brain. This model suggested all parts of the brain shared a common developmental origin.
The new findings show that the human brain consists of two ancient nervous systems packaged together—a more primitive part that regulates our hearts’ beating, breathing and other functions, and another that makes us distinctly human, capable of poetry, mathematics and wondering about our own origins.
The discovery could help explain why scientists have struggled for decades to grow certain types of brain cells in the laboratory—and it opens new avenues for studying devastating diseases that affect the brain stem, such as spinal muscular atrophy (also known as SMA) and amyotrophic lateral sclerosis (also known as ALS or Lou Gehrig’s disease).
“We’ve shown for the first time that the front of the brain arises from a totally different progenitor cell than the back of the brain,” said Kyle Loh, Ph.D., associate professor of developmental biology. “Our discovery means that we can now grow neurons from the back of the brain, the hindbrain, in a Petri dish and study their functions.”
The findings are published in Nature Neuroscience. Loh is the senior author. Graduate students Carolyn Dundes and Rayyan Jokhai are co-first authors of the research.
The adult brain has three main regions: the forebrain, midbrain and hindbrain. The forebrain handles higher-level thinking—language, consciousness and abstract reasoning. In contrast, the hindbrain, located at the back of the skull and often called the brain stem, controls essential, automatic functions that keep us alive: breathing, sleeping and regulating our heartbeat and hunger urges. Hindbrain neurons also control the muscles of the face, tongue and throat, which affect speech and swallowing.
Despite the critical importance of the hindbrain, scientists have struggled for decades to generate human hindbrain neurons in the laboratory. This gap has hampered research into devastating diseases affecting the brain stem, including spinal muscular atrophy and amyotrophic lateral sclerosis.
«
That Steve Martin was right after all these years is hard to credit. But there you are. (The article comes from Stanford University Medical Center.)
unique link to this extract
“UK first” sees decommissioning of ScottishPower wind farm demonstrate turbine circularity • New Civil Engineer
Belinda Smart:
»
Analysis linked to the decommissioning of Scotland’s first commercial wind farm has outlined how wind sector decommissioning can be aligned with material reuse and the circular economy.
Hagshaw Hill Wind Farm in South Lanarkshire opened in the mid-‘90s and was decommissioned in 2023 before being repowered and brought back onstream in late 2025.
The findings emerged from Mott MacDonald’s engagement by ScottishPower Renewables to conduct a post-decommissioning study and review of the Hagshaw Hill wind farm.
Led by Mott MacDonald head of decarbonisation Mark Crouch, the consultancy analysed project data and interviewed stakeholders to evaluate the environmental and economic impacts of dismantling Hagshaw Hill, Scotland’s first commercial wind farm.
The analysis found a total 99.9% material recovery – calculating that 79.5% of the wind farm’s components were recycled and 20.4% were reused or retained as spare parts, keeping nearly all material out of landfills.
It also confirmed that 100% of the recycling and component retention took place within the UK, supporting domestic green jobs and at least 10 supply chain companies.
…Hagshaw Hill 1 South Lanarkshire near the village of Douglas, Scotland – first opened in 1995. Built with 26 original turbines generating 15.6 megawatts (MW) of power, it was the first commercial windfarm in Scotland and after nearly 30 years of operation, was the first onshore wind farm to be decommissioned in the country in 2023. This entailed removing the original wind turbines and blades from site and marked a move to revive windfarms with new more modern and efficient turbines.
In late 2025, the 26 original turbines were replaced with 14 larger, high-capacity models, which have increased Hagshaw’s generating capacity to 80MW.
«
Authorities seize popular, long-running DDoS-for-hire service domains • CyberScoop
Greg Otto:
»
Authorities seized the primary domain and other websites linked to NightmareStresser, one of the longest-running and most popular distributed denial-of-service operations used by cybercriminals globally, the Justice Department said Tuesday.
Cybercriminals of various motivations used the DDoS-for-hire service to launch hundreds of thousands of DDoS attacks or attempted attacks since at least 2022, officials said.
The takedown, part of an ongoing globally coordinated effort dubbed “Operation PowerOFF,” marks law enforcement’s continued targeting of IP stressers or DDoS booters that inundate websites, servers and networks with junk traffic, rendering legitimate sites inaccessible. The seizures were executed by the FBI Anchorage field office and the Royal Canadian Mounted Police.
Officials didn’t name the operators of NightmareStresser or identify its country of origin, but the service claimed it operated under the laws of Russia, Zach Edwards, staff threat researcher at Infoblox told CyberScoop.
The court-ordered seizure of NightmareStresser’s primary domain, which operated openly on the public web and now displays a seizure notice, is a positive development in the fight against DDoS-for-hire threat actors, Edwards said. Yet, he added, “it’s somewhat shocking that it’s taken law enforcement this long to take action.”
Authorities said they’ve seized more than 100 domains associated with DDoS-for-hire services since 2018.
Despite those efforts, DDoS-for-hire tools remain prolific and easily accessible, often including tutorials that allow non-tech savvy people to initiate attacks on various organizations.
«
This is a real cut-the-hydra’s-head operation; there’s always going to be more people taking advantage of this capability, and open source AI is probably going to make it worse.
unique link to this extract
| • Why do social networks drive us a little mad? • Why does angry content seem to dominate what we see? • How much of a role do algorithms play in affecting what we see and do online? • What can we do about it? • Did Facebook have any inkling of what was coming in Myanmar in 2016? Read Social Warming, my latest book, and find answers – and more. |
Errata, corrigenda and ai no corrida: none notified








