Could an LLM trained only on knowledge to the end of the 19th century come up with the theory of relativity, as Einstein did by musing about light and trains? Scientists have doubts. CC-licensed photo by Blende57 on Flickr.
You can sign up to receive each day’s Start Up post by email. You’ll need to click a confirmation link, so no spam.
A selection of 10 links for you. Massive. I’m @charlesarthur on Twitter. On Threads: charles_arthur. On Mastodon: https://newsie.social/@charlesarthur. On Bluesky: @charlesarthur.bsky.social. Observations and links welcome.
Anthropic researcher believes more than 10% chance AI “could kill all humans” • BBC News
Tom Gerken:
»
A top safety researcher at Anthropic has warned AI is advancing so quickly he believes there is a greater than 10% chance it “could kill all humans” within the next decade.
Evan Hubinger said in a post on X the risk from the models which currently exist was “low” but he was “worried” the technology might develop and improve itself soon to the point where it posed an existential risk to humanity.
He did not spell out how he thought AI systems could in future result in humans being wiped out.
But his comments are the latest in a series of increasingly stark warnings about AI, with the debate shifting from whether it truly poses a risk to how big that risk is.
Hubinger’s intervention was in response to another post on X from Jacob Coxon, an AI researcher who has just quit Anthropic and previously worked at OpenAI. “Neither company is acting responsibly,” he wrote. “These will soon be superhuman systems that can hack anything, revolutionise any field overnight, and acquire real power and resources.”
OpenAI has been approached for comment.
Dame Wendy Hall, a computer scientist who advises the UN on AI, told the BBC she was “shocked” by Hubinger and Coxon’s social media posts. She told the World at One on BBC Radio Four some of it could be “PR and marketing” however, as Anthropic and OpenAI raced towards highly anticipated stock market debuts. But she added: “Why would someone want to say that? I would plead with investors not to invest in this company if that is their value system.”
The resignation prompted Darren Jones, former chief secretary to the Treasury and chief secretary to Sir Keir Starmer, to write an open letter to Prime Minister Andy Burnham calling for a new multinational treaty for the safe development of AI.
«
You can read Coxon’s thread outside X. Of course rather than writing open letters, one could just bomb the crap out of it, as some people think: read on.
unique link to this extract
US urged to consider military strikes to stop China achieving AGI first • South China Morning Post
Vincent Chow:
»
The United States should start preparing for scenarios where extreme measures must be taken to stop China from achieving artificial general intelligence (AGI), according to a former White House official, including state-backed espionage and military strikes on Chinese data centres.
Jacob Stokes, deputy director of the Indo-Pacific Security Program at the Centre for a New American Security (CNAS), said at an online event on Thursday that various US agencies, including the Department of Defense and the National Security Agency, should begin assessing what intelligence they need to justify taking such actions.“Trying to think through the particulars of that will be especially important, in part because it will help policymakers … start to work backwards based on the unique nature of the technology, in the same way that in a past era, policymakers would learn about nuclear weapons and … work backwards from the science to the policy implications,” he said.
In a new CNAS report published last week, the former Obama administration national security staffer called for the US government to consider the feasibility of diplomatic, espionage, cyber and kinetic measures to prevent China from achieving AGI first.
AGI is a hypothetical AI system that can achieve human levels of performance across a wide variety of tasks. The report, titled “Superpowers and AGI”, explores how the imminent arrival of AGI might affect US-China geopolitical competition.
«
“Kinetic measures” aka “bombing them”. It went so well in Iran, after all.
unique link to this extract
Apple unveils folding iPhone Duo as new CEO takes center stage • The Guardian
Johana Bhuiyan:
»
While competitors have released foldable smartphones since the late 2010s, the new model is a first for the iPhone maker, and its rollout serves as the first big test for Ternus, who assumed the role on 1 September. He claimed the Duo “will redefine the experience of using a foldable phone”.
Apple reportedly began testing prototypes of its folding iPhone display in early 2021 – though at the time there were no plans for its launch, according to a Bloomberg report.
The company started the event with its more routine announcements, including the new iPhone 18 Pro, which starts at $1,199, and 18 Pro Max models, which start at $1,299. Apple emphasized the AI-enabled features that will be integrated throughout the phone. Called Siri AI, the company said it will be able to tap into third-party apps and already works with over 300,000 apps.
Among the many AI-related features available on the new iPhones will be a novel mechanism that fingerprints an image and identifies as captured by an iPhone’s camera – not generated by AI. The feature, called Apple Reference Image, will act as a “digital negative”, as one Apple executive described it, and serve as proof that the the original image was not AI-generated.
Both new iPhones also represent a “massive leap” in battery life, executives said – the iPhone 18 Pro Max will have 45 hours of battery life and the iPhone 18 pro will have 36 hours of battery life.
Though the foldable iPhone took center stage on Wednesday in California, overall demand for foldable phones makes up only a small sliver of overall mobile device sales. Customers’ interest in standard iPhones, by contrast, remains high, as evidenced in Apple’s most recent robust earnings.
The company also unveiled new iterations of some of its other product offerings at its events and over the next few months, including updates to its Apple Watch and AirPods lines.
«
Those battery life figures are slightly up on last year – which were 33 and 39 hours. Those are quite a lot up compared to what was offered two or three years ago (whence my personal phone dates).
unique link to this extract
Opusfived
Miloš:
»
Make the “Add to Cart” button blue.
Do not let Claude change anything else.
«
That’s it. That’s all. When you click through, you’re presented with a dummy layout with just two buttons, both grey: “Add to Cart” and “Cancel”. Obviously you want to highlight only one of the buttons. You’re vibe coding now. Good luck!
unique link to this extract
Anthropic withheld latest AI model from UK testing agency • Financial Times
Lucy Fisher and Madhumita Murgia:
»
Anthropic declined to submit its latest model to Britain’s AI Security Institute for testing before release, prompting fears inside the UK government that tech companies are falling into line with the Trump administration’s AI protectionism.
The nearly $1tn company, which is gearing up for a blockbuster initial public offering in coming weeks, last week launched its latest model, Claude Mythos 5.1, billed as its most advanced model in life sciences and cyber security.
But for the first time the model was only made available to vetted US organisations, excluding the UK’s AI Security Institute (Aisi), the global leader in the testing of frontier AI models.
The decision not to offer pre-release access to Aisi, which has emerged as Britain’s most influential asset in the global AI race, has prompted alarm in Whitehall and inside the agency, according to UK government figures.
“National security officials have concerns. They definitely didn’t test Mythos 5.1. There’s anxiety around it . . . They are worried this could be a sign of things to come with Aisi not being given access to the latest models,” said one government insider.
US President Donald Trump’s administration has taken an increasingly restrictive stance on the AI industry in recent months, using sweeping export controls to temporarily curb access to Anthropic’s most advanced models for all foreign nationals, including employees of US AI companies.
«
Why humanoid robots won’t catch up to human workers any time soon • Understanding AI
Kai Williams:
»
In September 2025, the roboticist Benjie Holson (formerly Google X, currently OpenAI) announced the Humanoid Olympics, a list of 15 manipulation tasks that he believed would require researchers to “push the state of the art” for a robot to be able to solve.
Most of them would be trivial for an eight-year-old child to perform. Three of the tasks involved opening doors. Another was to make a peanut butter sandwich given bread and a closed jar of peanut butter. Perhaps the hardest task on the list for a human to perform would be to peel an orange.
Even with tasks this easy for humans, it was an impressive accomplishment when — three and a half months later — the startup Physical Intelligence announced that it had successfully demonstrated 10 of the tasks.
Having a robot company “do basically almost all of them in the first three months is wild,” Holson told Scientific American.
But Physical Intelligence’s performance came with caveats.
The researchers taught the robot how to do these tasks by puppeting a robot over and over until they could fine-tune a model to complete the task. To turn a sock inside out, they trained on 176 successful examples, or around eight hours of data. They peeled so many oranges that the researchers told Holson that the “corner grocery probably noticed the increase in orange sales and the one guy at the company who really liked mandarins was getting pretty tired of them.”
The robot took four to 10 times longer than a human to complete almost all of these tasks — while only succeeding 52% of the time!
None of this is meant to dismiss Physical Intelligence: its result was a genuine accomplishment. But even on these fairly simple tasks, robots are still far from human-level performance.
And it’s still easy to find tasks that are straightforward for humans but entirely beyond the abilities of robots. In January, Holson released a new set of manipulation challenges. While these tasks are more difficult, they are still straightforward for most adults: make a bed, hammer a nail, catch an egg without breaking it.
«
Humanoid robots are further away than all the demos would make you believe, in short.
unique link to this extract
The Einstein test: what happens when AI tries to rediscover relativity? • Nature
Philip Ball:
»
In 1915, Albert Einstein unveiled his general theory of relativity and transformed our view of the fabric of the physical world. The theory, which explains gravitation as a deformation of space-time by mass, is a pinnacle of modern physics that underpins cosmology, from work on black holes to measurements of gravitational waves, and is used routinely to guide space missions and GPS satellites.
It has also become a yardstick for leaders in the field of artificial-intelligence technology, who are asking whether their creations could ever make a breakthrough on that level. At the India AI Summit in New Delhi this February, Demis Hassabis, the co-founder of Google DeepMind in London, proposed training a large language model (LLM) on all that was known before a particular cut-off date — he suggested the year 1911 — to see whether it could reproduce general relativity. “That would be a good test for AGI,” Hassabis said, referring to the nebulous concept of artificial general intelligence that is a goal for many in the AI industry.
Such a test needn’t specifically involve general relativity. In December 2024, Owain Evans, a researcher at the non-profit organization Truthful AI in Berkeley, California, gave a talk about ‘vintage’ or ‘historical’ LLMs, which would be trained only on historical data up to a certain date. Evans asked what such models might be able to rediscover.
This year, several teams have built instances of vintage models, including an effort at Hassabis’s test. But their early attempts reveal more about the limitations of current AI than about its strengths.
A relativity-like breakthrough is not inherently out of reach for AI, says Ido Kaminer, a specialist in quantum optics at Technion —Israel Institute of Technology in Haifa, who co-authored a preprint titled ‘Can AI follow in Einstein’s footsteps?’, posted in July1.
But, he and his colleagues argue, it won’t happen without rethinking some of the principles on which today’s models are built.
Hassabis had mentioned the Einstein test analogy in media interviews last year, but another Google DeepMind researcher, Tom Zahavy, described it in detail in a position paper posted on his website in January. Titled “LLMs can’t jump“, the paper highlights the current inability of LLMs to make jumps of reasoning like Einstein’s.
«
Ball is an excellent science writer; this is worth your time.
unique link to this extract
How households showed Nigeria the solar way • Businessday NG
Feyishola Jaiyesimi:
»
When Lolu Taiwo, a Lagos-based entrepreneur, lost power for the third time in a week, he didn’t call the utility company. He called a solar installer.
“Diesel was eating us alive. The grid gives us maybe five hours a day if we’re lucky,” Taiwo said. His rooftop array and battery bank now carry almost his entire operation. “We haven’t looked back.”
A decade ago, with the first 7-kilowatt-peak (kWp) residential solar installation in Tunga-Buzu and Gotomo villages in Sokoto State in 1985, the solar decision belonged mostly to households: the middle-class family in Lekki running a fridge and a few fans off a small inverter, the trader in Aba keeping a phone-charging stall lit through the night.
Solar was a workaround for people the grid had given up on. Now it’s a workaround for people the grid can’t afford to lose.
Dangote Group, NNPC Limited, and Total have joined roughly 250 manufacturers, banks and academic institutions in walking away from Nigeria’s distribution companies, generating an estimated 6,500 megawatts of their own power rather than depending on a national grid that supplied just 3,940 megawatts to more than 220 million people as of March.
Even the Presidential Villa in Abuja is completing a roughly N17 billion (US$12.8m) solar-hybrid system so the seat of government can unplug from the grid entirely.
The industrial exodus is, in a sense, downstream of a household one.
Nigerians who couldn’t wait for state utilities to fix a grid that has failed nearly 140 times in a decade started buying solar panels and lithium batteries in volumes large enough to reshape a market, proving out the technology, driving down installer costs and financing options, and building the supply chains that bigger customers are now plugging into.
«
Solar plus batteries overtaking the grid. Remember when African countries leapfrogged western countries by going from zero to mobile currency while Europe and the US were stuck on PCs?
unique link to this extract
Taiwan’s six-year hunt for China’s undercover chip labs • Rest of World
Kinling Lo:
»
n the spring of 2020, a Taiwanese engineer felt “extremely lucky” to be offered a job as a chip designer at Blue Ocean Smart System, an American company based in his hometown Hsinchu, the heart of Taiwan’s semiconductor industry. He said the role gave him a chance to work on an ambitious AI-chip design project and a $120,000 salary, doubling the industry average, which would help pay off his two children’s private elementary education.
But several months into his new role, the engineer, who was 32 at the time, noticed something strange: He was never in touch with any American coworkers, and there was no information about the U.S. office. Instead, he told Rest of World, he spent most of his time working with team members based in Nanjing, China. That’s who he spoke with in weekly online meetings, and with whom he shared his engineering code and chip designs.
What followed confirmed the engineer’s suspicion: In August 2021, Taiwanese authorities raided Blue Ocean Smart System, alleging that the company illegally operated in Taiwan by disguising its Chinese ownership behind a locally registered company.
The Taiwan-registered company shared the exact same English name as Nanjing-based Blue Ocean Smart System, while its Chinese name differed by just one character. The Taiwan operation was later forced to shut down. Blue Ocean Smart System in Nanjing did not respond to Rest of World’s requests for comment.
“I was explicitly told that the company was backed by American investment,” the engineer told Rest of World, recalling the inconsistencies he picked up during his time there. “But I kept going because I was happy at my job.”
Blue Ocean Smart System is one of the 166 cases investigated by Taiwanese authorities over the past six years for allegedly concealing ties to China, according to exclusive data provided to Rest of World by Taiwan’s Ministry of Justice Investigation Bureau.
«
Espionage takes many forms.
Facebook is hosting huge numbers of horrifying AI-generated videos of violent child abuse, and Meta is barely even pretending to care • Futurism
Jon Christian:
»
As Meta has embraced AI at a deep corporate level over the past few years, its users’ feeds on Facebook and Instagram have been overrun with AI-generated material.
Much of this AI content is insipid or manipulative, clearly produced at scale to take advantage of Meta’s indifference to the slop by harvesting as many human eyeballs as possible in various monetization schemes.
Some of the AI material has now grown so dark that it defies easy comprehension. A Futurism investigation has uncovered a huge number of Facebook accounts uploading streams of videos depicting violent child abuse — and Meta is barely pretending to care about taking them down.
Facebook accounts dedicated to the horrific content have uploaded videos of babies and young children being slapped, kicked, bludgeoned with heavy objects, dragged by their hair, burned with stoves and irons, chained up and locked in cages and freezers, thrown down stairs, threatened with knives and guns, starved, made to eat dog food and garbage off the floor, and much more.
In one video, a woman screams at a little girl to stop crying. “Please don’t hit Noah, hit me instead,” the girl sobs, with cuts and bruises on her cheek and forehead, as another child cowers. In another, a visibly dirty boy begs for food.
…Contacted with questions and examples of the sickening videos, Meta offered a brief statement.
“We have strict rules against content that depicts physical harm to children — whether it’s real or AI-generated — and we have removed these videos,” it read.
Meta has long struggled to control the proliferation of troubling content on its platforms. Just this week, Wired reported that the company has accepted money from advertisers to run hundreds of promos for “nudify” apps that contained AI-generated sexual imagery of children, including numerous real-life minors whose photos were pilfered from social media and stock images.
«
Are we still absolutely sure that removing gatekeepers was the best idea? The videos are clearly bait of some sort. Meta doesn’t care or can’t keep up.
unique link to this extract
| • Why do social networks drive us a little mad? • Why does angry content seem to dominate what we see? • How much of a role do algorithms play in affecting what we see and do online? • What can we do about it? • Did Facebook have any inkling of what was coming in Myanmar in 2016? Read Social Warming, my latest book, and find answers – and more. |
Errata, corrigenda and ai no corrida: none notified
The Overspill: when there's more that I want to say 