Start Up No.2676: the superintelligence problem, Instagram AI helps hackers take over accounts, trouble in the solar business, and more


If a picture on a wall is described as a Monet painting, you probably believe it’s real. What if you’re told it’s actually AI, though? CC-licensed photo by Gerard Stolk on Flickr.

You can sign up to receive each day’s Start Up post by email. You’ll need to click a confirmation link, so no spam.


A selection of 9 links for you. Floating along. I’m @charlesarthur on Twitter. On Threads: charles_arthur. On Mastodon: https://newsie.social/@charlesarthur. On Bluesky: @charlesarthur.bsky.social. Observations and links welcome.


Superintelligence: the idea that eats smart people • Idle Words

Maciej Cieglowski, in a speech given to a conference:

»

Our intelligence level, cognitive speed, set of biases and so on is not predetermined, but an artifact of our evolutionary history.

In particular, there’s no physical law that puts a cap on intelligence at the level of human beings.

A good way to think of this is by looking what happens when the natural world tries to maximize for speed.

If you encountered a cheetah in pre-industrial times (and survived the meeting), you might think it was impossible for anything to go faster.

But of course we know that there are all kinds of configurations of matter, like a motorcycle, that are faster than a cheetah and even look a little bit cooler.

But there’s no direct evolutionary pathway to the motorcycle. Evolution had to first make human beings, who then build all kinds of useful stuff.

So analogously, there may be minds that are vastly smarter than our own, but which are just not accessible to evolution on Earth. It’s possible that we could build them, or invent the machines that can invent the machines that can build them.

There’s likely to be some natural limit on intelligence, but there’s no a priori reason to think that we’re anywhere near it. Maybe the smartest a mind can be is twice as smart as people, maybe it’s sixty thousand times as smart.

That’s an empirical question that we don’t know how to answer.

«

Cieglowski (who runs the Pinboard service which this newsletter relies on – 26,940 bookmarks so far!) has always been an excellent analyst of ideas. This talk is from 2016, which is why it’s worth looking at again now, when everything has changed. (On the way he comes up with a version of the idea that became the TV series Pluribus. That’s how good he is.)
unique link to this extract


Someone shared a real Monet painting as AI and asked for critiques • PetaPixel

Michael Zhang:

»

A fascinating art social experiment unfolded on social media this week after someone shared an actual Monet painting as an AI-generated artwork and asked people to explain what makes the “AI image” inferior to a genuine Monet piece. There was no shortage of “sharp-eyed” critics eager to chime in.

It all started after X user @SHL0MS posted the painting and wrote: “I just generated an image in the style of a Monet painting using AI. Please describe, in as much detail as possible, what makes this inferior to a real Monet painting.”

The user even marked the post with X’s “Made with AI” label to add to the deception.

In reality, the painting is one of the 250 oil paintings in the renowned French Impressionist painter Claude Monet’s Water Lilies series in which he depicted scenes from his home flower garden over the final 31 years of his life.

The painting is one of the 250 or so paintings in the famous “Water Lilies” series by French painter Claude Monet.

Critics, however, were eager to point out all kinds of “obvious” details that show why the “AI” Monet can’t hold a candle to a genuine Monet. One person even took the time to write out an 850-word breakdown of the AI work’s shortcomings.

…[Another critic wrote:] “As an amateur art enjoyer, the only criticism I can offer is that the AI generated image does not make me feel anything,” writes @ThrosturTh. “It does not conjure emotion, thought or wonder. It’s just a colorful wallpaper pattern. If you look up ‘monet painting’ in Google images, you feel something.”

«

That last one is probably where the other “critics” begin, but don’t say so. (Monet’s deteriorating eyesight means that the lilies series shows us how his vision was changing.) What we want to know, it seems, is that a human held the paintbrush.
unique link to this extract


Instagram Meta AI vulnerability allegedly enables password reset for accounts • Cyber Security News

Guru Baran:

»

A critical flaw in Meta’s AI-powered account recovery tool on Instagram allowed attackers to hijack high-value accounts by tricking the chatbot into forwarding password reset codes with no verification required.

Security researchers ZachXBT and Dark Web Informer were among the first to publicly expose the vulnerability, revealing that threat actors had found a way to manipulate Instagram’s Meta AI assistant a tool designed to help users recover access to their accounts.

Attackers engaged the AI chatbot in conversation and prompted it to forward password reset codes to unauthorized parties, entirely bypassing identity verification checks. The flaw stemmed from insufficient controls in how the AI processed account recovery requests, effectively allowing anyone who knew a target’s username to initiate the takeover process.

The exploit was not a traditional server breach Meta confirmed no backend systems were compromised. Instead, the vulnerability lived in the AI’s logic layer, which lacked proper rate-limiting or authentication enforcement before acting on reset requests.

These coveted accounts, some valued at over $1m combined, were quickly flipped through private Telegram channels before Meta could intervene. The speed of the operation highlighted how organized and financially motivated threat actors have become in exploiting social media platform vulnerabilities.

«

There have been suggestions that Meta has seriously cut staffing in the Trust & Safety part of Instagram/Meta, while encouraging staff to use more and more AI. It was bad enough before when you might struggle for hours or days to get in touch with a human over a hacked account; to have the company’s AI helping the hackers is definitely worse. (There’s a good writeup too at Brian Krebs’s security blog.)
unique link to this extract


Nvidia announces RTX Spark as ‘the most efficient PC chip ever built’ • The Verge

Sean Hollister:

»

This fall, Nvidia will officially become a consumer PC chipmaker like Intel, AMD, Apple, and Qualcomm, putting a complete computing chip — not just graphics — into the very heart of laptops and mini-PCs. After many months of leaks, it’s finally announcing the RTX Spark, the first in a family of chips that will meet or beat the most powerful thin-and-light Windows machines ever, it claims.

“This is the most efficient PC chip ever built,” says Nvidia senior director of product management Mark Aevermann — without sharing so much as a single statistic or chart to back that up.

The RTX Spark is effectively the same GB10 chip that’s in the DGX Spark, the tiny “personal AI supercomputer” that Nvidia released last year, only now it’s a family of chips instead of just one. The flagship version appears to be spec-to-spec identical with 20 CPU cores, 6,144 GPU cores, and 128GB of LPDDR5X memory.

But Nvidia says there’ll be lesser versions later, targeting lower prices, and with as little as 16GB of RAM.

Like Apple and Qualcomm’s chips, this Nvidia chip is Arm-based silicon, meaning legacy Windows software made for Intel and AMD’s x86 processors needs to run through an emulation layer to work. That can mean lower performance. But Microsoft has now spent years getting Windows and its Prism emulator ready for Qualcomm and now Nvidia chips, and Nvidia claims its own graphics and AI chops will take the idea further than ever before.

«

Promises, promises. There’s also the question of quite how much AI people want thrust in their face.
unique link to this extract


Intel: Our upcoming AI chip will be cheaper, run cooler than Nvidia, AMD options • Financial Times via Ars Technica

»

Intel plans to ship an AI chip by the end of this year that uses cheaper memory and cooling technology than rival offerings from Nvidia and AMD, as the US chipmaker seeks to capitalize on a sharp turnaround in its fortunes.

Kevork Kechichian, who leads Intel’s data center group, told the FT that the company is “starting with the basics” as it tries to challenge its rivals in the booming market for semiconductors that power AI.

Its new “Crescent Island” graphics processing unit is designed to speed up “inference” tasks, the stage when a user makes their request, rather than the training of models, an area where Nvidia’s processors are dominant.

An earlier attempt at building a GPU for training AI models called “Gaudi” saw poor sales, and its planned successor was cancelled last year.

“We decided to start rebuilding our muscles in AI… [but] we are not particularly aiming for [the training market] based on past experience,” said Kechichian, who joined Intel last year from chip designer Arm.

He added the new chip would start shipping in limited quantities to customers by the end of this year, following an 18-month development process.

«

Intel used to have it easy: there was it, and AMD as a very distant second. Then smartphones came along, Arm architecture became huge, and then Apple and Nvidia began making inroads, and everything went south. Nowadays you can feel the desperation to be counted alongside the sector leaders who are making money hand over fist.
unique link to this extract


China’s world-beating solar industry is in turmoil • The Economist

»

China’s solar exports have enjoyed a surge since the bombing began. But that will be small cheer to its companies, as they face three daunting problems. Domestic demand for their products is falling for the first time in decades because the country’s power grids—far and away the biggest market for solar panels—have become overloaded with the things. Solar-panel supply, meanwhile, is overabundant because of years of splashy investment in factories. And even while the war is helping solar sales in South-East Asia and Africa, protectionism has been on the rise in the bigger Western markets.

These problems converge at an ugly time. Most companies have been running at a loss since 2024 because of brutal price wars; bankruptcies are mounting. After experiencing blistering growth, the world’s solar factory now faces a reckoning.

Globally, the solar industry has not always been kind to investors. One solar panel is much like another, and when improvements are made by one producer, they are rapidly copied by competitors. So companies typically try to scale up production as quickly as possible in order to seize market share. That means production can race far ahead of demand, causing margins to collapse.

…Installations this year could fall by between 24% and 43% from 2025, according to an industry group. That would be enough to cause global demand for solar panels to fall in 2026 for the first time in two decades, says BloombergNEF, a consultancy. For China’s grids to cope, it needs to be able to store excess solar power or move it long distances to where it might be needed. That requires big investments in batteries and power lines, as well as figuring out flexible market mechanisms to co-ordinate everything (in some regions long-term contracts for coal-fired power lock out renewables even though they are cheaper).

All this is happening, helped by the fact that batteries, like solar panels, are becoming much cheaper as production of them increases. But it takes time. That means that even if solar installations start rising again next year, growth will probably be much slower than before.

«

unique link to this extract


Best Sim only deals: compare cheap contracts • Money Saving Expert

»

Switching to a cheaper deal with another network is really easy. Under rules from regulator Ofcom, you no longer have to call your current provider to ditch it, so you avoid the hard sell of it trying to make you stay. Here’s how to switch providers and keep your number:

Request a switching code by text. Simply text ‘PAC’ for free to 65075 on any network and you’ll be sent your ‘porting authorisation code’ (PAC) immediately by text so you can keep your number. To get a new number, text ‘STAC’ to 75075. Its text must include important info, such as any exit penalties, outstanding handset costs or credit balances.

You need to give the switching code to your new provider within 30 days. You’ll then be switched within one working day. You won’t need to contact your old provider again.

What’s more, if you’re out of your minimum contract period, mobile providers are now banned from charging you for the remainder of your notice period after you’ve switched (which is typically 30 days), putting an end to paying for your old and new contract at the same time.

«

Martin Lewis noticed the article yesterday about the rising tide of mobile/broadband pricing, and linked to this page as a riposte. So if you’re looking to save some money…
unique link to this extract


Traffic to DuckDuckGo’s proudly ‘No AI’ search page has tripled since latest Google AI search update • PC Gamer

Ted Litchfield:

»

Privacy-focused search engine DuckDuckGo has announced that its opt-in, generative AI-limiting “No AI” search page has received three times as much traffic since Google’s latest round of updates and PR for AI mode search. In addition to user privacy, DuckDuckGo is positioning itself as a destination for user choice—one that’s still keeping its options open with AI.

Three days ago, we reported on how installs of DuckDuckGo’s app were up by almost a third following Google’s latest push for AI in search. This latest announcement from DuckDuckGo appears to exclusively pertain to noai.duckduckgo.com, its AI opt-out search option.

“Since Google revealed its plans for an AI search overhaul, visits to our ‘No AI’ search page have tripled,” the company wrote on Bluesky. “And they’re still rising! Want to make it your default on Chrome or Firefox? Grab our No-AI extensions and banish AI-assisted answers, chat, and AI images.”

«

What we don’t know – and can’t know (because DDG doesn’t track its users) – is whether those are users who previously used DDG but are now going to the no-AI page (its normal page includes an AI-generated answer), or whether they’re people utterly disgusted with Google’s AI offering seeking respite from it. Slight suspicion it’s the former.
unique link to this extract


United flight forced to turn around because of a Bluetooth speaker name • The Verge

Terrence O’Brien:

»

United flight 236 from Newark to Palma de Mallorca on Saturday night was forced to turn around just an hour after takeoff due to security concerns around a Bluetooth signal. Multiple Redditors claimed to be on the flight and reported that the crew repeatedly requested passengers to turn off their Bluetooth. According to one poster, the crew issued a one-minute warning, saying that two devices were still active.

One Redditor reported flight attendants making comments like, “This little joke is ruining it for everyone.”

An archived recording from Air Traffic Control (embedded below) confirms that the root of the issue was the name of a discoverable Bluetooth speaker.

»

“There’s a security detail out there. Someone had a Bluetooth speaker, and they named it a certain four-letter word. So they have to inspect the whole aircraft, including the cargo area, and the passengers have to evacuate.”

«

«

Doesn’t take a genius to guess that the word is b-o-m-b, which is an exceptionally stupid name to give your Bluetooth speaker on a commercial aircraft.
unique link to this extract


• Why do social networks drive us a little mad?
• Why does angry content seem to dominate what we see?
• How much of a role do algorithms play in affecting what we see and do online?
• What can we do about it?
• Did Facebook have any inkling of what was coming in Myanmar in 2016?

Read Social Warming, my latest book, and find answers – and more.


Errata, corrigenda and ai no corrida: Simon Calder, formerly The Independent’s travel correspondent, isn’t retiring; he’s moving to The Telegraph. Should have known. (Thanks Simon D for pointing that out.)

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.