Start Up No.2406: AI slop’s attack on social media, don’t trust the chatbots!, US rural broadband faces cuts, and more


The tolerances required for the ball bearing of a ballpoint pen eluded Chinese manufacturing for decades, and only in the past few years has it met them. CC-licensed photo by Andrew Magill on Flickr.

You can sign up to receive each day’s Start Up post by email. You’ll need to click a confirmation link, so no spam.


Last week’s Social Warming Substack post is available for your delectation.


A selection of 9 links for you. Flowing. I’m @charlesarthur on Twitter. On Threads: charles_arthur. On Mastodon: https://newsie.social/@charlesarthur. On Bluesky: @charlesarthur.bsky.social. Observations and links welcome.


AI slop is a brute force attack on the algorithms that control reality • 404 Media

Jason Koebler:

»

Consider, for a moment, that this AI-generated video of a bizarre creature turning into a spider turning into a nightmare giraffe inside of a busy mall has been viewed 362 million times. That means this short reel has been viewed more times than every single article 404 Media has ever published, combined and multiplied tens of times. 

Any of these Reels could have been and probably was made in a matter of seconds or minutes. Many of the accounts that post them post multiple times per day. There are thousands of these types of accounts posting thousands of these types of Reels and images across every social media platform. Large parts of the SEO industry have pivoted entirely to AI-generated content, as has some of the internet advertising industry. They are using generative AI to brute force the internet, and it is working. 

One of the first types of cyberattacks anyone learns about is the brute force attack. This is a type of hack that relies on rapid trial-and-error to guess a password. If a hacker is trying to guess a four-number PIN, they (or more likely an automated hacking tool) will guess 0000, then 0001, then 0002, and so on until the combination is guessed correctly. 

As you may be able to tell from the name, brute force attacks are not very efficient, but they are effective. An attacker relentlessly hammers the target until a vulnerability is found or a password is guessed. The hacker is then free to exploit that target once the vulnerability is found.

The best way to think of the slop and spam that generative AI enables is as a brute force attack on the algorithms that control the internet and which govern how a large segment of the public interprets the nature of reality. It is not just that people making AI slop are spamming the internet, it’s that the intended “audience” of AI slop is social media and search algorithms, not human beings.

What this means, and what I have already seen on my own timelines, is that human-created content is getting almost entirely drowned out by AI-generated content because of the sheer amount of it. On top of the quantity of AI slop, because AI-generated content can be easily tailored to whatever is performing on a platform at any given moment, there is a near total collapse of the information ecosystem and thus of “reality” online. I no longer see almost anything real on my Instagram Reels anymore, and, as I have often reported, many users seem to have completely lost the ability to tell what is real and what is fake, or simply do not care anymore.

«

It’s that latter point which is relevant. People don’t care. They don’t reject it and so it inveigles its way into their content, and the platforms don’t care because it all gets viewed and keeps people on there.
unique link to this extract


AI search has a citation problem • Columbia Journalism Review

Klaudia Jaźwińska and Aisvarya Chandrasekar:

»

AI search tools are rapidly gaining in popularity, with nearly one in four Americans now saying they have used AI in place of traditional search engines. These tools derive their value from crawling the internet for up-to-date, relevant information—content that is often produced by news publishers. 

Yet a troubling imbalance has emerged: while traditional search engines typically operate as an intermediary, guiding users to news websites and other quality content, generative search tools parse and repackage information themselves, cutting off traffic flow to original sources. These chatbots’ conversational outputs often obfuscate serious underlying issues with information quality. There is an urgent need to evaluate how these systems access, present, and cite news content.
Building on our previous research, the Tow Center for Digital Journalism conducted tests on eight generative search tools with live search features to assess their abilities to accurately retrieve and cite news content, as well as how they behave when they cannot.

We found that…
• Chatbots were generally bad at declining to answer questions they couldn’t answer accurately, offering incorrect or speculative answers instead
• Premium chatbots provided more confidently incorrect answers than their free counterparts
• Multiple chatbots seemed to bypass Robot Exclusion Protocol preferences
• Generative search tools fabricated links and cited syndicated and copied versions of articles
• Content licensing deals with news sources provided no guarantee of accurate citation in chatbot responses.

Our findings were consistent with our previous study, proving that our observations are not just a ChatGPT problem, but rather recur across all the prominent generative search tools that we tested. 

«

One of the sections is headed “Chatbots’ responses to our queries were often confidently wrong”, which certainly sums up a lot of what one sees.
unique link to this extract


AI failed to detect critical health conditions: study • Axios

Maya Goldman:

»

Hospitals increasingly use tools that harness machine learning, a subset of AI that focuses on systems that continuously learn and adjust as they’re given new data.

A separate study recently published in Health Affairs found that about 65% of U.S. hospitals use AI-assisted predictive models, most commonly to figure out inpatient health trajectories.

Researchers looked at several machine learning models commonly cited in medical literature for use in predicting patient deterioration and fed them publicly available sets of data about the health and metrics of patients in ICUs or with cancer.

The researchers then created test cases for the models to predict potential health issues and risk scores if some patient metrics were altered from the initial data set.

The models for in-hospital mortality prediction could only recognize an average of 34% of patient injuries, the study found.

What they’re saying: “We are asking the models to make big decisions, and so we really need to figure out … in what kind of situations they can perform,” said Danfeng (Daphne) Yao, an author of the study and a computer science professor at Virginia Tech.

«

unique link to this extract


Here’s a “dead” person on Social Security in Seattle, with plenty to say • The Seattle Times

Danny Westneat:

»

“DOGE Has 10 Staffers at Social Security in Hunt for Dead People,” the headlines read this past week.

I found a dead person on Social Security. Right here in Seattle, on Capitol Hill.

Of course the circumstances of Ned Johnson’s death were completely the opposite of what Elon Musk and his Department of Government Efficiency had claimed was rampant.

“You wake up one day and discover you’re dead,” Johnson told me. “It’s been truly surreal.”

That’s the biggest difference — my deceased guy turns out to be very much alive. Musk is contending that hordes of dead people are listed as alive in the Social Security databases, and are fraudulently still drawing benefits (which the Social Security director disputes).

Johnson is 82 and still kicking. Yet sometime last month, someone or something led Social Security to both tag him as dead and start clawing back his benefits.

Johnson’s strange trip through the netherworld began in February, when a letter from his bank arrived addressed to his wife, Pam. “We recently received notification of LEONARD A. JOHNSON’s passing,” it began. “We offer our sincerest condolences …”

At first she figured it was a scam — her husband, after all, was sitting right there. But then the bank got to the point. “We know this is a difficult time, and we’re here to help,” the bank wrote. “We received a request from Social Security Administration to return benefits paid to LEONARD A. JOHNSON’s account after their passing.”

“There’s nothing you need to do — we’ve deducted the funds from LEONARD A. JOHNSON’s account.” Uh oh. It itemized how $5,201 had been stricken from their bank account, on the grounds that Ned wasn’t justified to get those benefits — because he was dead. That was for payments he’d received in December and January.

…What followed was a nearly three-week battle to resurrect himself. He called Social Security two or three times a day for two weeks, with each call put on hold and then eventually disconnected. Finally someone answered and gave him an appointment for March 13. Then he got a call delaying that to March 24.

In a huff, he went to the office on the ninth floor of the Henry Jackson Federal Building downtown. It’s one of the buildings proposed to be closed under what the AP called “a frenetic and error-riddled push by Elon Musk’s budget-cutting advisers.”

It was like a Depression-era scene, he said, with a queue 50-deep jockeying for the attentions of two tellers. The employees were kind but beleaguered.

«

This scene is going to be repeated all over the US, and journalism is going to be almost powerless to describe it – individual stories like this don’t give any scale, and stories of thousands of people being denied benefits lack focus.
unique link to this extract


Top broadband official exits Commerce Department with sharp Musk warning • POLITICO

John Hendel:

»

A top Commerce Department official sent a blistering email to his former colleagues on his way out the door Sunday warning that the Trump administration is poised to unduly enrich Elon Musk’s satellite internet company with money for rural broadband.

The technology offered by Starlink, Musk’s company, is inferior, wrote Evan Feinman, who had directed the $42.5bn broadband program for the past three years.

“Stranding all or part of rural America with worse internet so that we can make the world’s richest man even richer is yet another in a long line of betrayals by Washington,” Feinman said.

Feinman’s lengthy email, totaling more than 1,100 words and shared with POLITICO, is a sign of deep discomfort about the changes underway that will likely transform the Broadband Equity, Access and Deployment Program. Commerce Secretary Howard Lutnick recently pledged a vigorous review of BEAD, with an aim to rip out what he sees as extraneous requirements and remove any preference for particular broadband technologies like fiber.

Musk, who runs the Starlink satellite broadband service, stands to reap a greater share of these subsidies under the revised rules.

Musk and Starlink did not respond to requests for comment.

The program, created in the 2021 infrastructure law program, became a source of partisan fighting last year on the campaign trail as Republicans attacked the Biden administration for its slow pace. No internet expansion projects have begun using BEAD money, although some states were close at the beginning of this year.

Feinman’s critique: In his email, Feinman notes Friday was his last day leading BEAD and that he’s “disappointed not to be able to see this project through.”

«

Certainly the problem with fixed internet is.. getting it in. Once that’s done, it’s cheap. Satellite internet is quick to get in, but expensive forever.
unique link to this extract


Alphabet spins off laser-based internet project from “moonshot” hub • Financial Times

Stephen Morris:

»

Alphabet is spinning out laser-based internet company Taara from its “moonshot” incubator, hoping to turbocharge the start-up that provides high-bandwidth services to hard-to-reach areas in competition with Elon Musk’s Starlink network of satellites.

Taara is the latest project to spring from X — Alphabet’s experimental hub that produced artificial intelligence lab Google Brain and Waymo’s self-driving cars — and has its origins in a concept called Loon. That envisaged shooting beams of light between thousands of balloons floating on the edge of space to provide phone and internet services across remote areas.

Loon was wound up in 2021 because of political and regulatory hurdles to flying the balloons and the difficulty of servicing the 20-mile-high equipment. However, its lasers found a second life on Taara’s towers under engineer Mahesh Krishnaswamy.

The technology works by firing a beam of light the width of a pencil from one traffic light-sized terminal to another, using a system of sensors, optics and mirrors to fix it on a 1.5 inch receiver. Alphabet says the system can transmit data at 20 gigabits per second over 20km, extending traditional fibre-optics networks with minimal construction and lower costs.

Based in Sunnyvale near Google’s headquarters in Mountain View, California, Taara has two dozen staff and is hiring aggressively. The start-up has secured backing from Series X Capital and Alphabet will retain a minority stake, but the company refused to disclose any details about its seed funding or financial targets.

«

Feels like a long time since Alphabet spun off something from its moonshot.
unique link to this extract


Regional newspaper ABCs: no daily now has print circulation of 20,000 or more • Press Gazette

Bron Maher:

»

Print circulations at UK regional daily newspapers collectively declined 16% between 2023 and 2024, according to the latest data from ABC.

That marks a slight slowing after circulations dropped 17% between the first halves of 2023 and 2024.

But it nonetheless marks a milestone as circulations at the last two regional dailies to circulate 20,000 print copies or more per day, the Irish News and Aberdeen’s Press & Journal, sank below that benchmark and the Evening Standard [in London] ceased as a daily title.

The Standard’s average daily circulation in its final month reported to ABC, August, was 273,631. The weekly London Standard is now the non-daily regional paper with the largest circulation, averaging 148,021 copies per issue between October and December.

Total circulations per issue at the 417 non-daily UK local newspapers audited by ABC fell at a slightly slower rate of 14% between 2023 and 2024.

Across the 69 dailies, the total number of single copies sold at newsstands circulated per day also dropped 16%, to 279,000. Paid subscriptions fell 15% to 67,000 per day and free copies dropped more than a third to a total average of 1,364 per day.

«

The golden days for local papers were 1985 to 2004, according to this article in the New Statesman (free registration). The slope now is very steep, and downhill.

Losing local coverage means the loss of so much accountability.
unique link to this extract


BYD unveils new super-charging EV tech, to build charging network in China • Reuters

Qiaoyi Li, Zhang Yan and Brenda Goh:

»

BYD on Monday unveiled a new platform for electric vehicles (EVs) that it said could charge EVs as quickly as it takes to pump gas and announced for the first time that it would build a charging network across China.

The so-called “super e-platform” will be capable of peak charging speeds of 1,000 kilowatts (kW), enabling cars that use it to travel 400 km (249 miles) on a five-minute charge, founder Wang Chuanfu said at an event livestreamed from the company’s Shenzhen headquarters.

Charging speeds of 1,000 kW would be twice as fast as Tesla’s superchargers whose latest version offers up to 500 kw charging speeds. Fast-charging technology has been key to increasing EV adoption as it is seen to help assure EV drivers’ concerns over being able to charge their cars quickly.

“In order to completely solve our user’s charging anxiety, we have been pursuing a goal to make the charging time of electric vehicles as short as the refuelling time of petrol vehicles,” Wang said.

“This is the first time in the industry that the unit of megawatt (charge) has been achieved on charging power,” he said.
The new charging architecture will be initially available in two new EVs – Han L sedan and Tang L SUV priced from 270,000 yuan ($37,330) and BYD said it would build over 4,000 ultra-fast charging piles, or units, across China to match the new platform.

The company didn’t specify the time frame or how much it would invest in building such facilities. To date, BYD owners have largely relied on other automakers’ charging facilities or public charging poles run by third-party operators to charge their vehicles.

«

This is certainly the part of the trifecta to make EVs completely acceptable: plentiful locations with rapid charging at low prices. It’s the latter which may still be out of reach.
unique link to this extract


2017: Finally, China manufactures a ballpoint pen all by itself • The Washington Post

Adam Taylor, in 2017:

»

To anyone outside of the ballpoint pen manufacturing world, it might seem hard to understand what, exactly, is so surprising about this development. China already produces 38 billion ballpoint pens a year, according to China Daily, which is about 80% of all ballpoint pens in the world. That’s a lot of pens, but there was a catch: China had long been unable to produce a high-quality version of the most important part of the pen, its tip.

The tip of a ballpoint pen is what makes it a ballpoint pen. At the tip, a freely rotating ball is held in a small socket which connects it to an ink reservoir that allows the pen to write or draw lines. Manufacturing a ballpoint pen tip that can write comfortably for a long period of time requires high-precision machinery and precisely thin steel, but for years China was unable to match those crafted by foreign companies.

While there were over 3,000 companies manufacturing pens in China, none had their own high-end technology for the tip. Instead, about 90% of the pen tips and refills, too, were imported from Japan, Germany and Switzerland, according to Chinese state media. This cost the industry $17.3m a year, according to the China National Light Industry Council.

«

The fact that ballpoints require such very careful manufacturing isn’t that obvious – we take it for granted – but shows how China has speedrun through manufacturing history.
unique link to this extract


• Why do social networks drive us a little mad?
• Why does angry content seem to dominate what we see?
• How much of a role do algorithms play in affecting what we see and do online?
• What can we do about it?
• Did Facebook have any inkling of what was coming in Myanmar in 2016?

Read Social Warming, my latest book, and find answers – and more.


Errata, corrigenda and ai no corrida: none notified

1 thought on “Start Up No.2406: AI slop’s attack on social media, don’t trust the chatbots!, US rural broadband faces cuts, and more

  1. These days, if I see a breathless article that proclaims something along the lines of “Donald Trump ate a dog to demonstrate how DEI illegal criminals are ruining the country, and RFK Jr endorsed doing that as consuming healthy protein”, I just feel like it’s not worth the bother to point out it probably didn’t happen, since something similarly crazy certainly did (or even worse). Someone might wag a finger and say we should always speak up for the truth because that increases public trust, but I’m really dubious that’s correct, especially nowadays. In specific, the Social Security article is written quite disingenuously to leave an impression that Musk is somehow connected to the guy being deemed dead, but there’s no evidence of any cause and effect. Quite the opposite, the timeline actually implies there’s no connection at all. It’s also obviously wrong because 82 is not at all suspiciously old. If the man was 110, which is possible, but extraordinarily rare, that might be worth wondering about. But as the “buried lede” eventually notes, Musk is making things horrible, even if he didn’t cause this particular problem.

Leave a reply to Seth Finkelstein Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.