Start Up No.2639: sycophant chatbots harm human judgement, Australia’s fuel stocks dip, PC shipments forecast to fall 5%, and more


Shortages of helium shipments from Qatar are already hitting tech manufacturing. CC-licensed photo by fdecomite on Flickr.

You can sign up to receive each day’s Start Up post by email. You’ll need to click a confirmation link, so no spam.


A selection of 9 links for you. Deflated. I’m @charlesarthur on Twitter. On Threads: charles_arthur. On Mastodon: https://newsie.social/@charlesarthur. On Bluesky: @charlesarthur.bsky.social. Observations and links welcome.


Study finds sycophantic AI can undermine human judgment • Ars Technica

Jennifer Ouellette:

»

We all need a little validation now and then from friends or family, but sometimes too much validation can backfire—and the same is true of AI chatbots. There have been several recent cases of overly sycophantic AI tools leading to negative outcomes, including users harming themselves and/or others. But the harm might not be limited to these extreme cases, according to a new paper published in the journal Science. As more people rely on AI tools for everyday advice and guidance, their tendency to overly flatter and agree with users can have harmful effects on those users’ judgment, particularly in the social sphere.

The study showed that such tools can reinforce maladaptive beliefs, discourage users from accepting responsibility for a situation, or discourage them from repairing damaged relationships. That said, the authors were quick to emphasize during a media briefing that their findings were not intended to feed into “doomsday sentiments” about such AI models. Rather, the objective is to further our understanding of how such AI models work and their impact on human users, in hopes of making them better while the models are still in the early-ish development stages.

Co-author Myra Cheng, a graduate student at Stanford University, said she and her co-authors were inspired to study this issue after they began noticing a pronounced increase in the number of people around them who had started relying on AI chatbots for relationship advice—and often ended up receiving bad advice because the AI would take their side no matter what. Their interest was bolstered by recent surveys showing nearly half of Americans under 30 have asked an AI tool for personal advice. “Given how common this is becoming, we wanted to understand how an overly affirming AI advice might impact people’s real-world relationships,” said Cheng.

Granted, there has been some prior research looking at AI sycophancy, but these focused on very limited settings, such as how often an AI tool will agree with you even if means contradicting a well-established fact. Cheng and her co-authors wanted to look more closely at the broader social implications.

«

Given the number of people who (wrongly) seem to think chatbots are a sufficient replacement for search engines, this is a worrying finding. The ideal would be for all the chatbot makers to make their systems grumpier, but that’s just not going to happen, because people like smarmy robots.
unique link to this extract


Model collapse is already happening, we just pretend it isn’t • Communications of the ACM

Alex Williams:

»

People tend to imagine model collapse as some dramatic cliff where a model suddenly starts producing gibberish. The reality is more subtle and, honestly, more dangerous because of that subtlety. What you get is a slow erosion of variance. Outputs become more generic. Edges get sanded down. The model starts producing text that reads fine on a surface level but carries less information per sentence.

Think of it like making a photocopy of a photocopy. The first few generations look almost identical to the original. But by the tenth copy, the image is washed out. Everything is legible, technically; it’s just lost the detail that made it useful.

In language, that loss shows up as homogenization. The model reaches for the same sentence structures, the same hedging phrases, the same predictable cadences. If you’ve ever read a block of text and thought “something about this feels AI-generated” without being able to point to a specific error, you’ve already felt what an AI-native web looks like from the outside post model collapse.

The obvious response is: just filter the synthetic data out. Use classifiers to detect AI-generated text and exclude it from training sets. In theory, that’s fine. In practice, it’s becoming nearly impossible at scale.

Detection tools are in an arms race with the models producing the content, and the models are winning. As generation quality improves, the statistical signatures that classifiers rely on get weaker. Watermarking has been proposed, but there’s no universal standard, no adoption incentive, and plenty of ways to strip watermarks after the fact. The metadata layer of the Internet was never designed to track whether a piece of text was written by a person or a machine.

What the field actually needs is a robust data provenance infrastructure. We need to know where training data came from, how it was generated, and whether it’s been through a model already. That’s a hard systems problem, and it’s not glamorous enough to attract the attention it deserves. Everyone wants to build the next frontier model. Very few people want to build the plumbing that makes frontier models trustworthy.

«

unique link to this extract


Helium shortage has started impacting tech supply chains, execs say • Reuters

Eduardo Baptista:

»

Tightened supply of helium due to the Middle East conflict has started affecting some production in the global tech supply chains, leaving ​companies scrambling to secure alternative supplies, industry executives said.

Helium is used in several key stages of chipmaking, including cooling, leak detection and precision manufacturing processes, and its prices have soared since the Middle East crisis began.

Supply of helium, a byproduct of natural ​gas processing, is highly concentrated geographically, with Qatar producing nearly one third ​of world supply, according to data from the US Geological Survey.

Qatar contributes a major share of global helium supply—around 63 million cubic feet in 2025 [34% of the global total]—making it the largest producer outside the U.S [which produces 81m cu ft, 44% of global output].

“A helium shortage is an absolute concern,” said Cameron Johnson, senior partner at supply ​chain consultancy Tidal Wave Solutions, at Semicon China in Shanghai, one of the ​industry’s largest annual gatherings.

He said companies had few immediate options beyond slowing output and prioritising critical products, adding that many were hoping for a quick resolution.

Prolonged shortages could force production cuts ​and ripple through industries from electronics to automobiles, Johnson added.

“As there’s a shortage, ​companies might start slowing production or ultimately shutting production down, making chips,” he said. “If that happens, you will see an impact on things like electronics, automobiles, even smartphones.”

Jerry Zhang, China sales head at Swiss semiconductor components firm VAT, said the conflict in the Middle East had tightened helium supply and was already affecting production at his and other ​companies, adding that transport ​delays were compounding the impact. The company is seeking alternative sources, including from the United States.

«

Nobody, absolutely nobody, thought this through. And now everyone gets to pay the price.
unique link to this extract


‘Things are going to get worse’: swing-state mothers sour on government • Semafor

David Weigel:

»

They felt “overwhelmed” by bad news and grocery bills. They weren’t sure who “you can trust these days.” And they couldn’t name a politician that might fix it.

By convening a focus group of seven swing state voters, all mothers between 27 and 48 years old, the Democratic firm Navigator Research found angst about the country’s direction and little faith in either party’s ability to fix its problems.

“It’s just very overwhelming to me trying to figure out what direction we’re actually going in,” said a 40-year-old mother of three from Georgia, who’d voted for neither President Donald Trump nor Kamala Harris in 2024.

“I feel like things are going to get worse before they get better,” said a 38-year-old mother in Wisconsin who’d supported Trump. “It’s going to take a lot to break down those evil walls that were built up and all the evil behind everything so that things can be better eventually.”

Navigator Research regularly gathers groups of representative Americans to check in with voters who aren’t overly engaged with politics. Last week’s session, which was viewed by Semafor, focused on women who felt it had become too expensive to comfortably support a family in the US.

“The gas prices are going up right when taxes hit,” said a 27-year-old mother from Arizona.

But the voters surveyed didn’t seem to see either party as the solution. Even those who had voted for Harris couldn’t name a Democrat who spoke to their needs.

«

I think anyone would be hard-pressed right now to name a Democrat politician who has all the answers and has the attention of voters. However:

»

Asked what news sources she relied on, the Michigan mother said traditional news had “turned into the shows my grandma used to watch” — soap operas with forced narratives. Two women who had voted for Harris said that they got news and analysis from right-wing personality Candace Owens; one of them even said that she’d like to see Owens, who is known for spreading conspiracy theories, run for president. “She does fact-check,” she explained. “She has a lot of resources.”

«

OK, that is quite worrying. If there’s anyone who should be kept as far as possible from anything resembling a lever of power, it’s Owens.
unique link to this extract


Find out where Australia’s fuel shortage is most acute • Yahoo News Australia

Zac de Silva:

»

Each jurisdiction measures fuel shortages shortage differently but Energy Minister Chris Bowen has provided a state-by-state breakdown in federal parliament:
• As of Wednesday In NSW, 187 service stations had no diesel and 32 had no fuel at all
• In Queensland, 55 had no diesel and 35 had no regular unleaded
• In Victoria, 134 were out of at least one grade of fuel
• In South Australia, 49 stations were dealing with shortages
• In Western Australia, six are out of all fuel and four have no diesel
• In Tasmania, one service station has no diesel and six have no unleaded
• In the Northern Territory, there are no shortages because of the Middle East war, Mr Bowen said, but an ongoing flooding and cyclone disaster was affecting supplies
• In the ACT [Australian Central Territory], one service station was without diesel

How has the government responded?
• Fuel companies have been told to release more than 500 million litres of petrol and diesel from their emergency reserves, which is being directed towards regional areas where shortages are most acute
• Fuel quality standards for petrol and diesel have also been reduced, meaning lower-quality fuel usually reserved for export can also be used onshore

Government officials say they’re not considering rationing fuel at this stage, but have encouraged motorists to only take as much as they need, and no more.

«

This is a direct effect of the Iran conflict, and particularly the closure of the Strait of Hormuz. (Subbing note: it’s Strait, singular, not Straits. There’s only one navigational strait that goes part the island of Hormuz.)
unique link to this extract


Global PC shipments to decline 5% YoY in 2026 amid rising memory prices • Counterpoint Research

»

The market is witnessing a clear polarization in performance across major vendors:

Lenovo, HP and Dell: All three are expected to see mid-single-digit shipment declines. Notably, Dell is projected to record the mildest contraction among the three, buoyed by its high exposure to the commercial and premium segments, which typically exhibit higher price inelasticity.

Apple: Apple is uniquely positioned to turn this market volatility into a growth opportunity through strategic aggression:
• Entry-level disruption: By launching its first-ever $599 laptop (MacBook Neo), Apple is aggressively capturing demand in the budget and education sectors. This strategy aims to maximize long-term revenue from services and lock younger demographics into the Apple ecosystem at an earlier stage
• Portfolio premiumization: With the expected launch of its first OLED-based laptop in the second half of the year, Apple’s product portfolio will reach an unprecedented level of diversification and technical sophistication.

ASUS, Acer and Tier-2 brands: These vendors are forecast to suffer more pronounced declines. Their high exposure to low-end segments makes them vulnerable to price sensitivity. Furthermore, smaller procurement volumes may lead to weaker bargaining power, resulting in steeper cost hikes or supply chain marginalization during memory shortages.

Senior Analyst Minsoo Kang said, “The continuous surge in memory prices is forcing PC OEMs into a difficult choice – absorb margin compression or pass the costs on to consumers through price hikes. As major players are already implementing or planning price increases, an immediate softening of consumer demand is highly anticipated.

While higher prices may provide some ASP (average selling price) uplift, the projected 5% YoY drop in shipment volumes will result in revenue declines for most of the OEMs. However, around 40% of the current installed base still consists of Windows 10 or older systems, suggesting that a significant portion of devices still requires upgrades. Consequently, the PC market’s decline is likely to be less pronounced compared to other consumer electronics segments.

«

PC makers will be relieved that Microsoft is helping them out again. Though the smaller players are going to get squeezed for sure: this might be an extinction event for some of them.
unique link to this extract


The internet loves this band of lost dogs journeying home. Too bad the story is fake • CNN

Jessie Yeung:

»

By now, the video has been seen tens of millions of times. The internet is enamored; how can you not be?

The short clip shows a group of dogs in China who were purportedly captured to be eaten but escaped and made the long journey home as a merry band of misfits – including a golden retriever, an injured German shepherd, and a brave corgi leading the way.

The problem: It’s not real. Though the original clip is authentic, showing seven dogs wandering down the side of a highway in northeastern Jilin province, Chinese state media has since debunked the narrative of their escape and journey home.

…The phenomenon illustrates how misinformation can multiply after a viral moment, spreading what can seem to be harmless narratives that are made harder to verify in the age of AI. In this case, some of the false storylines included racist stereotypes.

Amid the doom and gloom of news coverage, audiences are hungry for wholesome feel-good content like animal videos.

They offer an escape but their popularity also encourages social media creators to invent or embellish content for clicks, said TJ Thomson, associate professor of digital media at RMIT University in Melbourne, Australia.

“Folks are trying to capitalize on existing viral content or trends,” he said. “Attention is money online and on social media. So, the more attention you get, the more engagement you get.”

…All of the dogs belonged to villagers who lived a few kilometers from the highway where they were filmed, according to the Chinese state-owned City Evening News, which tracked down the owners. The German shepherd had been in heat, which is why other dogs were drawn to it, the owners said.

Most dogs in the village were free-roaming and often disappeared for a day or two during their heat cycle, Cover News reported. The seven dogs in question have since returned home, with the German shepherd now restrained on a leash until its heat cycle ends.

«

Oh well. Look, it’s still a happy story – dogs some distance from home return home. It just doesn’t have the theft angle.
unique link to this extract


That’s all, for now • Asimov Press and Niko McCarty

Asimov Press:

»

Asimov Press is going on hiatus. While a few more articles will appear over the next month — and our stunning hardcover book, Making the Modern Laboratory, will come out this summer — operations will pause in April. All articles will remain freely accessible online.

As we prepare to pause, I want to celebrate what we’ve accomplished. In June 2023, Asimov Press was just a vague idea. (I remember posting the announcement from my phone in the Amsterdam metro, feeling a deep sense of fear about what we were starting.) A few months later, I brought on Xander Balwit. The two of us were its only editors and only full-time staff. We launched with about 7,000 subscribers, ported over from my personal blog, Codon. Our first article appeared in January 2024.

While I had attended journalism school and thought I knew about publishing, I soon discovered that literally everything — from managing people, to recruiting and paying writers, to fact-checking stories, printing books, and much else — doesn’t play out as it’s taught in school. By trial and error, Xander and I had to learn how to develop an eye for writers with exceptional promise, separate good pitches from vague ones, figure out which parts of a draft were most compelling, and make sure stories were told deeply enough to satisfy scientists, while remaining accessible to all “ambitious” readers.

…We’ve come a long way since our inception. Today, we have about 42,000 subscribers between Substack and our website. The press has published 149 original articles, reaching about half a million readers each month.

…Our decision to pause is not related to funding. Asimov has supported us for the last two years, and we’ve received generous grants from Astera Institute and Stripe. But new projects have called away Xander and I, making it a good time to recalibrate.

«

A relief that this is not about funding, because Asimov really has been a terrific entrant to the science publishing space (using Substack under the hood). Let’s hope the pause is not too long.
unique link to this extract


OpenAI puts erotic chatbot plans on hold ‘indefinitely’ • Financial Times

Cristina Criddle and Stephen Morris:

»

OpenAI has shelved plans to release an erotic chatbot “indefinitely” as it refocuses on its core products, following concerns from staff and investors about the effect of sexualised AI content on society.

Sam Altman’s start-up had already delayed the release of its “adult mode” amid internal discussions over whether to scrap the model entirely, according to multiple people familiar.

The sexual chatbot faced growing pushback over how it could encourage unhealthy attachments to AI systems and expose minors to problematic sexual content.

The erotic model was on hold with no timeline for its release, OpenAI confirmed. The $730bn AI lab said it wanted to have long-term research on the effects of sexually explicit chats and emotional attachments, as part of product development, acknowledging there was no “empirical evidence” at present.

It marked the latest decision by OpenAI to drop what executives have termed “side quests” in favour of devoting resources to productivity tools, bringing together products such as coding assistants and ChatGPT into one “super app”.

OpenAI on Tuesday said it was winding down its Sora video generation model and social app.

«

Finally, some good news! Though the idea of OpenAI trying to create a superapp seems quixotic. People in the west simply don’t want Swiss Army knives; they prefer the individual tools.

More generally, OpenAI is definitely pulling its horns in. Perhaps the first sign of winter.
unique link to this extract


• Why do social networks drive us a little mad?
• Why does angry content seem to dominate what we see?
• How much of a role do algorithms play in affecting what we see and do online?
• What can we do about it?
• Did Facebook have any inkling of what was coming in Myanmar in 2016?

Read Social Warming, my latest book, and find answers – and more.


Errata, corrigenda and ai no corrida: none notified

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.