Start Up No.2230: Google scrambles to clean up AI search, the stolen phone text scammers, life as an OnlyFans chatter, and more


A number of hotels in the Wyndham chain turn out to have had spyware on their checkin computers – making guest details visible to hackers. CC-licensed photo by Greg Grimes on Flickr.

You can sign up to receive each day’s Start Up post by email. You’ll need to click a confirmation link, so no spam.

A selection of 10 links for you. Which credit card? I’m @charlesarthur on Twitter. On Threads: charles_arthur. On Mastodon: https://newsie.social/@charlesarthur. Observations and links welcome.


Google scrambles to manually remove weird AI answers in search • The Verge

Kylie Robison:

»

Social media is abuzz with examples of Google’s new AI Overview product saying weird stuff, from telling users to put glue on their pizza to suggesting they eat rocks. The messy rollout means Google is racing to manually disable AI Overviews for specific searches as various memes get posted, which is why users are seeing so many of them disappear shortly after being posted to social networks.

It’s an odd situation, since Google has been testing AI Overviews for a year now — the feature launched in beta in May 2023 as the Search Generative Experience — and CEO Sundar Pichai has said the company served over a billion queries in that time.

But Pichai has also said that Google’s brought the cost of delivering AI answers down by 80% over that same time, “driven by hardware, engineering and technical breakthroughs.” It appears that kind of optimization might have happened too early, before the tech was ready.

“A company once known for being at the cutting edge and shipping high-quality stuff is now known for low-quality output that’s getting meme’d,” one AI founder, who wished to remain anonymous, told The Verge.

Google continues to say that its AI Overview product largely outputs “high quality information” to users. “Many of the examples we’ve seen have been uncommon queries, and we’ve also seen examples that were doctored or that we couldn’t reproduce,” Google spokesperson Meghann Farnsworth said in an email to The Verge. Farnsworth also confirmed that the company is “taking swift action” to remove AI Overviews on certain queries “where appropriate under our content policies, and using these examples to develop broader improvements to our systems, some of which have already started to roll out.”

Gary Marcus, an AI expert and an emeritus professor of neural science at New York University, told The Verge that a lot of AI companies are “selling dreams” that this tech will go from 80% correct to 100%. Achieving the initial 80% is relatively straightforward since it involves approximating a large amount of human data, Marcus said, but the final 20% is extremely challenging. In fact, Marcus thinks that last 20% might be the hardest thing of all.

«

Two points: note that Google doesn’t say how the cost of delivering AI answers compares to normal searches; and every day 15% of queries are completely novel, which amounts to millions every day. Google will never succeed at this.
unique link to this extract


The Elmer’s Glue pizza error is more fascinating than you think • The End(s) of Argument

Mike Caulfield:

»

Since intent can’t always be accurately read, a good search result for an ambiguous query will often be a bit of a buffet, pulling a mix of search results from different topical domains. When I say, for example, “why do people like pancakes but hate waffles” I’m going to get two sorts of results. First, I’ll get a long string of conversations where the internet debates the virtues of pancakes vs. waffles. Second, I’ll get links and discussion about a famous Twitter post about the the hopelessness of conversation on Twitter.

For an ambiguous query, this is a good result set. If you see a bit of each in the first 10 blue links you can choose your own adventure here. It’s hard for Google to know what matches your intent, but it’s trivially easy for you to spot things that match your intent. In fact, the wider apart the nature of the items, the easier it is to spot the context that applies to you. So a good result set will have a majority of results for what you’re probably asking and a couple results for things you might be asking, and you get to choose your path.

Likewise, when you put in terms about cheese sliding off pizza, Google could restrict the returned results to recipe sites, advice which would be relatively glue-free. But maybe you want advice, or maybe you want to see discussion. Maybe you want to see jokes. Maybe you are looking for a very specific post you read, and not looking to solve an issue at all, in which case you just want relevance completely determined by closeness of word match. Maybe you’re looking for a movie scene you remember about cheese sliding off pizza.

In the beforetimes, it would be hard to imagine a user getting upset that the cheese sliding query pulls up a joke on Reddit as well as some recipe tips. The user can spot, quite easily, that these are two different sorts of things.

The problem comes when the results get synthesized into a common answer. To some extent, this is a form of “context collapse”, where the different use contexts (jokes, movies, recipes, whatever) get blended into a single context.

«

unique link to this extract


Things the guys who have my stolen phone have texted me to try to get me to unlock it • Read Max

Veronica de Souza:

»

My phone was stolen in early March, most likely while I was standing on the platform at the York Street station waiting for the F train. Fifteen minutes later, in a food hall under the movie theater where my boyfriend and I were supposed to see Dune 2, I reached into my pocket and realized it was gone. Josh looked at the “Find My” app on his phone: My phone was “last seen” at York Street but wasn’t registering a current location. Someone had turned it off.

As quickly as possible, I did all the things you’re supposed to do when your phone is lost or stolen–mark it as lost, cut off service, and remotely erase it–and spent the rest of the night anxiously refreshing the Find My App, watching my phone move around Manhattan before finally coming to a stop at Rockefeller Center. I didn’t bother confronting the thief.

Worst of all, we didn’t even see Dune 2.

After two hours in the Williamsburg Apple Store the next morning, I had a new iPhone 15 and I stopped stressing. As long as I didn’t remove the phone from my Apple account or the “Find My” app, the phone was essentially bricked to anyone without the passcode–unusable by the thieves, or the fences who I assume bought it from them. Now my phone was their problem.

…Here are some things the guys who stole (or later purchased) my iPhone have told me to try to get me to unlock it:

1) “Your iPhone 14 Pro is trying to pay with Apple pay in China.”

«

And that’s only the beginning. The fight between scammers and intelligent people who recognise scammers is always interesting.
unique link to this extract


Spyware found on US hotel check-in computers • TechCrunch

Zack Whittaker:

»

A consumer-grade spyware app has been found running on the check-in systems of at least three Wyndham hotels across the United States, TechCrunch has learned.

The app, called pcTattletale, stealthily and continually captured screenshots of the hotel booking systems, which contained guest details and customer information. Thanks to a security flaw in the spyware, these screenshots are available to anyone on the internet, not just the spyware’s intended users. 

This is the most recent example of consumer-grade spyware exposing sensitive information because of a security flaw in the spyware itself. It’s also the second known time that pcTattletale has exposed screenshots of the devices on which the app is installed. Several other spyware apps in recent years had security bugs or misconfigurations that exposed the private and personal data of unwitting device owners, in some cases prompting action by government regulators.

pcTattletale allows whomever controls it to remotely view the target’s Android or Windows device and its data, from anywhere in the world. pcTattletale’s website says the app “runs invisibly in the background on their workstations and can not be detected.”

But the bug means that anyone on the internet who understands how the security flaw works can download the screenshots captured by the spyware directly from pcTattletale’s servers. 

…It’s not known who planted the app or how the app was planted — for example, if hotel employees were tricked into installing it, or if the hotel owner intended the spyware to be used to monitor employee behavior.

«

unique link to this extract


Media companies are making a huge mistake with AI • The Atlantic

Jessica Lessin:

»

For as long as I have reported on internet companies, I have watched news leaders try to bend their businesses to the will of Apple, Google, Meta, and more. Chasing tech’s distribution and cash, news firms strike deals to try to ride out the next digital wave. They make concessions to platforms that attempt to take all of the audience (and trust) that great journalism attracts, without ever having to do the complicated and expensive work of the journalism itself. And it never, ever works as planned.

Publishers like News Corp did it with Apple and the iPad, investing huge sums in flashy content that didn’t make them any money but helped Apple sell more hardware. They took payouts from Google to offer their journalism for free through search, only to find that it eroded their subscription businesses. They lined up to produce original video shows for Facebook and to reformat their articles to work well in its new app. Then the social-media company canceled the shows and the app. Many news organizations went out of business.

The Wall Street Journal recently laid off staffers who were part of a Google-funded program to get journalists to post to YouTube channels when the funding for the program dried up. And still, just as the news business is entering a death spiral, these publishers are making all the same mistakes, and more, with AI.

Publishers are deep in negotiations with tech firms such as OpenAI to sell their journalism as training for the companies’ models. It turns out that accurate, well-written news is one of the most valuable sources for these models, which have been hoovering up humans’ intellectual output without permission. These AI platforms need timely news and facts to get consumers to trust them. And now, facing the threat of lawsuits, they are pursuing business deals to absolve them of the theft. These deals amount to settling without litigation. The publishers willing to roll over this way aren’t just failing to defend their own intellectual property—they are also trading their own hard-earned credibility for a little cash from the companies that are simultaneously undervaluing them and building products quite clearly intended to replace them.

«

Lessin is a founder of The Information, which charges as much – more? – than most national papers for its paywall. And is going from strength to strength, unlike a lot of media. But not everyone can do that.
unique link to this extract


I went undercover as a secret OnlyFans chatter. It wasn’t pretty • WIRED

Brendan Koerner:

»

To seal the deal, I needed to pass an elaborate written test. Daniel sent me a biographical sketch for a fictional “adult influencer from Tokyo” named Miko; she was a fan of karate, green tea, and the tongue emoji. My assignment was to write four extended back-and-forth dialogs between Miko and a hypothetical subscriber—two had to involve X-rated material, while the other two were meant to be clean. “Each bot’s reply should contain a call to action, a question, a compliment, or an inspiration to do something,” the instructions dictated, though I was forbidden from using question marks in more than 20% of Miko’s responses.

I found it quite easy at first to write the sort of run-of-the-mill smut the Serbs expected. (I’ll spare you the gory details, except to say I cribbed some color from Kathryn Bigelow’s 1995 sci-fi film Strange Days.) For the less explicit chats, I imagined Miko offering to cook the subscriber a pasta dinner and feigning appreciation for his TV recommendations. I did make one glaring error that could have led to an entire chat being voided as unusable: Due to my hasty misreading of Miko’s bio, I characterized her as a fan of spicy ramen when she actually prefers her food mild. “I have to ask you to pay attention to these little facts,” Daniel wrote in his assessment. “In this case, these lines mentioning the food could have been rejected, and that could have led to the dialog’s rejection.”

But despite that mistake and a few other hiccups—my punctuation seemed unnatural because it was too accurate—Daniel offered me the job. I was to be paid 7 cents per line of dialog, with each dialog running for a minimum of 40 lines. For my first assignment, I had to compose 20 dialogs involving sex in public places—10 at the beach, five inside a car, and five in a forest or garden. There was a list of particular sex acts I had to include, as well as a stricture that I refrain from using emoji in more than 30% of lines. I had only 48 hours to complete the task.

By the time I wrapped up my fifth dialog, my brain was a puddle of goo.

«

It turns out there’s an enormous hinterland of this stuff – agents and hiring and recruitment and levels and schemes – but when you weigh it up, all nonsense. So much human effort, for what?
unique link to this extract


Millennials are waving away privacy concerns and eyeing Chinese cars • Quartz

William Gavin:

»

The US Department of Commerce in February announced an investigation into Chinese-made smart cars, noting that EVs and autonomous vehicles collect massive amounts of data.

About 44% of all respondents [to a survey] said they would be very concerned about their privacy if Chinese cars were sold in the US, with another 34% saying they would be somewhat concerned, according to AutoPacific. But that won’t stop Americans under 40 from considering buying one of those vehicles, despite 73% saying they would be concerned about their privacy.

“Privacy concerns about Chinese-brand vehicles are likely to eventually subside given that most of the connected smartphones, smart watches, laptops, connected home devices we are comfortable using every day are in fact manufactured in China,” [chief analyst and president of research firm AutoPacific, Ed] Kim said.

Chinese cars — especially EVs — have a major leg up on Western competitors. Many Chinese automakers — thanks to a combination of cheap manufacturing and state subsidies — are able to sell their cars for dirt-cheap.

BYD, for example, sells the Seagull hatchback car for less than $10,000. China is also home to the cheapest electric car in the world, the roughly $1,000 Changli Freeman. Chinese cars also come with impressive technology and features that consumers demand, from better driver assistance tech to free fridges and portable rooms.

“Younger generations of shoppers are clearly aware of the enticing products Chinese automakers are cooking up overseas,” Robby DeGraff, AutoPacific’s manager of product and consumer insights, said in a statement. “It’s only a matter of ‘when’ they’ll be able to get their hands on them.”

«

unique link to this extract


Who actually uses Instagram’s Threads app? Taiwanese protestors • Rest of World

Viola Zhou:

»

As thousands of people gathered outside Taiwan’s legislature on Tuesday to protest against a bill that would give more power to China-friendly parties, Yuan, who was volunteering at a nearby church, noticed that the large crowd was running short on supplies. 

He fired off posts on the Threads app listing items that protesters needed, such as snacks, bottled water, and plastic bags. Supplies arrived within minutes. 

“My Threads page was like a wishing well,” Yuan, who requested to be identified with part of his first name for privacy reasons, told Rest of World. “We got everything we asked for.” 

A 32-year-old bar owner in Taipei, Yuan has been lurking on Threads since Meta launched the Instagram-linked alternative to X last year. He posted on the app for the first time last weekend to help organize a protest against the island’s opposition lawmakers. His posts about the protests have been “liked” thousands of times. 

Threads, which had 150 million monthly active users globally by April, is doing exceptionally well in Taiwan, where it’s commonly loosely transliterated as cui — because the “th” sound doesn’t appear in Mandarin. It works like X, allowing users to post 500-character-long text posts as well as audio, photos and short videos. Despite its small population of 23 million, Taiwan had 1.88 million active users on Threads from May 5 to 11, behind only the U.S., Japan, and Brazil, according to app-tracking site Data.ai. 

«

unique link to this extract


Apple’s WWDC may include AI-generated emoji and an OpenAI partnership • The Verge

Wes Davis:

»

Apple will finally tell its own AI story at WWDC 2024, but it may not mean the sorts of showy features demoed by the likes of Google, Microsoft, or OpenAI. Instead, the event may see Apple rolling out basic AI features like transcribing voice memos or auto-generated emoji — and announcing a rumored partnership with OpenAI, according to Mark Gurman’s Power On newsletter for Bloomberg today.

…The Voice Memo app could also get a big boost in AI-generated transcripts, Gurman writes. Selfishly, that will be key for referring to interview recordings, but it could also be handy for, say, students recording their lessons for later reference. Apple devices have similar features already, like auto-generated voicemail transcripts and system-wide captions for videos, audio, and conversations.

The company also reportedly plans to announce AI-powered improvements to on-device Spotlight search, internet searches with Safari, as well as writing suggestions for emails and texts. And the company may also use AI to retouch photos and generate emoji on the fly, based on what you’re texting — a type of feature that seems to consistently lead to trouble for these companies. (See Meta’s gun-toting Waluigi AI stickers or Google’s inappropriately racially diverse nazi pictures.)

Apple could showcase a better, more natural-sounding voice for Siri, based on Apple’s own large language models, as well as better Siri functionality on the Apple Watch.

«

unique link to this extract


California can make climate polluters pay for the mess they have made of Earth • Los Angeles Times

»

As climate change exacts a mounting toll on California, who should pay for the damage from rising sea levels and increasingly ferocious wildfires, floods and heat waves?

Fossil fuel companies would like taxpayers to keep footing the bill while they reap the profits from the burning of coal, oil and methane gas. That’s not right. Companies whose products are responsible for the vast majority of the greenhouse gas emissions should be held liable for the costs.

That’s the concept behind the Polluters Pay Climate Cost Recovery Act, a bill in the California Legislature that would create a Superfund-style program to collect money from major fossil fuel companies such as Chevron and ExxonMobil to help the state pay for the environmental damage caused by their products.

Other states, including New York, Massachusetts and Maryland, are considering similar “Climate Superfund” bills. They’re modeled on the 1980 federal Superfund law that established an industry-funded trust fund to pay for cleaning abandoned hazardous waste sites and holds current and past operators and other responsible parties liable for the costs. California would not be the first to act; Vermont lawmakers earlier this month sent a Climate Superfund bill to their governor’s desk.

…The fund would come from the estimated 41 fossil fuel extraction and refining companies that meet the bill’s threshold of being responsible for more than 1 billion metric tons of emissions from 2000 to 2020. Each would pay a share of the climate costs to the state based on a study to be conducted by the California Environmental Protection Agency. The total cost recovery could amount to tens or even hundreds of billions of dollars, according to legislative analysts, and could be paid in installments over 20 years.

«

You would feel fairly sure that the monies raised wouldn’t really go into climate amelioration – though taxes are rarely ringfenced to their purposed. This tax is overdue, to be honest. (This was an Editorial opinion in the paper, hence no byline.)
unique link to this extract


• Why do social networks drive us a little mad?
• Why does angry content seem to dominate what we see?
• How much of a role do algorithms play in affecting what we see and do online?
• What can we do about it?
• Did Facebook have any inkling of what was coming in Myanmar in 2016?

Read Social Warming, my latest book, and find answers – and more.


Errata, corrigenda and ai no corrida: none notified

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.