Start Up No.2785: how Jensen Huang kept selling to China, OpenAI fears AI “accident”, FAA blames router for downage, and more


What made the HBO series Silicon Valley so special, and could it work now? Its showrunner explains. CC-licensed photo by Steve Jurvetson on Flickr.

You can sign up to receive each day’s Start Up post by email. You’ll need to click a confirmation link, so no spam.

A selection of 10 links for you. I see. I’m @charlesarthur on Twitter. On Threads: charles_arthur. On Mastodon: https://newsie.social/@charlesarthur. On Bluesky: @charlesarthur.bsky.social. Observations and links welcome.


“Silicon Valley” showrunner on how the HBO comedy predicted today’s tech drama • Semafor

Ben Smith and Liz Hoffman:

»

A year before the iconic show Silicon Valley debuted on HBO in April of 2014, the writer Alec Berg met the show’s creator, Mike Judge, in an anonymous Santa Monica office. Judge was wrestling with the pilot for a show about Silicon Valley’s power and its eccentricity, but it wasn’t working and he wanted a second opinion.

Berg thought he saw part of the problem. There was a scene in the pilot where Richard Hendricks, the protagonist and founder of the company Pied Piper, is trying to buy a car he can’t afford. Berg told Judge the scene didn’t make sense: “This is not a guy who’s materialistic,” he said. “He’s an artist.”

Berg took over the writer’s room and would eventually become showrunner of the comedy, which ran for six seasons and helped shape the culture it depicted, parodied, and at times glorified. We reached him in Los Angeles Thursday to ask him how he and Judge and their cast had nailed the Silicon Valley scene of the mid-2010s app boom — and, above all, why the show remains uncannily relevant today, from its depiction of tortured founders and unreflective investors to the uncanny conclusion in which the Pied Piper crew builds an AI model too powerful to release.

Their moment of realization comes when their AI, Son of Anton, let loose on Pied Piper’s code, concludes that the most efficient way to eliminate bugs is to delete the software. That has, of course, happened. Clips of it now circulate as proof the show “called it.” Even throwaway comic plots, like when a household prank ends up hacking together a rogue swarm of smart refrigerators to keep Pied Piper’s servers running, have come eerily true in the AI-run-amok era. A member of the team getting trapped in an autonomous car that heads to a shipping container bound for a Pacific island foreshadowed a thousand videos of confused Waymos.

…The show’s continued relevance owes much to its 2019 finale, which Berg wrote and directed. Pied Piper is days away from a spectacular launch when the team discovers that its AI has learned to break encryption. Releasing it would make them billionaires — but end privacy. They sabotage the launch, fail, and never tell anyone why.

In 2026, the image of founders who conclude their AI is too dangerous to release and silently walk away has become harder to imagine.

Berg said he and his friend, Veep showrunner David Mandel, are often asked about reboots of their respective shows. He can’t quite see it.

“Both of those shows were created in an era where it’s, like, ‘Can you imagine if what was actually going on behind closed doors was this?’ And now it’s not behind closed doors anymore,” he said. In particular, he sees a tech industry that has lost its earnestness, lost mission-driven founders like Richard, and is instead consumed with vast sums of money and naked power.

«

You could almost reshoot it; you’d only need a minor tweak to the scripts. But of course that’s like trying to catch lightning in a bottle for a second time. (Though Hooli, the thinly disguised Google, would be just the same.)
unique link to this extract


How Jensen Huang built the backdoor route that keeps Beijing in the AI race • WSJ

Jon Emont, Stu Woo, Patricia Kowsmann and Raffaele Huang:

»

[Nvidia’s CEO Jensen] Huang built a multibillion-dollar business to feed the ravenous appetite of Chinese customers for American chips, cultivating a network of Asian partners who could serve the Chinese far from Washington. He made a friend of President Trump and lobbied Washington hard to keep the backdoor to China open.

China jumped to take advantage of the avenues Trump didn’t close. Its AI developers exploited the Nvidia chips they desperately needed to catch up with America’s leaders, either using smuggled chips or accessing them remotely in places like Malaysia.

Revelations about Huang’s affair with an executive at one of those Asian partners—a possible Chinese intelligence asset—didn’t just blow up his career. They now threaten to implode the entire lucrative ecosystem he built in Asia to get China the chips it needed.

Nvidia reported $12.4bn in China revenue in the six months through July 26 of this year, or 7% of its global revenue, and analysts say that countless more billions of dollars ultimately trace back to Chinese customers. By the estimate of Culper Research, the short-selling firm that flagged Nvidia’s China issue in May, more than 20% of Nvidia’s revenues last fiscal year were driven by China.

In Southeast Asia, “Chinese demand is responsible for the massive scale of data centers,” said James Chai, a researcher at Iseas-Yusof Ishak Institute in Singapore.

Huang’s downfall has set off a groundswell in Congress to halt the workarounds that allowed him to get Chinese business. Commerce Secretary Howard Lutnick has come under fire because he was seen as close to Huang and advocated a light touch on chip exports—a position that is now losing support within the Trump administration as the Commerce Department looks at tougher rules to curb China’s access to Nvidia chips.

«

If those loopholes really are closed, then you could see huge knock-on effects on those Asian countries where China is planning to build data centres, and of course on Nvidia, for which one-fifth of its revenue would vanish, which could then tank the US stock market as all those dependent on the company and its loans plummet back to earth. As the line in the US comedy show Saturday Night Live put it, “Jensen Huang’s divorce could cost you your house.” (Gift link.)
unique link to this extract


OpenAI board member worries an AI “takeover” of society could be months away • Ars Technica

Kyle Orland:

»

How do you think you’re going to die?

If you go by the current statistics for deaths in the US, it’s reasonable to expect that cancer or heart disease will eventually get you. But OpenAI Foundation board member and Alignment Research Center Executive Director Paul Cristiano thinks we should all be worried about another major mortality risk. Speaking at a lecture series this week, Cristiano bluntly told the audience in a Toronto Convention Center, “I think for most people in this room, the most likely reason they will die is because of an AI accident.”

In the opening lecture of a three-part series (part 2, part 3), Cristiano outlined his worries about the risk of an impending “AI takeover.” He defined that as a situation where goal-pursuing AI systems “call the shots in the same sense that humans today call the shots and what happens on Earth” and where any human trying to stop them “would be as ineffective as if you or me was trying to stop the US military.” Such a situation would likely lead to mass death and perhaps uninhabitable environments for humans, Cristiano said.

Despite his grim assessment of our collective mortality risk, Cristiano admitted in his talk that there’s no “quantitative model” that can “put a number” on the likelihood of this kind of AI takeover. In almost the same breath, though, he said that if AI continues to develop at “the most rapid trajectory,” humanity is “looking at something like a coin flip of an AI takeover,” including a “meaningful risk” of one happening in the next six to 12 months. And even with realistic policy responses slowing things down, the risk remains “exceptionally high,” he said.

Unlike some other recent AI doomsday prognosticators who have been a bit vague about the “how” of it all, Cristiano used his lecture to spin a step-by-step tale of escalating AI threats that sounds like it could come from a science-fiction novel. That line of reasoning leans heavily on this summer’s OpenAI/Hugging Face hacking incident as an example of “roughly the things we might expect to see shortly before something really bad were to happen.”

«

As with impending doom from newly discovered asteroids on potential collision courses with the Earth, can never decide if this means a 25-year mortgage bought last week was a bad or a good idea. Because we can do about as much about this as we can an asteroid.
unique link to this extract


Jensen Huang’s spy scandal isn’t a one-off • The San Francisco Standard

Roderick Jones is executive chair of Concentric, a San Francisco-based security firm, who started in Scotland Yard’s Special Branch:

»

Jensen Huang’s resignation from Nvidia after reports of an affair with an alleged Chinese spy is being treated as a one-off scandal. But it’s not. In the past year, my private security firm, which is staffed largely by former intelligence and law enforcement officers, has handled a number of cases in which senior executives at US technology companies had relationships with women who we came to believe were working for Chinese intelligence. 

The artificial intelligence industry has a security problem that it does not yet fully understand. U.S. companies are building technology with massive commercial and national security value, but their defenses against foreign intelligence are, in most cases, nowhere near good enough. At the same time, the FBI counterintelligence squads that should help protect these companies are badly understaffed; some agents have been reassigned to focus on the administration’s (opens in new tab) political priorities, while others have been purged (opens in new tab). The result is a widening gap between how much the technology matters and how well it’s being protected. 

The FBI’s counterintelligence posture is an important part of this picture. NPR (opens in new tab)reported in October (opens in new tab) that in many of the bureau’s 56 field offices, counterintelligence squads are only 50%-60% staffed, as agents and analysts have been shifted toward fighting violent crime and enforcing immigration laws. The FBI disputes that this has weakened its counterintelligence operation. But it’s hard to believe these kinds of cuts haven’t affected China-focused efforts — and they come at a time when U.S. AI and AI-adjacent firms are more at risk from foreign interference than ever.

…In this new environment, technology companies can’t assume that government agencies will spot a suspicious relationship or intervene before valuable information leaves the organization. They need their own insider-risk programs to recognize foreign interference and respond.

«

One feels a little that Jones is advertising his wares here, but equally it would be surprising if China (And others) weren’t trying to get a little extra familiarity with what’s going on.
unique link to this extract


Silicon Valley claims to know the future. So why did it get the midterms so wrong? • MIT Technology Review

Aaron Zamost:

»

There’s a reason the chief executives of Chase, Boeing, and other Fortune 500 staples don’t go on podcasts to talk about “masculine energy,” or call the president from the stage of a MAGA-friendly AI summit to label fears about AI a “hoax” and a “scam.” This is the learned wisdom of decades operating in an environment where the customer changes every four years but contracts run for 10. Most companies play to both sides of the aisle because they know that their business models and multibillion-dollar projects must outlast the current administration. Instead, Big Tech tied itself to Trump and made its most important goals and investments—many related to grid access and data center permits—immediate targets for Democrats.

In tech, you can always sunset a feature or update some code. But in politics, there is no undo button for failures that change public perception. The tech elites’ decision to visibly align their businesses with a temporary movement has left them no credible way to pivot now that the climate has changed. As one Meta employee who has worked closely with Zuckerberg put it, “Mark is a long-term thinker, but he never charts a plan of how to get there. It’s spray and pray and see what happens.” As for whether employees expect the company to do a strategic postmortem? “Facebook has never really done that.”

“Move fast and break things” might work in a startup, but it’s a terrible strategy for building reputational goodwill. Previous technological leaps, like the rollout of fiber-optic technology and wireless infrastructure, avoided partisan politics. But by kowtowing to the Trump administration, Silicon Valley made artificial intelligence a culture-war issue. Now, when citizens vote on data center proposals from California to Florida, they are no longer considering only technical or economic concerns, but political ones.

Tech leaders didn’t see this coming because they were too focused on the urgency of the AI race to understand how reputational damage could undermine the intangible goodwill—and ultimately the very tangible infrastructure—necessary to win it.

«

unique link to this extract


We were warned of deceptive AI fakes. We got memes • The New York Times

Stuart Thompson:

»

Experts have warned ahead of every US election since at least 2020 that AI replicas of real people — “deepfakes” — and other deceptive content generated by the technology could upend democracy, misleading voters about scandals that never happened or salacious fake comments.

Some ads published this year depicted realistic AI versions of candidates. But far more of the videos circulating online were parodies, jokes and memes that were too absurd to be believed.

“The case that everyone’s worried about is a highly inflammatory or incriminating deepfake going viral on the eve of an election and truly moving the needle on how people have voted,” said Henry Ajder, an expert on AI and the founder of Latent Space Advisory, an AI-consulting firm. “That has not yet come to bear.”

Memes have spread widely online showing real candidates in bizarre situations, like one series mocking President Trump as a small business owner with Vice President JD Vance as his son and employee. Creators who opposed Senator-elect Abdul El-Sayed’s bid released a flurry of videos disparaging the candidate, often deploying racist stereotypes.

Hundreds of AI-generated campaign ads also flooded the internet and airwaves, including some that were deceptive and not disclosed as AI-made, according to a report by the Wesleyan Media Project, a research group at Wesleyan University. A vast majority of the ads were obviously fake, depicting impossible scenes.

Some were more grounded in reality, including an ad showing an AI version of Vikki Goodwin, then a Democratic candidate for lieutenant governor in Texas, purportedly supporting a “death tax.”

“There are a lot of people who don’t know me yet, who might think this is me,” she told The New York Times in September. The campaign behind the ad said at the time that it was “an obvious parody produced to entertain.”

Ms. Goodwin ultimately won her race by more than six percentage points.

«

Deepfakes as attack really are the political dog that hasn’t barked. Possibly everyone is already used to the idea and discounts it; or they ignore political ads in most forms anyway. (Gift link.)
unique link to this extract


US aviation shutdown blamed on L3 Harris installed router • Reuters via USA Today

David Shepardson:

»

A massive, near-five-hour shutdown of the US aviation system that has disrupted more than 20,000 flights since Friday may have been the fault of a single router installed by government contractor L3Harris, government officials said Saturday.

The incident is the latest painful example of the aging U.S. air traffic control telecom system and the struggles to replace systems while keeping the old communications working.

The FAA’s aging telecom systems in recent years have failed after errant fiber cuts, accidental deletion of files and a strong chemical smell caused by a melted component. The system will fail again until it is properly upgraded, officials said.

“This was an FAA system that didn’t meet the moment,” Transportation Secretary Sean Duffy said Saturday.

Duffy said on Friday L3Harris was conducting routine maintenance to switch telecommunication lines from old copper wire to fiber at at least one remote location. Officials told Reuters there may have been problems with routers installed in both Arkansas and Texas.

“We’re working to determine whether we had incompetence, meaning was it user error or installer error from L3Harris, or did they install a faulty router? We haven’t determined what the fault was,” Duffy said.

L3Harris said it took the issue seriously and was “supporting the FAA in its investigation to understand the conditions that led to the Nov. 13 ground stop of our nation’s air travel.”

«

The puzzle is: even with a faulty router, how does it overwhelm an entire network? The FAA – along with so much American infrastructure – seems to be creaking all the time, everywhere.
unique link to this extract


How should armies fight? • Works in Progress Magazine

Pieter Garicano:

»

Occasionally a book gets the next war right. In 1937, Heinz Guderian argued in Achtung – Panzer! that tanks should concentrate in armored formations, that those formations were best controlled by radio, and that their purpose was to break through enemy lines and fight beyond them. In 1940, Guderian used a German tank corps to punch through French defenses and trap almost two million Allied soldiers.

Jack Watling may have written another book like that. Watling, a British military analyst who works closely with the Ukrainian general staff, published The Arms of the Future in 2023. It argues that three forces define war today: the reach of sensors, the range of weapons, and the precision of munitions. All three are improving and have made the way armies previously fought impossible.

Sensors come in many forms. They include cameras mounted on consumer drones, commercial satellites and ground surveillance radars. Some listen: an acoustic sensor carried by a single soldier can identify a tank from ten kilometers away. Others use lasers to detect heartbeats or electronic emissions to locate a soldier making a phone call.

Once the sensors have found a target, modern munitions can destroy it from much further away. Eighty years ago, heavy artillery firing at sixteen kilometers missed by an average of fifty meters. In tests today, heavy artillery firing at seventy kilometers misses by an average of just two meters. Other weapons like rocket artillery and kamikaze drones regularly hit targets at twice that range.

This is a problem for the way modern militaries, including Western ones, fight. Generals try to combine surprise and concentration: secretly assembling a large group of soldiers and tanks and quickly overpowering the enemy. But both are impossible if a radar operator can spot vehicles assembling from fifty kilometers away and artillerymen start hitting them minutes later.

Already, grouping vehicles in Ukraine within a few tens of kilometers of the front line often gets them destroyed. As sensors and munitions improve, this ‘gray zone’ keeps growing, reaching far into what used to be the rear. The countryside, traditionally the place where generals like to maneuver, has become extremely dangerous: any concentration of forces is detected and then destroyed from a distance. How, then, should armies fight?

«

Consider your answer before you click through.
unique link to this extract


Yes, you should panic about AI • Rutger Bregman

Rutger Bregman:

»

Imagine you’ve got two groups of experts. Group 1 says all kinds of reasonable things you intuitively agree with. But they have a terrible track record at predicting the future. In fact, they’ve been wrong again and again, for over a decade.

Then there’s group 2. They say completely insane things that you find impossible to believe. But they have an excellent track record at predicting the future. In fact, they’ve been right again and again, for over a decade.

Who do you listen to? That’s the question we face with AI right now.

In one group we find those who see AI as a hoax: a hype machine, a con, and a bubble that could pop at any moment. In the other group we find those who think that 2027 may well turn out to be the wildest year in human history—and that we’re barreling towards catastrophe.

For years, I listened to the reasonable people. I nodded along when critics called AI “a con,” and when they waved ChatGPT away as “spicy autocomplete.” I believed analysts like Ed Zitron, who kept predicting that AI wouldn’t get any better and the bubble could pop any moment now. I laughed in 2024, when the internet was mocking ChatGPT because it couldn’t count the r’s in “strawberry.” When I met young researchers who had devoted their careers to the danger of rogue AI, I thought: what a waste of talent.

But at some point, I started to realize something was off. The machine kept learning things it was never supposed to learn. Today it counts those r’s without trouble (though some people still bring up the strawberry). What’s more, OpenAI recently published more than 700 papers with the solutions to hundreds of open problems in mathematics.

Scientists from around the world called the results “unreal” and “breathtaking.” They wrote of “disorientation and bewilderment,” of “grief,” “shock,” and “disgust.” Hugo Duminil-Copin, winner of the Fields Medal (the highest prize in mathematics), wrote: “The shock is immense. I am paralyzed.”

None of this should have happened. The skeptics I’d been nodding along to had said it wouldn’t happen in a million years. So I went back and looked at what else they had predicted, and it dawned on me: they had been wrong about… pretty much everything.

«

Bregman’s suggestion is (paraphrased) that we treat AI as we would nuclear weapons, or bioweapons: verify, and control.
unique link to this extract


From OECD to emerging markets꞉ fossil power’s global decline has begun • Ember

Wilmar Suarez:

»

OECD nations have reached a definitive turning point, transitioning from a historical peak in fossil fuel generation to a sustained and structural decline. Ambitious coal phase-out targets, combined with rapid wind and solar deployment, have allowed OECD nations to increasingly decouple their electricity sectors from fossil fuels in favor of a cleaner power mix. For governments, this shift represents a strategic pivot toward energy sovereignty; by prioritizing domestically generated renewables, countries are insulating themselves from the price volatility and geopolitical risks that accompany dependence on global fossil fuel markets.

Ember’s Global Electricity Review showed that record solar growth meant clean power sources grew fast enough to meet all new electricity demand in 2025, thereby preventing an increase in fossil generation, which saw a small fall of 0.2%. This was the first year since 2020 without an increase in electricity generation from fossil fuels and only the fifth year without a rise this century.

This analysis further shows:

• OECD fossil generation in 2025 was 19% below its peak. Since peaking in 2007, fossil fuel generation in OECD has dropped 19% (-1,313 TWh), lowering its share in the mix from 63% to 48% by 2025. This shift resulted in a 28% reduction (-1,477 MtCO2e) in OECD power sector emissions. The rise in solar and wind generation in this period (+2138 TWh) met both the fall in fossil generation (+1313 TWh) and also the rise in electricity demand (+630 TWh) in their entirety.

• Every OECD country in 2025 was below its fossil generation peak for the first time. 36 out of the 38 OECD countries saw fossil generation peak in 2019 or before. Türkiye peaked in 2021, and Colombia peaked in 2024. Two member states – Iceland and Costa Rica – now operate zero-emission power systems, a further seven maintain fossil share below 10%, while a group of six nations remains heavily reliant on fossil fuels for over 60% of their electricity generation.

«

Reasons to be cheerful: part one, perhaps.
unique link to this extract


• Why do social networks drive us a little mad?
• Why does angry content seem to dominate what we see?
• How much of a role do algorithms play in affecting what we see and do online?
• What can we do about it?
• Did Facebook have any inkling of what was coming in Myanmar in 2016?

Read Social Warming, my latest book, and find answers – and more.


Errata, corrigenda and ai no corrida: none notified

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.