AI’s existential crisis explodes as AI companies plunge ahead anyway – but urge a slowdown

The resignation this week of Anthropic researcher Jacob Coxon over concerns that artificial intelligence labs are “gambling with our lives” lit up the long-smoldering argument about whether AI is heading in a direction hazardous to human life.

Not only is he not alone at Anthropic — whose alignment science lead agreed with the possibility that AI has a more than 10% chance to “kill all humans” in the next decade — but OpenAI chimed in, as its chief scientist called for an AI research slowdown. And CEO Sam Altman reportedly told staff that the company is considering slowing down leading-edge AI work and wants other companies to do the same — if the government says that’s not an antitrust problem.

We’ll have to see if any of this existential worry actually changes anything — even Congress may be waking up (or maybe not) along with the rest of the world — but I’ll believe it when I see more than talk. Just this week, as almost every week, there was a flurry of new models, agents and apps galore — from Meta, OpenAI (twice), China’s DeepSeek and more. I understand why, and I’m still a bit skeptical of the AI doomers given the lack of specific reasons why AI can’t be controlled, but there’s definitely a disconnect here.

One thing that may slow things down regardless is the rising opposition to data centers. In a new feature story, Paul Gillin digs into exactly why AI requires so much compute — and why there isn’t a short-term fix. Tech giants have a lot of work ahead of them persuading communities not to shut down the next era of tech innovation.

By the way, don’t count out Europe in AI: Mistral AI this week raised almost $3.5 billion. And Paris-based startup Arlequin AI got $28 million to explore a fundamentally new kind of AI model.

Oracle looks like it’s finally benefiting from the AI boom, as its cloud infrastructure revenue more than doubled, and its stock rose 4% in late trading after falling more than that in regular trading, on an ugly day for the market. Adobe had a strong quarter too, but a light outlook sent shares down a bit after-hours Thursday.

As expected, autonomous software engineering firm Cognition AI raised another big round, a $32 billion Series E that values it at — wait for it — $48 billion.

Positron AI thinks it can solve the memory crunch in AI with an appliance that uses regular RAM instead of pricey and scarce high-bandwidth memory, raising an $875 million Series C round to prove it. Normally I might take such a claim with a grain of salt, but familiar old-school backers such as Silicon Graphics’ Jim Clark, Forest Baskett and Tom Jermoluk know what they’re doing, so this will be worth watching.

Oh, and Apple introduced some new iPhones, including a very expensive foldable. I find myself yawning, but I’m sure most of them will do fine.

Another flurry of events is coming next week: Salesforce’s Dreamforce in San Francisco, the AI Infra Summit in Santa Clara with lots of AI luminaries, Workiva Amplify in Las Vegas and the agent-focused developer conference AGNTCon + MCPCon in Amsterdam.

AI’s existential crisis explodes as AI companies plunge ahead anyway – but urge a slowdown

Analysis, opinion and food for thought

AI and data: AI still could kill us all — say AI companies

Policy

New models and services

Money matters

Around the enterprise: Oracle gets an AI boost

Money matters

New products and services

Cyber beat: More bitcoin mischief

Attack & response

New services

Money matters

Elsewhere in tech: Apple fields a foldable

Comings and goings

What’s next

Events

Image: SiliconANGLE/Reve

Similar Posts