Microsoft CEO Satya Nadella’s big AI warning: ‘It’s not worth pursuing’ without human control

Nadella said the development of AI must ensure that the technology does not slip beyond human control.

“Any pursuit of superintelligence has to be grounded in the core principle that if the AI we build is not helping humanity and under human control, it’s not worth pursuing,” Nadella wrote on X.

He also called for the benefits of AI to be accelerated and distributed more widely across countries, communities and companies.

“We also need to accelerate and spread the benefits of AI, such that they are diffused broadly across countries, communities, and companies. This requires a frontier ecosystem in which both closed and open-source models can thrive,” he said.

Nadella further argued that companies should retain control over their proprietary knowledge and avoid becoming overly dependent on any single AI model provider.

“And for firms, it’s imperative that they retain full control over their unique and tacit knowledge. Every organization should be able to build its own continuous learning loop/hill climbing machine, without becoming dependent on any one model provider, and have the ability to embed its own knowledge into models and weights they control,” he said.

He added that research, focus and deliberate pacing were necessary to ensure that AI alignment remained a core design objective.

“So, in this context, we welcome the research, focus, and deliberate pacing needed to get alignment right as the design goal. We also welcome ideas like ‘embedded evaluators’ and the broader efforts to develop the mechanisms to make this more than just talk,” Nadella said.

He stressed that AI development should not be controlled by a small number of companies or individuals, calling for broader representation across countries, industries and academia.

“The key is that this cannot be controlled by a handful of entities, but must have broad representation across the ecosystem, countries, and fields, including academia,” he said.

Trump rejects calls to slow AI development

US President Donald Trump has dismissed calls to slow the development of artificial intelligence and introduce new regulations for the industry, arguing that “negative forces” are fueling concerns about the technology.

Speaking to reporters, Trump said the US was ahead of China in AI and argued that maintaining that lead was crucial.

“We’re leading China in AI. We’re the most sophisticated country in the world, and frankly I want to keep it that way because whoever wins AI wins,” Trump said.

Rejecting calls from industry experts for a slowdown in AI development, Trump said regulations could be introduced but suggested that concerns about the technology were being overstated.

“We could put guardrails. We can do this and that. But I think you have a lot of very negative forces that are bringing it up that shouldn’t be bringing it up and they’re bringing up things that won’t happen,” he said.

Trump’s remarks came as several prominent figures in the AI industry, including Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman and xAI CEO Elon Musk, have called for greater caution over the pace of AI development.

The call for a slowdown was initially made by Amodei, who argued that the rapid improvement in AI capabilities should be moderated.

“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” Amodei wrote in an essay.