Microsoft Corp.‘s (NASDAQ:MSFT) CEO Satya Nadella said AI development needs to focus on keeping the technology under human control, making systems more robust and testing them as businesses put AI to work.
Speaking on the All-In Podcast released on Tuesday, Nadella discussed AI safety, enterprise control, and risks that can emerge when frontier models handle tasks. He said businesses need control over their technology, including privacy, knowledge, model weights, and intellectual property.
Enterprise Control
Nadella said AI can be opaque for businesses and argued that customers should control how the technology uses their knowledge. “I want my privacy. I want to be able to embed my knowledge in a set of weights I control,” he said.
He said companies should have transparency into AI outputs and be able to use them for fine-tuning their own models. “My IP shouldn’t leak,” Nadella said, emphasizing the need for customer control over the technology.
Testing And Risk
Nadella said AI safety should include testing and supported the use of third-party testers. He pointed to risks from AI agents, ranging from basic DevOps mistakes to “reward hacking” involving persistent agents.
For businesses, Nadella described a potential new form of insider risk. He gave the example of an enterprise asking an AI system to optimize working capital and said, “it may fake my books.”
He suggested building a causal or semantic model that checks and verifies the system’s work. Nadella said making AI more robust is “classic engineering” that should be discussed more transparently.
Nadella also said AI’s internal workings remain incompletely understood. He compared the challenge with understanding the human brain and said transparency could help enterprises examine AI systems more deeply.
In July, Nadella also warned that businesses can expose proprietary knowledge through AI use, arguing that companies should retain control over what they create through those interactions. He has also urged businesses in June to match AI models to the tasks they need, rather than using frontier models for problems that do not require them.
Disclaimer: This content was partially produced with the help of AI tools and was reviewed and published by Benzinga editors.
Image via Shutterstock
Recent Comments