It turns out the lay of today’s AI landscape can be traced back to — what do you know — fear, jealousy and intense capitalist ambition. Emails revealed in the Department of Justice’s antitrust case against Google, first reported by Business Insider, show Microsoft executives expressing alarm and envy over Google’s AI lead. That spurred an urgency that led to the Windows maker’s initial billion-dollar investment in its now-indispensable partner, OpenAI.
In a heavily redacted 2019 email thread titled “Thoughts on OpenAI,” Microsoft CEO Satya Nadella forwards a lengthy message from CTO Kevin Scott to CFO Amy Hood. “Very good email that explains, why I want us to do this … and also why we will then ensure our infra folks execute,” Nadella wrote.
Scott wrote that he was “very, very worried” about Google’s rapidly growing AI capabilities. He says he initially dismissed the company’s “game-playing stunts,” likely referring to Google’s AlphaGo models. One of them beat Go world champion Ke Jie in 2017, a remarkable feat at the time. (Google’s later models surpassed that one, dropping the need for human training altogether.)
But Scott says brushing off Google’s game-playing progress “was a mistake.” “When they took all of the infrastructure that they had built to build [natural language] models that we couldn’t easily replicate, I started to take things more seriously,” Scott wrote. “And as I dug in to try to understand where all of the capability gaps were between Google and us for model training, I got very, very worried.”
Scott recounts how Microsoft struggled to copy Google’s BERT-large, an AI model that deciphers the meaning and context of words in a sentence. Scott pinned the blame on infrastructure leaps its rival had made — and that Microsoft hadn’t.
“Turns out, just replicating BERT-large wasn’t easy to do for us. Even though we had the template for the model, it took us ~6 months to get the model trained because our infrastructure wasn’t up to the task,” the Microsoft CTO wrote. “Google had BERT for at least six months prior to that, so in the time that it took us to hack together the capability to train a 340M parameter model, they had a year to figure out how to get it into production and to move on to larger scale, more interesting models.”
He also admired and envied Google’s Gmail auto-complete capabilities, saying it was “getting scarily good.” He commented that Microsoft was “multiple years behind the competition in terms of [machine learning] scale.” He commented on the “interesting” growth of OpenAI, DeepMind and Google Brain.
Scott touted Microsoft’s “very smart” people on its machine-learning teams but said their ambitions were curbed. “But the core deep learning teams within each of these bigger teams are very small, and their ambitions have also been constrained, which means that even as we start to feed them resources, they still have to go through a learning process to scale up,” Scott wrote. “And we are multiple years behind the competition in terms of ML scale.”
After prompting Hood that Scott’s concerns were “why I want us to do this,” meaning invest in OpenAI, the company made good on its CEO’s wishes. Microsoft invested a billion dollars in the Sam Altman-led startup in 2019, and the rest is a rapidly changing history. (It’s now invested $13 billion.) It’s a technology that does some incredible things but threatens to gut the labor market and give propagandists their most powerful tools to date in what was already an age of rampant disinformation.
Credit: Source link