It turns out that today's AI landscape can be traced back to fear, jealousy, and intense capitalist ambition. The emails revealed in the Justice Department's antitrust case against Google, first mentioned by Interested in trade, shows Microsoft executives expressing their annoyance and envy at Google's progress in artificial intelligence. This catalyzed the urgency that led the Windows maker to make an initial $1 billion investment in its now indispensable partner, OpenAI.
In a heavily redacted 2019 email series titled “Thoughts on OpenAI,” Microsoft CEO Satya Nadella sends a lengthy letter from CTO Kevin Scott to CFO Amy Hood. “A very good email explaining why I want us to do this… and also why we will then ensure our infrastructure staff do this,” Nadella wrote.
Scott wrote that he was “deeply concerned” about Google's rapidly growing AI capabilities. He says he initially rejected the company's “exciting gameplay games,” likely a reference to Google's AlphaGo models. One of them beat world champion Go Ke Jie in 2017, an impressive feat at the time. (Later Google models went beyond this model, eliminating the need for human training altogether.)
But Scott says ignoring Google's progress in gaming “was a mistake.” “When they took over all the infrastructure they had built [natural language] “Models that we couldn't easily replicate started to take things more seriously,” Scott wrote. “As I dug deeper into trying to understand where all the capability gaps were between Google and us in terms of modular training, I felt very, very concerned.”
Scott recounts how Microsoft struggled to copy Google's big BERT model, the AI model that decodes the meaning and context of words in a sentence. Scott blamed leaps in infrastructure made by its competitor, which Microsoft did not.
“It turned out that simply replicating the large BERT model was not easy for us. “Even though we had the model in place, it took us approximately 6 months to train the model because our infrastructure was not up to the task,” Microsoft’s CTO wrote. “Google had had BERT for at least six months before this, so in the time it took us to hack together the ability to train a 340M parameter model, they had a year to figure out how to get it into production and move forward to broader, more interesting models.
He also expressed his admiration and envy of Google's Gmail autocomplete capabilities, saying they have “become frighteningly good.” He commented that Microsoft was “several years behind the competition in terms of… [machine learning] Volume.” He commented on the “interesting” growth in OpenAI, DeepMind and Google Brain.
Scott praised Microsoft's “very smart” people on its machine learning teams, but said their ambitions were curbed. “But the core deep learning teams within each of these large teams are very small, and their ambitions have also been constrained, which means that even when we start feeding them resources, they still have to undergo a learning process to scale the work,” Scott wrote. “And we are several years behind the competition in terms of the machine learning scale.”
After Hood urged that Scott's concerns were “why do I want us to do this,” namely investing in OpenAI, the company carried out its CEO's wishes. Microsoft invested $1 billion in the Sam Altman-led startup in 2019, and the rest is fast-changing history. (It has now invested $13 billion.) It's a technology that does incredible things but threatens to destroy the job market and gives preachers their most powerful tool yet in what was already an age of rampant misinformation.
