Why it issues: As the dimensions of generative AI fashions grows sooner than the computing energy required to coach and run them, it is no surprise corporations like Microsoft are exploring in-house options as an alternative choice to off-the-shelf {hardware} from corporations that dominate the specialised house. graphics card house.
The race for AI supremacy among the many tech giants is simply simply starting, and it is gone past feeding the reams of information to fashions just like the well-known (or notorious, relying in your perspective) ChatGPT that is part of almost each information cycle.
A report from The Information claims that Microsoft is presently creating customized AI chips to coach its fashions. The mysterious piece of silicon is code-named “Athena,” and the corporate reportedly started engaged on it in 2019. Currently, solely a small group of Microsoft and OpenAI workers have entry to it to check how effectively it performs like GPT-4 when used with giant language fashions.
Companies giant and small are utilizing each enterprise-class Nvidia GPU potential to construct the highly effective programs wanted to coach AI fashions utilizing curated datasets. It’s the identical {hardware} they use to run fashions and do one thing known as “inference” — the method of taking real-world info or enter and producing one thing helpful for a selected software.
The downside is that Nvidia can solely make so many A100 and H100 GPUs, and each prices a fortune. A100 GPUs value round $10,000 every, whereas extra trendy H100 GPUs fetch over $40,000 on eBay. Team Green is so pleased with the pent-up demand that it is shifting a few of its GeForce RTX 4090 GPU capability to make extra H100 GPUs.
So it is no shock that corporations like Microsoft are attempting to scale back their reliance on a single vendor for machine studying. It might additionally result in important value financial savings, which is why Amazon, Meta, and Google are taking the identical route to construct in-house chips.
With 300 folks engaged on the Athena chip and a number of generations of improvement deliberate for the following few years, it will likely be attention-grabbing to see what occurs to Microsoft’s current partnership with Nvidia. Either approach, the Redmond big has been busy incorporating AI know-how into each services or products it provides, whether or not it is the Edge browser, the Bing search engine, the Microsoft 365 suite, GitHub, and extra.
Interestingly, one other Thurrott report means that Microsoft can be engaged on a Neural Processing Unit (NPU) for its Surface line of gadgets. It presently makes use of CPUs and SoCs from Intel, AMD, and Qualcomm — to this point, solely the Qualcomm model of the Surface Pro 9 has an NPU.
