Almost all AI models you see or hear about these days use a powerful but expensive chip called the Nvidia A100, which costs around $10,000, to train or respond to users. Now the popularity of artificial intelligence tools has significantly increased the revenue of this major chip maker.
Companies like Microsoft and Google are competing with each other to add artificial intelligence tools to their search engines, and companies like OpenAI and Stable Diffusion are also looking to attract more users. But all these efforts are for the benefit of Nvidia, which provides the necessary platform for the production and supply of these services with its chip.
According to New Street Research, Nvidia has about 95 percent of the market for graphics processors used for machine learning. The A100 chip uses an interesting technology that was originally designed for rendering complex 3D graphics in games, but this chip is now used for configuration and machine learning in data centers.
Nvidia A100 is the most important player in the world of artificial intelligence processing
Companies operating in the field of supplying artificial intelligence models need hundreds or thousands of Nvidia chips or must obtain them through cloud services. Chips like the A100 can not only process several terabytes of data to teach AI models their content, but are also used to “infer” or generate text, predict and recognize objects in images.
The need for processing power has increased over time. Stability AI says it had just 32 A100 processors a year ago, but that number has now grown to more than 5,400. As we said, the main winner of artificial intelligence models coming to market is Nvidia.
In its Q4 2022 earnings report last week, the company announced that although its total sales were down 21%, its AI chip business grew 11% to more than $3.6 billion in sales in the quarter. In response to this report, Nvidia’s stock value rose by around 14% the next day. Nvidia’s stock price is up about 65% year-to-date since 2023.
“Jensen Huang”, the CEO of Nvidia, also emphasizes that the trend of artificial intelligence is at the center of the company’s business strategy. He says that regardless of all the plans that have been planned for this year, the trend that has emerged in the last two to three months has changed everything. Nvidia has already produced the H100 chip, which is considered the successor of the A100, and its supply has just started.
Performing artificial intelligence processes requires much more power compared to simple tasks such as web browsing. As a result, companies have to buy more graphics processors to be able to upgrade their models and provide them to users.
For the convenience of its customers, Nvidia has released a system called DGX A100, which offers 8 A100 chips along with 320 or 640 GB of RAM and some other features at a price of around 200,000 dollars. The company announced last week that it will also enable direct cloud access to DGX so that researchers can access it at a lower cost.
Larger products require more processors
However, you can see how expensive such systems can be. New Street Research says the ChatGPT AI model implemented in Bing requires 8 GPUs to answer a question in less than a second. Therefore, if Microsoft wants to provide public access to this service, it needs more than 20 thousand servers with 8 graphics processors. That means the people of Redmond have to spend $4 billion on their infrastructure.
But Microsoft has a very small share of the search engine market. If Google wants to provide such a model to all users at its scale, its costs will skyrocket. This search engine responds to 8-9 billion searches per day, so it must spend 80 billion dollars to provide the necessary DGXs.
But the CEO of Nvidia believes that the company’s products are actually cheap to perform the processing that these models require. If it weren’t for the company’s products, for example, instead of spending $100 million on a GPU-powered data center, you’d need to invest $1 billion in a CPU-powered data center, says Huang. Even this $100 million becomes an almost insignificant figure when it is placed in the cloud and divided among 100 companies.
Huang claims that you can currently train a large language model like GPT for roughly $10 million to $20 million, but if it weren’t for the company’s GPUs, the cost of using traditional processors to do the job would be much higher.
Nvidia is not the only company involved in the production of GPUs for artificial intelligence. AMD and Intel have their own processors, and even Google and Amazon work on their own processors; But analysts believe that the future will largely be in Nvidia’s hands. As of December, more than 21,000 open-source AI papers had announced that they used Nvidia chips for their experiments.
Currently, the biggest competitor to the Nvidia A100 chip is said to be its successor, the H100, which will be announced in 2022. The company says sales of the H100 surpassed the A100 in the quarter ending in January, even though the H100 is a more expensive chip.
The Nvidia H100 chip is one of the first graphics processors in the world of data centers, which is specially optimized for transformers. This technique, which is becoming more important day by day, is used by many new artificial intelligence tools. Nvidia announced last week that it wants to make training AI models more than 1 million times faster, potentially reducing the need for businesses to increase the number of chips.
