Skip to content

AI models that cost $1 billion to train are underway, $100 billion models coming — largest current models take 'only' $100 million to train: Anthropic CEO

AI models that cost $1 billion to train are underway, $100 billion models coming — largest current models take 'only' $100...
AI training costs are growing exponentially year after year.

Table of Contents

AI training costs are growing exponentially year after year. Anthropic CEO Dario Amodei said in the In Good Company podcast that AI models in development today can cost up to $1 billion to train. Current models like ChatGPT-4o only cost about $100 million, but he expects the cost of training these models to go up to $10 or even $100 billion in as little as three years from now. The Anthropic CEO mentioned these numbers when he discussed the development of AI from generative artificial intelligence (like ChatGPT) to artificial general intelligence (AGI). He said that there wouldn't be a single point where we suddenly reach AGI. Instead, it would be a gradual development where models build upon the developments of past models, much like how a human child learns. So, if AI models grow ten times more powerful each year, we can rationally expect the hardware required to train them to be at least ten times more powerful, too. As such, hardware could be the biggest cost driver in AI training. Last year, over 3.8 million GPUs were delivered to data centers. With Nvidia's latest B200 AI chip costing around $30,000 to $40,000, we can surmise that Dario's billion-dollar estimate is on track for 2024. However, aside from the supply of the actual chip hardware, these AI firms need to be concerned with power supply and related infrastructure, too. The total estimated power consumption of all data center GPUs sold just last year could power 1.3 million homes. If the data center power requirements continue to grow exponentially, then it's possible that we could run out of enough economically-priced electricity. Furthermore, while these data centers need power plants, they also need an entirely upgraded grid that can handle all the electrons the power-hungry AI chips need to run. Artificial intelligence is quickly gathering steam, and hardware innovations seem to be keeping up. However, as our AI technologies perform exponentially better every new generation, one big question still remains: how will it affect the future of our society?

Source

Latest