CPUs are Leading AI Inference Workloads, Not GPUs

Stefan Mileschin · 4th March 2024, 07:12

The AI infrastructure of today is mostly fueled by the expansion that relies on GPU-accelerated servers. Google, one of the world's largest hyperscalers, has noted that CPUs are still a leading compute for AI/ML workloads, recorded on their Google Cloud Services cloud internal analysis. During the TechFieldDay event, a speech by Brandon Royal, product manager at Google Cloud, explained the position of CPUs in today's AI game. The AI lifecycle is divided into two parts: training and inference. During training, massive compute capacity is needed, along with enormous memory capacity, to fit ever-expanding AI models into memory. The latest models, like GPT-4 and Gemini, contain billions of parameters and require thousands of GPUs or other accelerators working in parallel to train efficiently.

https://www.techpowerup.com/319880/g...loads-not-gpus

4th March 2024, 07:12	#1
Stefan Mileschin [M] Reviewer Join Date: May 2010 Location: Romania Posts: 148,726	CPUs are Leading AI Inference Workloads, Not GPUs The AI infrastructure of today is mostly fueled by the expansion that relies on GPU-accelerated servers. Google, one of the world's largest hyperscalers, has noted that CPUs are still a leading compute for AI/ML workloads, recorded on their Google Cloud Services cloud internal analysis. During the TechFieldDay event, a speech by Brandon Royal, product manager at Google Cloud, explained the position of CPUs in today's AI game. The AI lifecycle is divided into two parts: training and inference. During training, massive compute capacity is needed, along with enormous memory capacity, to fit ever-expanding AI models into memory. The latest models, like GPT-4 and Gemini, contain billions of parameters and require thousands of GPUs or other accelerators working in parallel to train efficiently. https://www.techpowerup.com/319880/g...loads-not-gpus

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
NVIDIA's RTX GPUs will speed up Maya 2020 workloads	Stefan Mileschin	WebNews	0	11th December 2019 09:28
Intel's latest Nervana processor is focused on AI inference	Stefan Mileschin	WebNews	0	8th January 2019 08:05
Xivlia Photographs Naked CPUs and GPUs	Stefan Mileschin	WebNews	0	29th December 2018 10:50
AMD CPUs Take Charge over Intel at Leading German Retailer	Stefan Mileschin	WebNews	0	3rd September 2018 08:06
Leading DRAM Makers Release Modules Validated with AMD Ryzen 2000 CPUs	Stefan Mileschin	WebNews	0	28th May 2018 06:09
Raja Koduri is running Intel's CPUs and GPUs	Stefan Mileschin	WebNews	0	16th November 2017 18:03
AMD Bets on DirectX 12 for Not Just GPUs, but Also its CPUs	Stefan Mileschin	WebNews	0	24th March 2015 06:29
WPA/WPA2 Cracking With CPUs, GPUs, and the Cloud	jmke	WebNews	0	19th August 2011 16:32
Price & Performance Stats for CPUs and GPUs	Sidney	WebNews	0	30th November 2007 20:48
CPUs and GPUs for 2007: A Peek Into the Future	jmke	WebNews	1	4th October 2006 22:09