“We almost went out of business several times.” Usually founders don’t talk about their company’s near-death experiences. But Jen-Hsun Huang, the boss of Nvidia, has no reason to be coy. His firm, which develops microprocessors and related software, is on a winning streak. In the past quarter its revenues increased by 55%, reaching $2.2 billion, and in the past 12 months its share price has almost quadrupled.
A big part of Nvidia’s success is because demand is growing quickly for its chips, called graphics processing units, or GPUs, which turn personal computers into fast gaming devices. But the GPUs also have new destinations: notably data centers where artificial intelligence programs gobble up the vast quantities of computing power that they generate.
Soaring sales of these chips are the clearest sign yet of a secular shift in information technology. The architecture of computing is fragmenting because of the slowing of Moore’s law, which until recently guaranteed that the power of computing would double roughly every two years, and because of the rapid rise of cloud computing and AI. The implications for the semiconductor industry and for Intel, its dominant company, are profound.
Things were straightforward when Moore’s law, named after Gordon Moore, a founder of Intel, was still in full swing. Whether in PCs or in servers (souped-up computers in data centers), one kind of microprocessor, known as a “central processing unit,” could deal with most “workloads,” as classes of computing tasks are called. Because Intel made the most powerful CPUs, it came to rule not only the market for PC processors (it has a market share of about 80%) but the one for servers, where it has an almost complete monopoly. In 2016 it had revenues of nearly $60 billion.
This unipolar world is starting to crumble. Processors are no longer improving quickly enough to be able to handle, for instance, machine learning and other AI applications, which require huge amounts of data and hence consume more number-crunching power than entire data centers did just a few years ago. Intel’s customers, such as Google and Microsoft together with other operators of big data centers, are opting for more and more specialized processors from other companies and are designing their own to boot.
Nvidia’s GPUs are one example. They were created to carry out the massive, complex computations required by interactive video games. GPUs have hundreds of specialized “cores” (the “brains” of a processor), all working in parallel, whereas CPUs have only a few powerful ones that tackle computing tasks sequentially. Nvidia’s latest processors boast 3,584 cores; Intel’s server CPUs have a maximum of 28.
And GPUs are only one sort of “accelerator,” as such specialized processors are known. The range is expanding as cloud-computing firms mix and match chips to make their operations more efficient and stay ahead of the competition. “Finding the right tool for the right job,” is how Urs Hölzle, in charge of technical infrastructure at Google, describes balancing the factors of flexibility, speed and cost.
At one end of the range are ASICs, an acronym for “application-specific integrated circuits.” As the term suggests, they are hard-wired for one purpose and are the fastest on the menu as well as the most energy-efficient. Google has built an ASIC called “Tensor Processing Unit” for speech recognition.
The other extreme is field-programmable gate arrays, or FPGAs. These can be programmed, meaning greater flexibility, which is why even though they are tricky to handle, Microsoft has added them to many of its servers, for instance those underlying Bing, its online-search service. “We now have more FPGAs than any other organization in the world,” says Mark Russinovich, chief technology officer at Azure, the firm’s computing cloud.
Instead of making ASICS or FPGAs, Intel focused in recent years on making its CPU processors ever more powerful. Yet the quickening rise of accelerators appears to be bad news for the company, says Alan Priestley of Gartner, an IT consultancy. The more computing happens on them, the less is done on CPUs.
In 2015 Intel bought Altera, a maker of FPGAs, for a whopping $16.7 billion. In August it paid more than $400 million for Nervana, a 3-year-old startup that is developing specialized AI systems ranging from software to chips. In the summer it will start selling a new processor, code-named Knights Mill, to compete with Nvidia. Intel is also working on another chip, Knights Crest, which will come with Nervana technology. At some point, Intel is expected also to combine its CPUs with Altera’s FPGAs.
Predictably, competitors see the future differently. Nvidia reckons it has already established its own computing platform. IBM is also trying to make Intel’s life harder. Taking a page from open-source software, the firm in 2013 “opened” its processor architecture, which is called Power, turning it into a semiconductor commons of sorts. Makers of specialized chips can more easily combine their wares with Power CPUs, and they get a say in how the platform develops.
Certainly, the age of the big, hulking CPU that handles every workload, no matter how big or complex, is over. It suffered, a bit like Humpty Dumpty, a big fall. And all of Intel’s horses and all of Intel’s men cannot put it together again.
© 2017 The Economist Newspaper Ltd., London (Feb. 25, 2017). All rights reserved. Reprinted with permission.
Image credits: Nicholas Roberts/Agence France-Presse/Getty Images