AlexNet: the 2012 breakthrough that started the deep learning boom
How a neural network trained on two gaming graphics cards in a Toronto bedroom won the 2012 ImageNet competition by a huge margin, and set off the deep learning boom behind today's AI.
First published · AI Digested
Every modern AI story, from ChatGPT to self-driving cars, traces back to a single result announced in September 2012. A neural network called AlexNet won an image recognition competition, not narrowly, but by a margin so large it changed the direction of an entire field.
The problem: getting computers to see
For decades, getting a computer to recognise what's in a photo was painfully hard. Researchers hand-designed features, like edges, corners and colour patterns, and fed them into learning algorithms. Progress was slow and incremental.
Neural networks, which could in theory learn their own features, had been out of fashion since the AI winters. A small group of researchers kept faith, most famously Geoffrey Hinton in Toronto, Yann LeCun in New York and Yoshua Bengio in Montreal.
Ingredient one: ImageNet
In 2006, a young professor named Fei-Fei Li began building something unfashionable: not a better algorithm, but a better dataset. Her team collected millions of images from the internet and paid thousands of online workers through Amazon Mechanical Turk to label them.
ImageNet, presented in 2009, grew to more than 14 million labelled images. From 2010, an annual contest challenged teams to classify images into 1,000 categories, from "container ship" to "Siamese cat".
Ingredient two: graphics cards
Training neural networks involves vast amounts of simple arithmetic, most of which can be done in parallel. GPUs, graphics processing units built by companies like Nvidia to render video games, happen to be brilliant at exactly that. In 2007, Nvidia released CUDA, software that made it practical to use GPUs for general calculations.
Ingredient three: better training tricks
Hinton's group and others had found ways to train deeper networks: new activation functions (ReLU) that sped up learning, and a technique called dropout that stopped networks memorising their training data.
AlexNet
Hinton's student Alex Krizhevsky built a deep convolutional neural network with eight learning layers and about 60 million parameters. He trained it on two Nvidia GeForce GTX 580 graphics cards, gaming cards that cost around $500 each, running for about a week. Fellow student Ilya Sutskever pushed the project forward, and Hinton supervised.
The results in the 2012 ImageNet contest:
| Entry | Top-5 error rate |
|---|---|
| AlexNet (Toronto) | 15.3% |
| Next best entry | 26.2% |
("Top-5 error" means the right answer wasn't among the model's five best guesses.) In a field used to improvements of a percentage point or two, it was a thunderclap.
The gold rush
Things moved fast:
- 2013: Hinton, Krizhevsky and Sutskever's newly founded company, DNNresearch, was auctioned. Google won, reportedly for $44 million, beating Baidu, Microsoft and DeepMind. Hinton split his time between Google and Toronto for the next decade.
- 2014: Google bought DeepMind. Facebook had opened an AI lab led by Yann LeCun the year before.
- 2015: A deep network from Microsoft Research beat estimated human performance on ImageNet.
- 2015: Ilya Sutskever left Google to co-found OpenAI.
Deep learning soon took over speech recognition, translation, recommendation systems and more. Nvidia, the maker of those gaming cards, transformed into the world's most valuable chipmaker.
Recognition
In 2018, Hinton, LeCun and Bengio received the Turing Award, computing's highest honour, as the "fathers of the deep learning revolution". In 2024, Hinton shared the Nobel Prize in Physics with John Hopfield.
Hinton has since become one of the most prominent voices warning about AI's risks. He left Google in 2023, saying he wanted to speak freely about the dangers of the technology he helped create.
Why it matters
AlexNet proved a simple, powerful recipe: big data, plus big computing power, plus deep neural networks, produces results nobody could hand-engineer. That same recipe, scaled up enormously and combined with the transformer, produced the chatbots we use today.
Read on
Common questions
What is ImageNet?
A huge database of labelled photos created by Fei-Fei Li and colleagues, first presented in 2009. Its annual competition asked programs to sort about 1.2 million training images into 1,000 categories, then label new ones.
Why were graphics cards so important?
Training a neural network involves enormous numbers of simple calculations that can be done at the same time. Graphics cards, built to colour millions of pixels at once for video games, turned out to be perfect for it, and far cheaper than supercomputers.
What happened to the AlexNet team?
Their company, DNNresearch, was bought by Google in 2013. Ilya Sutskever went on to co-found OpenAI in 2015. Geoffrey Hinton shared the 2018 Turing Award and the 2024 Nobel Prize in Physics.
Sources and further reading
Spotted a mistake or something out of date? Tell us and we will fix it. Read how we research and write our articles.
Read next
Nvidia: the chipmaker powering the AI boom
How Nvidia went from making graphics cards for video games to supplying the chips that train almost every major AI model, and became one of the most valuable companies on Earth.
Updated 22 November 2026
Alan Turing and the Turing test: the man who asked 'Can machines think?'
The story of Alan Turing, the British mathematician and codebreaker who imagined thinking machines in 1950, invented the Turing test, and laid the foundations of computing and AI.
Updated 22 November 2026
Dartmouth 1956: the summer artificial intelligence got its name
How a small 1956 summer workshop at Dartmouth College named the field of artificial intelligence, who was there, what they hoped to achieve, and why their optimism was so wrong about timing.
Updated 22 November 2026