A short history of AI: study notes

0.03 A short history of AI: symbolic AI, AI winters, the deep-learning boom: detailed study notes

Learn AI by Story · Module 0: What is AI · Level: Beginner

The idea in one sentence

AI is over 70 years old: early symbolic systems built on hand-written rules, two "winters" when hype faded and funding dried up, and then a comeback driven by big data, fast chips and deep learning.

The story from the video

Grandpa Raman has watched AI for decades and says it has summers and winters. He tells the story: Alan Turing's question in 1950, the Dartmouth workshop in 1956, rule-based AI, two winters, and the boom after 2012 that led to transformers and ChatGPT.

Key definitions

Term	Definition	In plain English
Symbolic AI	An approach in which intelligence is built from symbols and explicit rules written by people (also called good old-fashioned AI, GOFAI).	Hand-written if-then logic.
Expert system	A program that captures an expert's knowledge as a set of rules to give advice in a narrow area.	A rulebook that acts like a specialist.
AI winter	A period of reduced interest and funding after AI failed to meet inflated expectations.	The hype fades and the money goes away.
Turing test	A 1950 idea by Alan Turing: if a person cannot reliably tell a machine from a human in conversation, the machine can be said to think.	A chat-based imitation game.
Dartmouth workshop	The 1956 summer research project where the term artificial intelligence was introduced.	The birthday of the field's name.
Perceptron	An early (1958) simple learning model of a single artificial neuron.	The first neural network idea.
Neural network	A model made of layers of connected simple units that learns by adjusting connection strengths.	A web of tiny calculators.
Backpropagation	The method that computes how to adjust each connection in a neural network to reduce errors.	How the network learns from mistakes.
GPU	Graphics processing unit, a chip with thousands of small cores that is very good at the parallel maths used to train neural networks.	The chip that made deep learning fast.
ImageNet	A large labelled image dataset and a yearly contest that became a benchmark for image recognition.	The exam that proved deep learning.
Transformer	A neural-network design from 2017 that uses attention to process sequences, and is the basis of modern language models.	The architecture behind ChatGPT-style models.
Foundation model	A very large model trained on broad data that can be adapted to many tasks.	One big model, many uses.

Detailed explanation

1. The birth of the field (1950 to the 1960s)

In 1950 Alan Turing published a paper asking whether machines can think and proposed the imitation game, now called the Turing test. In 1956 a group of researchers met at Dartmouth College and proposed studying how machines could use language, form concepts and improve themselves. The term artificial intelligence comes from this workshop. In the 1950s and 1960s, programs proved mathematical theorems, played checkers well enough to beat their creators, and early chatbots such as ELIZA imitated conversation using simple pattern rules.

2. Symbolic AI

Early AI was mostly symbolic. Researchers believed intelligence could be built by writing down facts and rules, for example: IF fever AND cough THEN suspect flu. This works well in tidy, narrow areas where the rules are known and the world is simple. It breaks down in messy real life, where there are too many exceptions, and where the knowledge, such as recognising a face, is hard to write as rules.

3. The first winter (around 1974)

Early promises were large. Machine translation and general problem solving turned out to be much harder than predicted, computers were slow, and memory was small. Critical reports, such as the 1973 Lighthill report in the UK, led governments to cut funding. Research on simple neural networks was also set back after a 1969 book showed limits of single-layer perceptrons. This is called the first AI winter.

4. Expert systems and the second winter (1980s to early 1990s)

In the 1980s, expert systems brought AI back into business: companies used rule-based programs for tasks such as configuring computers or diagnosing faults. But the rule sets became huge, hard to update, and brittle outside their narrow domain. Specialised hardware also lost out to ordinary computers. By around 1987 the market collapsed, and a second winter followed.

5. Quiet progress: learning from data (1990s and 2000s)

During and after the winters, a different idea kept growing: let machines learn from data instead of writing rules. Backpropagation was popularised in 1986 for training multi-layer neural networks. Statistical machine learning methods such as support vector machines worked well on many problems. In 1997 IBM's Deep Blue beat the world chess champion using search and hand-tuned evaluation, rather than learning from data.

6. Why the boom came back

Three things came together:

1. Big data: the internet and smartphones created huge datasets, such as ImageNet with millions of labelled images.
2. GPUs: cheap parallel computing made it possible to train large networks in days instead of years.
3. Better algorithms: improved training tricks and deeper network designs made deep networks trainable.

7. The deep-learning boom (2012 onwards)

In 2012 a deep convolutional network called AlexNet won the ImageNet competition by a large margin, and the field shifted rapidly to deep learning. In 2016 AlphaGo beat a top human player at Go. In 2017 researchers at Google introduced the transformer architecture. Large language models built on transformers led to systems like ChatGPT, released to the public in late 2022, which brought generative AI to hundreds of millions of people.

8. Symbolic versus learning

Symbolic AI has humans write the rules, so it is transparent but brittle, and needs little data. Deep learning has the machine learn the rules from data, so it is flexible but needs lots of data and is harder to explain. Many modern systems combine both.

What the video did not explain

• Names behind the milestones: Turing (1950), McCarthy and Minsky (Dartmouth 1956), Rosenblatt (perceptron 1958), Rumelhart, Hinton and Williams (backpropagation 1986), Krizhevsky, Sutskever and Hinton (AlexNet 2012).
• Why hype cycles happen. Early results are exciting, expectations are inflated, real-world difficulty appears, funding drops, and then new tools restart progress. Today's boom may also have limits, so careful expectations are healthy.
• Other key milestones. Deep Blue (1997), IBM Watson winning Jeopardy! (2011), AlphaGo (2016). Video 0.09 covers milestones in detail.
• Compute is a driver. Training compute for the largest models has grown enormously, which is one reason progress sped up.
• Different schools. Symbolic, connectionist (neural), and statistical approaches competed. Many current research efforts try to combine reasoning and learning.
• Regional and ethical context. Questions about bias, safety, jobs and regulation have grown with the technology and are covered later in the roadmap.

Worked example

A small timeline (dates approximate):

Year	Event	Era
1950	Turing asks whether machines can think	Beginning
1956	Dartmouth workshop names the field	Beginning
1958	Perceptron	Early neural idea
1974	Funding cuts after unmet promises	First winter
1980s	Expert systems boom	Symbolic AI in business
1987	Expert system market collapses	Second winter
1997	Deep Blue beats the world chess champion	Search-based AI
2012	AlexNet wins ImageNet	Deep-learning boom
2017	Transformers introduced	Modern language models
2022	ChatGPT released	Generative AI to the public

Exercise: count the years between the Dartmouth workshop and AlexNet. 2012 minus 1956 is 56 years, so deep learning's breakthrough came more than five decades after the field was named.

Common mistakes and myths

• "AI was invented recently." The field is about 70 years old. Only the latest wave is new.
• "Neural networks are a new idea." The perceptron dates from the 1950s. What changed is data, computing power and training methods.
• "AI winters mean AI failed." Winters were slowdowns in funding and expectations. Research continued and later paid off.
• "ChatGPT started AI." It popularised generative AI, but built on decades of work.
• "Dates are exact." Eras have fuzzy boundaries, and historians place them slightly differently.

Where it is used

History helps you judge today's claims: when you hear a bold promise, you can compare it with earlier cycles. The ideas also live on in practice: rule-based logic in business software and safety systems, statistical machine learning in finance and ads, and deep learning in vision, speech and language.

Self-check questions

1. Who asked "can machines think?" in 1950? Alan Turing.
2. Where and when was the term artificial intelligence introduced? At the 1956 Dartmouth workshop.
3. What is symbolic AI? AI built on hand-written symbols and rules.
4. What is an AI winter? A period when hype fades and funding drops after unmet promises.
5. Name the two winters and roughly when they happened. Around 1974 and around 1987.
6. Name three things that brought AI back. Big data, GPUs and better algorithms.
7. What did AlexNet do in 2012? Won the ImageNet contest by a large margin using a deep neural network.
8. What did the 2017 transformer design lead to? Modern large language models.
9. Give one strength and one weakness of symbolic AI. Strength: transparent and needs little data. Weakness: brittle in messy real life.
10. Why is knowing AI history useful? It helps you judge hype and see that progress comes in waves.

Key takeaways

• AI began in the 1950s and the name dates to 1956.
• Early AI used hand-written rules and hit limits.
• Two winters followed overpromising.
• Data, GPUs and better algorithms drove the deep-learning boom after 2012.
• Transformers and ChatGPT brought generative AI to everyone.

Go deeper and what next

• Read about the Turing test and the Lighthill report.
• Search for the AlexNet ImageNet result and see how large the improvement was.
• Next in the roadmap: 0.04 Rules vs learning: two ways to build smart software.