How does AI think? (Theory behind ChatGPT)

Creating apps, generating content, solving problems, tasks once reserved for experts can now be handled with a few well-phrased questions. The shift is significant, and understanding how we arrived at this point means exploring the development of artificial intelligence.

This article follows the progression of AI through key stages:

• What is AI and where does it come from?
  An overview of its origins and early development.

• The Rise of Deep Learning
  How increased computing power and data reshaped machine learning.

• The Birth of Language Models
  The emergence of systems capable of processing and generating human language.

• What is an LLM, really?
  A breakdown of large language models and how they function.

• What’s Generative AI?
  Exploring AI’s ability to create new content in text, image, and beyond.

• Digital Guides: How Chatbots Lead Us Through AI
  The role of conversational interfaces in making AI accessible.

Each section builds toward a clearer picture of the systems shaping today’s technology landscape.

What is AI and where does it come from?

Humans have always wondered if we could build thinking machines. When computers were first created this was accelerated and in 1950 Alan Turing asked the same question and followed it up with the famous Turing Test, a thought experiment where a machine tries to fool a human into thinking it’s also human. This was the spark that lit the AI flame. They defined it as performing tasks which normally require human intelligence, they could understand language, recognizing images, solving problems and making their own decisions, essentially becoming a virtual person that can answer all your questions and solve all your problems. This is why Turing Test was important, where essentially you would put an Artificial Intelligence face-to-face with human who has to now determine whether they are talking to a human or a robot. AI essentially mimics human thinking. This is why John McCarthy gave it the name Artificial Intelligence. They thought it would take a summer to reach level where it would pass these tests and work perfectly by itself, but in reality development of AI is still ongoing.

What is AI and where does it come from?

Early AI, in the 60s and 70s, was rule-based. If you wanted a computer to “think,” you had to tell it exactly how to think. These were expert systems, where every rule had to be coded by a human. This worked until it didn’t, you can’t teach AI to do every single decision for every possible scenario, it’s impossible, or at least it wouldn’t, they had to figure out how computers could make new decisions by themselves, decision no one faced them with before.

Enter Machine Learning. In the 1980s and 1990s, researchers shifted toward a new idea, what if we could teach computers to learn from data instead of just rules? That’s machine learning, training an algorithm on a bunch of examples, so it can spot patterns and make predictions, what does that mean? Let’s imagine that in past you would teach AI how to follow grammar by writing out every single grammar rule, what machine learning meant as a concept is that AI was given thousands of articles, books and documents to read and figure out how English works by itself, self-learning.

The Rise of Deep Learning

Machine learning was great, but limited. It often needed humans to tell it which features to look at. Then came Deep Learning, powered by neural networks, a structure loosely inspired by how the human brain works, looking at this large amount of data, but in steps, which helped it to see more and more patterns.

The real breakthrough happened around 2012, when AlexNet, a deep neural network, crushed a major image recognition competition. Suddenly, deep learning could beat humans at recognizing cats on the internet. This wasn’t just better, it was scary good. Deep learning meant you could feed raw data (text, images, sound) into a model and it would figure out the important patterns itself. No more hand-holding. Just more data, more layers, more compute. AI started to learn exponentially.

The Birth of Language Models

Once deep learning cracked images, researchers asked: can it crack language, too? The answer, yes, but not easily. Language is full of nuance. But with enough data, and enough clever architecture, deep learning models like Recurrent Neural Networks (RNN) which could understand data in sequence, meaning it wouldn’t just look at one word, but how those words come one after another and why they do it in that way and later Transformers which didn’t only look on those words individually in the sequence, but could look at the text as a whole all at once also helped start understanding and generating text.

In 2017, Google introduced the Transformer architecture. It changed the game. Transformers could process language in parallel, faster, and pay attention to different parts of a sentence, mimicking human-like focus. This architecture powers Large Language Models or LLMs, like GPT, Gemini, Mistral, suddenly everyone wanted to create their own LLM that’s better than the other.

What Is an LLM, Really?

A Large Language Model (LLM) is a type of artificial intelligence system designed to generate and understand human language. It is trained on vast amounts of text data, such as books, websites, articles, and code and it is built using deep learning. Instead of understanding words like a human, it learns the patterns in how we write and speak.

The tech behind it? Something called a Transformer architecture that lets it process and generate language at scale. That’s where the “GPT” in ChatGPT comes from:

• Generative – it creates new content
• Pre-trained – it learns from general data first
• Transformer – the model structure doing the heavy lifting

Depending on the version of LLM, the chatbot’s intelligence, accuracy, and conversational abilities can vary dramatically. Newer versions understand context better, make fewer mistakes, and provide more helpful responses.

This difference comes down to parameters – the billions of connections that determine how the model processes information. More parameters generally mean better memory and deeper understanding.

You definitely heard about GPT-4, Claude, Gemini, LLaMA. So now it’s important to understand one important thing: any of these models doesn’t “understand” what it says, they’re just really good at predicting the next word, based on the context.

What is Generative AI?

Generative AI is a concept you will often hear connected to AIs. It’s an umbrella term for any AI that creates new stuff. If it can write, draw, speak, or sing without copying existing material, it’s generative — it generates new things. It can create new text (think ChatGPT), images (such as DALL·E or Midjourney), videos (like Sora), or code (like GitHub Copilot). There are many different types supported by many different LLMs.

Chatbots: Our Digital Guides

Chatbots are our friendly entry point into the complex knowledge of the entire world. Instead of needing technical knowledge, we simply start a conversation and explore AI naturally. They translate intimidating technology into our language.

Chatbot uses:

• Deep learning: to learn language patterns from vast text data
• Transformer architecture: for scalable, efficient understanding of context
• Machine learning: to continually improve and adapt based on feedback
• Generative AI: to craft human-like responses in real-time

But let’s not forget: it doesn’t “understand” the way humans do. It imitates understanding. That’s fine for now. We’re not quite at the AI singularity, but we’re definitely on the highway. And ChatGPT? It’s just the latest mile marker in a much longer trip.