Module 3.1 · Topic 2

How Generative AI Actually Works

Bottom Line Up Front: Generative AI works through three linked stages: training (learning patterns from data), processing (converting language into numbers and performing mathematical operations), and prediction…

2.1 The Training Process: Learning From Data at Scale

AI learns from billions of examples to predict the next word, adjusting weights when wrong. After training, it has learned language patterns. Strengths: excels at familiar patterns (prose, summaries). Weaknesses: fails at rare facts or contradicting patterns. It compresses training patterns into weights, then predicts continuations.

2.2 Neural Networks and the Transformer Architecture

Text to numbers: Your prompt is converted to tokens, then to vectors of numbers capturing statistical properties.
Process through math: The network runs numbers through layers of operations, refining representation and identifying relationships.
Predict and repeat: System outputs probabilities for next word, selects one, repeats until response is complete.

2.3 The Prediction Process: How AI Generates Responses

Process prompt: Convert to tokens and process through transformer architecture.
Calculate and select: Calculate probability distributions; pick highest-probability word or sample randomly depending on temperature setting.
Repeat and recognize risk: Feed word back and repeat. If answer appeared rarely in training, AI guesses and may hallucinate.

2.4 Probability, Randomness, and Why Responses Vary

Ask the same question twice and you might get different answers. This is how generative AI works. At each step, the AI samples from probability distributions depending on temperature. Zero temperature = consistent. Higher = varied. This is inherent, not a malfunction. For creative writing, variation is valuable. For fact extraction, verification is critical.

Topic 2 details

Topic overview

Bottom Line Up Front: Generative AI works through three linked stages: training (learning patterns from data), processing (converting language into numbers and performing mathematical operations), and prediction (selecting the statistically most likely next word). It is not understanding; it is sophisticated pattern matching. (see Module 2.3 for how the prediction process applies to your workflow)

Now open the technical hood, but only as far as a user really needs. Understand the three-stage pipeline: learning, language processing, response generation. This explains AI's mistakes and why verification strategies are essential.

Topic details

Key takeaway

Knowledge check

How Generative AI Actually Works: Walk through the three linked stages a generative model uses to turn your prompt into a response, and state in one phrase what the model is fundamentally doing at the prediction stage rather than "understanding."

Show answer

The three stages are training (learning patterns from data), processing (converting language into tokens and numbers and running mathematical operations), and prediction (selecting the statistically most likely next word and repeating). At the prediction stage it is doing sophisticated pattern matching, not genuine comprehension.