🧠 AI with Python – 🤖 Text Generation with GPT-2

Large Language Models (LLMs) have transformed the way computers understand and generate human language. From writing emails and answering questions to generating code and creative stories, these models power many of today’s AI applications.

At the heart of this revolution lies a simple idea:

Given some text, predict what comes next.

In this project, we’ll build our first text generation application using GPT-2, one of the most influential language models introduced by OpenAI. Although newer models exist today, GPT-2 remains an excellent starting point for understanding how modern LLMs work.

---

What Is GPT-2?

GPT-2 (Generative Pre-trained Transformer 2) is an autoregressive language model.

Instead of following predefined rules, it learns language patterns from massive amounts of text and predicts the next word—or more accurately, the next token—based on the context it has already seen.

For example:

Python is one of the most popular

GPT-2 might continue with:

programming languages used for data science and artificial intelligence.

It generates text one token at a time until the desired length is reached.

---

Loading the Pretrained Model

Thanks to the Hugging Face Transformers library, loading GPT-2 requires only a few lines of code.

First, we load the tokenizer.

tokenizer = GPT2Tokenizer.from_pretrained(
    "gpt2"
)

The tokenizer converts human-readable text into numerical tokens that the model can process.

Next, we load the pretrained GPT-2 model.

model = GPT2LMHeadModel.from_pretrained(
    "gpt2"
)

The first execution downloads the model weights, after which they are cached locally for future use.

---

Creating a Prompt

Every text generation task begins with a prompt.

For example:

prompt = (
    "Artificial Intelligence is transforming"
)

This prompt provides the initial context from which GPT-2 begins generating text.

The quality and wording of the prompt can significantly influence the generated output.

---

Tokenizing the Input

Before GPT-2 can process the prompt, it must be converted into tokens.

inputs = tokenizer.encode(
    prompt,
    return_tensors="pt"
)

Each token represents a piece of text, allowing the model to work with numerical representations instead of raw strings.

---

Generating New Text

With the input prepared, GPT-2 predicts one token after another.

outputs = model.generate(
    inputs,
    max_length=80,
    temperature=0.8,
    do_sample=True,
    top_k=50,
    top_p=0.95
)

The model repeatedly predicts the next most likely token until it reaches the specified length.

This iterative prediction process is the foundation of modern text generation.

---

Decoding the Output

The generated tokens are then converted back into readable text.

generated_text = tokenizer.decode(
    outputs[0],
    skip_special_tokens=True
)

The result is a coherent continuation of the original prompt.

---

Understanding Generation Parameters

One of the strengths of LLMs is the ability to control how text is generated.

Maximum Length

Determines how many tokens the model can generate.

max_length=80

Longer outputs provide more detailed responses but require additional computation.

---

Temperature

Controls randomness during generation.

temperature=0.8

• Lower values produce more predictable text.
• Higher values encourage creativity and variation.

Choosing the right temperature depends on the application.

---

Top-K Sampling

Instead of considering every possible next token, the model selects from only the top K most likely candidates.

top_k=50

This reduces unlikely or nonsensical outputs.

---

Top-P (Nucleus Sampling)

Rather than selecting a fixed number of tokens, Top-P chooses the smallest group of tokens whose combined probability reaches a specified threshold.

top_p=0.95

This often produces more natural and diverse text.

---

Experimenting with Prompts

One of the most interesting aspects of GPT-2 is how sensitive it is to the initial prompt.

Try prompts such as:

The future of healthcare...

Machine learning enables...

Python is becoming...

Climate change will...

Even small wording changes can lead to dramatically different outputs.

This demonstrates the importance of prompt design when working with language models.

---

Where GPT-2 Is Used

Although newer language models are available today, GPT-2 still illustrates the core ideas behind modern LLMs.

It can be used for:

• text completion
• content drafting
• story generation
• brainstorming ideas
• chatbot responses
• writing assistants

Many of the concepts learned with GPT-2 also apply to more advanced transformer-based models.

---

Key Takeaways

1. GPT-2 is a pretrained autoregressive language model that generates text one token at a time.
2. Every generation task begins with a prompt that provides context.
3. Tokenization converts text into a format the model can understand.
4. Parameters such as temperature, top-k, and top-p control the style and diversity of generated text.
5. GPT-2 provides an excellent foundation for understanding modern Large Language Models.

---

Conclusion

Text generation is one of the most recognizable applications of Large Language Models. By loading a pretrained GPT-2 model, supplying a prompt, and controlling the generation process with a few key parameters, we can create coherent and context-aware text with surprisingly little code.

This marks the beginning of the LLM Foundations track in the AI with Python series. In the upcoming projects, we’ll build on these concepts as we explore transformers, embeddings, prompt engineering, Retrieval-Augmented Generation (RAG), and AI agents—the technologies powering today’s most advanced AI applications.

---

# =========================================================
# 📦 Install Required Libraries
# =========================================================

# Run this in terminal if transformers/torch are not installed:
# pip install transformers torch


# =========================================================
# 📦 Import Required Libraries
# =========================================================

from transformers import (
    GPT2Tokenizer,
    GPT2LMHeadModel
)

import torch


# =========================================================
# 🤖 Load GPT-2 Tokenizer
# =========================================================

tokenizer = GPT2Tokenizer.from_pretrained(
    "gpt2"
)


# =========================================================
# 🧠 Load GPT-2 Model
# =========================================================

model = GPT2LMHeadModel.from_pretrained(
    "gpt2"
)

model.eval()


# =========================================================
# ✍️ Create Prompt
# =========================================================

prompt = (
    "Artificial Intelligence is transforming"
)


# =========================================================
# 🔤 Tokenize Prompt
# =========================================================

inputs = tokenizer.encode(
    prompt,
    return_tensors="pt"
)


# =========================================================
# 🚀 Generate Text
# =========================================================

with torch.no_grad():

    outputs = model.generate(
        inputs,
        max_length=80,
        temperature=0.8,
        do_sample=True,
        top_k=50,
        top_p=0.95,
        pad_token_id=tokenizer.eos_token_id
    )


# =========================================================
# 📖 Decode Generated Text
# =========================================================

generated_text = tokenizer.decode(
    outputs[0],
    skip_special_tokens=True
)

print("Generated Text:\n")
print(generated_text)


# =========================================================
# 🔁 Try Multiple Prompts
# =========================================================

prompts = [
    "The future of healthcare",
    "Python is one of the best",
    "Machine learning enables",
    "The biggest advantage of AI is"
]


for prompt in prompts:

    print("\n" + "=" * 60)
    print("Prompt:", prompt)
    print("=" * 60)

    inputs = tokenizer.encode(
        prompt,
        return_tensors="pt"
    )

    with torch.no_grad():

        outputs = model.generate(
            inputs,
            max_length=70,
            temperature=0.8,
            do_sample=True,
            top_k=50,
            top_p=0.95,
            pad_token_id=tokenizer.eos_token_id
        )

    generated_text = tokenizer.decode(
        outputs[0],
        skip_special_tokens=True
    )

    print(generated_text)