DistilGPT-2 Text Generation

Fine-tunes DistilGPT-2 on a conversation dataset and compares base vs fine-tuned text generation.

Overview

This project fine-tunes a DistilGPT-2 language model on the HuggingFace UltraFeedback dataset and compares text generation between the original base model and the fine-tuned version. The script first trains the model for 3 epochs on 10 000 prompts, saves the checkpoint, then reloads both models to ensure a fair comparison. Task 1 generates 5 alternative completions per prompt using both models side by side. Task 2 uses a HuggingFace pipeline with the fine-tuned model to produce ranked completions.

Requirements

• Python 3.8+
• transformers >= 4.30
• datasets >= 2.12
• tensorflow >= 2.12

Installation

pip install -r requirements.txt

Project Structure

distilgpt2-text-generation/
├── generate_text.py        # Fine-tuning and text generation script
├── requirements.txt        # Python dependencies
├── .gitignore
└── README.md

Usage

python generate_text.py

Steps performed:

1. Downloads the DistilGPT-2 model and the UltraFeedback dataset from HuggingFace.
2. Fine-tunes the model for 3 epochs on 10 000 conversation prompts.
3. Reloads a fresh base model and the saved fine-tuned checkpoint.
4. Task 1: generates 5 completions per prompt for both base and fine-tuned models; prints comparison dictionary to stdout.
5. Task 2: runs a text-generation pipeline with the fine-tuned model on 3 prompts; prints ranked completions to stdout.

Results

All output is printed to stdout. Task 1 produces a nested dictionary keyed by prompt, with not_tuned and tuned sub-dictionaries each containing 5 generated completions. Task 2 produces ranked completion lists per prompt. The fine-tuned model reflects the conversational style present in the UltraFeedback training data.

Author

Biswajeet Sahoo

License

MIT License