CUNY Course - RAG and LLM Core Concepts

Course materials and code examples for learning about RAG (Retrieval Augmented Generation) and LLM fundamentals.

Setup

Prerequisites

• Python 3.9 or higher
• Poetry for dependency management
• Optional: Azure/OpenAI API access for the existing structured-output example

Installation

1. Clone this repository:

git clone https://github.com/drdk/CUNY_course.git
cd CUNY_course

2. Install dependencies using Poetry:

poetry install

3. (Optional) Set up your environment variables in a .env file for API-based examples:

# For Azure OpenAI
AZURE_OPENAI_ENDPOINT=your_endpoint_here
AZURE_OPENAI_API_KEY=your_key_here
AZURE_OPENAI_DEPLOYMENT=your_deployment_name

# Or for OpenAI
OPENAI_API_KEY=your_key_here

Running Code Examples

Using Poetry

Run Python scripts directly:

poetry run python CUNY_course/example_code/structured_output.py
poetry run python CUNY_course/example_code/format_errror.py
poetry run python CUNY_course/example_code/rag_pipeline/01_data_prep.py
poetry run python CUNY_course/example_code/rag_pipeline/02_chunking.py
poetry run python CUNY_course/example_code/rag_pipeline/03_embedding.py
poetry run python CUNY_course/example_code/rag_pipeline/04_metadata.py
poetry run python CUNY_course/example_code/rag_pipeline/05_retrieval.py
poetry run python CUNY_course/example_code/rag_pipeline/06_generation.py
poetry run python CUNY_course/example_code/rag_pipeline/pipeline_demo.py

Or activate the virtual environment first:

poetry shell
python CUNY_course/example_code/structured_output.py

Using Jupyter

Start Jupyter:

poetry run jupyter notebook

Open and run:

• CUNY_course/example_code/rag_pipeline/rag_pipeline_demo.ipynb

Gradio Web App (Easiest Sharing)

Run the interactive RAG app locally:

poetry run python app.py

Then open http://localhost:7860.

Deploy for a Small Group (Hugging Face Spaces)

1. Create a new Gradio Space.
2. Push this repo (or at least app.py, requirements.txt, and CUNY_course/).
3. In Space Settings -> Variables and secrets, add:
   • AZURE_OPENAI_ENDPOINT
   • AZURE_OPENAI_API_KEY
   • AZURE_OPENAI_DEPLOYMENT
4. (Recommended) Set Space visibility to Private and invite your users.

The app entrypoint is app.py and uses the same RAG pipeline code as the notebook.

Project Structure

CUNY_course/
├── CUNY_course/
│   ├── example_code/       # Python code examples
│   │   ├── structured_output.py
│   │   ├── format_errror.py
│   │   └── rag_pipeline/   # Local in-memory RG pipeline demo
│   ├── example_data/       # Sample data files
│   │   ├── 10languages.txt
│   │   ├── usconstitution.txt
│   │   └── ...
│   ├── data_types/         # Pydantic models and schemas
│   │   └── person.py
│   ├── outline.md          # Course outline
│   └── links.md            # Useful links and resources
├── pyproject.toml          # Poetry dependencies
└── README.md               # This file

Key Files

• structured_output.py - Demonstrates how to extract structured data from text using LLMs with Pydantic models
• format_errror.py - Simple example showing Python data structures
• person.py - Pydantic models for representing people and relationships
• outline.md - Full course outline with lecture topics
• example_data/ - Various text files for experimentation

Local RG Pipeline Demo

• Fully local and in-memory (no API keys, no persistence layer)
• Uses open-source libraries only (sentence-transformers, faiss-cpu, transformers, torch)
• One Python file per pipeline step plus a combined notebook demo
• Default corpus file: CUNY_course/example_data/yosemite_guide.md
• Default chunking: character-based chunks with overlap (word-based strategy still available)

Classroom Quick Commands

Use the included Makefile for one-command runs:

make install     # install dependencies
make rg-steps    # run all six RG step scripts
make rg-demo     # run full end-to-end RG demo
make notebook    # open the RG demo notebook
make all         # install + run full end-to-end RG demo

Development

Code Formatting

poetry run black .

Running Tests

poetry run pytest

Resources

• See outline.md for the full course outline and lecture structure
• See links.md for useful tools and resources

License

See the LICENSE file for details.