GitHub OAuth Authentication Setup

This document explains how to set up GitHub OAuth authentication for the GitHub Scraper application.

Overview

The application now supports GitHub OAuth authentication, allowing users to sign in with their GitHub account and automatically use their GitHub token for API requests. This provides:

✅ Better rate limits (5,000/hour vs 60/hour unauthenticated)
✅ Access to private repositories
✅ Secure token storage (tokens never exposed to frontend)
✅ Seamless user experience

Architecture Note: The backend runs in Docker containers, while the frontend runs locally on your machine. This setup allows for faster frontend development while keeping backend services containerized.

Step 1: Create GitHub OAuth Apps

Important: You need to create TWO separate OAuth Apps - one for development and one for production. GitHub only allows one callback URL per OAuth app, so we use separate apps for each environment.

Development OAuth App

Go to GitHub Developer Settings
Click "New OAuth App"
Fill in the application details:
- Application name: GitHub Scraper (Dev) (or your preferred name)
- Homepage URL: http://localhost:3001
- Authorization callback URL: http://localhost:3000/auth/github/callback
Click "Register application"
Copy the Client ID and generate a Client Secret
Save these for your local .env file

Production OAuth App

Go to GitHub Developer Settings
Click "New OAuth App" again
Fill in the application details:
- Application name: GitHub Scraper (Prod) (or your preferred name)
- Homepage URL: Your Vercel production URL (e.g., https://github-scraper-xxx.vercel.app)
- Authorization callback URL: https://your-backend-url.run.app/auth/github/callback (your Cloud Run backend URL)
Click "Register application"
Copy the Client ID and generate a Client Secret
Save these for GCP Secret Manager (use scripts/secrets/create-secrets.sh script)

Step 2: Environment Variables

Local Development

Create a .env file in the root directory of the project:

# GitHub OAuth App Credentials (Development)
GITHUB_CLIENT_ID=your_dev_github_oauth_client_id
GITHUB_CLIENT_SECRET=your_dev_github_oauth_client_secret

# Application URLs
# Frontend runs locally on port 3001, backend runs in Docker on port 3000
FRONTEND_URL=http://localhost:3001
BACKEND_URL=http://localhost:3000

# Session Secret (generate a random string)
SESSION_SECRET=your-random-session-secret-string-min-32-characters

# Optional: Fallback GitHub Token
GITHUB_TOKEN=your_github_personal_access_token

# Frontend API URL (used by Next.js frontend)
NEXT_PUBLIC_API_URL=http://localhost:3000

Important Notes:

Docker Compose will automatically pass these variables to the backend container
SESSION_SECRET should be a long, random string (at least 32 characters)
Generate it using: openssl rand -base64 32
The frontend runs locally on port 3001 (not in Docker)
The backend runs in Docker on port 3000
Use the Development OAuth App credentials in your local .env file

Production

For production, secrets are stored in GCP Secret Manager and configured in cloudrun.yaml. Use the create-secrets.sh script to set them up:

./scripts/secrets/create-secrets.sh

The script will prompt you for:

GITHUB_CLIENT_ID (Production OAuth App)
GITHUB_CLIENT_SECRET (Production OAuth App)
FRONTEND_URL (Your Vercel production URL)
BACKEND_URL (Your Cloud Run URL, e.g., https://your-backend-url.run.app)

Production URLs:

FRONTEND_URL: Your Vercel production URL (e.g., https://github-scraper-xxx.vercel.app)
BACKEND_URL: Your Cloud Run URL (e.g., https://your-backend-url.run.app)

Step 3: Database Migration

Run the Prisma migration inside the Docker container:

docker-compose -f docker-compose.services.yml exec backend npx prisma migrate dev --name add_user_model

Or if the containers aren't running yet, start them first:

docker-compose -f docker-compose.services.yml up -d db
docker-compose -f docker-compose.services.yml exec backend npx prisma migrate dev --name add_user_model

Note: The backend runs inside Docker, so all migrations must be run using docker-compose -f docker-compose.services.yml exec backend.

Step 4: Install Dependencies

Backend Dependencies

Backend dependencies will be installed automatically when Docker builds the containers. If you need to manually install:

docker-compose -f docker-compose.services.yml exec backend npm install

The following packages are installed in the backend:

@fastify/cookie - Cookie handling
@fastify/cors - CORS support
@fastify/oauth2 - OAuth2 integration
@fastify/session - Session management

Frontend Dependencies

Install frontend dependencies locally (frontend runs outside Docker):

cd frontend
pnpm install

Step 5: Start the Application

Start Backend Services (Docker)

Make sure your .env file is in the root directory with all required variables
Start backend services using Docker Compose:

# Using docker-compose.services.yml (recommended)
docker-compose -f docker-compose.services.yml up -d

# Or using the start script
   ./scripts/dev/start-services.sh

This will start:

PostgreSQL database (port 5432)
Redis (port 6379)
Backend API (port 3000)
Worker service

Start Frontend (Local)

In a separate terminal, start the frontend locally:

cd frontend
pnpm run dev

The frontend will start on http://localhost:3001

Access the application:
- Frontend: http://localhost:3001 (runs locally)
- Backend API: http://localhost:3000 (runs in Docker)
The application should now be running with OAuth authentication enabled!

Note:

Backend services run in Docker containers
Frontend runs locally for faster development
All backend operations (migrations, npm commands, etc.) must be performed using docker-compose -f docker-compose.services.yml exec backend

How It Works

Authentication Flow

User clicks "Sign in with GitHub" button
User is redirected to GitHub OAuth authorization page
User authorizes the application
GitHub redirects back to backend callback URL (/auth/github/callback)
Backend exchanges authorization code for access token
Backend stores user info and token in database
Backend creates a session and redirects to frontend
Frontend detects successful authentication and fetches user info

Token Usage

Authenticated users: Their GitHub token is automatically used for all API requests
Unauthenticated users: Can still manually enter tokens (backward compatible)
Fallback: If no user token and no manual token, uses GITHUB_TOKEN env variable

Security Features

✅ Tokens stored encrypted in database
✅ Tokens never exposed to frontend
✅ Session-based authentication with secure cookies
✅ CORS configured for secure cross-origin requests
✅ HttpOnly cookies prevent XSS attacks

Troubleshooting

"Not authenticated" error

Check that cookies are enabled in your browser
Verify SESSION_SECRET is set correctly
Check browser console for CORS errors
Ensure FRONTEND_URL matches your actual frontend URL

OAuth callback fails

Verify callback URL in GitHub OAuth App settings matches backend URL
Check that GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET are correct
Ensure backend is accessible at the callback URL

CORS errors

Verify FRONTEND_URL in root .env file matches your frontend URL (http://localhost:3001)
Check that credentials: true is set in CORS config (already configured)
Ensure frontend uses withCredentials: true in axios requests (already configured)
Restart Docker containers after changing environment variables: docker-compose -f docker-compose.services.yml restart backend

Session not persisting

Check that cookies are being set (inspect browser DevTools → Application → Cookies)
Verify SESSION_SECRET is set and consistent
In production, ensure secure: true is set for HTTPS (already configured)

Production Deployment

Setting Up Production Secrets

Create Production OAuth App (see Step 1 above)
Run the secrets setup script:
```
./scripts/secrets/create-secrets.sh
```
This will prompt you for all production secrets including:
- GitHub OAuth credentials (Production app)
- Frontend URL (Vercel)
- Backend URL (Cloud Run)

Update Cloud Run service:

gcloud run services replace cloudrun.yaml \
  --project=YOUR_GCP_PROJECT_ID \
  --region=us-east1

Additional Considerations

HTTPS Required: OAuth requires HTTPS in production ✅ (Cloud Run and Vercel both use HTTPS)
Session Storage: Redis is already configured for session storage in production ✅
Environment Variables: Secrets are stored in GCP Secret Manager ✅
Rate Limiting: Consider adding rate limiting to OAuth endpoints
Token Refresh: GitHub tokens don't expire, but consider implementing token refresh logic

Local Development Setup

The docker-compose.services.yml file has already been updated with the necessary environment variables. Make sure your root .env file contains all the required values (see Step 2).

Quick Setup Summary:

Create .env file in root directory with Development OAuth App credentials
Set FRONTEND_URL=http://localhost:3001 and BACKEND_URL=http://localhost:3000
Start backend services: docker-compose -f docker-compose.services.yml up -d or ./scripts/dev/start-services.sh
Run migrations: docker-compose -f docker-compose.services.yml exec backend npx prisma migrate dev --name add_user_model
Start frontend locally: cd frontend && pnpm run dev
Access frontend at http://localhost:3001

Important:

Backend runs in Docker, frontend runs locally
All backend operations (migrations, npm commands, etc.) must be run inside the Docker container using docker-compose -f docker-compose.services.yml exec backend
Use Development OAuth App credentials in local .env file
Use Production OAuth App credentials in GCP Secret Manager

API Endpoints

Authentication Endpoints

GET /auth/github - Initiates OAuth flow (redirects to GitHub)
GET /auth/github/callback - OAuth callback handler
GET /auth/me - Get current authenticated user
POST /auth/logout - Logout current user
GET /auth/token - Get user's GitHub token (internal use)

Updated Endpoints

POST /leaderboard - Now uses authenticated user's token automatically if available
All endpoints support session-based authentication via cookies

Testing

Start the application
Click "Sign in with GitHub"
Authorize the application on GitHub
You should be redirected back and see your GitHub username in the top right
Try submitting a private repository - it should work without entering a token manually

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GitHub OAuth Authentication Setup

Overview

Step 1: Create GitHub OAuth Apps

Development OAuth App

Production OAuth App

Step 2: Environment Variables

Local Development

Production

Step 3: Database Migration

Step 4: Install Dependencies

Backend Dependencies

Frontend Dependencies

Step 5: Start the Application

Start Backend Services (Docker)

Start Frontend (Local)

How It Works

Authentication Flow

Token Usage

Security Features

Troubleshooting

"Not authenticated" error

OAuth callback fails

CORS errors

Session not persisting

Production Deployment

Setting Up Production Secrets

Additional Considerations

Local Development Setup

API Endpoints

Authentication Endpoints

Updated Endpoints

Testing

FilesExpand file tree

OAUTH_SETUP.md

Latest commit

History

OAUTH_SETUP.md

File metadata and controls

GitHub OAuth Authentication Setup

Overview

Step 1: Create GitHub OAuth Apps

Development OAuth App

Production OAuth App

Step 2: Environment Variables

Local Development

Production

Step 3: Database Migration

Step 4: Install Dependencies

Backend Dependencies

Frontend Dependencies

Step 5: Start the Application

Start Backend Services (Docker)

Start Frontend (Local)

How It Works

Authentication Flow

Token Usage

Security Features

Troubleshooting

"Not authenticated" error

OAuth callback fails

CORS errors

Session not persisting

Production Deployment

Setting Up Production Secrets

Additional Considerations

Local Development Setup

API Endpoints

Authentication Endpoints

Updated Endpoints

Testing