Skip to content

SETUP.md

Latest commit

 

History

History
205 lines (158 loc) · 5.65 KB

SETUP.md

File metadata and controls

205 lines (158 loc) · 5.65 KB

🚀 Setup Guide for API Documentation Quality Experiment

📋 Prerequisites

  1. Python 3.8+ installed
  2. API keys from LLM providers (optional for demo, required for real testing)

🔑 Setting Up API Keys

Step 1: Get Your API Keys

Claude (Anthropic)

  1. Go to https://console.anthropic.com/
  2. Sign up or log in
  3. Navigate to "API Keys"
  4. Create a new API key
  5. Copy the key (starts with sk-ant-)

OpenAI GPT-4

  1. Go to https://platform.openai.com/api-keys
  2. Sign up or log in
  3. Click "Create new secret key"
  4. Copy the key (starts with sk-)

Google Gemini

  1. Go to https://makersuite.google.com/app/apikey
  2. Sign up or log in to Google AI Studio
  3. Click "Create API Key"
  4. Copy the key

Step 2: Configure API Keys

Option A: Using .env File (Recommended)

  1. Copy the template:

    cp .env.template .env

  2. Edit the .env file with your actual API keys:

    # Open .env in your text editor
notepad .env  # Windows
nano .env     # Linux/Mac

  3. Replace the placeholder values:

    ANTHROPIC_API_KEY=sk-ant-your-actual-claude-key-here
OPENAI_API_KEY=sk-your-actual-openai-key-here
GOOGLE_API_KEY=your-actual-gemini-key-here

  4. Save the file

Option B: Environment Variables

Windows Command Prompt:

set ANTHROPIC_API_KEY=sk-ant-your-actual-claude-key-here
set OPENAI_API_KEY=sk-your-actual-openai-key-here
set GOOGLE_API_KEY=your-actual-gemini-key-here

Windows PowerShell:

$env:ANTHROPIC_API_KEY="sk-ant-your-actual-claude-key-here"
$env:OPENAI_API_KEY="sk-your-actual-openai-key-here"
$env:GOOGLE_API_KEY="your-actual-gemini-key-here"

Linux/Mac Terminal:

export ANTHROPIC_API_KEY="sk-ant-your-actual-claude-key-here"
export OPENAI_API_KEY="sk-your-actual-openai-key-here"
export GOOGLE_API_KEY="your-actual-gemini-key-here"

📦 Installing Dependencies

Install Required Packages

pip install --user pandas numpy matplotlib seaborn scipy scikit-learn python-dotenv

Install LLM API Clients (for real API calls)

pip install --user anthropic openai google-generativeai

🧪 Testing Your Setup

1. Test with Demo (No API Keys Required)

python demo.py

2. Test API Key Configuration

python -c "
import os
print('Claude API Key:', '✅ Set' if os.getenv('ANTHROPIC_API_KEY') else '❌ Missing')
print('OpenAI API Key:', '✅ Set' if os.getenv('OPENAI_API_KEY') else '❌ Missing')
print('Gemini API Key:', '✅ Set' if os.getenv('GOOGLE_API_KEY') else '❌ Missing')
"

3. Run Full Experiment

# With API keys - real LLM testing
python controls/experiment_execution.py

# Analyze results
python evaluation/results_analysis.py --results-dir experiment_results

🔒 Security Best Practices

Keep Your API Keys Safe

  • ✅ DO: Use .env files (added to .gitignore)
  • ✅ DO: Use environment variables
  • ❌ DON'T: Hardcode keys in source code
  • ❌ DON'T: Commit keys to version control
  • ❌ DON'T: Share keys in screenshots or logs

API Key Formats to Expect

  • Claude: sk-ant-api03-... (long string)
  • OpenAI: sk-... (starts with sk-)
  • Gemini: Usually a shorter alphanumeric string

💰 Cost Considerations

Estimated Costs per Full Experiment Run

  • Claude: ~$2-5 (depending on response length)
  • OpenAI GPT-4: ~$3-8 (higher cost per token)
  • Gemini: ~$1-3 (generally lower cost)

Cost Control Tips

  1. Start with demo to test the framework
  2. Use fewer APIs initially (modify experiment_config.json)
  3. Reduce task complexity (test only basic tasks first)
  4. Monitor usage in your provider dashboards

🛠️ Troubleshooting

Common Issues

"ModuleNotFoundError"

# Install missing dependencies
pip install --user [missing_package_name]

"API Key Not Found"

# Check if .env file exists and has correct format
cat .env

# Verify environment variables are set
echo $ANTHROPIC_API_KEY  # Linux/Mac
echo %ANTHROPIC_API_KEY%  # Windows CMD

"Permission Denied" on Windows

# Use --user flag for pip installs
pip install --user package_name

"Invalid API Key" Errors

  1. Double-check the key format
  2. Ensure no extra spaces or quotes
  3. Verify the key is active in your provider dashboard
  4. Check if you have sufficient credits/quota

Getting Help

  1. Check the logs in experiment_results/ for detailed error messages
  2. Run the demo first to isolate API key issues
  3. Verify your setup with the test commands above
  4. Check provider status pages for API outages

🎯 Quick Start Checklist

  • Python 3.8+ installed
  • Dependencies installed (pip install --user pandas numpy matplotlib seaborn scipy scikit-learn)
  • API keys obtained from providers
  • .env file created with your API keys
  • Demo runs successfully (python demo.py)
  • API key test passes
  • Ready to run full experiment!

🚀 Next Steps

Once setup is complete:

  1. Run the demo: python demo.py
  2. Execute full experiment: python controls/experiment_execution.py
  3. Analyze results: python evaluation/results_analysis.py
  4. Review insights: Check experiment_results/insights_report.md
  5. Examine visualizations: Look in experiment_results/visualizations/

Need help? The experiment will provide helpful error messages and guidance if API keys are missing or incorrectly configured.