Skip to content

TESTING.md

Latest commit

 

History

History
215 lines (153 loc) · 6.15 KB

TESTING.md

File metadata and controls

215 lines (153 loc) · 6.15 KB

Testing Guide for QuietPage

This document contains important information about testing in the QuietPage project, including known issues, best practices, and how to run specific test suites.

Table of Contents

Quick Start

# Run all tests
make test

# Run specific test file
uv run pytest apps/journal/tests/test_models.py -v

# Run tests by marker
uv run pytest -m unit -v
uv run pytest -m statistics -v

Running Tests

Full Test Suite

# Run all tests with coverage report
make test

# Run tests without coverage
uv run pytest

Specific Test Files

# Run a specific test file
uv run pytest apps/api/tests/test_statistics_views.py -v

# Run a specific test class
uv run pytest apps/api/tests/test_statistics_views.py::TestStatisticsViewMoodAnalytics -v

# Run a specific test method
uv run pytest apps/api/tests/test_statistics_views.py::TestStatisticsViewMoodAnalytics::test_mood_timeline_aggregation -v

Using Test Markers

# Run only unit tests
uv run pytest -m unit -v

# Run only integration tests
uv run pytest -m integration -v

# Run tests for a specific domain
uv run pytest -m statistics -v
uv run pytest -m streak -v
uv run pytest -m encryption -v

Known Test Isolation Issues

Rate Limiting Tests (Django REST Framework)

Location: apps/api/tests/test_statistics_views.py::TestStatisticsViewRateLimiting

Issue: These tests pass when run in isolation but may fail when run with all 171 tests in the file.

Root Cause: Django REST Framework caches its api_settings at module load time. When tests modify settings.REST_FRAMEWORK['DEFAULT_THROTTLE_RATES'] to test different throttle limits, DRF's cached settings don't automatically update. Despite extensive efforts to reload modules and clear caches, pytest-django's deep integration creates persistent references that prevent proper test isolation when many tests run together.

Important: This is a test isolation challenge, NOT a production bug. The rate limiting implementation in apps/api/statistics_views.py is correct and works properly in production.

How to Run Rate Limiting Tests:

# Run rate limiting tests in isolation (recommended)
uv run pytest apps/api/tests/test_statistics_views.py::TestStatisticsViewRateLimiting -v

# Or use the rate_limiting marker
uv run pytest -m rate_limiting -v

Expected Results:

  • When run in isolation: All 6 tests PASS ✓
  • When run with all tests in the file: 5 tests may fail (false negatives due to caching)

Implementation Details: The StatisticsView is correctly configured:

  • Uses ScopedRateThrottle throttle class
  • Has throttle_scope = "statistics"
  • Settings define 'statistics': '100/hour' rate limit
  • Production behavior is correct

Tests Affected:

  1. test_rate_limit_prevents_excessive_requests
  2. test_different_periods_count_toward_same_limit
  3. test_rate_limit_per_user_isolation
  4. test_throttle_status_code_and_message
  5. test_cache_and_throttle_interaction

What We Tried:

  • Reloading rest_framework.settings module
  • Reloading rest_framework.throttling module
  • Reloading apps.api.statistics_views module
  • Clearing Django cache before/after each test
  • Clearing Django URL resolver caches
  • Using autouse fixtures at class level
  • Various combinations of the above

The caching is too deep in the pytest-django/DRF interaction to reliably isolate when 171 tests run together.

Test Markers

The project uses pytest markers to categorize tests:

Test Speed/Scope

  • unit - Unit tests (fast, isolated)
  • integration - Integration tests (slower, multiple components)
  • slow - Slow running tests

Test Categories

  • models - Model tests
  • views - View tests
  • forms - Form tests
  • utils - Utility function tests
  • signals - Signal handler tests
  • api - API endpoint tests

Domain-Specific

  • encryption - Encryption/decryption tests
  • streak - Streak calculation tests
  • statistics - Statistics and analytics tests
  • celery - Celery task tests
  • rate_limiting - Rate limiting/throttle tests (needs isolation)

Using Markers

# Run fast unit tests only
uv run pytest -m unit

# Run statistics tests
uv run pytest -m statistics

# Combine markers
uv run pytest -m "unit and statistics"

# Exclude slow tests
uv run pytest -m "not slow"

Best Practices

  1. Run tests frequently during development to catch issues early
  2. Use markers to run relevant test subsets for faster feedback
  3. Check coverage to ensure new code is tested (80% minimum)
  4. Run full test suite before creating pull requests
  5. For rate limiting tests, always run them in isolation for accurate results

Coverage

The project maintains 80% test coverage minimum. View coverage reports:

# Generate coverage report
make test

# Open HTML coverage report
open htmlcov/index.html

Troubleshooting

Tests Failing Locally But Should Pass

  1. Clear pytest cache: rm -rf .pytest_cache
  2. Clear Django cache: python manage.py shell -c "from django.core.cache import cache; cache.clear()"
  3. Recreate database: rm db.sqlite3 (development only)
  4. Run specific test in isolation to verify it's not a test interaction issue

ImportError or Module Not Found

  1. Ensure virtual environment is activated: source .venv/bin/activate (or use uv run)
  2. Install dependencies: uv sync
  3. Check PYTHONPATH includes project root

Database Errors

The test suite uses --reuse-db flag to speed up tests. If you see database errors:

# Drop and recreate test database
uv run pytest --create-db

CI/CD Considerations

When setting up CI/CD pipelines:

  1. Run rate limiting tests separately:

    # Run main test suite
- run: uv run pytest --ignore=apps/api/tests/test_statistics_views.py::TestStatisticsViewRateLimiting

# Run rate limiting tests in isolation
- run: uv run pytest -m rate_limiting -v

  2. Or accept that rate limiting tests may show false negatives when run with all tests and rely on isolated test runs for verification.