diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..cef52b6a --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,113 @@ +# Campus Quick Reference + +Essential reminders for working on the Campus codebase. This guide applies to both humans and AI assistantsβ€”everyone should follow the same workflow and principles. + +## Critical Reminders (Read First!) + +### 1. Use Poetry for All Python Commands + +Campus uses Poetry for dependency management. Never run Python directly. + +```bash +# Correct +poetry run python main.py +poetry run python tests/run_tests.py unit + +# Incorrect +python main.py +python -m unittest discover tests +``` + +### 2. Use `run_tests.py` for Testing + +The only supported test entrypoint is `tests/run_tests.py`. It handles environment setup, cleanup, and isolation. Running tests directly may produce false positives. + +```bash +# Run all tests +poetry run python tests/run_tests.py all + +# Run specific category +poetry run python tests/run_tests.py unit +poetry run python tests/run_tests.py integration +``` + +### 3. Campus Uses `unittest`, Not pytest + +Test files use the standard library `unittest.TestCase` framework. No pytest dependencies are installed. + +### 4. Read These Files Before Starting Work + +| File | Purpose | +|------|---------| +| [docs/GETTING-STARTED.md](docs/GETTING-STARTED.md) | Setup and navigation | +| [docs/CONTRIBUTING.md](docs/CONTRIBUTING.md) | Development workflow and branching | +| [docs/STYLE-GUIDE.md](docs/STYLE-GUIDE.md) | Code standards and import patterns | +| [docs/TESTING-GUIDE.md](docs/TESTING-GUIDE.md) | Testing strategies and how to run tests | +| [docs/development-guidelines.md](docs/development-guidelines.md) | Architecture patterns and gotchas | + +## Project Structure + +For detailed file locations and architecture, see: +- [docs/architecture.md](docs/architecture.md) - System design and service boundaries +- [campus/README.md](campus/README.md) - Package-level documentation + +## Quick Links + +### Tests +- [docs/TESTING-GUIDE.md](docs/TESTING-GUIDE.md) - Complete testing guide +- [tests/README.md](tests/README.md) - Test directory reference +- [tests/contract/README.md](tests/contract/README.md) - HTTP contract invariants + +### Code Standards +- [docs/STYLE-GUIDE.md](docs/STYLE-GUIDE.md) - Import patterns, docstrings, commit messages +- [docs/development-guidelines.md](docs/development-guidelines.md) - Architecture patterns, common pitfalls + +### Getting Started +- [docs/GETTING-STARTED.md](docs/GETTING-STARTED.md) - Installation and setup +- [docs/CONTRIBUTING.md](docs/CONTRIBUTING.md) - Branching strategy and workflow + +## Known Gotchas + +These are the things most commonly forgottenβ€”even by experienced developers: + +### Storage Initialization Order + +Test fixtures must lazy-import `campus.storage` modules. Otherwise storage backends initialize before test mode is set. + +```python +# Bad - initializes storage immediately +from campus.storage import tables + +# Good - lazy import in test method +def test_something(self): + from campus.storage import tables +``` + +### Flask Blueprint Registration + +Flask blueprints can only be registered once. The test infrastructure shares Flask apps across test classes to avoid "already registered" errors. + +### Import Patterns + +Import packages, not individual functions. This preserves context and prevents naming conflicts. + +```python +# Good +from campus.common import utils, devops + +# Bad - loses context +from campus.common.utils import uid, utc_time +``` + +See [docs/STYLE-GUIDE.md](docs/STYLE-GUIDE.md) for complete import guidelines. + +## Before You Start + +- [ ] Have you read the relevant documentation? +- [ ] Are you using `poetry run python` for all commands? +- [ ] Will you use `run_tests.py` for testing? +- [ ] Do you understand the storage-model-resources pattern? + +--- + +**New to Campus?** Start with [docs/GETTING-STARTED.md](docs/GETTING-STARTED.md). diff --git a/README.md b/README.md index f07c0347..aebd6bc0 100644 --- a/README.md +++ b/README.md @@ -38,9 +38,9 @@ cd campus # Install dependencies poetry install -# Run the deployment -poetry run python main.py campus.auth # or -poetry run python main.py campus.api +# Run the deployment (set DEPLOY environment variable) +export DEPLOY=campus.auth # or campus.api +poetry run python main.py ``` > **Note**: Configuration is managed through environment variables and the vault service. See [Configuration](#-configuration) section below for details. @@ -77,7 +77,7 @@ Secrets managed via `campus.auth.vaults`, accessed through `campus_python` clien - **[πŸ“– Getting Started](docs/GETTING-STARTED.md)** - New user guide and navigation - **[πŸ—οΈ Architecture](docs/architecture.md)** - Detailed architecture overview and design principles - **[🀝 Contributing](docs/CONTRIBUTING.md)** - Development workflow and guidelines -- **[πŸ§ͺ Testing](docs/testing-strategies.md)** - Testing approaches and strategies +- **[πŸ§ͺ Testing](docs/TESTING-GUIDE.md)** - Testing approaches and strategies - **[πŸ“¦ Packaging](docs/PACKAGING.md)** - Monorepo structure and distribution **Service Documentation:** diff --git a/docs/AGENTS.md b/docs/AGENTS.md deleted file mode 100644 index 4966bc20..00000000 --- a/docs/AGENTS.md +++ /dev/null @@ -1,98 +0,0 @@ -# AI Agent Guidelines - -This file contains reminders for AI agents (Claude, Copilot, etc.) working on the Campus codebase. - -## 🚨 Critical Reminders (Read First!) - -1. **Use Poetry for all Python commands** - - ❌ `python main.py` - - βœ… `poetry run python main.py` - - ❌ `python -m unittest discover tests` - - βœ… `poetry run python run_tests.py` - -2. **Use `run_tests.py` as the ONLY test entrypoint** - - Never run `unittest` or `pytest` directly - - The test entrypoint handles environment setup, cleanup, and isolation - - Running tests directly may produce false positives or miss failures - - Usage: `poetry run python run_tests.py [unit|integration|sanity|type|all]` - -3. **Read these files BEFORE starting work** - - [CONTRIBUTING.md](CONTRIBUTING.md) - Development workflow and branching - - [GETTING-STARTED.md](GETTING-STARTED.md) - Setup instructions - - [STYLE-GUIDE.md](STYLE-GUIDE.md) - Code standards and import patterns - -4. **Campus uses `unittest`, not `pytest`** - - Test files use `unittest.TestCase` framework - - Fixtures are in `tests/fixtures/` directory - - Test discovery uses `unittest` patterns - -## πŸ“ Key File Locations - -### Tests -- `tests/run_tests.py` - **ALWAYS use this to run tests** -- `tests/fixtures/` - Test fixtures and shared utilities -- `tests/unit/` - Unit tests -- `tests/integration/` - Integration tests -- `tests/contract/` - HTTP contract tests (black-box) - -### Code -- `campus/` - Main application code - - `campus/auth/` - Authentication service - - `campus/api/` - REST API - - `campus/common/` - Shared utilities - - `campus/model/` - Entity definitions - - `campus/storage/` - Data persistence layer - -### Docs -- `docs/CONTRIBUTING.md` - How to contribute -- `docs/GETTING-STARTED.md` - Setup guide -- `docs/STYLE-GUIDE.md` - Code standards -- `docs/architecture.md` - System design -- `docs/integration-test-refactor-plan.md` - Test improvement progress - -## 🎯 Common Tasks - -### Running Tests -```bash -# All tests -poetry run python run_tests.py - -# Specific category -poetry run python run_tests.py unit -poetry run python run_tests.py integration -``` - -### Creating Tests -1. Extend `unittest.TestCase` -2. Place in appropriate directory (`tests/unit/`, `tests/integration/`, `tests/contract/`) -3. Use fixtures from `tests/fixtures/` -4. Run via `run_tests.py` to verify - -### Import Patterns (from STYLE-GUIDE.md) -- Import packages, not individual functions -- Use `from campus.common import env` not `from campus.common.env import ENV` -- Lazy imports in tests to avoid early storage initialization - -## ⚠️ Known Gotchas - -1. **Storage initialization order matters** - - Test fixtures must lazy-import `campus.storage` modules - - Otherwise storage backends initialize before test mode - -2. **Test isolation requires proper cleanup** - - Each test class should use `ServiceManager` from `tests/fixtures/services.py` - - `reset_test_storage()` is called at start of `ServiceManager.setup()` - - Test clients load credentials dynamically from environment - -3. **Flask blueprints can only be registered once** - - Using `shared=False` with `ServiceManager` causes errors - - Flask apps are shared across test classes to avoid this - -## πŸ“‹ Session Checklist - -Before starting work, confirm: -- [ ] Read CONTRIBUTING.md -- [ ] Read GETTING-STARTED.md -- [ ] Read STYLE-GUIDE.md -- [ ] Using `poetry run python` for all commands -- [ ] Will use `run_tests.py` for testing diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md index 4df04ed3..291bb520 100644 --- a/docs/CONTRIBUTING.md +++ b/docs/CONTRIBUTING.md @@ -1,8 +1,10 @@ # Contributing to Campus -Welcome to Campus development! This guide covers our development workflow and contribution process. +This guide covers our development workflow, branch strategy, and contribution process. -## 🌳 Branch Structure +**New here?** See [GETTING-STARTED.md](GETTING-STARTED.md) for installation instructions. + +## Branch Strategy Campus uses a three-branch model: @@ -10,156 +12,140 @@ Campus uses a three-branch model: weekly β†’ staging β†’ main ``` -- **`weekly`** - Active development (target for all new work) -- **`staging`** - Pre-production validation (requires review) -- **`main`** - Production releases (maintainer only) +| Branch | Purpose | Who Commits | +|--------|---------|-------------| +| `weekly` | Active development | All contributors | +| `staging` | Pre-production validation | Maintainers (via PR) | +| `main` | Production releases | Maintainers (via PR) | -## πŸš€ Quick Start +## Workflow -### 1. Setup +### 1. Create a Feature Branch ```bash -git clone https://github.com/nyjc-computing/campus.git -cd campus +# Start from weekly git checkout weekly -poetry install +git pull origin weekly -# Set up pre-push hook to catch sanity check failures early -git config core.hooksPath .githooks +# Create your feature branch +git checkout -b feature/your-feature-name ``` -The pre-push hook will run sanity checks before allowing pushes to GitHub. This saves time by catching issues locally before CI/CD runs. To bypass the hook (not recommended): `git push --no-verify` - -### 2. Create Feature +### 2. Make Changes ```bash -# Create branch from weekly -git checkout weekly -git pull origin weekly -git checkout -b feature/your-feature-name +# Run tests before committing +poetry run python tests/run_tests.py -# Make changes and test -poetry run python run_tests.py -poetry run python main.py - -# Commit and push +# Commit with conventional commit format git add . -git commit -m "feat: describe your changes" -git push origin feature/your-feature-name +git commit -m "feat(auth): add OAuth provider support" ``` -### 3. Pull Request +### 3. Create Pull Request -1. Create PR targeting `weekly` branch -2. Use conventional commit format in title: +1. Push your branch: `git push origin feature/your-feature-name` +2. Create PR targeting `weekly` branch +3. Use conventional commit in title: - `feat:` - New features - `fix:` - Bug fixes - - `docs:` - Documentation - - `test:` - Tests + - `docs:` - Documentation changes + - `test:` - Test changes - `refactor:` - Code restructuring -## πŸ§ͺ Testing +### 4. Code Review -**⚠️ IMPORTANT: Always use `run_tests.py` as the entrypoint for running tests.** +- Address review feedback +- Ensure tests pass +- Wait for maintainer approval -```bash -# Run all tests (sanity, type, unit, integration) -poetry run python run_tests.py - -# Run specific test categories -poetry run python run_tests.py unit # Unit tests only -poetry run python run_tests.py integration # Integration tests only -poetry run python run_tests.py sanity # Sanity checks only -poetry run python run_tests.py type # Type checks only -``` +## Code Review Checklist -**Do NOT run tests directly with `unittest` or `pytest`** - the test entrypoint handles proper environment setup, cleanup, and isolation between test classes. Running tests directly may produce false positives or miss failures. +### Before Submitting a PR -See [Testing Strategies](testing-strategies.md) for comprehensive approaches. +- [ ] All tests pass locally (`poetry run python tests/run_tests.py all`) +- [ ] Code follows [STYLE-GUIDE.md](STYLE-GUIDE.md) +- [ ] Documentation is updated (docstrings, relevant docs) +- [ ] Commit messages follow conventional commit format +- [ ] No secrets or credentials in code +- [ ] New features have corresponding tests -## πŸ“¦ Package Structure +### Review Focus Areas -``` -campus/ -β”œβ”€β”€ auth/ # Authentication and OAuth services -β”‚ β”œβ”€β”€ oauth_proxy/ -β”‚ β”œβ”€β”€ resources/ -β”‚ └── routes/ -β”œβ”€β”€ api/ # RESTful API resources -β”‚ β”œβ”€β”€ resources/ -β”‚ └── routes/ -β”œβ”€β”€ common/ # Shared utilities -β”œβ”€β”€ model/ # Entity representation (dataclasses) -β”œβ”€β”€ services/ # Business services (email, etc.) -β”œβ”€β”€ storage/ # Data persistence layer -β”œβ”€β”€ integrations/# External service integrations -└── yapper/ # Logging framework -``` - -### Key Rules +When reviewing or when preparing your PR for review: -- `auth` and `api` contain business logic in `.resources` submodules -- `model` contains only entity definitions (no business logic) -- Use `poetry run python` for consistency -- Follow [Style Guide](STYLE-GUIDE.md) for imports and coding standards +- **Security**: Check for potential vulnerabilities +- **Performance**: Look for inefficient operations +- **Maintainability**: Ensure code is readable and well-structured +- **Testing**: Verify adequate test coverage +- **Documentation**: Confirm docs match implementation -## 🎯 Best Practices +## Commit Message Format -### Code Quality -- Write tests for new features -- Update documentation for changes -- Use type hints and docstrings -- Follow existing patterns +``` +type(scope): description -### Commit Messages -```bash -feat(auth): add OAuth provider support +# Examples +feat(api): add circle management endpoints fix(storage): resolve PostgreSQL connection timeout -docs(api): update circle management examples +docs(auth): update OAuth configuration examples +refactor(common): extract ID generation to utils module +test(integration): add user flow tests ``` -### Common Pitfalls -- Use `poetry run python` not bare `python` -- Import packages not individual functions -- Update `pyproject.toml` for new dependencies -- Test locally before pushing +## Testing -## πŸ”€ Maintainer Workflow +Always run tests before committing: -### Weekly β†’ Staging ```bash -# After sprint review -# Create PR: weekly β†’ staging -# Title: "T3W10: weekly PR" +# All tests +poetry run python tests/run_tests.py all + +# Specific category +poetry run python tests/run_tests.py unit +poetry run python tests/run_tests.py integration ``` -### Staging β†’ Main +See [TESTING-GUIDE.md](TESTING-GUIDE.md) for complete testing documentation. + +## Pre-Push Hooks (Recommended) + +Set up the pre-push hook to catch issues early: + ```bash -# After validation -# Create PR: staging β†’ main -# Title: "v0.2 release: staging PR" +git config core.hooksPath .githooks ``` -## πŸ†˜ Getting Help +The hook runs sanity checks before allowing pushes. To bypass (not recommended): `git push --no-verify` + +## Maintainer Workflow + +### Weekly β†’ Staging + +After sprint review: +1. Create PR: `weekly` β†’ `staging` +2. Title: `"T3W10: weekly PR"` (or appropriate week) +3. Validate on staging environment + +### Staging β†’ Main -- **Issues**: [GitHub Issues](https://github.com/nyjc-computing/campus/issues) -- **Discussions**: [GitHub Discussions](https://github.com/nyjc-computing/campus/discussions) -- **Code Reviews**: [Best Practices](https://nyjc-computing.github.io/nanyang-system-developers/contributors/training/code-reviews.html) +After staging validation: +1. Create PR: `staging` β†’ `main` +2. Title: `"v0.2.0: staging PR"` +3. Tag release after merge -## πŸ“š Documentation +## Development Guidelines -- **[Architecture](architecture.md)** - System design -- **[Development Guidelines](development-guidelines.md)** - Coding patterns -- **[Testing Strategies](testing-strategies.md)** - Testing approaches -- **[Style Guide](STYLE-GUIDE.md)** - Code standards +For code-level guidelines (patterns, architecture, imports), see: +- [development-guidelines.md](development-guidelines.md) - Architecture patterns +- [STYLE-GUIDE.md](STYLE-GUIDE.md) - Code standards and import patterns +- [architecture.md](architecture.md) - System design -## πŸŽ“ Educational Goals +## Getting Help -This workflow teaches: -- Industry-standard branching (weekly/staging/main) -- Release management and quality gates -- Modular architecture patterns -- Collaborative development practices +- **[Issues](https://github.com/nyjc-computing/campus/issues)** - Bug reports and feature requests +- **[Discussions](https://github.com/nyjc-computing/campus/discussions)** - Questions +- **[Code Reviews](https://nyjc-computing.github.io/nanyang-system-developers/contributors/training/code-reviews.html)** - Best practices --- diff --git a/docs/GETTING-STARTED.md b/docs/GETTING-STARTED.md index 9019f8aa..d3942061 100644 --- a/docs/GETTING-STARTED.md +++ b/docs/GETTING-STARTED.md @@ -1,47 +1,44 @@ - # Getting Started with Campus -Welcome to Campus! This guide helps you navigate the documentation based on your role and experience level. - -## πŸ‘‹ New to Campus? - -**Start here:** -1. Read the [main README](../README.md) for project overview -2. Check out [Architecture](architecture.md) to understand the system design -3. Follow [Installation](#installation) steps below +This guide helps you navigate the documentation based on your role and experience level. -## 🎯 Quick Navigation by Role +## By Role -### πŸ“š End Users -- **[Deployment Guide](../DEPLOY.md)** β€” Deploy vault or full apps -- **API Documentation** β€” *Coming soon* -- **User Manual** β€” *Coming soon* - -### πŸ‘¨β€πŸ’» New Developers -- **[Contributing Guide](CONTRIBUTING.md)** β€” Development workflow and branch strategy -- **[Development Guidelines](development-guidelines.md)** β€” Coding patterns and best practices -- **[Style Guide](STYLE-GUIDE.md)** β€” Code and documentation standards -- **[Testing Strategies](testing-strategies.md)** β€” How to test your changes +### πŸ‘¨β€πŸ’» New Developers +Start here if you're new to the codebase: +1. [CONTRIBUTING.md](CONTRIBUTING.md) - Development workflow and branch strategy +2. [development-guidelines.md](development-guidelines.md) - Coding patterns and best practices +3. [STYLE-GUIDE.md](STYLE-GUIDE.md) - Code standards and import patterns +4. [TESTING-GUIDE.md](TESTING-GUIDE.md) - How to run and write tests ### πŸ—οΈ Experienced Developers -- **[Architecture](architecture.md)** β€” System design and service boundaries -- **[Packaging Guide](PACKAGING.md)** β€” Monorepo structure and distribution -- **[Development Guidelines](development-guidelines.md)** β€” Advanced patterns and abstractions +For those familiar with Python/Flask projects: +1. [architecture.md](architecture.md) - System design and service boundaries +2. [development-guidelines.md](development-guidelines.md) - Architecture patterns and abstractions +3. [PACKAGING.md](PACKAGING.md) - Monorepo structure and distribution ### πŸš€ DevOps/Infrastructure -- **[Deployment Guide](../DEPLOY.md)** β€” Deployment options -- **[Packaging Guide](PACKAGING.md)** β€” Build and distribution -- **[Testing Strategies](testing-strategies.md)** β€” Testing infrastructure +For deployment and operations: +1. [DEPLOY.md](../DEPLOY.md) - Deployment options +2. [PACKAGING.md](PACKAGING.md) - Build and distribution +3. [TESTING-GUIDE.md](TESTING-GUIDE.md) - Testing infrastructure + +### πŸ“š End Users +For those deploying or using Campus: +1. [README.md](../README.md) - Project overview +2. [DEPLOY.md](../DEPLOY.md) - Deployment guide +3. [architecture.md](architecture.md) - System design -## πŸ› οΈ Installation +## Installation ### Prerequisites - Python 3.11 or higher -- Poetry for dependency management -- PostgreSQL (for vault service) +- Poetry for dependency management +- PostgreSQL (for auth service database) - MongoDB (optional, for alternative storage backend) ### Quick Setup + ```bash # Clone and enter directory git clone https://github.com/nyjc-computing/campus.git @@ -55,37 +52,30 @@ poetry run python main.py ``` ### Environment Configuration + ```bash ENV="development" # deployment environment CLIENT_ID="your-client-id" # OAuth credentials -CLIENT_SECRET="your-client-secret" +CLIENT_SECRET="your-client-secret" POSTGRESDB_URI="postgresql://..." # auth service database ``` -Other configuration managed via `campus.auth.vaults`. - -## πŸ“– Documentation Index - -### Core Documentation -- **[Architecture](architecture.md)** β€” System design and component overview -- **[Contributing](CONTRIBUTING.md)** β€” Development workflow and guidelines -- **[Development Guidelines](development-guidelines.md)** β€” Coding patterns and best practices -- **[Style Guide](STYLE-GUIDE.md)** β€” Code and documentation standards -- **[Testing Strategies](testing-strategies.md)** β€” Testing approaches and tools -- **[Packaging](PACKAGING.md)** β€” Monorepo structure and distribution - -### Service-Specific Documentation -- **[Storage Layer](../campus/storage/README.md)** β€” Data persistence -- **[Common Utilities](../campus/common/README.md)** β€” Shared utilities +Other configuration is managed via `campus.auth.vaults`. -## πŸ†˜ Need Help? +## Documentation Index -- **πŸ“‹ Issues**: Report bugs via [GitHub Issues](https://github.com/nyjc-computing/campus/issues) -- **πŸ’¬ Discussions**: Ask questions in [GitHub Discussions](https://github.com/nyjc-computing/campus/discussions) -- **πŸ”’ Security**: Report vulnerabilities to [security@nyjc.edu.sg](mailto:security@nyjc.edu.sg) +| Document | Purpose | +|----------|---------| +| [AGENTS.md](../AGENTS.md) | Quick reference for humans and AI assistants | +| [architecture.md](architecture.md) | System design and component overview | +| [CONTRIBUTING.md](CONTRIBUTING.md) | Development workflow and branch strategy | +| [development-guidelines.md](development-guidelines.md) | Coding patterns and best practices | +| [STYLE-GUIDE.md](STYLE-GUIDE.md) | Code standards and documentation | +| [TESTING-GUIDE.md](TESTING-GUIDE.md) | Testing approaches and strategies | +| [PACKAGING.md](PACKAGING.md) | Monorepo structure and distribution | -## πŸƒβ€β™‚οΈ Next Steps +## Getting Help -1. **New Contributors**: Start with [CONTRIBUTING.md](CONTRIBUTING.md) -2. **Developers**: Review [development-guidelines.md](development-guidelines.md) -3. **Users**: Check out [DEPLOY.md](../DEPLOY.md) for deployment options +- **[Issues](https://github.com/nyjc-computing/campus/issues)** - Bug reports and feature requests +- **[Discussions](https://github.com/nyjc-computing/campus/discussions)** - Questions and discussions +- **[security@nyjc.edu.sg](mailto:security@nyjc.edu.sg)** - Security vulnerabilities diff --git a/docs/PACKAGING.md b/docs/PACKAGING.md index e7ba30ee..1520bf7d 100644 --- a/docs/PACKAGING.md +++ b/docs/PACKAGING.md @@ -6,24 +6,26 @@ Campus uses a **monorepo with centralized dependencies** approach. All component **Key Benefits:** - Consistent versioning across all services -- Simplified dependency management +- Simplified dependency management - Clean modular structure - No PyPI publishing delays -## Architecture Decisions +## Architecture ### Monorepo Structure -Campus consolidates all services in a single repository with unified dependency management in the root `pyproject.toml`. This solves several challenges: + +Campus consolidates all services in a single repository with unified dependency management in the root `pyproject.toml`: - Eliminates complex inter-service dependency management - Ensures compatibility between components - Simplifies distribution and testing ### Branch Strategy + ``` weekly β†’ staging β†’ main ``` - **`weekly`**: Active development, expected breakage -- **`staging`**: Extended testing, pre-production validation +- **`staging`**: Extended testing, pre-production validation - **`main`**: Stable, production-ready releases ## Dependency Rules @@ -34,7 +36,7 @@ storage β†’ common model β†’ common ``` -**Key Constraints**: +**Key Constraints:** - `auth` and `api` contain business logic in `.resources` submodules - `model` contains only entity definitions (minimal dependencies) - `storage` provides backend-agnostic persistence @@ -42,24 +44,29 @@ model β†’ common ## Installation -### Production Use +### For Production Use + +Add to your `pyproject.toml`: + ```toml [tool.poetry.dependencies] campus-suite = {git = "https://github.com/nyjc-computing/campus.git", branch = "main"} ``` -### Development Integration +### For Development Integration + ```toml -[tool.poetry.group.dev.dependencies] +[tool.poetry.group.dev-dependencies] campus-suite = {git = "https://github.com/nyjc-computing/campus.git", branch = "weekly"} ``` ### CLI Installation + ```bash -# Production +# Production (stable) poetry add git+https://github.com/nyjc-computing/campus.git@main -# Development +# Development (bleeding edge) poetry add git+https://github.com/nyjc-computing/campus.git@weekly --group dev # Specific commit (reproducible builds) @@ -67,30 +74,14 @@ poetry add git+https://github.com/nyjc-computing/campus.git@abc123def456 ``` ### Client Library Installation + ```bash # Add campus_python client library poetry add git+https://github.com/nyjc-computing/campus-api-python.git@main ``` -## Development Workflow - -### Contributing to Campus -```bash -# Clone repository -git clone https://github.com/nyjc-computing/campus.git -cd campus - -# Install dependencies -poetry install - -# Create feature branch from weekly -git checkout weekly -git checkout -b feature/new-feature - -# Make changes, test, submit PR to weekly -``` +## Using Campus in External Projects -### Using Campus in External Projects ```bash # Add Campus as dependency poetry add git+https://github.com/nyjc-computing/campus.git@main @@ -102,9 +93,10 @@ poetry add git+https://github.com/nyjc-computing/campus-api-python.git@main python -c "import campus_python; campus = campus_python.Campus()" ``` -## Building and Distribution +## Building and Validation ### Local Development + ```bash # Install dependencies poetry install @@ -117,13 +109,16 @@ poetry run python tests/run_tests.py unit ``` ### Release Process + 1. **Development**: Work in `weekly` branch 2. **Integration**: Merge `weekly` β†’ `staging` for extended testing 3. **Release**: Merge `staging` β†’ `main` when stable 4. **Tagging**: Create version tags on `main` branch ### Dependency Updates + Update dependencies in root `pyproject.toml`: + ```bash # Update a specific package poetry add requests@^2.31.0 @@ -137,21 +132,26 @@ poetry lock ## Troubleshooting -### Common Issues +### Import Errors + +Ensure you're using `poetry run python` instead of system Python. -**Import errors**: Ensure you're using `poetry run python` instead of system Python. +### Dependency Conflicts -**Dependency conflicts**: Update to latest main branch: +Update to latest main branch: ```bash poetry add campus-suite@git+https://github.com/nyjc-computing/campus.git@main ``` -**Authentication issues**: Verify GitHub access for private repositories: +### Authentication Issues + +Verify GitHub access for private repositories: ```bash git config --global url."https://username:token@github.com/".insteadOf "https://github.com/" ``` ### Validation Commands + ```bash # Check environment poetry env info @@ -160,27 +160,9 @@ poetry env info poetry run python -c "import campus" # Test modules -for module in auth api storage model common; do - poetry run python -c "import campus.$module" -done +poetry run python -c "import campus.auth; import campus.api; import campus.storage" ``` ## Why Git Dependencies? -### vs PyPI Publishing -- βœ… **No publishing overhead** - Changes available immediately -- βœ… **Development flexibility** - Can push breaking changes without version conflicts -- βœ… **Branch-based stability** - Multiple stability levels available -- ❌ **Dependency resolution** - Slightly slower than PyPI packages - -### vs Local Path Dependencies -- βœ… **External projects** - Works for projects outside Campus repo -- βœ… **External builds** - CI/CD works automatically -- βœ… **Version control** - External projects can pin specific commits -- ❌ **Network requirement** - Requires git access during install - -### Migration Path -This approach provides a clear evolution: -1. **Current**: Git dependencies for active development -2. **Future**: PyPI releases for stable, mature packages -3. **Hybrid**: Git for bleeding-edge, PyPI for stability +Campus currently uses Git-based dependencies for active development. This approach provides immediate availability of changes and branch-based stability levels. When Campus stabilizes, we plan to publish to PyPI for stable releases while maintaining Git dependencies for bleeding-edge access. diff --git a/docs/STYLE-GUIDE.md b/docs/STYLE-GUIDE.md index f6a169dc..f93c4604 100644 --- a/docs/STYLE-GUIDE.md +++ b/docs/STYLE-GUIDE.md @@ -1,30 +1,22 @@ # Campus Style Guide -This document defines coding and documentation standards for the Campus project. +This document defines coding standards for the Campus project. -## Documentation Standards +**For documentation standards**, see [development-guidelines.md](development-guidelines.md). -### Writing Style -- **Brief and precise**: Avoid verbose explanations; focus on essential information -- **Clear structure**: Use headings, lists, and code blocks to organize content -- **Consistent terminology**: Use established terms throughout the project -- **Active voice**: Prefer "Configure the database" over "The database should be configured" - -### Documentation Format -- Use Markdown for all documentation files -- Include table of contents for documents longer than 50 lines -- End code blocks with proper language specification (```python, ```bash, etc.) -- Use relative links for internal documentation references +**For code review checklist**, see [CONTRIBUTING.md](CONTRIBUTING.md). ## Python Code Standards ### Code Conventions + Follow established Python conventions: - **PEP 8**: Python code style guide - **PEP 257**: Docstring conventions - **Type hints**: Use type annotations for function parameters and return values ### Import Structure + Organize imports in this order: 1. **Built-in modules** (standard library) 2. **Third-party packages** (installed via pip/poetry) @@ -42,9 +34,9 @@ import flask import requests # Campus package imports -from campus.vault import get_vault -from campus.storage import PostgreSQLStorage +from campus.common import utils from campus.common.errors import CampusError +import campus.storage # Local imports from .models import User @@ -53,8 +45,6 @@ from .utils import validate_input ### Package Import Requirements -Organize imports in ascending order by name, uppercase then lowercase. - **Preferred**: Module-level imports ```python @@ -115,19 +105,19 @@ Use Google-style docstrings: ```python def create_user(name: str, email: str, role: str = "student") -> User: """Create a new user with the specified details. - + Args: name: Full name of the user email: Valid email address role: User role (student, teacher, admin) - + Returns: User object with generated ID and timestamps - + Raises: ValidationError: If email format is invalid CampusError: If user creation fails - + Example: >>> user = create_user("John Doe", "john@example.com", "student") >>> print(user.id) @@ -164,37 +154,3 @@ def validate_user_data(data: Dict) -> None: - **refactor**: Code refactoring - **test**: Adding or updating tests - **chore**: Build process or auxiliary tool changes - -## Common Pitfalls - -### Development Environment -- **Always use Poetry**: Run `poetry run python` instead of just `python` -- **Check your environment**: Use `poetry env info` to verify active environment - -### Import Issues -- **Avoid circular imports**: Structure imports to prevent dependency cycles -- **Use absolute imports**: Prefer `campus.vault` over relative imports in most cases - -### Testing -- **Use appropriate test strategy**: See [testing-strategies.md](testing-strategies.md) -- **Test edge cases**: Include error conditions and boundary values - -### Configuration -- **Never commit secrets**: Use `campus.auth.vaults` for sensitive data -- **Document configuration**: Update docs for new config options - -## Code Review Guidelines - -### Before Submitting -- [ ] All tests pass locally -- [ ] Code follows style guide -- [ ] Documentation is updated -- [ ] Commit messages follow format -- [ ] No secrets in code - -### Review Focus Areas -- **Security**: Check for potential vulnerabilities -- **Performance**: Look for inefficient operations -- **Maintainability**: Ensure code is readable and well-structured -- **Testing**: Verify adequate test coverage -- **Documentation**: Confirm docs match implementation diff --git a/docs/test-plan.md b/docs/TESTING-GUIDE.md similarity index 70% rename from docs/test-plan.md rename to docs/TESTING-GUIDE.md index 5f2398bf..e9afa67c 100644 --- a/docs/test-plan.md +++ b/docs/TESTING-GUIDE.md @@ -1,74 +1,29 @@ -# Campus Test Plan +# Campus Testing Guide -This document defines what we test, what we don't test, and how tests are organized. +This guide covers testing in Campus: what we test, how tests are organized, and how to run and write tests. -## Scope +## Critical Reminders -### What We Test - -| Category | Description | Examples | -|----------|-------------|----------| -| **HTTP contracts** | Status codes, response formats, authentication | 401 without auth, 409 for not found, 400 for bad input | -| **Business logic** | CRUD operations, validation, state transitions | Creating assignments, updating circles, token validation | -| **Data flow** | Request β†’ processing β†’ response | Vault get/set, OAuth flow, email OTP delivery | -| **Integration** | Cross-service interactions | Auth + API, API + Yapper, storage + resources | - -### What We Don't Test - -| Category | Rationale | -|----------|-----------| -| **HTTP layer** | Flask/Werkzeug handles HTTP correctly | -| **Network errors** | Platform responsibility (Railway, Replit, etc.) | -| **Middleware** | WSGI stack is trusted | -| **Flask bugs** | Not our responsibility to test Flask itself | -| **Deployment config** | Covered by manual testing on development environment | -| **Performance** | Out of scope for unit/integration tests | -| **Security auditing** | Out of scope (follow secure coding practices instead) | +- **Only standard library `unittest`**β€”no pytest or other test dependencies +- **Always use Poetry**: `poetry run python` for all test commands +- **Use `run_tests.py`**: The official test entrypoint that handles environment setup -### Testing Philosophy +## Quick Start -> **We test our code, not our dependencies.** +```bash +# Run all unit tests (fast, no external dependencies) +poetry run python tests/run_tests.py unit -Our tests verify that Campus API endpoints are implemented correctly. We trust that: -- Flask handles HTTP requests/responses correctly -- The WSGI server (Gunicorn) works as advertised -- The deployment platform (Railway) manages networking +# Run all integration tests (requires environment setup) +poetry run python tests/run_tests.py integration -## Test Organization +# Run all tests +poetry run python tests/run_tests.py all -``` -tests/ -β”œβ”€β”€ unit/ # Unit tests (fast, isolated) -β”‚ β”œβ”€β”€ auth/ # Auth service unit tests -β”‚ β”œβ”€β”€ api/ # API service unit tests -β”‚ β”œβ”€β”€ storage/ # Storage layer unit tests -β”‚ β”œβ”€β”€ yapper/ # Yapper unit tests -β”‚ └── common/ # Shared utilities tests -β”‚ -β”œβ”€β”€ integration/ # Integration tests (slower, cross-service) -β”‚ β”œβ”€β”€ auth/ # Auth service integration -β”‚ β”œβ”€β”€ api/ # API service integration -β”‚ └── yapper/ # Yapper integration -β”‚ -β”œβ”€β”€ contract/ # HTTP contract tests (interface invariants) -β”‚ β”œβ”€β”€ test_auth_vault.py # Vault endpoint contracts -β”‚ β”œβ”€β”€ test_auth_clients.py # Client CRUD contracts -β”‚ β”œβ”€β”€ test_auth_*.py # Other auth contracts -β”‚ └── test_api_*.py # API endpoint contracts -β”‚ -β”œβ”€β”€ fixtures/ # Shared test fixtures -β”‚ β”œβ”€β”€ services.py # ServiceManager for test coordination -β”‚ β”œβ”€β”€ tokens.py # Test token creation utilities -β”‚ └── auth.py # Auth service initialization -β”‚ -└── flask_test/ # Flask test client adapters - β”œβ”€β”€ campus_request.py # Test-compatible CampusRequest - β”œβ”€β”€ client.py # FlaskTestClient wrapper - └── response.py # FlaskTestResponse adapter +# Run specific test file +poetry run python -m unittest tests.unit.auth.test_resources -v ``` -See [tests/README.md](../tests/README.md) for detailed test documentation. - ## Test Types ### Unit Tests @@ -84,6 +39,12 @@ See [tests/README.md](../tests/README.md) for detailed test documentation. - Utility functions - Schema conversions +**Guidelines:** +- Test only internal logic of each package +- No environment dependencies or cross-package interactions +- Mock external dependencies (may mock `campus.common` classes) +- Must not mock package classes (test real implementations) + ### Integration Tests **Purpose:** Test multiple components working together. @@ -97,6 +58,11 @@ See [tests/README.md](../tests/README.md) for detailed test documentation. - Database operations - State management +**Guidelines:** +- Test package as a whole including DB, API, cross-package interactions +- Use `tests/fixtures/services.py` for environment setup +- Test real implementations with actual dependencies + **Note:** Integration tests use shared state within a test class for performance. Tests that depend on empty state should be named with `test_00_*` prefix to run first. ### Contract Tests @@ -112,40 +78,96 @@ See [tests/README.md](../tests/README.md) for detailed test documentation. - Response structure validation - HTTP status code correctness -See [tests/contract/README.md](../tests/contract/README.md) for invariants tested. +See [tests/contract/README.md](../tests/contract/README.md) for specific invariants tested. + +## Test Type Decision Tree + +``` +Are you testing HTTP interface contracts (status codes, auth)? +β”œβ”€β”€ Yes β†’ Contract Test (tests/contract/) +└── No + Are you testing cross-service interactions or database operations? + β”œβ”€β”€ Yes β†’ Integration Test (tests/integration/) + └── No β†’ Unit Test (tests/unit/) +``` + +**Summary:** +- **Contract tests**: HTTP invariants (401 without auth, error formats) +- **Integration tests**: Service-to-service, database operations +- **Unit tests**: Everything else (individual functions, classes) ## Running Tests +### Using the Test Runner (Recommended) + ```bash -# Run all unit tests (fast, no external dependencies) +# All tests +poetry run python tests/run_tests.py all + +# Unit tests only poetry run python tests/run_tests.py unit -# Run all integration tests (requires environment setup) +# Integration tests only poetry run python tests/run_tests.py integration -# Run all contract tests (HTTP interface invariants) -poetry run python -m unittest discover -s tests/contract -p "test_*.py" +# Sanity checks +poetry run python tests/run_tests.py sanity -# Run all tests -poetry run python tests/run_tests.py all +# Type checks +poetry run python tests/run_tests.py type +# Package-specific tests +poetry run python tests/run_tests.py unit --module auth +poetry run python tests/run_tests.py unit --module api +``` + +### Using unittest Directly + +```bash # Run specific test file -poetry run python -m unittest tests.contract.test_auth_vault -v +poetry run python -m unittest tests.unit.auth.test_resources -v -# Run specific test module -poetry run python tests/run_tests.py unit --module auth +# Run specific test class +poetry run python -m unittest tests.unit.auth.test_resources.TestAuthResource -v + +# Run specific test method +poetry run python -m unittest tests.unit.auth.test_resources.TestAuthResource.test_authenticate -v + +# Run all contract tests +poetry run python -m unittest discover -s tests/contract -p "test_*.py" ``` -## Test Fixtures +## Test Scope -Key fixtures in `tests/fixtures/`: +### What We Test -| Fixture | Purpose | -|---------|---------| -| `services.create_service_manager()` | Sets up Flask apps for testing | -| `create_test_token(user_id)` | Creates bearer tokens for authenticated tests | -| `get_basic_auth_headers(id, secret)` | Creates Basic auth headers | -| `get_bearer_auth_headers(token)` | Creates Bearer auth headers | +| Category | Description | Examples | +|----------|-------------|----------| +| **HTTP contracts** | Status codes, response formats, authentication | 401 without auth, 409 for not found, 400 for bad input | +| **Business logic** | CRUD operations, validation, state transitions | Creating assignments, updating circles, token validation | +| **Data flow** | Request β†’ processing β†’ response | Vault get/set, OAuth flow, email OTP delivery | +| **Integration** | Cross-service interactions | Auth + API, API + Yapper, storage + resources | + +### What We Don't Test + +| Category | Rationale | +|----------|-----------| +| **HTTP layer** | Flask/Werkzeug handles HTTP correctly | +| **Network errors** | Platform responsibility (Railway, Replit, etc.) | +| **Middleware** | WSGI stack is trusted | +| **Flask bugs** | Not our responsibility to test Flask itself | +| **Deployment config** | Covered by manual testing on development environment | +| **Performance** | Out of scope for unit/integration tests | +| **Security auditing** | Out of scope (follow secure coding practices instead) | + +### Testing Philosophy + +> **We test our code, not our dependencies.** + +Our tests verify that Campus API endpoints are implemented correctly. We trust that: +- Flask handles HTTP requests/responses correctly +- The WSGI server (Gunicorn) works as advertised +- The deployment platform (Railway) manages networking ## Test Environment @@ -161,6 +183,17 @@ Storage backends in test mode: - **Documents:** Python dicts - **Vault:** In-memory storage +## Test Fixtures + +Key fixtures in `tests/fixtures/`: + +| Fixture | Purpose | +|---------|---------| +| `services.create_service_manager()` | Sets up Flask apps for testing | +| `create_test_token(user_id)` | Creates bearer tokens for authenticated tests | +| `get_basic_auth_headers(id, secret)` | Creates Basic auth headers | +| `get_bearer_auth_headers(token)` | Creates Bearer auth headers | + ## Writing New Tests ### Unit Tests @@ -224,6 +257,13 @@ def test_create_resource(self): # ... creates a resource ``` +## Known Gotchas + +See [AGENTS.md](../AGENTS.md) for common testing pitfalls: +- Storage initialization order (lazy imports required) +- Flask blueprint registration (shared across test classes) +- Test cleanup requirements + ## External Testing For tests that require real HTTP or deployment verification: @@ -235,7 +275,7 @@ These are not automated because they require network access and external service ## Related Documentation -- [tests/README.md](../tests/README.md) - Detailed test directory documentation -- [tests/contract/README.md](../tests/contract/README.md) - Contract test invariants - [CONTRIBUTING.md](CONTRIBUTING.md) - Contribution guidelines - [development-guidelines.md](development-guidelines.md) - Development practices +- [STYLE-GUIDE.md](STYLE-GUIDE.md) - Code standards +- [tests/contract/README.md](../tests/contract/README.md) - Contract test invariants diff --git a/docs/architecture.md b/docs/architecture.md index 5c666374..2cbdbd96 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -25,16 +25,16 @@ campus/ ## Core Principles -- **πŸ”„ Separation of Concerns**: Each service has a single, well-defined responsibility -- **πŸ”Œ Loose Coupling**: Services communicate through clean interfaces -- **οΏ½ API-First**: Most actions have corresponding HTTP API endpoints -- **οΏ½ Extensible Storage**: Backend-agnostic storage interfaces support multiple implementations +- **Separation of Concerns**: Each service has a single, well-defined responsibility +- **Loose Coupling**: Services communicate through clean interfaces +- **API-First**: Most actions have corresponding HTTP API endpoints +- **Extensible Storage**: Backend-agnostic storage interfaces support multiple implementations ## Service Architecture ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ πŸ” Auth β”‚ β”‚ 🌐 API β”‚ β”‚ πŸ› οΈ Common β”‚ +β”‚ Auth β”‚ β”‚ API β”‚ β”‚ Common β”‚ β”‚ OAuth & β”‚ β”‚ RESTful β”‚ β”‚ Utilities β”‚ β”‚ Credentials β”‚ β”‚ Resources β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ @@ -44,7 +44,7 @@ campus/ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ πŸ›οΈ Model β”‚ β”‚ οΏ½ Services β”‚ β”‚ οΏ½ Integr. β”‚ +β”‚ Model β”‚ β”‚ Services β”‚ β”‚ Integrations β”‚ β”‚ Entities β”‚ β”‚ Email, etc. β”‚ β”‚ External β”‚ β”‚ (dataclasses) β”‚ β”‚ β”‚ β”‚ APIs β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ @@ -52,7 +52,7 @@ campus/ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” - β”‚ πŸ’Ύ Storage β”‚ + β”‚ Storage β”‚ β”‚ Data Layer β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` @@ -161,4 +161,4 @@ Three-tier testing strategy: 2. **Integration Tests**: Service-to-service communication testing 3. **End-to-End Tests**: Full application workflow testing -See [testing-strategies.md](testing-strategies.md) for detailed testing approaches. +See [TESTING-GUIDE.md](TESTING-GUIDE.md) for detailed testing approaches. diff --git a/docs/development-guidelines.md b/docs/development-guidelines.md index 77c359f5..1a56f665 100644 --- a/docs/development-guidelines.md +++ b/docs/development-guidelines.md @@ -1,57 +1,48 @@ # Development Guidelines -This document outlines key architectural patterns and development practices for Campus. Follow these guidelines to maintain consistency and ensure proper integration between services. +This document outlines key architectural patterns and development practices for Campus. -## Core Abstractions +**Looking for the contribution workflow?** See [CONTRIBUTING.md](CONTRIBUTING.md). -### Campus Authentication (`campus.auth`) +**For coding standards and import conventions**, see [STYLE-GUIDE.md](STYLE-GUIDE.md). -**Purpose**: Authentication, OAuth, and credential management. +## The Storage-Model-Resources Pattern -**Key Principles**: -- User authentication and session management -- OAuth proxy (Google, GitHub, Discord) -- Client authentication via CLIENT_ID/CLIENT_SECRET -- Business logic in `.resources` submodule +This is the foundational architectural pattern in Campus. Understanding this pattern is essential before working with any part of the codebase. -**Usage Pattern**: -```python -import campus_python +``` +Routes β†’ Resources β†’ Storage β†’ Model +``` -campus = campus_python.Campus() +### Model Layer (`campus.model`) -# Authenticate -auth_result = campus.auth.root.authenticate( - client_id="your_client_id", - client_secret="your_client_secret" -) +**Purpose:** Entity representation only - pure data structures with no business logic. -# Access secrets -secret = campus.auth.vaults["deployment"]["DATABASE_URL"] -``` +Models are dataclasses that define the shape of your data: -**Implementation**: ```python -# Within campus.auth -from campus.auth import resources -from campus import model +from dataclasses import dataclass -# Cross-service (e.g., in campus.api) -from campus.auth import resources as auth_resources +@dataclass +class User: + id: str + name: str + email: str + created_at: str ``` -### Campus Storage (`campus.storage`) +**Key principles:** +- Keyword-only init +- No methods beyond data conversion helpers (`to_storage()`, `to_resource()`) +- No business logic +- No dependencies on other campus packages (except `campus.common` for types) -**Purpose**: Unified data persistence layer with multiple backend support. +### Storage Layer (`campus.storage`) -**Storage Interface Requirements**: -All storage implementations must provide: -- `id` field - unique identifier (string) -- `created_at` field - creation timestamp (RFC3339 UTC) -- Standard CRUD operations -- Consistent error handling +**Purpose:** Backend-agnostic data persistence with multiple backend support. + +Storage provides CRUD operations independent of the underlying database: -**Usage Pattern**: ```python import campus.storage @@ -59,78 +50,196 @@ import campus.storage users = campus.storage.get_collection("users") user = users.create({"name": "John", "email": "john@example.com"}) -# Table storage (SQL-style) +# Table storage (SQL-style) sessions = campus.storage.get_table("sessions") session = sessions.find_by_id("session_123") ``` -**Backend Abstraction**: +**Storage interface requirements:** +- `id` field - unique identifier (string) +- `created_at` field - creation timestamp (RFC3339 UTC) +- Standard CRUD operations +- Consistent error handling + +### Resources Layer (`.resources/`) + +**Purpose:** Business logic - the "brains" of your application. + +Resources are classes that encapsulate business operations. They bridge the gap between HTTP requests and data storage. + +**Resource types:** + +1. **Collection Resources** - Operate on multiple items ```python -from abc import ABC, abstractmethod +class CirclesResource: + """Represents the circles resource in Campus API Schema.""" + + def list(self, **filters) -> list[Circle]: + """List all circles matching filters.""" + records = circle_storage.get_matching(filters) + return [_from_record(r) for r in records] + + def new(self, **fields) -> Circle: + """Create a new circle with validation.""" + # Validation logic here + # Storage operations here + return circle +``` -class StorageInterface(ABC): - @abstractmethod - def create(self, data: dict) -> dict: - pass - - @abstractmethod - def find_by_id(self, doc_id: str) -> dict: - pass +2. **Item Resources** - Operate on single items +```python +class CircleResource: + """Represents a single circle in Campus API Schema.""" -class MongoDBStorage(StorageInterface): - def create(self, data: dict) -> dict: - # Implementation - pass + def __init__(self, circle_id: CampusID): + self.circle_id = circle_id + + def get(self) -> Circle: + """Get the circle record.""" + record = circle_storage.get_by_id(self.circle_id) + return _from_record(record) + + def update(self, **updates) -> None: + """Update the circle record.""" + circle_storage.update_by_id(self.circle_id, updates) + + def delete(self) -> None: + """Delete the circle record.""" + circle_storage.delete_by_id(self.circle_id) ``` -### Storage-Model-Resources Pattern +3. **Sub-Resources** - Nested resources under a parent +```python +class CircleMembersResource: + """Represents the circle members resource.""" + + def __init__(self, parent: CirclesResource): + self._parent = parent -**Architecture**: Separate data persistence, entity representation, and business logic. + def list(self, circle_id: CampusID) -> dict: + """List all members of a circle.""" + # Implementation + def add(self, circle_id: CampusID, member_id: CampusID, access_value: int) -> None: + """Add a member to a circle.""" + # Implementation ``` -Routes β†’ Resources (.resources/) β†’ Storage β†’ Model + +**Resources are exported from `__init__.py`:** +```python +# campus/api/resources/__init__.py +from .circle import CirclesResource +circle = CirclesResource() # Module-level instance ``` -**Implementation**: +### Routes Layer (`.routes/`) + +**Purpose:** HTTP endpoints - thin wrappers that call Resource methods. + +Routes use Flask blueprints and delegate to resource instances: + ```python -# Model layer - entity representation (campus.model) -from dataclasses import dataclass +from campus.api import resources -@dataclass -class User: - id: str - name: str - email: str - created_at: str +bp = flask.Blueprint('circles', __name__, url_prefix='/circles') -# Resources layer - business logic (campus.auth.resources) -from campus.auth import resources +@bp.get('/') +def list_circles(tag: str | None = None): + """List all circles matching filter requirements.""" + result = resources.circle.list(**{"tag": tag} if tag else {}) + return {"data": [c.to_resource() for c in result]}, 200 + +@bp.post('/') +def new_circle(name: str, description: str, tag: str, parents: dict | None = None): + """Create a new circle.""" + circle = resources.circle.new(name=name, description=description, tag=tag, parents=parents) + return {"data": circle.to_resource()}, 201 +``` + +**Key principles for routes:** +- Minimal logic - just parameter extraction and response formatting +- Delegate all business logic to Resources +- Use `flask_campus.unpack_request` decorator for request parsing + +## Package Structure + +``` +campus/ +β”œβ”€β”€ auth/ # Authentication and OAuth services +β”‚ β”œβ”€β”€ oauth_proxy/ +β”‚ β”œβ”€β”€ resources/ # Business logic +β”‚ └── routes/ # HTTP endpoints +β”œβ”€β”€ api/ # RESTful API resources +β”‚ β”œβ”€β”€ resources/ # Business logic +β”‚ └── routes/ # HTTP endpoints +β”œβ”€β”€ common/ # Shared utilities +β”œβ”€β”€ model/ # Entity representation (dataclasses) +β”œβ”€β”€ services/ # Business services (email, etc.) +β”œβ”€β”€ storage/ # Data persistence layer +β”œβ”€β”€ integrations/ # External service integrations +└── yapper/ # Logging framework +``` + +**Key rule:** Business logic lives in `.resources/` submodules. Models (`campus.model`) are pure data structures. + +## Core Abstractions + +### Campus Storage (`campus.storage`) + +Backend-agnostic data persistence with multiple backend support. + +**Usage:** +```python import campus.storage -def create_user(data: dict) -> dict: - # Validation logic - users = campus.storage.get_collection("users") - return users.create(data) +# Document storage (MongoDB-style) +users = campus.storage.get_collection("users") +user = users.create({"name": "John", "email": "john@example.com"}) -# Routes layer - HTTP endpoints +# Table storage (SQL-style) +sessions = campus.storage.get_table("sessions") +session = sessions.find_by_id("session_123") +``` + +### Campus Authentication (`campus.auth`) + +**Purpose:** Authentication, OAuth, and credential management. + +**Via campus_python client:** +```python +import campus_python + +campus = campus_python.Campus() + +# Authenticate +auth_result = campus.auth.root.authenticate( + client_id="your_client_id", + client_secret="your-client_secret" +) + +# Access secrets +secret = campus.auth.vaults["deployment"]["DATABASE_URL"] +``` + +**Within codebase:** +```python +# Within same service from campus.auth import resources -@app.route('/users', methods=['POST']) -def create_user(): - return resources.create_user(request.json) +# Cross-service (e.g., from campus.api) +from campus.auth import resources as auth_resources ``` ### Campus Common (`campus.common`) -**Purpose**: Shared utilities used across services. +Shared utilities used across services. -**Key Modules**: +**Key modules:** - `utils` - ID generation, time handling - `devops` - Environment detection - `errors` - Standardized error types - `http` - HTTP utilities -**Usage Pattern**: ```python from campus.common import utils, devops, errors @@ -141,14 +250,8 @@ env = devops.ENV ### Entity Models (`campus.model`) -**Purpose**: Entity representation only - no business logic. - -**Key Principles**: -- Dataclass definitions (User, Circle, Client, Session, Token, etc.) -- Keyword-only init -- No business logic +**Purpose:** Entity representation only - no business logic. -**Usage Pattern**: ```python from campus import model @@ -156,13 +259,11 @@ def process_user(user: model.User) -> dict: return {"id": user.id, "name": user.name} ``` -Business logic belongs in `.resources` submodules, not `campus.model`. - ## Development Patterns ### Interface-First Design -**Pattern**: Define abstract interfaces for polymorphic concrete classes. +Define abstract interfaces for polymorphic concrete classes. ```python from abc import ABC, abstractmethod @@ -178,125 +279,90 @@ class SMTPSender(EmailSender): pass ``` -**Benefits**: +**Benefits:** - Enables swappable implementations - Easier testing with mock implementations - Clear contracts between components -### Function Calling Conventions - -**Prefer module-level imports**: -```python -# Within same service -from campus.auth import resources +## Common Pitfalls -# Cross-service (e.g., from campus.api) -from campus.auth import resources as auth_resources -from campus.api import resources +### Storage Initialization Order -# Common utilities -from campus.common import utils, devops -``` +Test fixtures must lazy-import `campus.storage` modules. Otherwise storage backends initialize before test mode is set. -**Avoid importing individual functions**: ```python -# Avoid - loses context -from campus.common.utils import uid, utc_time -from campus.auth.resources import create_user +# Bad - initializes storage immediately +from campus.storage import tables -uid() # Unclear origin +# Good - lazy import +def test_something(): + from campus.storage import tables ``` -## Common Pitfalls - -### Environment Issues -- **Always use Poetry**: Run `poetry run python` instead of bare `python` -- **Check environment**: Use `poetry env info` to verify active environment -- **Install dependencies**: Run `poetry install` after pulling changes - ### Import Problems -- **Circular imports**: Structure dependencies to prevent cycles -- **Package imports**: Import packages, not individual functions (see [Style Guide](STYLE-GUIDE.md)) -- **Standard library shadowing**: Avoid directory names like `collections/`, `json/` -### Configuration Errors -- **Use campus_python client**: Access config via `campus.auth.vaults` -- **Environment variables**: Only for deployment (ENV, DEPLOY, CLIENT_ID, CLIENT_SECRET, POSTGRESDB_URI) -- **Secrets management**: Access via `campus_python.Campus().auth.vaults[deployment][key]` +- **Circular imports:** Structure dependencies to prevent cycles +- **Standard library shadowing:** Avoid directory names like `collections/`, `json/`, `os/` +- **Package imports:** Import packages, not individual functions (see [STYLE-GUIDE.md](STYLE-GUIDE.md)) ### Architecture Violations -- **Business logic in models**: Keep `campus.model` as pure data structures; put logic in `.resources` -- **Missing .resources**: Each service (auth, api) should have its business logic in `.resources` submodule -### Testing Issues -- **Use appropriate strategy**: See [Testing Strategies](testing-strategies.md) -- **Mock external dependencies**: Don't hit real databases/APIs in unit tests -- **Import testing**: Test that classes can be imported without external resources +- **Business logic in models:** Keep `campus.model` as pure data structures; put logic in `.resources` +- **Missing .resources:** Each service (auth, api) should have business logic in `.resources` submodule ### Storage Interface Violations -- **Missing required fields**: All storage objects need `id` and `created_at` -- **Inconsistent error handling**: Use standardized error types from `campus.common.errors` -- **Direct database access**: Use storage interfaces instead of direct DB connections -## Development Workflow +- **Missing required fields:** All storage objects need `id` and `created_at` +- **Inconsistent error handling:** Use standardized error types from `campus.common.errors` +- **Direct database access:** Use storage interfaces instead of direct DB connections + +### Configuration -### Adding New Features +- **Use campus_python client:** Access config via `campus.auth.vaults` +- **Environment variables:** Only for deployment (ENV, DEPLOY, CLIENT_ID, CLIENT_SECRET, POSTGRESDB_URI) +- **Secrets management:** Access via `campus_python.Campus().auth.vaults[deployment][key]` + +## Adding New Features 1. **Design interfaces first** - Define abstract classes for new functionality -2. **Follow storage pattern** - Implement storage-model-views separation -3. **Test thoroughly** - Use appropriate testing strategy for your changes +2. **Follow storage pattern** - Implement storage-model-resources separation +3. **Test thoroughly** - Use appropriate testing strategy (see [TESTING-GUIDE.md](TESTING-GUIDE.md)) 4. **Update documentation** - Include usage examples and interface contracts -### Modifying Existing Code +## Modifying Existing Code 1. **Understand dependencies** - Check which services depend on your changes 2. **Maintain interfaces** - Don't break existing abstract contracts 3. **Test backwards compatibility** - Ensure existing code still works 4. **Update all implementations** - If changing interfaces, update all backends -### Testing and CI/CD - -Campus provides three testing strategies: - -1. **Development Server Testing** - Live Railway services -2. **Local Service Testing** - Local background services (currently broken) -3. **Flask Test Client Testing** - In-process, no network (fastest) - -See [Testing Strategies](testing-strategies.md) for details. - -### Import Shadowing Prevention - -**Critical**: Avoid directory names that shadow Python standard library modules. - -**Examples to Avoid**: -- `collections/` (shadows `collections` module) -- `json/` (shadows `json` module) -- `os/` (shadows `os` module) +## Documentation Standards -**Solution**: Use descriptive names like `document_collections/`, `json_utils/`, etc. +### Writing Style -## Documentation Standards +- **Brief and precise**: Avoid verbose explanations; focus on essential information +- **Clear structure**: Use headings, lists, and code blocks to organize content +- **Consistent terminology**: Use established terms throughout the project +- **Active voice**: Prefer "Configure the database" over "The database should be configured" ### Docstring Requirements -**Pattern**: Use consistent docstring format. - ```python def function_name(param: str) -> dict: """Brief description of the function. - + Longer description if needed. Explain the purpose, behavior, and any important details. - + Args: param: Description of the parameter - + Returns: Description of the return value - + Raises: SpecificError: When this specific condition occurs - + Example: >>> result = function_name("test") >>> print(result) @@ -306,7 +372,6 @@ def function_name(param: str) -> dict: ### Code Comments -**Guidelines**: - Explain **why**, not **what** - Document non-obvious design decisions - Include rationale for architectural choices @@ -314,13 +379,4 @@ def function_name(param: str) -> dict: --- -## Contributing - -When contributing to Campus: - -1. **Follow the established patterns** documented above -2. **Add new patterns** to this document when you establish them -3. **Test your changes** in isolation to ensure they follow lazy loading principles -4. **Document your design decisions** in code comments - This document is living and should be updated as the project evolves. diff --git a/tests/README.md b/tests/README.md index 43e143e8..c101117e 100644 --- a/tests/README.md +++ b/tests/README.md @@ -1,6 +1,8 @@ # Campus Test Directories +> **Comprehensive testing guide:** See [docs/TESTING-GUIDE.md](../docs/TESTING-GUIDE.md) for complete testing documentation including what we test, how tests are organized, and how to write new tests. + ## Critical Information - The test suite **only uses the standard library `unittest` module**β€”no other test dependencies are required or supported. @@ -186,3 +188,13 @@ python tests/run_tests.py integration --module auth # Verbose output python tests/run_tests.py unit -v ``` + +## More Information + +For complete testing documentation including: +- What we test and what we don't test +- How to write new tests +- Test environment setup +- Known gotchas + +See [docs/TESTING-GUIDE.md](../docs/TESTING-GUIDE.md).```