Skip to content

✨ (providers): Implement Phase 1.2 Provider Engine#412

Open
juftin wants to merge 1 commit into
webfrom
implement-1-2-provider-engine
Open

✨ (providers): Implement Phase 1.2 Provider Engine#412
juftin wants to merge 1 commit into
webfrom
implement-1-2-provider-engine

Conversation

@juftin
Copy link
Copy Markdown
Owner

@juftin juftin commented May 28, 2026

Summary

Implement the Phase 1.2 "Provider Engine" async architecture and fully migrate the Recreation.gov provider to a strongly-typed, high-performance parallel async implementation.

Context

Transitioning the legacy command line interface into a robust, modern full-stack web application requires a highly performant, async-first, and de-duplicated campsite polling worker. This PR establishes the core provider abstraction framework and completely refactors the Recreation.gov provider into Pydantic v2 data models, standard StrEnum parameters, and safe asyncio.gather parallel queries.

Changes

Code Changes

  • backend/packages/providers/providers/dto.py
    • Created standardized CampsiteType and CampsiteDTO models to represent unified data transfer objects returned by all campsite booking APIs.
  • backend/packages/providers/providers/base.py
    • Updated the abstract BaseProvider to declare async signatures for find_availabilities, sync_metadata, and populate_database.
    • Added support for dynamic, randomized Chrome User-Agent generation via fake-useragent on initialization.
  • backend/packages/providers/providers/recreation_gov/config.py
    • Extracted Recreation.gov specific base hosts, endpoint routes, strftime formatters, and cache timeouts into a cohesive RecreationGovConfig class utilizing urllib urljoins.
  • backend/packages/providers/providers/recreation_gov/models/enums.py
    • Defined enums for availability status codes, equipment type keywords, and campsite categories to eliminate all hardcoded string lookup loops.
  • backend/packages/providers/providers/recreation_gov/models/api.py
    • Built raw Pydantic v2 data structures (RecDotGovCampsite, CampsiteAvailabilityResponse) and request query parameter models (RecDotGovCampsiteSearchParams, RecDotGovAvailabilityParams).
  • backend/packages/providers/providers/recreation_gov/provider.py
    • Fully refactored RecreationGovProvider using async-first HTTPX.
    • Implemented fast parallel paging for metadata and month-level availability searches.
    • Decomposed _process_single_campsite into modular, highly-focused helper methods (_get_available_dates, _create_campsite_dto) with type-safe typing.cast bindings.
    • Exclusively utilized keyword arguments for internal class helper invocations.
  • backend/packages/providers/tests/test_recreation_gov.py
    • Added integration and unit tests using pytest-vcr (VCRPy) without mocking the client, verifying actual API contract behavior.

Test Plan

  • Initial verification (completed by agent or user).
    • Executed static analysis & linting checks: task check and task lint passed with 0 errors across 51 files.
    • Executed monorepo test suite: task test passed all 13 tests in 0.58 seconds.
  • Manual verification step.

Behavior Diagram

sequenceDiagram
    participant W as Worker
    participant P as RecreationGovProvider
    participant API as Recreation.gov API

    W->>P: find_availabilities(park_id, start_date, end_date)
    rect rgb(30, 30, 30)
        Note over P: Determine months & generate Chrome User-Agent
        par Page Metadata
            P->>API: GET api/search/campsites (RecDotGovCampsiteSearchParams)
            API-->>P: RecDotGovCampsiteResponse
        and Month Availabilities
            loop For each Month
                P->>API: GET api/camps/availability/campground/{id}/month (RecDotGovAvailabilityParams)
                API-->>P: CampsiteAvailabilityResponse
            end
        end
    end
    Note over P: type-safe cast gather results
    P->>P: Decompose & match metadata (CampsiteDTO keyword args)
    P-->>W: list[CampsiteDTO]
Loading

- Created CampsiteDTO and BaseProvider ABC.
- Implemented RecreationGovProvider async engine with optimized chunked paging.
- Replaced raw dictionaries with typed Pydantic models for search/availability parameters and metadata.
- Resolved type safety across asyncio.gather with explicit casts.
- Added comprehensive pytest-vcr integration tests.
@juftin juftin changed the base branch from main to web May 28, 2026 05:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant