Skip to content

architecture.md

Latest commit

 

History

History
1432 lines (1246 loc) · 67.5 KB

architecture.md

File metadata and controls

1432 lines (1246 loc) · 67.5 KB

Technical Design Document: Scuba Dive Site & Center Review Platform

Table of Contents

  1. Introduction
  2. Goals
  3. Functional Requirements
  4. Non-Functional Requirements
  5. Architecture
  6. API Endpoints
  7. Technologies & Tools
  8. Future Considerations
  9. Implementation Phases
  10. Security Considerations
  11. Privacy and Data Protection Features
  12. Error Handling and Logging
  13. Testing Infrastructure
  14. Current Implementation Status

1. Introduction

This document outlines the technical design for a Python-based web application, with future mobile application compatibility, dedicated to scuba diving site and center reviews. The platform will allow users to rate dive sites, find detailed information about them, discover diving centers and their offerings, view upcoming dive trips, and log their personal dives with detailed information.

2. Goals

  • Provide a comprehensive database of scuba diving sites with rich multimedia content and practical information.
  • Enable users to rate dive sites and diving centers.
  • Facilitate user interaction through comments on rated entities (for eponymous users).
  • Offer a directory of scuba diving centers, including pricing and associated dive sites.
  • Implement a system for parsing dive store newsletters to extract and display upcoming dive trip information on an interactive map.
  • Provide contact mechanisms for users to book dives with centers.
  • Enable users to log and track their personal dives with detailed information.
  • Allow users to claim ownership of diving centers with admin approval.
  • Design for scalability and future expansion to a mobile application.
  • Implement secure authentication with Google OAuth for enhanced user experience.

3. Functional Requirements

3.1. User Management

  • User registration and authentication (email/password, Google OAuth).
  • User profiles (displaying user's ratings, comments, dives).
  • Password reset functionality.
  • Google OAuth integration for secure authentication.

3.2. Dive Site Management

  • Add/Edit Dive Site (Admin/Moderator Functionality):
    • Name, description.
    • Location (GPS coordinates, address).
    • Access instructions (shore, boat details).
    • Example photos and videos (upload and display).
    • Difficulty level.
    • Marine life encountered (optional, could be free text or predefined tags).
    • Safety information.
    • Aliases system for enhanced search and newsletter parsing.
  • View Dive Site:
    • Display all aforementioned details.
    • Average user rating.
    • List of associated diving centers that visit this site.
    • User comments (eponymous users only).
    • List of dives logged at this site.
  • Rate Dive Site:
    • Score from 1 to 10.
    • One rating per user per site.
  • URL Routing:
    • Access dive sites via /dive-sites/dive-site-name or /dive-sites/alias
    • Fallback to ID-based routing for compatibility.

3.3. Diving Center Management

  • Add/Edit Diving Center (Admin/Moderator Functionality):
    • Name, description.
    • Contact information (email, phone, website).
    • Location.
    • List of dive sites they visit.
      • Dive cost per site (with multi-currency support).
    • Scuba gear rental costs (full set, individual items) with currency selection.
    • Tank rental costs (per type/size) with currency selection.
  • View Diving Center:
    • Display all aforementioned details.
    • Average user rating.
    • List of dive sites they visit with costs.
    • User comments (eponymous users only).
  • Rate Diving Center:
    • Score from 1 to 10.
    • One rating per user per center.
  • Diving Center Ownership:
    • Users can claim ownership of a diving center.
    • Admins must approve ownership claims.
    • Admins can directly assign users as diving center owners.
    • Approved owners can edit their diving center details regardless of admin status.

3.4. Dive Logging System

  • Create/Edit Dive (User Functionality):
    • Link to existing dive site or create new dive site.
    • Dive information (text form for detailed description).
    • Dive plan media upload (PDF, JPG, PNG).
    • Max depth (in meters/feet).
    • Average depth (in meters/feet).
    • Gas bottles used (type, size, pressure).
    • Suit type used (Wet suit, Dry suit, Shortie).
    • Difficulty level (Beginner, Intermediate, Advanced, Expert).
    • Visibility rating (1 to 10).
    • User rating (1 to 10).
    • Media upload/links (pictures, videos, external links).
    • Tags (using same categories as dive sites).
    • Date and time of dive.
    • Duration of dive.
  • View Dive:
    • Display all dive information.
    • Link to associated dive site.
    • Media gallery for dive plan, photos, videos, and external links.
    • Tag display.
    • User can edit their own dives.
  • Dive Management:
    • Users can view all their logged dives.
    • Search and filter dives by various criteria.
    • Export dive logs.
    • Media management (upload, delete, organize).

3.5. Comments

  • Users can leave comments on dive sites and diving centers.
  • Only eponymous users (logged-in users with a verified identity/profile name) can leave comments.
  • Comments are associated with the user and the rated entity.
  • Comments can be edited/deleted by the original author or by administrators.

3.6. Newsletter Parsing & Dive Trip Display

  • Newsletter Upload/Submission (Admin Functionality):
    • Mechanism to upload or submit dive store newsletters (e.g., email attachment, direct text paste).
    • Support for multiple newsletter formats (PDF, DOCX, TXT, HTML).
    • Batch processing of multiple newsletters.
  • Automated Parsing:
    • Identify diving centers using NLP and pattern matching.
    • Identify dive sites by matching against existing database entries.
    • Extract dates and times of scheduled dives with natural language processing.
    • Parse pricing information and special offers.
    • Extract trip details (duration, difficulty level, group size).
  • Data Validation & Storage:
    • Validate parsed data against existing dive sites and diving centers.
    • Store parsed trips in parsed_dive_trips table with proper relationships.
    • Handle duplicate trip detection and merging.
    • Maintain audit trail of parsing operations.
  • Map Integration:
    • Display parsed dive trips on an interactive map (e.g., Google Maps, OpenStreetMap).
    • Markers for dive sites showing upcoming trips.
    • Clicking a marker reveals details: dive center, date, time, cost (if available).
    • Color-coded markers for different diving centers.
  • Booking/Contact:
    • Link to dive center's email or phone number for booking.
    • Direct contact forms for trip inquiries.
    • Integration with diving center booking systems (future enhancement).

3.7. Dive Trip Calendar System

  • Calendar Interface:
    • Interactive calendar widget for navigating through dates.
    • Monthly, weekly, and daily view options.
    • Date range selection for planning multiple-day trips.
    • Responsive design for mobile and desktop use.
  • Trip Listing View:
    • Chronological list of all dive trips for selected date(s).
    • Grouped by diving center for easy comparison.
    • Detailed trip information display:
      • Diving center name and contact information.
      • Dive site name and location.
      • Trip date and time.
      • Duration and difficulty level.
      • Pricing information with currency support.
      • Group size limits and availability.
      • Special requirements or notes.
  • Map View Integration:
    • Toggle between calendar and map views.
    • Map displays all dive trips for selected date(s).
    • Different markers for each diving center.
    • Click markers to view trip details and booking options.
    • Route planning between multiple dive sites.
  • Advanced Filtering & Search:
    • Filter trips by diving center, dive site, or date range.
    • Search for specific dive sites or diving centers.
    • Filter by difficulty level, price range, or group size.
    • Sort by date, price, or diving center rating.
  • Trip Management Features:
    • Save favorite trips for quick access.
    • Share trip information via social media or email.
    • Export trip calendar to personal calendar applications.
    • Set up notifications for new trips from preferred diving centers.
  • User Experience Enhancements:
    • Quick booking buttons for direct contact.
    • Trip comparison tools for multiple options.
    • Weather integration for trip planning.
    • User reviews and ratings for specific trips.
    • Photo galleries from previous trips to same sites.
  • Admin Management:
    • Manual trip creation and editing interface.
    • Bulk import of trip data from external sources.
    • Trip approval workflow for diving center submissions.
    • Analytics dashboard for trip popularity and booking trends.
  • Mobile Optimization:
    • Touch-friendly calendar navigation.
    • Swipe gestures for date navigation.
    • Offline access to saved trips.
    • Push notifications for trip updates.
  • Integration Features:
    • Google Calendar integration for trip scheduling.
    • WhatsApp/Telegram integration for direct booking.
    • Payment processing integration (future enhancement).
    • Weather API integration for trip planning.

3.8. Privacy and Data Protection System ✅ COMPLETED

  • User Data Management:
    • Data Export and Portability: ✅ Users can export all their personal data, dive records, and user-generated content in multiple formats (JSON, CSV, PDF).
    • Data Access and Correction: ✅ Users can view, update, and correct all personal information held by the platform through dedicated interface.
    • Data Deletion: ✅ Users can request complete removal of their personal data with automatic cleanup within 30 days and confirmation process.
    • Consent Management: ✅ Users can manage consent for data processing and withdraw consent at any time with immediate effect.
  • Privacy Controls:
    • Privacy Settings Dashboard: ✅ Comprehensive user-configurable privacy preferences and data sharing controls.
    • Communication Preferences: ✅ Granular control over email notifications, updates, marketing communications, and newsletter subscriptions.
    • Data Visibility Controls: ✅ Fine-grained management of who can see user-generated content and profile information.
    • Account Privacy: ✅ Options for public or private profiles and content with selective sharing capabilities.
  • Data Protection Features:
    • GDPR Compliance Tools: ✅ Complete implementation of all GDPR user rights and data protection requirements.
    • Data Retention Policies: ✅ Automated enforcement of data retention policies with configurable cleanup schedules and legal compliance.
    • Audit Trail: ✅ Complete logging of all data access, modifications, and deletions for compliance, security, and transparency.
    • Privacy Impact Assessments: ✅ Tools for evaluating and documenting privacy implications of new features and system changes.
    • Data Minimization: ✅ Collection and processing of only necessary data for specified purposes.
    • Purpose Limitation: ✅ Data used only for explicitly stated and legitimate purposes.
  • Privacy Technologies:
    • Cloudflare Turnstile Integration: ✅ Privacy-preserving bot protection without personal data collection or tracking.
    • OpenLayers Mapping: ✅ Client-side mapping with no third-party location data sharing or external tracking.
    • Encrypted Storage: ✅ All sensitive data encrypted at rest and in transit using industry-standard encryption.
    • Secure Authentication: ✅ Multi-factor authentication support and secure session management with privacy protection.
    • Cookie Management: ✅ Minimal cookie usage with user consent and transparency about data collection.
  • User Rights Implementation:
    • Right to Information: ✅ Clear documentation of data collection, processing, and usage practices.
    • Right to Access: ✅ Complete access to all personal data held by the platform.
    • Right to Rectification: ✅ Easy correction and updating of personal information.
    • Right to Erasure: ✅ Comprehensive data deletion with verification and confirmation.
    • Right to Restrict Processing: ✅ Options to limit or suspend data processing activities.
    • Right to Data Portability: ✅ Export capabilities in standard, machine-readable formats.
    • Right to Object: ✅ Mechanisms to object to specific types of data processing.

3.9. Search and Filtering

  • Basic Search: Search dive sites by name, location, difficulty.
  • Advanced Search: Full-text search across multiple fields with location-based filtering.
  • Diving Center Search: Search by name, location, associated dive sites.
  • Dive Search: Search by various criteria (depth, date, location, tags).
  • Rating Filtering: Filter dive sites/centers by average rating.
  • Advanced Trip Search: Full-text search across trip descriptions, special requirements, diving center names, dive site names, and dive descriptions.
  • Location-Based Filtering: Filter by country, region, and address with geocoding support.
  • Duration Filtering: Filter trips by minimum and maximum duration.
  • Advanced Sorting: Sort by date, price, duration, difficulty, popularity, and distance from user location.
  • Distance Calculations: Haversine formula implementation for accurate geographic distance calculations.
  • Pagination Support: Efficient handling of large datasets with skip/limit parameters.
  • User Location Integration: Geolocation API support with manual coordinate input fallback.

3.10. Multi-Currency Support System

  • Supported Currencies: 10 major world currencies (USD, EUR, JPY, GBP, CNY, AUD, CAD, CHF, HKD, NZD)
  • Default Currency: Euro (€) is the default currency for all cost fields
  • Currency Display: Proper formatting with currency symbols and flags
  • Flexible Input: Users can submit costs in any supported currency

3.11. Database Migration System

  • Alembic Integration: All database schema changes must use Alembic for version control
  • Automatic Migration Execution: Migrations run automatically before application startup
  • Environment Compatibility: Supports both development and Docker environments
  • Health Checks: Database availability verification before migration execution
  • Rollback Support: Full migration history with downgrade capabilities
  • Currency Validation: 3-letter ISO currency code validation
  • Database Storage: Currency fields with indexes for performance
  • API Integration: All cost-related endpoints support currency
  • Frontend Utility: Comprehensive currency formatting and selection functions

3.12. Admin Management System

  • Mass Operations: Bulk delete functionality for admin management pages
  • User Management: Complete user CRUD with role assignment and status control
  • Tag Management: Comprehensive tag system with usage statistics
  • Safety Features: Protection against deleting used tags and self-deletion
  • Diving Center Ownership Management: Approve/deny ownership claims and assign owners
  • General Statistics Dashboard: Detailed platform statistics and engagement metrics
  • System Metrics Dashboard: Comprehensive system health and infrastructure monitoring
  • Recent Activity Monitoring: Real-time tracking of user actions and system changes
  • Backup and Export Management: Data export capabilities and backup management

3.13. Admin Dashboard Pages

3.13.1. System Statistics & Metrics

The General Statistics and System Metrics dashboards provide administrators with comprehensive platform statistics and health monitoring capabilities:

General Statistics Dashboard:

  • User Statistics: Total users, active users (last 7/30 days), new registrations (last 7/30 days), user growth rate, email verification status
  • Content Statistics: Total dive sites, diving centers, dives, routes, trips, comments, ratings, media uploads, tags
  • Engagement Metrics: Average ratings, comment activity, user participation rates
  • Geographic Distribution: Dive sites and diving centers by country/region
  • System Usage: API calls per day, peak usage times, most accessed endpoints
  • Notification Analytics: In-app notification and email delivery statistics, delivery rates, category breakdown

System Metrics Dashboard:

  • Database Performance: Connection health, query response times
  • Application Health: Service status (Database, API, Frontend)
  • Resource Utilization: CPU usage, memory consumption, disk space
  • Cloud Storage Health: Cloudflare R2 connectivity and local fallback status
  • Bot Protection Metrics: Cloudflare Turnstile verification success rates, error breakdown, top IP addresses
  • System Alerts: Real-time summary of critical issues and warnings

Visual Dashboard Elements:

  • Charts and Graphs: Growth trends, distribution maps, metric visualizations
  • Status Indicators: Service health lights (green/yellow/red)
  • Quick Actions: Direct links to specific statistics and metrics pages
  • Refresh Controls: Real-time data updates with configurable intervals

3.13.2. Recent Activity Monitoring

The Recent Activity page provides real-time tracking of user actions and system changes for security and operational oversight:

User Activity Tracking:

  • Authentication Events: Login/logout events, failed login attempts, OAuth usage
  • Content Creation: New dive sites, diving centers, dives, comments, ratings
  • Content Modifications: Edits to existing content, ownership changes
  • Administrative Actions: User role changes, account approvals, content deletions
  • Search Activity: Popular search terms, search patterns, geographic search trends

System Activity Monitoring:

  • Database Operations: Schema changes, migration executions, backup operations
  • API Usage: Endpoint access patterns, rate limiting events, error responses
  • File Operations: Media uploads, file deletions, storage usage changes
  • External Integrations: Newsletter parsing results, geocoding requests, OAuth verifications

Activity Details:

  • Timestamp: Precise time of each activity
  • User Information: User ID, username, IP address, user agent
  • Action Details: Specific operation performed, affected resources
  • Outcome: Success/failure status, error messages if applicable
  • Context: Related actions, session information, geographic location

Filtering and Search:

  • Time Range: Filter by specific time periods (last hour, day, week, month)
  • User Filtering: View activity for specific users or user groups
  • Action Types: Filter by specific activity categories
  • Status Filtering: View only successful, failed, or pending actions
  • Geographic Filtering: Filter by user location or content location

Security Features:

  • Suspicious Activity Detection: Unusual patterns, potential security threats
  • Audit Trail: Complete history for compliance and investigation
  • Export Capabilities: Export activity logs for external analysis
  • Alert Configuration: Customizable alerts for specific activity patterns

3.13.3. Backup and Export Management

The Backup and Export page provides comprehensive data management capabilities for administrators:

Database Backup Management:

  • Automated Backups: Scheduled daily backups with configurable retention policies
  • Manual Backups: On-demand backup creation with custom naming
  • Backup Verification: Automatic integrity checks and restoration testing
  • Backup Storage: Local and cloud storage options with encryption
  • Backup History: Complete backup log with timestamps and sizes
  • Restore Operations: Point-in-time restoration with rollback capabilities

Data Export Capabilities:

  • Full Database Export: Complete database dump in SQL format
  • Selective Table Export: Export specific tables or data subsets
  • Format Options: SQL, CSV, JSON, XML export formats
  • Filtered Exports: Export data based on date ranges, user criteria, or content types
  • Incremental Exports: Export only changed data since last export
  • Compression Options: Compressed exports for large datasets

Export Categories:

  • User Data: User accounts, profiles, preferences, activity history
  • Content Data: Dive sites, diving centers, dives, comments, ratings
  • Media Assets: Photos, videos, documents with metadata
  • Configuration Data: System settings, tags, organizations, permissions
  • Analytics Data: Usage statistics, performance metrics, audit logs

Export Management Features:

  • Scheduled Exports: Automated exports on configurable schedules
  • Export Queuing: Background processing for large exports
  • Progress Tracking: Real-time progress indicators for export operations
  • Notification System: Email alerts for completed exports and failures
  • Storage Management: Automatic cleanup of old exports and backups

Data Privacy and Compliance:

  • GDPR Compliance: User data export for data subject requests
  • Data Anonymization: Option to anonymize sensitive data in exports
  • Access Controls: Role-based access to backup and export functions
  • Audit Logging: Complete audit trail of all backup and export operations
  • Encryption: Encrypted exports for sensitive data protection

Integration Features:

  • Cloud Storage: Direct integration with AWS S3, Google Cloud Storage
  • FTP/SFTP: Secure file transfer for backup distribution
  • Email Integration: Automated email delivery of export files
  • API Access: Programmatic access to backup and export functions
  • Monitoring Integration: Integration with system monitoring tools

Operational Features:

  • Health Checks: Verification of backup integrity and accessibility
  • Performance Monitoring: Export and backup performance metrics
  • Error Handling: Comprehensive error reporting and recovery procedures
  • Documentation: Automatic generation of backup and export reports
  • Disaster Recovery: Complete disaster recovery procedures and testing

4. Non-Functional Requirements

  • Performance:
    • Fast page load times (under 2 seconds for most pages).
    • Efficient database queries.
  • Scalability:
    • Ability to handle increasing user traffic and data volume.
    • Support for future mobile application integration.
  • Security:
    • User authentication and authorization (JWT + Google OAuth).
    • Protection against common web vulnerabilities (XSS, CSRF, SQL injection).
    • Data encryption (especially for user credentials).
    • Secure Google OAuth token verification.
    • Industry-standard security measures for data protection.
    • Secure authentication and access controls.
    • Regular security audits and monitoring.
  • Privacy & Data Protection:
    • GDPR compliance with user data rights (access, correction, deletion, portability).
    • No data sales or marketing use of personal information.
    • Data export and portability features.
    • Transparent data collection and usage policies.
    • User consent management for data processing.
    • Data retention policies with automatic cleanup.
  • Maintainability:
    • Clean, modular, and well-documented codebase.
    • Easy to deploy and update.
  • Usability:
    • Intuitive user interface for both web and future mobile applications.
    • Responsive design for various screen sizes.
    • Enhanced toast notifications and layout improvements.
  • Reliability:
    • High availability of the platform.
    • Robust error handling and logging.

5. Architecture

The application will follow a microservices-oriented or a well-separated monolithic architecture, with a clear distinction between frontend, backend, and database layers.

5.1. High-Level Architecture Diagram

+-------------------+ +-------------------+ +-------------------+ | | | | | | | User Devices | <-------> | Load Balancer | <-------> | Web Servers | | (Web Browser/App) | | | | (Nginx/Gunicorn) | | | | | | | +-------------------+ +-------------------+ +-------------------+ | | HTTP/HTTPS V +-----------------------+ | | | Backend API | | (Python/FastAPI) | | | +-----------------------+ | +---------------------------------------+ | | | Database Connection Pool | V V +-------------------+ +-------------------+ | | | | | MySQL DB | | Redis Cache | | (Main Data Store) | | (Session/Caching) | | | | | +-------------------+ +-------------------+ | V +-----------------------+ | | | Asynchronous Tasks | | (Celery/RabbitMQ) | | (e.g., Newsletter Parsing) | +-----------------------+

5.2. Component Breakdown

5.2.1. Frontend

  • Technology: React (for web application) / React Native (for future mobile application).
  • Key Features:
    • User-friendly interface for browsing, searching, rating, and commenting.
    • Interactive map display for dive trips.
    • Responsive design.
    • Communication with the backend via RESTful API calls.
    • Google OAuth integration with Google Identity Services.
    • Mass delete functionality for admin management.
    • Dive logging interface with comprehensive media upload capabilities (photos, videos, external links).
    • Diving center ownership management interface.

5.2.2. Backend (API)

  • Language: Python
  • Framework: FastAPI (chosen for its high performance, modern features, and automatic OpenAPI/Swagger documentation generation).
  • Key Services/Modules:
    • User Service: Handles user registration, login, authentication (JWT + Google OAuth), profile management.
    • Dive Site Service: CRUD operations for dive sites, rating logic, comment management, URL routing.
    • Diving Center Service: CRUD operations for diving centers, rating logic, comment management, association with dive sites and pricing, ownership management.
    • Dive Service: CRUD operations for user dives, media upload handling, dive statistics, external link management.
    • Google OAuth Service: Token verification and user management for Google authentication.
    • Newsletter Parsing Service:
      • Receives newsletter content.
      • Utilizes NLP techniques (e.g., SpaCy, NLTK) or rule-based parsing to extract entities (dive center names, dive site names, dates).
      • Requires a mapping between recognized entities and database IDs.
      • Queues parsing tasks for asynchronous processing (Celery).
    • Search Service: Implements full-text search and filtering capabilities.
    • Image/Video Upload Service: Handles secure storage and retrieval of multimedia content (e.g., integration with cloud storage like CloudFlare R2, AWS S3 or Google Cloud Storage).
  • Web Server Gateway Interface (WSGI): Gunicorn (production-ready WSGI server for Python).
  • Reverse Proxy: Nginx (for serving static files, load balancing, SSL termination, and proxying requests to Gunicorn).

5.2.3. Database

  • Type: Relational Database
  • System: MySQL (chosen for its robustness, reliability, rich feature set, and strong support for spatial data if needed for advanced mapping).
  • Migration System: Alembic for version-controlled database schema changes
  • Schema (Conceptual):
    • users table:
      • id (PK)
      • username
      • email (unique)
      • password_hash
      • google_id (unique, nullable) - NEW FIELD
      • created_at
      • updated_at
      • is_admin (boolean)
      • is_moderator (boolean)
    • dive_sites table:
      • id (PK)
      • name
      • description
      • latitude
      • longitude
      • access_instructions
      • difficulty_level
      • created_at
      • updated_at
      • alternative_names (JSON array for URL routing)
    • site_media table:
      • id (PK)
      • dive_site_id (FK to dive_sites)
      • media_type (e.g., 'photo', 'video')
      • url (link to stored media)
      • description (optional)
    • site_ratings table:
      • id (PK)
      • dive_site_id (FK to dive_sites)
      • user_id (FK to users)
      • score (1-10)
      • created_at
    • site_comments table:
      • id (PK)
      • dive_site_id (FK to dive_sites)
      • user_id (FK to users)
      • comment_text
      • created_at
      • updated_at
    • diving_centers table:
      • id (PK)
      • name
      • description
      • email
      • phone
      • website
      • latitude
      • longitude
      • created_at
      • updated_at
      • owner_id (FK to users, nullable) - NEW FIELD
      • ownership_status (enum: 'unclaimed', 'claimed', 'approved') - NEW FIELD
    • center_ratings table:
      • id (PK)
      • diving_center_id (FK to diving_centers)
      • user_id (FK to users)
      • score (1-10)
      • created_at
    • center_comments table:
      • id (PK)
      • diving_center_id (FK to diving_centers)
      • user_id (FK to users)
      • comment_text
      • created_at
      • updated_at
    • center_dive_sites (junction table for many-to-many relationship):
      • id (PK)
      • diving_center_id (FK to diving_centers)
      • dive_site_id (FK to dive_sites)
      • dive_cost
    • gear_rental_costs table:
      • id (PK)
      • diving_center_id (FK to diving_centers)
      • item_name (e.g., "Full Set", "BCD", "Regulator", "12L Tank")
      • cost
    • dives table: - NEW TABLE
      • id (PK)
      • user_id (FK to users)
      • dive_site_id (FK to dive_sites, nullable)
      • dive_information (text)
      • max_depth (decimal)
      • average_depth (decimal)
      • gas_bottles_used (text)
      • suit_type (enum: 'wet_suit', 'dry_suit', 'shortie')
      • difficulty_level (enum: 'beginner', 'intermediate', 'advanced', 'expert')
      • visibility_rating (1-10)
      • user_rating (1-10)
      • dive_date (date)
      • dive_time (time)
      • duration (integer, minutes)
      • created_at
      • updated_at
    • dive_media table: - NEW TABLE
      • id (PK)
      • dive_id (FK to dives)
      • media_type (enum: 'dive_plan', 'photo', 'video', 'external_link')
      • url (link to stored media or external URL)
      • description (optional)
      • title (optional, for external links)
      • thumbnail_url (optional, for external links). Note: UI no longer exposes a thumbnail display toggle; images are handled contextually per page.
    • dive_tags table: - NEW TABLE
      • id (PK)
      • dive_id (FK to dives)
      • tag_id (FK to tags)
    • parsed_dive_trips table:
      • id (PK)
      • diving_center_id (FK to diving_centers)
      • dive_site_id (FK to dive_sites)
      • trip_date (Date)
      • trip_time (Time - optional)
      • trip_duration (integer, minutes)
      • trip_difficulty_level (enum: 'beginner', 'intermediate', 'advanced', 'expert')
      • trip_price (decimal with currency support)
      • trip_currency (3-letter ISO currency code)
      • group_size_limit (integer, nullable)
      • current_bookings (integer, default 0)
      • trip_description (text, optional)
      • special_requirements (text, optional)
      • trip_status (enum: 'scheduled', 'confirmed', 'cancelled', 'completed')
      • source_newsletter_id (FK to newsletters table, if storing raw newsletters)
      • extracted_at
      • created_at
      • updated_at
    • trip_favorites table: - NEW TABLE
      • id (PK)
      • user_id (FK to users)
      • trip_id (FK to parsed_dive_trips)
      • created_at
    • trip_notifications table: - NEW TABLE
      • id (PK)
      • user_id (FK to users)
      • diving_center_id (FK to diving_centers, nullable)
      • dive_site_id (FK to dive_sites, nullable)
      • notification_type (enum: 'new_trips', 'price_changes', 'cancellations')
      • is_active (boolean, default true)
      • created_at
      • updated_at
    • newsletters table (optional, for storing raw newsletters for auditing/re-parsing):
      • id (PK)
      • content (text blob)
      • received_at

5.2.4. Caching

  • Technology: Redis (for session management, frequently accessed data like average ratings, and rate limiting).

5.2.5. Asynchronous Task Queue

  • Technology: Celery with RabbitMQ (broker).
  • Purpose: Offload long-running tasks like newsletter parsing, image processing, and sending notifications to background workers to avoid blocking the main API.

5.2.6. Object Storage

  • Technology: AWS S3, Google Cloud Storage, or a self-hosted MinIO instance.
  • Purpose: Store large binary objects like photos, videos, and dive plan PDFs, keeping them separate from the main database.

6. API Endpoints (Conceptual)

  • /api/v1/auth/register (POST)
  • /api/v1/auth/login (POST)
  • /api/v1/auth/google-login (POST) - NEW ENDPOINT
  • /api/v1/users/{user_id} (GET, PUT)
  • /api/v1/dive-sites (GET, POST)
  • /api/v1/dive-sites/{site_id} (GET, PUT, DELETE)
  • /api/v1/dive-sites/{site_id}/rate (POST)
  • /api/v1/dive-sites/{site_id}/comments (GET, POST)
  • /api/v1/dive-sites/search (GET)
  • /api/v1/dive-sites/by-name/{name} (GET) - NEW ENDPOINT
  • /api/v1/diving-centers (GET, POST)
  • /api/v1/diving-centers/{center_id} (GET, PUT, DELETE)
  • /api/v1/diving-centers/{center_id}/rate (POST)
  • /api/v1/diving-centers/{center_id}/comments (GET, POST)
  • /api/v1/diving-centers/search (GET)
  • /api/v1/diving-centers/{center_id}/gear-rental (GET, POST) - Supports currency
  • /api/v1/dive-sites/{site_id}/diving-centers (GET, POST) - Supports currency
  • /api/v1/diving-centers/{center_id}/claim (POST) - NEW ENDPOINT
  • /api/v1/diving-centers/{center_id}/approve-ownership (POST) - NEW ENDPOINT
  • /api/v1/dives (GET, POST) - NEW ENDPOINT
  • /api/v1/dives/{dive_id} (GET, PUT, DELETE) - NEW ENDPOINT
  • /api/v1/dives/{dive_id}/media (GET, POST, DELETE) - NEW ENDPOINT
  • /api/v1/dives/{dive_id}/media/{media_id} (GET, PUT, DELETE) - NEW ENDPOINT
  • /api/v1/dives/search (GET) - NEW ENDPOINT
  • /api/v1/admin/newsletters/parse (POST - upload newsletter, trigger parsing)
  • /api/v1/dive-trips (GET - retrieve parsed trips for map)
  • /api/v1/dive-trips/calendar (GET - retrieve trips for calendar view with date filtering)
  • /api/v1/dive-trips/{trip_id} (GET - retrieve specific trip details)
  • /api/v1/dive-trips/search (GET - search trips by criteria)
  • /api/v1/dive-trips/favorites (GET, POST, DELETE - manage favorite trips)
  • /api/v1/dive-trips/export (GET - export trips to calendar format)
  • /api/v1/media/upload (POST - for image/video uploads)
  • /api/v1/admin/system/statistics (GET - platform statistics and engagement)
  • /api/v1/admin/system/metrics (GET - system health and infrastructure metrics)
  • /api/v1/admin/system/activity (GET - recent user and system activity)
  • /api/v1/admin/system/backup (POST - create database backup)
  • /api/v1/admin/system/export (GET - export data in various formats)
  • /api/v1/admin/system/health (GET - system health check)
  • /api/v1/privacy/data-export (GET - export user's personal data)
  • /api/v1/privacy/data-deletion (POST - request data deletion)
  • /api/v1/privacy/consent-management (GET, PUT - manage data processing consent)
  • /api/v1/privacy/privacy-settings (GET, PUT - manage privacy preferences)
  • /api/v1/privacy/audit-log (GET - view user's data access audit trail)

7. Technologies & Tools

  • Backend:
    • Python 3.x
    • FastAPI
    • Pydantic (for data validation)
    • SQLAlchemy (ORM for database interaction)
    • Alembic (for database migrations)
    • Gunicorn
    • Nginx
    • Celery
    • RabbitMQ
    • MySQL
    • Redis
    • Requests (for external API calls if any)
    • Pillow (for image processing - resizing, compression)
    • SpaCy / NLTK (for NLP in newsletter parsing)
    • Google Auth (for OAuth verification) - NEW
  • Frontend:
    • React / React Native
    • Redux / Zustand / React Query (for state management and data fetching)
    • OpenLayers (for interactive maps)
    • Axios (for API calls)
    • Google Identity Services (for OAuth) - NEW
    • Privacy-focused UI components for data control
  • DevOps & Deployment:
    • Docker / Docker Compose (for local development and deployment)
    • Kubernetes (for container orchestration in production - long-term)
    • CI/CD Pipeline (e.g., GitLab CI/CD, GitHub Actions)
    • Terraform (for infrastructure as code - long-term)
    • Prometheus / Grafana (for monitoring)
    • Sentry (for error tracking)
  • Version Control: Git
  • Documentation: OpenAPI/Swagger UI (generated by FastAPI)

8. Future Considerations

  • Mobile Application: Leverage React Native for cross-platform mobile development, reusing a significant portion of the frontend logic.
  • Advanced Search: Implement more sophisticated search capabilities (e.g., fuzzy search, spatial search).
  • Social Features: User following, sharing dive sites/trips.
  • AI/ML for Newsletter Parsing: Improve parsing accuracy and adaptability using machine learning models trained on various newsletter formats.
  • Personalized Recommendations: Suggest dive sites or centers based on user preferences and past activity.
  • Booking Integration: Direct booking functionality with diving centers (requires integration with their booking systems, potentially via APIs).
  • User-Generated Content Review Workflow: For comments and ratings to prevent abuse.
  • Internationalization (i18n): Support for multiple languages.
  • Additional OAuth Providers: Facebook, GitHub, etc.
  • Dive Statistics and Analytics: Advanced dive logging analytics and statistics.
  • Dive Buddy System: Connect divers and share dive experiences.
  • Enhanced Privacy Features: Advanced data anonymization, differential privacy, and privacy-preserving analytics.
  • Privacy Compliance Tools: Automated GDPR compliance checking, privacy impact assessment automation, and regulatory reporting tools.

9. Implementation Phases (High-Level)

Phase 1: Core MVP (Minimum Viable Product)

  • Basic User Management (registration, login, profile).
  • CRUD for Dive Sites (Admin only initially).
  • View Dive Sites (with all details).
  • Interactive map display of dive sites.
  • User Rating for Dive Sites.
  • Basic Search and Filtering for Dive Sites.
  • Deployment to a staging environment.

Phase 2: Diving Centers & Comments

  • CRUD for Diving Centers (Admin only initially).
  • View Diving Centers (with details, associated sites, pricing).
  • Diving centers also have a location and appear on the dive map using a different icon.
  • User Rating for Diving Centers.
  • Eponymous User Comments on Dive Sites and Diving Centers.
  • Basic Search and Filtering for Diving Centers.

Phase 3: Newsletter Parsing & Map

  • Admin interface for newsletter upload.
  • Initial implementation of newsletter parsing logic.
  • Populate parsed_dive_trips table.
  • Interactive map display of dive trips.
  • Contact details for booking (email/phone).

Phase 4: Dive Trip Calendar System

  • Interactive calendar widget for date navigation.
  • Trip listing view with detailed information display.
  • Map view integration for trip visualization.
  • Advanced filtering and search capabilities.
  • Trip management features (favorites, sharing, export).
  • User experience enhancements (quick booking, trip comparison).
  • Admin management interface for trip creation and editing.
  • Mobile optimization for touch-friendly navigation.
  • Integration features (Google Calendar, messaging apps).

Phase 5: Dive Logging System

  • CRUD for user dives with comprehensive dive information.
  • Media upload for dive plans and photos.
  • Dive statistics and analytics.
  • Search and filter dives by various criteria.
  • Integration with dive sites and tags.

Phase 6: Diving Center Ownership

  • User claiming system for diving centers.
  • Admin approval workflow for ownership claims.
  • Owner editing capabilities for diving center details.
  • Ownership management interface for admins.

Phase 7: URL Routing & Enhanced Features

  • URL routing for dive sites by name/alias.
  • Enhanced search and filtering capabilities.
  • Performance optimizations and scaling.
  • Mobile application development.

Phase 8: Refinement & Scaling

  • Performance optimizations (caching, query tuning).
  • Robust error handling and logging.
  • Security enhancements.
  • CI/CD pipeline setup.
  • Scalable deployment infrastructure (Docker/Kubernetes).
  • User-friendly UI/UX improvements.

Phase 9: Mobile Application (Future)

  • Design and development of the React Native mobile application.
  • Adaptation of existing frontend components.
  • Mobile-specific features (e.g., location services for "dives near me").

10. Security Considerations

  • Authentication: JWT (JSON Web Tokens) for stateless authentication + Google OAuth.
  • Authorization: Role-based access control (RBAC) to distinguish between regular users, administrators, and moderators.
  • Password Hashing: Use strong, industry-standard hashing algorithms (e.g., bcrypt) with salts.
  • Input Validation: Sanitize and validate all user inputs to prevent injection attacks (SQL injection, XSS).
  • CORS: Properly configure Cross-Origin Resource Sharing.
  • HTTPS: Enforce HTTPS for all communication.
  • Rate Limiting: Protect against brute-force attacks and API abuse.
  • Secret Management: Securely store API keys and sensitive credentials (e.g., environment variables, dedicated secret management services).
  • Regular Security Audits: Conduct periodic vulnerability assessments and penetration testing.
  • Google OAuth Security: Secure token verification with Google's servers.
  • Media Upload Security: Validate file types and sizes, scan for malware.

11. Privacy and Data Protection Features

11.1. Data Collection and Usage

  • Personal Information Management:
    • User registration data (name, username, email, contact information).
    • Profile information and preferences.
    • Diving certifications and experience.
    • Authentication details and OAuth tokens.
  • Usage Data Collection:
    • Platform usage patterns and interactions.
    • Pages visited and features used.
    • Search queries and user behavior.
    • Device and browser information for security.
  • User-Generated Content:
    • Dive site reviews, ratings, and comments.
    • Dive logs and trip reports.
    • Photos, videos, and media uploads.
    • Community contributions and interactions.

11.2. Data Protection Measures

  • Encryption and Security:
    • Data encryption in transit (HTTPS/TLS).
    • Data encryption at rest in database and storage.
    • Secure authentication and access controls.
    • Regular security audits and monitoring.
    • Employee training on data protection.
  • Access Control:
    • Role-based access control (RBAC).
    • Principle of least privilege.
    • Secure session management.
    • Multi-factor authentication support.

11.3. User Rights and Control

  • GDPR Compliance Features:
    • Right to Access: Users can view all personal data held about them.
    • Right to Correction: Users can update or correct inaccurate information.
    • Right to Deletion: Users can request removal of their personal data.
    • Right to Portability: Users can export their data in machine-readable format.
    • Right to Objection: Users can object to certain types of processing.
    • Right to Withdrawal: Users can revoke consent for data processing.
  • Data Export and Portability:
    • Download dive data in multiple formats.
    • Export profile and account information.
    • Access all uploaded content and media.
    • Transfer data to other platforms.
    • Machine-readable export formats.

11.4. Data Sharing and Third Parties

  • No Data Sales Policy:
    • No selling, renting, or trading of personal information.
    • No marketing use of user data.
    • No advertising partnerships or data monetization.
  • Limited Third-Party Sharing:
    • Service providers under strict confidentiality agreements.
    • Legal requirements and regulatory compliance.
    • Protection of platform rights and safety.
    • Explicit user consent for specific purposes.

11.5. Data Retention and Cleanup

  • Retention Policies:
    • Personal data retained only as long as necessary.
    • Automatic cleanup after account deletion (30 days).
    • Legal and regulatory compliance retention.
    • Audit trail maintenance for security.
  • Data Lifecycle Management:
    • Automated data retention enforcement.
    • Regular data cleanup processes.
    • Backup and archive management.
    • Data anonymization options.

11.6. Privacy Technologies and Services

  • Cloudflare Turnstile Integration:
    • Bot protection and spam prevention.
    • Privacy-preserving human verification.
    • No personal data collection by Cloudflare.
    • Enhanced security without compromising privacy.
  • OpenLayers Mapping Service:
    • Client-side mapping library.
    • No third-party location data sharing.
    • User-controlled location information.
    • Privacy-focused map interactions.

11.7. Privacy Policy and Transparency

  • Transparent Data Practices:
    • Clear privacy policy documentation.
    • Regular policy updates and notifications.
    • User consent management interface.
    • Data usage transparency tools.
  • Privacy Controls:
    • User preference management.
    • Communication preferences.
    • Data sharing consent controls.
    • Privacy settings dashboard.

11.8. Compliance and Auditing

  • Regulatory Compliance:
    • GDPR compliance framework.
    • Data protection impact assessments.
    • Privacy by design implementation.
    • Regular compliance audits.
  • Audit and Monitoring:
    • Complete audit trail of data operations.
    • Privacy event logging and monitoring.
    • Data access and modification tracking.
    • Compliance reporting and documentation.

12. Error Handling and Logging

  • Implement centralized error logging (e.g., using Sentry, ELK stack).
  • Provide meaningful error messages to the client without exposing sensitive internal details.
  • Log sufficient information for debugging (request details, stack traces, timestamps).
  • Graceful degradation for external service failures.

13. Testing Infrastructure

13.1 Testing Strategy

  • Automated Testing: Comprehensive test suite for backend API endpoints using Pytest.
  • Frontend Validation: Automated scripts to validate frontend functionality and catch regressions.
  • Data Type Safety: Validation of API response types and frontend data handling.
  • Regression Prevention: Automated testing to prevent common frontend errors.

13.2 Testing Tools

  • Backend Testing: Pytest with fixtures for isolated test database and authentication.
  • Frontend Validation: Node.js scripts for API health checks and data type validation.
  • Regression Testing: Automated scripts to test common issues like data type mismatches.
  • Manual Testing: Comprehensive checklist for user experience validation.

13.3 Test Categories

A. Backend API Testing

  • Unit tests for all API endpoints (auth, users, dive sites, diving centers, dives)
  • Integration tests for database operations
  • Authentication and authorization testing
  • Error handling and edge case testing
  • Google OAuth testing

B. Frontend Validation

  • Data type validation (lat/lng as strings, ratings as numbers)
  • API endpoint connectivity testing
  • Common error prevention (array safety, type conversion)
  • User interface functionality testing
  • Google OAuth integration testing

C. Regression Prevention

  • Automated testing for common frontend errors
  • Data type safety validation
  • API parameter filtering testing
  • Cross-browser compatibility testing

12.4 Testing Commands

# Backend tests
cd backend && python -m pytest

# Frontend validation
node validate_frontend.js

# Regression testing
node test_regressions.js

13. Current Implementation Status

13.1 Completed Features

Phase 1: Core MVP ✅ COMPLETED

  • ✅ Basic User Management (registration, login, profile)
  • ✅ CRUD for Dive Sites (Admin only initially)
  • ✅ View Dive Sites (with all details)
  • ✅ Interactive map display of dive sites
  • ✅ User Rating for Dive Sites
  • ✅ Basic Search and Filtering for Dive Sites
  • ✅ Deployment to a staging environment

Phase 2: Diving Centers & Comments ✅ COMPLETED

  • ✅ CRUD for Diving Centers (Admin only initially)
  • ✅ View Diving Centers (with details, associated sites, pricing)
  • ✅ Diving centers appear on the dive map using different icons
  • ✅ User Rating for Diving Centers
  • ✅ Eponymous User Comments on Dive Sites and Diving Centers
  • ✅ Basic Search and Filtering for Diving Centers
  • ✅ Comprehensive Add/Edit functionality for dive sites and diving centers
  • ✅ Media management for dive sites (photos and videos)
  • ✅ Gear rental cost management for diving centers
  • ✅ Enhanced dive site details (address, marine life, safety information)
  • ✅ Rating display improvements (numeric format instead of stars)
  • ✅ Edit functionality for admin/moderator users

Phase 3: Newsletter Parsing & Map 🔄 IN PROGRESS

  • ✅ Admin interface for newsletter upload
  • ✅ Initial implementation of newsletter parsing logic
  • ✅ Populate parsed_dive_trips table
  • 🔄 Interactive map display of dive trips (database structure ready, frontend display pending)
  • 🔄 Contact details for booking (email/phone) (backend ready, frontend integration pending)

Missing Newsletter Features:

  • 🔄 Frontend map display of parsed dive trips
  • 🔄 Trip booking interface integration
  • 🔄 User-facing trip browsing and search
  • 🔄 Trip calendar view integration

Phase 4: Dive Trip Calendar System 🔄 PLANNED

  • 🔄 Interactive calendar widget for date navigation
  • 🔄 Trip listing view with detailed information display
  • 🔄 Map view integration for trip visualization
  • 🔄 Advanced filtering and search capabilities
  • 🔄 Trip management features (favorites, sharing, export)
  • 🔄 User experience enhancements (quick booking, trip comparison)
  • 🔄 Admin management interface for trip creation and editing
  • 🔄 Mobile optimization for touch-friendly navigation
  • 🔄 Integration features (Google Calendar, messaging apps)

Phase 5: Refinement & Scaling ✅ COMPLETED

  • ✅ Performance optimizations (caching, query tuning)
  • ✅ Robust error handling and logging
  • ✅ Security enhancements
  • ✅ Comprehensive testing infrastructure
  • ✅ Scalable deployment infrastructure (Docker)
  • ✅ User-friendly UI/UX improvements

13.2 Recent Enhancements

Privacy and Data Protection System 🔄 IN PROGRESS

  • ✅ Comprehensive privacy API endpoints for GDPR compliance (data export and audit log implemented)
  • ✅ Data export endpoint for user personal data
  • ✅ Audit log endpoint for user activity history
  • ✅ Privacy Policy page with comprehensive data protection information
  • ✅ Privacy route integration in navigation with Shield icon
  • 🔄 Complete GDPR compliance framework implementation (missing: consent management, data deletion, data correction, communication preferences)
  • 🔄 User data portability and access rights (basic export implemented, advanced rights pending)

Missing Privacy Features:

  • 🔄 Consent management system (consent tracking, withdrawal, granular controls)
  • 🔄 Data deletion endpoint (/api/v1/privacy/data-deletion)
  • 🔄 Data correction endpoint (/api/v1/privacy/data-correction)
  • 🔄 Communication preferences management (/api/v1/privacy/communication-preferences)
  • 🔄 Cloudflare Turnstile integration (mentioned in docs but not implemented)
  • 🔄 Privacy settings dashboard for users

Diving Center Ownership Management ✅ COMPLETED

  • ✅ Comprehensive ownership request management system
  • ✅ Permanent history tracking for all ownership actions
  • ✅ Ownership revocation functionality with reason requirement
  • ✅ Complete audit trail for ownership changes (claim, approve, deny, revoke)
  • ✅ Admin interface for ownership management
  • ✅ History view with detailed information and timestamps
  • ✅ Ownership status management (unclaimed, claimed, approved, denied)

Admin Dashboard System ✅ COMPLETED

  • ✅ General Statistics and System Metrics dashboards with comprehensive platform statistics
  • ✅ Real-time system health monitoring and performance metrics
  • ✅ Recent Activity monitoring with user and system activity tracking
  • ✅ Activity filtering by time range and activity type
  • ✅ Auto-refresh functionality and real-time updates
  • ✅ System health checks and database performance monitoring
  • ✅ Platform statistics and engagement metrics

Database Export/Import System ✅ COMPLETED

  • ✅ Robust database export/import functionality for diving data
  • ✅ Full database backup creation before operations
  • ✅ Foreign key constraint-aware table clearing
  • ✅ Comprehensive error handling and logging
  • ✅ Dry-run mode for testing
  • ✅ Utility scripts for safe database operations between environments
  • ✅ Database synchronization between development and production

Dive Tag Editing System ✅ COMPLETED

  • ✅ Proper dive tag editing functionality in backend and frontend
  • ✅ Tag field support in DiveUpdate schema
  • ✅ Tag handling in update_dive endpoint
  • ✅ Tag retrieval in get_dive and get_dives endpoints
  • ✅ Permission enforcement for dive editing
  • ✅ Improved error handling for permission denials

Google OAuth Authentication ✅ COMPLETED

  • ✅ Complete OAuth 2.0 integration with Google Identity Services
  • ✅ Backend token verification with Google's servers
  • ✅ Automatic user creation and account linking
  • ✅ Frontend Google Sign-In buttons
  • ✅ Environment configuration and setup guide
  • ✅ Security features (rate limiting, error handling)

Mass Delete Functionality ✅ COMPLETED

  • ✅ Bulk operations for all admin management pages
  • ✅ Safety features (protection against deleting used tags and self-deletion)
  • ✅ Confirmation dialogs with item names
  • ✅ Visual feedback (loading states, success/error messages)
  • ✅ Responsive design for all screen sizes

Toast Notification Enhancements ✅ COMPLETED

  • ✅ Notifications appear below navbar to prevent navigation blocking
  • ✅ Reduced duration to 500ms for quicker disappearance
  • ✅ Proper z-index management with navbar
  • ✅ Responsive design for all screen sizes

Layout Improvements ✅ COMPLETED

  • ✅ Fixed navbar with proper z-index
  • ✅ Adjusted content spacing to account for fixed navbar
  • ✅ Text wrapping to prevent horizontal scrollbars
  • ✅ Increased container width for better content display

Testing Infrastructure ✅ COMPLETED

  • ✅ Comprehensive backend test suite with Pytest
  • ✅ Frontend validation scripts for regression prevention
  • ✅ Data type safety testing and validation
  • ✅ Automated testing for common frontend errors

User Experience Improvements ✅ COMPLETED

  • ✅ Rating display changed from stars to numeric format (X.X/10)
  • ✅ Enhanced dive site details with comprehensive information
  • ✅ Improved search and filtering with parameter validation
  • ✅ Better error handling and loading states

Admin Functionality ✅ COMPLETED

  • ✅ Comprehensive edit forms for dive sites and diving centers
  • ✅ Media management for dive sites
  • ✅ Gear rental cost management for diving centers
  • ✅ Protected routes for admin/moderator users

Data Type Safety ✅ COMPLETED

  • ✅ Fixed latitude/longitude type conversion issues
  • ✅ Improved array safety checks
  • ✅ API parameter filtering to prevent 422 errors
  • ✅ Comprehensive error prevention guidelines

Tag Management System ✅ COMPLETED

  • ✅ Comprehensive tag/label system for dive sites
  • ✅ Tag display in dive site details page
  • ✅ Multiple tag selection in edit forms with checkboxes
  • ✅ Bulk tag operations (add/remove all tags at once)
  • ✅ Create new tags functionality for admins/moderators
  • ✅ Efficient tag management with proper state handling

Map UI and Zoom Management ✅ COMPLETED

  • ✅ Interactive map display with OpenLayers integration
  • ✅ Different icons for dive sites and diving centers
  • ✅ Zoom level debugging indicator for optimal zoom configuration
  • ✅ Map counter box positioned at bottom-left for better UX
  • ✅ Configurable maximum zoom level (currently set to 18)
  • ✅ Smart zoom behavior: keeps zoom 5 levels before maximum for context
  • ✅ Real-time zoom level tracking and display
  • ✅ Map fit behavior optimization for single vs multiple site selection

13.3 Technical Improvements

Frontend Enhancements ✅ COMPLETED

  • ✅ Centralized API client with Axios
  • ✅ React Query for efficient data fetching
  • ✅ Comprehensive error boundaries and loading states
  • ✅ Responsive design with Tailwind CSS
  • ✅ Google OAuth integration with Google Identity Services

Backend Enhancements ✅ COMPLETED

  • ✅ FastAPI with automatic OpenAPI documentation
  • ✅ SQLAlchemy ORM with proper relationships
  • ✅ JWT authentication with role-based access control
  • ✅ Comprehensive API validation with Pydantic
  • ✅ Google OAuth token verification

DevOps & Deployment ✅ COMPLETED

  • ✅ Docker Compose for local development
  • ✅ MySQL database with proper schema
  • ✅ Nginx reverse proxy configuration
  • ✅ Automated testing and validation scripts

Admin Management System ✅ COMPLETED

  • ✅ Comprehensive admin dashboard with multiple management sections
  • ✅ Tag management with dive site count display
  • ✅ User management with role and status control
  • ✅ User approval system (new users disabled by default)
  • ✅ Admin-only user creation, editing, and deletion
  • ✅ Role-based access control (User, Moderator, Admin)
  • ✅ User status management (enabled/disabled)
  • ✅ Mass delete functionality with safety features
  • ✅ Quick Actions section with statistics, metrics, activity monitoring, and growth visualizations
  • ✅ General Statistics and System Metrics dashboards with comprehensive platform statistics and health monitoring
  • ✅ Recent Activity Monitoring with real-time user and system activity tracking

User Registration and Approval System ✅ COMPLETED

  • ✅ New users created with enabled=False by default
  • ✅ Admin approval required for account activation
  • ✅ Google OAuth integration for secure authentication
  • ✅ Registration success message with approval notice
  • ✅ Disabled users blocked from accessing protected endpoints
  • ✅ User-friendly approval workflow

13.4 Recent Bug Fixes

Google OAuth Implementation ✅ COMPLETED

  • ✅ Fixed ModuleNotFoundError for Google packages
  • ✅ Successfully added google_id field to users table
  • ✅ Fixed dependency conflicts with pyasn1
  • ✅ Rebuilt Docker containers with new dependencies

Frontend Linting Issues ✅ COMPLETED

  • ✅ Fixed missing icon imports (X, Loader, Save)
  • ✅ Fixed useEffect dependency warnings with useCallback
  • ✅ Removed unused navigate imports
  • ✅ Fixed all ESLint errors and warnings

Layout and UX Issues ✅ COMPLETED

  • ✅ Fixed toast notifications appearing behind navbar
  • ✅ Prevented horizontal scrollbars with text wrapping
  • ✅ Proper z-index management for fixed navbar
  • ✅ Improved container width and spacing

API Serialization Issues ✅ COMPLETED

  • ✅ Fixed dive sites API tag serialization causing 500 errors
  • ✅ Updated AvailableTag model field mapping (removed non-existent 'category' field)
  • ✅ Fixed Pydantic response validation errors for dive sites endpoint
  • ✅ Proper tag dictionary serialization in all dive site endpoints

Schema Validation Updates ✅ COMPLETED

  • ✅ Added 'expert' difficulty level support to all dive site schemas
  • ✅ Updated difficulty level patterns in DiveSiteBase, DiveSiteUpdate, and DiveSiteSearchParams
  • ✅ Fixed query parameter validation for difficulty level filtering

Frontend Create Pages ✅ COMPLETED

  • ✅ Added missing CreateDiveSite.js component with comprehensive form
  • ✅ Added missing CreateDivingCenter.js component with comprehensive form
  • ✅ Added proper React Router routes for create pages
  • ✅ Implemented form validation and error handling
  • ✅ Added proper navigation and user experience features

Authentication and Docker Issues ✅ COMPLETED

  • ✅ Resolved admin login issues with updated password requirements
  • ✅ Fixed slowapi import errors in containerized environment
  • ✅ Updated admin password to meet new security requirements
  • ✅ Rebuilt Docker images to include latest dependencies

Testing and Validation ✅ COMPLETED

  • ✅ Updated test data to include 'expert' difficulty level
  • ✅ Fixed test tag data to match actual model fields
  • ✅ Added comprehensive API response validation tests
  • ✅ Improved error handling and logging for debugging

13.5 Planned Features

Phase 4: Dive Trip Calendar System 🔄 PLANNED

  • 🔄 Interactive calendar widget for date navigation
  • 🔄 Trip listing view with detailed information display
  • 🔄 Map view integration for trip visualization
  • 🔄 Advanced filtering and search capabilities
  • 🔄 Trip management features (favorites, sharing, export)
  • 🔄 User experience enhancements (quick booking, trip comparison)
  • 🔄 Admin management interface for trip creation and editing
  • 🔄 Mobile optimization for touch-friendly navigation
  • 🔄 Integration features (Google Calendar, messaging apps)

Phase 5: Dive Logging System 🔄 PLANNED

  • 🔄 CRUD for user dives with comprehensive dive information
  • 🔄 Media upload for dive plans, photos, videos, and external links
  • 🔄 Media management (upload, delete, organize, external link handling)
  • 🔄 Dive statistics and analytics
  • 🔄 Search and filter dives by various criteria
  • 🔄 Integration with dive sites and tags
  • 🔄 Remove gas tanks necessary and dive plans from dive sites
  • ✅ Aliases system implemented for enhanced search and newsletter parsing

Phase 6: Diving Center Ownership ✅ COMPLETED

  • ✅ User claiming system for diving centers
  • ✅ Admin approval workflow for ownership claims
  • ✅ Owner editing capabilities for diving center details
  • ✅ Ownership management interface for admins
  • ✅ Comprehensive ownership request management system
  • ✅ Permanent history tracking for all ownership actions
  • ✅ Ownership revocation functionality with reason requirement
  • ✅ Complete audit trail for ownership changes

Phase 7: URL Routing & Enhanced Features 🔄 PLANNED

  • 🔄 URL routing for dive sites by name/alias
  • 🔄 Enhanced search and filtering capabilities
  • 🔄 Performance optimizations and scaling
  • 🔄 Mobile application development

Phase 8: Admin Dashboard Enhancement 🔄 IN PROGRESS

  • ✅ General Statistics and System Metrics dashboards with comprehensive platform statistics and health monitoring
  • ✅ Recent Activity Monitoring with real-time user and system activity tracking
  • 🔄 Backup and Export Management (placeholder UI exists, actual functionality pending)
  • 🔄 Advanced analytics and reporting features (basic stats implemented, advanced features pending)
  • 🔄 Real-time alerts and notification system (basic monitoring implemented, alerts pending)
  • 🔄 Performance monitoring and capacity planning tools (basic health checks implemented, advanced monitoring pending)
  • ✅ Database export/import functionality for diving data (utility scripts implemented)
  • ✅ System health checks and database performance monitoring
  • ✅ Platform statistics and engagement metrics

Missing Admin Dashboard Features:

  • 🔄 Actual backup and export management interface (only placeholder exists)
  • 🔄 Advanced analytics and reporting dashboard
  • 🔄 Real-time alerts and notification system
  • 🔄 Performance monitoring dashboard with detailed metrics
  • 🔄 Capacity planning tools and resource monitoring

14. Summary of Implementation Status

✅ COMPLETED PHASES (6/8)

  1. Phase 1: Core MVP - Basic user management, dive sites CRUD, interactive maps
  2. Phase 2: Diving Centers & Comments - Diving centers management, user ratings, media management
  3. Phase 5: Refinement & Scaling - Performance optimization, security, testing, deployment
  4. Phase 6: Diving Center Ownership - User claiming system, admin approval workflow, ownership management

🔄 IN PROGRESS PHASES (2/8)

  1. Phase 3: Newsletter Parsing & Map - Backend parsing implemented, frontend display pending
  2. Phase 8: Admin Dashboard Enhancement - Basic monitoring implemented, advanced features pending

📋 PLANNED FEATURES (2 remaining)

  1. Phase 4: Dive Trip Calendar System - Interactive calendar, trip management, mobile optimization
  2. Phase 7: URL Routing & Enhanced Features - URL routing, enhanced search, mobile app development

🎯 IMPLEMENTATION PROGRESS

  • Overall Completion: 65% (6 out of 10 major phases completed, 2 in progress)
  • Core Functionality: 90% (Most essential features implemented)
  • Admin Features: 75% (Basic dashboard implemented, advanced features pending)
  • User Features: 80% (Most user-facing features completed)
  • Infrastructure: 100% (Deployment, testing, security, performance)

🚀 RECENT MAJOR COMPLETIONS

  • Diving Center Ownership Management (complete)
  • Admin Dashboard System (basic monitoring implemented)
  • Database Export/Import System (utility scripts implemented)
  • Dive Tag Editing System (complete)
  • Privacy System (basic endpoints implemented)

🔄 FEATURES IN PROGRESS

  • Privacy and Data Protection System (GDPR compliance - partial)
  • Newsletter Parsing and Trip Mapping (backend ready, frontend pending)
  • Admin Dashboard Enhancement (basic features implemented, advanced pending)

📊 NEXT PRIORITIES

  1. Complete Phase 3: Newsletter Frontend Display - High priority for user experience
  2. Complete Phase 8: Admin Dashboard Advanced Features - Medium priority for admin functionality
  3. Phase 4: Dive Trip Calendar System - High priority for user experience
  4. Complete Privacy System - Medium priority for GDPR compliance
  5. Phase 7: URL Routing & Enhanced Features - Medium priority for SEO and usability