feat: Set up Python testing infrastructure with Poetry and pytest

[llbbl]

Set up Python Testing Infrastructure

Summary

This PR establishes a complete testing infrastructure for the EventMonitor Python project using Poetry as the package manager and pytest as the testing framework. The setup provides a ready-to-use testing environment where developers can immediately start writing tests.

Changes Made

Package Management

• Poetry Configuration: Created pyproject.toml with complete project metadata and dependency management
• Dependencies: Added existing project dependencies (scrapy, redis, lxml) to Poetry configuration
• Development Dependencies: Added pytest (^7.4.3), pytest-cov (^4.1.0), and pytest-mock (^3.12.0)

Testing Configuration

• Pytest Configuration:

  • Configured test discovery patterns for test_*.py and *_test.py files
  • Added strict markers and configuration options
  • Set up coverage reporting with HTML and XML output formats
  • Defined custom markers: unit, integration, and slow

• Coverage Configuration:

  • Source directories: EventMonitor and collect_person_rel
  • Excluded test files, cache directories, and virtual environments
  • HTML reports generate in htmlcov/ directory
  • XML report generates as coverage.xml

Directory Structure

tests/
├── __init__.py
├── conftest.py
├── test_setup_validation.py
├── unit/
│   └── __init__.py
└── integration/
    └── __init__.py

Shared Fixtures (conftest.py)

• temp_dir: Creates temporary directories for testing
• mock_redis_connection & mock_redis_pool: Mock Redis connections
• sample_rel_data & sample_person_data: Create sample data files
• scrapy_response: Mock Scrapy HTTP responses
• mock_scrapy_settings: Mock Scrapy configuration
• mock_news_parser: Mock news parsing functionality
• sample_html_content: Provide test HTML content
• reset_sys_modules: Clean up imports between tests

Development Commands

Both commands are configured to run the test suite:

• poetry run test
• poetry run tests

Additional Setup

• Updated .gitignore with:
  • Python build artifacts and cache files
  • Testing outputs (.pytest_cache/, coverage files, htmlcov/)
  • Virtual environment directories
  • IDE files and OS-specific files
  • Claude settings (.claude/*)
  • Note: poetry.lock is NOT ignored (should be committed)

How to Use

1. Install dependencies:

   poetry install

2. Run tests:

   poetry run test
   # or
   poetry run tests

3. Run specific test markers:

   poetry run pytest -m unit        # Run only unit tests
   poetry run pytest -m integration # Run only integration tests
   poetry run pytest -m "not slow"  # Skip slow tests

4. Generate coverage reports:

   poetry run test  # Automatically generates coverage
   # View HTML report: open htmlcov/index.html

Validation

The setup includes validation tests (test_setup_validation.py) that verify:

• Python path configuration
• Module imports work correctly
• All fixtures are available and functional
• Test markers are properly configured
• Redis mocking works as expected
• Sample data fixtures generate correctly
• Coverage tools are installed

All validation tests pass successfully.

Notes

• Coverage threshold is currently set to 0% since no actual unit tests exist yet
• The infrastructure is ready for developers to start writing tests immediately
• All pytest standard options are available through the configured commands
• The setup follows Python testing best practices and conventions