Codex/scraper mcp

.kiro/specs/bulk-read-new-jobs/tasks.md

@@ -19,14 +19,14 @@ This plan implements a read-only MCP tool for retrieving jobs with `status='new'
     - Include `code`, `message`, and `retryable` fields in error schema
     - Implement error sanitization to remove sensitive details (paths, SQL, stack traces)
     - _Requirements: 5.1, 5.2, 5.3, 5.5_
-  
+
   - [x] 2.2 Implement input validation functions
     - Create validation for `limit` parameter (1-1000 range, default 50)
     - Create validation for `db_path` parameter (string type check)
     - Create validation for `cursor` parameter (format validation)
     - Return structured `VALIDATION_ERROR` for invalid inputs
     - _Requirements: 1.3, 1.4, 6.3_
-  
+
   - [x]* 2.3 Write unit tests for validation
     - Test `limit` validation: default, valid range, below 1, above 1000
     - Test `db_path` validation: valid paths, invalid types
@@ -41,12 +41,12 @@ This plan implements a read-only MCP tool for retrieving jobs with `status='new'
     - Use base64 encoding for cursor opacity
     - Handle malformed cursor strings with `VALIDATION_ERROR`
     - _Requirements: 1.5, 2.4, 6.3_
-  
+
   - [x]* 3.2 Write property test for cursor round-trip
     - **Property 1: Cursor round-trip consistency**
     - **Validates: Requirements 1.5**
     - For any valid `(captured_at, id)` pair, `decode_cursor(encode_cursor(captured_at, id))` should return the original values
-  
+
   - [ ]* 3.3 Write unit tests for cursor edge cases
     - Test cursor with null/empty values
     - Test cursor with special characters in timestamp
@@ -62,7 +62,7 @@ This plan implements a read-only MCP tool for retrieving jobs with `status='new'
     - Return `DB_ERROR` for connection failures
     - Ensure connections are always closed via context management
     - _Requirements: 2.1, 2.2, 2.5, 2.6, 4.5_
-  
+
   - [x] 4.2 Implement deterministic query function
     - Create `query_new_jobs(conn, limit, cursor)` function
     - Filter by `status='new'` using parameterized SQL
@@ -71,15 +71,15 @@ This plan implements a read-only MCP tool for retrieving jobs with `status='new'
     - Query `limit + 1` rows to compute `has_more` flag
     - Return list of row dictionaries with fixed schema fields
     - _Requirements: 2.2, 2.3, 2.4
-  
+
   - [ ]* 4.3 Write unit tests for database reader
     - Test connection with valid database path
     - Test connection with non-existent database (expect `DB_NOT_FOUND`)
     - Test query with empty result set
     - Test query returns only `status='new'` jobs
     - Test deterministic ordering (repeated queries return same order)
     - _Requirements: 2.1, 2.3, 2.4, 2.5, 2.6_
-  
+
   - [ ]* 4.4 Write property test for read-only guarantee
     - **Property 2: Read-only operations**
     - **Validates: Requirements 4.1, 4.2, 4.3, 4.4**
@@ -98,12 +98,12 @@ This plan implements a read-only MCP tool for retrieving jobs with `status='new'
     - Implement `paginate_results(rows, limit)` returning `(page, has_more, next_cursor)`
     - Handle edge case when result set is empty
     - _Requirements: 1.5, 2.4, 6.4
-  
+
   - [ ]* 6.2 Write property test for pagination determinism
     - **Property 3: Pagination determinism**
     - **Validates: Requirements 1.5, 2.4**
     - For any database state, paginating through all results should produce deterministic, non-overlapping pages that union to the complete result set
-  
+
   - [ ]* 6.3 Write unit tests for pagination edge cases
     - Test pagination with exactly `limit` results (no more pages)
     - Test pagination with `limit + 1` results (one more page)
@@ -119,7 +119,7 @@ This plan implements a read-only MCP tool for retrieving jobs with `status='new'
     - Ensure all fields are JSON-serializable
     - Do not include arbitrary additional database columns
     - _Requirements: 3.1, 3.2, 3.3, 3.4, 3.5_
-  
+
   - [ ]* 7.2 Write unit tests for schema mapping
     - Test complete row mapping with all fields present
     - Test row mapping with missing/null fields
@@ -137,7 +137,7 @@ This plan implements a read-only MCP tool for retrieving jobs with `status='new'
     - Build response with `jobs`, `count`, `has_more`, `next_cursor` fields
     - Wrap all errors in structured error format
     - _Requirements: 1.1, 1.2, 6.1, 6.2, 6.3, 6.4, 6.5_
-  
+
   - [ ]* 8.2 Write integration tests for tool handler
     - Test tool with default parameters (limit=50, no cursor)
     - Test tool with custom limit
@@ -155,7 +155,7 @@ This plan implements a read-only MCP tool for retrieving jobs with `status='new'
     - Define tool output schema matching design specification
     - Wire tool handler to MCP server invocation
     - _Requirements: 6.1, 6.2, 6.3, 6.4_
-  
+
   - [x] 9.2 Add tool metadata and documentation
     - Set tool name: `bulk_read_new_jobs`
     - Set tool description for LLM agents
@@ -176,17 +176,17 @@ This plan implements a read-only MCP tool for retrieving jobs with `status='new'
     - **Property 4: Batch size validation**
     - **Validates: Requirements 1.3, 1.4**
     - For any limit value outside [1, 1000], the tool should return `VALIDATION_ERROR`
-  
+
   - [ ]* 11.2 Write property test for result count accuracy
     - **Property 5: Result count accuracy**
     - **Validates: Requirements 6.5**
     - For any query result, `count` field should equal `len(jobs)` array
-  
+
   - [ ]* 11.3 Write property test for schema stability
     - **Property 6: Schema stability**
     - **Validates: Requirements 3.1, 3.2, 3.5**
     - For any job record returned, it should contain exactly the fixed schema fields and no additional fields
-  
+
   - [ ]* 11.4 Write property test for JSON serializability
     - **Property 7: JSON serializability**
     - **Validates: Requirements 3.4**

.kiro/specs/bulk-update-job-status/.config.kiro

@@ -1 +1 @@
-{"generationMode": "requirements-first"}
+{"generationMode": "requirements-first"}

.kiro/specs/bulk-update-job-status/design.md

@@ -195,25 +195,25 @@ ALLOWED_STATUSES = {"new", "shortlist", "reviewed", "reject", "resume_written",
 class JobsWriter:
     def __init__(self, db_path: str):
         """Initialize writer with database path"""
-    
+
     def __enter__(self):
         """Open connection and begin transaction"""
-    
+
     def __exit__(self, exc_type, exc_val, exc_tb):
         """Rollback on exception, close connection"""
-    
+
     def validate_jobs_exist(self, job_ids: list[int]) -> list[int]:
         """Returns list of missing job IDs (empty if all exist)"""
 
     def ensure_updated_at_column(self) -> None:
         """Raises schema error if jobs.updated_at column is missing"""
-    
+
     def update_job_status(self, job_id: int, status: str, timestamp: str) -> None:
         """Execute UPDATE for single job. Raises on DB error."""
-    
+
     def commit(self) -> None:
         """Commit transaction"""
-    
+
     def rollback(self) -> None:
         """Rollback transaction"""
 ```
@@ -327,10 +327,10 @@ with JobsWriter(db_path) as writer:
     timestamp = get_current_utc_timestamp()
     for update in updates:
         writer.update_job_status(update["id"], update["status"], timestamp)
-    
+
     # Commit if all succeed
     writer.commit()
-    
+
 # Rollback automatically on exception via __exit__
 ```
 
@@ -492,12 +492,12 @@ async def bulk_update_job_status(args: dict) -> dict:
                     update["status"],
                     timestamp
                 )
-            
+
             # Commit transaction
             writer.commit()
-            
+
             return build_success_response(updates)
-    
+
     except RequestValidationError as e:
         return build_validation_error(e)
     except DatabaseError as e:

.kiro/specs/bulk-update-job-status/tasks.md

@@ -13,13 +13,13 @@ This plan implements a write-only MCP tool for performing atomic batch status up
     - Check for case-sensitivity, whitespace, null/empty values
     - Return per-item validation messages for invalid statuses (not top-level error object)
     - _Requirements: 2.1, 2.2, 2.3, 2.4, 2.5_
-  
+
   - [x] 1.2 Add job ID validation function
     - Implement `validate_job_id(job_id)` function that validates positive integers
     - Check for null, non-integer, zero, and negative values
     - Return per-item validation messages for invalid job IDs (not top-level error object)
     - _Requirements: 3.4, 3.5_
-  
+
   - [x] 1.3 Add batch size validation function
     - Implement `validate_batch_size(updates)` function that checks 0-100 range
     - Return early for empty batches (valid case)
@@ -30,7 +30,7 @@ This plan implements a write-only MCP tool for performing atomic batch status up
     - Implement `validate_unique_job_ids(updates)` to reject duplicate job IDs within one batch
     - Raise `ValidationError` with descriptive message listing duplicate IDs
     - _Requirements: 3.6, 9.1_
-  
+
   - [ ]* 1.5 Write unit tests for new validation functions
     - Test status validation with valid statuses, invalid statuses, case variations, whitespace
     - Test job ID validation with positive integers, negative, zero, null, non-integers
@@ -46,7 +46,7 @@ This plan implements a write-only MCP tool for performing atomic batch status up
     - Rollback on exception in `__exit__`, close connection always
     - Handle `DB_NOT_FOUND` error when database file doesn't exist
     - _Requirements: 7.1, 7.2, 7.3, 7.5, 4.1, 4.2_
-  
+
   - [x] 2.2 Implement schema preflight for `updated_at`
     - Create `ensure_updated_at_column()` method
     - Check table schema before applying any updates
@@ -59,20 +59,20 @@ This plan implements a write-only MCP tool for performing atomic batch status up
     - Return list of missing job IDs (empty if all exist)
     - Use parameterized queries to prevent SQL injection
     - _Requirements: 3.1, 3.2, 3.3, 7.6_
-  
+
   - [x] 2.4 Implement status update method
     - Create `update_job_status(job_id: int, status: str, timestamp: str)` method
     - Execute `UPDATE jobs SET status = ?, updated_at = ? WHERE id = ?`
     - Use parameterized SQL for all values
     - Raise exception on database errors
     - _Requirements: 6.1, 7.6, 7.7_
-  
+
   - [x] 2.5 Implement transaction control methods
     - Create `commit()` method to commit transaction
     - Create `rollback()` method to rollback transaction
     - Ensure proper error handling for commit/rollback failures
     - _Requirements: 4.3, 4.1, 4.2_
-  
+
   - [ ]* 2.6 Write unit tests for database writer
     - Test connection with valid database path
     - Test connection with non-existent database (expect `DB_NOT_FOUND`)
@@ -89,7 +89,7 @@ This plan implements a write-only MCP tool for performing atomic batch status up
     - Use UTC and format `YYYY-MM-DDTHH:MM:SS.mmmZ`
     - Return string like `2026-02-04T03:47:36.966Z`
     - _Requirements: 6.1, 6.3_
-  
+
   - [ ]* 3.2 Write unit tests for timestamp generation
     - Test timestamp format matches ISO 8601 UTC with `Z` suffix and millisecond precision
     - Test timestamp is in UTC
@@ -112,7 +112,7 @@ This plan implements a write-only MCP tool for performing atomic batch status up
     - Validate each update item structure and values
     - Resolve database path (default or override)
     - _Requirements: 1.1, 1.2, 1.3, 1.4, 3.6, 7.1, 7.2, 10.1, 10.2_
-  
+
   - [x] 5.2 Implement transaction orchestration
     - Open `JobsWriter` context manager
     - Run `updated_at` schema preflight before update execution
@@ -123,15 +123,15 @@ This plan implements a write-only MCP tool for performing atomic batch status up
     - Commit transaction on success
     - Rollback automatically on any failure via context manager
     - _Requirements: 4.1, 4.2, 4.3, 6.1, 6.2, 6.4, 3.1, 3.2, 3.3_
-  
+
   - [x] 5.3 Implement response building
     - Create `build_success_response(updates)` helper function
     - Create `build_failure_response(updates, failures)` helper function
     - Build response with `updated_count`, `failed_count`, `results` array
     - Each result entry has `id`, `success`, and optional `error` fields
     - Maintain input order in results array
     - _Requirements: 8.1, 8.2, 8.3, 8.4, 8.5_
-  
+
   - [x] 5.4 Implement error handling and sanitization
     - Wrap all operations in try-except blocks
     - Catch request-level `ValidationError` and return structured top-level error response
@@ -140,7 +140,7 @@ This plan implements a write-only MCP tool for performing atomic batch status up
     - Sanitize error messages (remove SQL, paths, stack traces)
     - Include `retryable` boolean in error responses
     - _Requirements: 9.1, 9.2, 9.3, 9.4, 9.5, 9.6_
-  
+
   - [ ]* 5.5 Write integration tests for tool handler
     - Test successful batch update (all jobs exist, valid statuses)
     - Test empty batch returns success with zero counts
@@ -161,15 +161,15 @@ This plan implements a write-only MCP tool for performing atomic batch status up
     - Define output schema matching design specification
     - Wire tool handler to MCP server invocation
     - _Requirements: 10.1, 10.4, 10.5_
-  
+
   - [x] 6.2 Add tool metadata and documentation
     - Set tool name: `bulk_update_job_status`
     - Set tool description for LLM agents
     - Document input parameters: `updates` (required array), `db_path` (optional string)
     - Document output structure: `updated_count`, `failed_count`, `results`
     - Document error codes: `VALIDATION_ERROR`, `DB_NOT_FOUND`, `DB_ERROR`, `INTERNAL_ERROR`
     - _Requirements: 10.1, 10.4, 10.5_
-  
+
   - [ ]* 6.3 Write MCP server integration test
     - Test tool can be invoked through MCP server interface
     - Test tool registration and metadata are correct
@@ -190,57 +190,57 @@ This plan implements a write-only MCP tool for performing atomic batch status up
     - **Property 1: Atomic batch updates**
     - **Validates: Requirements 1.1, 4.1, 4.2, 4.3**
     - For any batch with one invalid update, verify no updates are committed
-  
+
   - [ ]* 8.2 Write property test for batch size validation
     - **Property 2: Batch size validation**
     - **Validates: Requirements 1.4**
     - For any batch > 100 updates, verify `VALIDATION_ERROR` is returned
-  
+
   - [ ]* 8.3 Write property test for status validation
     - **Property 4: Status value validation**
     - **Validates: Requirements 2.1, 2.2, 2.4, 2.5**
     - For any invalid status value, verify a per-item failure result is returned
-  
+
   - [ ]* 8.4 Write property test for job ID validation
     - **Property 5: Job ID validation**
     - **Validates: Requirements 3.4**
     - For any non-positive-integer job ID, verify a per-item failure result is returned
-  
+
   - [ ]* 8.5 Write property test for idempotency
     - **Property 7: Idempotent updates**
     - **Validates: Requirements 5.1, 5.3, 5.4, 5.5**
     - For any job, updating to current status should succeed and refresh timestamp
-  
+
   - [ ]* 8.6 Write property test for batch idempotency
     - **Property 8: Batch idempotency**
     - **Validates: Requirements 5.2**
     - For any valid batch, submitting twice should produce same final `status` state
-  
+
   - [ ]* 8.7 Write property test for timestamp consistency
     - **Property 9: Timestamp consistency**
     - **Validates: Requirements 6.1, 6.2, 6.3**
     - For any batch, all jobs should get same ISO 8601 timestamp
-  
+
   - [ ]* 8.8 Write property test for field isolation
     - **Property 10: Timestamp field isolation**
     - **Validates: Requirements 6.5, 11.5**
     - For any update, only `status` and `updated_at` should change
-  
+
   - [ ]* 8.9 Write property test for response structure
     - **Property 15: Response structure completeness**
     - **Validates: Requirements 8.1, 8.2, 8.3, 8.4**
     - For any invocation, response should have required fields with correct types
-  
+
   - [ ]* 8.10 Write property test for result ordering
     - **Property 16: Result ordering preservation**
     - **Validates: Requirements 8.5**
     - For any batch, result order should match input order
-  
+
   - [ ]* 8.11 Write property test for JSON serializability
     - **Property 18: JSON serializability**
     - **Validates: Requirements 10.3**
     - For any response, all fields should be JSON-serializable
-  
+
   - [ ]* 8.12 Write property test for SQL injection prevention
     - **Property 13: SQL injection prevention**
     - **Validates: Requirements 7.6**