Skip to content

Migrate event log to batched logging system #8544

@HyeockJinKim

Description

@HyeockJinKim

Objective

Migrate event log creation from direct DB writes to the batched logging system.

Implementation Details

Event Reporter Changes

  1. Update EventLogger (src/ai/backend/manager/event_dispatcher/reporters.py)
    • Replace direct DB write with queue enqueue
    • Batch multiple events together
class EventLogger(AbstractEventReporter):
    def __init__(self, db: ExtendedAsyncSAEngine, queue_manager: LogQueueManager) -> None:
        self._db = db
        self._queue_manager = queue_manager
    
    async def prepare_event_report(
        self, event: AbstractEvent, arg: PrepareEventReportArgs
    ) -> None:
        event_log = EventLogRow.from_event(event)
        await self._queue_manager.enqueue_event_log(event_log)
  1. Add Batch Writer for Events
    • Create EventLogBatchWriter
    • Handle bulk insert to event_logs table

Configuration

  • Batch size: 200 (higher than other logs, events are frequent)
  • Flush interval: 10 seconds (can tolerate delay)
  • Redis backup: disabled

Testing

  • Test event log enqueueing
  • Test high-volume event scenarios
  • Integration tests with event dispatcher

Acceptance Criteria

  • Event logs written in batches
  • No event data loss
  • Reduced DB load during high event volume
  • All tests passing

JIRA Issue: BA-4236

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No fields configured for Story.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions