Skip to content

Empty batch sent after oversized message rejection causes retry loop with JSON parse error #87

@johnnagro

Description

@johnnagro

While working on #81 I noticed some logging and investigated.

When a message exceeds the maximum allowed size (32KB), it is correctly rejected by MessageBatch, but two subsequent issues cause a noisy retry loop.

Issue 1: Empty batch is sent to API

In send_worker.rb, when all messages in a batch are rejected due to size limits, the worker still calls @transport.send with an empty batch:

# send_worker.rb:46
res = @transport.send @api_key, @batch  # @batch may be empty here

The log shows:

[posthog-ruby] a message exceeded the maximum allowed size
[posthog-ruby] Sending request for 0 items

Issue 2: JSON parsing fails on non-JSON response

When the server receives an invalid/empty request, it may return a non-JSON error response. The transport code assumes JSON:

# transport.rb:53
error = JSON.parse(body)['error']

This throws a JSON::ParserError, which triggers the retry logic, resulting in:

[posthog-ruby] Retrying request, 10 retries left
[posthog-ruby] Retrying request, 9 retries left
...
[posthog-ruby] unexpected character: 'request' at line 1 column 1

Suggested fixes

  1. Skip sending if batch is empty in send_worker.rb
  2. Wrap JSON parsing in a rescue block in transport.rb

Environment

  • Ruby 3.4.5
  • Rails 8.1.1
  • posthog-ruby (local/dev version)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions