Skip to content

A better implementation of RocketMQEventSubscriber #82

@supercym

Description

@supercym

Problem Description
The current message pull logic of rocketMQEventSubscriber has the following problems:

  1. Consumer pull is bound to commit, which affects pull efficiency
  2. The pull mode of the consumer will cause certain performance pressure on the rocketmq broker when there is no message on the broker.

Solution
Implement a LitePullConsumer, the change after implementation are:

  1. LitePullConsumer has a built-in cache queue to decouple poll and commit;
  2. Support proxy mode;
  3. Message pull is implemented using pullBlockIfNotFound + PullCallback;
    • 3.1 pullBlockIfNotFound uses the long polling mode to avoid frequent pulls when there is no message on the MQ server which resulting in an increase in server load;
    • 3.2 However, only using the long polling method will also bring certain problems, that is, when the number of MQ topic queues is greater than the number of pulling threads and all threads are long polling on the server side, some queues will be starved;
    • 3.3 So the callback is used here to submit the next pull request of this queue when dealing response of each pull request.
  4. Added configuration to support aliyun Rocketmq.
  5. Implementation on native pullConsumer implementation, based on the fact that not all versions and environments of rocketmq support LitePullConsumer.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions