MySQL Input 的配置
[input]
type = "mysql"
mode = "batch"
[input.config]
nr-scanner = 10
table-scan-batch = 10000
batch-per-second-limit = 10
max-full-dump-count = 10000
ES Output 的配置
[output]
type="custom-elasticsearch"
[output.config]
ignore-bad-request = true
Scheduler 的配置
[scheduler]
type = "batch-table-scheduler"
[scheduler.config]
nr-worker = 10
batch-size = 1000
queue-size = 1024
sliding-window-size = 10240
nr-retries = 3
retry-sleep = "2s"
按照这个配置,在执行到 func (output *customESOutput) Execute(msgs []*core.Msg) error 函数时,msgs 数量总是为1,导致 ES 的 bulk request 形同虚设。
请问,应该如何调整配置,解决这个问题。