local backend oom

[sleepymole]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. What did you do? If possible, provide a recipe for reproducing the error.
   I used lightning to restore csv files (tpcc 5000 warehouses).

2. What did you expect to see?
   Restored successfully.

3. What did you see instead?
   Lightning oom. In January, the memory usage of lightning is about 20~30 GB. But now it costs at least 60GB.

4. Versions of the cluster

   • TiDB-Lightning version (run tidb-lightning -V):

     Release Version: v5.0.0-rc-21-g230eef2
     Git Commit Hash: 230eef2a6e16648a49a4c74910dca693781012c4
     Git Branch: master
     UTC Build Time: 2021-02-04 03:10:38
     Go Version: go version go1.15.6 linux/amd64
     

   • TiKV-Importer version (run tikv-importer -V):

     none
     

   • TiKV version (run tikv-server -V):

     TiKV
     Release Version:   5.0.0-rc.x
     Edition:           Community
     Git Commit Hash:   81c4de98a9a21e4dcf3cce6d7783793b1238044e
     Git Commit Branch: limit-write-batch-ingest
     UTC Build Time:    2021-02-04 11:43:07
     Rust Version:      rustc 1.51.0-nightly (1d0d76f8d 2021-01-24)
     Enable Features:   jemalloc mem-profiling portable sse protobuf-codec test-engines-rocksdb
     Profile:           release
     

   • TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

     Release Version: v4.0.0-beta.2-2067-g415d14b6a
     Edition: Community
     Git Commit Hash: 415d14b6ac65e3c73529d07b4331c2f4917b2701
     Git Branch: master
     UTC Build Time: 2021-01-27 15:27:10
     GoVersion: go1.13
     Race Enabled: false
     TiKV Min Version: v3.0.0-60965b006877ca7234adaced7890d7b029ed1306
     Check Table Before Drop: false
     

   • Other interesting information (system version, hardware config, etc):

5. Operation logs

   • Please upload tidb-lightning.log for TiDB-Lightning if possible
   • Please upload tikv-importer.log from TiKV-Importer if possible
   • Other interesting logs

6. Configuration of the cluster and the task

   • tidb-lightning.toml for TiDB-Lightning if possible
   • tikv-importer.toml for TiKV-Importer if possible
   • inventory.ini if deployed by Ansible

7. Screenshot/exported-PDF of Grafana dashboard or metrics' graph in Prometheus for TiDB-Lightning if possible
   

[sleepymole]

#560 is trying to solve this issue.