[feature] sglang support

[wheresmyhair]

1. Supports sglang inference backend
2. bump lmflow version to 1.1.0

[research4pan]

Unittest is needed.

Main feature

• Update datasets to 3.6.0
• setup.py: add sglang as an optional dependency
• --inference_engine is for choosing vllm

Details

• src/lmflow/args.py

  • inference_tensor_parallel_size and inferece_gpu_memory_utilization is the latest argument for both vllm and sglang
    • vllm_tensor_parallel_size and vllm_gpu_memory_utilization will soon be deprecated (but still usable in this version with higher priority)

• src/lmflow/models/hf_decoder_model.py

  • Add @deprecated_args to allow automatic deprecated arg translation to latest args (improve backward compatibility, also in src/lmflow/utils/deprecated.py)
  • Add corresponding logics for sglang
  • line 539: bos_token -> "" for sglang following sglang's logics

Question

• src/lmflow/models/hf_decoder_model.py: line 539: bos_token -> "". This may lead to prompt distribution. Confirm its correctness
• src/lmflow/models/hf_model_mixin.py: line 555-559: Can be removed if not necessary

Suggestions

• Add README.md for how to use sglang inferencer
• Add unittest for vllm, SGLang inference
  • compare with the original version of vllm and SGLang with a small example
  • a small example, compare with huggingface generation, check inference probability

[wheresmyhair]

Notes on error ModuleNotFoundError: No module named 'common_ops' when using SGLang:

If you encounter this error, please try apt-get update and apt install numactl.

[wheresmyhair]

1. Added shutdown for sglang engine
2. Added unittest for sglang inferencer

[research4pan]

Add unittests. LGTM