feat: improve offline inference interface and fix several tp and vlm bugs.#968

RobbieLeung merged 3 commits intojd-opensource:mainjd-opensource/xllm:mainfrom

weizhehuang0827:offline_rebase_mainweizhehuang0827/xllm:offline_rebase_mainCopy head branch name to clipboard