6.work.md

评测

在 opencompass 目录下执行

python run.py --datasets ceval_gen \
--hf-path /root/internlm2/model/Shanghai_AI_Laboratory/internlm2-chat-7b \
--tokenizer-kwargs padding_side='left' truncation='left' \
trust_remote_code=True --model-kwargs trust_remote_code=True \
device_map='auto' --max-seq-len 2048 --max-out-len 16 --batch-size 4 --num-gpus 1 --debug \
-w=/root/internlm2/output

summary：

./6/summary_20240119_220149.csv ./6/summary_20240119_220149.txt

离线转换

lmdeploy convert internlm2-chat-7b  /root/internlm2/model/Shanghai_AI_Laboratory/internlm2-chat-7b

评测转换后模型

编写 config/eval.py

from mmengine.config import read_base
from opencompass.models.turbomind import TurboMindModel

with read_base():
    from .datasets.ceval.ceval_gen_5f30c7 import ceval_datasets
    from .summarizers.medium import summarizer


datasets = sum((v for k, v in locals().items() if k.endswith('_datasets')), [])

internlm_meta_template = dict(round=[
    dict(role='HUMAN', begin='<|User|>:', end='\n'),
    dict(role='BOT', begin='<|Bot|>:', end='<eoa>\n', generate=True),],eos_token_id=103028)

# config for internlm-chat-7b
internlm_chat_7b = dict(
    type=TurboMindModel,
    abbr='internlm2-chat-7b-turbomind',
    path='/root/internlm2/workspace',
    engine_config=dict(session_len=2048,
                       max_batch_size=32,
                       rope_scaling_factor=1.0),
    gen_config=dict(top_k=1,
                    top_p=0.8,
                    temperature=1.0,
                    max_new_tokens=100),
    max_out_len=100,
    max_seq_len=512,
    batch_size=2,
    concurrency=1,
    meta_template=internlm_meta_template,
    run_cfg=dict(num_gpus=1, num_procs=1),
)

models = [internlm2_chat_7b]

在 opencompass 目录下执行

python run.py configs/eval_internlm2_turbomind.py -w /root/internlm2/output

summary：

./6/summary_20240119_225618.csv ./6/summary_20240119_225618.txt