Skip to content

update eval config#818

Closed
Ceceliachenen wants to merge 3 commits intofeaturefrom
personal/ranxia/fix_eval
Closed

update eval config#818
Ceceliachenen wants to merge 3 commits intofeaturefrom
personal/ranxia/fix_eval

Conversation

@Ceceliachenen
Copy link
Copy Markdown
Collaborator

  1. 修复eval上传新的数据集时出现的错误
  2. run_config 支持并发评估eval数据,并在前端透出
  3. 支持自定义并保存eval llm评估prompt,并在前端透出

@github-actions
Copy link
Copy Markdown

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines Covered Coverage Threshold Status
13875 5523 40% 0% 🟢

New Files

File Coverage Status
backend/evaluation/evaluator/prompts/eval_prompts.py 53% 🟢
TOTAL 53% 🟢

Modified Files

File Coverage Status
backend/api/v1/config_apis/evaluation.py 19% 🟢
backend/db/models/evaluation/evaluator_config.py 89% 🟢
backend/db/models/evaluation/run_config.py 91% 🟢
backend/evaluation/evaluator/llm_judge_evaluator.py 22% 🟢
backend/evaluation/evaluator/utils.py 59% 🟢
backend/rag/evaluation_tool.py 23% 🟢
backend/service/tool/evaluation_service.py 15% 🟢
TOTAL 45% 🟢

updated for commit: 1105dfa by action🐍

@moria97
Copy link
Copy Markdown
Collaborator

moria97 commented Jan 29, 2026

这个pr可以拆成两个吗?一个fix bug,一个加自定义prompt的逻辑,我打算年前发布一个和线上0128一样的版本,不改动数据库的。自定义prompt的可以放在后面

# Use safe formatting to handle JSON schema braces in the prompt
# First, replace double braces with a placeholder, then format, then restore
# Replace {{ and }} with placeholders
template = self.prompt_template.replace('{{', '___DOUBLE_BRACE_OPEN___').replace('}}', '___DOUBLE_BRACE_CLOSE___')
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个替换是为了做什么呀

model_provider_name: Optional[str] = Field(default="openai_like")
case_sensitive: bool = Field(default=False)
ignore_punctuation: bool = Field(default=False)
llm_judge_prompt: Optional[str] = Field(default=None)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit,是不是直接叫prompt_template比较直接?

@moria97 moria97 closed this Apr 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants