-
Notifications
You must be signed in to change notification settings - Fork 4
[ENH] Added support for 7B level language model learnwares. #2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
[DOC] modify details in docs
[DOC] change repo url
[DOC] change html tags
[FIX, DOC] fix readthedocs config bug, modify framework svg
[FIX] update readthedocs yaml
[MNT] update readthedocs config
[MNT] add sphinx_book_theme req
[FIX] fix readthedocs req with pip
[FIX] fix readthedocs req to install all
[DOC, FIX] update badge, update autodoc config, fix no logo in doc bug
[MNT] publish 0.3.2 version
[DOC] polish contents
[DOC] modify details
Feature/llm workflow
[MNT] modify workflow details
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds support for 7B-level language model learnwares by introducing a new LLM workflow, benchmark configurations, and example task definitions, and bumps the package version to 0.4.0.post1.
- Added YAML task definitions for various financial datasets (cra_*, convfinqa, cd, etc.)
- Introduced
examples/dataset_llm_workflowwitheval_config.py,build_market.py, and updated benchmark code - Updated documentation (README, CHANGES.rst) to include new LLM results and versioning
Reviewed Changes
Copilot reviewed 100 out of 100 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| examples/dataset_llm_workflow/extra_tasks/flare/*.yaml | New task definitions for LLM workflows |
| examples/dataset_llm_workflow/eval_config.py | Configured benchmark sets for medical, math, and finance |
| examples/dataset_llm_workflow/build_market.py | Added LLM market-building script |
| examples/dataset_llm_workflow/benchmark/config.py | Updated LEARNWARE_FIN, commented task entries |
| examples/dataset_llm_workflow/benchmark/base.py | New Benchmark class to manage learnware datasets |
| examples/dataset_llm_workflow/README.md | Usage instructions and results for LLM workflow |
| README.md & README_zh.md | Added LLM experimental results sections |
| docs/components/market.rst | Fixed typo in add_learnware description |
| CHANGES.rst | Bumped version and listed new LLM support |
Comments suppressed due to low confidence (1)
examples/dataset_llm_workflow/extra_tasks/flare/convfinqa.yaml:2
- [nitpick] Class name 'ConvFinQA' uses PascalCase while other task function references are lowercase. Ensure consistency of naming conventions across task definitions.
class: !function flare.ConvFinQA
Description
How has his PR been tested?
pytest tests/test_workflowunder upper directory of learnware.Types of changes