Skip to content

feat: add ModelScope backend support with modelscope:// protocol#1673

Open
pmady wants to merge 1 commit intodragonflyoss:mainfrom
pmady:feat/modelscope-backend
Open

feat: add ModelScope backend support with modelscope:// protocol#1673
pmady wants to merge 1 commit intodragonflyoss:mainfrom
pmady:feat/modelscope-backend

Conversation

@pmady
Copy link

@pmady pmady commented Feb 9, 2026

What this PR does / why we need it:

Adds a new backend for downloading models and datasets from ModelScope Hub using the modelscope:// URL scheme, similar to the existing Hugging Face backend.

URL Format

modelscope://[<repo_type>/]<owner>/<repo>[/<path>][@<revision>]

Examples

# Download a model repository
dfget modelscope://inclusionAI/Ling-1T -O /tmp/ling-1t/ --recursive

# Download a single file
dfget modelscope://inclusionAI/Ling-1T/config.json -O /tmp/config.json

# Download with authentication
dfget modelscope://inclusionAI/Ling-1T/config.json -O /tmp/config.json --ms-token=<token>

# Download a dataset
dfget modelscope://datasets/damo/squad-zh/train.json -O /tmp/train.json

# Download from a specific revision
dfget modelscope://inclusionAI/Ling-1T/config.json@v1.0 -O /tmp/config.json

Features

  • Support for models and datasets repository types
  • File listing via ModelScope API (/api/v1/models/{id}/repo/files) for recursive downloads
  • File download via ModelScope API (/api/v1/models/{id}/repo?FilePath=...)
  • Authentication via --ms-token CLI flag (injected as Authorization: Bearer header)
  • Range download support
  • Default revision is master (matching ModelScope convention)

Which issue(s) this PR fixes:

Closes dragonflyoss/dragonfly#4420

Does this PR introduce a user-facing change?

Yes. Users can now download files and repositories from ModelScope Hub using dfget modelscope://... with P2P acceleration.

Add a new backend for downloading models and datasets from ModelScope Hub
(modelscope.cn) using the modelscope:// URL scheme.

URL format: modelscope://[<repo_type>/]<owner>/<repo>[/<path>][@<revision>]

Features:
- Support for models and datasets repository types
- File listing via ModelScope API for recursive downloads
- Authentication via --ms-token CLI flag
- Range download support
- Default revision is 'master' (matching ModelScope convention)

Closes dragonflyoss/dragonfly#4420

Signed-off-by: Pradumna Saraf <pradumnasaraf@gmail.com>
Signed-off-by: pmady <pmady@users.noreply.github.com>
@pmady
Copy link
Author

pmady commented Feb 9, 2026

Hi maintainers, could you please add the enhancement label to this PR? The PR Label check requires one of: bug, enhancement, documentation, or dependencies. Thank you!

@codecov
Copy link

codecov bot commented Feb 9, 2026

Codecov Report

❌ Patch coverage is 52.52101% with 226 lines in your changes missing coverage. Please review.
✅ Project coverage is 50.52%. Comparing base (035ae91) to head (9597b39).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
dragonfly-client-backend/src/modelscope.rs 51.20% 222 Missing ⚠️
dragonfly-client/src/bin/dfget/main.rs 33.33% 4 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1673      +/-   ##
==========================================
- Coverage   50.83%   50.52%   -0.32%     
==========================================
  Files          83       84       +1     
  Lines       20029    20631     +602     
==========================================
+ Hits        10182    10424     +242     
- Misses       9847    10207     +360     
Files with missing lines Coverage Δ
dragonfly-client-backend/src/lib.rs 96.24% <100.00%> (+0.19%) ⬆️
dragonfly-client/src/bin/dfget/main.rs 49.92% <33.33%> (-0.08%) ⬇️
dragonfly-client-backend/src/modelscope.rs 51.20% <51.20%> (ø)

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@gaius-qi gaius-qi added the enhancement New feature or request label Feb 10, 2026
@gaius-qi
Copy link
Member

@pmady Thanks, I'll finish the review by this week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Supports directly pulling repositories from ModelScope

2 participants