DFetch can manage dependencies

We make products that can last 15+ years; because of this we want to be able to have all sources available to build the entire project from source without depending on external resources. For this, we needed a dependency manager that was flexible enough to retrieve dependencies as plain text from various sources. svn externals, git submodules and git subtrees solve a similar problem, but not in a VCS-agnostic way or completely user-friendly way. We want self-contained code repositories without any hassle for end-users. Dfetch must promote upstreaming changes, but allow for local customizations. The problem is described thoroughly in managing external dependencies and sometimes is also known as vendoring.

Other tools that do similar things are Zephyr's West, CMake ExternalProject and other meta tools. See alternatives for a complete list.

Getting started | Manual | Troubleshooting | Contributing

Problems DFetch Solves

• Declarative code reuse across projects (inner sourcing)
• Compose multi-repo code bases into a single working tree
• Vendoring dependencies for reproducible builds
• Apply local patches while keeping upstream syncable
• VCS-agnostic dependency management
• Self-contained exports for releases or audits

Install

Stable

pip install dfetch

latest version

pip install git+https://github.com/dfetch-org/dfetch.git#egg=dfetch

Binary distributions

Each release on the releases page provides installers for all major platforms.

• Linux .deb & .rpm
• macOS .pkg
• Windows .msi

Example manifest

manifest:
  version: 0.0

  remotes:                                                        # declare common sources in one place
  - name: github
    url-base: https://github.com/                                 # Allow git modules
    default: true                                                 # Set it as default

  - name: sourceforge
    url-base: svn://svn.code.sf.net/p/

  projects:

  - name: cpputest-git-tag
    dst: Tests/cpputest-git-tag
    url: https://github.com/cpputest/cpputest.git                 # Use external git directly
    tag: v3.4                                                     # revision can also be a tag

  - name: tortoise-svn-branch-rev
    dst: Tests/tortoise-svn-branch-rev/
    remote: sourceforge
    branch: 1.10.x
    revision: '28553'
    src: src/*
    vcs: svn
    repo-path: tortoisesvn/code

  - name: tortoise-svn-tag
    dst: Tests/tortoise-svn-tag/
    remote: sourceforge
    tag: version-1.13.1
    src: src/*.txt
    vcs: svn
    repo-path: tortoisesvn/code

  - name: cpputest-git-src
    dst: Tests/cpputest-git-src
    repo-path: cpputest/cpputest.git
    src: src

Github Action

You can use DFetch in your Github Actions workflow to check your dependencies. The results will be uploaded to Github. Add the following to your workflow file:

jobs:
  dfetch-check:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      security-events: write
    steps:
      - name: Run Dfetch Check
        uses: dfetch-org/dfetch@main
        with:
          working-directory: '.' # optional, defaults to project root