Skip to content

Storage Layer - DAG Source Adapters #9

@kination

Description

@kination

Description

Implement the source abstraction for syncing and fetching DAGs from file systems or remote locations.

Tasks

  • Define @behaviour Levicon.DAG.Source (list/0, fetch/1, on_change/1)
  • Implement LocalSource (dev/test — watch local directory)
  • Implement GitSource (MVP — git clone on startup, git pull on interval)
  • Implement sync job (Quantum-scheduled): diff ETags -> register/update/deactivate DAGs
  • Phase 2: Implement S3Source (ListObjectsV2 + ETag diff)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions