See Discussion #35 and Pull Request #37
Originally posted by javide February 7, 2025
Summary:
I propose introducing a way to override the pipeline definition configured for a distributed system with remote elements to a local deployment. This will streamline development and debugging, reducing the need for maintaining separate pipeline definitions for local and production environments.
Background:
Currently, our pipeline definitions support remote deployment configurations, which work well for test and production environments. However, for development and debugging, using remote services can be slow, inconvenient, or even impractical. To work around this, developers often duplicate pipeline definitions—one for local use and another for test/prod—leading to potential misalignments and configuration drift.
Proposal:
To simplify development, I propose a solution that programmatically replaces remote elements in the pipeline definition with local equivalents. This will allow developers to run the pipeline entirely locally without modifying the core configuration files manually.
The function will:
-
Traverse the pipeline definition and locate all remote deployments.
-
Convert them into local deployments while preserving other configurations.
-
Remove the service_filter field, as it is specific to remote deployments.
See Discussion #35 and Pull Request #37
Originally posted by javide February 7, 2025
Summary:
I propose introducing a way to override the pipeline definition configured for a distributed system with remote elements to a local deployment. This will streamline development and debugging, reducing the need for maintaining separate pipeline definitions for local and production environments.
Background:
Currently, our pipeline definitions support remote deployment configurations, which work well for test and production environments. However, for development and debugging, using remote services can be slow, inconvenient, or even impractical. To work around this, developers often duplicate pipeline definitions—one for local use and another for test/prod—leading to potential misalignments and configuration drift.
Proposal:
To simplify development, I propose a solution that programmatically replaces remote elements in the pipeline definition with local equivalents. This will allow developers to run the pipeline entirely locally without modifying the core configuration files manually.
The function will:
Traverse the pipeline definition and locate all remote deployments.
Convert them into local deployments while preserving other configurations.
Remove the service_filter field, as it is specific to remote deployments.