Skip to content

[DOP-34706] add indirect dependencies into job hierarchy request#416

Open
TiGrib wants to merge 4 commits intodevelopfrom
feature/DOP-34706
Open

[DOP-34706] add indirect dependencies into job hierarchy request#416
TiGrib wants to merge 4 commits intodevelopfrom
feature/DOP-34706

Conversation

@TiGrib
Copy link
Contributor

@TiGrib TiGrib commented Mar 25, 2026

Change Summary

  • Add optional indirect job-to-job dependency inference for GET /v1/jobs/hierarchy via infer_from_lineage.
  • Extend hierarchy request with optional since/until bounds and validation (since < until, and infer_from_lineage requires since).

Related issue number

[DOP-34706]

Checklist

  • Commit message and PR title is comprehensive
  • Keep the change as small as possible
  • Unit and integration tests for the changes exist
  • Tests pass on CI and coverage does not decrease
  • Documentation reflects the changes where applicable
  • docs/changelog/next_release/<pull request or issue id>.<change type>.rst file added describing change
    (see CONTRIBUTING.rst for details.)
  • My PR is ready to review.

@TiGrib TiGrib requested a review from dolfinus March 25, 2026 08:21
@TiGrib TiGrib self-assigned this Mar 25, 2026
@github-actions
Copy link

github-actions bot commented Mar 25, 2026

Coverage

Coverage Report •
FileStmtsMissBranchBrPartCoverMissing
data_rentgen/server/schemas/v1
   job.py6594486%143–147, 151–154
data_rentgen/server/services
   job.py4710097%75
TOTAL7918980123821687% 

@TiGrib TiGrib force-pushed the feature/DOP-34706 branch from 06e6ff3 to 73ab952 Compare March 25, 2026 08:49
@TiGrib TiGrib marked this pull request as ready for review March 26, 2026 13:50
Comment on lines +149 to +154
@model_validator(mode="after")
def _check_indirect_flag(self):
if self.infer_from_lineage and not self.since:
msg = "Inferring from lineage graph only possible with 'since' param"
raise ValueError(msg)
return self
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
@model_validator(mode="after")
def _check_indirect_flag(self):
if self.infer_from_lineage and not self.since:
msg = "Inferring from lineage graph only possible with 'since' param"
raise ValueError(msg)
return self
@field_validator("since", mode="after")
@classmethod
def _check_since(cls, value: datetime | None, info: ValidationInfo) -> datetime | None:
infer_from_lineage = info.data.get("infer_from_lineage")
if infer_from_lineage and not value:
msg = "'since' is mandatory when 'infer_from_lineage' is used"
raise ValueError(msg)
return value



@pytest_asyncio.fixture
async def job_dependency_chain_with_indirect_dependencies(
Copy link
Member

@dolfinus dolfinus Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
async def job_dependency_chain_with_indirect_dependencies(
async def job_dependency_chain_with_lineage(

Comment on lines +484 to +485
- left_task -> task1 (indirect via input/output relation)
- task3 -> right_task (indirect via input/output relation)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- left_task -> task1 (indirect via input/output relation)
- task3 -> right_task (indirect via input/output relation)
- left_task -> task1 (inferred from via input/output relation)
- task3 -> right_task (inferred from via input/output relation)

@pytest.mark.parametrize(
["direction", "depth", "start_node_idx", "expected_deps"],
[
("UPSTREAM", 1, 1, [(0, 1, "INFERRED_FROM_LINEAGE")]),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
("UPSTREAM", 1, 1, [(0, 1, "INFERRED_FROM_LINEAGE")]),
pytest.param("UPSTREAM", 1, 1, [(0, 1, "INFERRED_FROM_LINEAGE")], id="indirect-upstream-depth-1"),

IMHO it's more convenient than list of ids

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants