Live/retrospective data views

[liamhuber]

And a toy WfMS. Follows the concept of the mutable instance views being separate, not-easily-serializable, and living alongside and coupled to their underlying recipe. All the retrospective views with internal data are structured to be plain DAGs.

I'm not even sure these should live in this repo, but I want a place to play around and have conversations against real code examples.

EDIT:

This turned out better than I'd hoped.

It introduces a new LiveNode data class, to be a basis for instance-views of recipes. This holds an associated recipe field, and input_ports and output_ports. The ports should have all the same keys as the recipe.inputs and recipe.outputs, but the ports make space for values and annotations (and defaults, in the case of input ports).

This basis is then extended for Atomic(LiveNode), which is associated with AtomicNode recipes and additionally holds a live function field (from instantiating the PythonReference in the recipe), and Workflow(Composite(LiveNode)) and FlowControl(Composite(LiveNode)) subclasses, which all have nodes: MutableMapping[base_models.Label, LiveNode], and dictionaries for input_edges, edges, and output_edges. The difference between the last two is that the Workflow live node knows all its child instances and edges from the get-go, so they can be populated at creation. The flow controls know only prospective nodes and edges from their recipes, so these are instantiated as empty containers and it's up to the WfMS to fill them at runtime.

The primary API is a recipe2live function, but you can also .from_recipe the Atomic, Workflow, and FlowControl classes directly if you already know what sort of recipe you've got.

Then there is a WfMS module with a toy implementation. It has an API run_recipe which converts a recipe object into an output-data-populated live object.

@samwaseda, FYI this is working basically as I envisioned and I'm pretty happy with it. What I'll do now is try and build a converter from a Workflow live node to the legacy nested dictionary. This might turn up some pieces that are missing here. Once that conversion is working, it works then the conversion and .run method binding can both be done over in semantikon, and then @semantikon.workflow.workflow can directly extend @flowrep.models.api.workflow.

EDIT:

And a super simple example

from flowrep.models.api import parsers, wfms

def add(x, y):
    return x + y

def negate(x):
    return -x

def mul(x, y):
    return x * y

@parsers.workflow
def my_wf(x: float, y: float) -> float:
    s = add(x, y)
    n = negate(x)
    p = mul(s, n)
    return p

retrospective = wfms.run_recipe(my_wf.flowrep_recipe, x=2., y=3.)
retrospective.output_ports.p.value, retrospective.output_ports.p.annotation
>>> (-10.0, float)

[github-actions]

👈 Launch a binder notebook on branch pyiron/flowrep/retrospective

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.23%. Comparing base (507e9b2) to head (bcf63b0).
⚠️ Report is 83 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #185      +/-   ##
==========================================
+ Coverage   97.96%   99.23%   +1.26%     
==========================================
  Files          34       32       -2     
  Lines        2259     2598     +339     
==========================================
+ Hits         2213     2578     +365     
+ Misses         46       20      -26     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[liamhuber]

@samwaseda, one of the key differences to the .run method appended on the semantikon workflows is to not accept *args but only **kwargs. This is possible because the recipes all log both the input names (which might differ from the signature of the underlying function) and the input kind (e.g. positional only).

It can sometimes be a bit more of a pain to write, but I find it reads much easier. Delaying the arg-vs-kwarg until we finally call the underlying atomic python function saves some maintenance headache too, since then it only becomes relevant at exactly one spot.

[Copilot]

Pull request overview

This PR introduces “live” (runtime) node objects and a minimal toy Workflow Management System (WfMS) that can execute flowrep recipes into populated live graphs, along with a comprehensive unit test suite to validate execution behavior across atomic/workflow/flow-control recipes.

Changes:

• Added flowrep.models.live live-node data structures (ports, atomic/composite nodes, recipe→live conversion).
• Added flowrep.models.wfms minimal executor (run_recipe) supporting atomic, workflow, for/while/if/try nodes.
• Added extensive unit tests and expanded the static test library with additional atomic functions.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
flowrep/models/live.py	New live node/port structures and function signature/type parsing for live ports.
flowrep/models/wfms.py	New minimal WfMS to execute recipes and populate live graphs and port values.
flowrep/models/api/live.py	API re-export for live recipe→live conversion.
flowrep/models/api/wfms.py	API re-export for run_recipe.
flowrep/models/api/schemas.py	Exposes live classes/types through the API schemas module.
flowrep/models/parsers/atomic_parser.py	Tweaks the output-label mismatch error message.
flowrep/models/nodes/for_model.py	Documentation update clarifying runtime expectations for iterated inputs.
tests/unit/models/test_live_wfms.py	New end-to-end unit tests covering live models and WfMS execution.
tests/flowrep_static/library.py	Adds more atomic functions used by the new WfMS/live tests.
.github/workflows/push-pull.yml	Adjusts CI coverage omission patterns for API modules.
.coveragerc	Simplifies coverage config (removes omit list).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[liamhuber]

Since the live data structures have room for annotations, I this PR gets far enough to make me comfortable closing #17. The rest is then application in other repos, and edits here to incorporate feedback from those experiences.