workflow visualization library

[frankiethull]

placing this here to request a 'workflow' visualization tool as part of the tidymodels org. Thinking of how we take raw data -> pre-process steps -> engine -> tuning / cv method -> post-processors as an autoplot diagram for workflow objects.

scikit-learn has something similar for 'pipelines' showcased here: https://scikit-learn.org/stable/auto_examples/compose/plot_column_transformer_mixed_types.html

[EmilHvitfeldt]

We have seen this request a few times before, do you mind elaborating what type of information you would like to be able to extract with this type of view?

Tidymodels workflows tend to be simpler since they don't allow branching as opposed to scikit learn pipelines

[frankiethull]

in my mind, I was thinking quarto + mermaid styled output diagrams to share with downstream non-ML / non-programmer stakeholders on a project. To show-off the data flow more or less.

Something similar to this example:

where we might have recipe steps for numerical vs categorical columns, see what those steps do before the data hits the engine, maybe the engine shows 'defaults' or tune::tune(), then any tailor post-process that is part of the workflow (?). Less about branching but how the workflow could be visually displayed. It could be a useful tool for me to teach folks how tidymodels works too.

[EmilHvitfeldt]

We might be able to add this information on a broad scale. but for now each individual step doesn't have input/output information stored in them. We have that tracked in tidymodels/recipes#1158, but it will take a while before we get to that as it is a very invasive change