interaction and dependencies

[caseyjlaw]

Can nbflow detect dependencies that can't be generated by nbconvert?

I have a use case for a notebook flow that has interaction early on. E.g., An interactive notebook (with a widget) in stage 1 defines a set of windows that I use to visualize data in stage 2. The window definition is best done interactively, so it can't be run by nbconvert (if I understand that right).

In my playing with nbflow, a dependency must be generated by a notebook with nbconvert in order to for the dependency identified by scons. That is, I defining depends in stage 2 to a text file that I create with a text editor. Updating/editing that file did not trigger a build when running scons.

So I guess I'm hoping there's a way to get scons to identify the dependency in this use case. Any thoughts would be welcome. I'm experimenting at this point.

BTW, I saw your talk at the BIDS meetup last week. Great stuff!

[jhamrick]

Argh, sorry I missed this! Apparently I wasn't watching the repo so I wasn't notified of new issues 😦 In any case, I am now...

The dependencies shouldn't necessarily need to be generated by nbconvert -- scons just looks at modification times. So if the modification time of a dependency is newer than the modification time of the notebook, it should trigger the build.

[caseyjlaw]

No worries. I let this slide for a bit, but will test again.
In the meantime, I have developed a bit of code on a concept related to nbflow (https://github.com/caseyjlaw/nbpipeline). The goal is to persist some data between repeated builds (either interactive or using nbconvert).
I was chatting with Matthias Bussonier at the BIDS Faire yesterday and he said this persistence issue is something the Jupyter team is keen to think about. Could I chat with you some time about this, to see if I can be useful?

[jhamrick]

Cool! I'd be happy to chat, though it would have to be in two weeks as I'm pretty swamped the next few days and then next week I will be away at a conference. Feel free to shoot me an email and we can set something up.