Tabular Deep Learning models in R

[topepo]

I wrote a few notes on what we might want to do here: https://github.com/tidymodels/planning/blob/main/tabular-DL/tabular-DL-methods.md

Some folks that might be interested: @edgararuiz, @frankiethull, @VisruthSK, @cregouby, and @dfalbel. Please point others towards this issue if you think that they are interested.

[frankiethull]

thanks @topepo - i have a few reticulated tabular foundation models to add to the list: https://github.com/frankiethull#tfms

I also started a realmlp repo, translating the 'standalone' simplified repo to torch for R. More advanced RealMLP embedding exists in pytabkit.

KANs might be worth adding to this list too.

[topepo]

That's great. Y'all can PR into this repo, and we can add things to the list.

[topepo]

I think I'll make a parsnip extension package for tabular models. The implementations can be pretty much anywhere. If we want tidymodels to use them, we can add model types and engines to that package (tidytabular?). I'll start by adding parsnip model types for TabPFN and ResNet there.

[frankiethull]

@kdpsingh suggested rutabaga 😆

[topepo]

🙄 😆

[cregouby]

I must admit I've let the Tabular domain go in favor of the vision domain, but I'm very happy to see it active !

[topepo]

I had time to spend on this in the last few days. My process was to build torch versions of ResNet and AutoInt, then integrate them into the R brulee package.

The idea is to have the direct implementations in some package (brulee or anywhere else, really), then have a parsnip extension package (as mentioned above).

@edgararuiz is working on Regularization Learning Networks and will probably follow the same process.

Is there any specific model that folks would like to work on (to avoid duplication or to help collaboration)?

[topepo]

After the PR for AutoInt, I'm going to spend some time on the tabpfn package so we can work out some of the issues posted there. That won't start for me for another 2 weeks, though. If you want to help out there, please let me know

[cregouby]

I have start working on TabICL2 on my spare time. I'll be happy to transfer it to the mlverse when it starts achieving sth.

[VisruthSK]

I'm down to help with tabpfn

[frankiethull]

I have lightweight reticulate bindings for tabicl, tabdpt, limix, iltm, mitra, orion, & contexttab. @cregouby - the tabicl repo I have does default to TabICL2. But note, none of these repos are registered to parsnip yet.

@topepo - would you prefer to have the parsnip registration in each package or in the 'tidytabular' package? Another thing I want to keep in mind is the virtual env design. limix was really gnarly to setup, I'm thinking the virtual env might be an engine parameter & could be different for each engine. Then each fit/pred could be ran without loading a virtual env in the current R session to avoid any venv clash. ?

[topepo]

would you prefer to have the parsnip registration in each package or in the 'tidytabular' package

Yes. I wouldn't put the implementation there, just the tidymodels bits (like we do with rules, bonsai, or other extensions).

The VE is potentially problematic. Using them as an engine parameter hadn't occurred to me. Let me talk with someone at Posit about this, who would probably have a better sense of whether that is a good solution.

I've been trying to use (R) torch if possible, but I recognize that it is pretty difficult in a lot of cases.

[frankiethull]

I've been trying to use (R) torch if possible, but I recognize that it is pretty difficult in a lot of cases.

for the ones mentioned above, I had created the reticulate bindings to avoid translating to torch with R. But the VE has been in the back of my mind, thought of using callr with different default VEs.. looks like @cregouby has went the complete opposite route with their TabICL2 R torch implementation 🥳

[t-kalinowski]

Hi, this is exciting work!

I'm thinking the virtual env might be an engine parameter & could be different for each engine. Then each fit/pred could be ran without loading a virtual env in the current R session to avoid any venv clash. ?

Rather than creating user-visible venvs, it might be simpler to make the appropriate py_require() in R. With py_require(), users encounter python-dependency conflicts much less frequently, since each R session materializes only the minimal set of required dependencies in an ephemeral (and cached) venv.

[topepo]

I updated the list based on Frank's PR and also added Chronos2.

In brulee we have:

• resnet (merged)
• Regularization learning networks (PR upcoming)
• AutoInt (PR upcoming)
• Chronos2 (nearly a PR)