Feature engineering for image data

[gustavomodelli]

Image Classification with Tidymodels

Suggest Recipe: step_image

Is It possible to create a recipe that makes image processing?

Is It possible to process images like array reshape in Keras?

The tidymodels are an excellent package but there are no options to image process

[topepo]

It's a great and would really improve the modeling environment.

I think that it would require the images to be stored within the recipe in a different format that is more conducive to how the data are usually handled. This could get a little tricky but @EmilHvitfeldt's textrecipes does something very similar for text and it works very well.

If you are interested in starting a side package to do this, let us know. Otherwise, this might be lower down the line in priority unless @dfalbel or @skeydan are interested. They have a better handle on processing tools for those data.

[EmilHvitfeldt]

sounds like an interesting idea. I haven't done a lot of work with images myself. the main issue you might run into is that recipes only allows output to be 2-dimensional and I don't know if that will be troublesome.

If you need to talk to someone who has spent some time handling non-rectangular data inside recipe then I'm happy to talk.

[dfalbel]

In theory it would be possible to use {tfhub} for that.

We have implemented steps for text embedding here: https://github.com/rstudio/tfhub/blob/master/R/recipe.R and we could have similar functionality for images.

I imagine that images would need to be stored in an list-column where each element is an array.

[juliasilge]

I'm moving this to our planning repo since this is more about long term ideas (and probably a separate package) than a feature request here in recipes. 👍