vuetify-style-transfer

Live URL: https://style-transfer.netlify.com (if the Netlify build is successful)

README

This project is based on: https://github.com/reiinakano/arbitrary-image-stylization-tfjs

I refactored the Javascript and re-implemented the client-side using Vue/Vuetify. There are some upgrades to the latest Tensorflow.js and a number of fixes and improvements for mobile web.

A Vue application can be deployed using Netlify very quickly

Vuetify is one of the material design implementations for Vue

Motivation

Finally I decided to explore machine learning starting with the famous Andrew Ng's coursera class. Then I proceeced to deep learning (neural networks) and computer vision.

• MIT's Intro to Deep Learning
• Kadenze's Creative apps with Tensorflow
• CS231, Stanford's cs231, CNN for visual recognition creative-applications-of-deep-learning-with-tensorflow/info)
• Udemy's PyTorch and Computer Vision

I found out about Style Transfer through Tensorflow examples. But training a model with a specific style would take about 4 hours on an nvidia GTX 1080.

Later, I found out about a much faster way to do Style Transfer without the GPU intensive training per single style through Reiiniko's work. Given my expertise on Javascript, I decided to refactor it to understand how Tensorflow.js worked. In the process I learned some limitations with Tensorflow.js with mobile web.

Architecture

Components

The App contains a Tabs component with two StylizePanelLayouts. Each StylizePanelLayout contains a CameraModal, a StylizeControl and three instances of ImageInputs (two for the styles and one for the image content). The StylizeControl contains the Stylize button and the Tensorflow model selectors.

App -> Tabs -> StylizePanelLayout -> ImageInput
                                  -> ImageInput
                                  -> ImageInput
                                  -> Cameramodal
                                  -> StylizeControl

            -> StylizePanelLayout -> ImageInput
                                  -> ImageInput
                                  -> ImageInput
                                  -> Cameramodal
                                  -> StylizeControl

Emitted Events

ImageInput -> imageSizeChanged -> StylizePanelLayout
           -> imageSelected    -> StylizePanelLayout

StylizedControl -> styleAction -> StylizePanelLayout
                -> modelLoaded -> StylizePanelLayout

Utilities in /lib

The lib folder contains three utility classes

• CameraCapture: handles camera capture for desktop and laptops, for mobile we use <input capture="camera">
• ImageUtils:
  • createDownloadLink(dataUrl, filename): creates the link to save an image
  • loadImageFromFileInput(fileInput, image, sizeOptions): reads a file into an image
• StyleTransfer: loads tensorflow models and executes style transfer using Tensorflow.js

Project setup

npm install

Compiles and hot-reloads for development

npm run serve

Compiles and minifies for production

npm run build

Lints and fixes files

npm run lint

Issues

• Safari browser on laptops will reload the web page due to high memory consumption (these models are several MBs)