GUI Support

[Earlz]

So, I've been trying for the past few days to get any form of GUI working in my VST host (Reaper) using rust-vst2, and I've faced nothing but difficulties. I'm also pretty new to Rust itself, so my inexperience doesn't help.

Anyway, have you had any success prototyping such a thing, and ideas on what to use? I know the readme says Conrod+SDL, and this is the primary thing I've focused on, but I can't help but wonder if there might be an easier and more stable way. The C++ VST kit that everyone, wdl-ol, includes an OpenGL framework with built in easy support for drawing knobs and such. I'd be curious if doing FFI to that might be easier. Do you have any ideas?

[overdrivenpotato]

So if you look at src/editor.rs, it is up to the plugin to create an editor that implements the Editor trait. One of the things that we have to work with is a window pointer, which can be used by the plugin to do something e.g. initialize an SDL2 context. It seems libraries like wdl-ol abstract over more than just the bare VST api, I'm thinking it might be out of scope for this library to do something like that.

There are a few approaches that we can take though, the main thing I'm interested in is seeing if we can get Piston's 2D graphics library to work with an editor. I don't necessarily want to use OpenGL for drawing as it might not work on all systems and we can use a simpler interface for drawing 2D UI elements. If it's easier though I'm good to get just something running with a UI even if it is OpenGL.

We would also need to have the appropriate button controls like knobs and sliders, I'm thinking these can be done in another crate to separate concerns (or we can use conrod!). It also opens up the possibility of code reuse for something such as an Audio Unit rust implementation which needs UI elements as well.

There's a lot of stuff that needs to be done and I'm OK with trying out a bunch of things, just not sure which direction to head in. If you want to take a stab at it, the very first thing would be to see if you can get a graphics context to work with the pointer given in Editor::open. I'll see if I can help out if you do end up making a public fork/repo.

[Earlz]

Yea, I definitely think it'd fall under a completely different crate, but I feel like it could belong to the same project, or at least be related..

I did some investigating in the wdl-ol code, and I think the way they handle it (very thick codebase, so difficult to be sure) is that they basically have a platform dependent native set of graphics code for each platform. This graphics code handles basically the bare minimum however. It exposes a framebuffer, and allows for the normal event setup (ie, keys, mouse, etc). Then, most programs use their LICE abstraction that works directly on the exposed framebuffer for shapes, glyphs, etc. And finally, the programs that care about OpenGL, basically setup OpenGL to render to an invisible hidden window, and manually blit/copy the framebuffer of the OpenGL window to the VST-host window.

I'm not sure that this is the best way to go, but I'm sure there's a reason why they chose this way rather than directly rendering everything using the window handle.

I've been trying to get Editor::open to work properly with SDL, but having very little success thus far.. I intend to keep chasing this though. It'd be awesome if VST work on Rust was actually preferred over C++, since already this is incredibly easy compared to making a VST plugin in C++.. just the GUI bit isn't there, and eventually the DSP other common audio operation helpers

[overdrivenpotato]

Sorry about the delay, holidays and all 🎉 (Happy new year!)

Alright well it looks like you're right about the wdl-ol implementation, we can definitely do the same thing and implement platform specific graphics code as I am not aware of a rust library that does this. One of my goals would be to have rust-vst2 work with Linux, OS X and Windows though so I would like to support all three platforms properly. I guess the first step is to decide whether we're going down the route of using graphics as a base or whether we should just create a new library from scratch, your thoughts?

[adamski]

You might want to look at how the JUCE C++ framework deals with different platforms. A rust equivalent to JUCE would be amazing, albeit a very ambitious project!

[suhr]

graphics provides primitives to draw, but you still need a backend for it to initialize an sdl context.

I have a different problem though. I want to implement a microtonal (31 tone) sequencer (like Hex). It will have a complex UI, so I rather should use gtk-rs. But I also want it to be a plugin, to use it in my DAW of choice.

So, how do I use gtk in vst? I heard this is not an easy thing to do, but I also know Carla uses Qt and can be loaded as a plugin.

[sklopi]

Did anyone have luck in initializing a graphics backend from the hwnd handle on windows?
I'm currently trying to get this working, but i'm kinda stuck.

[overdrivenpotato]

@suhr I'm not really sure how this should be handled, I do think the best way going forward would be to use some sort of a toolkit but as I don't have experience with either QT or GTK I don't really have any suggestions.

It looks like it boils down to either getting an existing toolkit to use the raw window handle for each platform, or creating a GUI system specific to this library. Personally I'd be interesting in seeing something such as electron being used for the GUI and delegating the implementation/DSP down to rust code, but again I'm not sure where I'd even begin.

[ycros]

I have proof-of-concept working currently on Windows at https://github.com/ycros/vparty using a fork of glutin ycros/glutin@dd0de44 - eventually I'd like to get conrod up and running, but this requires either implementing a whole new conrod backend or getting some changes into piston. I know how to get things running on OSX as well, though things are a little trickier there (your pointer may be a Carbon windowRef or an NSWindow depending on 32bit/64bit/your daw). I've been busy with a new job so I haven't had time to come back to working on this.

Basically, for whatever sort of gui lib you want to end up using, you will need to handle spawning your window as a child window under whatever window handle/pointer/thing you get from your host. There's other open source C++ VST wrappers such as wdl-ol that you can peek at to see how they handle gui initialisation.

[Boscop]

I forked glutin, conrod and the piston window stuff and modified it to support multiple child windows in the same thread (required for VST but by default glutin creates a new thread for each window and doesn't allow child windows) and passing the parent window handle all the way through to where the win32 window creation functions are called, you can find the forks on my page, but I ran into some issues like this:
PistonDevelopers/piston_window#167
(The problem is piston_window uses its own event loop. I either don't get draw events or no input events.)
Right now I'm working on my own lightweight GUI lib on top of glutin, I already have layouting, buttons, knobs, font rendering etc (with batched rendering). but then I got sidetracked working on a game.. I'm getting back to this though.
My plan is to create a VST framework on top of this lib that includes GUI support and common DSP routines that are useful and can be composed to create larger plugins, kind of like JUCE but for Rust.
I currently have a basic starting point, ported some code from Will Pirkle's book to Rust.
If you are interested, we can plan this together on IRC, #rust-music
I think Rust has great potential for audio DSP.

Btw, this is an old screenshot of my GUI lib (this example isn't running in a VST but it works in a VST too):

[Boscop]

Another old screenshot, this VST is not using my GUI lib, just my glutin fork.

[suhr]

My plan is to create a VST framework on top of this lib that includes GUI support and common DSP routines that are useful and can be composed to create larger plugins, kind of like JUCE but for Rust.

Then it should also support LV2 and AU.

[Boscop]

@suhr Yeah, but I don't know anything about those formats... So it would be helpful if we can work together.
Probably we should design an intermediate API for plugin architectures so that each plugin only has to interface that API and can be compiled to all supported backends. I think that's how WDL-OL does it.
Currently, my glutin fork only allows creating child windows on windows. But I abstracted the WindowID into a type, which can be defined in a platform-independent way, so feel free to submit a PR for other platforms. Also I added set_timer() and kill_timer() so that the VST GUI can be drawn at a consistent framerate triggered by the timer, that's how it should be done on other platforms, too. This requires storing a pointer to the plugin in the userdata of the window, this is how I do it on windows.
My glutin fork also supports sending a Timer event instead of using a callback for the timer, but you can't poll events out of nothing, something has to trigger it, and if the mouse is not over your VST window, the host will call idle on the plugin only at like 4 fps. That's why I switched to the timer callback, because my spectrum view needed a higher framerate.

[overdrivenpotato]

@Boscop This looks great, I definitely would be interested in discussing this further. Which IRC network is #rust-music on?