VtkRemoteView layout distortion

[AndiSchratti]

Hi,
I’m using trame with multiple VtkRemoteViews. I toggle between different layouts by changing their width via state-bound CSS styles.

The problem is that all views appear distorted for a brief moment (about half a second) when their width changes. View2 is only distorted the first time the layout is toggled, while afterwards only view1 shows this temporary distortion. The distortion happens immediately after the style update, even though the render windows themselves are not recreated.

I’ve also tried using different events for InvokeEvent(), because I noticed that after clicking in the view it automatically rerenders. I also checked camera resets and render calls, but the temporary distortion still occurs.

I’m looking for a proper way to update the views so they maintain correct aspect ratio when the layout changes.

The code I’m using to test this currently looks as follows:

from trame.app import get_server
from trame.ui.vuetify import SinglePageWithDrawerLayout
from trame.widgets import vtk, vuetify
import vtkmodules.vtkRenderingOpenGL2  # noqa
from vtkmodules.vtkInteractionStyle import vtkInteractorStyleSwitch, vtkInteractorStyleTrackballCamera, vtkInteractorStyleImage  # noqa
from vtkmodules.vtkFiltersSources import vtkSphereSource, vtkConeSource
from vtkmodules.vtkRenderingCore import vtkRenderer, vtkPolyDataMapper, vtkActor, vtkRenderWindow, \
    vtkRenderWindowInteractor

server = get_server(client_type="vue2")
ctrl = server.controller
state = server.state

sources = [
    vtkSphereSource(),
    vtkConeSource()
]
for src in sources:
    src.Update()

render_windows = []
render_window_interactors = []
views = []
state.setdefault("viewer2d_enabled", False)
state.setdefault("view_style", 'width: 100%; margin: 0.2%;')
state.setdefault("view_style_2d", 'width: 0%; margin: 0.2%;')

for i, src in enumerate(sources):
    ren = vtkRenderer()

    mapper = vtkPolyDataMapper()
    mapper.SetInputConnection(src.GetOutputPort())
    actor = vtkActor()
    actor.SetMapper(mapper)

    ren.AddActor(actor)

    ren_win = vtkRenderWindow()
    ren_win.AddRenderer(ren)
    ren_win.SetOffScreenRendering(1)

    ren_win_int = vtkRenderWindowInteractor()
    ren_win_int.SetRenderWindow(ren_win)
    ren_win_int.GetInteractorStyle().SetCurrentStyleToTrackballCamera()

    render_windows.append(ren_win)
    render_window_interactors.append(ren_win_int)

def trame_app():
    with SinglePageWithDrawerLayout(server) as layout:
        with layout.drawer:
            vuetify.VBtn(
                "Switch window",
                click=test,
            )
        with layout.content:
            with vuetify.VContainer(fluid=True, classes="pa-0 fill-height"):
                view1 = vtk.VtkRemoteView(render_windows[0], style=("view_style",))
                view2 = vtk.VtkRemoteView(render_windows[1], style=("view_style_2d",))
                views.append(view1)
                views.append(view2)

def test():
    global state
    state.viewer2d_enabled = not state.viewer2d_enabled

    if state.viewer2d_enabled:
        state.view_style = "width: 49%; margin: 0.2%;"
        state.view_style_2d = "width: 49%; margin: 0.2%;"
    else:
        state.view_style = "width: 100%; margin: 0.2%;"
        state.view_style_2d = "width: 0%; margin: 0.2%;"

if __name__ == "__main__":
    trame_app()
    server.start()

Thanks in advance for your help!

[bourdaisj]

Hi @AndiSchratti ,
Thanks for bringing this topic up. This is indeed a problem due to the way remote view works.
When you resize the container, the browser components sends the new size to the server which generate an updated image. In the meantime, you get that distorted effect on the render view.
You can take several approach to improve this:

1. You may try the same using trame-rca to see if it handles this better. i doubt it, but still worth the try.
2. We would have to improve the way the image aspect ratio is handle in the browser. We currently use a html <img /> tag. That would require a contribution either to VTK.js remote view or vue-vtk-js / trame-vtk.
   I've tried locally and those CSS properties seems to help quite a lot with the problem you are describing:

• aspect-ratio
• object-fit
  You may try it yourself using the browser HTML inspector. Just locate the img tag that is used by the remote view.
  You would also have to use a transparent VTK/paraView background and set the background color using an HTML/CSS background, which would be more robust.

Curious also about what @finetjul and @jourdain think about this

[finetjul]

object-fit could indeed be exposed to trame-vtk. object-fit: contain could be a good default for VTK IMO.

[AndiSchratti]

Thank you for the quick response!
Unfortunately, I was not able to achieve the desired result with trame-rca either.
I have already experimented a bit with object-fit and aspect-ratio. The best solution at the moment would be a combination of object-fit: contain; and object-fit: cover;
Is there currently a way to achieve this? I’m not sure how to properly apply these CSS properties to the image element created by VtkRemoteView, as it is wrapped in multiple nested HTML elements.

[bourdaisj]

I think it might be interesting to contribute it in VTK.js directly, from my understanding in the CanvasView, which manage the img tag.
Or you do some CSS yourself to add this capability to your application. One solution would be to give a specific id to a top level element that you have control over in your application. Then use CSS child selectors to specifically target the img tag.