Editor: images blink when upload is completed

[kean]

When you add images to the post, and the upload completes, the images blink (shows blank canvas until they are downloaded):

Simulator.Screen.Recording.-.iPad.Air.5th.generation.-.2023-08-17.at.11.43.27.mp4

[kean]

I also wonder why Gutenberg requires the media to have a url right after selection: https://github.com/WordPress/gutenberg/blob/c14792d535c372a045ff4fc262fcaca43c0326a6/packages/block-library/src/image/edit.js#L150. The app actually doesn't have any URLs the picker passes the app the user selection, so it complicates things significantly.

[twstokes]

Thanks for surfacing @kean! I've added it to one of our initiative issues related to media upload improvements and will un-assign myself from this for now.

I also wonder why Gutenberg requires the media to have a url right after selection

I'm actually not entirely sure offhand. A guess would be that the picker passes the editor the image, the image gets uploaded to the backend, and then it's selectable (and configurable) once we know it exists on the site.

[kean]

After removing WPMediaPicker, this is now the last place that uses MediaThumbnailService. So, I was thinking, shouldn't we remove this thumbnail generation? It makes sense not to show the thumbnail until it's uploaded. I'll also reduce the pressure on the resources quite a lot when you upload multiple larger images. And it fixes the animation. @twstokes , thoughts?

Simulator.Screen.Recording.-.iPhone.15.-.2023-11-27.at.16.37.42.mp4

[twstokes]

@twstokes , thoughts?

I think this is less distracting than the blinkenlights effect we have today, but what happens when an upload fails? Does the user know which image they are retrying?

[kean]

It's inefficient in one more way: the high-resolution thumbnails are generated and saved twice during the upload. I've noticed it during my work on 23.2, but I think it's now the right time to address it.

1. MediaImportService creates creates a high-resolution thumbnail and stores it on disk as thumbnailLocalURL. In my case, I see it being resized to a maximum size of 2556x2556 px.
2. MediaImportService also prepares an image for export, resizing it using the new method introduced in Add Optimize Images setting and enable it by default #21981, and stores as localURL. When I tested it, the maximum size was set to 2000x2000px.

As a result, the app ends up storing two high-resolution images on disk instead of just one. The resizing also puts extra pressure on the system, especially in terms of RAM usage.

The high-resolution preview created during step 1 is used mainly in the Editor. Most of the other parts of the app use localURL to show the original assets.

cc @fluiddot.

[kean]

but what happens when an upload fails? Does the user know which image they are retrying?

Hmm, good point. I think there is a better way of addressing the performance issue (but not the blinking) by exposing the URLs for thumbnails managed by MediaImageService. The .medium size should OK for this purpose. I'll investigate further.

[fluiddot]

For reference, we have a similar GitHub issue in Gutenberg related to this: wordpress-mobile/gutenberg-mobile#6510

[derekblank]

After some initial investigation from the editor side, I believe the blinking issue occurs within the Javascript editor code when the local media file ("thumbnail") is being replaced by the uploaded image from the server. The "blink" timespan may be the time spent downloading the new image from the server. On slower connections, it's more of a small nap than a blink.

The high-resolution preview created during step 1 is used mainly in the Editor. Most of the other parts of the app use localURL to show the original assets.

As @fluiddot mentioned, we can work on improving the blink as part of wordpress-mobile/gutenberg-mobile#6510 by persisting the local media file until the server image is downloaded to create a seamless transition between the two states. If that were the case, we could probably bypass the image generated from MediaThumbnailService as well.

While the performance improvements from the serving the image files are always welcome, the source of the blink itself probably doesn't require further investigation from the WPiOS side at the moment.

[fluiddot]

After some initial investigation from the editor side, I believe the blinking issue occurs within the Javascript editor code when the local media file ("thumbnail") is being replaced by the uploaded image from the server. The "blink" timespan may be the time spent downloading the new image from the server. On slower connections, it's more of a small nap than a blink.

Thanks @derekblank for investigating the issue 🙇 ! We could probably reproduce and confirm that the issue is related to waiting for the remote image download by throttling the network connection. Similarly, we could enforce a failure in the image download to check if the Image block results in a blank image.

[...] by persisting the local media file until the server image is downloaded to create a seamless transition between the two states. If that were the case, we could probably bypass the image generated from MediaThumbnailService as well.

Yep, we should only replace the local image with the remote one when it's fully downloaded. This will most likely address the blink issue.