[Formulus] issues with sync

[r0ssing]

Component

• formulus (React Native mobile app)
• formulus-formplayer (React web app)
• synkronus (Go backend server)
• synkronus-cli (Command-line utility)
• Documentation
• Other (please specify)

Description

In the latest update sync appears to be broken. It stops at various places, and can't survive the app getting background'ed correctly. I think also the cancel button doesn't work and finally we should consider it a transaction (i.e. we shouldn't overwrote the existing app before we have downloaded everything. After the failed app bundle sync, I got the placeholder app instead of the old version of the custom app

Also; It feels redunant to include the 'calculation' part of the progress percentage (eg. '14/100 - 14%') - let's just write '14%' in that case? (See screenshots).

We could consider optimizing the app bundle update, since it seems slower than it should be? Perhaps we should download the zip and unzip it locally instead of downloading each file individually?

Steps to Reproduce

1. Update Formulus to alpha.16
2. Click Update app_bundle
3. Foreground a different app on Android

[najuna-brian]

Thanks @r0ssing
That's true!
I have observed as well that after updating an app-bundle in which one has made some changes to the custom app, it takes long to show those changes or even one needs to close the app first before they can see the changes in the updated app bundle

[najuna-brian]

Considering that the bundle is already uploaded as a zip (via CLI), extracted on the server, then re-downloaded file-by-file by the client from what we have currently,

Could we?

1. synkronus: Add a GET /app-bundle/download-zip endpoint that streams the active bundle as a single zip (the server already imports archive/zip)
2. Formulus: Download the zip to a temp file, unzip locally with react-native-zip-archive, then swap into place

It would make the update transactional where the old bundle stays intact until the new one is fully downloaded and then extracted.

[najuna-brian]

Also considering that the zip is NOT saved. The server currently:

1. Receives the zip upload
2. Writes it to a temp file
3. Validates and extracts it into app-bundle-versions/0001/ (as individual files)
4. Deletes the temp zip (defer os.Remove(...))
5. When "activated", copies those individual files into the app-bundle/ serving directory

So the server currently stores extracted files, never the original zip.

[najuna-brian]

This makes me think of two approaches to consider:

Option A: Zip on-the-fly at download time

• Walks app-bundle/ directory and streams a zip to the client
• No storage overhead, always up-to-date
• Slight CPU cost per request (though bundles are small, and this endpoint is called infrequently)

Option B: Save the zip during upload and serve it directly

• Modify PushBundle() to keep the zip file alongside the extracted version (e.g., app-bundle-versions/0001/bundle.zip)
• Serve that pre-existing file for downloads, this gives us zero CPU overhead, just file streaming
• Requires also generating a zip when SwitchVersion() is called (since it copies files into app-bundle/)

[najuna-brian]

I think Option B is better for a production system? since it avoids re-zipping on every download. The zip is already there when uploaded; we just need to stop deleting it.

What do you think? Which of the two options can we explore or is there an alternative/adjustment we can consider?

[r0ssing]

Great analysis work!

Yes, I agree that we should go for option B, saving the zip file on the server. (Why do we even extract it - for verification/validation purposes?). Having the zip would also make it easy to compare the hash (md5) of the file with the current version on android if that should be needed (to ensure against malicious attacks we shuld consider signing the bundle with a pgp key - so the app has the servers public key to verify it with; but that's version 2 I guess)