Questions and ideas regarding how well does or will MWS work with Syncthing or when run on different devices

[alurm]

Please tell me if this should be posted on the TiddlyWiki forum as well (or instead).

I would like to use MWS but I'm hesistant since I'm not sure how well it would work with Syncthing or in an offline context (where a server is run on each device). Here are my thoughts on this manner.

There are many benefits to Syncthing. I like Syncthing a lot because it works as a narrow waist providing syncing and, effectively, caching, so I doesn't have to worry about my data being exposed by too many vulnerable servers. Using Syncthing also means that I don't need to rely on a public IP.

Here's my setup. I currently run a Node.js version of TiddlyWiki on my devices and what I find it important that TiddlyWiki works on all of my devices even if I'm offline. The only issues are that I have to restart the Node.js process sometimes to get synced changes and that I have to solve conflicts sometimes. But this is managable.

Given that MWS now uses an SQLite database (in my understanding), I wonder how well it (1) would behave under Syncthing and if (2) it will work in a scenario where each device runs an MWS server?

I imagine that it would still work but sync conflicts would be more difficult to resolve and more frequent, since the SQLite database is one file (and not a folder). In regular Node.js version if different devices change or create different tiddlers it does not create sync conflicts.

My understanding is that Syncthing doesn't understand contents of files. My desire would be so that read-only usage of MWS wouldn't create Syncthing conflicts. I'm not sure if that's how it currently works. Note that Syncthing has a "syncignore" and I use that to disable syncing of $__StoryList.tid so conflicts aren't created.

I guess another potential idea is that there could be a tool to manually "merge" two SQLite dbs, since Syncthing preserves both copies of the file.

[Arlen22]

Basically no. While the file is opened it should be locked, and when it closes SyncThing is going to have to resolve any conflicts that happen. If you're saying it keeps every file if there is a conflict, then that will work great, except that you have to make sure that you properly run MWS with each file, and if there's a crash you have to make sure you keep track of the second file that holds the rollback or transaction log. You would technically still be able to recover, but it would require loading each copy separately and copying out the data you want. It's technically easy if you know what you're doing, but it'd be far too complicated to call it syncing, and you have to be really careful if there was a crash.

I don't have a good solution on the MWS side. I once did a proof of concept where a wiki running in electron loaded tiddlers directly from the file system on page load (essentially running the same $tw instance as both browser and node). I was pretty impressed with the novelty but I've never seen anyone use this in the wild.

I would recommend asking more on the tiddlywiki forums. I can't see SyncThing working well for this scenario at all unless it can somehow provide all your devices with a connection to the server. It'd honestly be less headache (and it's still a lot of headache) to setup a VPN on all your devices that can connect directly to the device running the MWS server. As long as you only have once instance running you should be able to use SyncThing as a backup just fine.

File syncing the database, and trying to get it to merge properly, in my personal opinion, isn't even the wild west. It's practically no man's land.

I know your not the only person running multiple devices (some of them even including mobile and offline situations. Definitely ask on the TiddlyWiki forums.