using with a single websocket connection for the whole app

[chee]

hey, nik! fabulous library, i really need it.

this issue is more of a chat than an issue.

i'm wondering if there's any ideas for a strategy for using this with automerge-repo, where you have a single websocket connection for the current tab (or browser, if you use a worker) rather than the yjs style of one connection per doc.

i started hacking something together, but i tripped up on the requirement for websocketEndpoint on a per-doc basis. i'm imagining a world where i could basically wrap an automerge provider (like herb's local-first/auth) and then every doc fetched and used through that connection would be encoded and deciphered via the keys registered at init

[nikgraf]

hey @chee, I'm thinking exactly about this problem a lot these days

Automerge-repo has the concept of a https://github.com/automerge/automerge-repo/blob/06126099322474bb8c6251b58b1ee5e63c174f06/packages/automerge-repo/src/synchronizer/CollectionSynchronizer.ts
This makes it really nice to use, but also wondering how this can scale beyond a single server without a different implementation. Need to investigate more.

One thing I was thinking of is to create a collection concept, a HTTP API and a sync mechanism that does scheduled checks if a sync is needed. My thinking was that the currently open document could have an active websocket connection and all other documents could sync in the background.
This probably also could work happen as a background task for Mobile apps?

I'm curious if this would help you with your project. What's your use-case?

[chee]

Automerge-repo has the concept of a https://github.com/automerge/automerge-repo/blob/06126099322474bb8c6251b58b1ee5e63c174f06/packages/automerge-repo/src/synchronizer/CollectionSynchronizer.ts

oh, that's cool! i hadn't seen that before. i've recently learned the automerge-repo folks are working on a new protocol that will, quoting from the discord, make it "easier to manage synchronizing larger collections of documents - as well as integrating with E2EE and authentication and authorization". i've decided to pause the e2ee part of my project until i get a clearer picture of what that would look like

One thing I was thinking of is to create a collection concept, a HTTP API and a sync mechanism that does scheduled checks if a sync is needed. My thinking was that the currently open document could have an active websocket connection and all other documents could sync in the background. This probably also could work happen as a background task for Mobile apps?

I'm curious if this would help you with your project. What's your use-case?

this is interesting! my project is quite a large thing with folders and lots of different files in arbitrary formats like a web-based devonthink where everything is stored as rich automerge docs (for history and collab) with a solidjs ui. it's all gotten a bit out of hand.

i think i could get it working with a background checking mechanism. right now i have these docs that are "files" that are loaded eagerly (all of them in the whole app, all watched), and they contain a reference to a content doc that is only loaded when the thing is actually open, so it would be a slightly different architecture (those content docs might be massive binary files so i'd never want them being synced in the background) but i think it could still be made to map well

[nikgraf]

yeah, I'm talking to the automerge-repo people and I'm contributing and reviewing ideas. Looks promising, but still early :)

Thanks for the insights! Might be a good test for future versions :)

[chee]

yeah, I'm talking to the automerge-repo people and I'm contributing and reviewing ideas. Looks promising, but still early :)

Thanks for the insights! Might be a good test for future versions :)

extremely exciting! i'll work on the other 14,000 items on my todo list in the meantime ❤️