Overlap with 'Group Services'

[hyperthunk]

As per haskell-distributed/distributed-process-platform#47 - might be of interest here, though I suspect it's something to built over global rather than directly into it, not least because we might want to use an alternative 'synchronisation backend' in some circumstances (e.g., very large clusters, HPC, etc).

[jepst]

So if I understand correctly, an atomic broadcast layer with pluggable backends, one of which is d-p-global's distributed locking?

[hyperthunk]

So if I understand correctly, an atomic broadcast layer with pluggable backends, one of which is d-p-global's distributed locking?

Pretty much what I was thinking yes. Atomic Broadcast is a useful enabler. Group Services (as a concept) probably encompasses other things too, consensus and presence being the ones mentioned by ZK; Service Discovery, cluster state replication and sharding are interesting ones for me but I'm sure there are many others. With global we can do a lot of this using lock+cluster operations and providing these would help us shake out the kind of APIs we'd like to see.

I suspect the problem with plug-able backends will be that the semantics need to change because different implementations will offer different guarantees. We'll have to figure out where the commonality is and decide whether that's a good idea. The RabbitMQ 'guaranteed multicast' protocol (gm) is fully asynchronous, so the semantics are somewhat different that we'd get if we did a lock based protocol (gm is a ring protocol that uses a kind of lock-less multi-version concurrency control wherein members can leave or join at any time).

Interestingly, we implemented GM because the distributed transaction support offered by Erlang's built-in distributed database (mnesia) is both too slow (for a highly available messaging system like RabbitMQ) and doesn't tolerate net-splits well. GM is totally agnostic about network partitions, having been built to deliberately compensate for membership changes whilst messages are mid-flight. The essay at the top of http://hg.rabbitmq.com/rabbitmq-server/file/default/src/gm.erl is well worth a read if you're interested.