Techniques/algorithms for syncing between multiple client and server

So I've been using pouchdb as a kind of client/server sync, but I want to start transitioning away from it for various reasons.

I want the (single) server to be the master and the clients to be slaves. So multi-master architecture is not needed - which AFAIK makes a whole lot of things easier.

Is there some general techniques/algorithms I should be looking at? Maybe a well known algorithm in the distributed systems field? I mean I can think of a bunch of edge cases myself but I want to do this properly.

Posted Mar 10, 2019, 7:39 PM UTC

16
seanwilsonfull-stack (www.checkbot.io)•

What are you syncing and why? What's the minimal amount of syncing you really need? Have you considered Firebase or services that claim to be similar? Or even Dropbox?

From working on and near projects that sync data with a custom solution, syncing algorithms (or distributed algorithms in general) can be very complicated to get right and you should avoid coding this yourself whenever possible. There's endless edge cases and when things go wrong (i.e. data loss) it takes a lot of effort to figure out where the problem is and to know for sure you've fixed it.

2
Sky_Linx•

I agree that syncing algorithms can be really difficult if you do two way syncing. Much easier to have clients commit local changes to the server immediately, and have the sync just pull from the server, with the server's version of the data having priority.

1

Behold... rsync!

0
Sky_Linx•

I don't see how rsync is relevant in this case..

1

client/server sync

I didn't read that. Are client's just trying to be in sync with the server and their own data-set or are there multiple clients? Consensus is a pretty hard CS problem if I understand. Dealing with multiple clients and servers seems like it would be limited by weakest link (client with the slowest connection) so that doesn't really seem like something that's terribly scalable (to me). You're basically trying to make an ACID compliant database across servers and clients with bad connections and data in the browser. What's the use case?

1
WikiTextBot•

Two Generals' Problem

In computing, the Two Generals Problem is a thought experiment meant to illustrate the pitfalls and design challenges of attempting to coordinate an action by communicating over an unreliable link. In the experiment, two generals are only able to communicate with one another by sending a messenger through enemy territory. The experiment asks how they might reach an agreement on the time to launch an attack, while knowing that any messenger they send could be captured.

It is related to the more general Byzantine Generals Problem and appears often in introductory classes about computer networking (particularly with regard to the Transmission Control Protocol, where it shows that TCP can't guarantee state consistency between endpoints and why), though it applies to any type of two-party communication where failures of communication are possible.


^[ ^PM ^| ^Exclude ^me ^| ^Exclude ^from ^subreddit ^| ^FAQ ^/ ^Information ^| ^Source ^] ^Downvote ^to ^remove ^| ^v0.28

1