Fellow Data Hoarders, a unique situation with trying to replicate data and maintain two hoards
I am going to be living overseas for a couple of years and maintain a data hoard currently. I am going to be leaving my network storage connected at a relative's house to remain accessbile while I am remote but I am also going to take a mirror of this data with me on an external drive. I would like to keep these data sets in sync as much as possible though I would like to avoid constantly be running rsync over the internet to do so.
My hope is that when I make yearly visits back home I can re-merge this data in a way which doesn't require me to transport the external with me each time I go home. I remember reading about some file transfer protocols that allow for this kind of functionality to low bandwidth locations and was wondering if rsync supported such a mode.
The desire is the following:
SiteA has data set A SiteB has data set B
Both data sets are the same but have both been modified slightly.
usb drive C runs an analysis at Site A over ssh of data set A and B and records the difference. This difference then gathers the files at SiteA which need to be updated at SiteB as well as a list of instructions to replace files and then gather files at SiteB which will then be updated when returning to SiteA
This will allow be to run the script before I leave to return home, take the files I need with me, then gather files at the remote site and update others, then run the script when I am back at the remote site to replace files.
Does something like this exist in rsync, I read through the man pages and their batch mode seems similar but i wasn't sure how to integrate that with an intermediary. Like a temp location (which i see rsync also has) but still compare the two sites?
Should I look at a different piece of software? I am effectively trying to use rysnc but with a 'sneakernet' style protocol to sync the changes.
Posted Apr 26, 2017, 5:38 PM UTC