Long story, feel free to skip to the red for the tl;dr:-

I was a friend's house today, I asked them if I could retrieve a video they had recorded that day of an event we went to, and, he handed me his camera. I could of just copied it to my laptop and been done with it, but, as a run an extremely small SSD, I couldn't actually copy all the data to my SSD without filling it up and requiring me to delete my data.

Solution? Well, just synchronize it to my computer back at home (I was on his WiFi), and, my offsite remove redundancy server using rsync, now I have two copies right from the horse's mouse (In this case, the camcorder that was plugged in and mounted).

Issue? Uploading twice. That's a huge pain, especially when you're limited on time, and, I couldn't really take his camera with me. I could just upload to one server, but, since I'm there anyway, I might as well redundify (That's not a word) the data right now.

Solution #2? Bittorrent! I can upload half the torrent to one server, half the torrent to the other using the super-seed functionality of pretty much every bittorrent client ever (that only shares one copy of each block, equally distributed, then expects the swarm to share it between themselves). This is what I did in the end, unfortunately, it took me about ten minutes to get up and running (Configure RTorrent to read from mount point, google public trackers, make torrent, rsync the .torrent between servers, load torrent on remote RTorrents, force tracker updates on all boxes, then finally, synchronization!)

I'm wondering, if there any software I can install on my servers & laptop where if this ever occurs, it's a simple command? If over SSH, that'd be even better.

Like:-

Then just enter the private keys/passwords for 'ssh_aliases', 'from_the' and 'ssh_config', automagically upload 1/3rd of the file to each one, then magically synchronize between the three once I've already disconnected and be done with it. I know I could make a bash script that does this with RTorrent, but, it seems a really inefficient way to do it, and, I have absolutely zero idea how to properly develop a full-fledged secure bittorrent client that correctly implements everything. I also don't want to use BTSync due to the fact I don't want the files to stay updated, just that one-off initial synchronization to offload bandwidth from the original seeder (Also not even sure if BTSync super seeds!).

Thanks.

tl;dr is there software that I can use that I can point at a file/directory, and, a few ips & credentials, then, have it use the super-seed system that bittorrent uses to seed quickly get at-least one copy into the swarm (of computers I gave credentials too), then let me DC (The original peer) while the other servers all continue to seed and then terminate once all of the servers are synchronized (But not before)?

I know it's an extremely specific request, but, rsync is annoying with it's one-on-one transfers, and, bittorrent's setup time (I.E. for each new file, not installing it) is insanely long also. I want something snappy, and, preferably something that also has end-to-end encryption on the network (I.E. over SSH, or, built in encryption).

See twitter murder blog post.

Murder source code.

Murder fork implemented in python called Herd.

1 members found this post helpful.