Rsync half a directory to one folder, other half to another folder, based on name
 

Hi everybody. I have another post that this one is in relationship too, but thought it was different enough and off-topic of the original post enough that I'd make a new topic for it. Plus its a subject where I really struggle being new.

I have a huge directory that I need to move to another location with rsync. But I don't want to sync it to one folder at the destination, i want to split it into two folders. I guess ideally I'd want symbols,numbers, A through L in one folder on the destination, then M-Z in the other. since that's how i think it naturally sorts them. I just dont know how to do it to split the source into two destinations. For example, I know how to recursively rsync from say /data1/documents to /data2/documents, but don't know how to sync the first half of the folders when listed alphabetically from /data1/documents to /data2/A-L_documents then the second half to /data2/M-Z_documents. Under my /data1/documents is where all the folder names i'd want to copy are and separate based on those names, like /data1/documents/Al, /data1/documents/Zed, etc

I'm not even sure how to do a find to show me just those folders in question.

Hello,

You could of course write a script to sort the files and output them to a list, next using that list as input for rsync. But in my opinion it's a lot easier to use a tool like Unison, which synchronizes directories and files in a very easy way. It comes with a lot of possibilities including filters. I think you'd be better of with that utility.

Kind regards,

Eric

I'm only able to use the command line though. Will that work through there as well?

Hi,

Of course it runs from command line. I don't even know if it has a GUI interface, since I only run it on servers (no GUI at all). In my opinion it's worth looking into since the configuration is a lot easier then rsync.

Kind regards,

Eric

Thanks, I will definitely check it out, i never even heard of it before.

I'll leave the post open for a bit until i can try it, and that will also let people comment on rsync as well for future use by others perhaps.

Great, I hope it can help you out. If you encounter problems installing or configuring it you can of course post them here on LQ.

Kind regards,

Eric

if it goes like everything else i've tried in Linux, I will definitely have questions! :-)

Hey, who doesn't? There's nobody that has all the answers but a community like LinuxQuestions comes pretty close to offering an answer to almost any question you can think of. I know I've gotten a lot of answers from this site.

Kind regards,

Eric

Thanks Eric. I know I would have given up by now if it weren't for this site. Google doesn't even help me at this point, because I don't even know the proper terms to search for.

I dont know if I'm going to be able to install Unison on my server.

Does anyone know how I could do this via rsync?

If you look at the man page http://linux.die.net/man/1/rsync you'll see you can use the usual Unix filename wildcards eg *.c etc.

Hi Chris. I saw that it will accept wildcards, but this is where my limited knowledge fails me. I still struggle even using the find command. How do I tell it to only grab any folder who's name starts with a symbol, number, or letters A-L, to only rsync those folders to a destination. then i imagine i would run a second rsync to grab only the folders who's names start with M-Z.

Under my /data1/documents, there are a ton of folders, but i only want to grab half at a time, and breaking them up in alphabetical order seemed most logical to me, but technically i'm clueless on how to accomplish this.

There's loads of examples on that page, but something like

rsync -avz /src/bar/A* host:/data/tmp

choice of flags is up to you.

since i'm using fat32, i think I need to do the following line, then repeat for each letter I want, or is there a more streamline way to do it?

rsync -t --modify-window=1 /src/bar/A* host:/data/tmp

Also, if i do it without a -r, does that mean its only going to copy the folders beneath bar and not their contents, or do i need to add in -r to get all the pictures in all those folders?

Why would you say that? Unison is included in several repositories such as the Debian repos. So installation should be as simple as running (for Debian)
What distro are you using?

Kind regards,

Eric