Hi,
I have a cPanel/WHM web server, which hosts a whole bunch of sites.
I then have another dedicated server which is for backups. Every 3 days, the backup server connects to the web server and grabs a copy of all the home dirs.
Here's the basics of the bash script using rsync (I've simplified it here)...
Code:
# Todays date
TODAY=`date -I`
# Previous date
PASTDATE=`date -I -d "3 day ago"`
# RSYNC the new files down from remote
sudo rsync -ach --progress --delete \
--exclude /somedir \
--link-dest=/home/serverbackups/WEBSERVER/files/$PASTDATE -e "ssh -i $SSHKEY" --progress \
root@WEBSERVER:/home \
/home/serverbackups/WEBSERVER/files/$TODAY
This works... all the date directories are there, each with all the files inside.
On the web server, the combined disk usage for /home is about 450GB.
On the backup server, disk usage is about 600GB - I figure this is about right, showing that this rsync process isn't re-creating files it already has, but is creating symlinks (which is what I want to save disk space). If this wasn't the case, the disk usage would go up by ~450GB every time it runs... which it doesn't.
So I'm switching backup servers, to something with more space (4x 2TB HDDs).
I wrote a bash script using RSYNC which would copy everything from old backup server to the new backup server, and then I ran it.
Three days later, it's still running, with a disk usage of 1.4TB.
That can't be right... disk usage should be the same as it is on the old server.
I assume it's copied the actual files and has followed the symlinks, copying them over again and again, so I cancel this, make some changes to my transfer script and start over.
Before running this script again, I thought I'd double check the source (the old backup server) to make sure all the symlinks are in place.
I "cd" into about 15 different, random directories from the old backup location, and run "ls -l".
From what I can see, there aren't any symlinks... here's an example...
Code:
drwxr-xr-x 13 558 558 4096 Sep 18 18:53 application
drwxr-xr-x 5 558 558 4096 Sep 18 18:53 assets.admin
drwxr-xr-x 6 558 558 4096 Sep 18 18:53 assets.frontend
drwxr-xr-x 2 558 558 4096 Sep 2 19:08 cgi-bin
-rw-r--r-- 14 558 558 8962 Sep 17 15:31 index.php
drwxr-xr-x 8 558 558 4096 Sep 18 18:53 system
-rw-r--r-- 14 558 558 53330 Sep 17 15:31 thumb.php
drwxr-xr-x 3 558 558 4096 Nov 4 14:55 uploads
That's the contents of a websites "public_html" directory. The files are small in this instance, however there are some which are 200MB+ each.
So... a couple of questions...
- What the heck is going on at the old backup server? I can't see any symlinks, and yet disk usage isn't going up by 450GB each time.
- When I transfer everything to the new backup server, what's the best way to do this whilst preserving the symlinks?
Any help would be greatly appreciated.
Thanks!
