Archive of Web Logs for Urchin.

wademac · 07-24-2009, 11:14 AM

Hello All,

We use Urchin 6.6 for our stats from our 3 web servers Urchin runs on a dedicated SUSE 10.3 box and I have the live logs mounted to this box using NFS so it can read the live logs off the servers local to the Urchin Application.

for example NFS mounts to Urchin Server:
/mnt/live_wwwlogs
/mnt/live_www2logs
/mnt/live_www3logs

now those servers can only hold some much storage of those logs but I need to keep a archive of all web logs so I have setup a archive on the Urchin box which is on a SAN connection .

/Data is a LVM from a SAN

/data/logs/wwwlogs
/data/logs/www2logs
/data/logs/www3logs

each website has a directory

/data/logs/wwwlogs/website1

for example website1-combined_log-20080203.gz, website1-combined_log-20080204.gz etc.. going back over many years...

I would like to move anything older than 15 days to the /data/logs/@server/$website and cron it to run after midnight log roll.

emetib · 07-24-2009, 11:27 AM

how often do you have your logs rotating?

i know on mine it's every week.

it sounds like you should set up an rsync cron job.
take a look at the man for rsync, it's pretty straight forward.

wademac · 07-24-2009, 12:37 PM

Quote:

Originally Posted by emetib

how often do you have your logs rotating?

i know on mine it's every week.

it sounds like you should set up an rsync cron job.
take a look at the man for rsync, it's pretty straight forward.

it is mixed some are done daily some are never rolled and just build as they are small I want to automate the process I am thinking of taking the logs that dont roll and roll them monthly.