LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 12-03-2011, 10:01 AM   #1
keimdf
LQ Newbie
 
Registered: Dec 2004
Location: Wabasha, MN
Posts: 24

Rep: Reputation: 15
Backup w/tar and 2GB limits


For years now, I have been using RH Linux for development of my customer's code. To insure that I did loose code due to crashes, I have pieced together a cron job to do backups of all of my important code. I have been blessed with doing this as over the years my system has crashed and I have dealt with it by restoring my backup. However, I have always had an issue as my system/customers have grown. I believe this to be a tar file size limit of 2GB.

I am including the cron script for the back up here. I hope this does not get anyone upset.
Quote:
#!/bin/sh
# full and incr backup script
# created 27 Sep 99
# Based on a script by Daniel O'Callaghan <danny@freebsd.org>


#Change the 5 variables below to fit your computer/backup

COMPUTER=ANAKIN # name of this computer
DIRLIST="/u" # directoris to backup
BACKUPDIR=/backups # where to store the backups
TIMEDIR=/full # where to store time of full backup
TAR=/bin/tar # name and locaction of tar

#You should not have to change anything below here

PATH=/usr/local/bin:/usr/bin:/bin
DOW=`date +%a` # Day of the week e.g. Mon
DOM=`date +%d` # Date of the Month e.g. 27
DM=`date +%d%b` # Date and Month e.g. 27Sep
BM=`date +%b` # Month e.g. Oct
YR=`date +%Y` # Year e.g. 2004

#Figure up the number of the day of the week
WD="0"
if [ $DOW = "Sun" ]
then WD="1"
fi
if [ $DOW = "Mon" ]
then WD="2"
fi
if [ $DOW = "Tue" ]
then WD="3"
fi
if [ $DOW = "Wed" ]
then WD="4"
fi
if [ $DOW = "Thu" ]
then WD="5"
fi
if [ $DOW = "Fri" ]
then WD="6"
fi
if [ $DOW = "Sat" ]
then WD="7"
fi

# Mount the Intel SS4000-E Network Drive Raid 5
mount -t smbfs //midgard/CodeBKUP /backups -o rw,username=backup,password="backup"

# 10-Oct-2004
# Backups are now set up by day and stored by month and week
#
# On the 1st of the month a permanet full backup is made
# Every Sunday a full backup is made - overwriting last Sundays backup
# The rest of the time an incremental backup is made. Each incremental
# backup overwrites last weeks incremental backup of the same name.
#
# if NEWER = "", then tar backs up all files in the DIRLIST
# otherwise it backs up files newer than the NEWER date. NEWER
# gets it date from the file written every Sunday.

# Monthly Backups - Overwrites Last Year's Backup By Month
if [ $DOM = "01" ]; then # monthly full backup on 1st of Month
NEWER=""
$TAR $NEWER -z -c -f $BACKUPDIR/$COMPUTER/$BM/$COMPUTER-$DOM.tgz $DIRLIST
else # daily incremental backup for rest of Month
# Now get yesterday's date
NEWER="-N `cat $BACKUPDIR/$TIMEDIR/$COMPUTER-daily-date`"
$TAR $NEWER -z -c -f $BACKUPDIR/$COMPUTER/$BM/$COMPUTER-$DOM.tgz $DIRLIST
fi

# Weekly Backups - Overwrites Last Week's Backup
if [ $DOW = "Sun" ]; then # weekly full backup for Sunday
NEWER=""
NOW=`date +%d-%b`
echo $NOW > $BACKUPDIR/$TIMEDIR/$COMPUTER-full-date #update full backup date
$TAR $NEWER -z -c -f $BACKUPDIR/$COMPUTER/$COMPUTER-$WD-$DOW.tgz $DIRLIST
else # weekly backups for rest of Week
# Get date of last full backup
NEWER="-N `cat $BACKUPDIR/$TIMEDIR/$COMPUTER-full-date`"
# Week Day Backup
$TAR $NEWER -z -c -f $BACKUPDIR/$COMPUTER/$COMPUTER-$WD-$DOW.tgz $DIRLIST
fi

NOW=`date +%d-%b`
echo $NOW > $BACKUPDIR/$TIMEDIR/$COMPUTER-daily-date # update Today's Date


# Unmount the device
umount /backups
For those of you that understand scripting you will see that I am doing several different backups. I do a monthly and a weekly backup. At the beginning of a Month and each Week, I do a full backup. Then for the rest of the month/week, I perform an incremental backup. The incremental files are NOT the problem obviously.

For a couple of years now, I am guessing that the full backups have exceeded the 2GB limit.

What I would like to know is if anyone can help me with this? I need to have full backups that have full integrity. Right now, it I have to recover ALL of the backup, tar errors out towards the end of the file with a bad EOF or something like that.

I wish thanks to anyone who is willing to read and help me with this. I am NOT a Linux person though I have managed to muddle through things.

THANKS!!
Happy Holidays

Last edited by keimdf; 12-07-2011 at 02:32 PM.
 
Old 12-03-2011, 10:21 AM   #2
Nominal Animal
Senior Member
 
Registered: Dec 2010
Location: Finland
Distribution: Xubuntu, CentOS, LFS
Posts: 1,723
Blog Entries: 3

Rep: Reputation: 946Reputation: 946Reputation: 946Reputation: 946Reputation: 946Reputation: 946Reputation: 946Reputation: 946
Quote:
Originally Posted by keimdf View Post
I believe this to be a tar file size limit of 2GB.
No, there is no such limit in tar. The limit is due to the SMB mount you are using. You need to add the lfs mount option to be able to store files files larger than 2GB:
Quote:
Originally Posted by keimdf View Post
Code:
mount -t smbfs //midgard/CodeBKUP /backups -o rw,username=backup,password="backup",lfs
 
1 members found this post helpful.
Old 12-03-2011, 10:30 AM   #3
lithos
Senior Member
 
Registered: Jan 2010
Location: SI : 45.9531, 15.4894
Distribution: CentOS, OpenNA/Trustix, testing desktop openSuse 12.1 /Cinnamon/KDE4.8
Posts: 1,144

Rep: Reputation: 217Reputation: 217Reputation: 217
Quote:
Originally Posted by Nominal Animal View Post
No, there is no such limit in tar. The limit is due to the SMB mount you are using. ...
True, I can confirm that, I do kind of same backups with TAR files as big as 15GB with no problems.

Quote:
... tar errors out towards the end of the file with a bad EOF or something like that ...
if you have another place to copy the TAR file to and then extract it there, then it must be some problem with your 'current' drive that doesn't know how to handle big files, try the suggested mount option "Nominal Animal" gave.

good luck.
 
Old 12-07-2011, 02:43 PM   #4
keimdf
LQ Newbie
 
Registered: Dec 2004
Location: Wabasha, MN
Posts: 24

Original Poster
Rep: Reputation: 15
I tried adding the lfs to the end of the tar command and the size of the file still stops at 2GB. Actually, it stops at 2.15 rounded.
 
Old 12-07-2011, 04:10 PM   #5
michaelk
Moderator
 
Registered: Aug 2002
Posts: 16,124

Rep: Reputation: 1866Reputation: 1866Reputation: 1866Reputation: 1866Reputation: 1866Reputation: 1866Reputation: 1866Reputation: 1866Reputation: 1866Reputation: 1866Reputation: 1866
Quote:
Originally Posted by keimdf View Post
I tried adding the lfs to the end of the tar command and the size of the file still stops at 2GB. Actually, it stops at 2.15 rounded.
The end of the tar command or mount command?
BTW what linux distribution and samba version are you running? Even though lfs is a valid option your version of smbfs might not be capable.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Get around 2GB Tar Limit richinsc Debian 10 03-13-2008 10:24 AM
help needed to open 5.2GB tar.gz gilbertt Linux - Software 2 02-28-2007 03:47 PM
BackUp & Restore with TAR (.tar / .tar.gz / .tar.bz2 / tar.Z) asgarcymed Linux - General 5 12-31-2006 02:53 AM
tar backup maximum 2gb only per tape tsaravan Linux - General 6 09-20-2006 06:45 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 01:00 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration