LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 10-12-2005, 12:47 PM   #1
gauge73
Member
 
Registered: Jan 2003
Location: Dallas, TX
Distribution: Fedora Core 4
Posts: 420

Rep: Reputation: 30
Backup script in bash


I have a backup script that I wrote in bash that needs a little tweaking. It currently will not backup hidden files because it is simply a recursive script that will call itself with a * on subdirectories. I'm going to add a switch to have it include hidden files such that when it recursively calls itself, it will do so twice - once for "*" and once for ".*". I just wanted to be sure that this will cover all files in a directory without exception. Is there any other way a file could be exluded from * apart from the name beginning with a period? Is there any way to have * include all files within a directory with "." and ".." being the only exceptions?
 
Old 10-12-2005, 01:15 PM   #2
anomie
Senior Member
 
Registered: Nov 2004
Location: Texas
Distribution: RHEL, Scientific Linux, Debian, Fedora, Lubuntu, FreeBSD
Posts: 3,930
Blog Entries: 5

Rep: Reputation: Disabled
Maybe. Is the script too long to post here?
 
Old 10-12-2005, 01:23 PM   #3
gauge73
Member
 
Registered: Jan 2003
Location: Dallas, TX
Distribution: Fedora Core 4
Posts: 420

Original Poster
Rep: Reputation: 30
It's fairly long. I wouldn't mind hosting on my web server and linking it for download, though. Basically, the syntax is like this:

backup <options> <files_to_back_up> <location_of_backup>

If you use the -R option (for recursion) it will call itself recursively for each subdirectory and replace the <files_to_back_up> with <path>/<subdirectory>/*. I was going to have it run twice, once for * and once for .*. However, if there is a better way to do it in a single command, that would be nice.
 
Old 10-13-2005, 03:02 PM   #4
ioerror
Member
 
Registered: Sep 2005
Location: Old Blighty
Distribution: Slackware, NetBSD
Posts: 536

Rep: Reputation: 30
Just use .*

. and .. are not included in .*, at least not in a proper shell like zsh, I don't know about bish or whatever it's called.
 
Old 10-13-2005, 03:10 PM   #5
ioerror
Member
 
Registered: Sep 2005
Location: Old Blighty
Distribution: Slackware, NetBSD
Posts: 536

Rep: Reputation: 30
Just tried .* in bash. Looks like it does include . and ..

But, as I said, zsh does not. Thus, using zsh would be the easiest solution.
 
Old 10-13-2005, 05:06 PM   #6
BuckRogers01
Member
 
Registered: Mar 2005
Distribution: Gentoo
Posts: 232

Rep: Reputation: 30
tried *.* perhaps? and maybe adding an if statement to check where it is just . or ..

Hope this helps
 
Old 10-14-2005, 11:32 PM   #7
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
If you use "/bin/ls --almost-all", you will get a listing of regular and hidden files, but not the "." or ".." entries. You could pipe the list to xargs to perform the backup.

Usually, the ls command is aliased. Try 'alias ls'
Code:
> alias ls
alias ls='/bin/ls $LS_OPTIONS'
If your ls command is aliased the same as mine, you might want to set LS_OPTIONS to a different value in your backup script so that it displays hidden files but not "." and ".."
LS_OPTIONS=-AR

Using shell globbing such as "cp .*" as may be in your program, or the "ls -AR *" as in my above example may not be the best way of doing it. The shell expands the wild card before it gets to the cp or tar command. If there are enough files, this will fail due to not enough memory. Another option is to use the find command. The two programs find and xargs are designed to work together. There is a find -print0 option that corresponds to xarg's -0 option). Using the find command would allow you to locate files modified since the last backup. Piping the output of find to xargs, you are able to limit the number of arguments handled at a time. Also, a filter between the two could for example, remove items from an exclude list. Using the tee command would allow you to log the files handled to a file.

Good Luck!

Last edited by jschiwal; 10-14-2005 at 11:43 PM.
 
Old 10-15-2005, 04:15 AM   #8
/bin/bash
Senior Member
 
Registered: Jul 2003
Location: Indiana
Distribution: Mandrake Slackware-current QNX4.25
Posts: 1,802

Rep: Reputation: 46
You may also want to look at using rsync. A simple command like:
rsync -avz /src/mydir/ /backup/mydir
Will do a complete archival backup with compression.

There is also tar and cpio.
 
Old 10-15-2005, 05:46 AM   #9
/bin/bash
Senior Member
 
Registered: Jul 2003
Location: Indiana
Distribution: Mandrake Slackware-current QNX4.25
Posts: 1,802

Rep: Reputation: 46
Sorry I forgot to answer your questions.
  1. Q. Is there any other way a file could be exluded from * apart from the name beginning with a period?
    A. GLOBIGNORE. For example, to list all files except .html files:
    GLOBIGNORE=*.html;for i in *;do echo $i;done
  2. Q. Is there any way to have * include all files within a directory with "." and ".." being the only exceptions?
    A. See jschiwal reply above.
 
Old 10-15-2005, 09:43 AM   #10
eddiebaby1023
Member
 
Registered: May 2005
Posts: 378

Rep: Reputation: 33
Quote:
Originally posted by ioerror
Just tried .* in bash. Looks like it does include . and ..

But, as I said, zsh does not. Thus, using zsh would be the easiest solution.
Use .?* instead (are you guys really so clueless about pattern matching?).

BTW, don't use zsh unless you're happy to be bourne shell incompatible.

It seems to me that your recursive solution is way over the top - have you thought of using the Right Tool For The Job instead, find(1)? You can even use its prune facility to exclude the stuff you don't want.
 
Old 10-15-2005, 10:49 AM   #11
ioerror
Member
 
Registered: Sep 2005
Location: Old Blighty
Distribution: Slackware, NetBSD
Posts: 536

Rep: Reputation: 30
I don't know why I didn't ask before, but why are you specifying all the files in the directory?

Why not just use the directory name itself?

Quote:
BTW, don't use zsh unless you're happy to be bourne shell incompatible.
Who cares about compatibility with a 30 year old shell? Times change, software improves, syntax becomes richer.

Besides, zsh is more than (ba)sh compatible enough for most purposes.
 
Old 10-15-2005, 10:53 AM   #12
ioerror
Member
 
Registered: Sep 2005
Location: Old Blighty
Distribution: Slackware, NetBSD
Posts: 536

Rep: Reputation: 30
Quote:
Use .?* instead (are you guys really so clueless about pattern matching?).
WTF are you talking about? .?* will match .. you muppet.

Besides, I don't use obsolete shells. How many times to I have to say it, zsh does not match .* BY DESIGN. i.e. the developers are intelligent enough to know that you don't want . and .. with .*, unlike (ba)sh developers apparently.
 
Old 10-17-2005, 05:10 AM   #13
bigearsbilly
Senior Member
 
Registered: Mar 2004
Location: england
Distribution: FreeBSD, Debian, Mint, Puppy
Posts: 3,287

Rep: Reputation: 173Reputation: 173
ever heard of find tar or cpio ?
 
Old 10-17-2005, 06:25 AM   #14
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
If you are writing a backup script, you might want to print out the manuals for "find" and "tar". Installing the source will give you access to the documentation source also. After running ./configure, run 'make dvi' or 'make pdf' to generate the manual.

You will probably run into things in writing the script that the manuals cover.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Bash backup script failing using tar and --newer option saldkj Programming 3 03-12-2005 12:03 PM
Bash backup script - If multiple files starting with a exist problem demoncheese Programming 2 07-29-2004 10:47 PM
send automatic input to a script called by another script in bash programming jorgecab Programming 2 04-01-2004 12:20 AM
bash shell backup script turnip Programming 0 04-03-2003 06:00 PM
bash script prob: how can i tell the script that a 'dd' has finished? Frustin Linux - General 2 04-02-2003 05:34 AM


All times are GMT -5. The time now is 04:31 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration