LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 06-09-2006, 10:14 AM   #1
dfidler
LQ Newbie
 
Registered: Sep 2005
Posts: 14

Rep: Reputation: 0
bash, cp and filename (with spaces) woes


Hey All,

I am trying to get a script to move files from one dir to another when they are older than 180 days. What I have so far:
Quote:
#Modify this to change the destination of the various components
BASE_DIR="/sumfiles
TARGET_DIR="/backup-sumfiles
AGE=180

echo "Moving all files older than $AGE days from $BASE_DIR -> $TARGET_DIR"
echo
# Make sure that our target directory exists
mkdir -p $TARGET_DIR

# Move the files
for i in `find $BASE_DIR -type f -mtime +$AGE`; do
cp "$i" $TARGET_DIR/ --parents -p
if (( $? != 0 )); then
echo "FAILED TO COPY $i; BAILING OUT"
exit 1;
fi
rm "$i"
echo "Removed: \"$i\""
done
This works great, right up until I hit a filename that contains spaces. Just quoting my $i doesn't seem to work (now, why don't all mtools have a -0 option? for the love of god!) I would *LOVE* to be able to just move the files, but I need to maintain the directory structure under which the files reside (hence --parents).

Anyone have any suggestions on how I can make it that last foot-and-a-half?

I am sure that I could use something like:

Quote:
find $BASE_DIR -type f -mtime $AGE -print0 | xargs -0 -l1 cp --parents -p "{}" $TARGET_DIR/ && rm {} \;
But I couldn't figure out how to get xargs to like the &&.

Any suggestions? I'd prefer to use the for loop from the first method as it is cleaner, easier to read and way easier to comment for the next guy that comes along.

note: maintaining the file attribs and directory structure are the key requirements here.

Dave.
 
Old 06-09-2006, 10:50 AM   #2
MensaWater
Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 5,973
Blog Entries: 5

Rep: Reputation: 778Reputation: 778Reputation: 778Reputation: 778Reputation: 778Reputation: 778Reputation: 778
This is how I did something similar for Oracle binary files as they contain spaces in their names:

Code:
#!/bin/ksh
# Simple script to change owner of files from original owner to new owner.
# jlightner 23-Aug-2005
#
# Usage:  chownit olduid newuid
#
find / -user $1 -fsonly vxfs |awk '{print "\""$0"\""}'|xargs chown -h $2
What I found was I had to escape the quotes with the \ but then quote the escape characters themselves. $0 in awk means "the entire line" so what the abvoe does is send the entire line (file name) with a quote at the beginning and a quote at the end to xargs. All the other quotes in that print statement are just so it will use the escape as an escape rather than a litteral.

(REGEXP is such fun!).

P.S. Rant about Oracle: What kinds of morons use spaces in their file names AND name a directory "core"? grrrr
 
Old 06-09-2006, 11:10 AM   #3
dfidler
LQ Newbie
 
Registered: Sep 2005
Posts: 14

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by jlightner
This is how I did something similar for Oracle binary files as they contain spaces in their names:

Code:
#!/bin/ksh
# Simple script to change owner of files from original owner to new owner.
# jlightner 23-Aug-2005
#
# Usage:  chownit olduid newuid
#
find / -user $1 -fsonly vxfs |awk '{print "\""$0"\""}'|xargs chown -h $2
What I found was I had to escape the quotes with the \ but then quote the escape characters themselves. $0 in awk means "the entire line" so what the abvoe does is send the entire line (file name) with a quote at the beginning and a quote at the end to xargs. All the other quotes in that print statement are just so it will use the escape as an escape rather than a litteral.

(REGEXP is such fun!).

P.S. Rant about Oracle: What kinds of morons use spaces in their file names AND name a directory "core"? grrrr
re: Oracle rant; LOL

As for your code, you prob could have done:

Code:
find / -user $1 -fsonly vxfs -print0 | xargs -0 chown -h $2
-print0 delimits the return vals from find with the null char while -0 tells xargs to expect null delimited strings

As far as your awking goes, I will give it a shot. I tried using sed/tr to replace the spaces in my strings with "\ " before but that required the use of echo; perhaps this will meet with more success...

Thanks.

Last edited by dfidler; 06-09-2006 at 11:14 AM.
 
Old 06-09-2006, 11:12 AM   #4
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
The problem is takes place before the copy command. When find returns a file name with a space, it is in the top of the for loop where it is read as two separate arguments.

You can temporarily change the value of the the IFS variable so that this doesn't happen.
[code]
ifs=$IFS
IFS='\
'
for file in $(find $BASE_DIR -type f -mtime +$AGE); do
cp -p --parent "$file" $BACKUP_DIR && rm "$file" || echo failed to backup "$file"
done
IFS=ifs

It doesn't make sense to me to abort the backup because a single copy failed.
You may still have a problem with certain evil characters, such as "!" which have a special meaning to the shell, and are evaluated in double quotes.
 
Old 06-09-2006, 12:02 PM   #5
MensaWater
Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 5,973
Blog Entries: 5

Rep: Reputation: 778Reputation: 778Reputation: 778Reputation: 778Reputation: 778Reputation: 778Reputation: 778
The -0 option isn't available in HP-UX xargs which is where I came up with this script originally. Thanks for the tip though - It may come in handy if I need to do something similar in Linux (as is likely - we're looking at migrating our Oracle Apps tier to Linux from HP-UX).
 
Old 06-09-2006, 12:06 PM   #6
dfidler
LQ Newbie
 
Registered: Sep 2005
Posts: 14

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by jlightner
The -0 option isn't available in HP-UX xargs which is where I came up with this script originally. Thanks for the tip though - It may come in handy if I need to do something similar in Linux (as is likely - we're looking at migrating our Oracle Apps tier to Linux from HP-UX).
Ah, my bad
 
Old 06-09-2006, 12:15 PM   #7
dfidler
LQ Newbie
 
Registered: Sep 2005
Posts: 14

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by jschiwal
The problem is takes place before the copy command. When find returns a file name with a space, it is in the top of the for loop where it is read as two separate arguments.

You can temporarily change the value of the the IFS variable so that this doesn't happen.
[code]
ifs=$IFS
IFS='\
'
for file in $(find $BASE_DIR -type f -mtime +$AGE); do
cp -p --parent "$file" $BACKUP_DIR && rm "$file" || echo failed to backup "$file"
done
IFS=ifs
So, if this works, do you prefer prayer or hymns?

Quote:
Originally Posted by jschiwal
It doesn't make sense to me to abort the backup because a single copy failed.
Yeah, except that I am writing this knowing that people that know nothing about the script will be running it. It is not critical if we miss a file, but it could be a bad thing under certain circumstances; "side-effects = bad". I'd rather it puke and have the know-nots call the know-whats because they can't proceed. Nothing ticks me off more than asking someone, "So, after the script ran, did you read the log file for failures?" The response is always, "Well, no... the script completed just fine". *sigh*

Quote:
Originally Posted by jschiwal
You may still have a problem with certain evil characters, such as "!" which have a special meaning to the shell, and are evaluated in double quotes.
Thanks, that is a good warning; I'll do some testing for the different use cases.

Cheers,
Dave.
 
Old 04-10-2007, 02:53 PM   #8
d1663m
LQ Newbie
 
Registered: Feb 2005
Location: MO
Distribution: Debian Sarge
Posts: 5

Rep: Reputation: 0
elegant solution for spaces or any other weird chars in filenames

Props to this post, this is a very elegant solution to weird characters in filenames:
http://www.macgeekery.com/tips/cli/h...spaces_in_bash
Code:
find . -type f -print | while read i; do touch "${i}"; done
'for' splits on spaces. Period. Regardless of quoting.
'read' does not.

Last edited by d1663m; 04-10-2007 at 02:54 PM.
 
Old 04-10-2007, 03:01 PM   #9
MensaWater
Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 5,973
Blog Entries: 5

Rep: Reputation: 778Reputation: 778Reputation: 778Reputation: 778Reputation: 778Reputation: 778Reputation: 778
Of course although I put mine in a script to be used later and commented it and put in the shell interpreter my original solution really was one line also.

I'll admit it had a few more characters than the above line though.

On the plus side the OP didn't have to wait nearly a year for my answer.
 
Old 04-10-2007, 11:12 PM   #10
cfaj
Member
 
Registered: Dec 2003
Location: Toronto, Canada
Distribution: Mint, Mandriva
Posts: 221

Rep: Reputation: 31
Quote:
Originally Posted by d1663m
Props to this post, this is a very elegant solution to weird characters in filenames:
http://www.macgeekery.com/tips/cli/h...spaces_in_bash
Code:
find . -type f -print | while read i; do touch "${i}"; done

That will fail if any filenames have leading or trailing spaces or end in a backslash. (Not to mention filenames containing newlines.)

Quote:
'for' splits on spaces. Period. Regardless of quoting.

It does not split on spaces if there is no space in the value of $IFS.
Quote:
'read' does not.

It does if more than one variable is given as an argument (and IFS contains a space).

 
  


Reply

Tags
bash, cp, filename, quoting, read, script, scripting, spaces


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Bash script to remove capitalisation and spaces form a filename scuzzman Programming 11 05-18-2008 12:28 PM
Script: lame stumbles on spaces in filename browny_amiga Linux - General 5 05-14-2008 08:14 AM
Getting rm to like spaces in filename kyosuke Linux - General 2 02-28-2006 04:55 PM
Spaces in bash parameters rose_bud4201 Programming 5 03-11-2005 06:57 AM
spaces in filename in console...? AlThor880 Linux - Software 4 01-09-2003 02:36 PM


All times are GMT -5. The time now is 11:39 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration