PIPE find results to tar not behaving as expected.

a1danel · 01-15-2011, 05:30 AM

Ok, I feel like a newbie on this question but;

I am trying to pipe a find command to 'tar -cvf' to create a tar archive. I have tried all the techniques that I can find or think of and this is what's happening.

I have a data directory that I would like to backup files that are older than 365days using tar and then delete the files from the data directory. There are some directories in the path that I want to exclude from the backup. And yes, I know there is other methods of doing this; but right now I am very curious as to why the below method is not working.

The results that I am getting.

# This works as expected: Tar archive only contains files&folders older than 365days.
find /path-example/ -mtime +365 | xargs tar -cvf /backup-dir/example.tar

# This command creates a tar archive with EVERYTHING in /path-example/
# seems to ignore my options.
find /path-example/ -name "*DIR_TO_EXCLUDE*" -prune -o -mtime +365 | xargs tar -cvf /backup-dir/example.tar

#
# I have also tried to exclude with tar,
find /path-example/ -mtime +365 | xargs -iXXX tar -cvf /backup-dir/example.tar XXX --exlcude="*DIR_TO_EXCLUDE/*"
#
# and it's still backing up everything in /path-example/
# however the find command returns what I expect to stdout when I run just the find command
find /path-example/ -name "*DIR_TO_EXCLUDE*" -prune -o -mtime +365

Is this a bug? or is this expected results and I am just missing something?

either way it's sure bugging the heck out of me

I am using:

CentOS release 5.4 (Final)
2.6.18-164.6.1.el5 #1 SMP Tue Nov 3 16:18:27 EST 2009 i686 i686 i386 GNU/Linux
Shell=Bash

grail · 01-15-2011, 06:40 AM

Check out the -path option and combine with -prune. Also remember, as I recently found out, that you also need to include a -print option once you have it all in place.

If you still get stuck, come back and I will give an example.

colucix · 01-15-2011, 07:05 AM

In addition to what grail suggested, I would consider the --no-recursion option of tar. The find command will print out the directories and their content separately, so that the tar command will archive the directory descending into it recursively AND the files. Since a tar archive is a stream of files, they will actually be stored twice (or more) even if upon extraction they will be overwritten and placed into the same location.

Maybe this is what you're looking for:

Code:

find /path-example -wholename /path-example/DIR_TO_EXCLUDE -prune -o -mtime +365 -print0 | xargs -0 tar --no-recursion -cvf /backup-dir/example.tar

Also note the -print0 option and the related -0 option of xargs to take in account filenames with blank spaces (if any).

cin_ · 01-15-2011, 01:01 PM

I was at it for hours... my issue? I called it, well many things... /logs, /logs/, logs/ ./logs, ./logs/, /home/logs, /home/logs/.

Code:

find /home -name "logs" -prune -o -print0 | xargs -0 tar --no-recursion -cvf goods.tar

---
IN USE

Code:

11:02:29/mnt/cing::cd /home
11:02:32/home::find .
.
./pilot
./pilot/.jpilot
./logs
./logs/2011.01.08.infSys
./logs/userlist
./logs/2011.01.14.infSys
./logs/2011.01.09.infSys
./logs/2011.01.13.infSys
./logs/2011.01.04.infSys
./ftp
./home
./home/pilot
./home/pilot/.jpilot
./home/logs
./home/logs/2011.01.08.infSys
./home/logs/userlist
./home/logs/2011.01.14.infSys
./home/logs/2011.01.09.infSys
./home/logs/2011.01.13.infSys
./home/logs/2011.01.04.infSys
11:02:35/home::cd /tmp

11:02:46/tmp::find /home -name "logs" -prune -o -print0 | xargs -0 tar --no-recursion -cvf goods.tar
tar: Removing leading `/' from member names
/home/
/home/pilot/
/home/pilot/.jpilot/
/home/ftp/
/home/home/
/home/home/pilot/
/home/home/pilot/.jpilot/

11:02:55/tmp::ls
goods.tar


11:03:02/tmp::tar xvf goods.tar 
home/
home/pilot/
home/pilot/.jpilot/
home/ftp/
home/home/
home/home/pilot/
home/home/pilot/.jpilot/
11:04:51/tmp::ls
goods.tar      home/
11:04:55/tmp::cd home
11:04:57/tmp/home::find .
.
./pilot
./pilot/.jpilot
./ftp
./home
./home/pilot
./home/pilot/.jpilot
11:05:02/tmp/home::

If you look at the instances where only `find .` is called, find() first recognises the directories without the trailing '/', and then once descended the directory takes on the '/' leading the file names

---

colucix's --no-recursion is necessary to allow the right hand side to function properly.

Similarly, -wholename "/home/logs", and grail's -path "/home/logs" work as well... Note: not "/home/logs/" or even "./logs"

colucix · 01-15-2011, 01:50 PM

Quote:

Originally Posted by cin_

Similarly, -wholename "/home/logs", works... Note: not "/home/logs/"

Yes, because find prints out directory as /home/logs and the name must be strictly matched, otherwise it is not pruned. Glad to see my suggestion helped you.