bash shell command

DamianS · 04-27-2007, 10:58 AM

I am an advanced beginner wrestling with complex (for me) bash commands.
After some misadventures using subversion, I want to start again and recreate the repos properly.

The problem is I have dozens of repos containing many hundreds of .svn directories. I wish to delete these .svn directories and all the files in them, but nothing else.

Here is what I have figured out:
I can find all the .svn directories by using:
find / -name .svn -print0

so I should be able to pipe the results to xargs to delete them like so:
find / -name .svn -print0 | xargs -0 rm

However, this doesn't work since the .svn's are directories, so I've worked out the whole command should be:

find / -name .svn -print0 | xargs -0 rm -frd

I haven't been brave enough to try this yet, since it is potentially catastrophic and could wipe out my entire system.
So, can someone please confirm this is correct before I attempt it?

Thanks in advance.

MensaWater · 04-27-2007, 11:40 AM

Are the directories named ".svn" or are they suffixed ".svn" (e.g. file.svn)? If the latter your find wouldn't work - you need to make it "*.svn" to find them.

You don't need the "-d" option for rm if you're using "-f" - the "-f" will make it delete even if it is a directory.

It should work as you wrote it assuming the .svn is the name rather than the suffix but the recommendation would be to backup everything before you start "just in case". There's no way anyone here will "guarantee" you it works - all advice here is to be seen as "at your own risk".

taylor_venable · 04-27-2007, 01:11 PM

I'd use the following: `find / -name '.svn'` to make sure that you catch only what you mean to. Then do a: `find / -name '.svn' -exec rm -rfv {} \;` to do the removal. Unless you have experience with this command, try it in a test directory first to make sure it does what you want.

DamianS · 04-27-2007, 11:03 PM

The dirs are named .svn
Yeah, I better make a directory and test in that first

ADDED:
Ok, thanks guys, both methods work.
ie. find / -name .svn -print0 | xargs -0 rm -fr
and
find / -name '.svn' -exec rm -rfv {} \;
both find all the '.svn' dirs and delete them and their contents, leaving everything else untouched.

I'm going to have to study the exec expression since the last half looks double greek to me.

I have found this page, discussing both approaches:
http://www.digg.com/linux_unix/Linux..._Command_xargs

Based on that, my method will not work if there are huge numbers of files to be deleted, since only so many arguments can be passed to rm.
In which case, it would be necessary to do something like:
find / -name .svn -print0 | xargs -l10000 -0 rm -fr

taylor_venable · 04-28-2007, 06:30 AM

Quote:

Originally Posted by DamianS

Based on that, my method will not work if there are huge numbers of files to be deleted, since only so many arguments can be passed to rm.
In which case, it would be necessary to do something like:
find / -name .svn -print0 | xargs -l10000 -0 rm -fr

That's why the -exec approach is better, because it is not limited by the maximum number of command-line arguments enforced by your system. Each directory gets removed as it is discovered by the find command.

MensaWater · 04-28-2007, 07:40 AM

That shows a misunderstanding of the way pipes and xargs work. While it is true a command line like "rm -f *" (no xargs) would attempt to expand the command line to a single line that could easily exceed limits this is NOT what happens with pipe into xargs.

xargs executes the rm on each LINE that gets piped in - it does NOT put all the files onto a single rm command.

Indeed, while researching to verify this very information I found a section within the xargs man page that talks about limits that ends in the following quote:

Quote:

The problem doesn't occur with the output of find(1) because it emits just one filename per line.

Having said all that I would certainly recommend using the -exec of find over the xargs as a "more proper" way to do it simply because the act of piping output into another external command has some cost on memory and processing. It is better to use single commands for efficiencies.

Just wanted to point out that xargs does have a purpose and it does not work the way the misinformed link implied by linking it with a simple rm command that didn't use find or xargs.

taylor_venable · 04-28-2007, 12:34 PM

Oh, I didn't know that. I apologize for echoing that mis-information.

makyo · 04-28-2007, 02:33 PM

Hi, jlightner.

Quote:

Originally Posted by jlightner

xargs executes the rm on each LINE that gets piped in - it does NOT put all the files onto a single rm command.

My experience differs from this. Below is a pair of scripts. The first calls the second from an exec within a find, and then calls the second script indirectly through piping into xargs:

Code:

#!/bin/sh

# @(#) s1       Demonstrate xargs collection of filenames.

# create a number of files.

for ((i=1 ; i<=10 ; i++ ))
do
        touch t$i
done
NUMBER=$( ls -1 t* | wc -l )

echo
echo " Finding $NUMBER files using -exec:"
time find . -name 't*' -exec ./call-echo {} \;

echo " Finding files, pipe into xargs:"
time find . -name 't*' |
xargs ./call-echo

exit 0

and here is call-echo:

Code:

#!/bin/sh

# @(#) call-echo        Echo arguments, print process number.

echo " Process $$, called with:"
echo $*

exit 0

Which results in:

Code:

% ./s1

 Finding 10 files using -exec:
 Process 29692, called with:
./t1
 Process 29693, called with:
./t2
 Process 29694, called with:
./t3
 Process 29695, called with:
./t4
 Process 29696, called with:
./t5
 Process 29697, called with:
./t6
 Process 29698, called with:
./t7
 Process 29699, called with:
./t8
 Process 29700, called with:
./t9
 Process 29701, called with:
./t10

real    0m0.016s
user    0m0.006s
sys     0m0.009s
 Finding files, pipe into xargs:
 Process 29704, called with:
./t1 ./t2 ./t3 ./t4 ./t5 ./t6 ./t7 ./t8 ./t9 ./t10

real    0m0.003s
user    0m0.002s
sys     0m0.00

I draw two conclusions from this. First, the exec will fire off a process for each item found. Second, xargs does collect arguments. The time, even for this short test, favors xargs. I think pushing data through a pipe is cheaper than creating a new process.

Did I misinterpret your statement? ... cheers, makyo

( edit 1: typo )
( edit 2: remove extraneous pasted-in text )