[SOLVED] Unable to rm -Rf directory via script

theillien · 10-07-2011, 11:58 AM

Without going into the obvious ramifications of running rm -Rf on a directory via an automated script, I seem to be having a bugger of a time doing so.

Is bash set up to not allow that particular command from a script? I can run it against individual files and it works fine but not against one particular finding which happens to be a directory.

The idea is to look for unowned files (both user and group) and delete anything found. The script is below:

Code:

#!/bin/bash -x

WORKDIR=/var/local/orph_files

mv $WORKDIR/filelist.txt $WORKDIR/oldfilelist.txt
find / -nouser -o -nogroup > $WORKDIR/filelist.txt
#find / -nouser > $WORKDIR/filelist.txt
diff $WORKDIR/filelist.txt $WORKDIR/oldfilelist.txt > $WORKDIR/diffilelist.txt
sed -i '/^/ s/^<\s\|^>\s//' $WORKDIR/diffilelist.txt

if [ -s $WORKDIR/diffilelist.txt ]; then
        echo -e "The following files on $HOSTNAME are not owned by anyone and will be removed:\n " > $WORKDIR/mailfile.txt
    for i in `cat $WORKDIR/diffilelist.txt`;
    do
        if [[ $i == *proc* ]]
        then
            continue
        else
            rm -Rf $i
            echo "$i was removed" >> $WORKDIR/mailfile.txt
        fi
    done
    echo -e "\n\nThank You,\nYour Happily Running System" >> $WORKDIR/mailfile.txt
    cat $WORKDIR/mailfile.txt | mail -s "$HOSTNAME Orphaned Files" admin@company.com
else
    echo "There are no unowned files on $HOSTNAME." > $WORKDIR/mailfile.txt
    echo -e "\n\nThank You,\nYour Happily Running System" >> $WORKDIR/mailfile.txt
    cat $WORKDIR/mailfile.txt | mail -s "$HOSTNAME Orphaned Files" admin@company.com
fi

Even if I run

Code:

find / -nouser -o -nogroup -exec rm -Rf {} \;

doesn't work.

If I run the above find without -exec it spits out the files and directory as expected.

crts · 10-07-2011, 12:14 PM

Hi,

are you running this script as normal user or root? Also, do not use a for-loop for this task. Use a while-loop instead:

Code:

while read FILE; do
    <your commands>
done < inputfile

Read more about the problems with your for-solution:
http://mywiki.wooledge.org/BashPitfa..._arg_in_.24.2A
http://mywiki.wooledge.org/BashPitfa...8ls_.2A.mp3.29

ntubski · 10-07-2011, 12:16 PM

Quote:

doesn't work

It helps if you post error messages: hard to tell what's happening otherwise. For instance, I found that rm -rf works fine on directories, but I needed to add -a -prune, otherwise find would search inside the deleted directory and complain that it no longer existed!

Code:

~/tmp/scratch$ mkdir dir
~/tmp/scratch$ ls -a
.  ..  dir
~/tmp/scratch$ find . -name dir -exec rm -rf {} \;
find: `./dir': No such file or directory
~/tmp/scratch$ ls -a
.  ..
~/tmp/scratch$ mkdir dir
~/tmp/scratch$ find . -name dir -exec rm -rf {} \; -a -prune
~/tmp/scratch$ ls -a
.  ..

theillien · 10-07-2011, 12:16 PM

Running as root via cron.

I'll read up on the issue you pointed out.

crts · 10-07-2011, 12:27 PM

Quote:

Originally Posted by theillien

Running as root via cron.

I'll read up on the issue you pointed out.

There are more issues with your script:

Code:

if [[ $i == *proc* ]]

Always quote variables. If $i contains spaces then you will get an error.

As ntubski said, post the complete error message.

theillien · 10-07-2011, 12:28 PM

Quote:

Originally Posted by ntubski

It helps if you post error messages: hard to tell what's happening otherwise.

This is the output of running the script with debugging on a server that has several unowned files that aren't the directory I'm having problems with (it does have that directory, though):

Code:

[root@server ~]# /usr/local/sbin/orph_files.sh 
+ WORKDIR=/var/local/orph_files
+ mv /var/local/orph_files/filelist.txt /var/local/orph_files/oldfilelist.txt
+ find / -nouser -o -nogroup
find: /proc/11436/task/11436/fd/4: No such file or directory
find: /proc/11436/task/11436/fd/4: No such file or directory
find: /proc/11436/task/11436/fdinfo/4: No such file or directory
find: /proc/11436/task/11436/fdinfo/4: No such file or directory
find: /proc/11436/fd/4: No such file or directory
find: /proc/11436/fd/4: No such file or directory
find: /proc/11436/fdinfo/4: No such file or directory
find: /proc/11436/fdinfo/4: No such file or directory
+ diff /var/local/orph_files/filelist.txt /var/local/orph_files/oldfilelist.txt
+ sed -i '/^/ s/^<\s\|^>\s//' /var/local/orph_files/diffilelist.txt
+ '[' -s /var/local/orph_files/diffilelist.txt ']'
+ echo -e 'The following files on server are not owned by anyone and will be removed:\n '
++ cat /var/local/orph_files/diffilelist.txt
+ for i in '`cat $WORKDIR/diffilelist.txt`'
+ [[ 2901,3104d2900 == *proc* ]]
+ rm -rf 2901,3104d2900
+ echo '2901,3104d2900 was removed'
+ for i in '`cat $WORKDIR/diffilelist.txt`'
+ [[ /home/user/app/thing/sgchroot/tmp/sess_lel29584oll6on5fufc865sie7 == *proc* ]]
+ rm -rf /home/user/app/thing/sgchroot/tmp/sess_lel29584oll6on5fufc865sie7
+ echo '/home/user/app/thing/sgchroot/tmp/sess_lel29584oll6on5fufc865sie7 was removed'

<snip>
+ for i in '`cat $WORKDIR/diffilelist.txt`'
+ [[ /proc/28305/attr == *proc* ]]
+ continue
+ for i in '`cat $WORKDIR/diffilelist.txt`'
+ [[ 3108,3110d2903 == *proc* ]]
+ rm -rf 3108,3110d2903
+ echo '3108,3110d2903 was removed'
+ for i in '`cat $WORKDIR/diffilelist.txt`'
+ [[ /var/run/sudo/user0 == *proc* ]]
+ rm -rf /var/run/sudo/user0
+ echo '/var/run/sudo/user0 was removed'
+ for i in '`cat $WORKDIR/diffilelist.txt`'
+ [[ /var/run/sudo/user1 == *proc* ]]
+ rm -rf /var/run/sudo/user1
+ echo '/var/run/sudo/user1 was removed'
+ for i in '`cat $WORKDIR/diffilelist.txt`'
+ [[ /var/run/sudo/testuser == *proc* ]]
+ rm -rf /var/run/sudo/testuser
+ echo '/var/run/sudo/testuser was removed'
+ echo -e '\n\nThank You,\nYour Happily Running System'
+ cat /var/local/orph_files/mailfile.txt
+ mail -s 'server Orphaned Files' admin@company.com

It doesn't remove the directory in question. It does remove all other unowned files. I've used both 'rm -rf' and 'rm -Rf' neither works.

crts · 10-07-2011, 12:40 PM

Which file is not removed that you think should have been removed?
The only undeleted file that I see is

Code:

/proc/28305/attr

which is not an error since it contains 'proc'. According to your script such files should not be deleted.

ntubski · 10-07-2011, 12:46 PM

You should remove the -f option from the call to rm: it suppresses error messages (I should have remembered that

).

Quote:

+ rm -rf 2901,3104d2900

Fishy, doesn't look like a full path.

Also, instead of avoiding proc files in the loop, you can tell find to skip them:

Code:

find / -wholename /proc -prune -o -nouser -o -nogroup

theillien · 10-07-2011, 02:14 PM

Quote:

Originally Posted by crts

Hi,

are you running this script as normal user or root? Also, do not use a for-loop for this task. Use a while-loop instead:

Code:

while read FILE; do
    <your commands>
done < inputfile

Read more about the problems with your for-solution:
http://mywiki.wooledge.org/BashPitfa..._arg_in_.24.2A
http://mywiki.wooledge.org/BashPitfa...8ls_.2A.mp3.29

How can I get the file name being removed into the email like I do in the for loop?:

Code:

echo "$i was removed" >> $WORKDIR/mailfile.txt

theillien · 10-07-2011, 03:01 PM

Quote:

Originally Posted by crts

Which file is not removed that you think should have been removed?
The only undeleted file that I see is

Code:

/proc/28305/attr

which is not an error since it contains 'proc'. According to your script such files should not be deleted.

The directory that should be removed (and all files under it) doesn't show up because the script doesn't remove them. If I didn't <snip> the output it would have been several more session files as well as the skipping of several /proc files. The latter of which will go away since I've been told about the the -prune option to find.

theillien · 10-07-2011, 03:18 PM

Code:

[root@server sbin]# ./orph_files.sh 
+ WORKDIR=/var/local/orph_files
+ mv /var/local/orph_files/filelist.txt /var/local/orph_files/oldfilelist.txt
+ find / -wholename /proc -prune -o -nouser -o -nogroup
+ diff /var/local/orph_files/filelist.txt /var/local/orph_files/oldfilelist.txt
+ sed -i '/^/ s/^<\s\|^>\s//' /var/local/orph_files/diffilelist.txt
+ '[' -s /var/local/orph_files/diffilelist.txt ']'
+ echo 'There are no unowned files on server.'
+ echo -e '\n\nThank You,\nYour Happily Running System'
+ cat /var/local/orph_files/mailfile.txt
+ mail -s 'server Orphaned Files' admin@company.com
[root@server sbin]# ll /usr/src/
total 28
drwxr-xr-x 2 root root 4096 Oct  1  2009 debug
drwxr-xr-x 5 root root 4096 Aug 31 22:56 kernels
drwxr-xr-x 7 root root 4096 Nov 17  2010 redhat
drwxr-xr-x 7  201  201 4096 May 24  2010 vmware-tools-distrib

ntubski · 10-07-2011, 04:09 PM

Okay, the problem is the bit of code where you produce diffilelist.txt:

Code:

diff $WORKDIR/filelist.txt $WORKDIR/oldfilelist.txt > $WORKDIR/diffilelist.txt
sed -i '/^/ s/^<\s\|^>\s//' $WORKDIR/diffilelist.txt

I'm not sure what exactly your intention is, but it should be pointed out that the sed command leaves some stuff from the diff that is not a filename, thus the "fishy" "2901,3104d2900".

Ignoring that, diffilelist.txt will contain files from the new file list, and files from the old list, but not files that are in both. So the answer as to why you are not removing a file now is: you didn't remove it last time (so it's in both lists). Also, you will attempt to remove non-existant files that were successfully removed last time (because they will only be in the old list).

I don't really understand why you are interested in oldfilelist.txt at all.

Quote:

How can I get the file name being removed into the email like I do in the for loop?:
Code:

echo "$i was removed" >> $WORKDIR/mailfile.txt

Code:

while read FILE; do
    echo "$FILE was removed" >> $WORKDIR/mailfile.txt
done < inputfile

You can use "i" instead of "FILE" (in either style of loop) if you prefer shorter inscrutable variable names.

theillien · 10-19-2011, 10:50 AM

I've eliminated the whole difffile thing. Looking back, I'm not sure why I was interested in the oldfiles.txt file either. I've removed it and simply set the script to run a find and remove the files.