Deleting remote directories on multiple hosts with a bash script

zirtik · 04-15-2009, 02:02 PM

Hi,

I have a text file which has a bunch of host names in it. It looks as follows:

Code:

host1.universitydomain.edu
host2.universitydomain.edu
host3.universitydomain.edu
host4.universitydomain.edu
.
.
.

I want to delete some directories from each of these nodes if they exist.
I can ssh to all of the nodes but the file contents change dynamically so I need to write a script that does this job automatically.

My script looks like:

Code:

cat nodes.txt | while read line;
do
    echo "ssh $line rm -rf /tmp/hadoop-0.19.1 &"
    ssh $line rm -rf /tmp/hadoop-0.19.1 &
done

but when I run this script I figure that it doesn't work, for some reason when I manually ssh to the nodes and check /tmp directory, I still see hadoop-0.19.1 under /tmp. I tried to remove the & from the end so it looks like:

Code:

cat nodes.txt | while read line;
do
    echo "ssh $line rm -rf /tmp/hadoop-0.19.1"
    ssh $line rm -rf /tmp/hadoop-0.19.1
done

but in this case my script only removed the directory from the first host in the list and skipped the rest without even printing the echo message.

I'd appreciate any help, thanks in advance.

ncsuapex · 04-15-2009, 02:07 PM

I have a similar script that I use to do work on remote servers:

Quote:

for i in `cat hostnames`
do
echo "host $i returned the following results:"
ssh root@$i -C rm -rf /tmp/hadoop-0.19.1
done

see if that works for you. I also have SSH keys setup so I dont have to input passwords for each server.

zirtik · 04-15-2009, 02:35 PM

Thanks ncsuapex. That really worked. So what is the difference between iterating over the lines with a for loop and a while loop? I just want to understand why my approach failed and learn something from my mistake.

ncsuapex · 04-15-2009, 02:42 PM

To be honest.. I dont know... Im not much of a programmer.. I had to get help to figure mine out so I just posted mine because I knew it should do what you wanted. I, too, would be interested in knowing why yours didnt work.

zirtik · 04-15-2009, 02:53 PM

No worries, thanks again ncsuapex. I appreciate you being so responsive.

ghostdog74 · 04-15-2009, 07:23 PM

Quote:

Originally Posted by zirtik

difference between iterating over the lines with a for loop and a while loop?

there is no differences in your case. cut the useless cat in your while loop

Code:

while read line
do
 ssh "$line" <your command>
done < host.txt

check also how ssh us the "host" portion, eg ssh root@server.com

ntubski · 04-16-2009, 09:52 AM

Quote:

Originally Posted by zirtik

Thanks ncsuapex. That really worked. So what is the difference between iterating over the lines with a for loop and a while loop? I just want to understand why my approach failed and learn something from my mistake.

The difference is that when you do cat file | command and command is a compound command like a while statement, every command inside reads from file. In your case there were 2 commands that use input: the read command, and ssh. If you use ssh -n it should work fine.

Code:

     -n      Redirects stdin from /dev/null (actually, prevents reading from
	     stdin)...

Another way would be to open a new file descriptor:

Code:

exec 3< file # open file for reading
while read -u3 line ...