[SOLVED] Deleting files on multiple nodes on a cluster

vaibhavtewari · 08-11-2013, 07:23 PM

I was wondering how to do this

I have files residing on folders

/atlas/node/*/user/vaibhav

where * is node1, node2, node2, .....node2000

I want to delete files residing on all these nodes, but if I try to do
rm /atlas/node/*/user/vaibhav/*

it doesn't work, saying "rm: No match."

it does work when I do
rm /atlas/node/node1234/vaibhav/*

so I was wondering how to delete files at once on all the nodes.

Thanks in advance

chrism01 · 08-11-2013, 08:47 PM

You should b able to use find cmd with a wildcard spec for the dir eg http://www.unix.com/shell-programmin...directory.html.
NB: test this in a test dir first, or (better) use 'echo' instead of 'rm' to check before you do it for real.

TRUNoise · 08-11-2013, 10:04 PM

A quick shell script could take care of this easily.

Code:

#!/bin/bash
nodes=$(ls /atlas/node/)
#nodes=( node1 node2 node3 ) #Incase you want to delete from specific folders

for i in "${nodes[@]}"
do
    Deleting from $i
    rm /atlas/node/$i/user/vaibhav/* -r
done

The script will remove everything in /user/vaibhav/ from all folders inside of /atlas/node/. I'm sure you can edit the script accordingly to fit your needs.

vaibhavtewari · 08-12-2013, 06:12 AM

Thank You for replying. Unfortunately wild cards didnt work for me, I tried something like

for n in {1..2000}; do cd '/atlas/node/n'$n'/user/vaibhav'; rm *; ls | wc -l; done

which worked. I figured this out on my own, but thank you for the second post which recommends the same thing.

TRUNoise · 08-12-2013, 11:23 AM

Quote:

Originally Posted by vaibhavtewari

Thank You for replying. Unfortunately wild cards didnt work for me, I tried something like

for n in {1..2000}; do cd '/atlas/node/n'$n'/user/vaibhav'; rm *; ls | wc -l; done

which worked. I figured this out on my own, but thank you for the second post which recommends the same thing.

If the post has no further problems and is done, please mark it [SOLVED] at the top of the thread. Thanks!

jpollard · 08-12-2013, 01:03 PM

One note - if, for some reason your delete loop is interrupted, you will get a lot of errors trying it again as some of the files will be missing. To avoid any abort, try "rm -f", so that the rm command will ignore missing files.

vaibhavtewari · 08-12-2013, 02:08 PM

Thanks. How to mark this solved ?

chrism01 · 08-13-2013, 10:41 PM

Try the 'Thread Tools' link at the top

TRUNoise · 08-13-2013, 10:43 PM

Quote:

Originally Posted by vaibhavtewari

Thanks. How to mark this solved ?

Click this link. http://www.linuxquestions.org/questi...d&t=4175473029