Help with piping

mijohnst · 10-20-2005, 01:30 PM

I'm having trouble understanding something. I've posted in another section about this, but maybe this is a better place for it. I'm trying to run a single command from one machine to another machine, but I can't get it to work right. I hope someone can explain what I'm doing wrong.

I run this command to find a file with a certain size and move it another directory:

ssh machine2 ls -al /tmp/test* | grep 4106 | awk '{ print "cp", $9, "/tmp/junk/" }' | bash

I get this error:

cp: cannot stat `/tmp/testfile2-1.txt': No such file or directory

When I terminal to the node and remove the ssh machine2 this command works just fine.

ls -al /tmp/test* | grep 4106 | awk '{ print "cp",$9,"/tmp/junk/" }' | bash

I'm I doing something wrong here? Thanks for the help!

homey · 10-20-2005, 01:53 PM

I haven't put it into ssh yet but on your original concept, the part of incrementing count can be trimmed down to something like this...
for (( i=1; i=<3; i++ )) ; do
echo n$i
ssh n$i

Quote:

declare COUNT=1

while [ $COUNT -le 32 ]; do
echo "n""$COUNT"
ssh n"$COUNT" ls -al /tmp/*.txt | awk '{print $9, $5}'\
if [[ $5 = 4106 ]] ; then # Look for file size of 4106k
echo "Deleting $9"
fi
COUNT=$(($COUNT+1))
done

Here's a sample I was putzing with on the local machine.

Code:

#!/bin/bash

clear
for i in tmp/*.txt ;do
   a=`echo $i | cut -d/ -f2`
   b=`stat -c %s "$i"`
if [ $b -ge 4106 ] ; then
   echo $a is large $b
   echo `mv "$i" tmp/junk`
else
   echo $a is small $b
fi
done

mijohnst · 10-20-2005, 05:37 PM

Wow homey... I like you're script a lot better then I do mine. It's much more advanced. I hope that I can get it working with my while loop to do many machines. Thanks for the input!

homey · 10-20-2005, 06:16 PM

You're welcome!
A couple of notes: Where I have tmp/*.txt ... I made a tmp in the home folder. In reality, you might have to change that to /tmp/*.txt and change the cut command to match. .... -f3 or what ever gives you the correct file name.

For example:

Code:

#!/bin/bash

clear
for i in /tmp/*.txt ;do
   a=`echo $i | cut -d/ -f3`
   b=`stat -c %s "$i"`
if [ $b -ge 4106 ] ; then
   echo $a is large $b
   echo `mv "$i" /tmp/junk`
else
   echo $a is small $b
fi
done

Also, on the machine loop, you might want to use something other than i as that is already being used in the script.
For example:
for (( c=1; c=<3; c++ )) ; do
echo n$c
ssh n$c

I have never tried something like this on ssh so I'm not sure how to proceed with that part yet.

Agrouf · 10-20-2005, 06:35 PM

Quote:

Originally posted by mijohnst
I'm having trouble understanding something. I've posted in another section about this, but maybe this is a better place for it. I'm trying to run a single command from one machine to another machine, but I can't get it to work right. I hope someone can explain what I'm doing wrong.

I run this command to find a file with a certain size and move it another directory:

ssh machine2 ls -al /tmp/test* | grep 4106 | awk '{ print "cp", $9, "/tmp/junk/" }' | bash

I get this error:

cp: cannot stat `/tmp/testfile2-1.txt': No such file or directory

When I terminal to the node and remove the ssh machine2 this command works just fine.

ls -al /tmp/test* | grep 4106 | awk '{ print "cp",$9,"/tmp/junk/" }' | bash

I'm I doing something wrong here? Thanks for the help!

I think the problem here is that the pipe is applied after the ssh. In other words, you do a local grep and a local awk and a local bash cp on a remote file name and the file does not exist locally.
try :
ssh machine2 "ls -al /tmp/test* | grep 4106 | awk '{ print "cp", $9, "/tmp/junk/" }' | bash"
(with the quotes so the whole piped commands are ssh'ed and executed remotely and not just the ls)

homey · 10-21-2005, 08:28 AM

I was putzing with the ssh part. I think you would best put the machine names into a file and read that file. For me at least, the machines aren't likely to be called m1, m2 m3 etc etc.
Here is a sample list.txt which has the machine names.

Code:

curly
tower
moe
vaio
desk1

There is a nice example of "Here Document" in the Advanced Bash Scripting Guide which you can read online.
Note: Unless you have ssh to run without passwords, you will have to put in a password twice for each machine.

Code:

#!/bin/bash
LIST="/home/list.txt"
FILE=test
(
cat <<'EOF'
#!/bin/bash

for i in /tmp/*.txt ; do
   a=`echo $i | cut -d/ -f3`
   b=`stat -c %s "$i"`
if [ $b -ge 4106 ] ; then
   echo $a is large $b
   echo `mv "$i" /tmp/junk`
else
   echo $a is small $b
fi
done

exit 0
EOF
) > $FILE
chmod +x $FILE

for name in `cat ${LIST}` ; do
   scp $FILE root@$name:/home
   ssh root@$name exec /home/test
done
exit

Edit: made some changes.

mijohnst · 10-21-2005, 01:26 PM

Homey,

You are too cool for words... Thank you so much for taking the time to figure out a problem for such a newbie. I wish there was a way to repay you for your work.

I'm started tweaking your script and it's working!!

homey · 10-21-2005, 04:14 PM

Glad to help!

Don't forget to google for the Advanced Bash Scripting Guide. It has a lot of great info and examples.