bash script using sed/scp/ssh has issues with delimited file

ScottThornley · 03-17-2009, 09:11 PM

I pulled the nixCraft line-by-line processing script, and edited it to search for and modify a pattern, save the modified pattern to a temp file, then scp the modified file to a remote host.

This works beautifully for the first line in a file containing two arguments per line: the string to insert in the sed search and replace above, and the ip address of the target host

contents of FILE:

100 10.1.1.1
200 10.1.1.2
300 10.1.1.3
.
.
3100 10.1.1.31

This subroutine just works on line one of FILE:

Code:

# User define Function (UDF)
processLine(){
     line="$@" # get all args

     ID=$(echo $line | awk '{ print $1 }')
     Coll_ID="id="$ID""
     Coll_IP=$(echo $line | awk '{ print $2 }')

     echo "Setting Collector at $Coll_IP to be AD ID $Coll_ID"

     # writing temporary file on local system
     # this version of sed does not support -i
     sed -e "s/id=[0123456789]*/$Coll_ID/" /<path>/<filename1> > <tempfile>

     # copying temp file to target
     scp <tempfile> root@"$Coll_IP":/<targetpath/<targetfile>

     # Now  showing contents of remote file to verify the change
     echo "This is the modified line from $Coll_IP"
     ssh -l root "$Coll_IP" grep -e "id=" /<targetpath>/<targetfile>

     echo -e "\n"
}

Generated output:

Quote:

Code:

Setting Collector at 10.1.1.1 to be AD ID 100
<tempfile>                                   100%  253     0.3KB/s   00:00
This is the modified line from 10.1.1.1
/<targetpath>/<targetfile>:id=100

As can be seen, lines 2 and beyond of FILE are not processed, so there are 30 other systems that do not get the file pushed out to them.

Now, if I just echo the two arguments, all lines are processed:

Code:

# User define Function (UDF)
processLine(){
     line="$@" # get all args

     ID=$(echo $line | awk '{ print $1 }')
     Coll_ID="id="$ID""
     Coll_IP=$(echo $line | awk '{ print $2 }')
     echo "$Coll_ID and $Coll_IP"
}

Generated output:

Quote:

id=100 and 10.1.1.1
id=200 and 10.1.1.2
id=300 and 10.1.1.3
.
.
.
id=3100 and 10.1.1.31

Yes, all target systems are set to use passwordless scp, all target keys are on local system.

I've tried moving the procedure processLine to a completely different script, with the same results.

Any ideas? Google has not been my friend so far...

Regards,
Scott

BrianK · 03-17-2009, 09:27 PM

What is it that you're trying to accomplish? I don't see a question other than "any ideas"? Do you want the second output to look like the first?

... and you should put code inside code tags so their formatting is preserved

Code:

  like this(arg)
     indents are visible

ScottThornley · 03-18-2009, 12:18 PM

Brian,

I've added code tags, but to be quite frank I don't think they'll do much for you, as this is essentially inline code. I've also edited a bit for clarity as well.

Goal:

Push a configuration file out to 31 systems. Each file is unique to each system, in that one line, containing the string "id=<number>" is different for each system. There is a prototype config file on the local system, which is run through sed to create a temporary file to be copied to the remote system. I then use grep via ssh to look at the contents of the file on the remote system to 1) verify that it exists remotely, and 2) that the ID is correct for the IP.

I'm using a file (FILE) that contains ID and IP pairs to provide arguments for the script. When I just echo the manipulated ID and the IP address, all 31 lines of FILE are processed. When I attempt to sed/scp/ssh as in the first example of proccessLine, only a single ID/IP pair is processed, and then the script terminates.

The feedback I'm looking for, is just why only only one line of FILE is processed when I sed/scp/ssh, rather than all 31.

Regards,
Scott

jan61 · 03-18-2009, 01:38 PM

Moin,

I'm not familiar with nixCraft - but I wonder if the exit code of a user defined function is evaluated. Try to add "return 0" at the end of the function.

Jan

ntubski · 03-18-2009, 03:00 PM

My guess is ssh is eating the input, try with

Code:

ssh -nl root "$Coll_IP" grep -e "id=" /<targetpath>/<targetfile>

Code:

man ssh
...
     -n      Redirects stdin from /dev/null (actually, prevents reading from stdin).  This must be
             used when ssh is run in the background.  A common trick is to use this to run X11 pro‐
             grams on a remote machine.  For example, ssh -n shadows.cs.hut.fi emacs & will start an
             emacs on shadows.cs.hut.fi, and the X11 connection will be automatically forwarded over
             an encrypted channel.  The ssh program will be put in the background.  (This does not
             work if ssh needs to ask for a password or passphrase; see also the -f option.)

ScottThornley · 03-18-2009, 03:45 PM

Well, I can definitely rule out sed and ssh as the issue, as a similar script that is just using scp to push a couple files and then uses ssh to show a remote directory listing is failing in the same manner. I commented out the ssh commands and the batch "push" fails after a single upload as the original code above.

Thanks!
Scott