LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
LinkBack Search this Thread
Old 03-17-2009, 09:11 PM   #1
ScottThornley
LQ Newbie
 
Registered: Mar 2009
Posts: 3

Rep: Reputation: 0
Question bash script using sed/scp/ssh has issues with delimited file


I pulled the nixCraft line-by-line processing script, and edited it to search for and modify a pattern, save the modified pattern to a temp file, then scp the modified file to a remote host.

This works beautifully for the first line in a file containing two arguments per line: the string to insert in the sed search and replace above, and the ip address of the target host

contents of FILE:

100 10.1.1.1
200 10.1.1.2
300 10.1.1.3
.
.
3100 10.1.1.31

This subroutine just works on line one of FILE:

Code:
# User define Function (UDF)
processLine(){
     line="$@" # get all args

     ID=$(echo $line | awk '{ print $1 }')
     Coll_ID="id="$ID""
     Coll_IP=$(echo $line | awk '{ print $2 }')

     echo "Setting Collector at $Coll_IP to be AD ID $Coll_ID"

     # writing temporary file on local system
     # this version of sed does not support -i
     sed -e "s/id=[0123456789]*/$Coll_ID/" /<path>/<filename1> > <tempfile>

     # copying temp file to target
     scp <tempfile> root@"$Coll_IP":/<targetpath/<targetfile>

     # Now  showing contents of remote file to verify the change
     echo "This is the modified line from $Coll_IP"
     ssh -l root "$Coll_IP" grep -e "id=" /<targetpath>/<targetfile>

     echo -e "\n"
}
Generated output:

Quote:
Code:
Setting Collector at 10.1.1.1 to be AD ID 100
<tempfile>                                   100%  253     0.3KB/s   00:00
This is the modified line from 10.1.1.1
/<targetpath>/<targetfile>:id=100
As can be seen, lines 2 and beyond of FILE are not processed, so there are 30 other systems that do not get the file pushed out to them.




Now, if I just echo the two arguments, all lines are processed:
Code:
# User define Function (UDF)
processLine(){
     line="$@" # get all args

     ID=$(echo $line | awk '{ print $1 }')
     Coll_ID="id="$ID""
     Coll_IP=$(echo $line | awk '{ print $2 }')
     echo "$Coll_ID and $Coll_IP"
}
Generated output:

Quote:
id=100 and 10.1.1.1
id=200 and 10.1.1.2
id=300 and 10.1.1.3
.
.
.
id=3100 and 10.1.1.31

Yes, all target systems are set to use passwordless scp, all target keys are on local system.

I've tried moving the procedure processLine to a completely different script, with the same results.

Any ideas? Google has not been my friend so far...

Regards,
Scott

Last edited by ScottThornley; 03-18-2009 at 12:11 PM. Reason: for clarity, and added code tags
 
Old 03-17-2009, 09:27 PM   #2
BrianK
Senior Member
 
Registered: Mar 2002
Location: Los Angeles, CA
Distribution: Debian, Ubuntu
Posts: 1,334

Rep: Reputation: 51
What is it that you're trying to accomplish? I don't see a question other than "any ideas"? Do you want the second output to look like the first?

... and you should put code inside code tags so their formatting is preserved

Code:
  like this(arg)
     indents are visible
 
Old 03-18-2009, 12:18 PM   #3
ScottThornley
LQ Newbie
 
Registered: Mar 2009
Posts: 3

Original Poster
Rep: Reputation: 0
Brian,

I've added code tags, but to be quite frank I don't think they'll do much for you, as this is essentially inline code. I've also edited a bit for clarity as well.

Goal:

Push a configuration file out to 31 systems. Each file is unique to each system, in that one line, containing the string "id=<number>" is different for each system. There is a prototype config file on the local system, which is run through sed to create a temporary file to be copied to the remote system. I then use grep via ssh to look at the contents of the file on the remote system to 1) verify that it exists remotely, and 2) that the ID is correct for the IP.

I'm using a file (FILE) that contains ID and IP pairs to provide arguments for the script. When I just echo the manipulated ID and the IP address, all 31 lines of FILE are processed. When I attempt to sed/scp/ssh as in the first example of proccessLine, only a single ID/IP pair is processed, and then the script terminates.

The feedback I'm looking for, is just why only only one line of FILE is processed when I sed/scp/ssh, rather than all 31.

Regards,
Scott

Last edited by ScottThornley; 03-18-2009 at 12:19 PM.
 
Old 03-18-2009, 01:38 PM   #4
jan61
Member
 
Registered: Jun 2008
Posts: 235

Rep: Reputation: 46
Moin,

I'm not familiar with nixCraft - but I wonder if the exit code of a user defined function is evaluated. Try to add "return 0" at the end of the function.

Jan
 
Old 03-18-2009, 03:00 PM   #5
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian
Posts: 2,307

Rep: Reputation: 767Reputation: 767Reputation: 767Reputation: 767Reputation: 767Reputation: 767Reputation: 767
My guess is ssh is eating the input, try with
Code:
ssh -nl root "$Coll_IP" grep -e "id=" /<targetpath>/<targetfile>
Code:
man ssh
...
     -n      Redirects stdin from /dev/null (actually, prevents reading from stdin).  This must be
             used when ssh is run in the background.  A common trick is to use this to run X11 pro‐
             grams on a remote machine.  For example, ssh -n shadows.cs.hut.fi emacs & will start an
             emacs on shadows.cs.hut.fi, and the X11 connection will be automatically forwarded over
             an encrypted channel.  The ssh program will be put in the background.  (This does not
             work if ssh needs to ask for a password or passphrase; see also the -f option.)
 
Old 03-18-2009, 03:45 PM   #6
ScottThornley
LQ Newbie
 
Registered: Mar 2009
Posts: 3

Original Poster
Rep: Reputation: 0
Well, I can definitely rule out sed and ssh as the issue, as a similar script that is just using scp to push a couple files and then uses ssh to show a remote directory listing is failing in the same manner. I commented out the ssh commands and the batch "push" fails after a single upload as the original code above.


Thanks!
Scott

Last edited by ScottThornley; 03-18-2009 at 04:48 PM. Reason: new input
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
using sed to remove line in a comma-delimited file seefor Programming 4 03-10-2009 03:35 PM
Manipulating comma delimited CSV - newbie help with sed etc jonnymorris Programming 16 09-19-2008 06:14 AM
scp in bash script only getting to 11% just_me_then Linux - General 5 04-27-2008 03:02 AM
Sed doesn't update file in bash script chilebiker Programming 4 06-14-2007 08:25 AM
bash script problems: scp/ssh from the node of a cluster to the other server frankie_DJ Programming 2 01-27-2007 06:29 PM


All times are GMT -5. The time now is 09:37 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration