LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 07-26-2012, 01:21 PM   #1
ali2011
Member
 
Registered: Nov 2011
Location: USA, CA
Distribution: Ubuntu+Fedora
Posts: 80

Rep: Reputation: Disabled
Looping in Many Files with Shell Programming


I have a file_one that contains 200 IPs, on which I execute the following command to be broken down to 2 IPs per column:

Code:
awk '{a[NR%2+1]=a[NR%2+1]" "$0}  END {for (i in a){print a[i]}}' file_one > file_two
As a result, each column of file_two contains two IPs, and I save each column using awk to 100 files of names ip_list# (i.e., the # sign goes from 1 to 100). After that I run the following script on these 100 files so each file will have its own terminal.

Code:
while read machine; do
echo "  ";
echo "# Outer loop: $machine"; 
echo "=============================";
ssh -n -i /home/.ssh/key user_one@$machine 'myCommand';
done < ip_list#
Is there another way to perform this task in more convenient manner so the above script will loop through all the 100 files simultaneously from a single terminal (i.e., one call to the script)?

Last edited by ali2011; 07-26-2012 at 01:22 PM.
 
Old 07-27-2012, 06:31 AM   #2
divyashree
Senior Member
 
Registered: Apr 2007
Location: bbsr,orissa,India
Distribution: RHEL5 ,RHEL4,CENT OS5,FEDORA,UBUNTU
Posts: 1,347

Rep: Reputation: 135Reputation: 135
If you want to perform task simultaneously from a single terminal with ssh, then use pssh(parallel ssh)/cssh(clustered ssh).

e.g.
Quote:
pssh -h <host-file> date
where the host-file contains the list of the hosts/ips.

Last edited by divyashree; 07-27-2012 at 10:26 AM.
 
Old 07-27-2012, 07:09 AM   #3
Reuti
Senior Member
 
Registered: Dec 2004
Location: Marburg, Germany
Distribution: openSUSE 11.4
Posts: 1,319

Rep: Reputation: 252Reputation: 252Reputation: 252
What’s the problem with running your script will the original file? Do you want to have 2 machines handled in each terminal window instead in one loop across all 200 machines?
 
Old 07-27-2012, 11:52 AM   #4
ali2011
Member
 
Registered: Nov 2011
Location: USA, CA
Distribution: Ubuntu+Fedora
Posts: 80

Original Poster
Rep: Reputation: Disabled
Hi divyashree,

Your suggestion seems to work, but for each node I have to provide an RSA-key, username in order to access. I tried the command below, but it gives the following error:

Code:
./parallel-ssh -h q5 -A -l user_name myCommand
Code:
Traceback (most recent call last):
File "./parallel-ssh", line 115, in <module>
hosts = psshutil.read_hosts(opts.host_files, default_user=opts.user)
File "/usr/lib/pymodules/python2.6/psshlib/psshutil.py", line 18, in read_hosts
f = open(pathname)
IOError: [Errno 2] No such file or directory: 'q5
I tried many formats I found online, but no success. Do you know what is the correct format for q5 should be? And please check if the command is not correct.
 
Old 07-27-2012, 12:00 PM   #5
ali2011
Member
 
Registered: Nov 2011
Location: USA, CA
Distribution: Ubuntu+Fedora
Posts: 80

Original Poster
Rep: Reputation: Disabled
Hi Reuti,

The script above has no error; it just a matter of simplicity. I need to ssh an X-number of machines every time, say 100 out of 200 simultaneously from one terminal. Clearly, I ask 100 out of 200 nodes to measure certain metric at once using single terminal, and then move to the next set of nodes remaining. I'm looking for some thing, on which I can have control on the number of nodes to be ssh-ed simultaneously.
 
Old 07-27-2012, 01:30 PM   #6
ali2011
Member
 
Registered: Nov 2011
Location: USA, CA
Distribution: Ubuntu+Fedora
Posts: 80

Original Poster
Rep: Reputation: Disabled
Now I'm having the first pssh command working correctly, but the second when I try to pass my shell script, it times out although the script is working fine!

The first:
Code:
pssh -h q1 -l -P user_name "cd ping/final; /sbin/ifconfig;"
The second:
Code:
pssh -h q1 -l user_name "cd ping/final; chmod +x shell.sh;./shell.sh"
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
I would like redirect output to files using shell programming ayamkudy Programming 5 05-23-2011 08:51 AM
pseudo-files in shell programming? ocicat Programming 11 01-31-2010 06:12 PM
Shell programming: Question about source files and read data from the files ccwq Programming 3 08-05-2007 12:13 AM
shell programming - removing files with spaces ANU Programming 4 08-08-2006 04:42 AM
shell script looping ust Linux - General 2 01-19-2005 04:05 AM


All times are GMT -5. The time now is 07:52 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration