LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 09-10-2011, 04:23 PM   #1
wolverene13
Member
 
Registered: May 2010
Location: Matiland, FL
Distribution: Debian Squeeze
Posts: 57

Rep: Reputation: 0
Bash - While Loop reading from two lists simultaneously - nested while loop


Hi,

I use while loops to make configuration changes on network devices all the time, so I am pretty familiar with while loops, but I can't seem to get this to work.

I need to make configuration changes in separate parts of the command tree on the multiple devices. This basically requires a while loop to read from two separate lists of commands and echo alternating output from those lists. I usually use expect, but in this case all I need is to echo some command output to a file and then paste it into the terminal. Here is an example:

The command output starts like this:

subnet 71.1.24.0/21 name dhcp03
range 71.1.24.2 71.1.31.254
subnet 71.50.128.0/21 name dhcp02
range 71.50.128.2 71.50.135.254
subnet 184.3.144.0/21 name dhcp01
range 184.3.144.2 184.3.151.254

I need to add "threshold falling 100 log" to the end of the lines that start with "range". However, in order to get to that point, you have to enter the lines that start with "subnet" first. So, I created two lists, similar to the following:

subnet 71.1.24.0/21 name dhcp03
subnet 71.50.128.0/21 name dhcp02
subnet 184.3.144.0/21 name dhcp01

range 71.1.24.2 71.1.31.254
range 71.50.128.2 71.50.135.254
range 184.3.144.2 184.3.151.254

I would think I would need to crete a nested while loop to do this, but I can't get it to work at all. Here is what I have so far:

#!/bin/bash
# dhcpbugchange.sh

outer=1 # Set outer loop counter.

# Beginning of outer loop.
cat subnets.txt | while read subnet
do
echo "$subnet"
inner=1 # Reset inner loop counter.
# Beginning of inner loop.
cat ranges.txt | while read range
do
echo "$range threshold falling 100 log"
let "inner+=1" # Increment inner loop counter.
done
# End of inner loop.
let "outer+=1" # Increment outer loop counter.
echo # Space between output blocks in pass of outer loop.
done
# End of outer loop.

exit 0

I chmod +x and chmod 777 and this is what I get:

bash-3.00$ ./dhcpbugchanges.sh
: bad interpreter: No such file or directory
bash-3.00$

what am I doing wrong?
 
Old 09-10-2011, 04:57 PM   #2
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
For the collation task, try paste -d'\n' file1.txt file2.txt. Adding the string to the "range" lines can be done a few different ways.
Code:
paste -d'\n' file1.txt file2.txt | sed '/^range /s/$/ threshold falling 100 log/'
Code:
sed 's/$/ threshold falling 100 log/' file2.txt | paste -d'\n' file1.txt -
Kevin Barry
 
Old 09-10-2011, 05:02 PM   #3
Reuti
Senior Member
 
Registered: Dec 2004
Location: Marburg, Germany
Distribution: openSUSE 15.2
Posts: 1,339

Rep: Reputation: 260Reputation: 260Reputation: 260
Is your bash in /bin? While it is in openSUSE and OS X, other distributions put it in /usr/bin. What is:
Code:
$ which bash
saying?
 
Old 09-10-2011, 05:08 PM   #4
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
Quote:
Originally Posted by Reuti View Post
Is your bash in /bin? While it is in openSUSE and OS X, other distributions put it in /usr/bin. What is:
Code:
$ which bash
saying?
It looks like no interpreter is specified, based on the error, which could mean there's a hidden character between "#!" and "/bin/bash".
Kevin Barry
 
Old 09-10-2011, 08:33 PM   #5
wolverene13
Member
 
Registered: May 2010
Location: Matiland, FL
Distribution: Debian Squeeze
Posts: 57

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by ta0kira View Post
For the collation task, try paste -d'\n' file1.txt file2.txt. Adding the string to the "range" lines can be done a few different ways.
Code:
paste -d'\n' file1.txt file2.txt | sed '/^range /s/$/ threshold falling 100 log/'
Code:
sed 's/$/ threshold falling 100 log/' file2.txt | paste -d'\n' file1.txt -
Kevin Barry
Thanks! That worked!

Code:
allen@tornado:~$ paste -d'\n' subnets.txt ranges.txt | sed '/^range /s/$/ threshold falling 100 log/'
subnet 71.1.24.0/21 name dhcp03
range 71.1.24.2 71.1.31.254 threshold falling 100 log
subnet 71.50.128.0/21 name dhcp02
range 71.50.128.2 71.50.135.254 threshold falling 100 log
subnet 184.3.144.0/21 name dhcp01
range 184.3.144.2 184.3.151.254 threshold falling 100 log
Code:
allen@tornado:~$ sed 's/$/ threshold falling 100 log/' ranges.txt | paste -d'\n' subnets.txt -
subnet 71.1.24.0/21 name dhcp03
range 71.1.24.2 71.1.31.254 threshold falling 100 log
subnet 71.50.128.0/21 name dhcp02
range 71.50.128.2 71.50.135.254 threshold falling 100 log
subnet 184.3.144.0/21 name dhcp01
range 184.3.144.2 184.3.151.254 threshold falling 100 log
However, I'm one of those "give a man a fish and he'll eat for a day, teach a man to fish and he'll eat for the rest of his life" type of people. I like to learn why I'm doing what I'm doing instead of just cutting and pasting things so that I can potentially help others too. I believe your syntax says the following:

1.) paste -d'\n' subnets.txt ranges.txt | sed '/^range /s/$/ threshold falling 100 log/'

Says: "Use the paste command, using a line feed (\n) as a delimiter for the files "subnets.txt" and "ranges.txt". Pipe the output to sed. With sed, anything starting (hence the ^) with the word "range" needs to have something added to the end (hence the $) of the string (hence the usage of the s option). That "something" is "threshold falling 100 log".

2.) sed 's/$/ threshold falling 100 log/' ranges.txt | paste -d'\n' subnets.txt -

Says: Use sed to add something to the end (hence the $) of the string (hence the usage of the s option) in the file "ranges.txt". That something is "threshold falling 100 log". Pipe the output to the paste command, using a new line feed (\n) as the delimiter (hence the -d option) for the file subnets.txt. However, this is where it gets fuzzy. I'm assuming that the - tells sed to somehow start with the output of the subnets.txt file first? But, I tried removing the hyphen to see what happened and it didn't print the ranges.txt output at all?

Thanks in advance!

Last edited by wolverene13; 09-10-2011 at 09:13 PM. Reason: Found the answers to most of my follow up questions by playing with the command, yet I still had one more question unanswered
 
Old 09-10-2011, 09:08 PM   #6
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
Quote:
Originally Posted by wolverene13 View Post
However, I'm one of those "give a man a fish and he'll eat for a day, teach a man to fish and he'll eat for the rest of his life" type of people. I like to learn why I'm doing what I'm doing instead of just cutting and pasting things so that I can potentially help others too. Can you clarify what s/$/, ^range, and \n do? According to the man page, it looks like -d designates a delimiter, so I'm guessing \n is referencing a new line? If that's the case, can you use \r too? Does s/$/ essentially say "add the following to the end of the string"? And ^range appears to say "anything beginning with the word "range", is that right? I always thought that ^ meant the inverse of whatever your string is, but is it different with sed?
"s" is the command to call in sed, and "/^range /" is an address specification: This means "s" will only be executed on lines that start ("^") with "range ". In the second version "s" is unconditionally called because all lines are assumed to start with "range ". '\n' is considered a newline by most of the standard command-line utilities. (One notable exception is cut, which doesn't seem to accept any escaped characters.) "s" requires 2 arguments: The pattern and the replacement. The character right after "s" is the separator ("/"). The pattern is "$", or "the end of the line" (which all lines have), which is replaced by " threshold falling 100 log". Note that the spaces I used are not separators; they're literal text.

This is a good sed reference:
http://www.grymoire.com/Unix/Sed.html

Here is a decent regular expression reference:
http://www.zytrax.com/tech/web/regex.htm

Kevin Barry
 
Old 09-11-2011, 08:38 AM   #7
Nominal Animal
Senior Member
 
Registered: Dec 2010
Location: Finland
Distribution: Xubuntu, CentOS, LFS
Posts: 1,723
Blog Entries: 3

Rep: Reputation: 948Reputation: 948Reputation: 948Reputation: 948Reputation: 948Reputation: 948Reputation: 948Reputation: 948
While it is rarely useful, you can, technically, read multiple files in a Bash loop simultanously, if you use file descriptors. Here are two examples:
Code:
#!/bin/bash

exec 11<input-file-A
exec 12<input-file-B
exec 13<input-file-C

# Read in tandem, until any one ends
while read -u 11 LINE1 && read -u 12 LINE2 && read -u 13 LINE3 ; do
    echo "$LINE1 $LINE2 $LINE3"
done

# OR

# Read first one primarily, the others optionally
while read -u 11 LINE1 ; do
    read -u 12 LINE2
    read -u 13 LINE3
    echo "$LINE1 $LINE2 $LINE3"
done

# Close the descriptors
exec 11<&- 12<&- 13<&-
The bash-builtins man page describes the options you can use with Bash read. It is surprisingly versatile.

(As Reuti pointed out below, I had b0rked the descriptor closing above. Fixed; thanks, Reuti!)

Last edited by Nominal Animal; 09-11-2011 at 09:18 AM. Reason: Fixed closing the descriptors -- thanks, Reuti!
 
1 members found this post helpful.
Old 09-11-2011, 09:03 AM   #8
Reuti
Senior Member
 
Registered: Dec 2004
Location: Marburg, Germany
Distribution: openSUSE 15.2
Posts: 1,339

Rep: Reputation: 260Reputation: 260Reputation: 260
Quote:
Originally Posted by Nominal Animal View Post
Code:
exec <&11- <&12- <&13-
The bash-builtins man page describes the options you can use with Bash read. It is surprisingly versatile.
Yes, it’s a nice feature and I use it to often as the assignment of the real file is only done once. Small correction, IIRC the closing is done by:
Code:
exec 11<&- 12<&- 13<&-
(section duplicating file descriptors) Otherwise stdin would get the last assigned one of &13.
 
1 members found this post helpful.
Old 09-13-2011, 08:17 AM   #9
huaihaizi3
LQ Newbie
 
Registered: Oct 2009
Location: BeiJing China
Distribution: Fedora
Posts: 19

Rep: Reputation: 0
Read multiple files in a Bash loop simultanously--the final solution

Code:
#!/bin/bash
# 
# crulat@gmail.com

(( $# == 0 )) && exit 1
File=(${@})

Len=$((${#File[@]}-1))
fdiscrp=$$

l_oo_p()
{

for((i=0; i<${#File[@]}; ++i))
do
    FD=$((i+10))
    ${1}
done

}

creat_fd()
{
     echo "exec ${FD}<${File[$i]}" >> /tmp/${fdiscrp}0
     Read=("${Read[*]}" read "-u" "${FD}" Line${i} "&&")
}

destroy_fd()
{
    echo "exec ${FD}<&-" >> /tmp/${fdiscrp}2
}

l_oo_p creat_fd 
. /tmp/${fdiscrp}0

echo "${Read[@]:0:${#Read[@]}-1}" >> /tmp/${fdiscrp}1

while . /tmp/${fdiscrp}1
do
    eval  echo $(eval echo \\\${Line{0..${Len}}} | sed -e 's#\$#\"\$#g; s#}#}\"#g')
done

l_oo_p destroy_fd
. /tmp/${fdiscrp}2

rm -rf /tmp/${fdiscrp}*
unset i

Last edited by huaihaizi3; 09-13-2011 at 08:25 AM.
 
Old 09-13-2011, 09:15 AM   #10
Reuti
Senior Member
 
Registered: Dec 2004
Location: Marburg, Germany
Distribution: openSUSE 15.2
Posts: 1,339

Rep: Reputation: 260Reputation: 260Reputation: 260
Quote:
Originally Posted by huaihaizi3 View Post
Read multiple files in a Bash loop simultanously--the final solution

Code:
destroy_fd()
{
    echo "exec ${FD}<&-" >> /tmp/${fdiscrp}2
}
Why are you using a temporary file here?
Code:
destroy_fd()
{
    eval exec "${FD}<&-"
}
The same should be possible for the creation.
 
Old 09-13-2011, 09:29 AM   #11
huaihaizi3
LQ Newbie
 
Registered: Oct 2009
Location: BeiJing China
Distribution: Fedora
Posts: 19

Rep: Reputation: 0
Quote:
Originally Posted by Reuti View Post
Why are you using a temporary file here?
Code:
destroy_fd()
{
    eval exec "${FD}<&-"
}
The same should be possible for the creation.
Thanks, it's not necessary to create a temporary file!
and so:
Code:
while eval "${Read[@]:0:${#Read[@]}-1}"

Last edited by huaihaizi3; 09-13-2011 at 09:39 AM.
 
Old 10-01-2011, 05:00 PM   #12
wolverene13
Member
 
Registered: May 2010
Location: Matiland, FL
Distribution: Debian Squeeze
Posts: 57

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by ta0kira View Post
For the collation task, try paste -d'\n' file1.txt file2.txt. Adding the string to the "range" lines can be done a few different ways.
Code:
paste -d'\n' file1.txt file2.txt | sed '/^range /s/$/ threshold falling 100 log/'
Code:
sed 's/$/ threshold falling 100 log/' file2.txt | paste -d'\n' file1.txt -
Kevin Barry
This worked perfectly, thanks!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
how to loop over text file lines within bash script for loop? johnpaulodonnell Linux - Newbie 9 07-28-2015 03:49 PM
nested loop-bash script- issue on logic yathin Linux - Newbie 6 05-31-2010 06:30 AM
BASH: Reading long filenames into an array using a loop DaneM Programming 12 09-11-2009 07:24 AM
bash scripting problem with nested if statements in while loop error xskycamefalling Programming 4 05-11-2009 03:14 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 04:28 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration