LinuxQuestions.org - [SOLVED] Bash - While Loop reading from two lists simultaneously

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - Bash - While Loop reading from two lists simultaneously - nested while loop (https://www.linuxquestions.org/questions/programming-9/bash-while-loop-reading-from-two-lists-simultaneously-nested-while-loop-902334/)

Bash - While Loop reading from two lists simultaneously - nested while loop

Hi,

I use while loops to make configuration changes on network devices all the time, so I am pretty familiar with while loops, but I can't seem to get this to work.

I need to make configuration changes in separate parts of the command tree on the multiple devices. This basically requires a while loop to read from two separate lists of commands and echo alternating output from those lists. I usually use expect, but in this case all I need is to echo some command output to a file and then paste it into the terminal. Here is an example:

The command output starts like this:

subnet 71.1.24.0/21 name dhcp03
range 71.1.24.2 71.1.31.254
subnet 71.50.128.0/21 name dhcp02
range 71.50.128.2 71.50.135.254
subnet 184.3.144.0/21 name dhcp01
range 184.3.144.2 184.3.151.254

I need to add "threshold falling 100 log" to the end of the lines that start with "range". However, in order to get to that point, you have to enter the lines that start with "subnet" first. So, I created two lists, similar to the following:

subnet 71.1.24.0/21 name dhcp03
subnet 71.50.128.0/21 name dhcp02
subnet 184.3.144.0/21 name dhcp01

range 71.1.24.2 71.1.31.254
range 71.50.128.2 71.50.135.254
range 184.3.144.2 184.3.151.254

I would think I would need to crete a nested while loop to do this, but I can't get it to work at all. Here is what I have so far:

#!/bin/bash
# dhcpbugchange.sh

outer=1 # Set outer loop counter.

# Beginning of outer loop.
cat subnets.txt | while read subnet
do
echo "$subnet"
inner=1 # Reset inner loop counter.
# Beginning of inner loop.
cat ranges.txt | while read range
do
echo "$range threshold falling 100 log"
let "inner+=1" # Increment inner loop counter.
done
# End of inner loop.
let "outer+=1" # Increment outer loop counter.
echo # Space between output blocks in pass of outer loop.
done
# End of outer loop.

exit 0

I chmod +x and chmod 777 and this is what I get:

bash-3.00$ ./dhcpbugchanges.sh
: bad interpreter: No such file or directory
bash-3.00$

what am I doing wrong?

For the collation task, try paste -d'\n' file1.txt file2.txt. Adding the string to the "range" lines can be done a few different ways.

Code:

paste -d'\n' file1.txt file2.txt | sed '/^range /s/$/ threshold falling 100 log/'

Code:

sed 's/$/ threshold falling 100 log/' file2.txt | paste -d'\n' file1.txt -

Kevin Barry

Is your bash in /bin? While it is in openSUSE and OS X, other distributions put it in /usr/bin. What is:

Code:

$ which bash

saying?

Quote:

Originally Posted by Reuti (Post 4468400)

Is your bash in /bin? While it is in openSUSE and OS X, other distributions put it in /usr/bin. What is:

Code:

$ which bash

saying?

It looks like no interpreter is specified, based on the error, which could mean there's a hidden character between "#!" and "/bin/bash".
Kevin Barry

Quote:

Originally Posted by ta0kira (Post 4468394)

For the collation task, try paste -d'\n' file1.txt file2.txt. Adding the string to the "range" lines can be done a few different ways.

Code:

paste -d'\n' file1.txt file2.txt | sed '/^range /s/$/ threshold falling 100 log/'

Code:

sed 's/$/ threshold falling 100 log/' file2.txt | paste -d'\n' file1.txt -

Kevin Barry

Thanks! That worked!

Code:

allen@tornado:~$ paste -d'\n' subnets.txt ranges.txt | sed '/^range /s/$/ threshold falling 100 log/'

subnet 71.1.24.0/21 name dhcp03

range 71.1.24.2 71.1.31.254 threshold falling 100 log

subnet 71.50.128.0/21 name dhcp02

range 71.50.128.2 71.50.135.254 threshold falling 100 log

subnet 184.3.144.0/21 name dhcp01

range 184.3.144.2 184.3.151.254 threshold falling 100 log

Code:

allen@tornado:~$ sed 's/$/ threshold falling 100 log/' ranges.txt | paste -d'\n' subnets.txt -

subnet 71.1.24.0/21 name dhcp03

range 71.1.24.2 71.1.31.254 threshold falling 100 log

subnet 71.50.128.0/21 name dhcp02

range 71.50.128.2 71.50.135.254 threshold falling 100 log

subnet 184.3.144.0/21 name dhcp01

range 184.3.144.2 184.3.151.254 threshold falling 100 log

However, I'm one of those "give a man a fish and he'll eat for a day, teach a man to fish and he'll eat for the rest of his life" type of people. I like to learn why I'm doing what I'm doing instead of just cutting and pasting things so that I can potentially help others too. I believe your syntax says the following:

1.) paste -d'\n' subnets.txt ranges.txt | sed '/^range /s/$/ threshold falling 100 log/'

Says: "Use the paste command, using a line feed (\n) as a delimiter for the files "subnets.txt" and "ranges.txt". Pipe the output to sed. With sed, anything starting (hence the ^) with the word "range" needs to have something added to the end (hence the $) of the string (hence the usage of the s option). That "something" is "threshold falling 100 log".

2.) sed 's/$/ threshold falling 100 log/' ranges.txt | paste -d'\n' subnets.txt -

Says: Use sed to add something to the end (hence the $) of the string (hence the usage of the s option) in the file "ranges.txt". That something is "threshold falling 100 log". Pipe the output to the paste command, using a new line feed (\n) as the delimiter (hence the -d option) for the file subnets.txt. However, this is where it gets fuzzy. I'm assuming that the - tells sed to somehow start with the output of the subnets.txt file first? But, I tried removing the hyphen to see what happened and it didn't print the ranges.txt output at all?

Thanks in advance!

Quote:

Originally Posted by wolverene13 (Post 4468512)

"s" is the command to call in sed, and "/^range /" is an address specification: This means "s" will only be executed on lines that start ("^") with "range ". In the second version "s" is unconditionally called because all lines are assumed to start with "range ". '\n' is considered a newline by most of the standard command-line utilities. (One notable exception is cut, which doesn't seem to accept any escaped characters.) "s" requires 2 arguments: The pattern and the replacement. The character right after "s" is the separator ("/"). The pattern is "$", or "the end of the line" (which all lines have), which is replaced by " threshold falling 100 log". Note that the spaces I used are not separators; they're literal text.

This is a good sed reference:
http://www.grymoire.com/Unix/Sed.html

Here is a decent regular expression reference:
http://www.zytrax.com/tech/web/regex.htm

Kevin Barry

While it is rarely useful, you can, technically, read multiple files in a Bash loop simultanously, if you use file descriptors. Here are two examples:

Code:

#!/bin/bash



exec 11<input-file-A

exec 12<input-file-B

exec 13<input-file-C



# Read in tandem, until any one ends

while read -u 11 LINE1 && read -u 12 LINE2 && read -u 13 LINE3 ; do

    echo "$LINE1 $LINE2 $LINE3"

done



# OR



# Read first one primarily, the others optionally

while read -u 11 LINE1 ; do

    read -u 12 LINE2

    read -u 13 LINE3

    echo "$LINE1 $LINE2 $LINE3"

done



# Close the descriptors

exec 11<&- 12<&- 13<&-

The bash-builtins man page describes the options you can use with Bash read. It is surprisingly versatile.

(As Reuti pointed out below, I had b0rked the descriptor closing above. Fixed; thanks, Reuti!)

Quote:

Originally Posted by Nominal Animal (Post 4468857)

Code:

exec <&11- <&12- <&13-

The bash-builtins man page describes the options you can use with Bash read. It is surprisingly versatile.

Yes, it’s a nice feature and I use it to often as the assignment of the real file is only done once. Small correction, IIRC the closing is done by:

Code:

exec 11<&- 12<&- 13<&-

(section duplicating file descriptors) Otherwise stdin would get the last assigned one of &13.

Read multiple files in a Bash loop simultanously--the final solution

Code:

#!/bin/bash

# 

# crulat@gmail.com



(( $# == 0 )) && exit 1

File=(${@})



Len=$((${#File[@]}-1))

fdiscrp=$$



l_oo_p()

{



for((i=0; i<${#File[@]}; ++i))

do

    FD=$((i+10))

    ${1}

done



}



creat_fd()

{

    echo "exec ${FD}<${File[$i]}" >> /tmp/${fdiscrp}0

    Read=("${Read[*]}" read "-u" "${FD}" Line${i} "&&")

}



destroy_fd()

{

    echo "exec ${FD}<&-" >> /tmp/${fdiscrp}2

}



l_oo_p creat_fd 

. /tmp/${fdiscrp}0



echo "${Read[@]:0:${#Read[@]}-1}" >> /tmp/${fdiscrp}1



while . /tmp/${fdiscrp}1

do

    eval  echo $(eval echo \\\${Line{0..${Len}}} | sed -e 's#\$#\"\$#g; s#}#}\"#g')

done



l_oo_p destroy_fd

. /tmp/${fdiscrp}2



rm -rf /tmp/${fdiscrp}*

unset i

Quote:

Originally Posted by huaihaizi3 (Post 4470564)

Read multiple files in a Bash loop simultanously--the final solution

Code:

destroy_fd()

{

    echo "exec ${FD}<&-" >> /tmp/${fdiscrp}2

}

Why are you using a temporary file here?

Code:

destroy_fd()

{

    eval exec "${FD}<&-"

}

The same should be possible for the creation.

Quote:

Originally Posted by Reuti (Post 4470607)

Why are you using a temporary file here?

Code:

destroy_fd()

{

    eval exec "${FD}<&-"

}

The same should be possible for the creation.

Thanks, it's not necessary to create a temporary file!
and so:

Code:

while eval "${Read[@]:0:${#Read[@]}-1}"

Quote:

Originally Posted by ta0kira (Post 4468394)

For the collation task, try paste -d'\n' file1.txt file2.txt. Adding the string to the "range" lines can be done a few different ways.

Code:

paste -d'\n' file1.txt file2.txt | sed '/^range /s/$/ threshold falling 100 log/'

Code:

sed 's/$/ threshold falling 100 log/' file2.txt | paste -d'\n' file1.txt -

Kevin Barry

This worked perfectly, thanks!