LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-28-2017, 03:36 AM   #1
sid87
LQ Newbie
 
Registered: Feb 2017
Posts: 9

Rep: Reputation: Disabled
Question read multiple variables in while loop


I am trying to read multiple variables from a file and trying to use them for desired output.

while read x y; do cat details.txt | grep -Ei '$x[[:space:]]+$y' | awk '{print $1}' | tr '\n' '\t';echo $x | tr '\n' '\t'; echo $y ; done < data.txt

data.txt
was CB3-WAS-APP
CB-Pro CB4-CB-Pro-APP
CB-Pro CB4-CB-Pro-Catal
CCAPZP01 CB4-CCAPZP01-APP
CCAPZP02 CB4-CCAPZP02-APP

details.txt
USSD WAS CB3-WAS-APP
Virtu CB-Pro CB4-CB-Pro-APP
Virtu CB-Pro CB4-CB-Pro-Catal
BI CCAPZP01 CB4-CCAPZP01-APP
BI CCAPZP02 CB4-CCAPZP02-APP

grep -Ei '$x[[:space:]]+$y' I am using it as there is tab in between in details file. I am not getting desired output, meaning for the 1st line it does take x as was but doesn't give me USSD from details file instead it displays $x. But when I execute manually it does give me USSD. What am i doing wrong

cat detail | grep -Ei 'WAS[[:space:]]+CB3-WAS-APP' | awk '{print $1}'
USSD

However when the loop runs this is the output

was CB3-WAS-APP
CB-Pro CB4-CB-Pro-APP
CB-Pro CB4-CB-Pro-Catal
CCAPZP01 CB4-CCAPZP01-APP
CCAPZP02 CB4-CCAPZP02-APP

Expected output should be, which is same as details.txt, However i need to do it through while loop as I need to display many other things other than just these 3 details each line

USSD WAS CB3-WAS-APP
Virtu CB-Pro CB4-CB-Pro-APP
Virtu CB-Pro CB4-CB-Pro-Catal
BI CCAPZP01 CB4-CCAPZP01-APP
BI CCAPZP02 CB4-CCAPZP02-APP

Last edited by sid87; 02-28-2017 at 03:43 AM.
 
Old 02-28-2017, 03:51 AM   #2
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,308
Blog Entries: 3

Rep: Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721
It will help to wrap the script in [code[ [/code] tags.

About the question, but not looking at the rest of the script, one way to read multiple variables to use additional file descriptors and read two files:

Code:
while read -u3 x; read -u4 y;
do
        echo $x $y;
done 3< file1.txt 4< file2.txt
Mind the spacing with the redirects, there can't be any space between the file descriptor number and the redirection symbol.

It will be up to you to ensure that the two files contain the exact same number of lines.
 
Old 02-28-2017, 03:55 AM   #3
sid87
LQ Newbie
 
Registered: Feb 2017
Posts: 9

Original Poster
Rep: Reputation: Disabled
Thanks Turbocapitalist, I have only one file so are you suggesting me to do some like the following, not sure of the syntax.

while read -u3 x; read -u4 y;
do
echo $x $y;
done 3< $(cat file1.txt | awk '{print $1}') 4< $(cat file1.txt | awk '{print $2}')
 
Old 02-28-2017, 04:01 AM   #4
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,308
Blog Entries: 3

Rep: Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721
Or are you trying to join them according to a common field?

Code:
join -1 2 -2 3 -o '2.1 2.2 1.2' data.txt details.txt
See

Code:
man join
 
Old 02-28-2017, 04:02 AM   #5
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,842

Rep: Reputation: 7309Reputation: 7309Reputation: 7309Reputation: 7309Reputation: 7309Reputation: 7309Reputation: 7309Reputation: 7309Reputation: 7309Reputation: 7309Reputation: 7309
you do not need cat:
Code:
awk '{print $1}' filename
would be sufficient
 
Old 02-28-2017, 04:06 AM   #6
sid87
LQ Newbie
 
Registered: Feb 2017
Posts: 9

Original Poster
Rep: Reputation: Disabled
No I am not joining it, i don't only need to displays 3 details in final output I need to use x & y and display other details. Main code is

#!/bin/ksh93

i=1
exec < data.txt
while read x y
do
echo $i | tr '\n' '\t'
cat details.txt | grep -Ei '$x[[:space:]]+$y' | awk '{print $1}' | tr '\n' '\t'
echo $x | tr '\n' '\t'
Pol_Type=`sudo /usr/openv/netbackup/bin/admincmd/bppllist $y -U | grep -i "Policy Type:" | awk '{print $3}'`

case "$Pol_Type" in
"DB2") echo "Application-DB2" | tr '\n' '\t'
;;
"MS-SQL-Server") echo "Application-SQL" | tr '\n' '\t'
;;
"MS-Windows") echo "Operating_System-Windows" | tr '\n' '\t'
;;
"NBU-Catalog") echo "NBU-Catalog" | tr '\n' '\t'
;;
"Oracle") echo "Application-Oracle" | tr '\n' '\t'
;;
"Standard") echo "Operating_System-Unix" | tr '\n' '\t'
;;
esac

cat /etc/hosts | grep -F "$x" | grep -v "^#" | head -1 | awk '{print $1}' | tr '\n' '\t'
echo "CB-BACK" | tr '\n' '\t'
echo "Con-B" | tr '\n' '\t'
echo "$y"
((i++))
done
 
Old 02-28-2017, 04:12 AM   #7
sid87
LQ Newbie
 
Registered: Feb 2017
Posts: 9

Original Poster
Rep: Reputation: Disabled
@pan64, I need to search $x tab $y in detail file then only it will be able to search and print $1 will be relavant.
 
Old 02-28-2017, 04:32 AM   #8
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,308
Blog Entries: 3

Rep: Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721
Quote:
Originally Posted by sid87 View Post
...
cat details.txt | grep -Ei '$x[[:space:]]+$y' | awk '{print $1}' | tr '\n' '\t'
...
Pol_Type=`sudo /usr/openv/netbackup/bin/admincmd/bppllist $y -U | grep -i "Policy Type:" | awk '{print $3}'`
...
cat /etc/hosts | grep -F "$x" | grep -v "^#" | head -1 | awk '{print $1}' | tr '\n' '\t'
...
As mentioned, cat is not needed in conjunction with either grep or awk. Further grep is almost never needed when you have awk. Same for head.

Code:
awk --assign x=$x --assign y=$y '$0 ~ x "\t" y { print $0 }' IGNORECASE=1 details.txt

awk --assign x=$x --assign y=$y '$0 ~ x "\t" y { printf ("%s", $0);} ' IGNORECASE=1 details.txt
etc.

Code:
awk --assign x=$x '$0 ~ x && !/^#/ && !n++ { print $1 }' /etc/hosts

awk --assign x=$x '$0 ~ x && !/^#/ && !n++ { printf( "%s", $1); }' /etc/hosts
 
Old 02-28-2017, 04:40 AM   #9
sid87
LQ Newbie
 
Registered: Feb 2017
Posts: 9

Original Poster
Rep: Reputation: Disabled
$ x=CB-Pro
$ y=CB4-CB-Pro-APP
$ cp detail details
$ x=CB-Pro
$ y=CB4-CB-Pro-APP
$ awk --assign x=$x --assign y=$y '$0 ~ x "\t" y { print $0 }' IGNORECASE=1 details
$ awk --assign x=$x --assign y=$y '$0 ~ x"\t"y { print $0 }' IGNORECASE=1 details
$ cat details | grep -Ei '$x[[:space:]]+$y' | awk '{print $1}'
$ cat details | grep -Ei 'CB-Pro[[:space:]]+CB4-CB-Pro-APP' | awk '{print $1}'
Virtu
$

I believe there is something wrong in my grep, when I use variables it doesn't give me anything However when I use exact text it does work.
 
Old 02-28-2017, 04:44 AM   #10
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,308
Blog Entries: 3

Rep: Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721
Quote:
Originally Posted by sid87 View Post
I believe there is something wrong in my grep, when I use variables it doesn't give me anything However when I use exact text it does work.
Yes, there are several things amiss there. However, one of them is the use of single quotes ' where double quotes " should be used. The latter allow for interpretation of shell variables, the former do not. Again, see the comments about cat and co.
 
1 members found this post helpful.
Old 02-28-2017, 04:52 AM   #11
sid87
LQ Newbie
 
Registered: Feb 2017
Posts: 9

Original Poster
Rep: Reputation: Disabled
Ohhh fish, my bad, stupid mistake, it is working fine now. However as you have mentioned How can I simplify/substitute these 2.

cat details | grep -Ei "$x[[:space:]]+$y" | awk '{print $1}'
cat /etc/hosts | grep -F "$x" | grep -v "^#" | head -1 | awk '{print $1}'
 
Old 02-28-2017, 04:56 AM   #12
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,308
Blog Entries: 3

Rep: Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721
Quote:
Originally Posted by sid87 View Post
How can I simplify/substitute these 2
See the examples in post #8, a simple awk will do. But if not, at least eliminate the unnecessary cat.

Code:
grep -Ei "$x[[:space:]]+$y" details | awk '{print $1}'
 
Old 02-28-2017, 05:03 AM   #13
sid87
LQ Newbie
 
Registered: Feb 2017
Posts: 9

Original Poster
Rep: Reputation: Disabled
yeah I have already removed cat, However your awk didn't do the job.
 
Old 02-28-2017, 05:16 AM   #14
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,308
Blog Entries: 3

Rep: Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721
What do you need to add to it or change so that it will do the job? From here it looks like it illustrates two ways of using awk to combine several activities into one.
 
Old 02-28-2017, 05:32 AM   #15
sid87
LQ Newbie
 
Registered: Feb 2017
Posts: 9

Original Poster
Rep: Reputation: Disabled
-E is extended-regex which interprets "$x[[:space:]]+$y", what would be equivalent search parameter for awk?
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
how can we use multiple variables in single for loop in shell script nagendrar Linux - Newbie 11 05-22-2017 01:21 PM
[SOLVED] Populate multiple variables via loop theillien Programming 12 05-27-2012 09:00 AM
[SOLVED] Bash script to read multiple variables from a list then output to command diplo707 Programming 6 04-23-2012 12:46 PM
Read multiple variables from a file santhosh_cv Programming 13 09-02-2011 09:24 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 11:35 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration