LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 04-08-2015, 04:34 AM   #1
jewfro2
LQ Newbie
 
Registered: Apr 2015
Posts: 3

Rep: Reputation: Disabled
process substitution with awk, output splitting incorrectly


I have the following code to extract two dates using awk, which are then read into two awk variables new and old respectively.
Each dates on the html file pulled with curl request is in this format:
2015-04-06 09:40:37
And two are being extracted
However the strings are being split on white space within the date strings. I tried changing OFS to ',', but it was still splitting incorrectly.
Code:
read dateStrNew dateStrOld < <(curl -k -q "$curl_call" | html2text | gawk '/Newest Sequence/ { new=$3" "$4 }/Oldest Sequence/ \
 {old=$3" "$4}END {OFS=","; print new,old }')  //new = date, old = date
Both parts of the date are being assigned to each variable using $3 and $4, then the space needs to be added back in so that the string can be used afterwards with a date command.

I just can't work out what is wrong, any help would be very much appreciated! Thanks!
 
Old 04-08-2015, 06:18 AM   #2
millgates
Member
 
Registered: Feb 2009
Location: 192.168.x.x
Distribution: Slackware
Posts: 840

Rep: Reputation: 380Reputation: 380Reputation: 380Reputation: 380
Hi,
the culprit here is the shell (bash?), rather than awk. awk outputs a line that contains spaces and read splits it. If you want to prevent that, you need to change the IFS variable in the shell:

Code:
IFS=, read dateStrNew dateStrOld < <(curl -k -q "$curl_call" | html2text | gawk '/Newest Sequence/ { new=$3" "$4 }/Oldest Sequence/ \
 {old=$3" "$4}END {OFS=","; print new,old }')  //new = date, old = date
 
Old 04-08-2015, 06:26 AM   #3
jewfro2
LQ Newbie
 
Registered: Apr 2015
Posts: 3

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by millgates View Post
Hi,
the culprit here is the shell (bash?), rather than awk. awk outputs a line that contains spaces and read splits it. If you want to prevent that, you need to change the IFS variable in the shell:

Code:
IFS=, read dateStrNew dateStrOld < <(curl -k -q "$curl_call" | html2text | gawk '/Newest Sequence/ { new=$3" "$4 }/Oldest Sequence/ \
 {old=$3" "$4}END {OFS=","; print new,old }')  //new = date, old = date
Thanks! I did actually have IFS="," followed by the read command on a new line it will work? So you are saying it needs to all be on the same line and it will work? I can't try it now until the server is up again tomorrow when we are testing. Will let you know how I go
 
Old 04-08-2015, 07:10 AM   #4
millgates
Member
 
Registered: Feb 2009
Location: 192.168.x.x
Distribution: Slackware
Posts: 840

Rep: Reputation: 380Reputation: 380Reputation: 380Reputation: 380
Quote:
Originally Posted by jewfro2 View Post
Thanks! I did actually have IFS="," followed by the read command on a new line it will work? So you are saying it needs to all be on the same line and it will work? I can't try it now until the server is up again tomorrow when we are testing. Will let you know how I go
do you mean something like this?

Code:
IFS=","
read dateStrNew dateStrOld < ...
That would also work. The difference is in this case the IFS variable will be set to "," permanently for the current shell, while the former assignment will only change IFS for the single command. If that doesn't work for you, then there is some other problem and it would help to show us the exact output of

Code:
curl -k -q "$curl_call" | html2text
and the exact results you got.
 
Old 04-08-2015, 08:04 AM   #5
jewfro2
LQ Newbie
 
Registered: Apr 2015
Posts: 3

Original Poster
Rep: Reputation: Disabled
Ah, well it's good to know that is a way of temporarily setting the IFS.
Yes I did.
Sample output:
newest ,
oldest
newest 2015-04-07 06
oldest 29:29,2015-04-06 09:40:37
newest ,
oldest
newest 2015-04-07 06
oldest 29:29,2015-04-06 09:40:37

So you can see that the comma is being read in the first variable, followed by whitespace/nothing.
Then the first part of the first date string
Then the second part of the first date string plus all of the second date string
then a comma
etc
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Column substitution with grep inside awk inconsistent r_hartman Programming 7 01-29-2014 04:00 AM
[SOLVED] Need help with awk and two csv: field match / substitution archduke83 Linux - Newbie 9 04-27-2012 12:49 PM
[SOLVED] shell script -- awk variable substitution r2d2#jedi Linux - General 2 10-07-2010 10:04 PM
[SOLVED] variable substitution in awk statement emmalg Linux - Software 12 07-02-2009 09:39 AM
Text substitution and processing with sed and awk shanecraddock@gmail.com Linux - Newbie 1 12-18-2008 12:34 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 11:06 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration