[SOLVED] bash script - awk split field on specific delimiter ignore spaces.

oly_r · 07-23-2013, 01:06 PM

I have an excel spreadsheet that i'm dropping into a csv file for manipulation. The spreadsheet has 12 columns, one of which is a date/time entry that is in the format "

Code:

...,Jul 23 2013  10:30am,..."

. I've been trying to awk -F"," split the fields and assign them to variables via while read loop. I need to validate the different fields. Even with the -F option i don't get a single variable (myLastUpdate) with the whole date and time, it breaks off at the whitespace.

Example row:

Code:

123.123.123.123,authoritative,ns.where.com,where.com,Jul 23 2013  10:30am,Mr Dumb guy,(201)555-1212,dguy@where.com,Books for Dumbies Inc,New York NY,entry11,entry12

what do i have to do to get the date (and other fields with whitespace in them) to be assigned to a single variable.

I have

Code:

awk -F"," '{print $1, $2,..., $12}' ${InFile} | \
while read myip mytype myns mydom mylupdt ... my12fld
do
  <test per field for valid entries>
done

I have changed reading the full line then assigning the variables via a cut line. So this is just a Can it be done question.

David the H. · 07-23-2013, 01:31 PM

You're sending the raw awk output through stdin, which read then breaks up and stores one white-space delimited "word" at a time. Other than removing the commas, awk isn't really doing anything for you..

So why bother with awk at all when, with the proper IFS setting, read can do the splitting for you?

Code:

wile IFS=, read -ra fields ; do

    echo "The date is ${fields[4]}"

done <"$InFile"

Of course you can use individual variable names as well, as in your original.

How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
http://mywiki.wooledge.org/BashFAQ/001

string manipulation
arrays

oly_r · 07-24-2013, 09:49 AM

Quote:

Originally Posted by David the H.

You're sending the raw awk output through stdin, which read then breaks up and stores one white-space delimited "word" at a time. Other than removing the commas, awk isn't really doing anything for you..

So why bother with awk at all when, with the proper IFS setting, read can do the splitting for you?

I have to admit, it was habit.

Quote:

Originally Posted by David the H.

Code:

wile IFS=, read -ra fields ; do

    echo "The date is ${fields[4]}"

done <"$InFile"

Of course you can use individual variable names as well, as in your original.

How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
http://mywiki.wooledge.org/BashFAQ/001

string manipulation
arrays

Ok, so i have converted it over to:

Code:

while IFS=, read -ra fields
   do
      myip=${fields[0]}
      mytype=${fields[1]}
      myns=${fields[2]}
      mydom=${fields[3]}
      mylupdt=${fields[4]}
<snip>
      if [[ test for field0 ]];then for ((i=0;i<=11;i++));do printf "${fields[${i}]}," >> field0_error.txt;done;echo >> field0_error.txt;fi
      if [[ test for field1 ]];then dothis;fi
      if [[ test for field2 ]];then dothis;fi
      if [[ test for field3 ]];then dothis;fi

  done

This does what i need and i get out of the "habit".
Thanks.