LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-18-2014, 03:31 PM   #1
kristo5747
Member
 
Registered: Jul 2010
Location: Earth
Distribution: Ubuntu 11.04 (Natty Narwhal)
Posts: 31

Rep: Reputation: 0
Grep and substitute?


I have to parse ASCII files, output the relevant data to a comma-delimited file and load it into a database table.

The specs for the file format have been recently updated and one section is causing problems. This is the original layout for that section.

Code:
    CSVHeaderAttr:PUIS,IdleImmediate,POH,Temp,WorstTemp
    CSVValuesAttr:NO,NO,9814,31,56
I parse it with `grep` thusly

Code:
    CSVAttributes=$(grep ^CSVValuesAttr:  ${filename}|cut -d':' -f2)
    [ -z "$CSVAttributes" ] && CSVAttributes="NA"
It works great but now that the section has new fields and they are named differently
Code:
CSVHeaderAttr:PUIS,IdleImmediateSupported,IdleImmediateEnabled,POH,Temp,WorstTemp
    CSVValuesAttr:NO,YES,YES,23861,31,51
Right now, I am grepping the files based on their layout (there is a field in the the header which tells me the version of the layout) to two different comma-delimited files and load them into two different tables. I would like to output both sections to the same file so the data scientist only has one table to use in his analysis.

Is there a way to use grep to produce an output like this and substitute empty fields with NA?

For one file type:
Code:
CSVHeaderAttr:PUIS,IdleImmediate,IdleImmediateSupported,IdleImmediateEnabled,POH,Temp,WorstTemp
    CSVValuesAttr:NO,NO,NA,NA,9814,31,56
For the other file type:
Code:
CSVHeaderAttr:PUIS,IdleImmediate,IdleImmediateSupported,IdleImmediateEnabled,POH,Temp,WorstTemp
    CSVValuesAttr:NO,NA,YES,YES,23861,31,51
Thanks for your input.
 
Old 07-19-2014, 07:56 AM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,331
Blog Entries: 55

Rep: Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529Reputation: 3529
You showed understanding of it here (https://www.linuxquestions.org/quest...values-912724/) so why not use awk to test fields for length == 0 and subst "NA"?
 
  


Reply

Tags
ksh


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
grep to file outputs more than grep to screen? tcpman Linux - Server 4 06-07-2013 05:46 AM
Creating an alias in ksh that uses grep and includes 'grep -v grep' doug248 Linux - Newbie 2 08-05-2012 03:07 PM
Substitute for grep -o? MikeyCarter Linux - Software 4 03-08-2012 02:17 PM
Trying to understand pipes - Can't pipe output from tail -f to grep then grep again lostjohnny Linux - Newbie 15 03-12-2009 11:31 PM
ps -ef|grep -v root|grep apache<<result maelstrombob Linux - Newbie 1 09-24-2003 12:38 PM


All times are GMT -5. The time now is 06:49 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration