LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 05-09-2012, 09:04 AM   #1
codestar1
LQ Newbie
 
Registered: May 2012
Posts: 1

Rep: Reputation: Disabled
Use awk to pipe output from one file into multiple


Hi All. Thanks for your help in advance.

I have a requirement to examine the number of delimiters in each record of a file. If the record has the expected number of delimiters it should be passed into a 'good' file. If it does not, the record should be passed into a 'bad' file. I have been able to complete this task, but it requires passing over the file twice. With bigger files this may slow down performance. I currently am using:

cat ${landing_dir}/${lfile}|awk -F "${file_delimiter}" "NF !=${file_field_count} {print NR, \$0 }" >> $bad_file

The similar code is used for the 'good' file with slight modifications. Is there a way to do this in only one pass over the file?

Thanks, again!
 
Old 05-09-2012, 10:15 AM   #2
pan64
Senior Member
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 4,487

Rep: Reputation: 1220Reputation: 1220Reputation: 1220Reputation: 1220Reputation: 1220Reputation: 1220Reputation: 1220Reputation: 1220Reputation: 1220
see the man page of awk. You can directly write into files in the ask script, so you can do something like this (for example):
<condition1> { print > file1 }
<condition2> { print > file2 }
 
1 members found this post helpful.
Old 05-09-2012, 12:27 PM   #3
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,426

Rep: Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876
Two suggestions:

1. Remove cat as awk is designed to read files

2. Use -v option to pass variables into awk as present method could quickly get messy and is extremely confusing to the eye

Other than that, what pan64 said.
 
1 members found this post helpful.
  


Reply

Tags
awk


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] awk with pipe delimited file (specific column matching and multiple pattern matching) lolmon Programming 4 08-31-2011 12:17 PM
pipe output of ls to file takayama Programming 4 05-03-2011 05:13 PM
[SOLVED] perl script output cannot pipe to file hattori.hanzo Programming 3 12-15-2010 02:26 AM
how to pipe/redirect awk output into a variable? johnpaulodonnell Linux - Newbie 2 01-25-2007 06:54 AM
pipe xine output to file sohmc Linux - Software 0 01-01-2005 10:03 PM


All times are GMT -5. The time now is 07:10 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration