-   Linux - Newbie (
-   -   Use awk to pipe output from one file into multiple (

codestar1 05-09-2012 10:04 AM

Use awk to pipe output from one file into multiple
Hi All. Thanks for your help in advance.

I have a requirement to examine the number of delimiters in each record of a file. If the record has the expected number of delimiters it should be passed into a 'good' file. If it does not, the record should be passed into a 'bad' file. I have been able to complete this task, but it requires passing over the file twice. With bigger files this may slow down performance. I currently am using:

cat ${landing_dir}/${lfile}|awk -F "${file_delimiter}" "NF !=${file_field_count} {print NR, \$0 }" >> $bad_file

The similar code is used for the 'good' file with slight modifications. Is there a way to do this in only one pass over the file?

Thanks, again!

pan64 05-09-2012 11:15 AM

see the man page of awk. You can directly write into files in the ask script, so you can do something like this (for example):
<condition1> { print > file1 }
<condition2> { print > file2 }

grail 05-09-2012 01:27 PM

Two suggestions:

1. Remove cat as awk is designed to read files

2. Use -v option to pass variables into awk as present method could quickly get messy and is extremely confusing to the eye

Other than that, what pan64 said.

All times are GMT -5. The time now is 05:20 PM.