[SOLVED] awk if statement with a string gives empty file

atjurhs · 05-25-2022, 07:00 AM

hi guys,

i've got an awk script that has a string as part of it's if statement. the script runs but gives me an empty output file.

here's an example of the data file

Code:

123, 54.1, abc-1, 100
123, 26.7, abc-1, 99
123, 31.8, def-2, 99
678, 59.7, ghi-2, 100
123, 22.1, abc-1, 100

so if i write the code as

Code:

{
if($1==123 && $4==99)
  print $2,$3
}

i get a file with

Code:

26.7, abc-1
31.8, def-2

i tried to write the code as

Code:

{
if($1==123 && $4==99 && $3=="abc-1")
  print $2,$3
}

but i get back an empty file. i was expecting to get back

Code:

26.7, abc-1

i've searched the net looking for what is wrong with my syntax, but everything i've found so far says i'm doing it right? it's probably something really simple

Turbocapitalist · 05-25-2022, 07:18 AM

AWK syntax allows you to put the conditional at the beginning of the clause:

Code:

awk '$1==123 &&  $3=="abc-1" && $4==99 { print $2, $3; }' < infile > outfile

Otherwise, you might try using curly braces for clarity:

Code:

awk '{ if ($1==123 &&  $3=="abc-1" && $4==99) { print $2, $3; } }' < infile > outfile

atjurhs · 05-25-2022, 07:31 AM

Turbocapitalist, that did not work, i got the same result, empty files.

michaelk · 05-25-2022, 07:36 AM

It looks like a space problem, with numbers it does not matter since they are basically ignored. Strings not so much.

Code:

'$1==123 && $3==" abc-1" && $4==99 {print $2, $3;}'

atjurhs · 05-25-2022, 07:56 AM

yea michaelk that didn't work out either

michaelk · 05-25-2022, 08:05 AM

Here is my test case using your data.

Code:

123, 54.1, abc-1, 100
123, 26.7, abc-1, 99
123, 31.8, def-2, 99
678, 59.7, ghi-2, 100
123, 22.1, abc-1, 100

#!/bin/bash

awk -F , '$1==123 && $3==" abc-1" && $4==99 {print $2, $3;}' test.txt

 26.7 abc-1

Is there any other information that you have not provided?

syg00 · 05-25-2022, 08:14 AM

regex anyone ?.

atjurhs · 05-25-2022, 08:36 AM

Code:

#! /bin/bash/awk -f
for file in *test.csv;
do
awk 'BEGIN {FS=OFS=","}
NR==1 {print  "Double", "String"}
{ 
  a bunch of awk scripting stuff
{
if($1==123 && $3==" abc-1" && $4==99)
  {print $2,$3;}
}
}' "$file" > out_"$file"
done

it runs if i don't have the $3==" abc-1" condition, it just doesn't give me what i need

atjurhs · 05-25-2022, 08:46 AM

michaelk, your command gives me back an empty file ???

grail · 05-25-2022, 09:54 AM

So just to be clear, you ran michaelk's exact code and got an empty file or you put it into your script?

Code:

$ awk -F , '$1==123 && $3==" abc-1" && $4==99 {print $2, $3;}' test.txt
 26.7  abc-1

His code works as described for me.

awk on your system is a symbolic link to gawk or a different type of awk?

This would be my alternate so as to ignore the spaces:

Code:

$ awk 'BEGIN{FS="[ ,]+";OFS=","}$1==123 && $3=="abc-1" && $4==99 {print $2, $3;}' f1
26.7,abc-1

michaelk · 05-25-2022, 09:57 AM

Or you might be forced to use regex.

atjurhs · 05-25-2022, 10:26 AM

i don't know regex

cormacj · 05-25-2022, 10:55 PM

The problem is that awk by default treats spaces as field separators, but your data is CSV.

The fix is to pass -F ", " when running awk to tell it to use comma and space as a separator so that it will treat the data correctly.

Quote:

~/ cat awkt.txt
{
if($1==123 && $4==99 && $3=="abc-1")
print $2,$3
}

~/ cat inp.txt |awk -f awkt.txt
~/ cat inp.txt |awk -F ", " -f awkt.txt
26.7 abc-1

As to why your first script worked, well it was treating the fields as numbers and dropping the comma. This caused it to work.

syg00 · 05-26-2022, 01:20 AM

That's fragile in that more blanks may appear - this would be my preference.

Code:

 awk -F, '{if ($1==123 && $4==99 && $3 ~ /abc-1/)   print $2,$3}' infile.txt

MadeInGermany · 05-26-2022, 05:11 AM

$3 ~ /abc-1/ is not good, causes another problem:
It matches xabc-1 and abc-1x

Better have a precise FS - as others stated already.

Code:

BEGIN { FS=", " }
# a comma followed by a space

Code:

BEGIN { FS=",[ \t]+" }
# a comma followed by a whitespace

This also fixes the cast to a number. There might be awk versions that cast 123, to a string "123,"
There is no standard how the number detection should work.

Last but not least, some awk versions treat
awk -F", " like
awk -F"[, ]"
(A misfeature IMHO.)
So I only recommend -F with one character, and use a BEGIN section for setting a more complex FS