[SOLVED] help with script that search through csv file

konkura · 12-10-2013, 03:59 AM

Hi,

I have to create script which will search through log file in csv format
which have lines like this:

"","100","0992342235676","office","100","main location","23"
"","101","666666666","office","101","main location","24"

so I need to search third column which have numbers that begin with 56.
So far I have this script:

#!/bin/sh

INPUT=file.csv
OLDIFS=$IFS
IFS=,
[ ! -f $INPUT ] && { echo "$INPUT file not found"; exit 99; }
while read col3
do
echo $col3
done < $INPUT
IFS=$OLDIFS

but I dont know how to include search for lines that have third
columns that begin with '56'

thanx!

druuna · 12-10-2013, 04:26 AM

Have a look at this:

Code:

#!/bin/bash

INPUT=file.csv

OLDIFS=$IFS
IFS=,

[ ! -f $INPUT ] && { echo "$INPUT file not found"; exit 99; }

while read col1 col2 col3 col4 col5 col6 col7
do
  [[ "$col3" =~ ^.56.*$ ]] && echo "$col1 $col2 $col3 $col4 $col5 $col6 $col7"
done < $INPUT

# another way of doing this using bash:
while read LINE
do
  csvArray=( ${LINE} )
  [[ "${csvArray[2]}" =~ ^.56.*$ ]] && echo "$LINE"
done < $INPUT

IFS=$OLDIFS

# and another using awk:
awk -F, '$3 ~ /"56/' $INPUT

konkura · 12-10-2013, 09:31 AM

Druuna- what to say thx!!! I learn somethin new :-)
I still got one more question- is it possible, if I
have log file with time stamps like this one here,
that script start to search (it will be run periodically through
cron job) from time stamp when last cron job finished.
Let say script is started, and start search from place in
log file that have 16:00 onward? Than again script is started at
18:00 and start search in log file from place that have time stamp
17:00 onward?
thx again!

druuna · 12-10-2013, 10:21 AM

Quote:

Originally Posted by konkura

Druuna- what to say thx!!! I learn somethin new :-)

You're welcome.

Quote:

I still got one more question- is it possible, if I
have log file with time stamps like this one here,
that script start to search (it will be run periodically through
cron job) from time stamp when last cron job finished.
Let say script is started, and start search from place in
log file that have 16:00 onward? Than again script is started at
18:00 and start search in log file from place that have time stamp
17:00 onward?

First of all: The file shown in post #1 isn't a log file, but a csv file.

Yes, you can check (log) files based on a time period. Here's a simple example using awk:

Code:

$ awk '$0 >= "Dec 10 08:00:00" && $0 <= "Dec 10 09:00:00" { print $0 }' logfile

The above will show lines between Dec 10 08:00:00 and Dec 10 09:00:00 (included).

Your date string must have the same layout as the one used in the log file. The date command can generate just about any form you like/want.

konkura · 12-10-2013, 01:54 PM

Again, respect! Thx! :-)