help with grep/sed/awk

nikunjbadjatya · 02-17-2010, 04:33 AM

Hi,
I have a directory containing files of the following pattern:

b04-10.232.146.23-20100123000005.csv.gz
b04-10.233.146.24-20100123000024.csv.gz
...........
...........

I want to copy the pattern 2010* from every filename and store it in a variable.

ex. for the above 1st file, i want,
20100123000005 to be stored in some variable.

Please help.!!

pixellany · 02-17-2010, 04:43 AM

You can get the desired number using sed or grep. The regex (regular expression) would be:
2010[0-9]* ## read this as: Literal "2010", followed by any number of digits.

For example, try "grep -o "2010[0-9]*" filename

Do you want a different variable for each line, or are you going to iterate in a loop and use the variable to do something else?

ashok.g · 02-17-2010, 05:02 AM

This code may help you!

Code:

#/usr/bin/perl -w
use strict;
my @files=`ls|grep \".csv.gz\"`;
foreach(@files)
{
s/.*-(2010[0-9]+).*/\1/g;
print;
}

nikunjbadjatya · 02-17-2010, 05:03 AM

Hi,
Yes, I am goin through a loop. basically, the pattern has to matched and stored into variable $timestamp, then this variable has to inserted in each and every line of that file.

here's my piece of code:

while read filename
do
echo $filename

#The matched pattern 2010* has to be found here
#it has to be stored in a variable $timestamp

cat $filename | while read line
do
sed 's/^/$timestamp,/' $line

#will insert $timestamp at the begginnig of every line
done
echo

done < $no_of_files

?????

pixellany · 02-17-2010, 05:14 AM

Code:

for filename in *; do
    timestamp=$(grep -o "2010[0-9]*" $filename)
    echo $timestamp
done

This loops thru all the files in the current directory and extracts the value to assign to timestamp.

Note: In your code, you have this:

Code:

while read filename
do
.
.
done < $no_of_files

This implies that you are trying to get the filenames from a variable "no_of_files". This will not work---the redirection operator is used to specify a file to read from.

pixellany · 02-17-2010, 05:17 AM

PS: Please put your code in [CODE] tags (Advanced mode--select text and then "#")

nikunjbadjatya · 02-17-2010, 05:41 AM

Hi again,
Thanks alot, It worked with a slight problem..

Code:

for filename in *; do
    timestamp=$(grep -o "2010[0-9]*" $filename)
    echo $timestamp
    cat $filename | while read line
    do
    echo $line
    sed 's/^/$timestamp,/' $line

#will insert $timestamp at the begginnig of every line
    done
done

The timestamp insertion at the begginning of the line using sed isnt working..!!

also,
while read filename
do
.
.
done < $no_of_files
$no_of_files has all the filenames present in a directory.

pixellany · 02-17-2010, 05:53 AM

To use a variable inside the SED command, you need to use double-quotes:

Code:

 sed "s/^/$timestamp,/" $line

for the redirection issue:

Code:

while read stuff; do
  things
done < $name

This works if the value of the variable "name" is the name of a file in the current directory or is an absolute pathname.

ghostdog74 · 02-17-2010, 07:29 PM

just use the shell, no need external commands

Code:

declare -a array
for file in 2010*.csv.gz
do
  t=${file##*-}
  t=${t%%.*}
  array=(${array[@]} $t)
done