Read text file column by column

RVF16 · 05-30-2009, 06:08 PM

Hello.

I have the following text file "devicesinfo" :

Code:

071202004143 2 1.33 1-7
081002004779 1 1.32 1-5 video0 audio0 mixer0 radio0 vbi0 dsp1 dvb0

I would like to read for example the second line column by column.
Problem is that not only the field values change (except for the first column) but also the number of columns (except for the first 4).

I know i can read a specific column :

Code:

# cat devicesinfo | grep 081002004779 | awk '{print $3}'
1.32

I would like to make that $3 parametric and make a loop that reads each column value until there is none left.

Thank you.
Regards.

colucix · 05-30-2009, 06:47 PM

Just an idea:

Code:

#!/bin/bash
while read line
do
  count=1
  ncols=$(echo $line | wc -w)
  while [ $count -le $ncols ]
  do
    echo $line | cut -d' ' -f$count
    ((count++))
  done
done < devicesinfo

Telemachos · 05-30-2009, 06:54 PM

Here's one way, using Perl. (You can obviously adjust to do other processing on the fields and to get the parametric value (the one you for) from a file or from the person running the program, etc.)

Code:

#!/usr/bin/env perl
use strict;
use warnings;

while (<>) {
  if (m/^081002004779/) {
    chomp; # Remove newline
    # Split line on whitespace and save into @line
    my @line = split /\s+/, $_; 
    print "In line $.:\n";

    foreach my $column (@line) {
      print "\tHere's a column: $column\n";
    }
  }
}

Also, if you know that this magic line will only occur once, you should break out of the while loop early, rather than read the rest of the file.

norobro · 05-30-2009, 06:58 PM

And here's another based on your code:

Code:

$cat awk.txt
081002009999 1 1.32 1-5 video0 audio0 mixer0 radio0 vbi0 dsp1 dvb0
081002004779 1 1.32 1-5 video0 mixer0 radio0 vbi0 dsp1 dvb0
081002008888 1 1.32 1-5 video0 audio0 mixer0 vbi0 dsp1
081002004779 1 1.32 1-5 video0 dsp1 dvb0

Code:

$cat awk.txt | grep 081002004779 | awk '{for(i = 1 ; i <= $NF ; i++)  {print '$i'; next}}' 
081002004779 1 1.32 1-5 video0 mixer0 radio0 vbi0 dsp1 dvb0
081002004779 1 1.32 1-5 video0 dsp1 dvb0

RVF16 · 05-30-2009, 07:18 PM

Thank you all.

Based upon your suggestion norobro i did the following as i wanted to manipulate each value as read :

Code:

#!/bin/bash

x=$(cat devicesinfo | grep 081002004779 | awk '{ print NF }')

for i in $(seq 1 $x)
  do
    y=$(cat devicesinfo | grep 081002004779 | awk '{j='$i'}{print $j}')
    echo $y
  done

ghostdog74 · 05-30-2009, 07:38 PM

Quote:

Originally Posted by colucix

Just an idea:

Code:

#!/bin/bash
while read line
do
  count=1
  ncols=$(echo $line | wc -w)
  while [ $count -le $ncols ]
  do
    echo $line | cut -d' ' -f$count
    ((count++))
  done
done < devicesinfo

if using bash in OP's case, no need to call wc or cut.

Code:

exec < file
read line1
read line2
set -- $line2
for items in $@
do
    echo $items
done

ghostdog74 · 05-30-2009, 07:40 PM

Quote:

Originally Posted by RVF16

Thank you all.

Based upon your suggestion norobro i did the following as i wanted to manipulate each value as read :

Code:

#!/bin/bash

x=$(cat devicesinfo | grep 081002004779 | awk '{ print NF }')

for i in $(seq 1 $x)
  do
    y=$(cat devicesinfo | grep 081002004779 | awk '{j='$i'}{print $j}')
    echo $y
  done

your code is inefficient as it uses cat/grep/awk and at the same time calling them again in the for loop. Just one awk command will suffice. Awk replaces the use of cat/grep/sed so there is no need to use them when using awk

Code:

# awk 'NR==2{$1=$1;print }' OFS="\n" file
081002004779
1
1.32
1-5
video0
audio0
mixer0
radio0
vbi0
dsp1
dvb0

RVF16 · 05-31-2009, 03:48 AM

Quote:

your code is inefficient as it uses cat/grep/awk and at the same time calling them again in the for loop. Just one awk command will suffice. Awk replaces the use of cat/grep/sed so there is no need to use them when using awk

Quite right.
Thank you.

colucix · 05-31-2009, 04:00 AM

Quote:

Originally Posted by ghostdog74

if using bash in OP's case, no need to call wc or cut.

Thank you Ghostdog, I forgot about set -- to unset positional parameters. You're the best!

Telemachos · 05-31-2009, 05:45 AM

Quote:

Originally Posted by ghostdog74

if using bash in OP's case, no need to call wc or cut.

Code:

exec < file
read line1
read line2
set -- $line2
for items in $@
do
    echo $items
done

It seems to me that neither Colucix's version nor this one does quite what the OP asked for. Colucix's treats every line the same way, ignoring the request to only process lines starting with '081002004779'. Ghostdog's assumes that the relevant line is on line 2 (and appears only once).

The OP's use of grep originally suggests that in a real sample, he may have no idea which line or lines begin with the magic '081002004779'.

ghostdog74 · 05-31-2009, 05:54 AM

Quote:

Originally Posted by Telemachos

Ghostdog's assumes that the relevant line is on line 2 (and appears only once).

well, that's because he says in his first post its the second line

Quote:

The OP's use of grep originally suggests that in a real sample, he may have no idea which line or lines begin with the magic '081002004779'.

its not difficult implementing a check

Code:

while read line
do
    case $line in
    *081002004779* ) 
        set -- $line
        for items in $@
        do
            echo $items
        done
        ;;
    esac
done < file

same with the awk

Code:

awk '/081002004779/{....}' file

Telemachos · 05-31-2009, 07:16 AM

Quote:

Originally Posted by ghostdog74

well, that's because he says in his first post its the second line

Well, yes and no. He said:

Quote:

I would like to read for example the second line column by column.
Problem is that not only the field values change (except for the first column) but also the number of columns (except for the first 4).

But yeah, I didn't mean that it was hard, only that it was part of the original problem.