[SOLVED] reading lines from file

zuze11 · 11-17-2011, 03:12 PM

Hi, trying to read data from a file with several blocks in it. Each block corresponds to a particular day and blocks are separated by empty lines. I managed to write a small and very simple bash script that selects the relevant lines and outputs them to separate files for each day.

Here is the script:

Code:

#! /bin/bash

DESCRIPTION='Date Time Ref avgU10 gstU10 dirU10 tC10 RH10 avgU4 avgU2 avgU1 tC0_5 avgU0_5 gstU0_5'

while read LINE
     do
       FILEID=${LINE:0:6}
       if [ ${LINE:0:1} == 9 ]; then
          if [ ! -f ${FILEID}.txt ]; then # if file does not exist, add description header
             echo ${DESCRIPTION} >> ${FILEID}.txt
             echo ${LINE} >> ${FILEID}.txt
          else
             echo ${LINE} >> ${FILEID}.txt
          fi
       fi
     done

It works simply by redirecting the data file into the script:

./script < datafile

I would like to modify it so I can specify the name of the data file to be processed within the script. Tried a bunch of different things I found on the net but nothing worked. Anybody knows a good way of doing it?

Thanks

colucix · 11-17-2011, 03:15 PM

Code:

while read LINE
do
  :
done < datafile

zuze11 · 11-17-2011, 03:30 PM

Quote:

Originally Posted by colucix

Code:

while read LINE
do
  :
done < datafile

So you basically move the redirect inside, to the end of the do construct. Very cool and works like a charm. Thanks!

Can you suggest a good bash reference with examples?

colucix · 11-17-2011, 03:37 PM

You might check the Bash Guide for Beginners and then the Advanced Bash Scripting Guide, here. A very nice guide about linux command line and shell scripting is here.

zuze11 · 11-17-2011, 06:57 PM

Thanks

grail · 11-17-2011, 08:13 PM

Whilst the current example works the input file would always need to be of the same name and in the current directory.
In the sites suggested I would look for command line arguments / parameters so you could then use a complete path to any file.

zuze11 · 11-22-2011, 12:47 PM

Quote:

Originally Posted by grail

Whilst the current example works the input file would always need to be of the same name and in the current directory.
In the sites suggested I would look for command line arguments / parameters so you could then use a complete path to any file.

Here is a further evolution of the script with the input file passed by the -i flag. It also checks for illegal flags and runs anyway as long as the input file is set and exists (although it still does not differentiate between these cases).

Still does not have error checking within the data being read but it's coming...

Code:

#! /bin/bash

# This script reads a NordlysStasjonen data acquisition file with data for several days
# and splits it into a series of daily measurement files 22/11/11

display_usage() {
        echo '####'
        echo 'USAGE: extractDays -i inputfile'
        echo 'where inputfile is the global NordlysStasjonen data acquisition file'
        echo '####'
        echo ''
        }

while getopts :i: option # flag construct with leading : for illegal flag detection
      do
        case "${option}"
        in
             i) INFILE=${OPTARG};;
             ?) HUH=${OPTARG};;
        esac
      done

if [[ -n "${INFILE:+x}" && -f ${INFILE} ]]; then # Fails if INFILE is unset or empty or does not exist
       echo -e '\n\t\tVariable INFILE has been set, and input file exists.'
     else
       echo -e '\n\t\tVariable INFILE is unset or empty, or input file does not exist.'
       echo -e '\t\tExiting!!!'
       display_usage
       exit 1
fi

if [ -n "${HUH+x}" ]; then # Fails if HUH is unset
     echo -e '\tIllegal use of additional flag'
     echo -e '\t\tWill ignore illegal flag and run script since INFILE has been correctly set!\n'
     display_usage
fi

# Initialize output file description header
DESCRIPTION='Date Time Ref avgU10 gstU10 dirU10 tC10 RH10 avgU4 avgU2 avgU1 tC0_5 avgU0_5 gstU0_5'

while read LINE
     do
       FILEID=${LINE:0:6}
       if [ ${LINE:0:1} == 9 ]; then
          if [ ! -f ${FILEID}.txt ]; then # if file does not exist, add description header
             echo ${DESCRIPTION} >> ${FILEID}.txt
             echo ${LINE} >> ${FILEID}.txt
          else
             echo ${LINE} >> ${FILEID}.txt
          fi
       fi
     done < ${INFILE}

echo 'Script execution complete. Exiting...'

exit 0

grail · 11-22-2011, 10:54 PM

Code:

if [[ -n "${INFILE:+x}" && -f ${INFILE} ]]; then # Fails if INFILE is unset or empty or does not exist

Not sure I understand your comment here? Both -n and -f seem to work fine for me whether INFILE is set or not?

Code:

if [ -n "${HUH+x}" ]; then # Fails if HUH is unset
     echo -e '\tIllegal use of additional flag'
     echo -e '\t\tWill ignore illegal flag and run script since INFILE has been correctly set!\n'
     display_usage
fi

The red highlighted section would seem to be a guess? Just because 'HUH' has been set does not mean 'INFILE' has been set

zuze11 · 11-23-2011, 10:47 AM

Quote:

Originally Posted by grail

Code:

if [[ -n "${INFILE:+x}" && -f ${INFILE} ]]; then # Fails if INFILE is unset or empty or does not exist

Not sure I understand your comment here? Both -n and -f seem to work fine for me whether INFILE is set or not?

This one checks whether INFILE is set, non-empty and the corresponding file exists. All conditions have to be present and it does fail if any one or more conditions is not met.

Quote:

Originally Posted by grail

Code:

if [ -n "${HUH+x}" ]; then # Fails if HUH is unset
     echo -e '\tIllegal use of additional flag'
     echo -e '\t\tWill ignore illegal flag and run script since INFILE has been correctly set!\n'
     display_usage
fi

The red highlighted section would seem to be a guess? Just because 'HUH' has been set does not mean 'INFILE' has been set

True, but if we made it so far into the script then according to the previous check INFILE has been indeed properly set.

I tested the script for all possible situations and it works as intended. Or maybe I misunderstood your comments?