LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices



Reply
 
Search this Thread
Old 11-17-2011, 04:12 PM   #1
zuze11
LQ Newbie
 
Registered: Sep 2011
Posts: 5

Rep: Reputation: Disabled
reading lines from file


Hi, trying to read data from a file with several blocks in it. Each block corresponds to a particular day and blocks are separated by empty lines. I managed to write a small and very simple bash script that selects the relevant lines and outputs them to separate files for each day.

Here is the script:

Code:
#! /bin/bash

DESCRIPTION='Date Time Ref avgU10 gstU10 dirU10 tC10 RH10 avgU4 avgU2 avgU1 tC0_5 avgU0_5 gstU0_5'

while read LINE
     do
       FILEID=${LINE:0:6}
       if [ ${LINE:0:1} == 9 ]; then
          if [ ! -f ${FILEID}.txt ]; then # if file does not exist, add description header
             echo ${DESCRIPTION} >> ${FILEID}.txt
             echo ${LINE} >> ${FILEID}.txt
          else
             echo ${LINE} >> ${FILEID}.txt
          fi
       fi
     done
It works simply by redirecting the data file into the script:

./script < datafile

I would like to modify it so I can specify the name of the data file to be processed within the script. Tried a bunch of different things I found on the net but nothing worked. Anybody knows a good way of doing it?

Thanks
 
Old 11-17-2011, 04:15 PM   #2
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Code:
while read LINE
do
  :
done < datafile
 
1 members found this post helpful.
Old 11-17-2011, 04:30 PM   #3
zuze11
LQ Newbie
 
Registered: Sep 2011
Posts: 5

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by colucix View Post
Code:
while read LINE
do
  :
done < datafile
So you basically move the redirect inside, to the end of the do construct. Very cool and works like a charm. Thanks!

Can you suggest a good bash reference with examples?
 
Old 11-17-2011, 04:37 PM   #4
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
You might check the Bash Guide for Beginners and then the Advanced Bash Scripting Guide, here. A very nice guide about linux command line and shell scripting is here.
 
Old 11-17-2011, 07:57 PM   #5
zuze11
LQ Newbie
 
Registered: Sep 2011
Posts: 5

Original Poster
Rep: Reputation: Disabled
Thanks
 
Old 11-17-2011, 09:13 PM   #6
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,698

Rep: Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988
Whilst the current example works the input file would always need to be of the same name and in the current directory.
In the sites suggested I would look for command line arguments / parameters so you could then use a complete path to any file.
 
Old 11-22-2011, 01:47 PM   #7
zuze11
LQ Newbie
 
Registered: Sep 2011
Posts: 5

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by grail View Post
Whilst the current example works the input file would always need to be of the same name and in the current directory.
In the sites suggested I would look for command line arguments / parameters so you could then use a complete path to any file.
Here is a further evolution of the script with the input file passed by the -i flag. It also checks for illegal flags and runs anyway as long as the input file is set and exists (although it still does not differentiate between these cases).

Still does not have error checking within the data being read but it's coming...

Code:
#! /bin/bash

# This script reads a NordlysStasjonen data acquisition file with data for several days
# and splits it into a series of daily measurement files 22/11/11

display_usage() {
        echo '####'
        echo 'USAGE: extractDays -i inputfile'
        echo 'where inputfile is the global NordlysStasjonen data acquisition file'
        echo '####'
        echo ''
        }

while getopts :i: option # flag construct with leading : for illegal flag detection
      do
        case "${option}"
        in
             i) INFILE=${OPTARG};;
             ?) HUH=${OPTARG};;
        esac
      done

if [[ -n "${INFILE:+x}" && -f ${INFILE} ]]; then # Fails if INFILE is unset or empty or does not exist
       echo -e '\n\t\tVariable INFILE has been set, and input file exists.'
     else
       echo -e '\n\t\tVariable INFILE is unset or empty, or input file does not exist.'
       echo -e '\t\tExiting!!!'
       display_usage
       exit 1
fi

if [ -n "${HUH+x}" ]; then # Fails if HUH is unset
     echo -e '\tIllegal use of additional flag'
     echo -e '\t\tWill ignore illegal flag and run script since INFILE has been correctly set!\n'
     display_usage
fi

# Initialize output file description header
DESCRIPTION='Date Time Ref avgU10 gstU10 dirU10 tC10 RH10 avgU4 avgU2 avgU1 tC0_5 avgU0_5 gstU0_5'

while read LINE
     do
       FILEID=${LINE:0:6}
       if [ ${LINE:0:1} == 9 ]; then
          if [ ! -f ${FILEID}.txt ]; then # if file does not exist, add description header
             echo ${DESCRIPTION} >> ${FILEID}.txt
             echo ${LINE} >> ${FILEID}.txt
          else
             echo ${LINE} >> ${FILEID}.txt
          fi
       fi
     done < ${INFILE}

echo 'Script execution complete. Exiting...'

exit 0
 
Old 11-22-2011, 11:54 PM   #8
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,698

Rep: Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988Reputation: 1988
Code:
if [[ -n "${INFILE:+x}" && -f ${INFILE} ]]; then # Fails if INFILE is unset or empty or does not exist
Not sure I understand your comment here? Both -n and -f seem to work fine for me whether INFILE is set or not?
Code:
if [ -n "${HUH+x}" ]; then # Fails if HUH is unset
     echo -e '\tIllegal use of additional flag'
     echo -e '\t\tWill ignore illegal flag and run script since INFILE has been correctly set!\n'
     display_usage
fi
The red highlighted section would seem to be a guess? Just because 'HUH' has been set does not mean 'INFILE' has been set
 
Old 11-23-2011, 11:47 AM   #9
zuze11
LQ Newbie
 
Registered: Sep 2011
Posts: 5

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by grail View Post
Code:
if [[ -n "${INFILE:+x}" && -f ${INFILE} ]]; then # Fails if INFILE is unset or empty or does not exist
Not sure I understand your comment here? Both -n and -f seem to work fine for me whether INFILE is set or not?
This one checks whether INFILE is set, non-empty and the corresponding file exists. All conditions have to be present and it does fail if any one or more conditions is not met.

Quote:
Originally Posted by grail View Post
Code:
if [ -n "${HUH+x}" ]; then # Fails if HUH is unset
     echo -e '\tIllegal use of additional flag'
     echo -e '\t\tWill ignore illegal flag and run script since INFILE has been correctly set!\n'
     display_usage
fi
The red highlighted section would seem to be a guess? Just because 'HUH' has been set does not mean 'INFILE' has been set
True, but if we made it so far into the script then according to the previous check INFILE has been indeed properly set.

I tested the script for all possible situations and it works as intended. Or maybe I misunderstood your comments?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
While reading lines of a file do this.. plax65 Linux - Newbie 4 11-13-2011 01:07 PM
Reading lines within a file (Perl) jack.barnes Programming 12 03-16-2011 12:09 PM
Reading specific lines from a file Mork2k4 Programming 2 09-11-2009 12:51 AM
PHP - Help With Reading Lines In A File windisch Programming 6 05-03-2006 01:48 PM
Reading the number of Lines in a File Mistro116@yahoo.com Programming 31 11-24-2005 01:36 AM


All times are GMT -5. The time now is 11:11 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration