LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices



Reply
 
Search this Thread
Old 11-22-2006, 07:33 PM   #1
cslink23
LQ Newbie
 
Registered: Nov 2006
Posts: 10

Rep: Reputation: 0
Help w/ script to read file and parse variables


I need to split a file into 3 smaller files. Specifically,
I'm trying to write a shell (bash) script to:
1. read a file
2. parse the variables in it
3. depending on the value of the 3rd field write the record to 1 of three files.

Here's the input data:
"0123456789",1," 2"
"0123456789",1," 1"
"0123456789",1," 3"
...

I'll ask the user for the input file name. The output files will have the same name as the input with a number added. The number comes from the last field in the input file (ie. 1,2,3). And, the records will be written to each output file based on the value of that last field.

Here's what I'm working with (piecing together from other web samples):

#!/bin/bash
echo -n "Please enter the IQ filename:"
read filename
# test filename and file exists

# loop through input file
branch=`echo $inputline | cut -d"," -f 3`
branch=`awk -F , '{ print $3 }' $inputline`
while branch=`cat $filename | awk -F, '{ print $3 }' $filename`
Note: None of these worked for me BUT I may be mixing recipes.

#while branch=`cat $inputline | cut -d, -f3`

do
case "$branch" in
" 1")
# write " 1" records to output.1
# count output records so I'll know input matches output.
;;
" 2")
# write " 2" records to output.2
;;
" 3")
# write " 3" records to output.3
;;
*)
echo "Bad Branch $branch"
;;
esac
done
exit 0

Thanks for your help.
Link
 
Old 11-22-2006, 08:06 PM   #2
Sepero
Member
 
Registered: Jul 2004
Location: Tampa, Florida, USA
Distribution: Ubuntu
Posts: 734
Blog Entries: 1

Rep: Reputation: 31
I don't know how to use awk, but I can see some obvious flaws in your script. First, you are trying to use $inputline, and you haven't even defined it. Second, you define $branch, but then you immediately define it again, making the first define useless. Then you define $branch a third time in your loop, making the second define useless.
 
Old 11-22-2006, 08:15 PM   #3
cslink23
LQ Newbie
 
Registered: Nov 2006
Posts: 10

Original Poster
Rep: Reputation: 0
I assumed that it would be viewed as the types of things I've tried. These lines are out of context since they would only "work" with the proper corresponding "while" loops. The whole script is just meant to be pseudocode. Thanks for the reply, though.
 
Old 11-23-2006, 02:32 AM   #4
Sepero
Member
 
Registered: Jul 2004
Location: Tampa, Florida, USA
Distribution: Ubuntu
Posts: 734
Blog Entries: 1

Rep: Reputation: 31
Done. Please pay all thanks to the GNU licenses. Without them, Linux could not exist.
EDIT: Oh yes, I did test this code before posting it.
Code:
#!/bin/bash

#Global count variable for counting lines output.
count=0

if [ -f output.1 -o -f output.2 -o -f output.3 ];
then
  echo "One or all output files already exist. Please move or delete them."
  exit 1
fi

echo -n "Please enter the IQ filename: "
filename="test.dat"
echo $filename
#read filename

# test filename and file exists
if [ ! -f "$filename" ];
then
  echo "Sorry, that file does not exist. Please make sure the name is correct."
  exit 1
fi

# loop through input file
while read inputline;
do
  branch=$(echo "$inputline" | cut -d "," -f 3)
  case "$branch" in
    '" 1"')
      echo "$inputline" >> output.1
      let count=count+1
      ;;
    '" 2"')
      echo "$inputline" >> output.2
      let count=count+1
      ;;
    '" 3"')
      echo "$inputline" >> output.3
      let count=count+1
      ;;
    *)
      echo "Bad Branch: $branch"
      let count=count+1
      ;;
esac
done < "$filename"

echo "$count lines processed."

Last edited by Sepero; 11-23-2006 at 02:34 AM.
 
Old 11-23-2006, 10:33 AM   #5
cslink23
LQ Newbie
 
Registered: Nov 2006
Posts: 10

Original Poster
Rep: Reputation: 0
Sepero -

Thank you so much for putting this together for me. The line:
branch=$(echo "$inputline" | cut -d "," -f 3)
is giving me a syntax error - "syntax error at line 27: `branch=$' unexpected"

Is it possible that my version of /bin/sh is too old? I don't know how to tell the version. This is a Point of Sale server that is supported by another company. If there is a way to update it (and maybe that would be for another forum) I can give it a try.

But, for simplicity, do you know of any way to modify that one statement to work around this problem?

Thanks
 
Old 11-23-2006, 07:41 PM   #6
Sepero
Member
 
Registered: Jul 2004
Location: Tampa, Florida, USA
Distribution: Ubuntu
Posts: 734
Blog Entries: 1

Rep: Reputation: 31
Try the backticks, like you had in your original. I believe backticks are generally a bad way to go, because someday you might need to use them inside your statement.
Code:
branch=`echo "$inputline" | cut -d "," -f 3`
 
Old 11-23-2006, 07:53 PM   #7
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 63
It seems very strange that bash should be used for this sort of task - I would think perl or awk would be much more suitable.

Perhaps a Haiku can help to answer the question why...
Code:
task to do with bash
it is inappropriate
must be a homework
 
Old 11-23-2006, 08:38 PM   #8
Sepero
Member
 
Registered: Jul 2004
Location: Tampa, Florida, USA
Distribution: Ubuntu
Posts: 734
Blog Entries: 1

Rep: Reputation: 31
I agree with you mathew. Especially when I noticed that the script is to output to 3 separate files. Since it was a short script (pretty much a hack), I figured I'd test my Bash skills a little bit. (sort of a self-challenge. )
 
Old 11-23-2006, 09:50 PM   #9
cslink23
LQ Newbie
 
Registered: Nov 2006
Posts: 10

Original Poster
Rep: Reputation: 0
OK - Now that's nice. One more problem (for now)...the "let" statement doesn't work.
"link2.bat: let: not found"

Is there another way to verify all records are accounted for?

Signed, picky
 
Old 11-23-2006, 10:23 PM   #10
Sepero
Member
 
Registered: Jul 2004
Location: Tampa, Florida, USA
Distribution: Ubuntu
Posts: 734
Blog Entries: 1

Rep: Reputation: 31
I'm not at home right now, so I can't test this, but you could try:
Code:
count=$(($count + 1))
or perhaps
count=$((count + 1))
It seems awful strange that your shell doesn't support these things. I'm guessing that you're either using a really old version of bash, or you're using a shell that is trying to emulate bash. My bet is on the latter.
 
Old 11-24-2006, 05:52 PM   #11
cslink23
LQ Newbie
 
Registered: Nov 2006
Posts: 10

Original Poster
Rep: Reputation: 0
I have no way to convince you that this is not homework. But, I can explain its use. Our Point of Sale server (SCO Unix) has a piece of software called IQ used to extract data from a UniBASIC database. We run several queries using IQ to extract files which are used to create purchase orders. The problem is that we have to run this IQ 3 times - once for each store - generating 3 different files. Each run takes over 1 hour. MY GOAL: I'd like to run this program only once (1 hour) to create a file containing data for all three stores then split the data into 3 files in a script (or whatever works).

I do know some PERL and could probably get it done in that but we don't have PERL on this server and I can't add it and keep our support contract. Also, you'll remember that in one of my posts above, I mentioned that my version of /bin/sh may be old. I can tell from my /etc/passwd that /bin/sh is what is being used - not bash. But, I don't know how to check the version of /bin/sh. If you know how please tell me and I'll report that to you.

I tried your last suggestion for increasing $count and received this message for both formats:
syntax error at line 35: `count=$' unexpected

I appreciate your continued help and assure you that it will be greatly appreciated.

Link
 
Old 11-24-2006, 05:56 PM   #12
cslink23
LQ Newbie
 
Registered: Nov 2006
Posts: 10

Original Poster
Rep: Reputation: 0
BTW, I could probably live with this and just add some "wc -l" statements to check the file sizes. But, that's a little less tidy.
 
Old 11-24-2006, 09:59 PM   #13
Sepero
Member
 
Registered: Jul 2004
Location: Tampa, Florida, USA
Distribution: Ubuntu
Posts: 734
Blog Entries: 1

Rep: Reputation: 31
Ok, so it's not bash. What shell is it?
If I know, I might be able to figure out how to do math with it.
 
Old 11-24-2006, 10:01 PM   #14
cslink23
LQ Newbie
 
Registered: Nov 2006
Posts: 10

Original Poster
Rep: Reputation: 0
The only clue I have is that in the /etc/passwd file the shell assigned to each user is "/bin/sh". I have no idea how to find out further. Please tell me where to look to find out the shell and the version (if you know).

Thanks.
 
Old 11-25-2006, 03:33 AM   #15
Sepero
Member
 
Registered: Jul 2004
Location: Tampa, Florida, USA
Distribution: Ubuntu
Posts: 734
Blog Entries: 1

Rep: Reputation: 31
whereis sh
Then when you find sh. Type this:
ls -l [location of sh]

I think the second command should tell you which shell you have.
 
  


Reply

Tags
awk, bash, script


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
perl script to parse this file ohcarol Programming 10 11-02-2006 10:50 AM
daunting task - read wml input, insert variables into URL, DL page, parse, write file jeffreybluml Programming 1 05-12-2005 07:31 AM
Need help with perl/bash script to parse PicBasic file cmfarley19 Programming 13 11-18-2004 06:06 PM
use php script to parse a file. blackzone Linux - Software 1 07-07-2004 05:43 AM
How to read ans parse MS word file using a Linux Shell script. Alek Linux - General 2 11-10-2003 03:07 PM


All times are GMT -5. The time now is 10:27 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration