LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 11-15-2007, 08:54 PM   #1
lynx81191
LQ Newbie
 
Registered: Mar 2006
Location: Phoenix, Arizona
Distribution: Mandriva 2007
Posts: 8

Rep: Reputation: 0
Bash read in variable length text records


I need to be able to read in variable length text records on like a daily basis with out loosing track I know to use date and a file to store what record im on because the program will execute every day and need to process the next record in the file but how to determine what record? the field between records is going to be an extra line so what I have is a long text file with several pages worth of text and I have an additional line of white space separator so as to delineate records I would use while but the time frame for record processing needs to be a day and the variable record length creates a problem? Thanks in advance.

Last edited by lynx81191; 11-15-2007 at 08:55 PM.
 
Old 11-15-2007, 10:10 PM   #2
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,287

Rep: Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034
It's a little unclear what you are asking for, especially as you have no punctuation.
IUIC, each data rec is in fact 2 recs, one with data, one empty.
All you need to do is count which rec you have reached, and as you have noted, save that value for next time in a temp file.
Use cron to run the prog each day, which then reads the temp file and uses the rec cnt therein to read to that rec in the data file and extract the next data rec, incrementing the rec cnt and saving it.
HTH
 
Old 11-15-2007, 11:06 PM   #3
lynx81191
LQ Newbie
 
Registered: Mar 2006
Location: Phoenix, Arizona
Distribution: Mandriva 2007
Posts: 8

Original Poster
Rep: Reputation: 0
clarification

I apologize. Let me clarify. I have a 14 page long text flle spaced with an extra flle between records. The records consist of words of text separated by spaces. I need to pull records one at a time 1 per day and the only field delimiter I have to use is an extra blank line. I can write bash scripts but I have no idea how to process the records using the field delimiter that I have to work with. I've looked into regular expressions but am unsure what command to use and the exact structure of the regular expression that I would need. If use a 'for' loop with 'cut' I cannot use a multiple character field delimiter (there are spaces between the words in the records). And I can't use various other commands because of this so I am looking to sed and awk but have no knowledge of how to use a regular expression to accomplish this.
 
Old 11-15-2007, 11:31 PM   #4
lynx81191
LQ Newbie
 
Registered: Mar 2006
Location: Phoenix, Arizona
Distribution: Mandriva 2007
Posts: 8

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by chrism01 View Post
It's a little unclear what you are asking for, especially as you have no punctuation.
IUIC, each data rec is in fact 2 recs, one with data, one empty.
All you need to do is count which rec you have reached, and as you have noted, save that value for next time in a temp file.
Use cron to run the prog each day, which then reads the temp file and uses the rec cnt therein to read to that rec in the data file and extract the next data rec, incrementing the rec cnt and saving it.
HTH
#!/bin/bash
while read text
do
cat text | mail -s record user@domain.com
done < data

will not work because of delimiting fields ?

I apologize. Let me clarify. I have a 14 page long text file spaced with an extra blank line between records. The records consist of words of text separated by spaces. I need to pull records one at a time 1 per day and the only field delimiter I have to use is an extra blank line. I can write bash scripts but I have no idea how to process the records using the field delimiter that I have to work with. I've looked into regular expressions but am unsure what command to use and the exact structure of the regular expression that I would need. If use a 'for' loop with 'cut' I cannot use a multiple character field delimiter (there are spaces between the words in the records). And I can't use various other commands because of this so I am looking to sed and awk but have no knowledge of how to use a regular expression to accomplish this.

Last edited by lynx81191; 11-15-2007 at 11:40 PM.
 
Old 11-17-2007, 08:53 PM   #5
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,287

Rep: Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034
Here's the basics

Code:
file=yourfile
#you want eg rec 3 from yourfile - see my previous
num1=3    

#Get rec
rec=`cat $file|head -$num1|tail -1`

#awk will use/accept any amt of whitespace
field=`echo $rec|awk '{print $3}'`
echo $field
 
  


Reply

Tags
bash, file, processing, record, scripting, text, variable


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
bash read a given line number from text bendeco13 Programming 7 08-31-2012 03:49 PM
bash read files to variable question babag Programming 7 06-25-2007 05:04 AM
Bash store last line from displayed text output in a variable carl0ski Programming 1 01-16-2007 03:38 AM
Read variables from a text file in Bash jakev383 Linux - General 5 12-20-2006 07:29 AM
bash script that can read lines of text palceksmuk Programming 1 12-25-2005 03:49 AM


All times are GMT -5. The time now is 03:43 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration