LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 07-28-2005, 01:21 PM   #1
joelhop
Member
 
Registered: Mar 2004
Location: Pennsylvania::USA
Distribution: Fedora Core 6
Posts: 100

Rep: Reputation: 15
Talking Data Processing


Hey Gang,

I got a pretty quick question. I have a file where blank lines are being used as a place marker, so i cannot remove them:

ex:

train





school" "business
truck


plant



wagon

I would like to put a switch in front of only non- blank lines, while leaving blank lines intact:

ex:


-a train





-a school" "business
-a truck


-a plant



-a wagon


I'm looking for the most simple solution, I am not looking to complicate the script too much as efficiency is a prerequisite.


Thanks KARL
 
Old 07-28-2005, 02:03 PM   #2
joelhop
Member
 
Registered: Mar 2004
Location: Pennsylvania::USA
Distribution: Fedora Core 6
Posts: 100

Original Poster
Rep: Reputation: 15
Thumbs up additionally

additionally, all non- blank lines do contain at least 1 period.
 
Old 07-28-2005, 02:28 PM   #3
demian
Member
 
Registered: Apr 2001
Location: Bremen, Germany
Distribution: Debian
Posts: 303

Rep: Reputation: 30
awk '{if(NF!=0)print "-a",$0; else print $0}' infile
 
Old 07-28-2005, 05:02 PM   #4
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,659
Blog Entries: 4

Rep: Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940
For almost any sort of task involving line-oriented data, awk is your best friend. Definitely an important tool to know.
 
Old 08-01-2005, 01:45 PM   #5
joelhop
Member
 
Registered: Mar 2004
Location: Pennsylvania::USA
Distribution: Fedora Core 6
Posts: 100

Original Poster
Rep: Reputation: 15
Post Solution

Hey Gang,

This is the solution I ended up going with:
#########################################

file=data.txt #sets variable file to data.txt

numlines=`wc -l $file | awk '{ print $1 }'` #sets $numlines to the number of lines in $file

r=0 #sets $r to 0

#########################################

while [ $r -le $numlines ]; do #runs loop as many times as there are lines in the file

line=`cat $file | head -$r | tail -1` # $line is equal to the data on the particular line we
## are on in the loop

if [[ $line = *.* ]] #if $line contains anything than a period than anything else
then
echo "-a" $line >> goodfile.txt # prints -a plus the data previous on that line
else
echo >> goodfile.txt # if it does not contain anything a period anything else prints a
## blank line to goodfile.txt
fi

r=`expr $r + 1` #adds 1 to $r
done

###########################################

Any questions feel free to mail.

Thanks for all the help,

-Karl
 
Old 08-01-2005, 01:52 PM   #6
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,659
Blog Entries: 4

Rep: Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940Reputation: 3940
It will work of course... but you could have done it all with awk. I would encourage you to read-up on it and see how the tool could have been applied to the problem.
 
Old 08-02-2005, 10:17 AM   #7
demian
Member
 
Registered: Apr 2001
Location: Bremen, Germany
Distribution: Debian
Posts: 303

Rep: Reputation: 30
*hmmms*
Code:
luna:~/tmp demian$ wc -l data.txt
    3200 data.txt
luna:~/tmp demian$ time ./yourscript.sh
head: illegal line count -- 0

real    3m50.525s
user    0m22.550s
sys     1m30.890s

luna:~/tmp demian$ time awk '{if(NF!=0)print "-a",$0; else print $0}' data.txt > goodfile2.txt

real    0m0.098s
user    0m0.014s
sys     0m0.012s

luna:~/tmp demian$ diff goodfile.txt goodfile2.txt
luna:~/tmp demian$
 
Old 01-01-2006, 07:19 PM   #8
joelhop
Member
 
Registered: Mar 2004
Location: Pennsylvania::USA
Distribution: Fedora Core 6
Posts: 100

Original Poster
Rep: Reputation: 15
I did use awk, where needed: numlines=`wc -l $file | awk '{ print $1 }'`
That was all it was needed for, the rest was processed through a while loop. ;p
 
Old 01-01-2006, 08:08 PM   #9
btmiller
Senior Member
 
Registered: May 2004
Location: In the DC 'burbs
Distribution: Arch, Scientific Linux, Debian, Ubuntu
Posts: 4,290

Rep: Reputation: 378Reputation: 378Reputation: 378Reputation: 378
Another possibility would have been to use sed -- something like:

Code:
sed 's/^\(.*\..*\)/-a \1/g'
ought to work if each line you want prefixed has a period in it.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Distributive Processing? ircmaxell Linux - General 2 09-15-2005 12:40 AM
Data Processing Server peter72 Linux - Software 1 06-14-2005 11:17 AM
Home Office Biotech Data Mining - Data Collection Adler Linux - General 20 11-03-2004 04:17 AM
Burn Data DVD... Read Data in Linux and Windows SaintStrive Linux - Newbie 3 09-18-2004 05:04 PM
need help on processing large data files eph Programming 3 03-11-2004 04:56 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 04:14 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration