LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-03-2008, 12:58 AM   #1
phillipseamore
LQ Newbie
 
Registered: Oct 2008
Posts: 2

Rep: Reputation: 0
Question Manipulating text


Hi all,

I'm in desperate need of reformatting an old text file with some 5300 lines of text. I think that awk or sed is the way to go but am getting confused.

The original format is like this:

Code:
in:  160.11
See our bus? It's really big.
out: 165.02
in:  165.12
-Is it the great big one here?
-That's right.
out: 171.03
...
But the new format needs to be like this:

Code:
@
160.11		165.02
See our bus? It's really big. 
@
165.12		171.03
-Is it the great big one here?
-That's right.
Anyone with a clue about how to proceed ?

Thanks,
Phil
 
Old 10-03-2008, 02:29 AM   #2
kenneho
Member
 
Registered: May 2003
Location: Oslo, Norway
Distribution: Ubuntu, Red Hat Enterprise Linux
Posts: 655

Rep: Reputation: 40
Hi.


I personally don't think sed it the way to go here. All you seem to be doing is moving text, not manupilating it much.

I haven't used awk much, so I don't have an opinion here. But a small shell or perl script would do the job quite easily. All you have to do i read a few lines at the time, store the lines or tokens in variables, and the write the stored variables to a file.
 
Old 10-03-2008, 04:08 AM   #3
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,240

Rep: Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324
Yeah, for re-formatting like that I'd use Perl. These links will help:
http://perldoc.perl.org/
http://www.perlmonks.org/?node=Tutorials
 
Old 10-03-2008, 04:19 AM   #4
phillipseamore
LQ Newbie
 
Registered: Oct 2008
Posts: 2

Original Poster
Rep: Reputation: 0
Thanks so much kenneho and chrism01. I feel like such an idiot now!

I managed to nail together a quick parser for this from PHP actually:

Code:
<?php

$handle = fopen("SUB.txt", "r");
$text = "";

while (!feof($handle)) {
    $buffer = fgets($handle, 4096);

		if (substr($buffer,0,3) == "in:") {
			echo "@\r";
			echo rtrim(substr($buffer,5,6),"\n");
		}
							

		if (substr($buffer,0,3) != "in:" && substr($buffer,0,4) != "out:" ) {
			$text .= $buffer; 
	  }

		if (substr($buffer,0,4) == "out:") {
		 	echo "\t\t";
			echo rtrim(substr($buffer,5,6),"\n");
			echo "\r";
			echo $text;
			$text = "";
		}

}
fclose ($handle);

?>
 
Old 10-03-2008, 10:45 PM   #5
MarkBurke
LQ Newbie
 
Registered: Nov 2003
Location: Orange County, CA, USA
Distribution: aix, rhel, ubuntu
Posts: 24

Rep: Reputation: 0
Smile awk attempt

1.awk
==
begin { INSIDE=NO ; }
/in:/ { INVALUE = $2 ; INSIDE=YES; }

!/in:/&&!/out:/ {
if (length(lines) > 0)
lines=sprintf("%s\n%s",lines,$0);
else lines=$0 ;
}

/out:/ {
OUTVALUE=$2 ;
printf("@\n%s\t\t%s\n",INVALUE,OUTVALUE);
print lines ; lines = "" ;
INSIDE=NO
}

end { print lines ; }
==

gawk -f 1.awk < 1.in


==
in:

in: 160.11
See our bus? It's really big.
out: 165.02
in: 165.12
-Is it the great big one here?
-That's right.
out: 171.03
...

out:

@
160.11 165.02
See our bus? It's really big.
@
165.12 171.03
-Is it the great big one here?
-That's right.
 
Old 10-03-2008, 11:37 PM   #6
Mr. C.
Senior Member
 
Registered: Jun 2008
Posts: 2,529

Rep: Reputation: 61
Here's a simpler version I think:

Code:
$ cat doit.pl
#!/usr/bin/perl

undef $/;
$_ = <>;

s/^in:[ \t]+(\d+\.\d+)\n(.*?)\n^out:[ \t]+(\d+\.\d+)\n/@\n\1      \3\n\2\n/gms;

print "$_";

$ ./doit.pl data
@
160.11      165.02
See our bus? It's really big.
@
165.12      171.03
-Is it the great big one here?
-That's right.
 
Old 10-05-2008, 01:51 PM   #7
archtoad6
Senior Member
 
Registered: Oct 2004
Location: Houston, TX (usa)
Distribution: MEPIS, Debian, Knoppix,
Posts: 4,727
Blog Entries: 15

Rep: Reputation: 233Reputation: 233Reputation: 233
If you don't mind preprocessing w/ sed , this is pretty short & sweet:
Code:
#! /bin/bash
cat $1                                 \
| sed -r 's,^out: *([^ ]*),@\1\n,                
   /^in/N;s,^in: *([^ ]+).*\n,\1\n@,'  \
| awk 'BEGIN{RS=""; FS="\n@"}; 
   {print "@","\n"$1,$3,"\n"$2}' 
exit  # end of script


## test output:   
@
160.11 165.02
See our bus? It's really big.
@
165.12 171.03
-Is it the great big one here?
-That's right.
I might add this is the 1st time I have seen a perl script of <100 lines that is shorter than my bash equivalent.

EDIT:
I spoke too soon -- here is an all sed script that is the shortest of all:
Code:
#!/bin/sed -rf
:n;N;s,out: *,@,;T n
s,in: *([^\n]*)(.*)\n@(.*),@\n\1 \3\2,
I don't know if this is good or bad -- we can only guess at the true format of the actual data -- but I don't assume that the "in:" & "out:" data will be digits. I could if needed.

Happy to answer any Q's about how it works.
(http://www.gnu.org/software/sed/manual/sed.html)

Last edited by archtoad6; 10-05-2008 at 04:10 PM. Reason: add more
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Manipulating Text File with awk or sed kushalkoolwal Programming 2 09-10-2008 08:35 PM
manipulating files tableyou Linux - Newbie 8 11-08-2005 03:38 AM
Editing/manipulating text files winchester169 Linux - Software 3 08-18-2005 07:01 PM
Manipulating NICs ? Infernal211283 Linux - Newbie 5 04-28-2005 12:46 PM
Manipulating Files in vi Baldorg Linux - General 1 09-19-2003 08:25 PM


All times are GMT -5. The time now is 06:55 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration