LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 07-19-2006, 04:39 AM   #1
ERBRMN
Member
 
Registered: Mar 2005
Location: Japan
Distribution: TurboLinux, RHEL, SUSE
Posts: 96

Rep: Reputation: 15
How to split file , .. awk or split


Structure of my result file is following :
-----------
11
12
13
14
15
END
21
22
END
31
32
34
45
61
62
END

----------

How can I split it to three text file. File name is free, but indexed.
"END" is split string for files (end of each file).
I want to do it using AWK, but shell is OK.
Have you any idea ???

Last edited by ERBRMN; 07-19-2006 at 04:52 AM.
 
Old 07-19-2006, 05:01 AM   #2
spirit receiver
Member
 
Registered: May 2006
Location: Frankfurt, Germany
Distribution: SUSE 10.2
Posts: 424

Rep: Reputation: 33
The following will append the corresponding parts to file1, file2, ... . It will read the content from STDIN, so you may invoke it as "command < your_data_file" from the shell.
Code:
#!/bin/bash
FILENAME=file
INDEX=1
while read
do
 if [[ "$REPLY" = "END" ]]
 then
  (( INDEX++ ))
 else
  echo "$REPLY" >> $FILENAME$INDEX
 fi
done
 
Old 07-19-2006, 05:58 AM   #3
ERBRMN
Member
 
Registered: Mar 2005
Location: Japan
Distribution: TurboLinux, RHEL, SUSE
Posts: 96

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by spirit receiver
The following will append the corresponding parts to file1, file2, ... . It will read the content from STDIN, so you may invoke it as "command < your_data_file" from the shell.
Code:
#!/bin/bash
FILENAME=file
INDEX=1
while read
do
 if [[ "$REPLY" = "END" ]]
 then
  (( INDEX++ ))
 else
  echo "$REPLY" >> $FILENAME$INDEX
 fi
done

Thank you very much, spirit receiver

It was good job.
 
Old 07-20-2006, 10:32 PM   #4
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,245
Blog Entries: 15

Rep: Reputation: 233Reputation: 233Reputation: 233
Hello there. I still recommend spirit receiver's script the first but just in case you'll want to use this too.

command.sh file

Code:
#!/bin/bash
FILENAME="$1"
INDEX=1

IFS=$'\n'
for REPLY in $(<$FILENAME); do
 if [[ "$REPLY" = "END" ]]; then
  (( INDEX++ ))
 else
  echo "$REPLY" >> $FILENAME$INDEX
 fi
done

#IFS=$' \t\n'
regards
 
Old 07-20-2006, 11:56 PM   #5
ERBRMN
Member
 
Registered: Mar 2005
Location: Japan
Distribution: TurboLinux, RHEL, SUSE
Posts: 96

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by konsolebox
Hello there. I still recommend spirit receiver's script the first but just in case you'll want to use this too.

command.sh file

Code:
#!/bin/bash
FILENAME="$1"
INDEX=1

IFS=$'\n'
for REPLY in $(<$FILENAME); do
 if [[ "$REPLY" = "END" ]]; then
  (( INDEX++ ))
 else
  echo "$REPLY" >> $FILENAME$INDEX
 fi
done

#IFS=$' \t\n'
regards
Thk you, konsolebox

This is also good job.
 
Old 08-14-2006, 06:58 PM   #6
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,966
Blog Entries: 11

Rep: Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865
A tad late, and not quite perfect, but worth looking at :}

Code:
awk 'BEGIN{RS="END";OFS="\n"}{$1=$1; print > "myfile"NR}' test.txt
I haven't quite figured out why it will produce a 4th file (with only
and empty line in it) even though there's just three records.


Cheers,
Tink
 
Old 08-14-2006, 07:21 PM   #7
homey
Senior Member
 
Registered: Oct 2003
Posts: 3,057

Rep: Reputation: 56
Tinkster,
On my box, it only makes three files if there is no \n after the last END in the file.txt .
 
Old 08-14-2006, 08:07 PM   #8
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,966
Blog Entries: 11

Rep: Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865
Ok, looked at it again. Behaves correctly if I do RS="END\n"

Odd.



Cheers,
Tink

Last edited by Tinkster; 08-14-2006 at 08:09 PM.
 
Old 08-14-2006, 09:37 PM   #9
frob23
Senior Member
 
Registered: Jan 2004
Location: Roughly 29.467N / 81.206W
Distribution: Ubuntu, FreeBSD, NetBSD
Posts: 1,449

Rep: Reputation: 47
Code:
#!/bin/sh
index=1
filebase=foo

while read LINE
do
        if [ "${LINE}" = "END" ];
        then
                index=`expr ${index} + 1`
        else
                echo "${LINE}" >> ${filebase}.${index}
        fi
done
Variation of the first response but is bourne shell compatible (/bin/sh) so it's portable on systems where bash is not in the base. I love bash but, with something this simple, bash specific functionality is not required to reduce complexity and only frustrates portability. You can see it's not much different from the first reply; you just have to include a variable for the read and use expr (an external command) to do the math because there's no shell arithmatic in the original bourne shell.

Oh, and I named the files foo.# because that's my personal preference... easy to change if you want.

Can anyone tell I am bored?

Last edited by frob23; 08-14-2006 at 09:43 PM.
 
Old 08-15-2006, 12:02 AM   #10
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,966
Blog Entries: 11

Rep: Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865
Quote:
Originally Posted by Tinkster
Ok, looked at it again. Behaves correctly if I do RS="END\n"

Odd.
Actually, thinking about it some more (and I'm kind of slow due to
a flu :}) it's not that odd at all. If END is the delimiter, the \n does
comprise a 4th record that's unterminated.


Cheers,
Tink
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Split large file in several files using scripting (awk etc.) chipix Programming 14 10-29-2007 11:16 AM
split files using awk (or similar) lgualteri Programming 1 06-13-2005 09:17 AM
split up text file jollyjoice Programming 4 06-10-2005 03:33 PM
split one file to two alaios Linux - General 1 05-19-2005 10:24 PM
split file farkmischer Linux - Laptop and Netbook 3 11-28-2004 07:19 AM


All times are GMT -5. The time now is 01:36 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration