LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 09-22-2004, 08:38 AM   #1
xmeson
Member
 
Registered: Jul 2003
Location: Random
Distribution: Slackware
Posts: 63

Rep: Reputation: 15
Header Removal


Hello,

Now i am doing some data analysis and i cannot help but generate about 369.5MB of data every 20seconds (luckily my experiments last only 60seconds), but the mess is that i stream this data to my analysis software - propriety(mathematica), and the software that generates this data also propriety has some header tags that mathematica does not like.

So i open this with a text editor but most times it simply crahes and does not complete the Job, does anyone know an efficient way of removing this tags? I mean simply open the "top" of the file and take out the tags?

Thanks,

xmeson
 
Old 09-22-2004, 09:05 PM   #2
wipe
Member
 
Registered: Jun 2004
Location: High Green
Distribution: Fedora Core 4
Posts: 180

Rep: Reputation: 30
Your files seem quite large, so I don't know what would be efficient. If you wish to remove N bytes from the beginning of a file, you can use tail. The command might look like this: tail -c +100 FILE1 > FILE2 (this would remove 99 bytes and create a second file).

If editing a large file crashes your editors, you can split the file into smaller chunks, edit the first one and merge them again. This: split -b 100m FILE chunk would create 100 Meg files called chunkaa, chunkab and so on. They could be merged with cat chunkaa chunkab chunkac > FILE. To get one small chunk and one big chunk, use head to extract data from the beginning and tail to get the rest, but be careful not to corrupt the data by extracting overlapping sections or leaving out a middle part.

Simple scripts can be written to edit files or data streams if the format is known. I use sed for simple tasks and perl for more complicated ones. Read the man and info pages for more information.

Regards
Simon
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
removal invinciblegod Linux - Laptop and Netbook 3 06-06-2005 03:47 PM
c header files in linux in place of header files in windows? harun_acs Programming 1 03-17-2004 02:24 AM
virus removal Kent Emia Linux - General 3 01-30-2004 03:13 AM
Mesa3d removal jfrey Linux - Software 0 05-24-2003 12:38 AM
Lilio Removal???? fhbooker Linux - Newbie 3 05-15-2002 12:14 PM


All times are GMT -5. The time now is 08:59 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration