LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 04-10-2006, 01:47 AM   #1
Deepak Inbasekaran
Member
 
Registered: Apr 2006
Location: India
Distribution: Red Hat Linux release 9 (Shrike)
Posts: 44

Rep: Reputation: 15
Ftruncate


is ftruncate used to truncate the file at the end only ?? i need to remove/truncate the top of my file as i keep appedning content at the bottom , how do i do it ?
 
Old 04-10-2006, 03:40 AM   #2
Flesym
Member
 
Registered: Aug 2005
Location: Germany
Distribution: Ubuntu, Debian
Posts: 189

Rep: Reputation: 31
No, you cannot remove a random line from a file. You can only append to or truncate the end, as well as overwriting existing data. If you want to remove/insert data at a random position, you have to copy the file into a buffer, modify that buffer and copy it back to the file. No way around, sorry.
 
Old 04-10-2006, 04:53 AM   #3
Deepak Inbasekaran
Member
 
Registered: Apr 2006
Location: India
Distribution: Red Hat Linux release 9 (Shrike)
Posts: 44

Original Poster
Rep: Reputation: 15
ok random line not possible but wat about FIRST LINE ? cant i remove a record from the first line of the file ?? or the first record ???
 
Old 04-10-2006, 05:18 AM   #4
Flesym
Member
 
Registered: Aug 2005
Location: Germany
Distribution: Ubuntu, Debian
Posts: 189

Rep: Reputation: 31
What I said above is also true for the first line. But it really isn't very hard to implement a short function that copies the file into a buffer, removes the first line and copy it back. Another way would be to use existing (higher level) functions to strip that line away. An example that uses the *NIX command 'sed' would be:
Code:
sed -i '1d' theFile.txt
In C you can call this command through the 'system()' function:
Code:
system("sed -i '1d' theFile.txt")

Last edited by Flesym; 04-10-2006 at 05:19 AM.
 
Old 04-10-2006, 05:47 AM   #5
Deepak Inbasekaran
Member
 
Registered: Apr 2006
Location: India
Distribution: Red Hat Linux release 9 (Shrike)
Posts: 44

Original Poster
Rep: Reputation: 15
hey thanx a lot , this is sort of thing i wanted now nearly my prob is solved, am able to remove the first line and do insertions at the end but the owner of the file has used the "NEWLINE" character as the delimiter for each message that is to be stored, so is it possivle to delete based on delimiter??
 
Old 04-10-2006, 07:01 AM   #6
Deepak Inbasekaran
Member
 
Registered: Apr 2006
Location: India
Distribution: Red Hat Linux release 9 (Shrike)
Posts: 44

Original Poster
Rep: Reputation: 15
is it possible to have the '\n' delimiter to remove the content till the delimiter using this sed ? or any other system call ??
 
Old 04-10-2006, 07:16 AM   #7
primo
Member
 
Registered: Jun 2005
Posts: 542

Rep: Reputation: 34
It can be done with a linked list with lines as elements. You may keep adding data to the end of the file and update a pointer to the last byte of data in the file to discard. Then you may use a combination of mmap() and memmove() to move this data to the top and ftruncate() it.
 
Old 04-11-2006, 01:08 AM   #8
Deepak Inbasekaran
Member
 
Registered: Apr 2006
Location: India
Distribution: Red Hat Linux release 9 (Shrike)
Posts: 44

Original Poster
Rep: Reputation: 15
matter is my file size is 1MB so when it reaches tat limit i got to copy them up to a buffer and do the above said mmap memmove and remove first record and use msync to write it back to file, now for every write i got to repeat this 1MB copy to buffer, record removal and write back to file - it sounds a bit too tedious isnt it ?
 
Old 04-11-2006, 02:53 AM   #9
primo
Member
 
Registered: Jun 2005
Posts: 542

Rep: Reputation: 34
Quote:
Originally Posted by Deepak Inbasekaran
matter is my file size is 1MB so when it reaches tat limit i got to copy them up to a buffer and do the above said mmap memmove and remove first record and use msync to write it back to file, now for every write i got to repeat this 1MB copy to buffer, record removal and write back to file - it sounds a bit too tedious isnt it ?
With careful coding, only one mmap() call is needed. With 1MB, why a file is needed anyway ?
 
Old 04-11-2006, 03:23 AM   #10
muha
Member
 
Registered: Nov 2005
Distribution: xubuntu, grml
Posts: 451

Rep: Reputation: 38
Quote:
Originally Posted by Deepak Inbasekaran
is it possible to have the '\n' delimiter to remove the content till the delimiter using this sed ? or any other system call ??
What are you trying to achieve then? Sed handles files per line, so if you remove till the delimiter, you're removing everything on a line except the \n which doesn't make sense to me. It looks like you want to search for empty lines: ^$
Also check out some sed links: http://bookmarks.linuxquestions.org/.../one-liner+sed
 
Old 04-11-2006, 05:26 AM   #11
Deepak Inbasekaran
Member
 
Registered: Apr 2006
Location: India
Distribution: Red Hat Linux release 9 (Shrike)
Posts: 44

Original Poster
Rep: Reputation: 15
those are actually log files each of size 1MB, and i need to keep logging my error msgs from my different processes and i must be able to retain around 1 MB of data all the time, so as i keep adding i need to delete the old messages so tat there is 1MB of data all the time.tats y now immap this 1MB and delete the first record by memove and then use msync to write it back to the file, matter is i got to do this procedure everytime a message comes in after reaching the 1MB limit, am i anywhere near the solution ?? any other alternatives ?
 
Old 04-11-2006, 05:29 AM   #12
Deepak Inbasekaran
Member
 
Registered: Apr 2006
Location: India
Distribution: Red Hat Linux release 9 (Shrike)
Posts: 44

Original Poster
Rep: Reputation: 15
my other doubt is : now tat my file reaches its full size of 1MB if i use
Quote:
system("sed -i '\n d' filename.txt")
each line will be deleted right ?? does tat mean - when first record is deleted the second one becomes first and then the new record cud be inserted at the end ? or will the place of the deleted record be occupied by a space ?
 
Old 04-11-2006, 05:56 AM   #13
muha
Member
 
Registered: Nov 2005
Distribution: xubuntu, grml
Posts: 451

Rep: Reputation: 38
Use
Code:
sed -i '1d' theFile.txt
to delete the first line and save to the original file.

Last edited by muha; 04-11-2006 at 05:58 AM.
 
Old 04-11-2006, 06:07 AM   #14
Flesym
Member
 
Registered: Aug 2005
Location: Germany
Distribution: Ubuntu, Debian
Posts: 189

Rep: Reputation: 31
Code:
sed -i '\n d' filename.txt
No, this isn't a valid sed command. Have a look at its man-page or the link muha posted to learn how to use it. To delete the first line, use the command I already posted; it will delete to the first '\n' (including):
Code:
sed -i '1d' theFile.txt
So when your file reaches 1MB call this command, then reopen the the file for appending and write the next record to its end. But: This method should only be used, if a new log entry is a relatively rare situation, because this is a quite expensive method (sed has to buffer the whole file every time).

Another more efficient way would be to use two (or even more) files: Begin by filling the first file until it reaches 1MB; then open the second file and keep on writing to this until it is full. Now blank the first file again and log to this for the next 1MB and so on... This way you can review at least your desired last 1MB and won't have to worry about performance, because you only append and truncate (both are cheap file operations), so there is no buffer and no additional string operation needed. But of course this has to fit into your software design.
 
Old 04-11-2006, 06:09 AM   #15
Deepak Inbasekaran
Member
 
Registered: Apr 2006
Location: India
Distribution: Red Hat Linux release 9 (Shrike)
Posts: 44

Original Poster
Rep: Reputation: 15
ya this
Quote:
1d
deletes jus one line , but i need to delete FROM first line till the delimiter "\n" is reached . let me give a sample

this is how my file looks
Quote:
timestamp1<space>id<space>message1...........<space>\n
timestamp2<space>id<space>message2.............................
...............................<space>\n
timestamp3<space>id<space>message3...........<space>\n
timestamp4<space>id<space>message4...........<space>\n
timestamp5<space>id<space>message5.............................
...............................<space>\n

so if i delete 1d not always the entire msg gets deleted as there are multiline msgs
 
  


Reply

Tags
sed



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 06:14 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration