LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 03-26-2008, 01:22 AM   #1
xemous
Member
 
Registered: Jun 2004
Location: Australia, Melbourne.
Distribution: Ubuntu
Posts: 80

Rep: Reputation: 15
C binary files: Deleting part of the file and resizing


Hi.

I'm trying to figure out how to delete part of a binary file and as a result have the file take up less space.

The reference I'm using writes a blank structure to the file in place of a deletion, this isn't what I want since the file remains the same size.

The way I can see C doing this is reading the entire file into memory by loading each structure into a list, removing the offending node and write the list back out to a new binary file. This will work however the file is large ( perhaps gigabytes ) and it'll be nice there was a way to copy the next structure ( which I can do ) over the offending one, right to left and remove the last sizeof( struct some_struct ) from the end of the file and reducing it's size that way because memory is tight.

I can't find any functions in C that can snip off the last part of the binary file. Do I need some sort of fancy system call to cut off the last few bytes of the file? I'm writing this for Solaris and Linux.



Last edited by xemous; 03-26-2008 at 01:28 AM.
 
Old 03-26-2008, 01:52 AM   #2
abolishtheun
Member
 
Registered: Mar 2008
Posts: 183

Rep: Reputation: 31
If the unwanted bytes are at the end, I think truncate or ftruncate (in unistd.h) would work. If it's somewhere in the middle of the file, I have no idea..
edit: and I don't know the portable version of truncate. anyone?

Last edited by abolishtheun; 03-26-2008 at 02:00 AM.
 
Old 03-26-2008, 01:59 AM   #3
xemous
Member
 
Registered: Jun 2004
Location: Australia, Melbourne.
Distribution: Ubuntu
Posts: 80

Original Poster
Rep: Reputation: 15
I found it in my UNIX programming book, it's truncate. I gotta shift all the valid structs right to left to remove all gaps in the file and snip off the end with truncate.

Thank ye good sir
 
Old 03-26-2008, 02:12 AM   #4
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,251

Rep: Reputation: 2026Reputation: 2026Reputation: 2026Reputation: 2026Reputation: 2026Reputation: 2026Reputation: 2026Reputation: 2026Reputation: 2026Reputation: 2026Reputation: 2026
If it's in the middle, you can save on RAM by reading and writing one block/struct at a time, no need to read the whole thing into RAM at once.
If your disk is so full you don't (temporarily) have space for the new & old versions, I think you need a bigger disk.
I could be wrong, but even if you went down to the filesystem level, I don't think you could stitch 2 parts of a file together (if you know what I mean) and free up the bit in the 'middle'. The odds of your struct being exactly on an fs block boundary would be low.
Certainly not portable either...
 
Old 03-26-2008, 04:12 AM   #5
xemous
Member
 
Registered: Jun 2004
Location: Australia, Melbourne.
Distribution: Ubuntu
Posts: 80

Original Poster
Rep: Reputation: 15
Thats what I've done, fseek'd to the point of deletion and read the next struct back to the previous point and so on until i reach the end of the file, then I truncate the file. I use a total of 36 bytes in memory, the size of each record, while reducing the file size after the deletions. I can also perform all deletions and leave a marker and shift all the records across to eliminate all the gaps in the file, then truncate file sizie - 36 * num_of_deletions and this works well I think.
 
Old 03-26-2008, 01:31 PM   #6
knudfl
LQ 5k Club
 
Registered: Jan 2008
Location: Copenhagen, DK
Distribution: pclos2014, Slack14.1 DebWheezy, +50+ other Linux OS, for test only.
Posts: 13,522

Rep: Reputation: 2503Reputation: 2503Reputation: 2503Reputation: 2503Reputation: 2503Reputation: 2503Reputation: 2503Reputation: 2503Reputation: 2503Reputation: 2503Reputation: 2503
If I remember wright, the 'strip' command should remove blank areas,
or am I wrong ??
 
Old 03-26-2008, 02:28 PM   #7
tronayne
Senior Member
 
Registered: Oct 2003
Location: Northeastern Michigan, where Carhartt is a Designer Label
Distribution: Slackware 32- & 64-bit Stable
Posts: 3,011

Rep: Reputation: 742Reputation: 742Reputation: 742Reputation: 742Reputation: 742Reputation: 742Reputation: 742
Quote:
Originally Posted by knudfl View Post
If I remember wright, the 'strip' command should remove blank areas,
or am I wrong ??
You're right (kind of) -- this is edited.

While stip operates on object files, deleting all symbols from them, generally making them significantly smaller, truncate() and ftruncat() operate on regular files and will either shorten a file or extend it depending upon the size of the file and the length of the argument passed to the function.

I blew my answer but it looks like the original problem got solved so I suppose all is well that ends.

Last edited by tronayne; 03-27-2008 at 07:24 AM.
 
Old 03-26-2008, 08:10 PM   #8
abolishtheun
Member
 
Registered: Mar 2008
Posts: 183

Rep: Reputation: 31
???

And which implementation of strip removes 'blank areas'?
 
Old 03-27-2008, 08:49 AM   #9
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
Quote:
Originally Posted by tronayne View Post
While stip operates on object files, deleting all symbols from them, generally making them significantly smaller, truncate() and ftruncat() operate on regular files and will either shorten a file or extend it depending upon the size of the file and the length of the argument passed to the function.
The operation of truncate when used with a larger file size is undefined and is explicitly unreliable, but it happens to work in some cases. It does save a hell of a lot of time when creating, for example, a 2GB file to use as a file system image (vs. 12+ hours with dd) when it does work, though.
ta0kira
 
  


Reply

Tags
binary, delete, file, resize


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Practical Steganography Part-1: Hiding information in Binary Executable File LXer Syndicated Linux News 0 02-14-2008 11:41 AM
Deleting files particular type of file using rm? vxc69 Linux - General 3 05-11-2007 09:47 PM
Deleting files after x days from the latest date file sriramsreedhars Programming 8 03-03-2007 09:30 AM
Replace Part of Binary File with 0s hansnueski Linux - General 2 06-14-2006 01:11 PM
Changing File Permission in a FTP directory to prevent deleting of files shawnbishop Linux - Software 3 01-10-2006 05:41 AM


All times are GMT -5. The time now is 07:33 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration