C binary files: Deleting part of the file and resizing
ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
C binary files: Deleting part of the file and resizing
Hi.
I'm trying to figure out how to delete part of a binary file and as a result have the file take up less space.
The reference I'm using writes a blank structure to the file in place of a deletion, this isn't what I want since the file remains the same size.
The way I can see C doing this is reading the entire file into memory by loading each structure into a list, removing the offending node and write the list back out to a new binary file. This will work however the file is large ( perhaps gigabytes ) and it'll be nice there was a way to copy the next structure ( which I can do ) over the offending one, right to left and remove the last sizeof( struct some_struct ) from the end of the file and reducing it's size that way because memory is tight.
I can't find any functions in C that can snip off the last part of the binary file. Do I need some sort of fancy system call to cut off the last few bytes of the file? I'm writing this for Solaris and Linux.
If the unwanted bytes are at the end, I think truncate or ftruncate (in unistd.h) would work. If it's somewhere in the middle of the file, I have no idea..
edit: and I don't know the portable version of truncate. anyone?
Last edited by abolishtheun; 03-26-2008 at 02:00 AM.
I found it in my UNIX programming book, it's truncate. I gotta shift all the valid structs right to left to remove all gaps in the file and snip off the end with truncate.
If it's in the middle, you can save on RAM by reading and writing one block/struct at a time, no need to read the whole thing into RAM at once.
If your disk is so full you don't (temporarily) have space for the new & old versions, I think you need a bigger disk.
I could be wrong, but even if you went down to the filesystem level, I don't think you could stitch 2 parts of a file together (if you know what I mean) and free up the bit in the 'middle'. The odds of your struct being exactly on an fs block boundary would be low.
Certainly not portable either...
Thats what I've done, fseek'd to the point of deletion and read the next struct back to the previous point and so on until i reach the end of the file, then I truncate the file. I use a total of 36 bytes in memory, the size of each record, while reducing the file size after the deletions. I can also perform all deletions and leave a marker and shift all the records across to eliminate all the gaps in the file, then truncate file sizie - 36 * num_of_deletions and this works well I think.
Location: Northeastern Michigan, where Carhartt is a Designer Label
Distribution: Slackware 32- & 64-bit Stable
Posts: 3,541
Rep:
Quote:
Originally Posted by knudfl
If I remember wright, the 'strip' command should remove blank areas,
or am I wrong ??
You're right (kind of) -- this is edited.
While stip operates on object files, deleting all symbols from them, generally making them significantly smaller, truncate() and ftruncat() operate on regular files and will either shorten a file or extend it depending upon the size of the file and the length of the argument passed to the function.
I blew my answer but it looks like the original problem got solved so I suppose all is well that ends.
While stip operates on object files, deleting all symbols from them, generally making them significantly smaller, truncate() and ftruncat() operate on regular files and will either shorten a file or extend it depending upon the size of the file and the length of the argument passed to the function.
The operation of truncate when used with a larger file size is undefined and is explicitly unreliable, but it happens to work in some cases. It does save a hell of a lot of time when creating, for example, a 2GB file to use as a file system image (vs. 12+ hours with dd) when it does work, though.
ta0kira
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.