Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I have been using the same bash script for archiving and backing up data for several years. I am getting concerned about possible data corruption in my .tar.bz2 backups because the size of them has grown immensely.
My bash script does the following:
1. makes a .tar.bz2 of each major data directory we have in the office.
2. then takes each .tar.bz2 and places it into an .iso file that can be mounted or burned to a DVD.
Each major directory holds a little over 5 gigs. of data.
When each is compressed into the .tar.bz2 file they are about 1.2-1.4 Gigs. in size.
I've done some googling on the subject and I'm not finding much empirical data about the limits of bzip2 files and potential data corruption or loss.
Does anyone have experiences with this issue?
And can you provide some good links for some worthwhile RTFM. :-)
I personally have not seen any data, but depending on what goes into the bzip2 file, you can get HUGE compression ratios. You can always test it... bunzip the compressed file and compare an md5sum.
Thanks Matir.
Maybe bzip2 is so solid, perhaps there are no size limitations to what it can successfully compress/uncompress.
I just don't want to be the guy to find the limit, heh.
The only problem I could potentially see is if they use 32-bit file sizes in bzip2, but I find this highly unlikely. I've seen plenty of large files bz2 compressed and all seemed to have worked fine. You could always give it a test... dump some files together into one big file and bzip2 it while testing md5sums.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.