LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 01-19-2013, 08:37 AM   #1
Changes
Member
 
Registered: Oct 2008
Posts: 190

Rep: Reputation: 16
Error-tolerant non-compressing archives


I just had the unfortunate experience of having a single bad sector ruin an entire 32GB uncompressed rar containing a whole lot of stuff, fortunately of secondary importance. I thought keeping the file uncompressed would make it just skip the relevant files in case of an error, but I was obviously wrong.

Is there anything equivalent that would do that in case of a small amount of bad data instead of aborting the whole procedure? What about uncompressed tar files?

I ask because I often have to backup lots of gigs of very small files, and a normal copy takes three forevers as it has to write all the tiny files, while an archive operation on a single huge file takes a lot less.
 
Old 01-19-2013, 09:37 AM   #2
ruario
Senior Member
 
Registered: Jan 2011
Location: Oslo, Norway
Distribution: Slackware
Posts: 2,557

Rep: Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762
afio. It defaults to cpio odc format (but will use extensions to handle large files). Also does compression per file, if you do want compression.
 
Old 01-19-2013, 09:58 AM   #3
ruario
Senior Member
 
Registered: Jan 2011
Location: Oslo, Norway
Distribution: Slackware
Posts: 2,557

Rep: Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762
Some redundancy data is a good idea for important backups and does work to safeguard against various levels of corruption. You could use parity files with any archive format. The advantage of parity files is that I could set say 10% redundancy (allowing you to fix corruptions of up to 10% anywhere in your file), without having to store a full extra copy. Check out parchive/par2cmdline (an implementation of PAR v2.0 specification).
 
Old 01-19-2013, 10:08 AM   #4
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
...also note RAR (lets skip OSS vs non-OSS for now) recover undamaged files, keep extracted files that fail CRC check and it can create recovery records / volumes as well. Just enable the switches you need.
 
Old 01-19-2013, 10:46 AM   #5
ruario
Senior Member
 
Registered: Jan 2011
Location: Oslo, Norway
Distribution: Slackware
Posts: 2,557

Rep: Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762
RAR is a nice tool but IIRC the format does not store all UNIX meta data (ownership, groups), nor support all file types (links, device files) so using an *nix native format (tar, cpio, dar, etc.) might be better if you care about these kinds of things.
 
Old 01-19-2013, 11:32 AM   #6
H_TeXMeX_H
LQ Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301
What I use for backups:
http://dvdisaster.net/en/index.html
It's mostly for DVDs.
 
Old 01-19-2013, 12:06 PM   #7
ruario
Senior Member
 
Registered: Jan 2011
Location: Oslo, Norway
Distribution: Slackware
Posts: 2,557

Rep: Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762
This is also interesting http://www.mondorescue.org/ (it uses afio as a backend)
 
Old 02-15-2013, 09:53 PM   #8
linuxbird
Member
 
Registered: Feb 2006
Distribution: Slackware
Posts: 543

Rep: Reputation: 36
This is an interesting discussion.

I am trying to backup selected areas, and I would like to use cpio, except it fails with 7 or 8 gb files.

Ideally, I would like to find the files I would like backed up, and then have them sorted size wise so that the nearly optimally pack on media. And then I would like cpio run against one or more media, so that I can save on bd, dvd, or cd.

I am using Slackware 14.0, 32 bit. Any pointers or ideas?
 
Old 02-16-2013, 09:06 AM   #9
ruario
Senior Member
 
Registered: Jan 2011
Location: Oslo, Norway
Distribution: Slackware
Posts: 2,557

Rep: Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762Reputation: 1762
afio will overcome the size limitations. it extends the cpio format when such limitations are reached. for the other stuff spend some time reading man find.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
error processing /var/cache/apt/archives/cnijfilter-common_3.50-2ubuntu3_amd64.deb docklc Linux - Software 1 08-02-2011 02:40 PM
/var/cache/apt/archives/udvev-151-3-i386.deb ERROR Antiz Linux - Newbie 15 04-05-2010 02:19 PM
Segmenting archives into a variety of smaller archives. ktek Linux - Software 4 09-25-2008 04:47 AM
/var/cache/apt/archives/lock error digsby0007 Linux - Software 3 06-23-2006 06:17 AM
crc error,stablin compressing ,can't install redhat ngan_yine Linux - Newbie 2 09-03-2003 07:52 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 09:30 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration