-   Linux - Software (
-   -   Long Term File Archiving with Linux (

chaoswings 11-01-2008 06:17 PM

Long Term File Archiving with Linux
Good file archiving application?

Is there any good file archiving application that works on Linux available in the repository? I tried all sorts of keywords in synaptic package manager and was unable to find what I wanted.

What I want to do: Watch a specified number of folders on my desktop PC and Add or replace files in an archive which I will keep on on my external hard drive.

The files will be anything from movies to music to games. I also want the program to be able to perform a CRC32 or MD5 check periodically (like once a year or something) on every individual file to verify its integrity. A separate program with advanced search features to index it all would be nice too.

I'll be burning a secondary copy to a bunch of blu ray discs once it is cheap enough.

The archive could reach terabytes so it needs to be scalable and possibly spread across hard drives. I'm a hobbyist who likes to create movies/music and draw. As any videophile or digital artist knows, those files can get huge. I'm already at about 500GB since I started doing this 5 years ago. Now I need a more reliable (but still relatively cheap) solution.

The plan:
I was thinking of installing ubuntu on my external hard drive having the files on one partition and the ubuntu OS on another so I can update the OS without fear of losing everything. Then the external hard drive is self contained and can play the files and verify the integrity all on it's own. I also need to able to transfer the files to a new hard drive when that hardware becomes obsolete or I need more storage space.

Hopefully 50 years from now ubuntu and VLC will still be able to play all the same formats (MKV,AVI, H264 codecs etc.). I am hoping linux and open source does not screw me over like windows and is able to maintain backwards compatibility.

Does this sound crazy?

Well maybe it does but I don't want to wind up like some old people I know who had vinyl records and are now unable to play them because they either degraded or can't be read by a machine.

Why did I decide to do this?

I'm only 20 and one of my friends asked me for my copy of Final Fantasy 9 cause his got destroyed along with his PS1. I have the original discs and I lent the game to him.

After that it got me thinking and I made ISOs of those discs and now store the digital copies away along with the original. Thanks to emulators I can still play it (my PS1 no longer works either).

This scenario happens often it seems I'm the only one who takes care of my stuff...This is even more painful when its a copy of one of my self-made movies or animations in which no other copies exist anywhere else.

acid_kewpie 11-01-2008 06:24 PM

i'd say just use rsync to maintain a copy. it's normally used over a network, but can work fine locally on loopback. as for scalability, i'd be using lvm to add additional disks to a single filesystem. everything else you ask for is just seperate tasks on a filesystem.

All times are GMT -5. The time now is 06:43 AM.