LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 03-09-2009, 06:54 PM   #1
Runge_Kutta
Member
 
Registered: Jul 2006
Location: H-town, TX
Distribution: SuSE10.1, Ubuntu 9.10, RHEL-4-WS, Fedora Core 6, CentOS 5.x, Fedora 13
Posts: 35

Rep: Reputation: 16
Is Tar file to tape drive faster than dir/files to tape drive?


I have about a 163GB tar file that contains a backup project. I will be putting this file onto SDLT1 tape using compatible drive (compression is confirmed to be on). My question is this:

will the transfer to the tape drive go faster if I use the tar file or the original directories and files:

dd if=project.tar.gz of=/dev/st0 bs=16k

vs

tar -cvzf /dev/st0 /project

will any combination of bit sizes, options, or compressions make one method faster than the other? Or is the dd command going to be the fastest for my situation?

Also, my problem with using the dd command is that it is going to limit my ability to check the files on the tape after data transfer is finished as supposed to using a straight tar command: i.e. tar -tvzf /dev/st0 would not be possible after using the dd command above without first copying entire file to disk before running tar -tvzf on copied tar file. Or is there a way around this?
 
Old 03-10-2009, 06:59 AM   #2
GazL
LQ Veteran
 
Registered: May 2008
Posts: 6,719

Rep: Reputation: 4838Reputation: 4838Reputation: 4838Reputation: 4838Reputation: 4838Reputation: 4838Reputation: 4838Reputation: 4838Reputation: 4838Reputation: 4838Reputation: 4838
There is a way around it, to write on the fly you can use the '-' stdin/stdout as the filename on tar.

Code:
tar -cvzf - /project | dd bs=16k of=/dev/st0
and similar to test.

Code:
dd bs=16k if=/dev/st0 | tar -tvzf -
Whether this would be faster or not would probably depend on the specifics of your tape device, and how the linux driver interacts with it. I've certainly used the above approach on commercial UNIXs before to good effect but I've no experience with tape on linux.

If your tape drive does it's own hardware compression, then you might be better off not using the -z tar option. It'll just use up cpu time for no gain (in some cases, compressing compressed data can actually make the data larger).

You could always do a couple of benchmarks on a subset of your data, trying a few different blocksizes/compression options and come back and let us know if it has an effect.
 
Old 03-11-2009, 07:13 AM   #3
choogendyk
Senior Member
 
Registered: Aug 2007
Location: Massachusetts, USA
Distribution: Solaris 9 & 10, Mac OS X, Ubuntu Server
Posts: 1,197

Rep: Reputation: 105Reputation: 105
If you are planning on going to tape in the first place, then it would be faster (from the beginning) to just go straight to tape with tar, and skip the gzip as well as the -z option on tar. You confirmed compression is on for the tape drive.

Of course, this is assuming you have no reason to have the tar file sitting around on your system (not on tape). The time spent doing the tar and the gzip then was wasted time, which you wouldn't do in the future. The gzip on something that large takes a long time, and then, if compression is on for the tape drive, it actually slows it down a bit as well as possibly losing some space.

With gnu tar, you can mess around with record sizes for tape, but I've never done that. I use Amanda for my tape backups. You could check the man page or the documentation at gnu.org. You can also get information from gnu tar on the format (including blocking factor) of an existing archive.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Unable to insert the DAT72 Tape in tape drive. peddip Red Hat 8 01-08-2009 09:27 AM
help me to take backup to tape drive using tar binuveranal Linux - Server 1 02-26-2008 03:33 AM
Tar not able to perform backup on tape drive livetoday Red Hat 2 09-29-2007 03:16 AM
Tar command and Tape drive perry.rabino Linux - Newbie 1 05-22-2006 03:26 AM
Using a DDS5 tape drive to restore from a DDS3 backup tape. AndrewCAtWayofthebit Linux - Hardware 1 05-14-2006 09:15 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 04:32 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration