LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 09-24-2009, 09:15 AM   #1
kschmitt
Member
 
Registered: Jul 2009
Location: Chicago Suburbs
Distribution: Crux, CentOS, RHEL, Ubuntu
Posts: 96

Rep: Reputation: 23
Question LTO tape filling too early.


I've got a situation where an LTO4 tape is filling too early, and I'm trying to figure out what the culprit is.

On an 800GB tape, I'm only getting about 600GB!

For those of you not familiar with LTO, it's an open standard tape format. LTO1 holds 100GB, LTO2 200GB, LTO3 400, and LTO4 800GB. That's raw data, and real GB, not the BS kind thats shown on the back of a hard drive box.

So I've got an LT04 tape loaded, about 616GB of data in an ext3 partition, and the data consists of 2431 files in 29 directories. I'm tarring it directly to tape, without compression (data is already compressed). Towards the end of the tar operation, I'm getting a, "Cannot write: No space left on device" error.

I know each file adds overhead past it's actual size (can't recall how much), but even if each file added 1MB of overhead, I'm still missing over 190 GB of space.....

Am I missing something here? Anyone with a deeper knowledge of tapes and tar out there?

Thanks
--Kyle

Last edited by kschmitt; 09-24-2009 at 09:16 AM. Reason: more info
 
Old 09-24-2009, 09:46 AM   #2
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
Blog Entries: 15

Rep: Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668Reputation: 1668
You've verified you're not using the no rewind device and starting late on the tape after another backup?

On this site for Dell's LTO4 drive:
http://www.dell.com/us/en/enterprise...0&cs=555&s=biz

is this informative footnote about "800 GB":
Quote:
GB means 1 billion bytes and TB equals 1 trillion bytes; actual capacity varies with preloaded material and operating environment and will be less.
Which is legalese telling you that marketroids don't talk about capacity the way techies do. You're losing in their definition about 56 GB.

Also you might want to rethink your decision not to use hardware compression. My experience is that it speeds the backup even when the data is already compressed although it might actually cause a slight increase in size (and reduction in capacity).
 
Old 09-24-2009, 10:38 AM   #3
kschmitt
Member
 
Registered: Jul 2009
Location: Chicago Suburbs
Distribution: Crux, CentOS, RHEL, Ubuntu
Posts: 96

Original Poster
Rep: Reputation: 23
Quote:
Originally Posted by jlightner View Post
Which is legalese telling you that marketroids don't talk about capacity the way techies do. You're losing in their definition about 56 GB.

Also you might want to rethink your decision not to use hardware compression. My experience is that it speeds the backup even when the data is already compressed although it might actually cause a slight increase in size (and reduction in capacity).

Yup. I'm using /dev/st1 (side note, /dev/st0 is a Dell re-branded lto2 drive, that stopped working after 2 months of use. After that experience, I've avoided Dell for all things tape/backup related, including their website )

That's interesting about the size of a GB. It directly contradicts what I read awhile ago, but searching, I can't find the article. It must've been wrong though, since I've found 3 other places that state 1,000,000,000 as a GB. Damn marketers! Surprising it's not more widely publicized though. I guess people have gotten used to it.

Anyway, even at that size, I'm still missing about 128 GB...

What types of compression were already on the data when you've enabled drive-level compression?
 
Old 09-29-2009, 09:35 AM   #4
kschmitt
Member
 
Registered: Jul 2009
Location: Chicago Suburbs
Distribution: Crux, CentOS, RHEL, Ubuntu
Posts: 96

Original Poster
Rep: Reputation: 23
Anyone have any ideas?

Even with the latest evidence that the manufacturer's state it's 800X1billion bytes, not 800GiB* I'm still missing out on 128GiB of storage on my tapes!


* http://en.wikipedia.org/wiki/GiB
 
Old 09-29-2009, 09:00 PM   #5
choogendyk
Senior Member
 
Registered: Aug 2007
Location: Massachusetts, USA
Distribution: Solaris 9 & 10, Mac OS X, Ubuntu Server
Posts: 1,197

Rep: Reputation: 105Reputation: 105
Could be conflict between software and hardware compression. If you have data that is already compressed, and hardware compression kicks in, you can actually expand the data.

So, make sure you aren't getting hardware compression. Just as a note, I saw a discussion once on one of the backup lists that if a tape had been previously used with compression on, and you tried to rewrite it with compression off, the hardware would overrule you, because it "knows" that that tape is supposed to be compressed. Don't remember all the details; but, basically, to be sure, you have to use a new tape after you are sure you have compression turned off on the drive.

Also, if you're not getting decent throughput (LTO4 drives are faster than some computers and their disk drives can handle), the LTO will shoe-shine, and you will lose a bit every time it does that. You would know, because your throughput would be incredibly low compared to the rated speed of LTO. If you are near or over 100MB/s, you are fine. If you are way down, say, 25MB/s or even less, then you losing time and space. I'm not sure quite where the transition is, but it is very clear cut, because the shoe shining pushes it down even more.
 
Old 10-01-2009, 11:48 AM   #6
kschmitt
Member
 
Registered: Jul 2009
Location: Chicago Suburbs
Distribution: Crux, CentOS, RHEL, Ubuntu
Posts: 96

Original Poster
Rep: Reputation: 23
Quote:
Originally Posted by choogendyk View Post
rated speed of LTO. If you are near or over 100MB/s, you are fine. If you are way down, say, 25MB/s or even less,
Humm. By my rough calculation, I'm writing 570GiB in 6hours, so that works out to about 27MiB/second...

I don't know why the machine would really be slow though: nothing else runs while it's writing to tape, it's got 4 cores, 4gigs of ram, scsi to the tape drive, and the source is on a 6 disc raid-5 part...

How would you work on improving the throughput to the tape drive then?

Thanks
--Kyle
 
Old 10-01-2009, 06:33 PM   #7
choogendyk
Senior Member
 
Registered: Aug 2007
Location: Massachusetts, USA
Distribution: Solaris 9 & 10, Mac OS X, Ubuntu Server
Posts: 1,197

Rep: Reputation: 105Reputation: 105
OK, so now you have a totally different problem. The issue is no longer why does my LTO tape fill up too early (you now know the answer). The issue is how do I tune my server to be able to maintain high throughput to an LTO4 tape drive.

I'm not sure I'm the best person to help. My servers are all Sun enterprise and T2 (Chip Multi-Threaded) based servers. I think you should post a new thread with this new question. Provide the details of what the server is, what the CPU is, what kind of SCSI card, how the RAID is connected (different SCSI bus or same), what OS, what software is driving the backup to the tape, and so on. I'm sure someone will be able to point out what the issue is.

Without knowing any of that, I would say that you should have the RAID and the LTO on separate SCSI controllers, and that they should both be LVD Ultra320 SCSI. Or, if the RAID is SAS, that might work. Dual multi-homed SAS would be better. SATA? I don't know.

Anyway, good luck, now that you have a direction.
 
  


Reply

Tags
lto, tape, tar


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
tar backup to LTO-4 tape drive wazzu62 Linux - Hardware 2 08-19-2009 10:07 AM
Slow speeds while dumping backup files to LTO-3 tape device dafart Linux - Hardware 3 09-04-2007 10:35 AM
dd and LTO tape drives with seek kernel bug jahearne Linux - Hardware 0 08-30-2007 04:33 PM
How do I use cleaning tape in LTO drive with Arkeia? bdb4269 Linux - Software 3 08-06-2006 06:24 PM
Utrium 2 LTO Tape: space used/available? JMCraig Linux - Hardware 0 09-05-2005 07:10 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 12:12 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration