Linux - Server This forum is for the discussion of Linux Software used in a server related context. |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
 |
09-24-2009, 09:15 AM
|
#1
|
Member
Registered: Jul 2009
Location: Chicago Suburbs
Distribution: Crux, CentOS, RHEL, Ubuntu
Posts: 96
Rep:
|
LTO tape filling too early.
I've got a situation where an LTO4 tape is filling too early, and I'm trying to figure out what the culprit is.
On an 800GB tape, I'm only getting about 600GB!
For those of you not familiar with LTO, it's an open standard tape format. LTO1 holds 100GB, LTO2 200GB, LTO3 400, and LTO4 800GB. That's raw data, and real GB, not the BS kind thats shown on the back of a hard drive box.
So I've got an LT04 tape loaded, about 616GB of data in an ext3 partition, and the data consists of 2431 files in 29 directories. I'm tarring it directly to tape, without compression (data is already compressed). Towards the end of the tar operation, I'm getting a, "Cannot write: No space left on device" error.
I know each file adds overhead past it's actual size (can't recall how much), but even if each file added 1MB of overhead, I'm still missing over 190 GB of space.....
Am I missing something here? Anyone with a deeper knowledge of tapes and tar out there?
Thanks
--Kyle
Last edited by kschmitt; 09-24-2009 at 09:16 AM.
Reason: more info
|
|
|
09-24-2009, 09:46 AM
|
#2
|
LQ Guru
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,831
|
You've verified you're not using the no rewind device and starting late on the tape after another backup?
On this site for Dell's LTO4 drive:
http://www.dell.com/us/en/enterprise...0&cs=555&s=biz
is this informative footnote about "800 GB":
Quote:
GB means 1 billion bytes and TB equals 1 trillion bytes; actual capacity varies with preloaded material and operating environment and will be less.
|
Which is legalese telling you that marketroids don't talk about capacity the way techies do. You're losing in their definition about 56 GB.
Also you might want to rethink your decision not to use hardware compression. My experience is that it speeds the backup even when the data is already compressed although it might actually cause a slight increase in size (and reduction in capacity).
|
|
|
09-24-2009, 10:38 AM
|
#3
|
Member
Registered: Jul 2009
Location: Chicago Suburbs
Distribution: Crux, CentOS, RHEL, Ubuntu
Posts: 96
Original Poster
Rep:
|
Quote:
Originally Posted by jlightner
Which is legalese telling you that marketroids don't talk about capacity the way techies do. You're losing in their definition about 56 GB.
Also you might want to rethink your decision not to use hardware compression. My experience is that it speeds the backup even when the data is already compressed although it might actually cause a slight increase in size (and reduction in capacity).
|
Yup. I'm using /dev/st1 (side note, /dev/st0 is a Dell re-branded lto2 drive, that stopped working after 2 months of use. After that experience, I've avoided Dell for all things tape/backup related, including their website  )
That's interesting about the size of a GB. It directly contradicts what I read awhile ago, but searching, I can't find the article. It must've been wrong though, since I've found 3 other places that state 1,000,000,000 as a GB. Damn marketers! Surprising it's not more widely publicized though. I guess people have gotten used to it.
Anyway, even at that size, I'm still missing about 128 GB...
What types of compression were already on the data when you've enabled drive-level compression?
|
|
|
09-29-2009, 09:35 AM
|
#4
|
Member
Registered: Jul 2009
Location: Chicago Suburbs
Distribution: Crux, CentOS, RHEL, Ubuntu
Posts: 96
Original Poster
Rep:
|
Anyone have any ideas?
Even with the latest evidence that the manufacturer's state it's 800X1billion bytes, not 800GiB* I'm still missing out on 128GiB of storage on my tapes!
* http://en.wikipedia.org/wiki/GiB
|
|
|
09-29-2009, 09:00 PM
|
#5
|
Senior Member
Registered: Aug 2007
Location: Massachusetts, USA
Distribution: Solaris 9 & 10, Mac OS X, Ubuntu Server
Posts: 1,197
Rep: 
|
Could be conflict between software and hardware compression. If you have data that is already compressed, and hardware compression kicks in, you can actually expand the data.
So, make sure you aren't getting hardware compression. Just as a note, I saw a discussion once on one of the backup lists that if a tape had been previously used with compression on, and you tried to rewrite it with compression off, the hardware would overrule you, because it "knows" that that tape is supposed to be compressed. Don't remember all the details; but, basically, to be sure, you have to use a new tape after you are sure you have compression turned off on the drive.
Also, if you're not getting decent throughput (LTO4 drives are faster than some computers and their disk drives can handle), the LTO will shoe-shine, and you will lose a bit every time it does that. You would know, because your throughput would be incredibly low compared to the rated speed of LTO. If you are near or over 100MB/s, you are fine. If you are way down, say, 25MB/s or even less, then you losing time and space. I'm not sure quite where the transition is, but it is very clear cut, because the shoe shining pushes it down even more.
|
|
|
10-01-2009, 11:48 AM
|
#6
|
Member
Registered: Jul 2009
Location: Chicago Suburbs
Distribution: Crux, CentOS, RHEL, Ubuntu
Posts: 96
Original Poster
Rep:
|
Quote:
Originally Posted by choogendyk
rated speed of LTO. If you are near or over 100MB/s, you are fine. If you are way down, say, 25MB/s or even less,
|
Humm. By my rough calculation, I'm writing 570GiB in 6hours, so that works out to about 27MiB/second...
I don't know why the machine would really be slow though: nothing else runs while it's writing to tape, it's got 4 cores, 4gigs of ram, scsi to the tape drive, and the source is on a 6 disc raid-5 part...
How would you work on improving the throughput to the tape drive then?
Thanks
--Kyle
|
|
|
10-01-2009, 06:33 PM
|
#7
|
Senior Member
Registered: Aug 2007
Location: Massachusetts, USA
Distribution: Solaris 9 & 10, Mac OS X, Ubuntu Server
Posts: 1,197
Rep: 
|
OK, so now you have a totally different problem. The issue is no longer why does my LTO tape fill up too early (you now know the answer). The issue is how do I tune my server to be able to maintain high throughput to an LTO4 tape drive.
I'm not sure I'm the best person to help. My servers are all Sun enterprise and T2 (Chip Multi-Threaded) based servers. I think you should post a new thread with this new question. Provide the details of what the server is, what the CPU is, what kind of SCSI card, how the RAID is connected (different SCSI bus or same), what OS, what software is driving the backup to the tape, and so on. I'm sure someone will be able to point out what the issue is.
Without knowing any of that, I would say that you should have the RAID and the LTO on separate SCSI controllers, and that they should both be LVD Ultra320 SCSI. Or, if the RAID is SAS, that might work. Dual multi-homed SAS would be better. SATA? I don't know.
Anyway, good luck, now that you have a direction.
|
|
|
All times are GMT -5. The time now is 10:49 PM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|