Debugging slow tape: Big disparity between iostat and tape speed
 

I am trying to debug a generic whitebox backup server running SLES 10 and Veritas Netbackup 6. It has a 3Ware RAID5 array hung off one SCSI bus and an Ultrium 4 tape hung off another SCSI bus.

I am only getting about 20 MBytes/sec off the drive, verified both by wall clock time and Veritas's MB/s stats. However, both iostat and vmstat report that the disk is being read at about 100000 blocks/sec, which would be about 50Mbytes/sec.

I am mystified. What explains the diferrence? The system is not paging and is otherwise quiet.

TIA - W Sanders
http://wsanders.net

In my experience, it depends on the data that's getting shoveled across. For example, if you have 10,000 1KB files, they'll restore FAR slower than a single 2GB file will.

Tape drives have great throughput...but suffer from 'shoeshining'. It'll be going so fast, it'll go past the end of a small file, and have to stop, back up, then go forward again....repeat a gazillion times for a bunch of small files. A large file, though, is a steady stream, and will blow off the drive in no time, limited only by the transfer rate of the hardware. If you can, sit next to the drive during the restore, and listen to it; if you hear alot of stopping/rewinding/starting, that's probably the case.

Try running some speed tests with large (>1GB is a good size) files, and then with smaller files.

If you are only getting 20MB/s with LTO4, then it is definitely shoe-shining. I would think that Netbackup would be sending a continuous data stream (in it's own format) to the tape, and not be breaking it up by file. But, is it trying to do something like software compression? Have you got plenty of memory? Is the CPU plenty fast to keep up with all of this? Is the server backplane fast enough to handle both SCSI buses at once? And are they both something like LVD Ultra320? Typical problem people run into with something like LTO4 is having the server appropriately beefy, balanced and scaled to keep up with the throughput of the LTO4.

Can you cut Netbackup out of the loop? Just for testing. Try doing a tar of a large partition to the tape? Then you can figure out whether it is something in your hardware setup or something in your Netbackup configuration.