LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware
User Name
Password
Linux - Hardware This forum is for Hardware issues.
Having trouble installing a piece of hardware? Want to know if that peripheral is compatible with Linux?

Notices


Reply
  Search this Thread
Old 12-22-2017, 01:53 AM   #1
Rawze
LQ Newbie
 
Registered: Dec 2017
Posts: 3

Rep: Reputation: Disabled
LTO Tape drive Linux experience...


- Well, i just pulled the trigger the other day and purchased an HP Ultrium LTO-6 tape drive for my Linux box. I have a lot of media and video content I want to make permanent backups for and thought to myself "Why not a tape drive". Any ways, I have had about a week or so now of working with this thing and thought I would share a bit of my experiences so far. I am by means no expert so I am sure there are better ways of doing things, but I thought I would share a bit of gathered information here because it was just so hard to find, so here goes my story...

Well, I purchased an internal LTO-6 Ultrium tape drive with the intent on putting it into a PC instead of a server. I do have a Poweredge server but chose to put it into the PC instead. Here are some of the challenges, problem, and solutions I have seen so far with it...


I ordered the tape drive and wanted to mount it internally or perhaps in an enclosure later on, so it was recommended I get an HP H241 PCIe controller card (external SAS) for it. I found out right away that the H241 card has the wrong connectors on it. The H241 controller has SFF-8644 HD (known as SAS HD) connection ports on it and the drive came with an SFF-8087 adapter cable. After a day of searching, no one I could find carries an adapter cable to convert SFF-8087 to SFF-8644 directly. Problem number one.. I ordered the wrong adapter card. I decided to keep it though, as I since have found a solution to making this card work with this drive, but in the mean time, I ordered and installed into my PC, an HP H240 card that has the correct SFF-8087 ports on it. The HP H240 card has SF-8087 ports.

The First problem I encountered is that the adapter card wanted to overheat and was really hot to the touch. It clearly was meant for a server that had a high rate of forced air flow (like my old poweredge). I made a shroud around the card heat sink + mounted a cpu fan on it. After that, the adapter card ran cool and just fine.

Next, I installed the drive and wanted to use the terminal to do my backups and restores instead of relying on some software. Learning to get this thing to work via only terminal commands was several more days of frustrations. Because of this, I though I would share this information for others ...

The first time I booted the PC with everything installed (Ubuntu 16.04 LTS), the SAS card did its auto-setup routine, and then the PC booted just fine. The kernel had no trouble with the card or the drive but I did have to install some basic packages for operating it. Here is what I ended up installing to get it fully functional. All commands were as root user ...
Code:
apt-get install lsscsi fuse procinfo
apt-get install mtx mt-st pv mbuffer
I also had to eventually edit the '/etc/modules' file and add this to the bottom of the file on a separate line and re-boot ...
Code:
st

-----

I then used this command to ensure it was visible and operating ...

Code:
lsscsi | grep tape
the reply
...
[0:0:1:0] tape HP Ultrium 6-SCSI 35GD /dev/st0
...

The device name was '/dev/st0' --- and for 2 days, I used this device name only to find out that this is actually wrong!. It drove me nuts because if you only use the 'st0' then it constantly rewinds after every command and will refuse to make more than one back on a single tape as well. I had to use the device name 'dev/nst0' adding in the 'n' to prevent it from rewinding constantly.

-------

I found out that the drive will not respond to some commands correctly unless you set an option on the drive to allow it to become a logical device. Here is that command, It has to be used every time the PC is booted up before the drive is used. Adding it to a startup file somewhere is likely a good idea...

Code:
mt -f /dev/nst0 stsetoptions scsi2logical

-------

It took me the better part of another half day to figure out how to get/show the device settings and status info from the terminal. Here are those 2 commands combined ...

Code:
tapeinfo -f /dev/nst0 ; mt -f /dev/nst0 status
here is what mine reveals...
...
Code:
Product Type: Tape Drive
Vendor ID: 'HP      '
Product ID: 'Ultrium 6-SCSI  '
Revision: '35GD'
Attached Changer API: No
SerialNumber: 'HUJ7308Y1E'
MinBlock: 1
MaxBlock: 16777215
SCSI ID: 1
SCSI LUN: 0
Ready: yes
BufferedMode: yes
Medium Type: Not Loaded
Density Code: 0x5a
BlockSize: 0
DataCompEnabled: yes
DataCompCapable: yes
DataDeCompEnabled: yes
CompType: 0x1
DeCompType: 0x1
Block Position: 155409288
ActivePartition: 0
EarlyWarningSize: 0
NumPartitions: 0
MaxPartitions: 3
SCSI 2 tape drive:
File number=1, block number=0, partition=0.
Tape block size 0 bytes. Density code 0x5a (no translation).
Soft error count since last status=0
General status bits on (81010000):
 EOF ONLINE IM_REP
...
I could tell that compression was on by default along with other options. This information can also be used in a bash script if someone wants to monitor status, check settings, etc. - The very last line where it says 'ONLINE' is very useful for determining if the drive has a loaded tape and it and is ready for its next command.

-----


Before I get too much further along, I would like to also share more woes with installing this thing into a PC. An internal HP LTO-6 tape drive does NOT have any kind of cooling fan in it. THE DRIVE WILL OVERHEAT WHEN USING IT if forced air is not pushed through the drive (I was told a minimum of 5 cfm). I did not learn this for a day or so and the drive did strange things. It also had a hint of "burnt motor coil" smell to it on occasion. What would happen as it got hot is that during long backups, it would suddenly stop, eject the tape only partially, sit there for about 5 seconds, then load the tape back on, then continue. When it is cooled properly with forced air through the drive, it does not do this. ([tags] LTO tape partially ejects during operation, LTO drive overheating, LTO tape drive ejects during backup, LTO Ultrium ejecting randomly[/tags])


-------
Continuing on ...

Here is what I found out with this thing playing with it over the last few days. The basic premise of using a tape drive with terminal commands is to always position the tape to the beginning of the desired record number before backing up or restoring data. This means you will always need 2 commands to stay consistent at all times. A positioning command, followed by what you want to do.

- To make a backup/append a folder+files to the tape The commands are...
Code:
mt -f /dev/nst0 eod; #move tape past end of last record
tar -cvf /dev/nst0 /etc; #back up the '/etc' folder. change what folder you want as necessary.
- This will make a record and store all the files onto the tape inside that record. You can make multiple backups on one tape and refer to them by record number. The first record will always be 0 on the tape. The next will be record 1, then record 2, and so forth. I found I needed to use BOTH of these commands each time when making a backup even if backups are consecutive for things to always be in sync.

-------

- To show the folder and get a basic overview of what is stored on the tape at record number 0, you would use this command...

Code:
mt -f /dev/nst0 asf 0; #positon the tape to record number 0.
tar tvf /dev/nst0 2> /dev/null | head -n 1; #show only the first entry of the record.
...
as another example, to show record #2, it would look like this...
Code:
mt -f /dev/nst0 asf 2; #positon the tape to record number 2.
tar tvf /dev/nst0 2> /dev/null | head -n 1; #show only the first entry of the record.

-------
- To show a basic overview of all the records on the tape, I made this simple script and saved it to a file ...
Code:
mt -f /dev/nst0 stsetoptions scsi2logical
mt -f /dev/nst0 asf 0;
i=0;
while true;
	r=$(tar tvf /dev/nst0 2> /dev/null | head -n 1);
	! [[ -z "$r" ]] || break;
	printf "\nrecord $i (block-$r) -- ";
	printf "$r\n"
	mt -f /dev/nst0 fsf 1;
do
	((i++));
done
------
Restoring Files...
- To restore the first record (record 0) to the '/tmp' folder, you would use these commands. (the #comments are not necessary)...
Code:
cd  /tmp; #change to the destination folder.
mt -f /dev/nst0 asf 0; #position the tape to record number 0.
tar xvf /dev/nst0; #extract all files to the current folder.
-------

- To generate a list of all the files within a single record (0 in this example) and save it to a file '/tmp/tape-record0.filelist.txt' , you would do something like this ...

Code:
mt -f /dev/nst0 asf 0; #positon the tape to record number 0.
tar tvf /dev/nst0 2> /dev/null > /tmp/tape-record0.filelist.txt
You can then keep file lists and records somewhere so that you can use it for searching what you want on your tapes without having to wait for them to be read.

------

- To overwrite an existing record (record #2 in this example) and back up the '/etc' folder over it ...

Code:
mt -f /dev/nst0 asf 2; #positon the tape to record number 2.
tar -cvf /dev/nst0 /etc; #back up the '/etc' folder.
NOTE: THIS WILL ERASE ALL RECORDS PAST THIS RECORD NUMBER!.

------

- Erasing the tape...

NOTE: To prevent wasting many many hours of time, DO NOT USE THE ERASE COMMAND mentioned in the 'mt' man page for linux. Once an erase command is issued to the tape drive there is no stopping it without powering down or hard resetting the unit. Not a good idea as it will constantly be moving very fast and can make the tape un-spool a bit if you do shut it off/reset it. Here is an alternative that I have found...

To do a quick-erase, simply overwrite record 0 with 'dev/null'.
Code:
mt -f /dev/nst0 asf 0; #positon the tape to record number 0.
tar -cvf /dev/nst0 /dev/null; #write a null to record 0.
------

- Finally, to rewind and eject the tape...

Code:
mt -f /dev/nst0 rewind; #rewind the tape to the beginning.
mt -f /dev/nst0 eject; #eject the tape.
----

- Hopefully this will help others out there like myself trying to get started with using one of these things. I am certainly just trying to learn it myself, and thought I would share what I have so far to save others all the wasted time I spent and hours of combing through manuals.
 
Old 12-23-2017, 12:11 PM   #2
jlinkels
LQ Guru
 
Registered: Oct 2003
Location: Bonaire, Leeuwarden
Distribution: Debian /Jessie/Stretch/Sid, Linux Mint DE
Posts: 5,191

Rep: Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039
Great post. Thank you for the effort.

I think the /dev/st0 vs /dev/nst0 is something invented in 1971 when a 5MB tape was huge and a unix system had typically 20 devices in total. I feel rewinding is something which belong in the command execution and not in the device type.

Anyway, as you describe it, it is no fun at all and why on earth would someone want to use a tape drive. My last tape drive on a Linux system was a 400 GB LTO. It performed reasonably well. I had set up scripts to make full and differential backups on /dev/st0 and the last day of the week of /dev/nst0 when the tape had to be changed and so on.

Then the point came when a 1 TB hard drive became cheaper than tape drives. And since then I make backups on multiple USB hard disks. They get swapped regularly to provide off-site storage. And I also use NAS-es which can be located elsewhere by definition. Although the speed is not that great as compared to SAS or USB-3.

I wonder if this project was for academic purposes, or for fun, or that really tape back serves a purpose for you.

jlinkels
 
Old 12-27-2017, 01:26 AM   #3
Rawze
LQ Newbie
 
Registered: Dec 2017
Posts: 3

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by jlinkels View Post
Great post. Thank you for the effort.

...
I wonder if this project was for academic purposes, or for fun, or that really tape back serves a purpose for you.
...
Thanks for the reply and the insight.

Although I am a complete newb at this point for running this tape drive on linux, I am not new to the whole "tape" thing actually. My first computers had cassette tape drive in them back so many years ago. As the years passed, cassette tape backups got converted into floppies(360K), then into high capacity floppies(720K and 1.44M), then onto zip-drives, jazz-drives, then into CD-r, then dvd-r, then onto whole collections of dvd-r's, then onto HDD's like a lot of people these days as they got cheaper and cheaper over these last few years.

I used travan tape systems quite heavily years ago, but back then I never used them on any unix/linux machines I had. Of all the backup media and devices I have used over the years, there were 2 that were the most solid and just worked without question when you were looking for that file/photo/song/family home movie/etc. you backed up maybe 5, 6, or even perhaps 10+ years ago. Those 2 were without question "tape" and/or "cd's" of some sort, brand, or capacity. I have lost a lot of data over the years off of HDD's that sat around in backup collections for many years, them refusing to start up, or perhaps you find out that someone all too conveniently overwrote it at some point thinking to themselves "Baah,, this back is so old, I will never need it again".. only to find out that the one file your looking for the most now 10 years later, you realize was only on that old drive you have over-written 3 times since.

I.E.> I think for most, using HDD's as backup devices is definitely not a bad option at all, including for myself. I prefer it for the sheer convenience and speed factor alone. CDr's just will not hold the massive collections of files, folders, and HDD images these days without it taking a dozen or more CDr's/dvd'r/blu'r etc. just to make one backup. HDD's are the wise choice for something you don't need for more than a couple 3 years retention.

- Fast forward 10 or even 15+ years though, and remembering that home movie you thought was mostly just useless raw video, or perhaps those pictures you copied from one of your relatives/friends way back then, hearing they have just passed away,.. and out that old drive comes, you knocking the dust off it and praying like mad now that it still fires up.

Actually for myself, I kept finding myself drawn back to tape as sort of the "shoe that best fits" after weighing the options available while looking at this now mostly full drawer of HDD's that are getting out of date once again. I have dozens of "backup HDD's", some recent and as large as 6TB for short term backup purpose, some that have sat around for years as backups.

The re-visiting of such an old technology like tape, after looking at its now internationally standardized platform + somewhat re-growing popularity due to its sheer tenacity and revised access speeds became attractive again for me. I am not replacing my "HDD's as backup devices", but plan on using tape as the sort of "last resort, el-on-the-cheap-o" place to dump old backup HDD content before overwriting it with updated content. I am definitely not one to want to hassle with tape, even if they are faster than they used to be vs slapping that HDD you saved your files on last month. It is more of a "spare tire for the spare tire" on the cheap ($20 for 6 more TB on the bookshelf) kind of thing where several years later, I can say "yup, I have that old family picture collection by my third cousin, let me go dig it up" knowing that tape is simply going to work after knocking the dust off it.
 
Old 12-28-2017, 02:44 PM   #4
jefro
Moderator
 
Registered: Mar 2008
Posts: 19,149

Rep: Reputation: 2918Reputation: 2918Reputation: 2918Reputation: 2918Reputation: 2918Reputation: 2918Reputation: 2918Reputation: 2918Reputation: 2918Reputation: 2918Reputation: 2918
Good info.

I just wonder why all the overheating issues. Too low of current, too high of voltage or just as you say low cooling. I've seen many a rack server that has what I feel is less airflow than a pc. Dunno, would like to know.
 
Old 12-29-2017, 12:36 AM   #5
Rawze
LQ Newbie
 
Registered: Dec 2017
Posts: 3

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by jefro View Post
Good info.

I just wonder why all the overheating issues. Too low of current, too high of voltage or just as you say low cooling. I've seen many a rack server that has what I feel is less airflow than a pc. Dunno, would like to know.
They simply required a bit of direct air flow to keep them cool, power/voltage, etc. is all correct.
 
Old 04-03-2018, 09:12 AM   #6
VintageIT
LQ Newbie
 
Registered: Apr 2018
Posts: 1

Rep: Reputation: Disabled
Hi and thanks for your helpful post!

Question: did you by any chance find out how to dump the entire content of a tape? with floppy tapes it was possible to do a dd if=/dev/ftape of=dump.bin to save an image of the tape into a file for further analysis but seems like this can't be easily done with SCSI tapes

Problem is that I've got a bunch of old tapes and don't know how to identify with which software they have been written (beside trying to recognize some of them with NTBackup).

Thanks a lot!
 
Old 04-03-2018, 09:23 AM   #7
TenTenths
Senior Member
 
Registered: Aug 2011
Location: Dublin
Distribution: Centos 5 / 6 / 7
Posts: 2,927

Rep: Reputation: 1192Reputation: 1192Reputation: 1192Reputation: 1192Reputation: 1192Reputation: 1192Reputation: 1192Reputation: 1192Reputation: 1192
Just out of curiosity, did you ever consider using ltfs? https://en.wikipedia.org/wiki/Linear...pe_File_System
 
Old 04-03-2018, 12:12 PM   #8
Soadyheid
Senior Member
 
Registered: Aug 2010
Location: Near Edinburgh, Scotland
Distribution: Cinnamon Mint 17.3 (Netbook) and 18.3 (Desktop)
Posts: 1,419

Rep: Reputation: 338Reputation: 338Reputation: 338Reputation: 338
@Rawze
Quote:
let me go dig it up" knowing that tape is simply going to work after knocking the dust off it.
Er.... Knock the dust off it?! Wha...?

Dust and tape don't work, neither does temperature and humidity.

I used to maintain some IBM 3584 tape libraries loaded with LTO1 and LT02 drives and remember when the customer decided to do some reconstruction work in one of their Data Centres. It involved removing some plasterboard/drywall sheets which polluted the room with plaster dust next to one of the libraries. No, don't ask.

Each drive in the library looked like it had been snorting cocaine, fan ports smeared with dust, we had to remove each one, strip and clean them plus deep clean the whole library. Logs jammed with tape errors. Luckily, the other equipment in the room escaped pretty much Scott free.

Perhaps you'd like to check up on what environmental problems tape media has to put up with?

Surprisingly I found this old document, from June 1983, which highlights the problems with tape though it's obviously aimed at the reel-to-reel stuff (remember them?) Fun reading - 145 pages, Ghak!!

There are now companies who exist purely to recover data from long dead tape formats and save them to something more current. I don't think this is likely to affect you though.

Just a thought...

Play Bonny!

 
Old 10-09-2018, 03:27 PM   #9
m00dawg
LQ Newbie
 
Registered: Oct 2018
Posts: 1

Rep: Reputation: Disabled
Resurrecting an old thread, so my apologies. First I wanted to say thanks for the info! This will be a huge help as I am about to setup a tape backup solution as well (only using LTO-4).

I was curious - since it's a been a while, how has your tape backup strategy worked? I've been pondering just using 'tar' versus something like Bacula. I'm leaning towards the former because of the frequency of backups is, well, infrequent (these are for long term offline storage of photos, e.g.) and I wanted to keep things simple.
 
Old 10-10-2018, 12:17 PM   #10
AwesomeMachine
LQ Guru
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian testing/sid; OpenSuSE; Fedora; Mint
Posts: 5,513

Rep: Reputation: 1004Reputation: 1004Reputation: 1004Reputation: 1004Reputation: 1004Reputation: 1004Reputation: 1004Reputation: 1004
A footnote: as previously noted by the OP, HDDs and even more so SSDs, are not archival storage media. HDDs can be refreshed every few years, but not SSDs. To refresh a HDD for another 2-3 years of storage,
Code:
$ dd if=/dev/sdb of=/dev/sdb
That will write the drive to itself, trimming the tracks and refreshing the magnetic flux points that form the data.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] LTO-5 Tape drive issue - mt -f /dev/st0 fsf 1 not working callis07 Linux - Newbie 2 07-17-2014 02:29 AM
[SOLVED] HP Ultrium 448 External LTO Tape Drive not detecting in RHEL5 sree.m Linux - Server 5 07-19-2012 07:08 PM
[SOLVED] Enabling Compression on Ultrium LTO-2 Tape Drive Tyler_H72 Linux - Server 4 05-31-2011 03:09 PM
tar backup to LTO-4 tape drive wazzu62 Linux - Hardware 2 08-19-2009 10:07 AM
How do I use cleaning tape in LTO drive with Arkeia? bdb4269 Linux - Software 3 08-06-2006 06:24 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware

All times are GMT -5. The time now is 03:50 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration