Adding 4TB disk, need help figuring out BIOS/gdisk/parted

kaza · 12-21-2018, 12:33 PM

Hello!

I'm adding a 4TB SAS disk (MG03SCA400, firmware 5702) connected to the same IDT5165BR
controller to which the old 4 disks are connected (2 pairs of hardware RAID 0 arrays).
My BIOS is rather old - the PC had been bought in 2009 and the IDT disks
utility (which uses BIOS) sees only 2TB (at first I thought I felt victim
to fraud but later read about the 2TB BIOS/MBR limitation).

I understand that I should use "gdisk" or "parted" tools to create the 4TB partition
(later I would like to add it to LVM) but from "man gdisk" and "man parted"
it seems that they need at least the disk to be visible as a device. I haven't done anything
yet (except connecting) with the new disk, haven't run any "format" by the IDT
disk utility because it sees only 2TB and I'm worried that if I'll try formatting it
using the IDT utility (which uses BIOS) I might end with only 2 TB usable.
Currently the new disk isn't visible as a device:

Code:

<root localhost.localdomain>.../root>lsscsi
[0:0:0:0]    cd/dvd  HL-DT-ST DVDRAM GH24NSD1  LG00  /dev/sr0 
[4:0:3:0]    tape    HP       Ultrium 1-SCSI   N2CG  /dev/st0 
[8:0:0:0]    disk    ICP      KA_RAID_0        V1.0  /dev/sda 
[8:0:1:0]    disk    ICP      KA_RAID_1        V1.0  /dev/sdb 
[8:1:0:0]    disk    FUJITSU  MAU3147RC        0104  -        
[8:1:1:0]    disk    FUJITSU  MBA3147RC        HPF1  -        
[8:1:2:0]    disk    TOSHIBA  MK2001TRKB       0105  -        
[8:1:3:0]    disk    TOSHIBA  MK2001TRKB       0105  -        
[8:1:4:0]    disk    TOSHIBA  MG03SCA400       5702  -        


<root localhost.localdomain>.../root>ls /dev | grep sd
brw-rw----. 1 root disk             8,   0 Dec 21 18:50 sda
brw-rw----. 1 root disk             8,   1 Dec 21 18:50 sda1
brw-rw----. 1 root disk             8,   2 Dec 21 18:50 sda2
brw-rw----. 1 root disk             8,  16 Dec 21 18:50 sdb
brw-rw----. 1 root disk             8,  17 Dec 21 18:50 sdb1

What should I do in order to view the new disk as a device?

TIA,
kaza.

kaza · 12-21-2018, 12:56 PM

Some more info: looking at the "dmesg" output I see the following related to SCSI disks deection:

Code:

[    2.923492] scsi host8: aacraid
[    2.923779] scsi 8:0:0:0: Direct-Access     ICP      KA_RAID_0        V1.0 PQ: 0 ANSI: 2
[    2.924121] scsi 8:0:1:0: Direct-Access     ICP      KA_RAID_1        V1.0 PQ: 0 ANSI: 2
[    2.946747] scsi 8:1:0:0: Direct-Access     FUJITSU  MAU3147RC        0104 PQ: 0 ANSI: 3
[    2.948298] scsi 8:1:1:0: Direct-Access     FUJITSU  MBA3147RC        HPF1 PQ: 0 ANSI: 5
[    2.951631] scsi 8:1:2:0: Direct-Access     TOSHIBA  MK2001TRKB       0105 PQ: 0 ANSI: 5
[    2.953634] scsi 8:1:3:0: Direct-Access     TOSHIBA  MK2001TRKB       0105 PQ: 0 ANSI: 5
[    2.955489] scsi 8:1:4:0: Direct-Access     TOSHIBA  MG03SCA400       5702 PQ: 0 ANSI: 5
[    3.015194] firewire_core 0000:09:08.0: created device fw0: GUID 001e8c0001843f96, S400
[    3.023172] clocksource: Switched to clocksource tsc
[    3.032327] sd 8:0:0:0: Attached scsi generic sg1 type 0
[    3.032360] sd 8:0:0:0: [sda] 573030400 512-byte logical blocks: (293 GB/273 GiB)
[    3.032392] sd 8:0:0:0: [sda] Write Protect is off
[    3.032395] sd 8:0:0:0: [sda] Mode Sense: 12 00 10 08
[    3.032428] sd 8:0:0:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA
[    3.032543] sd 8:0:1:0: [sdb] Very big device. Trying to use READ CAPACITY(16).
[    3.032556] sd 8:0:1:0: [sdb] 7801405440 512-byte logical blocks: (3.99 TB/3.63 TiB)
[    3.032565] sd 8:0:1:0: [sdb] Write Protect is off
[    3.032567] sd 8:0:1:0: [sdb] Mode Sense: 12 00 10 08
[    3.032583] sd 8:0:1:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
[    3.032700] sd 8:0:1:0: Attached scsi generic sg2 type 0
[    3.032715] sd 8:0:1:0: [sdb] Very big device. Trying to use READ CAPACITY(16).
[    3.033433]  sda: sda1 sda2
[    3.035263] scsi 8:1:0:0: Attached scsi generic sg3 type 0
[    3.035514] sd 8:0:0:0: [sda] Attached SCSI removable disk
[    3.035528] scsi 8:1:1:0: Attached scsi generic sg4 type 0
[    3.037756] scsi 8:1:2:0: Attached scsi generic sg5 type 0
[    3.037908] scsi 8:1:3:0: Attached scsi generic sg6 type 0
[    3.038152] scsi 8:1:4:0: Attached scsi generic sg7 type 0
[    3.054072]  sdb: sdb1
[    3.054301] sd 8:0:1:0: [sdb] Very big device. Trying to use READ CAPACITY(16).
[    3.054395] sd 8:0:1:0: [sdb] Attached SCSI removable disk

For some reason it detected the new disk as removable.

Still the main question: how to make it to be detected like an ordinary disk?
What will happen if I'll use the IDT disk utility (using BIOS) to format the disk?

TIA,
kaza.

dc.901 · 12-21-2018, 01:35 PM

What does "fdisk -l /dev/sdb" show?

syg00 · 12-21-2018, 07:07 PM

Yes all the userspace tools require a device node to work with - I'd just let IDT create the device. The kernel probes devices itself and generally doesn't trust dodgy firmware, so it'll be interesting to see what it determines it to be.
Even if you ultimately only get 2T, it's 2T more than you currently have.

Note - I have no experience with SAS disks.

ehartman · 12-22-2018, 01:51 AM

Quote:

Originally Posted by kaza

For some reason it detected the new disk as removable.

Still the main question: how to make it to be detected like an ordinary disk?
What will happen if I'll use the IDT disk utility (using BIOS) to format the disk?

The "old" disk (sda) is deemed to be removable too, so that is not specific to the 4 TB one.
And the kernel did assign it the name sdb, though, because you didn't partition it yet, no sdb? partition names yet.
When the system boots up, is the device file /dev/sdb there?

If so, just use gdisk or parted on it to partition it.
If you used IDT first, you still will have to use those programs, but then first will have to remove the partitions IDT created.

Brains · 12-22-2018, 02:16 AM

Quote:

Originally Posted by kaza

I'm worried that if I'll try formatting it
using the IDT utility (which uses BIOS) I might end with only 2 TB usable.
Currently the new disk isn't visible as a device:

Nothing to worry about, formatting a disk is not a one shot and live with it situation. Go ahead and format it with the IDT, so what if you only get 2GB, at least it will be visible by the OS, then you can resize the partition with another utility from within the OS. The OS generally don't care what the BIOS says, it will see the whole disk and use the whole disk.

My old Dell laptop had a 120GB BIOS limit, I installed a 250GB drive and partitioned it with Bootitng dos based boot manager, zapped images of three Operating systems at the front of the drive, booted Windows and formatted the Data partition NTFS which encompassed the rest of the drive. And happily used the whole 250GB.

BTW: Although this was like 10 years ago, Gparted would adhere to BIOS limits and only offer 120GB, that's why I used Bootitng which was my boot manager on the limited laptop at the time, it flipped the BIOS a finger and gave me the whole drive. And all operating systems saw all partitions and used them. Gparted or parted (same) may be different today, but there are utilities that will give you what you want if parted won't. You may also want to try creating the partition(s) from a computer without the 2GB limit first, then format after when visible by the OS.

kaza · 12-23-2018, 11:50 AM

Thanks for all the replies, sorry for the late answer:
I've run low level format with the IDT controller BIOS utility
(16 hours) and then decided to verify the media. At first the progress
indicator showed that it'll take about 6 hours (started on Saturday
at 15:00) but as it completed around 21:00 it started another iteration
of yet another 6 hours. I thought: OK, it'll end around 03:00, I wake up around
04:00 and leave to work close to 05:00 so I'll have few minutes to complete the boot
and see if the device is visible by FC26. But low and behold, when I looked at the screen
around 04:15 I saw the progress indicator dropped to 25% which means: it started another
6 hours iteration. I went to work in hope to get back to find the verify completed but
when I got back around 18:30 I saw that it's still in the middle of yet another
6 hours iteration! I'm currently posting from the laptop, wondering how many 6 hours
iteration will there be (unless I'll lose patience and abort it). So far it completed
8 iterations and is at 70% of the 9-th one. What does it do: writing and verifying different
patterns?

Anyway, that's not the main issue, the main issue was the fact that the new disk is visible
to "lsscsi" and is mentioned in "dmesg" output but not becoming a visible /dev/sdc.

The /dev/sda is a hardware RAID 0 pair of 147GB Fujitsu disks (on which the OS is installed),
the /dev/sdb is a hardware RAID 0 pair of 2TB Toshiba disks

so I expect the newly added disk to become /dev/sdc but it didn't
happen so far. I'll wait for the "verify media" iterations to end
(unless someone will tell me that IDT5165BR controller utility should only
do one verify iteration in which case it's stuck in an infinite loop),
proceed with the boot and check if the low level format solved the issue
of the new disk not appearing as a device.

Thanks for the advices so far, "to be continued..." :-)
kaza.

Brains · 12-23-2018, 01:25 PM

Hopefully it's not wiping out all 5 disks. A low level format checks every 512 byte sector for defects, which is not required on a new disk, a simple format is all that's needed on newer disks as the disk controllers of most newer disks are smart enough to isolate bad sectors on their own.

So either it's checking all 5 disks, then formatting them, or it's doing a secure wipe. A secure wipe random/zero fills every sector a specified amount of passes across the whole disk. The utility I mentioned using in the past called Bootitng, if I remember correctly made a minimum of 10 passes for a secure wipe.

kaza · 12-23-2018, 02:08 PM

OK, after yet another verify iteration started I aborted it, completed the boot
and returned to my original problem - the lsscsi shows the new disk but it's not detected as a device.

So, how to proceed from here: what prevents the OS from detecting a disk as a device?

TIA,
kaza.

Brains · 12-23-2018, 03:31 PM

In Windows, a disk does not get assigned a drive letter till it is initialized. This could be how Fedora handles new disks. Have you checked Fedora's logical volume management.
You may need to partition and format the drive from another computer that can access 4TB drive either by attaching it with a USB adapter or installing directly into a desktop. It don't matter which type of file system it is formatted with if this is what it takes to initialize it, once visible by Fedora the file system can be changed.

kaza · 12-23-2018, 03:48 PM

After rebooting again and going to the BIOS utility I noticed that I can't initialize this disk
(neither others because they're pard of RAID arrays) so I'm beginning to think I might have a hardware problem.
The hour is late now, will continue tomorrow.

There are LEDs headers on the controller and I think some of them migh form "error code",
need to look again at the card doc.

As for initializing in other PC - I don't have any other.

Any ideas, meanwhile?

Brains · 12-23-2018, 03:49 PM

Try booting the unit with Ubuntu live or similar to see if it can see the drive.

syg00 · 12-23-2018, 04:56 PM

Did the controller utility add the new disk to one of the existing RAID arrays ?. Would account for the longer initialization.
Lets see a rerun of all of the output in posts #1 and #2.

kaza · 12-23-2018, 08:32 PM

No, the new disk wasn't added to the arrays. And I was wrong about old disks not being
possible to initialize - they are, although I, of corse, don't do that as I'll wipe
my system.

The "lsscsi stills gives the same:

Code:

[0:0:0:0]    cd/dvd  HL-DT-ST DVDRAM GH24NSD1  LG00  /dev/sr0 
[6:0:3:0]    tape    HP       Ultrium 1-SCSI   N2CG  /dev/st0 
[7:0:0:0]    disk    ICP      KA_RAID_0        V1.0  /dev/sda 
[7:0:1:0]    disk    ICP      KA_RAID_1        V1.0  /dev/sdb 
[7:1:0:0]    disk    FUJITSU  MAU3147RC        0104  -        
[7:1:1:0]    disk    FUJITSU  MBA3147RC        HPF1  -        
[7:1:2:0]    disk    TOSHIBA  MK2001TRKB       0105  -        
[7:1:3:0]    disk    TOSHIBA  MK2001TRKB       0105  -        
[7:1:4:0]    disk    TOSHIBA  MG03SCA400       5702  -

and it's still not i "/dev":

Code:

ls /dev | grep sd
brw-rw----. 1 root disk             8,   0 Dec 24 2018 04:09:34 sda
brw-rw----. 1 root disk             8,   1 Dec 24 2018 04:09:37 sda1
brw-rw----. 1 root disk             8,   2 Dec 24 2018 04:09:34 sda2
brw-rw----. 1 root disk             8,  16 Dec 24 2018 04:09:34 sdb
brw-rw----. 1 root disk             8,  17 Dec 24 2018 04:09:34 sdb1

and the dmesg:

Code:

[    2.724463] aacraid 0000:07:0e.0: 64 Bit DAC enabled
[    2.724779] [drm] ib test on ring 0 succeeded in 0 usecs
[    2.882477] scsi host7: aacraid
[    2.883047] scsi 7:0:0:0: Direct-Access     ICP      KA_RAID_0        V1.0 PQ: 0 ANSI: 2
[    2.883370] scsi 7:0:1:0: Direct-Access     ICP      KA_RAID_1        V1.0 PQ: 0 ANSI: 2
[    2.886937] usb 1-6: New USB device found, idVendor=03f0, idProduct=8904
[    2.886939] usb 1-6: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[    2.886941] usb 1-6: Product: Deskjet 6940 series
[    2.886942] usb 1-6: Manufacturer: HP
[    2.886944] usb 1-6: SerialNumber: MY87ICS2F004QF
[    2.905930] scsi 7:1:0:0: Direct-Access     FUJITSU  MAU3147RC        0104 PQ: 0 ANSI: 3
[    2.907450] scsi 7:1:1:0: Direct-Access     FUJITSU  MBA3147RC        HPF1 PQ: 0 ANSI: 5
[    2.910158] scsi 7:1:2:0: Direct-Access     TOSHIBA  MK2001TRKB       0105 PQ: 0 ANSI: 5
[    2.912038] scsi 7:1:3:0: Direct-Access     TOSHIBA  MK2001TRKB       0105 PQ: 0 ANSI: 5
[    2.913871] scsi 7:1:4:0: Direct-Access     TOSHIBA  MG03SCA400       5702 PQ: 0 ANSI: 5
[    2.992167] sd 7:0:0:0: Attached scsi generic sg1 type 0
[    2.992346] sd 7:0:1:0: Attached scsi generic sg2 type 0
[    2.992485] scsi 7:1:0:0: Attached scsi generic sg3 type 0
[    2.992618] scsi 7:1:1:0: Attached scsi generic sg4 type 0
[    2.992755] scsi 7:1:2:0: Attached scsi generic sg5 type 0
[    2.992917] scsi 7:1:3:0: Attached scsi generic sg6 type 0
[    2.993793] scsi 7:1:4:0: Attached scsi generic sg7 type 0
[    2.993866] sd 7:0:0:0: [sda] 573030400 512-byte logical blocks: (293 GB/273 GiB)
[    2.993877] sd 7:0:0:0: [sda] Write Protect is off
[    2.993879] sd 7:0:0:0: [sda] Mode Sense: 12 00 10 08
[    2.993895] sd 7:0:0:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA
[    2.994070] sd 7:0:1:0: [sdb] Very big device. Trying to use READ CAPACITY(16).
[    2.994087] sd 7:0:1:0: [sdb] 7801405440 512-byte logical blocks: (3.99 TB/3.63 TiB)
[    2.994101] sd 7:0:1:0: [sdb] Write Protect is off
[    2.994103] sd 7:0:1:0: [sdb] Mode Sense: 12 00 10 08
[    2.994124] sd 7:0:1:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
[    2.994554]  sda: sda1 sda2
[    2.994842] sd 7:0:0:0: [sda] Attached SCSI removable disk
[    2.995463] sd 7:0:1:0: [sdb] Very big device. Trying to use READ CAPACITY(16).
[    3.023120] clocksource: Switched to clocksource tsc
[    3.038506]  sdb: sdb1
[    3.038744] sd 7:0:1:0: [sdb] Very big device. Trying to use READ CAPACITY(16).
[    3.038830] sd 7:0:1:0: [sdb] Attached SCSI removable disk

syg00 · 12-23-2018, 10:04 PM

Ah sorry, my bad - I didn't realise the new disk was in the lsscsi listing.
I'm guessing the udev rule(s) is missing that device - the /dev/sd? device is added by udev. You'll need to find out why, and get attributes for the new disk. Have a read of this for some things to try.