LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 01-27-2011, 02:16 PM   #1
vila99
LQ Newbie
 
Registered: Jan 2011
Posts: 4

Rep: Reputation: 0
Device Mapping sdaj, sdak


Hi folks!

I am pretty new to this forum and linux server administration.

Recently I have added a new LUN to a Oracle RAC. The new LUN has been mapped to the devices sdaj and sdak by the server. After I reboot the server, redhat automatically re-align all devices and LUNs. This causes all the disk mount problem.

So, any one know how to identify which device map to which LUN; or which device is equivalent to which (both devices point to the same LUN).

The OS of the subject server are Redhat 4.5

Please guide me to the right direction to fix the problem.

Thank you,
Wei
 
Old 01-27-2011, 03:39 PM   #2
tommylovell
Member
 
Registered: Nov 2005
Distribution: Fedora, Redhat
Posts: 372

Rep: Reputation: 101Reputation: 101
Unhappy

I'm not really conversant in RAC, or ocfs2, but I think what you are looking at is at a lower level that that.

On Red Hat 4.5 you have DM-Multipath and LVM2, both of which I'm familiar with and we use heavily at my company.

This is a little wordy, so bear with me.

If you do an 'lsscsi', you should see all of your scsi block devices, i.e. /dev/sdaj and /dev/sdak. It won't tell you very much about them though, just that they have been recognized and are there.

If you are running DM-multipath (we do on our RAC systems) the underlying SAN scsi devices are "matched up" (by reading the serial number from each device) and are represented by an 'mpath' device, e.g. /dev/mapper/mpath2.

If you do a 'multipath -ll', you should see those underlying devices mapped to 'mpath' devices.

(There is also the dmsetup command, 'dmsetup ls', that shows the multipath devices (lvm, dm-crypt, dm-multipath, ...) whatever you are using.)

The ext2/3/4 LABEL or the UUID of a device can be used to mount it and eleviates the problem where you mount an explicit device name and the names have "shuffled" on you.

With LVM you have metadata written to the various "volumes" of LVM. When you do a 'pvcreate' on a partition (or full volume), metadata is written to that partition (or full volume); then when you do a 'vgcreate' or 'vgextend', more metatdata is written there; same with 'lvcreate' or 'lvextend'. When LVM is initialized as the system comes up, it scans for that metatdata, again alleviating a dependence on explicit device names.

On a "normal" non-RAC system, the mpath devices can be formatted (preferably with an ext2/3/4 LABEL so you don't have to worry about device names changing) and mounted (I think). A better practice is to use LVM.

With ocfs2 you don't use LVM or mount the devices directly. It does (at least in my shop) use the multipath device (e.g. /dev/mapper/mpath2) but mounts what look like partitions (e.g. /dev/mapper/mpath2p8 on /u08 and /dev/mapper/mpath2p9 on /u19).

Do a 'mount' command to see what you have mounted now.

Sorry I can't tell you much more than that. If you are using ocsf2, you can post another question that is ocfs2 specific and you should find someone that is more authoritative than me.

btw, RHEL4.5 is very old and was pretty buggy. We tried it and then went to RHEL4.6 + a higher level kernel. When Oracle finally supported RHEL5, we went to RHEL5.2 and then RHEL5.4.

Last edited by tommylovell; 01-27-2011 at 03:48 PM.
 
Old 01-27-2011, 04:09 PM   #3
vila99
LQ Newbie
 
Registered: Jan 2011
Posts: 4

Original Poster
Rep: Reputation: 0
Thank you Tommy;

Thank you for your quick reply.

Because RHEL 4.5 is old, we don't have multipath installed. Therefore, your suggestion doesn't work for me.
Do you have any otherway to work around?
 
Old 01-27-2011, 05:12 PM   #4
tommylovell
Member
 
Registered: Nov 2005
Distribution: Fedora, Redhat
Posts: 372

Rep: Reputation: 101Reputation: 101
I can't see (from googling) that ocfs2 does multipathing itself.

So, first question is "are you doing any kind or multipathing, using md for multipathing, or are you doing no multipathing at all?"
Second question is "are you using ocfs2?"

If you supply the output of a 'mount' command (so I can see what and how you have your filesystems mounted) it will help.

btw, DM-multipath was introduced in RHEL4.2. Probably not a good idea to implement it now. Before that md was used to support multipathing but did not perform very well and was not popular.
 
Old 01-28-2011, 10:03 AM   #5
vila99
LQ Newbie
 
Registered: Jan 2011
Posts: 4

Original Poster
Rep: Reputation: 0
Hi Tommy:

I don't believe we use any multipath.

We do use ocfs

Here is the mount result.

/dev/sda7 on / type ext3 (rw)
none on /proc type proc (rw)
none on /dev/pts type devpts (rw,gid=5,mode=620)
usbdevfs on /proc/bus/usb type usbdevfs (rw)
/dev/sda3 on /boot type ext3 (rw)
/dev/sda8 on /home type ext3 (rw)
/dev/sda5 on /opt/oracle type ext3 (rw)
none on /dev/shm type tmpfs (rw)
/dev/sda12 on /tmp type ext3 (rw)
/dev/sda6 on /usr type ext3 (rw)
/dev/sda13 on /data type ext3 (rw)
/dev/sdb8 on /u01 type ocfs (rw,_netdev)
/dev/sdc1 on /u03 type ocfs (rw,_netdev)
rac0:/data/oracle/local on /opt/oracle/local type nfs (rw,addr=167.165.###.###)


Thank you for your help.
 
Old 01-28-2011, 10:42 PM   #6
tommylovell
Member
 
Registered: Nov 2005
Distribution: Fedora, Redhat
Posts: 372

Rep: Reputation: 101Reputation: 101
I'm seeing posts for ocfs2 similar to yours where people are mounting SAN LUNs by device name and from boot to boot the device name is changing. The advice given is "you need to mount by UUID", and "you should not mount by UUID, you should mount by LABEL." (I'm hoping this advice is applicable to ocfs...)

I've done both LABEL= and UUID= mounts with ext3 and ext4 file systems. (And you can LABEL an ext2/3/4 filesystem after it's been formatted. It's not clear if you can add a label to an ocfs filesystem after it's been formatted, but it should have a UUID.)

With ext2/3/4, you can UUID mount with an /etc/fstab entry like:
UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /<mountpoint> ext3 defaults 1 2

or LABEL mount with:
LABEL=label /<mountpoint> ext3 defaults 1 2

You can try using the vol_id command to determine the UUID of the filesystem that you want to mount.

As an example, you should be able to do '/lib/udev/vol_id -u /dev/sdb8' to find the UUID of that FS.
I'm not sure you can use UUID=<uuid> the way you can with ext2/3/4, but I've seen a recommendation that it can by mounted with an fstab entry similar to this,
/dev/disk/by-uuid/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /<mountpoint> ocfs etc...

(An 'ls -l /dev/disk/by-uuid/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' will tell you if this works on RHEL4.5.)

Hope this is going in the right direction. I have no RHEL4 systems to try any of this on.
 
Old 02-01-2011, 03:01 PM   #7
vila99
LQ Newbie
 
Registered: Jan 2011
Posts: 4

Original Poster
Rep: Reputation: 0
Hi Tommy:

Thank you for your help, my problem has been solved by 3rd party vendor.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
HD labeling/device mapping RWTH Linux - Newbie 13 07-26-2010 07:37 PM
device mapping switch james2b Linux - Newbie 4 06-17-2008 06:07 PM
mapping usb device to device file scerenon Linux - Hardware 12 07-20-2006 04:13 PM
physical scsi channel mapping to scsiX device node mapping, how to configure manually drthornt Linux - Hardware 3 02-09-2003 11:50 AM
Memory mapping a device PLEASE HELP!!!! mpalmer Linux - Newbie 1 08-16-2001 03:09 PM


All times are GMT -5. The time now is 12:33 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration