LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices

Reply
 
Search this Thread
Old 05-25-2009, 04:02 PM   #1
MQMan
Member
 
Registered: Jan 2004
Location: Los Angeles
Distribution: Slack64 13.37
Posts: 535

Rep: Reputation: 36
highmem64g Kills LVM


I've just upgraded my box, from 4G to 8G, and so recompiled the "generic" kernel, and modules, to support this.

The only option I changed, from the config that was shipped with Slack 12.2, was to change High Memory Support, from 4GB to 64GB. Nothing else.

After compiling, installing, creating a new initrd, and running lilo, I rebooted. A quick display showed me that the system was now seeing all the memory. Great, or so I thought.

Now, none of my LVM volumes show up. Running pvdisplay, or lvdisplay, returns nothing. I tried to re-run the commands from rc.S, vgscan, and vgchange, which both completed without errors. But still no volumes showed up. Also, I couldn't find any errors in any logs, that might account for this.

After resetting my system back to it's "original" state, the volumes now are available again.

What could have caused this, and what can I check, next time I boot into the 64GB system to see if I can spot what went wrong.

Cheers.
 
Old 05-25-2009, 04:49 PM   #2
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 12,222

Rep: Reputation: 1019Reputation: 1019Reputation: 1019Reputation: 1019Reputation: 1019Reputation: 1019Reputation: 1019Reputation: 1019
Not that I don't believe you, but it might be worth diff'ing those config files.
 
Old 05-25-2009, 05:00 PM   #3
MQMan
Member
 
Registered: Jan 2004
Location: Los Angeles
Distribution: Slack64 13.37
Posts: 535

Original Poster
Rep: Reputation: 36
Code:
root@The-Tardis:/usr/src/linux# diff .config /boot/config
4c4                                                                     
< # Sun May 24 14:21:24 2009                                            
---                                                                     
> # Thu Nov 20 22:04:22 2008                                            
222d221                                                                 
< CONFIG_X86_CMPXCHG64=y
250,251c249,250
< # CONFIG_HIGHMEM4G is not set
< CONFIG_HIGHMEM64G=y
---
> CONFIG_HIGHMEM4G=y
> # CONFIG_HIGHMEM64G is not set
254d252
< CONFIG_X86_PAE=y
268c266
< CONFIG_RESOURCES_64BIT=y
---
> # CONFIG_RESOURCES_64BIT is not set
1583d1580
< CONFIG_I2O_EXT_ADAPTEC_DMA64=y
3670a3668,3681
> CONFIG_DMADEVICES=y
>
> #
> # DMA Devices
> #
> CONFIG_INTEL_IOATDMA=m
> CONFIG_DMA_ENGINE=y
>
> #
> # DMA Clients
> #
> CONFIG_NET_DMA=y
> CONFIG_DMATEST=m
> CONFIG_DCA=m
4091a4103
> CONFIG_LGUEST=m
BTW I forgot to say, that all the LVM volumes are on an LSI megaRaid card, which is recognised as SCSI.

Code:
May 24 23:00:33 The-Tardis kernel: megaraid: fw version:[N661] bios version:[1.01]
May 24 23:00:33 The-Tardis kernel: scsi4 : LSI Logic MegaRAID driver

May 24 23:00:33 The-Tardis kernel: scsi 4:4:0:0: Direct-Access     MegaRAID LD 0 RAID5 1430G N661 PQ: 0 ANSI: 2
May 24 23:00:33 The-Tardis kernel: sd 4:4:0:0: [sda] 2930307072 512-byte hardware sectors (1500317 MB)
May 24 23:00:33 The-Tardis kernel: sd 4:4:0:0: [sda] Write Protect is off
May 24 23:00:33 The-Tardis kernel: sd 4:4:0:0: [sda] 2930307072 512-byte hardware sectors (1500317 MB)
May 24 23:00:33 The-Tardis kernel: sd 4:4:0:0: [sda] Write Protect is off
May 24 23:00:33 The-Tardis kernel:  sda: sda1
May 24 23:00:33 The-Tardis kernel: sd 4:4:0:0: [sda] Attached SCSI disk
May 24 23:00:33 The-Tardis kernel: sd 4:4:0:0: Attached scsi generic sg1 type 0
Cheers.

Last edited by MQMan; 05-25-2009 at 06:07 PM.
 
Old 05-25-2009, 06:43 PM   #4
gargamel
Senior Member
 
Registered: May 2003
Distribution: Slackware, SLAX, OpenSuSE
Posts: 1,595

Rep: Reputation: 142Reputation: 142
Are you sure, that your initrd has the modules required for RAID and LVM support?

You can check this by looking into the files under /boot/intrd-tree. Or you can just download a very nice script written by Eric Hameleers, called mkinitrd-command-generator.sh. This script analyses your system and makes a (usually correct) suggestion what your mkinitrd command should look like. Very helpful!

mkinitrd-command-generator.sh

Just compare the output with your mkinitrd command.

gargamel
 
Old 05-25-2009, 06:48 PM   #5
MQMan
Member
 
Registered: Jan 2004
Location: Los Angeles
Distribution: Slack64 13.37
Posts: 535

Original Poster
Rep: Reputation: 36
The initrd does not have, and has never had the modules for RAID or LVM, as they are not needed.

My boot disk is not LVM, and the RAID is a "real" hardware RAID.

Here's what the script suggests:
Code:
mkinitrd -c -k 2.6.27.7-smp -f xfs -r /dev/hda2 -m usbhid:ehci-hcd:ohci-hcd:xfs -o /boot/initrd.gz
My usual command is: mkinitrd -c -k 2.6.27.7-smp -m xfs

BTW, the link should be this.

Cheers.

Last edited by MQMan; 05-25-2009 at 06:57 PM.
 
Old 05-31-2009, 07:23 PM   #6
MQMan
Member
 
Registered: Jan 2004
Location: Los Angeles
Distribution: Slack64 13.37
Posts: 535

Original Poster
Rep: Reputation: 36
OK, here's what seems to be the problem, but can anyone explain why:
Code:
root@The-Tardis:~# vgscan --mknodes --verbose
    Wiping cache of LVM-capable devices
    Wiping internal VG cache
  Reading all physical volumes.  This may take a while...
    Finding all volume groups
  /dev/sda1: Checksum error
    Finding all logical volumes
This *ONLY* happens when I boot the kernel that has the HIGHMEM64G set. When I reboot back into the "generic" kernel, that has HIGHMEM4G, then vgscan works fine:
Code:
root@The-Tardis:~# vgscan --mknodes --verbose
    Wiping cache of LVM-capable devices
    Wiping internal VG cache
  Reading all physical volumes.  This may take a while...
    Finding all volume groups
    Finding volume group "raid_vg"
  Found volume group "raid_vg" using metadata type lvm2
    Finding all logical volumes
And, yes, /dev/sda1 is where all my LVMs reside.

Cheers,
Eddie
 
Old 06-01-2009, 05:48 AM   #7
GazL
Senior Member
 
Registered: May 2008
Posts: 3,380

Rep: Reputation: 912Reputation: 912Reputation: 912Reputation: 912Reputation: 912Reputation: 912Reputation: 912Reputation: 912
The checksum error seems to suggest the lvm tools are having a problem validating the lvm metadata on that partition. As it still works under your old non HIGHMEM64/PAE kernel then that seems to suggest that there's nothing wrong with the metadata or the raid device.

if you want to completely rule out a device/driver issue, you might want to try something like copying the first MB or so of raw data from /dev/sda1 to a file using dd under each kernel and compare the 2 files to look for any corruption. Probably need to have the volume group deactivated to get an exact comparison.

I'd also be inclined to investigate updating to the latest version of the lvm2 sources from redhat and see if that resolves it before going any further. At first sight, it looks like you may have found an honest-to-goodness bug/incompatibility, but the upstream developers aren't going to be interested unless you're on the latest version. I suppose it could also be a kernel bug. so that's something to keep in mind too.

Worst case scenario, it could even be some dodgy implementation of PAE in your hardware/bios/firmware or whatever, but it's probably wise to leave the 'Sky is falling!' scenarios until all the other possibilities have been ruled out.

I think it's fair to say, you've probably stepped outside the knowledge levels of most of us who dwell here and are firmly in the LKML territory with this one.

Hope these suggestions help. Best of luck.

Last edited by GazL; 06-01-2009 at 05:50 AM.
 
Old 06-01-2009, 01:13 PM   #8
MQMan
Member
 
Registered: Jan 2004
Location: Los Angeles
Distribution: Slack64 13.37
Posts: 535

Original Poster
Rep: Reputation: 36
Quote:
Originally Posted by GazL View Post
I think it's fair to say, you've probably stepped outside the knowledge levels of most of us who dwell here and are firmly in the LKML territory with this one.
Yeah, that's what I was thinking.

I actually signed up to the LVM mailing list yesterday, and posted it there, so hopefully someone there can shed some light, or at least a "plan of attack".

And I'm with you, in hoping it's not a PAE issue.

BTW what's the weather like in England at the moment. I'm coming back home for a couple of weeks vacation in a couple of days time.

Cheers,
Eddie
 
Old 06-01-2009, 02:40 PM   #9
GazL
Senior Member
 
Registered: May 2008
Posts: 3,380

Rep: Reputation: 912Reputation: 912Reputation: 912Reputation: 912Reputation: 912Reputation: 912Reputation: 912Reputation: 912
If they find something, please drop us a follow-up note in here. This one has raised my curiosity.


UK weather has been pretty damn wonderful over the last few days. It'll probably change by the time you get here though, so don't get your hopes up too much!
 
Old 06-03-2009, 08:19 PM   #10
MQMan
Member
 
Registered: Jan 2004
Location: Los Angeles
Distribution: Slack64 13.37
Posts: 535

Original Poster
Rep: Reputation: 36
Quote:
Originally Posted by GazL View Post
UK weather has been pretty damn wonderful over the last few days. It'll probably change by the time you get here though, so don't get your hopes up too much!
Ha, bloody typical.

I never got any replies on the lvm mailing list.

So, I went ahead and downloaded the latest version, 2.02.47, which is now a combined device-mapper and lvm tools. After cobbling together a suitable slackbuild script, and building the package, I replaced the existing device-mapper and lvm packages with this, crossed fingers, and re-booted.

Yaaaay. It works. I haven't got time to see what the differences are, in the checksum calculation, as I'm in the middle of packing, for a morning flight tomorrow.

Cheers,
Eddie

Last edited by MQMan; 06-03-2009 at 10:42 PM.
 
Old 06-04-2009, 02:37 AM   #11
GazL
Senior Member
 
Registered: May 2008
Posts: 3,380

Rep: Reputation: 912Reputation: 912Reputation: 912Reputation: 912Reputation: 912Reputation: 912Reputation: 912Reputation: 912
Glad you got it working Eddie. Have a nice flight and a good time in England.
 
Old 06-27-2009, 04:20 PM   #12
MQMan
Member
 
Registered: Jan 2004
Location: Los Angeles
Distribution: Slack64 13.37
Posts: 535

Original Poster
Rep: Reputation: 36
OK, now I'm back from vacation, I've had time to check further. It seems I was a little premature in saying that the latest version of LVM2 fixed it. It didn't.

What it did do, was mount the volumes. Any file I tried to read, came back corrupted. I was able to confirm this by comparing the md5sum of a bunch of files, where I had the checksums from before I made the switch.

Or, again, it gives a checksum error with vgscan.

Going back to my HIGHMEM4G kernel, the md5sums now match again.

I guess it's back to the LVM mailing list again.

Cheers,
Eddie
 
Old 06-28-2009, 04:20 AM   #13
GazL
Senior Member
 
Registered: May 2008
Posts: 3,380

Rep: Reputation: 912Reputation: 912Reputation: 912Reputation: 912Reputation: 912Reputation: 912Reputation: 912Reputation: 912
Welcome back.

If you're still running 2.6.27.7 it might be worth trying a newer kernel. Can't think of anything else to suggest.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
4Gb not seen on Slack 12.2 with HIGHMEM64G set brooky9999 Slackware 28 05-27-2009 04:52 AM
LVM and multiple iSCSI disks performance of LVM xxx_anuj_xxx Linux - Server 4 05-01-2008 12:26 PM
LXer: Managing LVM with the LVM Manager LXer Syndicated Linux News 0 03-16-2008 11:10 AM
Picasa kills X! R_Shackleford Slackware 2 12-01-2007 09:22 AM
LXer: Back Up (And Restore) LVM Partitions With LVM Snapshots LXer Syndicated Linux News 0 04-17-2007 11:16 AM


All times are GMT -5. The time now is 08:34 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration