LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Server (https://www.linuxquestions.org/questions/linux-server-73/)
-   -   Root on lvm (https://www.linuxquestions.org/questions/linux-server-73/root-on-lvm-703174/)

serge 02-08-2009 02:11 PM

Root on lvm
 
I am in the process of building a two node cluster.
Based on gentoo (mainly for the extensive customization)
I decided to use LVM for everything except /boot
Currently I decided to use 5 partitions on a 80GB disk
/dev/sda1 /boot 32mb
/dev/vg/swap 12GB
/dev/vg/root 4GB
/dev/vg/data 21GB
/dev/vg/drbd 21GB

The issue now is that when I try to boot I get a kernel panic with the error:
VFS: unable to mount root fs on unknown block (0,0)
I compiled the kernel with LVM support and added dolvm to the kernel line in grub.conf
I read something about needing to create an initramfs with lvmcreate_initrd, but could not find it.
I have a log of all the steps I took during the install (it isn't really clear, since it is mostly for personal reference)

grub.conf:
Code:

title Gentoo Linux 2.6.26-9
root (hd0,0)
kernel /boot/kernel-hardened-2.6.26-gentoo-r9 root=/dev/mapper/vg-root

Steps to install:
Code:

At gentoo boot prompt:
boot: gentoo dosshd dolvm docache
#passwd
#ifconfig eth0
Use IP to connect with putty
Set the date
#/etc/init.d/ntp-client start
#dd if=/dev/zero of=/dev/sda bs=1M count=1
#fdisk /dev/sda
:w
#fdisk /dev/sda
:n
:p
:1
:enter
:+32M
:t
:83
:n
:p
:2
:enter
:enter
:t
:8e
:w
#pvcreate /dev/sda2
#vcreate vg /dev/sda2
#lvcreate –L12288M –nswap vg
#lvcreate –L4096M –nroot vg
#lvcreate –l50%FREE –ndata vg
#lvcreate –l100%FREE –ndrbd vg
#mkfs.ext2 /dev/sda1
#mkswap /dev/vg/swap
#mkfs.ext3 /dev/vg/root
#mkfs.ext3 /dev/data
#mount /dev/vg/root /mnt/gentoo
#mkdir /mnt/gentoo/boot
#mount /dev/sda1 /mnt/gentoo/boot
#mkdir -p /mnt/gentoo/mnt/data
#swapon /dev/vg/swap
#mount /dev/vg/data /mnt/gentoo/mnt/data
#cd /mnt/gentoo/mnt/data
#wget ftp://ftp.snt.utwente.nl/pub/os/linux/gentoo/releases/amd64/2008.0/stages/stage3-amd64-2008.0.tar.bz2
#wget ftp://ftp.snt.utwente.nl/pub/os/linux/gentoo/releases/amd64/2008.0/stages/stage3-amd64-2008.0.tar.bz2.DIGESTS
#md5sum –c stage3-amd64-2008.0.tar.bz2.DIGESTS
#tar xvjpf stage3-amd64-2008.0.tar.bz2 -C /mnt/gentoo/
#wget ftp://ftp.snt.utwente.nl/pub/os/linux/gentoo/snapshots/portage-20090204.tar.bz2
#wget ftp://ftp.snt.utwente.nl/pub/os/linux/gentoo/snapshots/portage-20090204.tar.bz2.md5sum
#md5sum -c portage-20090204.tar.bz2.md5sum
#tar xvjpf portage-20090204.tar.bz2 -C /mnt/gentoo/usr
#nano -w /mnt/gentoo/etc/make.conf
CFLAGS="-march=nocona -O2 -pipe -fomit-frame-pointer"
MAKEOPTS=”-j5 –l6”
CXXFLAGS="${CFLAGS}"
#mirrorselect -i -o >> /mnt/gentoo/etc/make.conf
#cp -L /etc/resolv.conf /mnt/gentoo/etc/
#mount -t proc none /mnt/gentoo/proc
#mount -o bind /dev /mnt/gentoo/dev
#chroot /mnt/gentoo /bin/bash
#env-update
#source /etc/profile
#export PS1="(chroot) $PS1"
#emerge --sync
#emerge --oneshot portage
#find /etc -iname ._cfg????_*
-- diff file ._cfg????_*file
Update the new files as desirable
#ln -snf /usr/portage/profiles/hardened/amd64 /etc/make.profile
#nano /etc/make.conf
USE="-3dfx -3dnow -X -Xaw3d -a52 -aac -aalib -accessibility acl acpi -adabas -adns -afs -aim -alsa -altivec -ao apache2 apm arts -atm -audiofile avahi bash-completion -bcmath -berkdb -bidi bindist -birdstep -blas -bluetooth branding bsf bzip2 -cairo -calendar -canna caps -cdb -cddb -cdinstall -cdparanoia -cdr cgi -chasen -cjk clamav cracklib crypt cscope -css ctype -cups curl curlwrappers -cvs cxx -db2 -dbase -dbm -dbmaker dbus -dbx -debug -dedicated -dga -directfb -djvu -doc -dri -dts -dv -dvb -dvd -dvdr dvdread -eds -emacs -emboss -empress -empress-bcs -encode -enscript -esd -esoob -evo -examples -exif expat -fam fastcgi fbcon -ffmpeg -fftw -firebird -firefox -flac -flatfile -fltk -fontconfig -foomaticdb -fortran -freetds -freewnn -frontbase ftp -gcj gd gd-external -gdbm geoip -ggi gif -gimp -ginac -glut -gmp -gnome -gnome-keyring -gnuplot -gnustep gnutls -gphoto2 gpm -gps -graphviz gzip -gsl -gstreamer -gtk -gtkhtml -guile hal hardened -hddtemp -hdf5 -htmlhandbook -ibm iconv icq -icu -idn -ieee1394 imagemagick imap -imlib inifile innodb -interbase -iodbc -ipod -ipv6 -jabber -jack java java5 java6 javascript -jbig -jingle -joystick jpeg jpeg2k -kontact -kde -kdeenablefinal -kdehiddenvisibility -kdeprefix kerberos -kolab krb4 -ladcca -ladspa -lame -lapack -lash -latex -lcms -ldap -libcaca -libedit -libgda -libnotify -libsamplerate -libwww -lirc -lm_sensors -lua lzo -m17n-lib -mad -maildir mailwrapper -matroska -matrox mbox -mcve memlimit -mhash -migemo -mikmod milter mime -minimal mmap mmx -mng -modplug -mono -motif -mozilla -mp3 -mpeg mpi -mplayer -msn -msql -mssql -mule -multilib -musepack -musicbrainz mysql mysqli -nas ncurses -neXt netboot -netcdf -networkmanager nis -nls -nntp -nocd nptl -nsplugin -ocaml -ocamlopt -oci8 -oci8-instant-client -odbc -offensive -ofx -ogg -old-linux -openal -openexr -opengl openmp -oracle -osc -oscar -oss pch -pcmcia pcntl pcre -pda pdf perl php -plasma -plotutils png -portaudio posix postgres -ppds -prelude -pulseaudio python -qdbm -qmail-spp -qt3 -qt4 -quicktime radius -raw rdesktop readline -recode -rss -ruby samba -sapdb sasl -savedconfig -scanner -sdl -seamonkey session sharedext sharedmem -shorten simplexml skey -slang slp -smartcard smp -sndfile snmp soap sockets -socks5 -solid -source -sox -speex spell spl -sqlite -sqlite3 sse sse2 ssl startup-notification subversion suid -svg -svga -sybase -sybase-ct symlink syslog sysvipc -szip taglib -tcl tcpd -theora threads tidy -tiff -timidity -tk tokenizer truetype -uclibc unicode usb -v4l -v4l2 -vcd vhosts -videos vim-syntax -vnc -voodoo3 -vorbis -wavpack -wddx -wifi -win32codecs -wmf -wxwindows -x264 xattr -xcb -xcomposite -xemacs -xface -xft -xine -xinerama xinetd xml xmlrpc -xosd -xpm -xprint -xscreensaver -xsl -xulrunner -xv -xvid -yahoo -yaz zeroconf zlib"
#nano /etc/locale.gen
en_US ISO-8859-1
en_US.UTF-8 UTF-8
#locale-gen
#cp /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime
#emerge hardened-sources
#find /etc -iname ._cfg????_*
-- diff file ._cfg????_*file
Update the new files as desirable
#cd /usr/src/linux
#emerge pciutils
#make menuconfig
Processor type and features
        Subarchitecture Type
                [X] Support for ScaleMP vSMP (press space or enter)
        Processor family (Core 2/newer Xeon)
        (4) Maximum number of CPUs
        [ ] SMT (Hyperthreading) scheduler support
        Preemption Model (No Forced Preemption (Server)
        [ ]  AMD MCE features
        [ ] Numa Memory Allocation and Scheduler Support (EXPERIMENTAL)
        [ ] Support for suspend on SMP and hot-pluggable CPUs
Power management options
        [ ] Suspend to RAM and standby
        [ ] Hibernation (aka 'suspend to disk'
        [*] ACPI (Advanced Configuration and Power Interface) Support  --->
                < >  AC Adapter
                < >  Battery
                < >  Fan
        CPU Frequency scaling  --->
                Default CPUFreq governor (performance
                        (X) performance
                < >  'userspace' governor for userspace frequency scaling
                < >  'ondemand' cpufreq policy governor
                < >  'conservative' cpufreq governor
                < >  AMD Opteron/Athlon64 PowerNow!
                <*>  Intel Pentium 4 clock modulation
Executable file formats / Emulations
        [ ] IA32 Emulation
Networking
        Networking options
                [ ]    IP: DHCP support
                <*>  Large Receive Offload (ipv4/tcp
                < >  The IPv6 protocol  --->
                [*] Network packet filtering framework (Netfilter)
                <*>  IP virtual server support (EXPERIMENTAL)
                        [*]  TCP load balancing support
                        [*]  UDP load balancing support
                        <*>  round-robin scheduling
                        <*>  weighted round-robin scheduling
                        <*>  least-connection scheduling
                        <*>  weighted least-connection scheduling
                        < >  shortest expected delay scheduling (NEW)
                        <*>  FTP protocol helper
                [*] Network packet filtering framework (Netfilter)
                        [*]  Advanced netfilter configuration
                                Core Netfilter Configuratio
                                        <*> Netfilter NFQUEUE over NFNETLINK interface
                                        <*> Netfilter LOG over NFNETLINK interface
                                        <*> Netfilter connection tracking support
                                        <*>  FTP protocol support
                                        <*>  NetBIOS name service protocol support
                                        <*> Netfilter Xtables support (required for ip_tables)
                                        <*>  "CLASSIFY" target support
                                        <*>  "MARK" target support
                                        <*>  "NFQUEUE" target Suppor
                                        <*>  "NFLOG" target support
                                        <*>  "connbytes" per-connection counter match support
                                        <*>  "connlimit" match support"
                                        <*>  "connmark" connection mark match support
                                        <*>  "conntrack" connection tracking match support
                                        <*>  "helper" match support
                                        <*>  "iprange" address range match support
                                        <*>  "length" match support
                                        <*>  "limit" match support
                                        <*>  "mac" address match support
                                        <*>  "mark" match support
                                        <*>  "owner" match support
                                        <*>  "multiport" Multiple port match support
                                        <*>  "pkttype" packet type match support
                                        <*>  "quota" match support
                                        <*>  "rateest" match support       
                                        <*>  "realm" match support
                                        <*>  "state" match support
                                        <*>  "string" match support
                                        <*>  "time" match support
                                IP: Netfilter Configuration
                                        <*> IP tables support (required for filtering/masq/NAT)
                                        <*>  "recent" match support
                                        <*>  "state" match support
                                        <*>  "addrtype" address type match support
                                        <*>  Packet filtering
                                        <*>    REJECT target support
                                        <*>  LOG target support
                                        <*>  ULOG target support
Device Drivers
        <*> Connector - unified userspace <-> kernelspace linker
        [*] Block device
                < >  Normal floppy disk support
        <*> ATA/ATAPI/MFM/RLL support
                < >    Include IDE/ATA-2 DISK support
                < >    AMD and nVidia IDE support
                < >    ATI IXP chipset IDE support
                < >    PROMISE PDC202{68|69|70|71|75|76|77} support
        SCSI device support
                < > SCSI disk support
                < > SCSI CDROM support
                < > SCSI generic support
                [ ] Verbose SCSI error reporting
                SCSI Transports
                        < > FiberChannel Transport Attributes
                [*] SCSI low-level driver
                        <*>  3ware 5/6/7/8xxx ATA-RAID suppor
                        < >  Adaptec AIC79xx U320 support
        <*> Serial ATA (prod) and Parallel ATA (experimental) drivers
                < >    ServerWorks Frodo / Apple K2 SATA support
                < >    NVIDIA SATA support
                < >    Silicon Image SATA support
                < >    VIA SATA support
        [*] Multiple devices driver support (RAID and LVM)
        [ ] Fusion MPT device support
        IEEE 1394 (FireWire) support
                < > Stable FireWire stack
        [ ] Macintosh device driver
        [*] Network device suppor
                <*>  Bonding driver support
                < >  Universal TUN/TAP device driver support
                [ ]  Ethernet (10 or 100Mbit)
                [*]  Ethernet (1000 Mbit
                        < >  Intel(R) PRO/1000 Gigabit Ethernet support
                        <*>  Intel(R) 82575 PCI-Express Gigabit Ethernet support
                        < >  Broadcom Tigon3 support
                        < >  Broadcom NetXtremeII support
                [ ]  Ethernet (10000 Mbit
        Character devices
                < >  AMD HW Random Number Generator support
        Multimedia devices
                [ ] DAB adapters
        Sound
                < > Sound card support
        [*] USB support
                < >    USB Printer support
File systems
        < > Reiserfs support
        <*> GFS2 file system support
        <*>  GFS2 DLM locking module
        [*] Network File Systems
                < >  NFS file system support
                < >  NFS server support
                <*>  CIFS support (advanced network filesystem, SMBFS successor)
                [*]    CIFS extended attributes
                [*]      CIFS POSIX Extension
Partition Types
        [*] Advanced partition selection
#make
#cp arch/x86_64/boot/bzImage /boot/kernel-hardened-2.6.26-gentoo-r9
#emerge gentoolkit
#nano -w /etc/fstab
/dev/sda1              /boot          ext2            defaults,noatime        1 2
/dev/vg/root            /              ext3            noatime                0 1
/dev/vg/swap            none            swap            sw                      0 0
#nano -w /etc/conf.d/hostname
HOSTNAME="node01"
#emerge ifenslave
#nano -w /etc/conf.d/net
config_eth0=( "192.168.1.1 netmask 255.255.255.0 brd 192.168.1.255" )
routes_eth0=( "default via 192.168.1.254" )

slaves_bond0=("eth2 eth3")
config_bond0=("10.0.0.1 netmask 255.255.255.248 brd 10.0.0.7")
depend_bond0() {
        need net.eth2 net.eth3
}
#cd /etc/init.d
#ln -snf net.lo net.eth0
#ln -snf net.lo net.eth2
#ln -snf net.lo net.eth3
#ln -snf net.lo net.bond0
#rc-update add net.lo default
#rc-update add net.eth0 default
#rc-update add net.eth2 default
#rc-update add net.eth3 default
#nano -w /etc/hosts
192.168.1.1    node01.fonville-it.nl node01
192.168.1.2    node02.fonville-it.nl node02
#passwd
#nano -w /etc/conf.d/clock
CLOCK="local"
TIMEZONE="Europe/Amsterdam"
#emerge syslog-ng
#rc-update add syslog-ng default
#emerge vixie-cron
#rc-update add vixie-cron default
#emerge grub-static
#emerge lvm2
#nano /boot/grub/grub.conf
title Gentoo Linux 2.6.26-9
root (hd0,0)
kernel /boot/kernel-hardened-2.6.26-gentoo-r9 root=/dev/mapper/vg-root
#grep -v rootfs /proc/mounts > /etc/mtab
#grub-install --no-floppy /dev/sda

Thanks in advance

serge 02-09-2009 04:30 AM

I changed a couple of things.
Enable SCSI disk support, otherwise sda* devices weren't available
Device Drivers
SCSI device support
<*> SCSI disk support
I found lvm2create_initrd at HOWTO_Install_Gentoo_on_an_LVM2_root_partition a link to lvm2create_initrd from redhat.
I saved lvm2create_initrd in /sbin and chmodded it +x
and ran 'lvm2create_initrd -M gentoo 2.6.26-gentoo-r9'
I edited grub.conf to:
Code:

kernel /boot/kernel-hardened-2.6.26-gentoo-r9 root=/dev/ram0 lvm2root=/dev/vg/root
initrd /boot/initrd-2.6.26-gentoo-r9.gz

Now I get an error that the ramdisk isn't big enough, that should be resolved by either specifying the ramdisk size on the kernel line or recompiling the kernel (I prefer the latter)

I changed:
Code:

#rc-update add net.lo default
#rc-update add net.eth0 default
#rc-update add net.eth2 default
#rc-update add net.eth3 default
#rc-update add net.bond0 default

to:
Code:

#rc-update add net.eth0 boot
#rc-update add net.eth2 boot
#rc-update add net.eth3 boot
#rc-update add net.bond0 boot

I am not entirely sure I the scripts for starting network interfaces should be in boot instead (but I didn't start the system fully yet)

serge 02-09-2009 02:15 PM

After setting up the ramdisk image, I got an error about not being able to find libncurses.
This was solved by
Code:

#cd /boot
#mkdir tmp
#gzip -d initrd-2.6.26-gentoo-r9.gz
#mount initrd-2.6.26-gentoo-r9 tmp -o loop
#cd tmp
#mkdir sys
#mv lib/* lib64
#rmdir lib
#ln -s lib64 lib
#cd ..
#umount tmp
#gzip initrd-2.6.26-gentoo-r9

Unfortunately, know I get an error that a mount point for sysfs could not be found, pivot root: file not found and it drops to the lvm shell.
When I type exit the system starts up normally.
I tried editing lvm2create_initrd and init (inside the ramdisk) but either did not help.

Any help is greatly appreciated


All times are GMT -5. The time now is 06:01 AM.