Need help trying to get PXE boot to use NBD for rootfs

dbrazeau · 08-15-2018, 03:56 PM

So I am not sure where I am going wrong. I have all my stuff setup on my server and am able to PXE boot using both an NFS file system and using a file system image with http, but I can't seem to get it to work using a file system image with NBD.

nbd-server config:

Code:

[generic]
[export]
    exportname = /mnt/tftpboot/rootfs.img
    port = 2000
    copyonwrite = false

pxelinux config:

Code:

DEFAULT pxeboot
TIMEOUT 20
PROMPT 0
LABEL pxeboot
        KERNEL vmlinuz
        APPEND initrd=initrd.img ip=enp10s0f1:dhcp root-path=nbd:192.168.1.2:export root=dhcp console=tty0 console=ttyS0,115200n8
ONERROR LOCALBOOT 0

client output:

Code:

[    7.330300] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
[    7.337341] igb: Copyright (c) 2007-2014 Intel Corporation.
[    7.400354] igb 0000:0a:00.0: added PHC on eth0
[    7.404975] igb 0000:0a:00.0: Intel(R) Gigabit Ethernet Network Connection
[    7.411931] igb 0000:0a:00.0: eth0: (PCIe:5.0Gb/s:Width x4) 00:1b:ac:00:42:27
[    7.419151] igb 0000:0a:00.0: eth0: PBA No: Unknown
[    7.424114] igb 0000:0a:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
[    7.441632] 8021q: 802.1Q VLAN Support v1.8
[    7.487602] igb 0000:0a:00.1: added PHC on eth1
[    7.492228] igb 0000:0a:00.1: Intel(R) Gigabit Ethernet Network Connection
[    7.499193] igb 0000:0a:00.1: eth1: (PCIe:5.0Gb/s:Width x4) 00:1b:ac:00:42:28
[    7.506415] igb 0000:0a:00.1: eth1: PBA No: Unknown
[    7.511381] igb 0000:0a:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s)
[    7.520442] igb 0000:0a:00.1 enp10s0f1: renamed from eth1
[    7.534336] igb 0000:0a:00.0 enp10s0f0: renamed from eth0
[  OK  ] Started udev Wait for Complete Device Initialization.
[    7.623234] audit: type=1130 audit(1532747553.622:11): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-udev-settle comm="systemd
" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
         Starting Device-Mapper Multipath Device Controller...
[  OK  ] Started Device-Mapper Multipath Device Controller.
[  OK  ] Reached target Local File Systems (Pre).
[  OK  ] Reached target Local File Systems.
         Starting Create Volatile Files and Directories...
[  OK  ] Started Create Volatile Files and Directories.
[  OK  ] Reached target System Initialization.
[  OK  ] Reached target Basic System.
[    7.773625] IPv6: ADDRCONF(NETDEV_UP): enp10s0f1: link is not ready
[    7.780010] 8021q: adding VLAN 0 to HW filter on device enp10s0f1
[   11.351492] igb 0000:0a:00.1 enp10s0f1: igb: enp10s0f1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   11.362031] IPv6: ADDRCONF(NETDEV_CHANGE): enp10s0f1: link becomes ready
[   15.196203] dracut-initqueue[545]: Warning: No dhcp root-path received for '' trying other interfaces if available
[   91.744264] random: crng init done
[  147.296972] dracut-initqueue[545]: Warning: dracut-initqueue timeout - starting timeout scripts
[  147.868973] dracut-initqueue[545]: Warning: dracut-initqueue timeout - starting timeout scripts
[  148.418432] dracut-initqueue[545]: Warning: dracut-initqueue timeout - starting timeout scripts
[  148.970222] dracut-initqueue[545]: Warning: dracut-initqueue timeout - starting timeout scripts
[  149.523491] dracut-initqueue[545]: Warning: dracut-initqueue timeout - starting timeout scripts
[  150.074389] dracut-initqueue[545]: Warning: dracut-initqueue timeout - starting timeout scripts
[  150.623440] dracut-initqueue[545]: Warning: dracut-initqueue timeout - starting timeout scripts
[  151.173637] dracut-initqueue[545]: Warning: dracut-initqueue timeout - starting timeout scripts
[  151.719875] dracut-initqueue[545]: Warning: dracut-initqueue timeout - starting timeout scripts

No matter what I do I always seem to get the error

Code:

dracut-initqueue[545]: Warning: No dhcp root-path received for '' trying other interfaces if available

as if I am not providing the root-path option (which I am). As a test I was able to use nbd-client to connect to NBD export locally on the server, so as far as I can tell the nbd-server configure is ok and working. Any ideas what I might be doing wrong? Is it possible that the kernel/initrd I'm using doesn't support NBD?

For reference I'm using the kernel and initrd shipped with the Fedora 28 LiveCD image.

jefro · 08-16-2018, 01:05 PM

Does this require a port entry? nbd:192.168.1.2:

dbrazeau · 08-17-2018, 03:22 PM

Quote:

Originally Posted by jefro

Does this require a port entry? nbd:192.168.1.2:

I'm able to mount it find just using the export name both locally and on a remote system (after boot). I just can't get it to work when trying to use the NBD as a rootfs.

dbrazeau · 08-17-2018, 03:29 PM

So it turns out my initial issue was that my initrd.img didn't contain the nbd module. After rebuilding my initrd.img I seem to be getting a little closer, although still having some issues.

Here is the output from the client after rebuilding initrd.img to include nbd.

Code:

[    7.886365] Sending DHCP requests ., OK
[    9.400428] IP-Config: Got DHCP answer from 0.0.0.0, my address is 192.168.1.27
[    9.449365] IP-Config: Complete:
[    9.452674]      device=eth1, addr=192.168.1.27, mask=255.255.255.0, gw=255.255.255.255,
[    9.460751]      host=192.168.1.27, domain=, nis-domain=(none),
[    9.466797]      bootserver=0.0.0.0, rootserver=0.0.0.0, rootpath=nbd:192.168.1.2:export
[    9.478358] Freeing unused kernel memory: 1328k freed
[    9.483649] Write protecting the kernel read-only data: 10240k
[    9.493229] Freeing unused kernel memory: 992k freed
[    9.503818] Freeing unused kernel memory: 1560k freed
[    9.543556] dracut: dracut-009-12.fc15
[    9.572103] udev[168]: starting version 167
[    9.625897] NET: Registered protocol family 10
[    9.723094] ADDRCONF(NETDEV_UP): eth0: link is not ready
[   12.668081] nbd: registered device at major 43
RTNETLINK answers: File exists
too many non-option arguments sp[   12.698231] dracut Warning: Mounting root via 'eth1' failed
ecifiedUsage: nbd-client host port nbd_device [-block-size|-b block size] [-timeout|-t timeout] [-swap|-s] [-sdp|-S] [-persist|-p] [-nofork|-n] [-name|-N na
me]
Or   : nbd-client -d nbd_device
Or   : nbd-client -c nbd_device
Or   : nbd-client -h|--help
Default value for blocksize is 1024 (recommended for ethernet)
Allowed values for blocksize are 512,1024,2048,4096
Note, that kernel 2.4.2 and older ones do not work correctly with
blocksizes other than 1024 without patches


[   93.726991] dracut Warning: No root device "dhcp" found

Dropping to debug shell.

When I boot the client off of its local HD, nbd-client can connect to the NBD fine after boot. However if I PXE Boot the client and try to use the NBD as the root file system I get the above error. It looks like for whatever reason nbd-client does not like the format of the command being issued by the boot script/application.

dbrazeau · 08-17-2018, 04:34 PM

I'm starting to wonder if my older client system is incompatible with my newer server. It seems like the client is trying to use a port number instead of the export name.

Whenever I try to connect to the NBD using just a port number I get this error.

Code:

# nbd-client 192.168.1.2 2000 /dev/nbd0
Error: Socket failed: Connection refused
Exiting.

But if I use the command "nbd-client -N export 192.168.1.2 /dev/nbd0" it works fine.

I've even tried experiments such as setting the root-path to "nbd:192.168.1.2:-N export" with no luck. Is there anyway to figure out what nbd-client command my client is trying to use at boot to mount the NBD?

dbrazeau · 08-17-2018, 04:58 PM

Well it looks like nbd-server uses the default port regardless of what is in the configuration file.

Code:

# netstat -tlnp
Active Internet connections (only servers)
tcp        0      0 0.0.0.0:10809           0.0.0.0:*               LISTEN      1296/nbd-server

So using the default port on the client still doesn't work but I get a different error.

Code:

nbd-client 192.168.1.2 10809 /dev/nbd0
Negotiation: .Error: Not enough cliserv_magic
Exiting.

jefro · 08-17-2018, 07:42 PM

Maybe this old thread? https://bugs.debian.org/cgi-bin/bugr...cgi?bug=699373

I was kind of hoping others may jump in on this. On web tutorials they claim it ought to be easier to do that standard pxe but I've never tried this.

Ztcoracat · 08-17-2018, 09:04 PM

https://etbe.coker.com.au/2009/08/23...ooting-debian/

https://wiki.archlinux.org/index.php/diskless_system

HTH