Thanks a lot Alim for the detail step-by-step, for both modes. I'd like to stay with legacy mode with pxelinux.0 for now. I'm doing pretty much the same as what you described below, with exceptions - 1. I'm not using vesamenu.c32. Instead menu.32 is used, though I don't think that would be an issue. 2. I use "kernel image/bzImage" instead of "linux image/bzImage". Still this shouldn't be an issue. 3. With the "append" line, I've a question - what is the significance of the options "hard, tcp, intr"? I currently don't use them. Since my last email this morning, I'd made some progress - 1. I figured out how to reconfigure the kernel with bitbake menuconfig to change my Ethernet driver from .ko to part of kernel. 2. I now have a confusing situation, due to Avahi-daemon in the mix. PXEBoot would request IP address via DHCP. Then when this NFS boot runs, it too uses DHCP, and allocates a different IP address. I suppose this is fine, so long as I've enough IP address to spare. However, Avahi-daemon also request IP address and get the board yet another IP address. I've yet to figure out how to turn off Avahi-daemon. And NFS boot still doesn't work (I don't see the mount point with "df -h"). 3. Client side uses NFS v4.1 while server side uses v3. Would this be a problem? I suppose there must be backward compatibility built into NFS. Also, in the "Append" line of pxelinux.cfg/default, I add nfsrootdebug. What sort of NFS debugs should I see? I only see two NFS logs, and 4 RPC calls. Is there anyway to turn up the verbosity so I know how NFS client fails? Until I get some more insights to above, I'd proceed to investigate the other issue - why my kernel initialization would need to wait for any "removable media" indefinitely, and see if I could fix/change this. Thanks, Raymond ________________________________ From: Hussin, Mohamad Noor Alim Sent: Thursday, April 26, 2018 8:30 PM To: Raymond Yeung; yocto@yoctoproject.org Subject: RE: PXE Boot NFS not working You are using PXE with legacy BIOS as you use pxelinux.0 . To configure PXE with legacy BIOS follow these instruction Note: PXE server IP = 192.168.1.2 Target IP = 192.168.1.1 On PXE server 1. make sure DHCP, NFS & TFTP services are up and running. 2. change the filename “pxelinux.0”; in /etc/dhcp/dhcpd.conf. Then restart your DHCP service. 3. Put your kernel to /path/to/tftpboot/image/ and extract your rootfs to /srv/rootfs/. 4. Create a file in /path/to/tftpboot/pxelinux.cfg/default. The content should like this include menu.cfg default vesamenu.c32 prompt 0 timeout 5 label Manual Netboot genericx86-64 menu label Netboot genericx86-64 linux image/bzImage append root=/dev/nfs rw nfsroot=192.168.1.2:/srv/rootfs,hard,tcp,intr ip=dhcp if you want to use PXE with GRUB, bootx64.efi, you need to enable UEFI instead of legacy BIOS in BIOS configuration. Follow these instruction. on PXE server 1. make sure DHCP, NFS & TFTP services are up and running. 2. change the filename “bootx64.efi”; in /etc/dhcp/dhcpd.conf. Then restart your DHCP service. 3. Put your kernel to /path/to/tftpboot/image/ and extract your rootfs to /srv/rootfs/. 4. If you want to get into GRUB, a. comment the commands from /path/to/tftpboot/grub/grub.cfg to avoid grub automatically load the kernel from network. b. On GRUB, press “TAB” key to list available command. Or you can try use this command i. net_bootp ii. linux (tftp,pxeserverip)/image/bzImage root=/dev/nfs rw nfsroot=192.168.1.2:/srv/rootfs,hard,tcp,intr ip=dhcp iii. boot 5. if you want boot automatically to NFS, create a file called grub.cfg in /path/to/tftpboot/grub.cfg. your grub.cfg should look like this Set timeout 5 menuentry " genericx86-64" { linux image/bzImage root=/dev/nfs rw nfsroot=192.168.1.2:/srv/rootfs,hard,tcp,intr ip=dhcp initrd image/initrd } 6. When your device boot, it will download bootx64.efi and the message something like this >>Start PXE over IPv4. Station IP address is 192.168.1.1 Server IP address is 192.168.1.2 NBP filename is bootx64.efi NBP filesize is 1137016 Bytes Downloading NBP file... NBP file downloaded successfully. Please note that I did not use initramfs or initrd as my kernel already have network module build-in. To build your kernel with network module, you need to know which module need to build-in into your kernel. Simply boot your device using *.hhdimg from USB stick. Then check loaded kernel module from your device. $ lsmod Find the module something with network. Yes it is a little bit tricky. Build your kernel in host machine. $ bitbake virtual/kernel –c menuconfig A menu will popup and search your kernel module using key “/”. Make sure symbol module [*] indicate build-in. $ bitbake virtual/kernel You also can create a kernel bundle with initramfs. Read this https://www.yoctoproject.org/docs/2.5/mega-manual/mega-manual.html#building-an-initramfs-image. Maybe need to ask someone who are expert on kernel development on how to enable build-in kernel module. Regards, Alim Hussin From: Raymond Yeung [mailto:rksyeung@hotmail.com] Sent: Friday, April 27, 2018 1:12 AM To: Hussin, Mohamad Noor Alim ; yocto@yoctoproject.org Subject: Re: PXE Boot NFS not working Follow-up to my previous post, after seeing the reply below suggesting me not to use initramfs (but initrd is usable, right?). BTW, I don't use bootx64.efi. Instead, I use pxelinux.0 along with the other lib***.c32 files. 1. Is there a way to verify if GRUB (that I'm using) supports networking or not? It seems to have very limited user commands available. 2. How do I get into GRUB? I'd been into GRUB, more like accidentally. The reply below refers to grub.cfg. Is this cfg file stored on target, or on host (like pxelinux.cfg)? 3. How do I configure my build (and what yocto tool to use, if any) in order to get my network driver part of kernel, instead of being a LKM that isn't there when I need it for NFS boot? ________________________________ From: Hussin, Mohamad Noor Alim > Sent: Thursday, April 26, 2018 12:26 AM To: Raymond Yeung; yocto@yoctoproject.org Subject: RE: PXE Boot NFS not working Refer to my post here https://lists.yoctoproject.org/pipermail/yocto/2018-April/040860.html Don’t use initramfs/initrd as it not working. Regards, Alim Hussin