PXE booting ISO image fails with message "Waiting for removable media..."

PXE booting ISO image fails with message "Waiting for removable media..."

[Vincent Daanen]

[-- Attachment #1: Type: text/plain, Size: 1070 bytes --]

Hi,

We want to deploy image created with Yocto (Rocko) via PXE.
The target board is a Intel-based SBC (AAEON GENE BT05).

At first, we tried with a ISO from Archlinux and the target-board successfully booted.

Then we create a ISO using Yocto but boot hangs with a message “Waiting for removable media ...”.
Googling this message points me to this post: http://thread.gmane.org/gmane.linux.embedded.yocto.general/20611 which relates exactly the same problem, explain why the issue raises, gives an indication to how to fix .. and that’s all ☹

So at this point, we know that the problem comes from the /recipes-core/initrdscripts/files/init-live.sh file.
It seems searching for an /dev/sdx should be protected by a timeout set by default to 30 secs but during our trials, no timeout seems to exist.

What we do not understand, is how to highlight the differences between the Yocto-based and the Archlinux ISO files so that we can “unblock” the boot using the Yocto-based ISO file.

Is the someone here how could help?

Thanks

Vincent


[-- Attachment #2: Type: text/html, Size: 3719 bytes --]

Re: PXE booting ISO image fails with message "Waiting for removable media..."

[Raymond Yeung]

[-- Attachment #1: Type: text/plain, Size: 2412 bytes --]

Hi Vincent,


I'd recently gone through similar issue that you're experiencing.  The "Waiting for Removable Media" hang had been there for at least 7-8 years.  A workaround was put in around 2013.  See here:

https://patchwork.openembedded.org/patch/42291/


Add "debugshell=30" (30 is in second, for timeout, or any other reasonable value that you like) to the append
statement in your pxelinux.cfg/default file.  Then you'd timeout when you "normally" hangs, break into a shell,
where you could initiate udhcpc to configure networking, after which you could transfer your bootable image onto
your RAM, and write out onto your HDD/SSD.

It seems this kernel boot option of debugshell isn't documented officially in kernel-parameters.txt file.

You could also look for init-live.sh on your yocto tree, and look for "Waiting for Removable Media", and see the
surrounding code to get a fell how this debugshell thing works.

Raymond




Date: Thu, 3 May 2018 14:27:46 +0000
From: Vincent Daanen <vincent.daanen@orthotaxy.com>
To: "yocto@yoctoproject.org" <yocto@yoctoproject.org>
Subject: [yocto] PXE booting ISO image fails with message "Waiting for
        removable media..."
Message-ID:
        <VI1P192MB01430B16A66D217CB44829098D870@VI1P192MB0143.EURP192.PROD.OUTLOOK.COM>

Content-Type: text/plain; charset="utf-8"

Hi,

We want to deploy image created with Yocto (Rocko) via PXE.
The target board is a Intel-based SBC (AAEON GENE BT05).

At first, we tried with a ISO from Archlinux and the target-board successfully booted.

Then we create a ISO using Yocto but boot hangs with a message ?Waiting for removable media ...?.
Googling this message points me to this post: http://thread.gmane.org/gmane.linux.embedded.yocto.general/20611 which relates exactly the same problem, explain why the issue raises, gives an indication to how to fix .. and that?s all ?

So at this point, we know that the problem comes from the /recipes-core/initrdscripts/files/init-live.sh file.
It seems searching for an /dev/sdx should be protected by a timeout set by default to 30 secs but during our trials, no timeout seems to exist.

What we do not understand, is how to highlight the differences between the Yocto-based and the Archlinux ISO files so that we can ?unblock? the boot using the Yocto-based ISO file.

Is the someone here how could help?

Thanks

Vincent


[-- Attachment #2: Type: text/html, Size: 5135 bytes --]

Re: PXE booting ISO image fails with message "Waiting for removable media..."

[Vincent Daanen]

[-- Attachment #1: Type: text/plain, Size: 3325 bytes --]

Hi Raymond,

> I'd recently gone through similar issue that you're experiencing.  The "Waiting for Removable Media" hang had been there for at least 7-8 years.  A workaround was put in around 2013.  See here: https://patchwork.openembedded.org/patch/42291/
>
> Add "debugshell=30" (30 is in second, for timeout, or any other reasonable value that you like) to the append
> statement in your pxelinux.cfg/default file.  Then you'd timeout when you "normally" hangs, break into a shell,
> where you could initiate udhcpc to configure networking, after which you could transfer your bootable image onto
> your RAM, and write out onto your HDD/SSD.

Unfortunately, we cannot use this trick because the MB which boots using PXE is not accessible and we want to use PXE booting  for every boot...

> It seems this kernel boot option of debugshell isn't documented officially in kernel-parameters.txt file.

>  You could also look for init-live.sh on your yocto tree, and look for "Waiting for Removable Media", and see the
> surrounding code to get a fell how this debugshell thing works.
I think this is what we will have to do ..

I read the description of the internals of the initrd provided by Archlinux. The main difference is that Yocto initrd searches for the final rootfs on physical device (/dev/sdX) whereas the Archlinux initrd identifies (using LABEL) the support it was launched from and then searches on this support for the final rootfs.

I guess this approach would unify all booting: from USB, CD-ROM and PXE.. But at the moment, I don't know how to reproduce the archlinux initrd works with Yocto...

If only a guru of Yocto could help ...

Vincent

Date: Thu, 3 May 2018 14:27:46 +0000
From: Vincent Daanen <vincent.daanen@orthotaxy.com<mailto:vincent.daanen@orthotaxy.com>>
To: "yocto@yoctoproject.org<mailto:yocto@yoctoproject.org>" <yocto@yoctoproject.org<mailto:yocto@yoctoproject.org>>
Subject: [yocto] PXE booting ISO image fails with message "Waiting for
        removable media..."
Message-ID:
        <VI1P192MB01430B16A66D217CB44829098D870@VI1P192MB0143.EURP192.PROD.OUTLOOK.COM<mailto:VI1P192MB01430B16A66D217CB44829098D870@VI1P192MB0143.EURP192.PROD.OUTLOOK.COM>>

Content-Type: text/plain; charset="utf-8"

Hi,

We want to deploy image created with Yocto (Rocko) via PXE.
The target board is a Intel-based SBC (AAEON GENE BT05).

At first, we tried with a ISO from Archlinux and the target-board successfully booted.

Then we create a ISO using Yocto but boot hangs with a message ?Waiting for removable media ...?.
Googling this message points me to this post: http://thread.gmane.org/gmane.linux.embedded.yocto.general/20611 which relates exactly the same problem, explain why the issue raises, gives an indication to how to fix .. and that?s all ?

So at this point, we know that the problem comes from the /recipes-core/initrdscripts/files/init-live.sh file.
It seems searching for an /dev/sdx should be protected by a timeout set by default to 30 secs but during our trials, no timeout seems to exist.

What we do not understand, is how to highlight the differences between the Yocto-based and the Archlinux ISO files so that we can ?unblock? the boot using the Yocto-based ISO file.

Is the someone here how could help?

Thanks

Vincent

[-- Attachment #2: Type: text/html, Size: 7704 bytes --]

Re: PXE booting ISO image fails with message "Waiting for removable media..."

[Raymond Yeung]

[-- Attachment #1: Type: text/plain, Size: 4383 bytes --]

Perhaps you could try NFS kernel boot option.  Set up the remote (tftp root) directory to reflect the path the init code expects.


I tried this for a little while but ran into what 2 issues - network driver init is later than NFS (due to it being a LKM rather than part of kernel), and Avahi screwing around with IP addresses.  I looked into the log to confirm timing for the first one, and fixed this with kernel reconfiguration.  However, I've yet to try fixing Avahi (remove it) as I'm not sure how to do it yet.


My time ran out, and I got another workable solution.  So I shelved this NFS approach.  If you get this working, would appreciate it if you could publish it, and any issues you run into implementing it.


Raymond


________________________________
From: Vincent Daanen <vincent.daanen@orthotaxy.com>
Sent: Thursday, May 3, 2018 11:43 PM
To: Raymond Yeung; yocto@yoctoproject.org
Subject: RE: [yocto] PXE booting ISO image fails with message "Waiting for removable media..."


Hi Raymond,



> I'd recently gone through similar issue that you're experiencing.  The "Waiting for Removable Media" hang had been there for at least 7-8 years.  A workaround was put in around 2013.  See here: https://patchwork.openembedded.org/patch/42291/

>

> Add "debugshell=30" (30 is in second, for timeout, or any other reasonable value that you like) to the append

> statement in your pxelinux.cfg/default file.  Then you'd timeout when you "normally" hangs, break into a shell,

> where you could initiate udhcpc to configure networking, after which you could transfer your bootable image onto

> your RAM, and write out onto your HDD/SSD.



Unfortunately, we cannot use this trick because the MB which boots using PXE is not accessible and we want to use PXE booting  for every boot…



> It seems this kernel boot option of debugshell isn't documented officially in kernel-parameters.txt file.



>  You could also look for init-live.sh on your yocto tree, and look for "Waiting for Removable Media", and see the

> surrounding code to get a fell how this debugshell thing works.

I think this is what we will have to do ..



I read the description of the internals of the initrd provided by Archlinux. The main difference is that Yocto initrd searches for the final rootfs on physical device (/dev/sdX) whereas the Archlinux initrd identifies (using LABEL) the support it was launched from and then searches on this support for the final rootfs.



I guess this approach would unify all booting: from USB, CD-ROM and PXE.. But at the moment, I don’t know how to reproduce the archlinux initrd works with Yocto…



If only a guru of Yocto could help …



Vincent



Date: Thu, 3 May 2018 14:27:46 +0000
From: Vincent Daanen <vincent.daanen@orthotaxy.com<mailto:vincent.daanen@orthotaxy.com>>
To: "yocto@yoctoproject.org<mailto:yocto@yoctoproject.org>" <yocto@yoctoproject.org<mailto:yocto@yoctoproject.org>>
Subject: [yocto] PXE booting ISO image fails with message "Waiting for
        removable media..."
Message-ID:
        <VI1P192MB01430B16A66D217CB44829098D870@VI1P192MB0143.EURP192.PROD.OUTLOOK.COM<mailto:VI1P192MB01430B16A66D217CB44829098D870@VI1P192MB0143.EURP192.PROD.OUTLOOK.COM>>

Content-Type: text/plain; charset="utf-8"

Hi,

We want to deploy image created with Yocto (Rocko) via PXE.
The target board is a Intel-based SBC (AAEON GENE BT05).

At first, we tried with a ISO from Archlinux and the target-board successfully booted.

Then we create a ISO using Yocto but boot hangs with a message ?Waiting for removable media ...?.
Googling this message points me to this post: http://thread.gmane.org/gmane.linux.embedded.yocto.general/20611 which relates exactly the same problem, explain why the issue raises, gives an indication to how to fix .. and that?s all ?

So at this point, we know that the problem comes from the /recipes-core/initrdscripts/files/init-live.sh file.
It seems searching for an /dev/sdx should be protected by a timeout set by default to 30 secs but during our trials, no timeout seems to exist.

What we do not understand, is how to highlight the differences between the Yocto-based and the Archlinux ISO files so that we can ?unblock? the boot using the Yocto-based ISO file.

Is the someone here how could help?

Thanks

Vincent

[-- Attachment #2: Type: text/html, Size: 8059 bytes --]