calling request_firmware() from module init will not work with recent/future udev versions

calling request_firmware() from module init will not work with recent/future udev versions

[Kay Sievers]

Hey,

just a heads-up, when bug reports are coming in now. The kernel log
might look like a kernel failure, but it's caused by recent userspace
changes.

We changed udev to strictly enforce sequence order and dependency
handling of events. This seems to break some netdev drivers which
block in modprobe until the firmware from userspace is loaded into the
driver.

I think it's out of question, that nothing must block in module init
and rely on a userspace transaction to be fulfilled to finish the
init_module() syscall. If userspace is not running properly, nothing
will work. Built-in drivers and the transition from initramfs to the
real root can't be handled properly that way.

These drivers need to be fixed to load their firmware during ifup,
which would be the most flexible solution. That way, it should also
work if the driver is built-in, or is loaded from initramfs and no
firmware is available.
Alternatively, the firmware request should be able to use the aync
firmware_request API and not the blocking one.

We might need to work around that in the current udev for now, but
these drivers will definitely break in future udev versions.
Userspace, these days, should not be in charge of papering over
obvious kernel bugs like this.

These are the drivers we received bug reports so far. We didn't look
into any of details of the drivers, but according to the logs, they
seem to block for 60 seconds in modprobe, when userspace is not
behaving like expected:
  bnx2/bnx2-mips-06-6.2.1.fw
  rtlwifi/rtl8192sefw.bin
  brcm/bcm43xx-0.fw

Any help here would be appreciated.

Thanks,
Kay

Re: calling request_firmware() from module init will not work with recent/future udev versions

[John W. Linville]

Kay, thanks for the reports.  Drivers should definitely be loading
firmware at IFF_UP time.

John

On Sat, Jan 14, 2012 at 04:25:23PM +0100, Kay Sievers wrote:
> Hey,
> 
> just a heads-up, when bug reports are coming in now. The kernel log
> might look like a kernel failure, but it's caused by recent userspace
> changes.
> 
> We changed udev to strictly enforce sequence order and dependency
> handling of events. This seems to break some netdev drivers which
> block in modprobe until the firmware from userspace is loaded into the
> driver.
> 
> I think it's out of question, that nothing must block in module init
> and rely on a userspace transaction to be fulfilled to finish the
> init_module() syscall. If userspace is not running properly, nothing
> will work. Built-in drivers and the transition from initramfs to the
> real root can't be handled properly that way.
> 
> These drivers need to be fixed to load their firmware during ifup,
> which would be the most flexible solution. That way, it should also
> work if the driver is built-in, or is loaded from initramfs and no
> firmware is available.
> Alternatively, the firmware request should be able to use the aync
> firmware_request API and not the blocking one.
> 
> We might need to work around that in the current udev for now, but
> these drivers will definitely break in future udev versions.
> Userspace, these days, should not be in charge of papering over
> obvious kernel bugs like this.
> 
> These are the drivers we received bug reports so far. We didn't look
> into any of details of the drivers, but according to the logs, they
> seem to block for 60 seconds in modprobe, when userspace is not
> behaving like expected:
>   bnx2/bnx2-mips-06-6.2.1.fw
>   rtlwifi/rtl8192sefw.bin
>   brcm/bcm43xx-0.fw
> 
> Any help here would be appreciated.
> 
> Thanks,
> Kay
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
John W. Linville		Someday the world will need a hero, and you
linville@tuxdriver.com			might be all we have.  Be ready.

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Larry Finger]

On 01/14/2012 11:58 AM, John W. Linville wrote:
> Kay, thanks for the reports.  Drivers should definitely be loading
> firmware at IFF_UP time.

As the maintainer for several drivers with this problem, I need to make some 
changes. What call from mac80211 corresponds to IFF_UP?

I have looked into using asynchronous firmware loading, but I have not yet found 
a good implementation.

Larry

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Larry Finger]

On 01/14/2012 09:25 AM, Kay Sievers wrote:
> Hey,
>
> just a heads-up, when bug reports are coming in now. The kernel log
> might look like a kernel failure, but it's caused by recent userspace
> changes.
>
> We changed udev to strictly enforce sequence order and dependency
> handling of events. This seems to break some netdev drivers which
> block in modprobe until the firmware from userspace is loaded into the
> driver.
>
> I think it's out of question, that nothing must block in module init
> and rely on a userspace transaction to be fulfilled to finish the
> init_module() syscall. If userspace is not running properly, nothing
> will work. Built-in drivers and the transition from initramfs to the
> real root can't be handled properly that way.
>
> These drivers need to be fixed to load their firmware during ifup,
> which would be the most flexible solution. That way, it should also
> work if the driver is built-in, or is loaded from initramfs and no
> firmware is available.
> Alternatively, the firmware request should be able to use the aync
> firmware_request API and not the blocking one.
>
> We might need to work around that in the current udev for now, but
> these drivers will definitely break in future udev versions.
> Userspace, these days, should not be in charge of papering over
> obvious kernel bugs like this.
>
> These are the drivers we received bug reports so far. We didn't look
> into any of details of the drivers, but according to the logs, they
> seem to block for 60 seconds in modprobe, when userspace is not
> behaving like expected:
>    bnx2/bnx2-mips-06-6.2.1.fw
>    rtlwifi/rtl8192sefw.bin
>    brcm/bcm43xx-0.fw

Thanks for the heads-up. Do you have a reference for the problem with 
rtlwifi/rtl8192sefw.bin? Google did not find anything that looked appropriate.

Thanks,

Larry

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Tom Gundersen]

Hi Larry,

On Sat, Jan 14, 2012 at 7:45 PM, Larry Finger <Larry.Finger@lwfinger.net> wrote:
> Thanks for the heads-up. Do you have a reference for the problem with
> rtlwifi/rtl8192sefw.bin? Google did not find anything that looked
> appropriate.

I believe the logs attached to this bug comment deal with rtlwifi:
<https://bugs.archlinux.org/task/27938#comment87952>.

Cheers,

Tom

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Arend van Spriel]

On 01/14/2012 07:20 PM, Larry Finger wrote:
> On 01/14/2012 11:58 AM, John W. Linville wrote:
>> Kay, thanks for the reports.  Drivers should definitely be loading
>> firmware at IFF_UP time.
> 
> As the maintainer for several drivers with this problem, I need to make some 
> changes. What call from mac80211 corresponds to IFF_UP?

Hi, Larry

The mac80211 start callback describes following:

 * @start: Called before the first netdevice attached to the hardware
 *	is enabled. This should turn on the hardware and must turn on
 *	frame reception (for possibly enabled monitor interfaces.)
 *	Returns negative error codes, these may be seen in userspace,
 *	or zero.
 *	When the device is started it should not have a MAC address
 *	to avoid acknowledging frames before a non-monitor device
 *	is added.
 *	Must be implemented and can sleep.

Seems to me a good place to do the firmware loading although it is not
really IFF_UP.

> I have looked into using asynchronous firmware loading, but I have not yet found 
> a good implementation.
> 
> Larry

In brcmsmac we request the firmware in the probe function so not in
module_init, but I guess upon driver registration in the module_init the
probe is called.

Gr. AvS

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Arend van Spriel]

On 01/14/2012 07:20 PM, Larry Finger wrote:
> On 01/14/2012 11:58 AM, John W. Linville wrote:
>> Kay, thanks for the reports.  Drivers should definitely be loading
>> firmware at IFF_UP time.
> 
> As the maintainer for several drivers with this problem, I need to make some 
> changes. What call from mac80211 corresponds to IFF_UP?

Hi, Larry

The mac80211 start callback describes following:

 * @start: Called before the first netdevice attached to the hardware
 *	is enabled. This should turn on the hardware and must turn on
 *	frame reception (for possibly enabled monitor interfaces.)
 *	Returns negative error codes, these may be seen in userspace,
 *	or zero.
 *	When the device is started it should not have a MAC address
 *	to avoid acknowledging frames before a non-monitor device
 *	is added.
 *	Must be implemented and can sleep.

Seems to me a good place to do the firmware loading although it is not
really IFF_UP.

> I have looked into using asynchronous firmware loading, but I have not yet found 
> a good implementation.
> 
> Larry

In brcmsmac we request the firmware in the probe function so not in
module_init, but I guess upon driver registration in the module_init the
probe is called.

Gr. AvS

--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Larry Finger]

On 01/14/2012 01:59 PM, Arend van Spriel wrote:
> On 01/14/2012 07:20 PM, Larry Finger wrote:
>> On 01/14/2012 11:58 AM, John W. Linville wrote:
>>> Kay, thanks for the reports.  Drivers should definitely be loading
>>> firmware at IFF_UP time.
>>
>> As the maintainer for several drivers with this problem, I need to make some
>> changes. What call from mac80211 corresponds to IFF_UP?
>
> Hi, Larry
>
> The mac80211 start callback describes following:
>
>   * @start: Called before the first netdevice attached to the hardware
>   *	is enabled. This should turn on the hardware and must turn on
>   *	frame reception (for possibly enabled monitor interfaces.)
>   *	Returns negative error codes, these may be seen in userspace,
>   *	or zero.
>   *	When the device is started it should not have a MAC address
>   *	to avoid acknowledging frames before a non-monitor device
>   *	is added.
>   *	Must be implemented and can sleep.
>
> Seems to me a good place to do the firmware loading although it is not
> really IFF_UP.
>
>> I have looked into using asynchronous firmware loading, but I have not yet found
>> a good implementation.
>>
>> Larry
>
> In brcmsmac we request the firmware in the probe function so not in
> module_init, but I guess upon driver registration in the module_init the
> probe is called.

Thanks.

The rtlwifi drivers also load the firmware in the probe function, but do so 
synchronously, which is where they get in trouble with the new udev.

I'm planning on converting to asynchronous firmware loading with a check in the 
start routine.

Larry

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Larry Finger]

On 01/14/2012 01:59 PM, Arend van Spriel wrote:
> On 01/14/2012 07:20 PM, Larry Finger wrote:
>> On 01/14/2012 11:58 AM, John W. Linville wrote:
>>> Kay, thanks for the reports.  Drivers should definitely be loading
>>> firmware at IFF_UP time.
>>
>> As the maintainer for several drivers with this problem, I need to make some
>> changes. What call from mac80211 corresponds to IFF_UP?
>
> Hi, Larry
>
> The mac80211 start callback describes following:
>
>   * @start: Called before the first netdevice attached to the hardware
>   *	is enabled. This should turn on the hardware and must turn on
>   *	frame reception (for possibly enabled monitor interfaces.)
>   *	Returns negative error codes, these may be seen in userspace,
>   *	or zero.
>   *	When the device is started it should not have a MAC address
>   *	to avoid acknowledging frames before a non-monitor device
>   *	is added.
>   *	Must be implemented and can sleep.
>
> Seems to me a good place to do the firmware loading although it is not
> really IFF_UP.
>
>> I have looked into using asynchronous firmware loading, but I have not yet found
>> a good implementation.
>>
>> Larry
>
> In brcmsmac we request the firmware in the probe function so not in
> module_init, but I guess upon driver registration in the module_init the
> probe is called.

Thanks.

The rtlwifi drivers also load the firmware in the probe function, but do so 
synchronously, which is where they get in trouble with the new udev.

I'm planning on converting to asynchronous firmware loading with a check in the 
start routine.

Larry

--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Emmanuel Grumbach]

On Sat, Jan 14, 2012 at 22:13, Larry Finger <Larry.Finger@lwfinger.net> wrote:
> On 01/14/2012 01:59 PM, Arend van Spriel wrote:
>>
>> On 01/14/2012 07:20 PM, Larry Finger wrote:
>>>
>>> On 01/14/2012 11:58 AM, John W. Linville wrote:
>>>>
>>>> Kay, thanks for the reports.  Drivers should definitely be loading
>>>> firmware at IFF_UP time.
>>>
>>>
>>> As the maintainer for several drivers with this problem, I need to make
>>> some
>>> changes. What call from mac80211 corresponds to IFF_UP?
>>
>>
>> Hi, Larry
>>
>> The mac80211 start callback describes following:
>>
>>  * @start: Called before the first netdevice attached to the hardware
>>  *     is enabled. This should turn on the hardware and must turn on
>>  *     frame reception (for possibly enabled monitor interfaces.)
>>  *     Returns negative error codes, these may be seen in userspace,
>>  *     or zero.
>>  *     When the device is started it should not have a MAC address
>>  *     to avoid acknowledging frames before a non-monitor device
>>  *     is added.
>>  *     Must be implemented and can sleep.
>>
>> Seems to me a good place to do the firmware loading although it is not
>> really IFF_UP.
>>
>>> I have looked into using asynchronous firmware loading, but I have not
>>> yet found
>>> a good implementation.
>>>
>>> Larry
>>
>>
>> In brcmsmac we request the firmware in the probe function so not in
>> module_init, but I guess upon driver registration in the module_init the
>> probe is called.
>
>

I looked once at the pci_register_driver, and if I remember well, the
probe in called in the same context of pci_register_driver. The driver
registration fires a bus enumeration.
FWIW, drivers might need to fetch their fw before registering
mac80211. This can happen if they need their fw to know their
capabilities...

>
> The rtlwifi drivers also load the firmware in the probe function, but do so
> synchronously, which is where they get in trouble with the new udev.
>
> I'm planning on converting to asynchronous firmware loading with a check in
> the start routine.
>

so does iwlwifi

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Emmanuel Grumbach]

On Sat, Jan 14, 2012 at 22:13, Larry Finger <Larry.Finger-tQ5ms3gMjBLk1uMJSBkQmQ@public.gmane.org> wrote:
> On 01/14/2012 01:59 PM, Arend van Spriel wrote:
>>
>> On 01/14/2012 07:20 PM, Larry Finger wrote:
>>>
>>> On 01/14/2012 11:58 AM, John W. Linville wrote:
>>>>
>>>> Kay, thanks for the reports.  Drivers should definitely be loading
>>>> firmware at IFF_UP time.
>>>
>>>
>>> As the maintainer for several drivers with this problem, I need to make
>>> some
>>> changes. What call from mac80211 corresponds to IFF_UP?
>>
>>
>> Hi, Larry
>>
>> The mac80211 start callback describes following:
>>
>>  * @start: Called before the first netdevice attached to the hardware
>>  *     is enabled. This should turn on the hardware and must turn on
>>  *     frame reception (for possibly enabled monitor interfaces.)
>>  *     Returns negative error codes, these may be seen in userspace,
>>  *     or zero.
>>  *     When the device is started it should not have a MAC address
>>  *     to avoid acknowledging frames before a non-monitor device
>>  *     is added.
>>  *     Must be implemented and can sleep.
>>
>> Seems to me a good place to do the firmware loading although it is not
>> really IFF_UP.
>>
>>> I have looked into using asynchronous firmware loading, but I have not
>>> yet found
>>> a good implementation.
>>>
>>> Larry
>>
>>
>> In brcmsmac we request the firmware in the probe function so not in
>> module_init, but I guess upon driver registration in the module_init the
>> probe is called.
>
>

I looked once at the pci_register_driver, and if I remember well, the
probe in called in the same context of pci_register_driver. The driver
registration fires a bus enumeration.
FWIW, drivers might need to fetch their fw before registering
mac80211. This can happen if they need their fw to know their
capabilities...

>
> The rtlwifi drivers also load the firmware in the probe function, but do so
> synchronously, which is where they get in trouble with the new udev.
>
> I'm planning on converting to asynchronous firmware loading with a check in
> the start routine.
>

so does iwlwifi
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Johannes Berg]

Kay,

> These drivers need to be fixed to load their firmware during ifup,
> which would be the most flexible solution. That way, it should also
> work if the driver is built-in, or is loaded from initramfs and no
> firmware is available.
> Alternatively, the firmware request should be able to use the aync
> firmware_request API and not the blocking one.

Will udev now also return the async load only after root is booted if it
can't be satisfied earlier? There are a number of devices, particularly
wireless, that need firmware before they can register with mac80211 for
capability advertisement reasons.

johannes

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Kay Sievers]

On Sun, Jan 15, 2012 at 11:02, Johannes Berg <johannes@sipsolutions.net> wrote:
>> These drivers need to be fixed to load their firmware during ifup,
>> which would be the most flexible solution. That way, it should also
>> work if the driver is built-in, or is loaded from initramfs and no
>> firmware is available.
>> Alternatively, the firmware request should be able to use the aync
>> firmware_request API and not the blocking one.
>
> Will udev now also return the async load only after root is booted if it
> can't be satisfied earlier?

Not sure, let me explain what happens here, maybe it contains the answer:

Udev gets an event for a pci device:
  /devices/pci0000:00/0000:00:1c.1/0000:03:00.0

This device has a modalias, which let's udev load the matching module
into the kernel. The module_init() syscall triggers the firmware
loading request, which causes another event:
  /devices/pci0000:00/0000:00:1c.1/0000:03:00.0/firmware/0000:03:00.0

This event is a direct child of the pci device and udev delays the
execution of child devices until the parent devices return from
handling. This dependency logic is needed for many things to ensure a
proper operation, like partitions which need to make sure the events
for the disk devices are handled before the partition events are
started.

Now the problem, the pcidev event is blocking in modprobe and waits
for the child event it has generated to finish, but udev does not
start the event because the parent still blocks in modprobe ->
deadlock until default firmware timeout of 60 sec. What we want here,
for several reasons not only udev's dependency logic, is that modprobe
never waits for userspace transactions to finish.

> There are a number of devices, particularly
> wireless, that need firmware before they can register with mac80211 for
> capability advertisement reasons.

Right, ideally all firmware loading would be delayed until the netif
is brought up, and that's what we shoudl use if possible. In all other
cases, I think what iwlwifi is doing here looks fine from the
userspace side. It does not register any netdev/mac80211 device, the
firmware request is the only thing that it issued, and modprobe
returns immediately, regardless of the firmware request handling.

If userspace is not responding, the firmware request times out after
60 seconds and the driver is not associated with any hardware. To
retry the firmware loading, the module needs to be unloaded and
reloaded, or the driver needs to be asked to bind to a device again by
writing to the 'bind' in file in the sysfs driver directory.

Firmware requests stay around in the system for by default 60 seconds.
If the driver would be built-in the request would be issued long
before userspace is ready, but udev's coldplug step during bootup will
cause all events to be replayed, so it would catch also the
outstanding firmware requests, and would be able to handle them.

Kay

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Kay Sievers]

On Sun, Jan 15, 2012 at 11:02, Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org> wrote:
>> These drivers need to be fixed to load their firmware during ifup,
>> which would be the most flexible solution. That way, it should also
>> work if the driver is built-in, or is loaded from initramfs and no
>> firmware is available.
>> Alternatively, the firmware request should be able to use the aync
>> firmware_request API and not the blocking one.
>
> Will udev now also return the async load only after root is booted if it
> can't be satisfied earlier?

Not sure, let me explain what happens here, maybe it contains the answer:

Udev gets an event for a pci device:
  /devices/pci0000:00/0000:00:1c.1/0000:03:00.0

This device has a modalias, which let's udev load the matching module
into the kernel. The module_init() syscall triggers the firmware
loading request, which causes another event:
  /devices/pci0000:00/0000:00:1c.1/0000:03:00.0/firmware/0000:03:00.0

This event is a direct child of the pci device and udev delays the
execution of child devices until the parent devices return from
handling. This dependency logic is needed for many things to ensure a
proper operation, like partitions which need to make sure the events
for the disk devices are handled before the partition events are
started.

Now the problem, the pcidev event is blocking in modprobe and waits
for the child event it has generated to finish, but udev does not
start the event because the parent still blocks in modprobe ->
deadlock until default firmware timeout of 60 sec. What we want here,
for several reasons not only udev's dependency logic, is that modprobe
never waits for userspace transactions to finish.

> There are a number of devices, particularly
> wireless, that need firmware before they can register with mac80211 for
> capability advertisement reasons.

Right, ideally all firmware loading would be delayed until the netif
is brought up, and that's what we shoudl use if possible. In all other
cases, I think what iwlwifi is doing here looks fine from the
userspace side. It does not register any netdev/mac80211 device, the
firmware request is the only thing that it issued, and modprobe
returns immediately, regardless of the firmware request handling.

If userspace is not responding, the firmware request times out after
60 seconds and the driver is not associated with any hardware. To
retry the firmware loading, the module needs to be unloaded and
reloaded, or the driver needs to be asked to bind to a device again by
writing to the 'bind' in file in the sysfs driver directory.

Firmware requests stay around in the system for by default 60 seconds.
If the driver would be built-in the request would be issued long
before userspace is ready, but udev's coldplug step during bootup will
cause all events to be replayed, so it would catch also the
outstanding firmware requests, and would be able to handle them.

Kay
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Johannes Berg]

On Sun, 2012-01-15 at 16:33 +0100, Kay Sievers wrote:

> > Will udev now also return the async load only after root is booted if it
> > can't be satisfied earlier?
> 
> Not sure, let me explain what happens here, maybe it contains the answer:
> 
> Udev gets an event for a pci device:
>   /devices/pci0000:00/0000:00:1c.1/0000:03:00.0
> 
> This device has a modalias, which let's udev load the matching module
> into the kernel. The module_init() syscall triggers the firmware
> loading request, which causes another event:
>   /devices/pci0000:00/0000:00:1c.1/0000:03:00.0/firmware/0000:03:00.0
> 
> This event is a direct child of the pci device and udev delays the
> execution of child devices until the parent devices return from
> handling. This dependency logic is needed for many things to ensure a
> proper operation, like partitions which need to make sure the events
> for the disk devices are handled before the partition events are
> started.
> 
> Now the problem, the pcidev event is blocking in modprobe and waits
> for the child event it has generated to finish, but udev does not
> start the event because the parent still blocks in modprobe ->
> deadlock until default firmware timeout of 60 sec. What we want here,
> for several reasons not only udev's dependency logic, is that modprobe
> never waits for userspace transactions to finish.

Ok, thanks for the description. I guess to me that means nothing really
changes much in the situation I'm thinking of.

> If userspace is not responding, the firmware request times out after
> 60 seconds and the driver is not associated with any hardware. To
> retry the firmware loading, the module needs to be unloaded and
> reloaded, or the driver needs to be asked to bind to a device again by
> writing to the 'bind' in file in the sysfs driver directory.

Right.

> Firmware requests stay around in the system for by default 60 seconds.
> If the driver would be built-in the request would be issued long
> before userspace is ready, but udev's coldplug step during bootup will
> cause all events to be replayed, so it would catch also the
> outstanding firmware requests, and would be able to handle them.

Hmm, right, but I don't think that matters for the issue I'm asking
about. Let's say you have iwlwifi, built into the kernel, and an
initramfs that doesn't contain the iwlwifi firmware.
What will happen (last I checked anyway) is this: iwlwifi will do an
async firmware request, and the udev in initramfs will say "don't have
it", thus unbinding and you have to sysfs bind or module reload.

What I'm was asking then is this: Can udev know that it is running from
initramfs (presumably that can't be too hard) and simply not reply to
async requests it doesn't have firmware for? Then once the real root is
mounted it could satisfy (or not) firmware requests from the real root.

johannes

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Kay Sievers]

On Mon, Jan 16, 2012 at 09:57, Johannes Berg <johannes@sipsolutions.net> wrote:
> On Sun, 2012-01-15 at 16:33 +0100, Kay Sievers wrote:

> Let's say you have iwlwifi, built into the kernel, and an
> initramfs that doesn't contain the iwlwifi firmware.
> What will happen (last I checked anyway) is this: iwlwifi will do an
> async firmware request, and the udev in initramfs will say "don't have
> it", thus unbinding and you have to sysfs bind or module reload.

Right, currently. if no firmware is found, udev will write "-1" to the
"loading" file in /sys, which will cancel the kernel's firmware
request.

> What I'm was asking then is this: Can udev know that it is running from
> initramfs (presumably that can't be too hard) and simply not reply to
> async requests it doesn't have firmware for? Then once the real root is
> mounted it could satisfy (or not) firmware requests from the real root.

We can surely change it to not cancel the firmware request.

Either by making it aware that we run from initramfs, or by never
cancelling any firmware request and just leave it hanging around for
forever?

We need to decide what's the best model here, if we want a timeout at
all, if yes, how large it should be, and if and when we should cancel
requests.

Kay

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Johannes Berg]

On Mon, 2012-01-16 at 13:05 +0100, Kay Sievers wrote:

> > What I'm was asking then is this: Can udev know that it is running from
> > initramfs (presumably that can't be too hard) and simply not reply to
> > async requests it doesn't have firmware for? Then once the real root is
> > mounted it could satisfy (or not) firmware requests from the real root.
> 
> We can surely change it to not cancel the firmware request.
> 
> Either by making it aware that we run from initramfs, or by never
> cancelling any firmware request and just leave it hanging around for
> forever?

I think not cancelling it from initramfs and then providing or
cancelling it once we have normal root mounted should be sufficient? I
don't see how letting it hang around forever (a minute until it times
out) would be useful.

> We need to decide what's the best model here, if we want a timeout at
> all, if yes, how large it should be, and if and when we should cancel
> requests.

I do think cancelling requests still gives us desirable behaviour in
terms of being able to reproduce it etc., it's just that cancelling it
when we have only partial data (from initramfs) isn't helpful.

johannes

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Arend van Spriel]

On 01/16/2012 09:57 AM, Johannes Berg wrote:
>> Now the problem, the pcidev event is blocking in modprobe and waits
>> >  for the child event it has generated to finish, but udev does not
>> >  start the event because the parent still blocks in modprobe ->
>> >  deadlock until default firmware timeout of 60 sec. What we want here,
>> >  for several reasons not only udev's dependency logic, is that modprobe
>> >  never waits for userspace transactions to finish.
> Ok, thanks for the description. I guess to me that means nothing really
> changes much in the situation I'm thinking of.

I am working on changes in brcm80211 driver and the behaviour changes 
slightly. The async firmware request basically kicks of a kernel thread 
to do the actual request. So the probe finishes successfully regardless 
what the results will be of the actual firmware request. Hence the 
driver is associated with the hardware.

>> >  If userspace is not responding, the firmware request times out after
>> >  60 seconds and the driver is not associated with any hardware. To
>> >  retry the firmware loading, the module needs to be unloaded and
>> >  reloaded, or the driver needs to be asked to bind to a device again by
>> >  writing to the 'bind' in file in the sysfs driver directory.
> Right.
>

If my previous statement is true, what does it mean regarding retrying 
the firmware loading?

Gr. AvS

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Arend van Spriel]

On 01/16/2012 09:57 AM, Johannes Berg wrote:
>> Now the problem, the pcidev event is blocking in modprobe and waits
>> >  for the child event it has generated to finish, but udev does not
>> >  start the event because the parent still blocks in modprobe ->
>> >  deadlock until default firmware timeout of 60 sec. What we want here,
>> >  for several reasons not only udev's dependency logic, is that modprobe
>> >  never waits for userspace transactions to finish.
> Ok, thanks for the description. I guess to me that means nothing really
> changes much in the situation I'm thinking of.

I am working on changes in brcm80211 driver and the behaviour changes 
slightly. The async firmware request basically kicks of a kernel thread 
to do the actual request. So the probe finishes successfully regardless 
what the results will be of the actual firmware request. Hence the 
driver is associated with the hardware.

>> >  If userspace is not responding, the firmware request times out after
>> >  60 seconds and the driver is not associated with any hardware. To
>> >  retry the firmware loading, the module needs to be unloaded and
>> >  reloaded, or the driver needs to be asked to bind to a device again by
>> >  writing to the 'bind' in file in the sysfs driver directory.
> Right.
>

If my previous statement is true, what does it mean regarding retrying 
the firmware loading?

Gr. AvS

--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Johannes Berg]

On Thu, 2012-02-16 at 13:04 +0100, Arend van Spriel wrote:
> On 01/16/2012 09:57 AM, Johannes Berg wrote:
> >> Now the problem, the pcidev event is blocking in modprobe and waits
> >> >  for the child event it has generated to finish, but udev does not
> >> >  start the event because the parent still blocks in modprobe ->
> >> >  deadlock until default firmware timeout of 60 sec. What we want here,
> >> >  for several reasons not only udev's dependency logic, is that modprobe
> >> >  never waits for userspace transactions to finish.
> > Ok, thanks for the description. I guess to me that means nothing really
> > changes much in the situation I'm thinking of.
> 
> I am working on changes in brcm80211 driver and the behaviour changes 
> slightly. The async firmware request basically kicks of a kernel thread 
> to do the actual request. So the probe finishes successfully regardless 
> what the results will be of the actual firmware request. Hence the 
> driver is associated with the hardware.

This is true.

> >> >  If userspace is not responding, the firmware request times out after
> >> >  60 seconds and the driver is not associated with any hardware. To
> >> >  retry the firmware loading, the module needs to be unloaded and
> >> >  reloaded, or the driver needs to be asked to bind to a device again by
> >> >  writing to the 'bind' in file in the sysfs driver directory.
> > Right.
> >
> 
> If my previous statement is true, what does it mean regarding retrying 
> the firmware loading?

When the firmware loading fails, the driver should unbind from the
device that it failed for, and the retrying behaviour doesn't change
(and requires a rebind)

johannes

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Arend van Spriel]

On 02/16/2012 01:38 PM, Johannes Berg wrote:
> On Thu, 2012-02-16 at 13:04 +0100, Arend van Spriel wrote:
>> On 01/16/2012 09:57 AM, Johannes Berg wrote:
>>>> Now the problem, the pcidev event is blocking in modprobe and waits
>>>>>   for the child event it has generated to finish, but udev does not
>>>>>   start the event because the parent still blocks in modprobe ->
>>>>>   deadlock until default firmware timeout of 60 sec. What we want here,
>>>>>   for several reasons not only udev's dependency logic, is that modprobe
>>>>>   never waits for userspace transactions to finish.
>>> Ok, thanks for the description. I guess to me that means nothing really
>>> changes much in the situation I'm thinking of.
>>
>> I am working on changes in brcm80211 driver and the behaviour changes
>> slightly. The async firmware request basically kicks of a kernel thread
>> to do the actual request. So the probe finishes successfully regardless
>> what the results will be of the actual firmware request. Hence the
>> driver is associated with the hardware.
>
> This is true.
>
>>>>>   If userspace is not responding, the firmware request times out after
>>>>>   60 seconds and the driver is not associated with any hardware. To
>>>>>   retry the firmware loading, the module needs to be unloaded and
>>>>>   reloaded, or the driver needs to be asked to bind to a device again by
>>>>>   writing to the 'bind' in file in the sysfs driver directory.
>>> Right.
>>>
>>
>> If my previous statement is true, what does it mean regarding retrying
>> the firmware loading?
>
> When the firmware loading fails, the driver should unbind from the
> device that it failed for, and the retrying behaviour doesn't change
> (and requires a rebind)
>
> johannes
>
>

Thanks, Johannes

That helps. My driver was happily bound to the device until unloading 
the driver or unplugging the device.

Gr. AvS

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Arend van Spriel]

On 02/16/2012 01:38 PM, Johannes Berg wrote:
> On Thu, 2012-02-16 at 13:04 +0100, Arend van Spriel wrote:
>> On 01/16/2012 09:57 AM, Johannes Berg wrote:
>>>> Now the problem, the pcidev event is blocking in modprobe and waits
>>>>>   for the child event it has generated to finish, but udev does not
>>>>>   start the event because the parent still blocks in modprobe ->
>>>>>   deadlock until default firmware timeout of 60 sec. What we want here,
>>>>>   for several reasons not only udev's dependency logic, is that modprobe
>>>>>   never waits for userspace transactions to finish.
>>> Ok, thanks for the description. I guess to me that means nothing really
>>> changes much in the situation I'm thinking of.
>>
>> I am working on changes in brcm80211 driver and the behaviour changes
>> slightly. The async firmware request basically kicks of a kernel thread
>> to do the actual request. So the probe finishes successfully regardless
>> what the results will be of the actual firmware request. Hence the
>> driver is associated with the hardware.
>
> This is true.
>
>>>>>   If userspace is not responding, the firmware request times out after
>>>>>   60 seconds and the driver is not associated with any hardware. To
>>>>>   retry the firmware loading, the module needs to be unloaded and
>>>>>   reloaded, or the driver needs to be asked to bind to a device again by
>>>>>   writing to the 'bind' in file in the sysfs driver directory.
>>> Right.
>>>
>>
>> If my previous statement is true, what does it mean regarding retrying
>> the firmware loading?
>
> When the firmware loading fails, the driver should unbind from the
> device that it failed for, and the retrying behaviour doesn't change
> (and requires a rebind)
>
> johannes
>
>

Thanks, Johannes

That helps. My driver was happily bound to the device until unloading 
the driver or unplugging the device.

Gr. AvS

--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Johannes Berg]

On Thu, 2012-02-16 at 14:09 +0100, Arend van Spriel wrote:


> > When the firmware loading fails, the driver should unbind from the
> > device that it failed for, and the retrying behaviour doesn't change
> > (and requires a rebind)

> That helps. My driver was happily bound to the device until unloading 
> the driver or unplugging the device.

Right, if it fails you should call device_release_driver()

johannes

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Johannes Berg]

On Thu, 2012-02-16 at 14:09 +0100, Arend van Spriel wrote:


> > When the firmware loading fails, the driver should unbind from the
> > device that it failed for, and the retrying behaviour doesn't change
> > (and requires a rebind)

> That helps. My driver was happily bound to the device until unloading 
> the driver or unplugging the device.

Right, if it fails you should call device_release_driver()

johannes

--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Arend van Spriel]

On 01/14/2012 04:25 PM, Kay Sievers wrote:
> We changed udev to strictly enforce sequence order and dependency
> handling of events. This seems to break some netdev drivers which
> block in modprobe until the firmware from userspace is loaded into the
> driver.

Working on the issue complicated the behavior of the driver so I decided 
to take a step back to determine the actual issue.

> These are the drivers we received bug reports so far. We didn't look
> into any of details of the drivers, but according to the logs, they
> seem to block for 60 seconds in modprobe, when userspace is not
> behaving like expected:
>    bnx2/bnx2-mips-06-6.2.1.fw
>    rtlwifi/rtl8192sefw.bin
>    brcm/bcm43xx-0.fw

The main problem is that the init_module() syscall should not block. 
However, the driver I am responsible for (brcm80211) does not request 
firmware in the init_module() path. It does that on the probe() 
callback. So the problem is that the probe() code path is done in the 
context of the init_module() syscall. So the thing to do is decouple the 
probe() callback from the init_module() syscall.

One option is using the nowait version of request_firmware(), but 
another option I am considering is to defer the driver registration 
using a workqueue and schedule it in the init_module() syscall. That way 
I think I can avoid having to call the device_unregister_driver() when 
firmware loading fails.

Another thing to consider is to change the driver core subsystem and 
assure the probe() callback is never done in the init_module() context.

Any opinions?

Gr. AvS

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Arend van Spriel]

On 01/14/2012 04:25 PM, Kay Sievers wrote:
> We changed udev to strictly enforce sequence order and dependency
> handling of events. This seems to break some netdev drivers which
> block in modprobe until the firmware from userspace is loaded into the
> driver.

Working on the issue complicated the behavior of the driver so I decided 
to take a step back to determine the actual issue.

> These are the drivers we received bug reports so far. We didn't look
> into any of details of the drivers, but according to the logs, they
> seem to block for 60 seconds in modprobe, when userspace is not
> behaving like expected:
>    bnx2/bnx2-mips-06-6.2.1.fw
>    rtlwifi/rtl8192sefw.bin
>    brcm/bcm43xx-0.fw

The main problem is that the init_module() syscall should not block. 
However, the driver I am responsible for (brcm80211) does not request 
firmware in the init_module() path. It does that on the probe() 
callback. So the problem is that the probe() code path is done in the 
context of the init_module() syscall. So the thing to do is decouple the 
probe() callback from the init_module() syscall.

One option is using the nowait version of request_firmware(), but 
another option I am considering is to defer the driver registration 
using a workqueue and schedule it in the init_module() syscall. That way 
I think I can avoid having to call the device_unregister_driver() when 
firmware loading fails.

Another thing to consider is to change the driver core subsystem and 
assure the probe() callback is never done in the init_module() context.

Any opinions?

Gr. AvS

--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Kay Sievers]

On Mon, Jan 16, 2012 at 1:16 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
> On Mon, 2012-01-16 at 13:05 +0100, Kay Sievers wrote:
>
>> > What I'm was asking then is this: Can udev know that it is running from
>> > initramfs (presumably that can't be too hard) and simply not reply to
>> > async requests it doesn't have firmware for? Then once the real root is
>> > mounted it could satisfy (or not) firmware requests from the real root.
>>
>> We can surely change it to not cancel the firmware request.
>>
>> Either by making it aware that we run from initramfs, or by never
>> cancelling any firmware request and just leave it hanging around for
>> forever?

Never say 6 months is a long time to reply. :)

Fedora uses systemd in the initramfs now, which made it trivial to
implement this, and to leave the firmware requests hanging around
until we reach in the real rootfs and know if the firmware file is
available:
  http://cgit.freedesktop.org/systemd/systemd/commit/?id=39177382a4f92a834b568d6ae5d750eb2a5a86f9

The logic to tell udev that it runs in the initramfs could easily be
implemented by other initramfs tools than dracut, but they usually do
not really follow what we do here, so this might for now only work on
recent systems using dracut.

Cheers,
Kay

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Johannes Berg]

On Thu, 2012-07-19 at 12:46 +0200, Kay Sievers wrote:

> >> > What I'm was asking then is this: Can udev know that it is running from
> >> > initramfs (presumably that can't be too hard) and simply not reply to
> >> > async requests it doesn't have firmware for? Then once the real root is
> >> > mounted it could satisfy (or not) firmware requests from the real root.
> >>
> >> We can surely change it to not cancel the firmware request.
> >>
> >> Either by making it aware that we run from initramfs, or by never
> >> cancelling any firmware request and just leave it hanging around for
> >> forever?
> 
> Never say 6 months is a long time to reply. :)

Hehe :-)

> Fedora uses systemd in the initramfs now, which made it trivial to
> implement this, and to leave the firmware requests hanging around
> until we reach in the real rootfs and know if the firmware file is
> available:
>   http://cgit.freedesktop.org/systemd/systemd/commit/?id=39177382a4f92a834b568d6ae5d750eb2a5a86f9
> 
> The logic to tell udev that it runs in the initramfs could easily be
> implemented by other initramfs tools than dracut, but they usually do
> not really follow what we do here, so this might for now only work on
> recent systems using dracut.

Ok, too bad there wasn't a generic way, but at least there's a way
now :-)

Anyway, I think this is a good thing.

johannes

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Tom Gundersen]

On Tue, Jul 24, 2012 at 4:16 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
>> The logic to tell udev that it runs in the initramfs could easily be
>> implemented by other initramfs tools than dracut, but they usually do
>> not really follow what we do here, so this might for now only work on
>> recent systems using dracut.
>
> Ok, too bad there wasn't a generic way, but at least there's a way
> now :-)

If I understand the code correctly, it should be enough to put a file
/etc/initrd-release in the initramfs for udev to do the right thing.
But please correct me if I'm wrong Kay.

-t

Re: calling request_firmware() from module init will not work with recent/future udev versions

[Kay Sievers]

On Tue, Jul 24, 2012 at 4:32 PM, Tom Gundersen <teg@jklm.no> wrote:
> On Tue, Jul 24, 2012 at 4:16 PM, Johannes Berg
> <johannes@sipsolutions.net> wrote:
>>> The logic to tell udev that it runs in the initramfs could easily be
>>> implemented by other initramfs tools than dracut, but they usually do
>>> not really follow what we do here, so this might for now only work on
>>> recent systems using dracut.
>>
>> Ok, too bad there wasn't a generic way, but at least there's a way
>> now :-)
>
> If I understand the code correctly, it should be enough to put a file
> /etc/initrd-release in the initramfs for udev to do the right thing.
> But please correct me if I'm wrong Kay.

The current check in udev/systemd is the existence of:
  /etc/initrd-release
and the / filesystem must be tmpfs/ramfs to get the environment
recognized as 'initrd'.

Kay