network manager vs. missing firmware

network manager vs. missing firmware

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 592 bytes --]

Hi,

Just remembered this issue from last year's summit and figured I'd bring
it up again since we never made progress on it.

Is there any idea other than standardising on a new error code
-ENOFIRMWARE that can be returned from device up or association or
wherever makes sense for the driver. Or do we also mandate that it be
returned on device up, and for example never on module load? I think
it's mostly a question of documentation/driver acceptance policy/driver
review as well as a question of whether we can get a new error code into
the kernel or not.........

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: network manager vs. missing firmware

[Dan Williams]

On Tue, 2007-02-13 at 19:41 +0100, Johannes Berg wrote:
> Hi,
> 
> Just remembered this issue from last year's summit and figured I'd bring
> it up again since we never made progress on it.
> 
> Is there any idea other than standardising on a new error code
> -ENOFIRMWARE that can be returned from device up or association or
> wherever makes sense for the driver. Or do we also mandate that it be
> returned on device up, and for example never on module load? I think
> it's mostly a question of documentation/driver acceptance policy/driver
> review as well as a question of whether we can get a new error code into
> the kernel or not.........

The simpler, the better I think; but it's complicated.  A sample of
current drivers and when they call request_firmware() from a 2.6.19
kernel:

on dev->open: atmel, prism54, bcm43xx (softmac)
on dev->init: ipw2100, ipw2200
on probe: zd1201, libertas, spectrum_cs

So this isn't really consistent.  I'm unclear as to why the ipw cards
need it on init rather than on dev open, but hey, why make things
easier?

With USB devices (and spectrum_cs I guess), most can't do _anything_
until you upload firmware to them, so the firmware upload really does
need to happen on probe, not device up.  I guess we have two main
use-cases to target here.  We all know what the case on dev up is
(-ENOFILE or whatever gets returned from request_firmware()) but I have
no idea how to detect the error on module load (ie, probe) because
that's _waaay_ before NM gets into the business.  The kernel should
normally be autoloading the modules for you, otherwise half the time NM
won't even know the device is there in the first place.

So for USB devices and the like, if the firmware isn't there, the module
will have already failed to load, and HAL won't know it's a wireless
interface because it's not in sysfs as a wireless interface, and
therefore NM won't know about it, and therefore there is no error to
catch.  I don't think NM should be in the business of searching through
USB devices to find ones that don't report themselves as wireless
devices.

So there are two questions:

1) What's the failure mode for request_firmware() error on probe?  How
does a userspace program notice that the hardware has no firmware?

2) What's the failure case for ipw* drivers that do it on dev->init, why
are they different, and how would a userspace program know that the
firmware load failed?

We need to make sure that this is fixed for d80211 drivers too; at least
try to specify when firmware should be loaded if we can.

Dan

Re: network manager vs. missing firmware

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 3337 bytes --]

On Tue, 2007-02-13 at 14:23 -0500, Dan Williams wrote:

> The simpler, the better I think; but it's complicated.  A sample of
> current drivers and when they call request_firmware() from a 2.6.19
> kernel:
> 
> on dev->open: atmel, prism54, bcm43xx (softmac)
> on dev->init: ipw2100, ipw2200
> on probe: zd1201, libertas, spectrum_cs

ouch.

> So this isn't really consistent.  I'm unclear as to why the ipw cards
> need it on init rather than on dev open, but hey, why make things
> easier?

When is ->init() called anyway? Isn't it when you register the netdev so
it would be on probe?

> With USB devices (and spectrum_cs I guess), most can't do _anything_
> until you upload firmware to them, so the firmware upload really does
> need to happen on probe, not device up.  I guess we have two main
> use-cases to target here.  We all know what the case on dev up is
> (-ENOFILE or whatever gets returned from request_firmware())

-ENOENT :)
But then we need to specify that returning -ENOENT from dev up is not
valid in any other case I guess.

> but I have
> no idea how to detect the error on module load (ie, probe) because
> that's _waaay_ before NM gets into the business.  The kernel should
> normally be autoloading the modules for you, otherwise half the time NM
> won't even know the device is there in the first place.

Well, the kernel will just show in sysfs what the system can support and
udev will autoload modules based on that.

But actually, I don't think you fail module load. The thing is that when
->probe() fails the module will usually (unless you write code) be
loaded but not bound to any device.

> So for USB devices and the like, if the firmware isn't there, the module
> will have already failed to load, and HAL won't know it's a wireless
> interface because it's not in sysfs as a wireless interface, and
> therefore NM won't know about it, and therefore there is no error to
> catch.  I don't think NM should be in the business of searching through
> USB devices to find ones that don't report themselves as wireless
> devices.

Yeah, that's a good point.

> So there are two questions:
> 
> 1) What's the failure mode for request_firmware() error on probe?  How
> does a userspace program notice that the hardware has no firmware?

As I said above, the driver will be loaded but no device will be bound
to it. That could actually be taken as a sign... If you look into
/sys/module/*/drivers you can find those values for * that are known
wireless modules and have no devices associated. ;)

Of course, DON'T ACTUALLY DO THIS! This breaks for various reasons:
 * drivers that are built-in
 * computers with multiple wireless devices that have a single driver
   but need different firmware (can happen with bcm43xx)
(and that's just those that I can think of right now)

This is a really tough case to recognise for a program.

But then again, even usb drivers don't need to load firmware to know
that there's a device and to register a netdev. They might not be able
to assign a mac address but that shouldn't hurt (though we have to make
sure d80211 can cope with this), so we can mandate deferring it at least
to netdev registration, and then I don't see why ipw* needs it right
away there as opposed to open().

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: network manager vs. missing firmware

[Michael Wu]

[-- Attachment #1: Type: text/plain, Size: 609 bytes --]

On Tuesday 13 February 2007 14:37, Johannes Berg wrote:
> > So this isn't really consistent.  I'm unclear as to why the ipw cards
> > need it on init rather than on dev open, but hey, why make things
> > easier?
>
> When is ->init() called anyway? Isn't it when you register the netdev so
> it would be on probe?
>
Yes, ipw2200 does it on probe... but I know it can be pushed off to open. I've 
done it. However.. I think intersil cards really need firmware to read the 
eeprom. prism54 currently sets a fake MAC address so they can push it off to 
open, but I think this is wrong.

-Michael Wu

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: network manager vs. missing firmware

[Luis R. Rodriguez]

On 2/13/07, Michael Wu <flamingice@sourmilk.net> wrote:
> On Tuesday 13 February 2007 14:37, Johannes Berg wrote:
> > > So this isn't really consistent.  I'm unclear as to why the ipw cards
> > > need it on init rather than on dev open, but hey, why make things
> > > easier?
> >
> > When is ->init() called anyway? Isn't it when you register the netdev so
> > it would be on probe?
> >
> Yes, ipw2200 does it on probe... but I know it can be pushed off to open. I've
> done it. However.. I think intersil cards really need firmware to read the
> eeprom.

This is correct

> prism54 currently sets a fake MAC address so they can push it off to
> open, but I think this is wrong.

This was suggested by Jean a long time ago after we asked for advice
on what to do since we couldn't read the MAC before loading the
firmware. Whatever we decide on we should standardize on it in case
other new cards end up with same issue. One option was to have the
probe() fail if no firmware was present but obviously this would force
you to reload the module/reboot if you then get the firmware. We
decided to leave the firmware upload on open() to overcome this but
the problem then was the lack of a MAC address.

Anyone know if wireless network devices the only ones requiring a
firmware? If not then we should think of slapping something onto
net_device. Otherwise we should distinguish the requirement on
cfg80211_config so userspace tools know WTF is going on and inform the
user appropriately.

  Luis

Re: network manager vs. missing firmware

[Dan Williams]

On Tue, 2007-02-13 at 15:13 -0500, Luis R. Rodriguez wrote:
> On 2/13/07, Michael Wu <flamingice@sourmilk.net> wrote:
> > On Tuesday 13 February 2007 14:37, Johannes Berg wrote:
> > > > So this isn't really consistent.  I'm unclear as to why the ipw cards
> > > > need it on init rather than on dev open, but hey, why make things
> > > > easier?
> > >
> > > When is ->init() called anyway? Isn't it when you register the netdev so
> > > it would be on probe?
> > >
> > Yes, ipw2200 does it on probe... but I know it can be pushed off to open. I've
> > done it. However.. I think intersil cards really need firmware to read the
> > eeprom.
> 
> This is correct
> 
> > prism54 currently sets a fake MAC address so they can push it off to
> > open, but I think this is wrong.
> 
> This was suggested by Jean a long time ago after we asked for advice
> on what to do since we couldn't read the MAC before loading the
> firmware. Whatever we decide on we should standardize on it in case
> other new cards end up with same issue. One option was to have the
> probe() fail if no firmware was present but obviously this would force
> you to reload the module/reboot if you then get the firmware. We
> decided to leave the firmware upload on open() to overcome this but
> the problem then was the lack of a MAC address.
> 
> Anyone know if wireless network devices the only ones requiring a
> firmware? If not then we should think of slapping something onto
> net_device. Otherwise we should distinguish the requirement on
> cfg80211_config so userspace tools know WTF is going on and inform the
> user appropriately.

No; others include Apple iSight cameras (the built-in USB variants),
bluetooth devices (usually USB too), Keyspan serial adapters (usually
USB), a bunch of stuff in drivers/media/video/, some non-USB network
adapters, and some adaptec and qlogic sci drivers.

It seems that most USB devices require firmware no matter what type they
are.  Beyond that:

probe: scsi/qla2xxx, net/myri10ge, scsi/aic94xx, net/spidernet
open: serial/icom

Dan

Re: network manager vs. missing firmware

[James Ketrenos]

Dan Williams wrote:

> The simpler, the better I think; but it's complicated.  A sample of
> current drivers and when they call request_firmware() from a 2.6.19
> kernel:
> 
> on dev->open: atmel, prism54, bcm43xx (softmac)
> on dev->init: ipw2100, ipw2200

request_firmware is also used during mode changes (ie, BSS to IBSS to 
MONITOR)

Perhaps solving the larger problem of "the driver knows something is 
horribly wrong, but has no way to tell the user" would be good. 
Firmware not being there is bad.  There are also periodic failures or 
usage conditions that keep things from working.  Right now we might have 
failures due to:

	FIRMWARE missing
	RF KILL active

but maybe tomorrow we would have:

	Conflict with other active wifi device / coexistence

or any number of solution specific error codes that the vendor / driver 
writer would be able to articulate in a text string how to resolve.

Perhaps a simple cfg80211 interface for querying the interface status 
and and associated text string (if in an error condition).  Missing 
firmware or RF kill could have standard numbers and the text string 
could be standardized to return just the name of the firmware file expected.

Even if it didn't get standardized across all devices, NM or other 
utilities could provide a standard place for the user to look at the 
error and then google on it.  Right now, things break and the user can't 
figure out why without looking at the kernel log (which many users don't 
even know exist).

James

Re: network manager vs. missing firmware

[Luis R. Rodriguez]

On 2/13/07, James Ketrenos <jketreno@linux.intel.com> wrote:
> Dan Williams wrote:
>
> > The simpler, the better I think; but it's complicated.  A sample of
> > current drivers and when they call request_firmware() from a 2.6.19
> > kernel:
> >
> > on dev->open: atmel, prism54, bcm43xx (softmac)
> > on dev->init: ipw2100, ipw2200
>
> request_firmware is also used during mode changes (ie, BSS to IBSS to
> MONITOR)

This is a bit off topic but if that's the case then we should also add
the check for Master mode and if so check to see if the AP firmware is
available and switch to on master mode. We'd then need to incorporate
the ap ipw2200 support on ipw2200

http://sourceforge.net/projects/ipw2200-ap

I wanted to try a patch to send along but I cannot find the relevent
code except the firmware there. Perhaps yorkliu can supply that
though.

> Perhaps solving the larger problem of "the driver knows something is
> horribly wrong, but has no way to tell the user" would be good.
> Firmware not being there is bad.  There are also periodic failures or
> usage conditions that keep things from working.  Right now we might have
> failures due to:
>
>         FIRMWARE missing

We should figure out a way to handle this gracefully and deal with it
in a standard way. See bellow for a suggestion.

>         RF KILL active

Based on previous conversation on this before it seems each laptop has
its own mechanism for handling RF KILL. I am not sure if there is a
way to detect if this button is on in all cases. Can someone confirm?
A suggestion on handling this is provided bellow.

>
> but maybe tomorrow we would have:
>
>         Conflict with other active wifi device / coexistence
>
> or any number of solution specific error codes that the vendor / driver
> writer would be able to articulate in a text string how to resolve.
>
> Perhaps a simple cfg80211 interface for querying the interface status
> and and associated text string (if in an error condition).  Missing
> firmware or RF kill could have standard numbers and the text string
> could be standardized to return just the name of the firmware file expected.
>
> Even if it didn't get standardized across all devices, NM or other
> utilities could provide a standard place for the user to look at the
> error and then google on it.

I think we should be able to standardize at least within wireless
devices. If we put a wrapper to request_firmware() which is specific
to wireless devices, for example, we could then handle the errors
gracefully ourselves without regard for now for the other devices. The
same applies to the RF KILL switch on open()/close() but I am not sure
if we can detect it on/off in all cases.

> Right now, things break and the user can't
> figure out why without looking at the kernel log (which many users don't
> even know exist).

True.

  Luis

Re: network manager vs. missing firmware

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 346 bytes --]

On Tue, 2007-02-13 at 16:36 -0800, James Ketrenos wrote:

> Perhaps solving the larger problem of "the driver knows something is 
> horribly wrong, but has no way to tell the user" would be good. 

Yes, but I think it's out of scope for wireless.

Maybe we should come up with a generic "driver problem" userspace
interface.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: network manager vs. missing firmware

[James Ketrenos]

Luis R. Rodriguez wrote:
> On 2/13/07, James Ketrenos <jketreno@linux.intel.com> wrote:
>> Dan Williams wrote:
>>
>> > The simpler, the better I think; but it's complicated.  A sample of
>> > current drivers and when they call request_firmware() from a 2.6.19
>> > kernel:
>> >
>> > on dev->open: atmel, prism54, bcm43xx (softmac)
>> > on dev->init: ipw2100, ipw2200
>>
>> request_firmware is also used during mode changes (ie, BSS to IBSS to
>> MONITOR)
> 
> This is a bit off topic but if that's the case then we should also add
> the check for Master mode and if so check to see if the AP firmware is
> available and switch to on master mode. We'd then need to incorporate
> the ap ipw2200 support on ipw2200
> 
> http://sourceforge.net/projects/ipw2200-ap
> 
> I wanted to try a patch to send along but I cannot find the relevent
> code except the firmware there. Perhaps yorkliu can supply that
> though.

Under the 'Download' link there is v0.3 available -- 
http://sourceforge.net/project/showfiles.php?group_id=149502

I haven't tried it myself.  It hasn't been updated since Sept so 
probably needs some kernel API cleanup.

James