Firmware versioning best practices

Firmware versioning best practices

[Luis R. Rodriguez]

The ath_hif_usb driver will require the ar9271 firmware file but in
the future an open firmware might become available. The ar9170 driver
already is under the same situation already: a closed firmware is
available but an open firmware can be used, only thing is ar9170 uses
the same firmware name for both. We *could* change ar9170 to use the
Intel practice of tagging a version at the end of each firmware
release, like ar9170-1.fw but ar9170 originally was implemented with a
2-stage firmware requirement and so ar9170-1.fw is already taken.

ar9170 still needs a solution for the different firmwares, once we
start supporting the open firmware through some sort of release but
I'd like to address ath_hif_usb now early so that we don't run into
these snags and use some decent convention that is easy to follow.

As I noted above, Intel seems to use the device-1.fw, device-2.fw
naming convention. Is this the best approach? Or shall we have the
same firmware filename and simply query the firmware for a map of
capabilities? Any other ideas?

  Luis

Re: Firmware versioning best practices

[Pavel Roskin]

On Mon, 2009-09-28 at 15:17 -0700, Luis R. Rodriguez wrote:
> The ath_hif_usb driver will require the ar9271 firmware file but in
> the future an open firmware might become available. The ar9170 driver
> already is under the same situation already: a closed firmware is
> available but an open firmware can be used, only thing is ar9170 uses
> the same firmware name for both. We *could* change ar9170 to use the
> Intel practice of tagging a version at the end of each firmware
> release, like ar9170-1.fw but ar9170 originally was implemented with a
> 2-stage firmware requirement and so ar9170-1.fw is already taken.

Versions don't have to start with 1.  We could start e.g. with 10.

> ar9170 still needs a solution for the different firmwares, once we
> start supporting the open firmware through some sort of release but
> I'd like to address ath_hif_usb now early so that we don't run into
> these snags and use some decent convention that is easy to follow.

We could use ar9170-apiversion-codeverestion.fw and link it to
ar9170-apiversion.fw.  That is, if the open firmware version is 0.9.0
and it was compiled for API version 12, the filename would be
ar9170-12-0.9.0.fw and it could be linked to ar9170-12.fw.

> As I noted above, Intel seems to use the device-1.fw, device-2.fw
> naming convention. Is this the best approach? Or shall we have the
> same firmware filename and simply query the firmware for a map of
> capabilities? Any other ideas?

Distinctive names are good for simplicity of administration and the
capabilities are good for the sanity of the driver.  But I don't see why
we cannot have both.

-- 
Regards,
Pavel Roskin

Re: Firmware versioning best practices

[Luis R. Rodriguez]

On Mon, Sep 28, 2009 at 3:33 PM, Pavel Roskin <proski@gnu.org> wrote:
> On Mon, 2009-09-28 at 15:17 -0700, Luis R. Rodriguez wrote:
>> The ath_hif_usb driver will require the ar9271 firmware file but in
>> the future an open firmware might become available. The ar9170 driver
>> already is under the same situation already: a closed firmware is
>> available but an open firmware can be used, only thing is ar9170 uses
>> the same firmware name for both. We *could* change ar9170 to use the
>> Intel practice of tagging a version at the end of each firmware
>> release, like ar9170-1.fw but ar9170 originally was implemented with a
>> 2-stage firmware requirement and so ar9170-1.fw is already taken.
>
> Versions don't have to start with 1.  We could start e.g. with 10.

Point taken.

>> ar9170 still needs a solution for the different firmwares, once we
>> start supporting the open firmware through some sort of release but
>> I'd like to address ath_hif_usb now early so that we don't run into
>> these snags and use some decent convention that is easy to follow.
>
> We could use ar9170-apiversion-codeverestion.fw and link it to
> ar9170-apiversion.fw.  That is, if the open firmware version is 0.9.0
> and it was compiled for API version 12, the filename would be
> ar9170-12-0.9.0.fw and it could be linked to ar9170-12.fw.

Nice, I like this convention.

>> As I noted above, Intel seems to use the device-1.fw, device-2.fw
>> naming convention. Is this the best approach? Or shall we have the
>> same firmware filename and simply query the firmware for a map of
>> capabilities? Any other ideas?
>
> Distinctive names are good for simplicity of administration and the
> capabilities are good for the sanity of the driver.  But I don't see why
> we cannot have both.

OK sure, thanks for the feedback, at least now I know what naming
scheme to use. The bitmap stuff will have to come later through some
sort of open firmware as I am not sure if we have this with the closed
one.

  Luis

Re: Firmware versioning best practices

[Marcel Holtmann]

Hi Luis,

> The ath_hif_usb driver will require the ar9271 firmware file but in
> the future an open firmware might become available. The ar9170 driver
> already is under the same situation already: a closed firmware is
> available but an open firmware can be used, only thing is ar9170 uses
> the same firmware name for both. We *could* change ar9170 to use the
> Intel practice of tagging a version at the end of each firmware
> release, like ar9170-1.fw but ar9170 originally was implemented with a
> 2-stage firmware requirement and so ar9170-1.fw is already taken.
> 
> ar9170 still needs a solution for the different firmwares, once we
> start supporting the open firmware through some sort of release but
> I'd like to address ath_hif_usb now early so that we don't run into
> these snags and use some decent convention that is easy to follow.
> 
> As I noted above, Intel seems to use the device-1.fw, device-2.fw
> naming convention. Is this the best approach? Or shall we have the
> same firmware filename and simply query the firmware for a map of
> capabilities? Any other ideas?

the general rule of thumb is that if you break the firmware API/ABI or
change it then the firmware name should be different. So for example if
you have some new driver functionality that requires new firmware then
you better use a new firmware name. Otherwise it is just fine to use the
same name if the functionality is not changing. If you can actually
detect the new firmware features from the firmware filename, then you
might not even have to bother with a different name. However keep in
mind that you still need to load at least the previous version of the
firmware and keep that working.

For open source firmware vs binary blobs, we don't really have a good
reference. In theory the driver should always try loading both and if
one succeeds then go with it. At no point the driver should stop working
only because a firmware is missing while either an open source or binary
one for that matter would have been available.

If you have a binary and an open source available, then you might wanna
have a Kconfig option which to try first. Like prefer the open source
over the binary one, but at the end of the day most system will ship
with only one anyway. And a module parameter would work here as well.

Regards

Marcel

Re: Firmware versioning best practices

[John W. Linville]

On Mon, Sep 28, 2009 at 04:52:17PM -0700, Marcel Holtmann wrote:
> Hi Luis,
> 
> > The ath_hif_usb driver will require the ar9271 firmware file but in
> > the future an open firmware might become available. The ar9170 driver
> > already is under the same situation already: a closed firmware is
> > available but an open firmware can be used, only thing is ar9170 uses
> > the same firmware name for both. We *could* change ar9170 to use the
> > Intel practice of tagging a version at the end of each firmware
> > release, like ar9170-1.fw but ar9170 originally was implemented with a
> > 2-stage firmware requirement and so ar9170-1.fw is already taken.
> > 
> > ar9170 still needs a solution for the different firmwares, once we
> > start supporting the open firmware through some sort of release but
> > I'd like to address ath_hif_usb now early so that we don't run into
> > these snags and use some decent convention that is easy to follow.
> > 
> > As I noted above, Intel seems to use the device-1.fw, device-2.fw
> > naming convention. Is this the best approach? Or shall we have the
> > same firmware filename and simply query the firmware for a map of
> > capabilities? Any other ideas?
> 
> the general rule of thumb is that if you break the firmware API/ABI or
> change it then the firmware name should be different. So for example if
> you have some new driver functionality that requires new firmware then
> you better use a new firmware name. Otherwise it is just fine to use the
> same name if the functionality is not changing. If you can actually
> detect the new firmware features from the firmware filename, then you
> might not even have to bother with a different name. However keep in
> mind that you still need to load at least the previous version of the
> firmware and keep that working.
> 
> For open source firmware vs binary blobs, we don't really have a good
> reference. In theory the driver should always try loading both and if
> one succeeds then go with it. At no point the driver should stop working
> only because a firmware is missing while either an open source or binary
> one for that matter would have been available.
> 
> If you have a binary and an open source available, then you might wanna
> have a Kconfig option which to try first. Like prefer the open source
> over the binary one, but at the end of the day most system will ship
> with only one anyway. And a module parameter would work here as well.

This seems like a good piece of advise (as does Pavel's).  Perhaps
someone should capture this on the wiki?

John
-- 
John W. Linville		Someday the world will need a hero, and you
linville@tuxdriver.com			might be all we have.  Be ready.

Re: Firmware versioning best practices

[Holger Schurig]

> Or shall we have the same firmware filename and simply query
> the firmware for a map of capabilities? Any other ideas?

Don't put the version into the filename. This is not a common 
practice for Linux / BSD / whatever systems. Usually you have 
a "kmail" file, not a kmail3.5, kmail4.0 and kmail4.2 file.

Versions or capability maps can be stored inside the firmware and 
queried at load time. E.g. the libertas driver does it that way.

If you make your firmware redistributable (which I recommend), 
the version will also be stored in the package metadata, e.g. 
the rpm or deb file and the infrastructure for rpm/yum deb/apt.

-- 
http://www.holgerschurig.de

Re: Firmware versioning best practices

[Tomas Winkler]

On Tue, Sep 29, 2009 at 8:59 AM, Holger Schurig
<hs4233@mail.mn-solutions.de> wrote:
>> Or shall we have the same firmware filename and simply query
>> the firmware for a map of capabilities? Any other ideas?
>
> Don't put the version into the filename. This is not a common
> practice for Linux / BSD / whatever systems. Usually you have
> a "kmail" file, not a kmail3.5, kmail4.0 and kmail4.2 file.

I think in this context libyyy.so.x.y.z is better analogy. firmware is
not an executable
What is the reasoning behind this common practice?

> Versions or capability maps can be stored inside the firmware and
> queried at load time. E.g. the libertas driver does it that way.

It's only check if it fits but cannot fall back to an older version.
.
>
> If you make your firmware redistributable (which I recommend),
> the version will also be stored in the package metadata, e.g.
> the rpm or deb file and the infrastructure for rpm/yum deb/apt.
>
> --
> http://www.holgerschurig.de
> --
> To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Thanks
Tomas

Re: Firmware versioning best practices

[Holger Schurig]

> > Don't put the version into the filename. This is not a common
> > practice for Linux / BSD / whatever systems. Usually you have
> > a "kmail" file, not a kmail3.5, kmail4.0 and kmail4.2 file.
> 
> I think in this context libyyy.so.x.y.z is better analogy.
> firmware is not an executable What is the reasoning behind this
> common practice? 

There's no version of the library in the file-name, but the 
version of the API. So if the API changes, and might break users 
of the API, you increase the filename.

But you won't have a libc-2.3.6.so file. Instead you have a 
package "libc6_2.3.6.ds1-13etch9_i386.deb" which contains the 
file libc.so.6.

-- 
http://www.holgerschurig.de

Re: Firmware versioning best practices

[Luis R. Rodriguez]

On Mon, Sep 28, 2009 at 4:42 PM, John W. Linville
<linville@tuxdriver.com> wrote:
> On Mon, Sep 28, 2009 at 04:52:17PM -0700, Marcel Holtmann wrote:
>> Hi Luis,
>>
>> > The ath_hif_usb driver will require the ar9271 firmware file but in
>> > the future an open firmware might become available. The ar9170 driver
>> > already is under the same situation already: a closed firmware is
>> > available but an open firmware can be used, only thing is ar9170 uses
>> > the same firmware name for both. We *could* change ar9170 to use the
>> > Intel practice of tagging a version at the end of each firmware
>> > release, like ar9170-1.fw but ar9170 originally was implemented with a
>> > 2-stage firmware requirement and so ar9170-1.fw is already taken.
>> >
>> > ar9170 still needs a solution for the different firmwares, once we
>> > start supporting the open firmware through some sort of release but
>> > I'd like to address ath_hif_usb now early so that we don't run into
>> > these snags and use some decent convention that is easy to follow.
>> >
>> > As I noted above, Intel seems to use the device-1.fw, device-2.fw
>> > naming convention. Is this the best approach? Or shall we have the
>> > same firmware filename and simply query the firmware for a map of
>> > capabilities? Any other ideas?
>>
>> the general rule of thumb is that if you break the firmware API/ABI or
>> change it then the firmware name should be different. So for example if
>> you have some new driver functionality that requires new firmware then
>> you better use a new firmware name. Otherwise it is just fine to use the
>> same name if the functionality is not changing. If you can actually
>> detect the new firmware features from the firmware filename, then you
>> might not even have to bother with a different name. However keep in
>> mind that you still need to load at least the previous version of the
>> firmware and keep that working.
>>
>> For open source firmware vs binary blobs, we don't really have a good
>> reference. In theory the driver should always try loading both and if
>> one succeeds then go with it. At no point the driver should stop working
>> only because a firmware is missing while either an open source or binary
>> one for that matter would have been available.
>>
>> If you have a binary and an open source available, then you might wanna
>> have a Kconfig option which to try first. Like prefer the open source
>> over the binary one, but at the end of the day most system will ship
>> with only one anyway. And a module parameter would work here as well.
>
> This seems like a good piece of advise (as does Pavel's).  Perhaps
> someone should capture this on the wiki?

I just did that as I had to dig this old e-mail to help someone with
this information. I'll send out another e-mail for wider review on
lkml.

  Luis