putting future iwlwifi firmware into intel/ ?

putting future iwlwifi firmware into intel/ ?

[Johannes Berg]

Hi,

Since I was just looking at some firmware related thing (files for the
hardware that might be shared between Intel BT and WiFi), I noticed that
just over 30% of the files/dirs in the top-level firmware tree are
iwlwifi-* files.

While we can't move the files that older drivers might consume, we could
e.g. change the driver to look up future versions/future hardware under
intel/ instead? Would that be worth doing?

johannes

Re: putting future iwlwifi firmware into intel/ ?

[Mario Limonciello]

On 4/17/24 06:18, Johannes Berg wrote:
> Hi,
> 
> Since I was just looking at some firmware related thing (files for the
> hardware that might be shared between Intel BT and WiFi), I noticed that
> just over 30% of the files/dirs in the top-level firmware tree are
> iwlwifi-* files.
> 
> While we can't move the files that older drivers might consume, we could
> e.g. change the driver to look up future versions/future hardware under
> intel/ instead? Would that be worth doing?
> 
> johannes
> 

How about moving them all now and then creating compat symlinks for 
older kernels at linux-firmware 'make install' time?

Re: putting future iwlwifi firmware into intel/ ?

[Johannes Berg]

On Wed, 2024-04-17 at 07:31 -0500, Mario Limonciello wrote:
> 
> On 4/17/24 06:18, Johannes Berg wrote:
> > Hi,
> > 
> > Since I was just looking at some firmware related thing (files for the
> > hardware that might be shared between Intel BT and WiFi), I noticed that
> > just over 30% of the files/dirs in the top-level firmware tree are
> > iwlwifi-* files.
> > 
> > While we can't move the files that older drivers might consume, we could
> > e.g. change the driver to look up future versions/future hardware under
> > intel/ instead? Would that be worth doing?
> > 
> > johannes
> > 
> 
> How about moving them all now and then creating compat symlinks for 
> older kernels at linux-firmware 'make install' time?
> 

Didn't realize there even was a 'make install' time :-)

But then I guess it'd be simpler in the driver since we could just
unconditionally add the intel/ prefix to the request_firmware call.

johannes

Re: putting future iwlwifi firmware into intel/ ?

[Mario Limonciello]

On 4/17/24 07:39, Johannes Berg wrote:
> On Wed, 2024-04-17 at 07:31 -0500, Mario Limonciello wrote:
>>
>> On 4/17/24 06:18, Johannes Berg wrote:
>>> Hi,
>>>
>>> Since I was just looking at some firmware related thing (files for the
>>> hardware that might be shared between Intel BT and WiFi), I noticed that
>>> just over 30% of the files/dirs in the top-level firmware tree are
>>> iwlwifi-* files.
>>>
>>> While we can't move the files that older drivers might consume, we could
>>> e.g. change the driver to look up future versions/future hardware under
>>> intel/ instead? Would that be worth doing?
>>>
>>> johannes
>>>
>>
>> How about moving them all now and then creating compat symlinks for
>> older kernels at linux-firmware 'make install' time?
>>
> 
> Didn't realize there even was a 'make install' time :-)
> 
> But then I guess it'd be simpler in the driver since we could just
> unconditionally add the intel/ prefix to the request_firmware call.
> 
> johannes

Do both.  All the stuff already there add the compat symlinks in linux 
firmware so that current and older kernels work with older hardware.

Any "new firmware" only put in the new path, and add commits to the 
kernel to look for "all" firmware in the new path.

Should hopefully cover everything without too much pain then.

Re: putting future iwlwifi firmware into intel/ ?

[Johannes Berg]

On Wed, 2024-04-17 at 07:42 -0500, Mario Limonciello wrote:
> 
> On 4/17/24 07:39, Johannes Berg wrote:
> > On Wed, 2024-04-17 at 07:31 -0500, Mario Limonciello wrote:
> > > 
> > > On 4/17/24 06:18, Johannes Berg wrote:
> > > > Hi,
> > > > 
> > > > Since I was just looking at some firmware related thing (files for the
> > > > hardware that might be shared between Intel BT and WiFi), I noticed that
> > > > just over 30% of the files/dirs in the top-level firmware tree are
> > > > iwlwifi-* files.
> > > > 
> > > > While we can't move the files that older drivers might consume, we could
> > > > e.g. change the driver to look up future versions/future hardware under
> > > > intel/ instead? Would that be worth doing?
> > > > 
> > > > johannes
> > > > 
> > > 
> > > How about moving them all now and then creating compat symlinks for
> > > older kernels at linux-firmware 'make install' time?
> > > 
> > 
> > Didn't realize there even was a 'make install' time :-)
> > 
> > But then I guess it'd be simpler in the driver since we could just
> > unconditionally add the intel/ prefix to the request_firmware call.
> > 
> > johannes
> 
> Do both.  All the stuff already there add the compat symlinks in linux 
> firmware so that current and older kernels work with older hardware.
> 
> Any "new firmware" only put in the new path, and add commits to the 
> kernel to look for "all" firmware in the new path.
> 
> Should hopefully cover everything without too much pain then.

Yeah, I guess. Though not actually sure, do we need to support new
kernel + old firmware install?

johannes

Re: putting future iwlwifi firmware into intel/ ?

[Kalle Valo]

Johannes Berg <johannes@sipsolutions.net> writes:

> Since I was just looking at some firmware related thing (files for the
> hardware that might be shared between Intel BT and WiFi), I noticed that
> just over 30% of the files/dirs in the top-level firmware tree are
> iwlwifi-* files.
>
> While we can't move the files that older drivers might consume, we could
> e.g. change the driver to look up future versions/future hardware under
> intel/ instead? Would that be worth doing?

Yes, please. It would be nice to cleanup linux-firmware.git top level
directory. But what about this:

* convert iwlwifi to first use _all_ files under intel/ and then as a
  fallback top level directory

* move existing iwlwifi firmware files to intel/ but create symlinks from
  the old location to the new one (using 'Link:' in WHENCE)

* add new files only to intel/ directory

* after few (4?) years remove the symlinks from linux-firmware and the
  fallback from iwlwifi

This should ensure backwards compatibility, right?

-- 
https://patchwork.kernel.org/project/linux-wireless/list/

https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches

Re: putting future iwlwifi firmware into intel/ ?

[Mario Limonciello]

On 4/17/24 07:45, Johannes Berg wrote:
> On Wed, 2024-04-17 at 07:42 -0500, Mario Limonciello wrote:
>>
>> On 4/17/24 07:39, Johannes Berg wrote:
>>> On Wed, 2024-04-17 at 07:31 -0500, Mario Limonciello wrote:
>>>>
>>>> On 4/17/24 06:18, Johannes Berg wrote:
>>>>> Hi,
>>>>>
>>>>> Since I was just looking at some firmware related thing (files for the
>>>>> hardware that might be shared between Intel BT and WiFi), I noticed that
>>>>> just over 30% of the files/dirs in the top-level firmware tree are
>>>>> iwlwifi-* files.
>>>>>
>>>>> While we can't move the files that older drivers might consume, we could
>>>>> e.g. change the driver to look up future versions/future hardware under
>>>>> intel/ instead? Would that be worth doing?
>>>>>
>>>>> johannes
>>>>>
>>>>
>>>> How about moving them all now and then creating compat symlinks for
>>>> older kernels at linux-firmware 'make install' time?
>>>>
>>>
>>> Didn't realize there even was a 'make install' time :-)
>>>
>>> But then I guess it'd be simpler in the driver since we could just
>>> unconditionally add the intel/ prefix to the request_firmware call.
>>>
>>> johannes
>>
>> Do both.  All the stuff already there add the compat symlinks in linux
>> firmware so that current and older kernels work with older hardware.
>>
>> Any "new firmware" only put in the new path, and add commits to the
>> kernel to look for "all" firmware in the new path.
>>
>> Should hopefully cover everything without too much pain then.
> 
> Yeah, I guess. Though not actually sure, do we need to support new
> kernel + old firmware install?
> 
> johannes

I'd say you should.  Here's a few examples of why:

1) Kernel documentation says so.  See 
https://github.com/torvalds/linux/blob/master/Documentation/driver-api/firmware/firmware-usage-guidelines.rst


2) Let's say you change this in latest linux-firmware and kernel 6.10 
but don't do a symlink.  A user upgrades to 6.10 and latest 
linux-firmware but finds a regression.  They're asked to come bisect it, 
but their kernel 6.9 can no longer work with wifi due to "missing" 
firmware.  The bisect gets WAY more painful.

3) Some distros do "HWE" kernels (for example Ubuntu).  This means they 
could do a HWE-6.10 or HWE-6.11 but expect that the exact same 
linux-firmware package works on all of them.  They could uprev their 
linux-firmware to the one matching this change but then their "LTS" 
kernel stops working.

So yes; do the symlinks when you move directories.  It's one line in 
WHENCE per file that you want a symlink.

Re: putting future iwlwifi firmware into intel/ ?

[Johannes Berg]

On Wed, 2024-04-17 at 08:58 -0500, Mario Limonciello wrote:
> > 
> > Yeah, I guess. Though not actually sure, do we need to support new
> > kernel + old firmware install?
> 
> I'd say you should.  Here's a few examples of why:

Right, agree.

> So yes; do the symlinks when you move directories.  It's one line in 
> WHENCE per file that you want a symlink.
> 

That doesn't achieve it though. It means the driver needs to try for the
file in intel/ and then fall back without the intel/ if it's not found.

Or just look for new files in intel/ and old files without intel/, but
then you don't really need the file move/symlinks?

johannes

Re: putting future iwlwifi firmware into intel/ ?

[Johannes Berg]

On Wed, 2024-04-17 at 15:56 +0300, Kalle Valo wrote:
> 
> Yes, please. It would be nice to cleanup linux-firmware.git top level
> directory. But what about this:
> 
> * convert iwlwifi to first use _all_ files under intel/ and then as a
>   fallback top level directory

Yeah, that's quite annoying though since we currently look for something
like "iwlwifi-<hw>-<version>.ucode", and version will go down from some
number to another number (e.g. 73 ... 43.) So now we have to double that
loop up and try intel/...-73, ...-73, intel/...-72, ...-72, etc.?

johannes

Re: putting future iwlwifi firmware into intel/ ?

[Mario Limonciello]

On 4/17/24 09:09, Johannes Berg wrote:
> On Wed, 2024-04-17 at 08:58 -0500, Mario Limonciello wrote:
>>>
>>> Yeah, I guess. Though not actually sure, do we need to support new
>>> kernel + old firmware install?
>>
>> I'd say you should.  Here's a few examples of why:
> 
> Right, agree.
> 
>> So yes; do the symlinks when you move directories.  It's one line in
>> WHENCE per file that you want a symlink.
>>
> 
> That doesn't achieve it though. It means the driver needs to try for the
> file in intel/ and then fall back without the intel/ if it's not found.
> 
> Or just look for new files in intel/ and old files without intel/, but
> then you don't really need the file move/symlinks?
> 
> johannes

It's probably easiest to look in intel/ and then fallback to the 
toplevel path if -ENOENT and try again.

Like Kalle mentioned, wait a few years and then tear out the fallback 
code and the symlink creation.