Re: [Sound-open-firmware] Signed firmware availability for kbl/cnl

Re: [Sound-open-firmware] Signed firmware availability for kbl/cnl

[Jaroslav Kysela]

Dne 24. 07. 19 v 18:09 Liam Girdwood napsal(a):
> On Wed, 2019-07-24 at 18:00 +0200, Jaroslav Kysela wrote:
>> Dne 24. 07. 19 v 16:41 Pierre-Louis Bossart napsal(a):
>>> On 7/24/19 8:59 AM, Jaroslav Kysela wrote:
>>>> Dne 23. 07. 19 v 16:22 Liam Girdwood napsal(a):
>>>>> On Thu, 2019-07-18 at 16:43 +0800, Daniel Drake
>>>>> wrote:_______________________________________________
>>>>>> Sound-open-firmware mailing list
>>>>>> Sound-open-firmware@alsa-project.org
>>>>>>
>>>>> https://mailman.alsa-project.org/mailman/listinfo/sound-open-firmware
>>>>>
>>>>>> On Wed, Jul 17, 2019 at 8:09 AM Pierre-Louis Bossart
>>>>>> <pierre-louis.bossart@linux.intel.com>
>>>>>> wrote:_______________________________________________
>>>>>>> Sound-open-firmware mailing list
>>>>>>> Sound-open-firmware@alsa-project.org
>>>>>>>
>>>>>> https://mailman.alsa-project.org/mailman/listinfo/sound-open-firmware
>>>>>>
>>>>>>> I was indeed told a while ago that there was a limited
>>>>>>> number of
>>>>>>> KBL-based devices with DMIC, but mistakenly assumed we
>>>>>>> could avoid
>>>>>>> dealing with this configuration and Murphy's Law applied of
>>>>>>> course.
>>>>>>> we'll have to huddle with our Intel colleagues to figure
>>>>>>> this one
>>>>>>> out.
>>>>>>
>>>>>> OK, thanks.
>>>>>>
>>>>>> Is there any update on the release of signed firmware files
>>>>>> for the
>>>>>> other platforms? We are under pressure to return the other
>>>>>> unit we
>>>>>> have to the vendor (which needs the cnl files), but we would
>>>>>> like to
>>>>>> try SOF first.
>>>>>>
>>>>>
>>>>> Apologies for the delay, I hurt my back and was off work for a
>>>>> few
>>>>> weeks. Signed binaries now on v1.3 github release tag. Will now
>>>>> be
>>>>> upstreaming into Linux FW repo.
>>>>
>>>> Liam, the sizes of signed firmware binaries are a lot different
>>>> than the
>>>> unsigned ones (v1.3 tag) which I can build in docker:
>>>>
>>>> -rw-rw-r--. 1 perex perex 270336 Jul 24 15:44 sof-apl-signed-
>>>> intel.ri
>>>> -rw-r--r--. 1 perex perex 167936 Jul 24 15:44 sof-apl.ri
>>>> -rw-rw-r--. 1 perex perex 278528 Jul 24 15:44 sof-cnl-signed-
>>>> intel.ri
>>>> -rw-r--r--. 1 perex perex 172032 Jul 24 15:44 sof-cnl.ri
>>>> -rw-rw-r--. 1 perex perex 278528 Jul 24 15:44 sof-icl-signed-
>>>> intel.ri
>>>> -rw-r--r--. 1 perex perex 172032 Jul 24 15:44 sof-icl.ri
>>>>
>>>> Is that ok?
>>>
>>> The firmware used for production is typically built with the
>>> Cadence 
>>> tools, which unfortunately are not available publicly (but can be
>>> made 
>>> available to Intel partners). It wouldn't be surprising if the code
>>> size 
>>> was different due to the use of intrinsics (though 100K seems like
>>> a lot 
>>> indeed).
>>>
>>> Liam, I think we ought to release binaries with the community key
>>> as 
>>> well so that people can use them as is, e.g. on the Up2 board which
>>> does 
>>> not require the Intel production key. Same for GLK Chromebooks.
>>
>> It would be probably more nice to create a tar ball with all firmware
>> and
>> topology files bundled with the proper (usual) filesystem location
>> (/lib/firmware/intel/sof/... and /lib/firmware/intel/sof-tplg/...).
>> So we
>> (users/distribution packagers) can just use it.
> 
> Yeah, been thinking about this atm. It may be better to package the
> binaries (firmware and topologies) as part of Linux firmware repo
> (since the driver expects to load them all from lib/firmware) and
> package the sources (firmware and topology) via sof tarball ?

It looks good in my eyes, because topology files are another pieces of the
driver from the user space perspective. The unanswered question is the UCM
configuration which is linked to the topology configuration (if I understand
this correctly). I proposed to place an unique identifier/version to the
topology file and propagate this identifier to the user space, so the alsa-lib
can pick the right UCM configuration when topology changes. The component
string (snd_component_add function / struct snd_ctl_card_info -> components)
can be used for this identification.

					Jaroslav

> 
> Would this be OK for the distros ?
> 
> Liam 
> 
>>
>> BTW: Do we need UCM config files for SOF, too?
>>
>> 					Jaroslav
>>
> 


-- 
Jaroslav Kysela <perex@perex.cz>
Linux Sound Maintainer; ALSA Project; Red Hat, Inc.

Re: [Sound-open-firmware] Signed firmware availability for kbl/cnl

[Liam Girdwood]

+ Mengdong

On Wed, 2019-07-24 at 18:23 +0200, Jaroslav Kysela wrote:
> > Yeah, been thinking about this atm. It may be better to package the
> > binaries (firmware and topologies) as part of Linux firmware repo
> > (since the driver expects to load them all from lib/firmware) and
> > package the sources (firmware and topology) via sof tarball ?
> 
> It looks good in my eyes, because topology files are another pieces
> of the
> driver from the user space perspective. The unanswered question is
> the UCM
> configuration which is linked to the topology configuration (if I
> understand
> this correctly). I proposed to place an unique identifier/version to
> the
> topology file and propagate this identifier to the user space, so the
> alsa-lib
> can pick the right UCM configuration when topology changes. The
> component
> string (snd_component_add function / struct snd_ctl_card_info ->
> components)
> can be used for this identification.

Apologizes for the delay, Pierre and I have been discussing this
internally as we have to synchronise the deployment of the topologies
and UCMs alongside the FW.

Current thinking has changed from shipping FW + tplg via linux-firmware 
repo to only shipping FW binaries in the FW repo and using alsa-ucm-
conf.git for UCMs + topologies (since the coupling between UCM and
topology is tighter than the FW coupling).

Any objections to using this repo for topologies too ? I know we
haven't yet used it for UCMs but now is probably a good point to move
(including moving the older UCMs over too).
The "make" rule would compile topologies, whilst the "make install"
rule would install the UCM's and binary topologies in the correct
places ?

Thanks

Liam

Re: [Sound-open-firmware] Signed firmware availability for kbl/cnl

[Pierre-Louis Bossart]

On 7/31/19 8:23 AM, Liam Girdwood wrote:
> + Mengdong
> 
> On Wed, 2019-07-24 at 18:23 +0200, Jaroslav Kysela wrote:
>>> Yeah, been thinking about this atm. It may be better to package the
>>> binaries (firmware and topologies) as part of Linux firmware repo
>>> (since the driver expects to load them all from lib/firmware) and
>>> package the sources (firmware and topology) via sof tarball ?
>>
>> It looks good in my eyes, because topology files are another pieces
>> of the
>> driver from the user space perspective. The unanswered question is
>> the UCM
>> configuration which is linked to the topology configuration (if I
>> understand
>> this correctly). I proposed to place an unique identifier/version to
>> the
>> topology file and propagate this identifier to the user space, so the
>> alsa-lib
>> can pick the right UCM configuration when topology changes. The
>> component
>> string (snd_component_add function / struct snd_ctl_card_info ->
>> components)
>> can be used for this identification.
> 
> Apologizes for the delay, Pierre and I have been discussing this
> internally as we have to synchronise the deployment of the topologies
> and UCMs alongside the FW.
> 
> Current thinking has changed from shipping FW + tplg via linux-firmware
> repo to only shipping FW binaries in the FW repo and using alsa-ucm-
> conf.git for UCMs + topologies (since the coupling between UCM and
> topology is tighter than the FW coupling).

more precisely, the topology file exposes stream numbers and control 
names, and if the UCM file is not aligned with topology changes then 
users will get errors thrown at them. It really makes sense to release 
topology and UCM files concurrently. We'll also need to work with 
distributions to make sure the files are updated in the right places. 
Currently topology files are in /lib/firmware/intel/sof-tplg and UCM in 
/usr/share/alsa/ucm.

> 
> Any objections to using this repo for topologies too ? I know we
> haven't yet used it for UCMs but now is probably a good point to move
> (including moving the older UCMs over too).

We'll need to figure out the license for all this, the topologies  and 
UCM files created for SOF are all BSD 3-clause but I am not clear on 
older UCM files stored in alsa-lib, I don't think there was any 
agreement that the LGPL license applied to them.

> The "make" rule would compile topologies, whilst the "make install"
> rule would install the UCM's and binary topologies in the correct
> places ?

Re: [Sound-open-firmware] Signed firmware availability for kbl/cnl

[Liam Girdwood]

On Wed, 2019-07-31 at 09:01 -0500, Pierre-Louis Bossart wrote:
> > Any objections to using this repo for topologies too ? I know we
> > haven't yet used it for UCMs but now is probably a good point to
> > move
> > (including moving the older UCMs over too).
> 
> We'll need to figure out the license for all this, the topologies 
> and 
> UCM files created for SOF are all BSD 3-clause but I am not clear on 
> older UCM files stored in alsa-lib, I don't think there was any 
> agreement that the LGPL license applied to them.

I remember this was discussed in the past, it is possible to make all
new UCMs and topologies BSD 3c in the new repo. We could also transfer
over and re-license any topoloies over from alsa-lib where we had
agreement from authors. 

This would eventually mean alsa-lib would contain "legacy" GPL
topologies whilst alsa-ucm-conf would contain any new and BSD licences
UCM/topologies.

Liam

Re: [Sound-open-firmware] Signed firmware availability for kbl/cnl

[Jaroslav Kysela]

Dne 31. 07. 19 v 15:23 Liam Girdwood napsal(a):
> + Mengdong
> 
> On Wed, 2019-07-24 at 18:23 +0200, Jaroslav Kysela wrote:
>>> Yeah, been thinking about this atm. It may be better to package the
>>> binaries (firmware and topologies) as part of Linux firmware repo
>>> (since the driver expects to load them all from lib/firmware) and
>>> package the sources (firmware and topology) via sof tarball ?
>>
>> It looks good in my eyes, because topology files are another pieces
>> of the
>> driver from the user space perspective. The unanswered question is
>> the UCM
>> configuration which is linked to the topology configuration (if I
>> understand
>> this correctly). I proposed to place an unique identifier/version to
>> the
>> topology file and propagate this identifier to the user space, so the
>> alsa-lib
>> can pick the right UCM configuration when topology changes. The
>> component
>> string (snd_component_add function / struct snd_ctl_card_info ->
>> components)
>> can be used for this identification.
> 
> Apologizes for the delay, Pierre and I have been discussing this
> internally as we have to synchronise the deployment of the topologies
> and UCMs alongside the FW.

My strong point is that the driver with the different firmware and the
topology file behaves differently from the user space perspective. It seems
that there is no way to propagate the firmware (and topology?) version to the
user space at the moment.

> Current thinking has changed from shipping FW + tplg via linux-firmware 
> repo to only shipping FW binaries in the FW repo and using alsa-ucm-
> conf.git for UCMs + topologies (since the coupling between UCM and
> topology is tighter than the FW coupling).

This is fine, but I think that we should have a check (compatibility
verification) in the user space level, too. More precisely, each level should
do a verification if it's compatible with the tied level (driver -> firmware
-> topology -> ucm).

					Jaroslav

-- 
Jaroslav Kysela <perex@perex.cz>
Linux Sound Maintainer; ALSA Project; Red Hat, Inc.

Re: [Sound-open-firmware] Signed firmware availability for kbl/cnl

[Jaroslav Kysela]

Dne 31. 07. 19 v 16:07 Liam Girdwood napsal(a):
> On Wed, 2019-07-31 at 09:01 -0500, Pierre-Louis Bossart wrote:
>>> Any objections to using this repo for topologies too ? I know we
>>> haven't yet used it for UCMs but now is probably a good point to
>>> move
>>> (including moving the older UCMs over too).
>>
>> We'll need to figure out the license for all this, the topologies 
>> and 
>> UCM files created for SOF are all BSD 3-clause but I am not clear on 
>> older UCM files stored in alsa-lib, I don't think there was any 
>> agreement that the LGPL license applied to them.
> 
> I remember this was discussed in the past, it is possible to make all
> new UCMs and topologies BSD 3c in the new repo. We could also transfer
> over and re-license any topoloies over from alsa-lib where we had
> agreement from authors. 
> 
> This would eventually mean alsa-lib would contain "legacy" GPL
> topologies whilst alsa-ucm-conf would contain any new and BSD licences
> UCM/topologies.

I created the alsa-ucm-conf repository on github and invited you as the
maintainer:

https://github.com/alsa-project/alsa-ucm-conf

					Jaroslav

-- 
Jaroslav Kysela <perex@perex.cz>
Linux Sound Maintainer; ALSA Project; Red Hat, Inc.

Re: [Sound-open-firmware] Signed firmware availability for kbl/cnl

[Pierre-Louis Bossart]

On 7/31/19 12:29 PM, Jaroslav Kysela wrote:
> Dne 31. 07. 19 v 15:23 Liam Girdwood napsal(a):
>> + Mengdong
>>
>> On Wed, 2019-07-24 at 18:23 +0200, Jaroslav Kysela wrote:
>>>> Yeah, been thinking about this atm. It may be better to package the
>>>> binaries (firmware and topologies) as part of Linux firmware repo
>>>> (since the driver expects to load them all from lib/firmware) and
>>>> package the sources (firmware and topology) via sof tarball ?
>>>
>>> It looks good in my eyes, because topology files are another pieces
>>> of the
>>> driver from the user space perspective. The unanswered question is
>>> the UCM
>>> configuration which is linked to the topology configuration (if I
>>> understand
>>> this correctly). I proposed to place an unique identifier/version to
>>> the
>>> topology file and propagate this identifier to the user space, so the
>>> alsa-lib
>>> can pick the right UCM configuration when topology changes. The
>>> component
>>> string (snd_component_add function / struct snd_ctl_card_info ->
>>> components)
>>> can be used for this identification.
>>
>> Apologizes for the delay, Pierre and I have been discussing this
>> internally as we have to synchronise the deployment of the topologies
>> and UCMs alongside the FW.
> 
> My strong point is that the driver with the different firmware and the
> topology file behaves differently from the user space perspective. It seems
> that there is no way to propagate the firmware (and topology?) version to the
> user space at the moment.

The topology may not be enough, e.g. for all Baytrail devices we use the 
same simple topology. To pick the right UCM file you really need the 
card information which may include the DMI info or some quirks 
(mono-speaker, analog mics). The topology is quite static and doesn't 
expose anything that is board-specific or may vary between skews.

> 
>> Current thinking has changed from shipping FW + tplg via linux-firmware
>> repo to only shipping FW binaries in the FW repo and using alsa-ucm-
>> conf.git for UCMs + topologies (since the coupling between UCM and
>> topology is tighter than the FW coupling).
> 
> This is fine, but I think that we should have a check (compatibility
> verification) in the user space level, too. More precisely, each level should
> do a verification if it's compatible with the tied level (driver -> firmware
> -> topology -> ucm).

The SOF driver checks if its supported ABI level is compatible with 
firmware and topology levels (both files embed the information, which 
doesn't have to be identical).

I don't see how UCM would be checked since there's no direct interaction 
with the driver, e.g. it's used by PulseAudio or CRAS and the only 
interaction is through the control and PCM APIs. Likewise UCM has no 
knowledge about topology or firmware.

Re: [Sound-open-firmware] Signed firmware availability for kbl/cnl

[Jaroslav Kysela]

Dne 31. 07. 19 v 20:14 Pierre-Louis Bossart napsal(a):
> On 7/31/19 12:29 PM, Jaroslav Kysela wrote:
>> Dne 31. 07. 19 v 15:23 Liam Girdwood napsal(a):
>>> + Mengdong
>>>
>>> On Wed, 2019-07-24 at 18:23 +0200, Jaroslav Kysela wrote:
>>>>> Yeah, been thinking about this atm. It may be better to package the
>>>>> binaries (firmware and topologies) as part of Linux firmware repo
>>>>> (since the driver expects to load them all from lib/firmware) and
>>>>> package the sources (firmware and topology) via sof tarball ?
>>>>
>>>> It looks good in my eyes, because topology files are another pieces
>>>> of the
>>>> driver from the user space perspective. The unanswered question is
>>>> the UCM
>>>> configuration which is linked to the topology configuration (if I
>>>> understand
>>>> this correctly). I proposed to place an unique identifier/version to
>>>> the
>>>> topology file and propagate this identifier to the user space, so the
>>>> alsa-lib
>>>> can pick the right UCM configuration when topology changes. The
>>>> component
>>>> string (snd_component_add function / struct snd_ctl_card_info ->
>>>> components)
>>>> can be used for this identification.
>>>
>>> Apologizes for the delay, Pierre and I have been discussing this
>>> internally as we have to synchronise the deployment of the topologies
>>> and UCMs alongside the FW.
>>
>> My strong point is that the driver with the different firmware and the
>> topology file behaves differently from the user space perspective. It seems
>> that there is no way to propagate the firmware (and topology?) version to the
>> user space at the moment.
> 
> The topology may not be enough, e.g. for all Baytrail devices we use the 
> same simple topology. To pick the right UCM file you really need the 
> card information which may include the DMI info or some quirks 
> (mono-speaker, analog mics). The topology is quite static and doesn't 
> expose anything that is board-specific or may vary between skews.

Yes, thus we need to use another UCM file (or make some parts conditional in
the UCM config) depending on this and I would like to pass the exact
hardware/firmware/topology identification which may affect the UCM, through
the ALSA API to the user space level (UCM parser). Think from the packaging
(Linux distributions) perspective. We have to handle all those situations, so
all the configs, pieces to support all hardware variations must be prepared in
the packages.

Also, the blind fw / topology / UCM relationship without any compatibility
checks might cause issues when the user upgrades only some parts. The binary
topology files might be packaged with the UCM files as proposed, but if an
incompatible DSP firmware will be loaded (it's placed in the another package -
linux-firmware), it should be reported to the user, too.

>>> Current thinking has changed from shipping FW + tplg via linux-firmware
>>> repo to only shipping FW binaries in the FW repo and using alsa-ucm-
>>> conf.git for UCMs + topologies (since the coupling between UCM and
>>> topology is tighter than the FW coupling).
>>
>> This is fine, but I think that we should have a check (compatibility
>> verification) in the user space level, too. More precisely, each level should
>> do a verification if it's compatible with the tied level (driver -> firmware
>> -> topology -> ucm).
> 
> The SOF driver checks if its supported ABI level is compatible with 
> firmware and topology levels (both files embed the information, which 
> doesn't have to be identical).

Ok, but if you add another functionality to the firmware or remove some, it
might break the compatibility with the topology (different ALSA controls for
example), right? I'm not sure if ABI checks are sufficient. It's more about
the overall sound hardware abstraction for the user space (managed ALSA controls).

> I don't see how UCM would be checked since there's no direct interaction 
> with the driver, e.g. it's used by PulseAudio or CRAS and the only 
> interaction is through the control and PCM APIs. Likewise UCM has no> knowledge about topology or firmware.

The UCM parser code in alsa-lib (not applications) can do the check /
configuration selection. This is exactly what I am proposing to do. Actually,
for example, the UCM parser looks for sof-skl_hda_card.conf file without any
other checks or conditions. You will agree that we cannot support all hardware
variants with this, because some vendors might use other GPIOs etc..

So my proposal is to pass all necessary information throught the ALSA control
API (struct snd_ctl_card_info -> components) so the UCM parser can pick the
right config file based on the complete identification. It might fallback to
sof-skl_hda_card.conf, but if new hardware variant exist, the file name might
look like 'sof-skl_hda_card-TOPOLOGYID-VENDORID-PRODUCTID.conf' etc, etc....

							Jaroslav

-- 
Jaroslav Kysela <perex@perex.cz>
Linux Sound Maintainer; ALSA Project; Red Hat, Inc.

Re: [Sound-open-firmware] Signed firmware availability for kbl/cnl

[Liam Girdwood]

On Fri, 2019-08-02 at 10:21 +0200, Jaroslav Kysela wrote:
> Dne 31. 07. 19 v 20:14 Pierre-Louis Bossart napsal(a):
> > On 7/31/19 12:29 PM, Jaroslav Kysela wrote:
> > > Dne 31. 07. 19 v 15:23 Liam Girdwood napsal(a):
> > > > + Mengdong
> > > > 
> > > > On Wed, 2019-07-24 at 18:23 +0200, Jaroslav Kysela wrote:
> > > > > > Yeah, been thinking about this atm. It may be better to
> > > > > > package the
> > > > > > binaries (firmware and topologies) as part of Linux
> > > > > > firmware repo
> > > > > > (since the driver expects to load them all from
> > > > > > lib/firmware) and
> > > > > > package the sources (firmware and topology) via sof tarball
> > > > > > ?
> > > > > 
> > > > > It looks good in my eyes, because topology files are another
> > > > > pieces
> > > > > of the
> > > > > driver from the user space perspective. The unanswered
> > > > > question is
> > > > > the UCM
> > > > > configuration which is linked to the topology configuration
> > > > > (if I
> > > > > understand
> > > > > this correctly). I proposed to place an unique
> > > > > identifier/version to
> > > > > the
> > > > > topology file and propagate this identifier to the user
> > > > > space, so the
> > > > > alsa-lib
> > > > > can pick the right UCM configuration when topology changes.
> > > > > The
> > > > > component
> > > > > string (snd_component_add function / struct snd_ctl_card_info
> > > > > ->
> > > > > components)
> > > > > can be used for this identification.
> > > > 
> > > > Apologizes for the delay, Pierre and I have been discussing
> > > > this
> > > > internally as we have to synchronise the deployment of the
> > > > topologies
> > > > and UCMs alongside the FW.
> > > 
> > > My strong point is that the driver with the different firmware
> > > and the
> > > topology file behaves differently from the user space
> > > perspective. It seems
> > > that there is no way to propagate the firmware (and topology?)
> > > version to the
> > > user space at the moment.
> > 
> > The topology may not be enough, e.g. for all Baytrail devices we
> > use the 
> > same simple topology. To pick the right UCM file you really need
> > the 
> > card information which may include the DMI info or some quirks 
> > (mono-speaker, analog mics). The topology is quite static and
> > doesn't 
> > expose anything that is board-specific or may vary between skews.
> 
> Yes, thus we need to use another UCM file (or make some parts
> conditional in
> the UCM config) depending on this and I would like to pass the exact
> hardware/firmware/topology identification which may affect the UCM,
> through
> the ALSA API to the user space level (UCM parser). Think from the
> packaging
> (Linux distributions) perspective. We have to handle all those
> situations, so
> all the configs, pieces to support all hardware variations must be
> prepared in
> the packages.

I think the UCM parser will currently only bail on cdev naming
differences, so I agree maybe something at the top level UCM "machine
global" level that can be used to check FW, topology (+anything else)
so we could bail earlier or warn and attempt to continue.

> 
> Also, the blind fw / topology / UCM relationship without any
> compatibility
> checks might cause issues when the user upgrades only some parts. The
> binary
> topology files might be packaged with the UCM files as proposed, but
> if an
> incompatible DSP firmware will be loaded (it's placed in the another
> package -
> linux-firmware), it should be reported to the user, too.
> 
> > > > Current thinking has changed from shipping FW + tplg via linux-
> > > > firmware
> > > > repo to only shipping FW binaries in the FW repo and using
> > > > alsa-ucm-
> > > > conf.git for UCMs + topologies (since the coupling between UCM
> > > > and
> > > > topology is tighter than the FW coupling).
> > > 
> > > This is fine, but I think that we should have a check
> > > (compatibility
> > > verification) in the user space level, too. More precisely, each
> > > level should
> > > do a verification if it's compatible with the tied level (driver
> > > -> firmware
> > > -> topology -> ucm).
> > 
> > The SOF driver checks if its supported ABI level is compatible
> > with 
> > firmware and topology levels (both files embed the information,
> > which 
> > doesn't have to be identical).
> 
> Ok, but if you add another functionality to the firmware or remove
> some, it
> might break the compatibility with the topology (different ALSA
> controls for
> example), right? I'm not sure if ABI checks are sufficient. It's more
> about
> the overall sound hardware abstraction for the user space (managed
> ALSA controls).
> 
> > I don't see how UCM would be checked since there's no direct
> > interaction 
> > with the driver, e.g. it's used by PulseAudio or CRAS and the only 
> > interaction is through the control and PCM APIs. Likewise UCM has
> > no> knowledge about topology or firmware.
> 
> The UCM parser code in alsa-lib (not applications) can do the check /
> configuration selection. This is exactly what I am proposing to do.
> Actually,
> for example, the UCM parser looks for sof-skl_hda_card.conf file
> without any
> other checks or conditions. You will agree that we cannot support all
> hardware
> variants with this, because some vendors might use other GPIOs etc..
> 
> So my proposal is to pass all necessary information throught the ALSA
> control
> API (struct snd_ctl_card_info -> components) so the UCM parser can
> pick the
> right config file based on the complete identification. It might
> fallback to
> sof-skl_hda_card.conf, but if new hardware variant exist, the file
> name might
> look like 'sof-skl_hda_card-TOPOLOGYID-VENDORID-PRODUCTID.conf' etc,
> etc....
> 

How would we get topology or FW version from the above identification ?

Would we also use semantic versioning to align the UCM with the
topology and FW ? Currently we use semantic versioning for topology and
FW.

Thanks

Liam

Re: [Sound-open-firmware] Signed firmware availability for kbl/cnl

[Jaroslav Kysela]

Dne 02. 08. 19 v 16:40 Liam Girdwood napsal(a):
> On Fri, 2019-08-02 at 10:21 +0200, Jaroslav Kysela wrote:
>> Dne 31. 07. 19 v 20:14 Pierre-Louis Bossart napsal(a):
>>> On 7/31/19 12:29 PM, Jaroslav Kysela wrote:
>>>> Dne 31. 07. 19 v 15:23 Liam Girdwood napsal(a):
>>>>> + Mengdong
>>>>>
>>>>> On Wed, 2019-07-24 at 18:23 +0200, Jaroslav Kysela wrote:
>>>>>>> Yeah, been thinking about this atm. It may be better to
>>>>>>> package the
>>>>>>> binaries (firmware and topologies) as part of Linux
>>>>>>> firmware repo
>>>>>>> (since the driver expects to load them all from
>>>>>>> lib/firmware) and
>>>>>>> package the sources (firmware and topology) via sof tarball
>>>>>>> ?
>>>>>>
>>>>>> It looks good in my eyes, because topology files are another
>>>>>> pieces
>>>>>> of the
>>>>>> driver from the user space perspective. The unanswered
>>>>>> question is
>>>>>> the UCM
>>>>>> configuration which is linked to the topology configuration
>>>>>> (if I
>>>>>> understand
>>>>>> this correctly). I proposed to place an unique
>>>>>> identifier/version to
>>>>>> the
>>>>>> topology file and propagate this identifier to the user
>>>>>> space, so the
>>>>>> alsa-lib
>>>>>> can pick the right UCM configuration when topology changes.
>>>>>> The
>>>>>> component
>>>>>> string (snd_component_add function / struct snd_ctl_card_info
>>>>>> ->
>>>>>> components)
>>>>>> can be used for this identification.
>>>>>
>>>>> Apologizes for the delay, Pierre and I have been discussing
>>>>> this
>>>>> internally as we have to synchronise the deployment of the
>>>>> topologies
>>>>> and UCMs alongside the FW.
>>>>
>>>> My strong point is that the driver with the different firmware
>>>> and the
>>>> topology file behaves differently from the user space
>>>> perspective. It seems
>>>> that there is no way to propagate the firmware (and topology?)
>>>> version to the
>>>> user space at the moment.
>>>
>>> The topology may not be enough, e.g. for all Baytrail devices we
>>> use the 
>>> same simple topology. To pick the right UCM file you really need
>>> the 
>>> card information which may include the DMI info or some quirks 
>>> (mono-speaker, analog mics). The topology is quite static and
>>> doesn't 
>>> expose anything that is board-specific or may vary between skews.
>>
>> Yes, thus we need to use another UCM file (or make some parts
>> conditional in
>> the UCM config) depending on this and I would like to pass the exact
>> hardware/firmware/topology identification which may affect the UCM,
>> through
>> the ALSA API to the user space level (UCM parser). Think from the
>> packaging
>> (Linux distributions) perspective. We have to handle all those
>> situations, so
>> all the configs, pieces to support all hardware variations must be
>> prepared in
>> the packages.
> 
> I think the UCM parser will currently only bail on cdev naming
> differences, so I agree maybe something at the top level UCM "machine
> global" level that can be used to check FW, topology (+anything else)
> so we could bail earlier or warn and attempt to continue.
> 
>>
>> Also, the blind fw / topology / UCM relationship without any
>> compatibility
>> checks might cause issues when the user upgrades only some parts. The
>> binary
>> topology files might be packaged with the UCM files as proposed, but
>> if an
>> incompatible DSP firmware will be loaded (it's placed in the another
>> package -
>> linux-firmware), it should be reported to the user, too.
>>
>>>>> Current thinking has changed from shipping FW + tplg via linux-
>>>>> firmware
>>>>> repo to only shipping FW binaries in the FW repo and using
>>>>> alsa-ucm-
>>>>> conf.git for UCMs + topologies (since the coupling between UCM
>>>>> and
>>>>> topology is tighter than the FW coupling).
>>>>
>>>> This is fine, but I think that we should have a check
>>>> (compatibility
>>>> verification) in the user space level, too. More precisely, each
>>>> level should
>>>> do a verification if it's compatible with the tied level (driver
>>>> -> firmware
>>>> -> topology -> ucm).
>>>
>>> The SOF driver checks if its supported ABI level is compatible
>>> with 
>>> firmware and topology levels (both files embed the information,
>>> which 
>>> doesn't have to be identical).
>>
>> Ok, but if you add another functionality to the firmware or remove
>> some, it
>> might break the compatibility with the topology (different ALSA
>> controls for
>> example), right? I'm not sure if ABI checks are sufficient. It's more
>> about
>> the overall sound hardware abstraction for the user space (managed
>> ALSA controls).
>>
>>> I don't see how UCM would be checked since there's no direct
>>> interaction 
>>> with the driver, e.g. it's used by PulseAudio or CRAS and the only 
>>> interaction is through the control and PCM APIs. Likewise UCM has
>>> no> knowledge about topology or firmware.
>>
>> The UCM parser code in alsa-lib (not applications) can do the check /
>> configuration selection. This is exactly what I am proposing to do.
>> Actually,
>> for example, the UCM parser looks for sof-skl_hda_card.conf file
>> without any
>> other checks or conditions. You will agree that we cannot support all
>> hardware
>> variants with this, because some vendors might use other GPIOs etc..
>>
>> So my proposal is to pass all necessary information throught the ALSA
>> control
>> API (struct snd_ctl_card_info -> components) so the UCM parser can
>> pick the
>> right config file based on the complete identification. It might
>> fallback to
>> sof-skl_hda_card.conf, but if new hardware variant exist, the file
>> name might
>> look like 'sof-skl_hda_card-TOPOLOGYID-VENDORID-PRODUCTID.conf' etc,
>> etc....
>>
> 
> How would we get topology or FW version from the above identification ?

It was just an example. We can compose the UCM filename from any other
additional information passed from the kernel. Example component strings for
USB and legacy HDA:


  Mixer name	: 'USB Mixer'
  Components	: 'USB0bda:58fe'

  Mixer name	: 'Realtek ALC298'
  Components	: 'HDA:10ec0298,17aa222e,00100103'

So we should consider what to export for SOF. Perhaps string like:

  'SOFP01234567:45670123,1:1:0-6cc8d,???TPLGVER???,3:7:0'
  'SOFP{PCIID}:{PCISUBSYS},FW-VER,TPLG-VER,TPLG-ABI-VER'

It's just a proposal for the discussion.

By the way:

  https://mailman.alsa-project.org/pipermail/alsa-devel/2019-May/149409.html

The component string extensions should be also considered for other Intel SOC
drivers. It seems that the long_name is misused as the UCM configuration
selector for other drivers like bytcr_rt5651.c etc. The long_name for the
soundcard like 'bytcht-es8316-mono-spk-in2-mic' is not really fancy. This
string is used in GUI.

> Would we also use semantic versioning to align the UCM with the
> topology and FW ? Currently we use semantic versioning for topology and
> FW.

If we have the versions exported to ther user space, the UCM configuration
loader / parser can use this information to select or verify the right UCM
configuration. The semantic versioning in UCM files sounds good to me, too.

						Jaroslav

> 
> Thanks
> 
> Liam
> 


-- 
Jaroslav Kysela <perex@perex.cz>
Linux Sound Maintainer; ALSA Project; Red Hat, Inc.

Re: [Sound-open-firmware] Signed firmware availability for kbl/cnl

[Pierre-Louis Bossart]

>> Would we also use semantic versioning to align the UCM with the
>> topology and FW ? Currently we use semantic versioning for topology and
>> FW.
> 
> If we have the versions exported to ther user space, the UCM configuration
> loader / parser can use this information to select or verify the right UCM
> configuration. The semantic versioning in UCM files sounds good to me, too.

My understanding semantic versioning is that it provides means to handle 
minor differences where a new capability is ignored in backwards 
compatible ways. This is what we use for SOF structures between driver 
and firmware, new fields might be added but used or not depending on 
versions.

For UCM, the interaction with other layers is limited to stream numbers 
and control names, so I am not sure what semantic versioning and 
backwards compatibility would mean here? I am all for it, but I don't 
get how it would work.

Re: [Sound-open-firmware] Signed firmware availability for kbl/cnl

[Liam Girdwood]

On Fri, 2019-08-02 at 21:01 +0200, Jaroslav Kysela wrote:
> > How would we get topology or FW version from the above
> > identification ?
> 
> It was just an example. We can compose the UCM filename from any
> other
> additional information passed from the kernel. Example component
> strings for
> USB and legacy HDA:
> 
> 
>   Mixer name    : 'USB Mixer'
>   Components    : 'USB0bda:58fe'
> 
>   Mixer name    : 'Realtek ALC298'
>   Components    : 'HDA:10ec0298,17aa222e,00100103'
> 
> So we should consider what to export for SOF. Perhaps string like:
> 
>   'SOFP01234567:45670123,1:1:0-6cc8d,???TPLGVER???,3:7:0'
>   'SOFP{PCIID}:{PCISUBSYS},FW-VER,TPLG-VER,TPLG-ABI-VER'
> 
> It's just a proposal for the discussion.

Ok, we will probably need TPLG-NAME in here so that we load the correct
UCM based on TPLG NAME + HW.
> 
> By the way:
> 
>   
> https://mailman.alsa-project.org/pipermail/alsa-devel/2019-May/149409.html
> 
> The component string extensions should be also considered for other
> Intel SOC
> drivers. It seems that the long_name is misused as the UCM
> configuration
> selector for other drivers like bytcr_rt5651.c etc. The long_name for
> the
> soundcard like 'bytcht-es8316-mono-spk-in2-mic' is not really fancy.
> This
> string is used in GUI.

Sorry, do you mean it would be better to include some more encoding
into the name to make them more unique and so that UI tools and users
can better understand the UCM file features without reading the UCM
file source ?

Thanks

Liam

Re: [Sound-open-firmware] Signed firmware availability for kbl/cnl

[Liam Girdwood]

On Fri, 2019-08-02 at 14:24 -0500, Pierre-Louis Bossart wrote:
> > > Would we also use semantic versioning to align the UCM with the
> > > topology and FW ? Currently we use semantic versioning for
> > > topology and
> > > FW.
> > 
> > If we have the versions exported to ther user space, the UCM
> > configuration
> > loader / parser can use this information to select or verify the
> > right UCM
> > configuration. The semantic versioning in UCM files sounds good to
> > me, too.
> 
> My understanding semantic versioning is that it provides means to
> handle 
> minor differences where a new capability is ignored in backwards 
> compatible ways. This is what we use for SOF structures between
> driver 
> and firmware, new fields might be added but used or not depending on 
> versions.
> 
> For UCM, the interaction with other layers is limited to stream
> numbers 
> and control names, so I am not sure what semantic versioning and 
> backwards compatibility would mean here? I am all for it, but I
> don't 
> get how it would work.
> 

My thinking here was to try and track the FW component kcontrol
surfaces for UCM. But I thought some more about it and it's better to
track using the single TPLG + UCM repo method to ensure alignment
between UCM and TPLGs.

Liam