HDA HDMI pin to converter mapping

HDA HDMI pin to converter mapping

[Stephen Warren]

Most cards supported by patch_hdmi.c have the property of a 1:1 mapping
or connection between converter widgets and pin widgets. Hence,
patch_hdmi.c sets up a PCM for each converter, and occasionally performs
some operations on the pin widget as required.

However, some new cards don't have this 1:1 mapping. In particular, the
new GeForce GT 520 and many future NVIDIA cards, and at least the Intel
Ibex Peak (in my wife's laptop), have more pin widgets than converters,
and each pin widget has a mux to select which converter to take the
audio from.

For pretty pictures, see the 4th (and 3rd) diagrams at:

ftp://download.nvidia.com/XFree86/gpu-hdmi-audio-document/gpu-hdmi-audio.html#_examples

I'd like to discuss how best to handle such cards.

I'd expect to see a PCM device created for each pin widget. This would
create a 1:1 mapping between PCM devices and attached monitors and hence
ELD data (with the possible exception of DisplayPort 1.2 daisy-chaining;
I have no idea yet how that plays into this).

Does this seem like a reasonable approach to everyone?

The current situation is that we only create a PCM for each converter
that is mux'd to a pin by default HW initialization. On both the GeForce
520 and Ibex Peak, this means that only 1 PCM gets created, since all
pin widgets' muxes point at the first converter by default. This:

a) Doesn't allow usage of both converters at once, since there's only 1
PCM object.

b) Ends up with up to 4 (in NVIDIA's case) pin widgets (and hence ELD
information) logically associated with the PCM object. Hence, it'll be
confusing for application to know what features they can really use on
the PCM. In fact, ALSA actually only enforces the ELD of the first pin
associated with the converter, and ignores the rest.

The basic mechanics of a-PCM-per-pin would require hdmi_pcm_open to look
at the list of converters, find one not in use by a stream, and then
dynamically associate the converter with the stream, or fail the open if
a converter was not available. We'd also need a custom close or cleanup
function to "unassign" the converter from the PCM, and set
hda_pcm_stream.nid to an invalid value.

That might be easiest to implement if hda_pcm_stream.nid were to refer
to the pin widget rather than the converter widget. However, I notice
that some core code in hda_codec.c would be confused by this, so
perhaps we'd better just have have hdmi_pcm_open use some other way to
determine which stream object is associated with the pin. I suppose
that it'd be pretty easy to set up a mapping from hda_pcm_stream to pin
nid as part of generic_hdmi_build_pcms.

As an aside, this rather reminds me of the mux'ing that goes on inside
codecs controlled by ASoC, although I don't think we need to expose this
muxing to user-space in the case, but instead make it automatic as I
outlined above.

Thanks for any thoughts.

-- 
nvpublic

Re: HDA HDMI pin to converter mapping

[Takashi Iwai]

At Wed, 25 May 2011 11:29:35 -0700,
Stephen Warren wrote:
> 
> Most cards supported by patch_hdmi.c have the property of a 1:1 mapping
> or connection between converter widgets and pin widgets. Hence,
> patch_hdmi.c sets up a PCM for each converter, and occasionally performs
> some operations on the pin widget as required.
> 
> However, some new cards don't have this 1:1 mapping. In particular, the
> new GeForce GT 520 and many future NVIDIA cards, and at least the Intel
> Ibex Peak (in my wife's laptop), have more pin widgets than converters,
> and each pin widget has a mux to select which converter to take the
> audio from.
> 
> For pretty pictures, see the 4th (and 3rd) diagrams at:
> 
> ftp://download.nvidia.com/XFree86/gpu-hdmi-audio-document/gpu-hdmi-audio.html#_examples

Wow, nice to see such a good document.

> I'd like to discuss how best to handle such cards.
> 
> I'd expect to see a PCM device created for each pin widget. This would
> create a 1:1 mapping between PCM devices and attached monitors and hence
> ELD data (with the possible exception of DisplayPort 1.2 daisy-chaining;
> I have no idea yet how that plays into this).
> 
> Does this seem like a reasonable approach to everyone?

Sounds reasonable and logical to me.
It won't break the older chips, and it will work with new chips.

> The current situation is that we only create a PCM for each converter
> that is mux'd to a pin by default HW initialization. On both the GeForce
> 520 and Ibex Peak, this means that only 1 PCM gets created, since all
> pin widgets' muxes point at the first converter by default. This:
> 
> a) Doesn't allow usage of both converters at once, since there's only 1
> PCM object.
> 
> b) Ends up with up to 4 (in NVIDIA's case) pin widgets (and hence ELD
> information) logically associated with the PCM object. Hence, it'll be
> confusing for application to know what features they can really use on
> the PCM. In fact, ALSA actually only enforces the ELD of the first pin
> associated with the converter, and ignores the rest.
> 
> The basic mechanics of a-PCM-per-pin would require hdmi_pcm_open to look
> at the list of converters, find one not in use by a stream, and then
> dynamically associate the converter with the stream, or fail the open if
> a converter was not available. We'd also need a custom close or cleanup
> function to "unassign" the converter from the PCM, and set
> hda_pcm_stream.nid to an invalid value.
> 
> That might be easiest to implement if hda_pcm_stream.nid were to refer
> to the pin widget rather than the converter widget. However, I notice
> that some core code in hda_codec.c would be confused by this, so
> perhaps we'd better just have have hdmi_pcm_open use some other way to
> determine which stream object is associated with the pin. I suppose
> that it'd be pretty easy to set up a mapping from hda_pcm_stream to pin
> nid as part of generic_hdmi_build_pcms.

Yes, we can keep simply an array of pins corresponding to each
hda_pcm_stream.

(The reason hda_pcm_stream containing the audio-widget is that it's
 an interface to HD-audio controller.  For the controller side, it
 doesn't matter what is actual output.  Only the entry-point matters.)

> As an aside, this rather reminds me of the mux'ing that goes on inside
> codecs controlled by ASoC, although I don't think we need to expose this
> muxing to user-space in the case, but instead make it automatic as I
> outlined above.

Sounds OK.


thanks,

Takashi

Re: HDA HDMI pin to converter mapping

[David Henningsson]

On 2011-05-26 08:56, Takashi Iwai wrote:
> At Wed, 25 May 2011 11:29:35 -0700,
> Stephen Warren wrote:
>>
>> Most cards supported by patch_hdmi.c have the property of a 1:1 mapping
>> or connection between converter widgets and pin widgets. Hence,
>> patch_hdmi.c sets up a PCM for each converter, and occasionally performs
>> some operations on the pin widget as required.
>>
>> However, some new cards don't have this 1:1 mapping. In particular, the
>> new GeForce GT 520 and many future NVIDIA cards, and at least the Intel
>> Ibex Peak (in my wife's laptop), have more pin widgets than converters,
>> and each pin widget has a mux to select which converter to take the
>> audio from.
>>
>> For pretty pictures, see the 4th (and 3rd) diagrams at:
>>
>> ftp://download.nvidia.com/XFree86/gpu-hdmi-audio-document/gpu-hdmi-audio.html#_examples
>
> Wow, nice to see such a good document.

Indeed!

>> I'd like to discuss how best to handle such cards.
>>
>> I'd expect to see a PCM device created for each pin widget. This would
>> create a 1:1 mapping between PCM devices and attached monitors and hence
>> ELD data (with the possible exception of DisplayPort 1.2 daisy-chaining;
>> I have no idea yet how that plays into this).
>>
>> Does this seem like a reasonable approach to everyone?
>
> Sounds reasonable and logical to me.
> It won't break the older chips, and it will work with new chips.

It makes PulseAudio's life harder, if that is to take into account; in 
the startup phase, PulseAudio usually only tries one of the pcm streams 
(which is bad here, it'll effectively rule out all ports but the first), 
if we change to all of the pcm streams, the latter two will fail since 
the former ones are already opened.

So we're effectively breaking some of the older chips by changing the 
PCM device from hw:X to hw:X,1 for a specific pin...?

OTOH, the bigger change here is what's inevitable; that PulseAudio at 
some point must start to take into account that the current capabilities 
of a PCM can change during its lifetime. (And that goes regardless of 
whether have one pcm per pin or converter.) And so the upcoming question 
is, can PulseAudio detect that the PCM capabilities have changed 
somehow, and then reprobe it?

>> The current situation is that we only create a PCM for each converter
>> that is mux'd to a pin by default HW initialization. On both the GeForce
>> 520 and Ibex Peak, this means that only 1 PCM gets created, since all
>> pin widgets' muxes point at the first converter by default. This:
>>
>> a) Doesn't allow usage of both converters at once, since there's only 1
>> PCM object.

So, create a second PCM object for the second converter...?

>> b) Ends up with up to 4 (in NVIDIA's case) pin widgets (and hence ELD
>> information) logically associated with the PCM object. Hence, it'll be
>> confusing for application to know what features they can really use on
>> the PCM. In fact, ALSA actually only enforces the ELD of the first pin
>> associated with the converter, and ignores the rest.

Looks like they should be combined; restricted to whatever formats they 
both support in case both have valid ELD's?

>> The basic mechanics of a-PCM-per-pin would require hdmi_pcm_open to look
>> at the list of converters, find one not in use by a stream, and then
>> dynamically associate the converter with the stream, or fail the open if
>> a converter was not available. We'd also need a custom close or cleanup
>> function to "unassign" the converter from the PCM, and set
>> hda_pcm_stream.nid to an invalid value.

Would it be more difficult for hdmi_pcm_open to open the associated 
converter and dynamically assign a pin?

That said, I think it's great that you brought it up Stephen, and I 
understand this doesn't help you associate PCMs with X screens - and I'm 
not sure I have a better proposal for you (and I don't want to block 
development either), just wanted to add my view to the discussion.

-- 
David Henningsson, Canonical Ltd.
http://launchpad.net/~diwic

Re: HDA HDMI pin to converter mapping

[Takashi Iwai]

At Thu, 26 May 2011 11:52:26 +0200,
David Henningsson wrote:
> 
> On 2011-05-26 08:56, Takashi Iwai wrote:
> > At Wed, 25 May 2011 11:29:35 -0700,
> > Stephen Warren wrote:
> >>
> >> Most cards supported by patch_hdmi.c have the property of a 1:1 mapping
> >> or connection between converter widgets and pin widgets. Hence,
> >> patch_hdmi.c sets up a PCM for each converter, and occasionally performs
> >> some operations on the pin widget as required.
> >>
> >> However, some new cards don't have this 1:1 mapping. In particular, the
> >> new GeForce GT 520 and many future NVIDIA cards, and at least the Intel
> >> Ibex Peak (in my wife's laptop), have more pin widgets than converters,
> >> and each pin widget has a mux to select which converter to take the
> >> audio from.
> >>
> >> For pretty pictures, see the 4th (and 3rd) diagrams at:
> >>
> >> ftp://download.nvidia.com/XFree86/gpu-hdmi-audio-document/gpu-hdmi-audio.html#_examples
> >
> > Wow, nice to see such a good document.
> 
> Indeed!
> 
> >> I'd like to discuss how best to handle such cards.
> >>
> >> I'd expect to see a PCM device created for each pin widget. This would
> >> create a 1:1 mapping between PCM devices and attached monitors and hence
> >> ELD data (with the possible exception of DisplayPort 1.2 daisy-chaining;
> >> I have no idea yet how that plays into this).
> >>
> >> Does this seem like a reasonable approach to everyone?
> >
> > Sounds reasonable and logical to me.
> > It won't break the older chips, and it will work with new chips.
> 
> It makes PulseAudio's life harder, if that is to take into account; in 
> the startup phase, PulseAudio usually only tries one of the pcm streams 
> (which is bad here, it'll effectively rule out all ports but the first), 
> if we change to all of the pcm streams, the latter two will fail since 
> the former ones are already opened.

The problem (of missing HDMI outputs) is already seen in the recent
Intel and Nvidia HDMI codecs.  They already provide multiple streams,
and snd-hda-intel also provides multiple streams, too.
So, we need a certain fix anyway.

> So we're effectively breaking some of the older chips by changing the 
> PCM device from hw:X to hw:X,1 for a specific pin...?
> 
> OTOH, the bigger change here is what's inevitable; that PulseAudio at 
> some point must start to take into account that the current capabilities 
> of a PCM can change during its lifetime. (And that goes regardless of 
> whether have one pcm per pin or converter.) And so the upcoming question 
> is, can PulseAudio detect that the PCM capabilities have changed 
> somehow, and then reprobe it?

The pin configuration won't change at runtime (unless
reconfiguration), so the configuration is static even if we bind the
HDMI PCM stream to a pin.

(I don't mean against PA changes for dynamic configuration, of course)

But maybe a question is when multiple streams are present but there
are less converters than pins.  In that case, you can't open all
streams simultaneously, but the driver won't expose that
information.

The conflict between PCM streams is always a headache.  A sane way to
expose the conflicts and relations is required...

> >> The current situation is that we only create a PCM for each converter
> >> that is mux'd to a pin by default HW initialization. On both the GeForce
> >> 520 and Ibex Peak, this means that only 1 PCM gets created, since all
> >> pin widgets' muxes point at the first converter by default. This:
> >>
> >> a) Doesn't allow usage of both converters at once, since there's only 1
> >> PCM object.
> 
> So, create a second PCM object for the second converter...?

Well, it's another option -- i.e. creating PCM streams per converter,
and provide also mux control if multiple sinks can be selected.
This avoids the problem of less-converters-than-pins, but introduces
another conflict / race, i.e. assigning a pin from multiple
converters.


> >> b) Ends up with up to 4 (in NVIDIA's case) pin widgets (and hence ELD
> >> information) logically associated with the PCM object. Hence, it'll be
> >> confusing for application to know what features they can really use on
> >> the PCM. In fact, ALSA actually only enforces the ELD of the first pin
> >> associated with the converter, and ignores the rest.
> 
> Looks like they should be combined; restricted to whatever formats they 
> both support in case both have valid ELD's?
> 
> >> The basic mechanics of a-PCM-per-pin would require hdmi_pcm_open to look
> >> at the list of converters, find one not in use by a stream, and then
> >> dynamically associate the converter with the stream, or fail the open if
> >> a converter was not available. We'd also need a custom close or cleanup
> >> function to "unassign" the converter from the PCM, and set
> >> hda_pcm_stream.nid to an invalid value.
> 
> Would it be more difficult for hdmi_pcm_open to open the associated 
> converter and dynamically assign a pin?
> 
> That said, I think it's great that you brought it up Stephen, and I 
> understand this doesn't help you associate PCMs with X screens - and I'm 
> not sure I have a better proposal for you (and I don't want to block 
> development either), just wanted to add my view to the discussion.


Takashi

Re: HDA HDMI pin to converter mapping

[David Henningsson]

On 2011-05-26 12:46, Takashi Iwai wrote:
> At Thu, 26 May 2011 11:52:26 +0200,
> David Henningsson wrote:
>>
>> On 2011-05-26 08:56, Takashi Iwai wrote:
>>> At Wed, 25 May 2011 11:29:35 -0700,
>>> Stephen Warren wrote:
>>>>
>>>> Most cards supported by patch_hdmi.c have the property of a 1:1 mapping
>>>> or connection between converter widgets and pin widgets. Hence,
>>>> patch_hdmi.c sets up a PCM for each converter, and occasionally performs
>>>> some operations on the pin widget as required.
>>>>
>>>> However, some new cards don't have this 1:1 mapping. In particular, the
>>>> new GeForce GT 520 and many future NVIDIA cards, and at least the Intel
>>>> Ibex Peak (in my wife's laptop), have more pin widgets than converters,
>>>> and each pin widget has a mux to select which converter to take the
>>>> audio from.
>>>>
>>>> For pretty pictures, see the 4th (and 3rd) diagrams at:
>>>>
>>>> ftp://download.nvidia.com/XFree86/gpu-hdmi-audio-document/gpu-hdmi-audio.html#_examples
>>>
>>> Wow, nice to see such a good document.
>>
>> Indeed!
>>
>>>> I'd like to discuss how best to handle such cards.
>>>>
>>>> I'd expect to see a PCM device created for each pin widget. This would
>>>> create a 1:1 mapping between PCM devices and attached monitors and hence
>>>> ELD data (with the possible exception of DisplayPort 1.2 daisy-chaining;
>>>> I have no idea yet how that plays into this).
>>>>
>>>> Does this seem like a reasonable approach to everyone?
>>>
>>> Sounds reasonable and logical to me.
>>> It won't break the older chips, and it will work with new chips.
>>
>> It makes PulseAudio's life harder, if that is to take into account; in
>> the startup phase, PulseAudio usually only tries one of the pcm streams
>> (which is bad here, it'll effectively rule out all ports but the first),
>> if we change to all of the pcm streams, the latter two will fail since
>> the former ones are already opened.
>
> The problem (of missing HDMI outputs) is already seen in the recent
> Intel and Nvidia HDMI codecs.  They already provide multiple streams,
> and snd-hda-intel also provides multiple streams, too.
> So, we need a certain fix anyway.
>
>> So we're effectively breaking some of the older chips by changing the
>> PCM device from hw:X to hw:X,1 for a specific pin...?
>>
>> OTOH, the bigger change here is what's inevitable; that PulseAudio at
>> some point must start to take into account that the current capabilities
>> of a PCM can change during its lifetime. (And that goes regardless of
>> whether have one pcm per pin or converter.) And so the upcoming question
>> is, can PulseAudio detect that the PCM capabilities have changed
>> somehow, and then reprobe it?
>
> The pin configuration won't change at runtime (unless
> reconfiguration), so the configuration is static even if we bind the
> HDMI PCM stream to a pin.

But capabilities will still change, e g if one plugs in a receiver that 
can do 8 channels, then unplug it and then plug in something that can 
only do 2 channels. That still changes the hw_params of the PCM, which 
is what PA tries to detect at startup, and never reprobes.

> (I don't mean against PA changes for dynamic configuration, of course)
>
> But maybe a question is when multiple streams are present but there
> are less converters than pins.  In that case, you can't open all
> streams simultaneously, but the driver won't expose that
> information.
>
> The conflict between PCM streams is always a headache.  A sane way to
> expose the conflicts and relations is required...

Yeah, and it seems like every change we could do would break something, 
somewhere...

I think maybe one problem could be that ALSA lacks the concept of a pin. 
Some object that you could connect jacks to (the /dev/input/event 
stuff), connect ELD data to, and one or more converters to.

>>>> The current situation is that we only create a PCM for each converter
>>>> that is mux'd to a pin by default HW initialization. On both the GeForce
>>>> 520 and Ibex Peak, this means that only 1 PCM gets created, since all
>>>> pin widgets' muxes point at the first converter by default. This:
>>>>
>>>> a) Doesn't allow usage of both converters at once, since there's only 1
>>>> PCM object.
>>
>> So, create a second PCM object for the second converter...?
>
> Well, it's another option -- i.e. creating PCM streams per converter,
> and provide also mux control if multiple sinks can be selected.
> This avoids the problem of less-converters-than-pins, but introduces
> another conflict / race, i.e. assigning a pin from multiple
> converters.

It is also the current way of doing things on the analogue side - should 
we have one-pcm-per-pin, would you see that happening on the analogue 
side as well, e g one pcm for headphone and another for speakers?

HDMI is different in the way that it changes the capabilities on the PCM 
device, so I'm not saying we should, just wondering.

Something to also keep in mind is that AFAICT it is perfectly possible 
to make a codec which has both HDMI outs and analogue outs. I don't 
think anyone has done such a codec yet, but it wouldn't surprise me if 
that would happen.

>>>> b) Ends up with up to 4 (in NVIDIA's case) pin widgets (and hence ELD
>>>> information) logically associated with the PCM object. Hence, it'll be
>>>> confusing for application to know what features they can really use on
>>>> the PCM. In fact, ALSA actually only enforces the ELD of the first pin
>>>> associated with the converter, and ignores the rest.
>>
>> Looks like they should be combined; restricted to whatever formats they
>> both support in case both have valid ELD's?
>>
>>>> The basic mechanics of a-PCM-per-pin would require hdmi_pcm_open to look
>>>> at the list of converters, find one not in use by a stream, and then
>>>> dynamically associate the converter with the stream, or fail the open if
>>>> a converter was not available. We'd also need a custom close or cleanup
>>>> function to "unassign" the converter from the PCM, and set
>>>> hda_pcm_stream.nid to an invalid value.
>>
>> Would it be more difficult for hdmi_pcm_open to open the associated
>> converter and dynamically assign a pin?
>>
>> That said, I think it's great that you brought it up Stephen, and I
>> understand this doesn't help you associate PCMs with X screens - and I'm
>> not sure I have a better proposal for you (and I don't want to block
>> development either), just wanted to add my view to the discussion.
>
>
> Takashi
> _______________________________________________
> Alsa-devel mailing list
> Alsa-devel@alsa-project.org
> http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
>


-- 
David Henningsson, Canonical Ltd.
http://launchpad.net/~diwic

Re: HDA HDMI pin to converter mapping

[Stephen Warren]

David Henningsson wrote at Thursday, May 26, 2011 3:52 AM:
> On 2011-05-26 08:56, Takashi Iwai wrote:
> > At Wed, 25 May 2011 11:29:35 -0700, Stephen Warren wrote:
> >>
> >> Most cards supported by patch_hdmi.c have the property of a 1:1 mapping
> >> or connection between converter widgets and pin widgets. Hence,
> >> patch_hdmi.c sets up a PCM for each converter, and occasionally performs
> >> some operations on the pin widget as required.
> >>
> >> However, some new cards don't have this 1:1 mapping. In particular, the
> >> new GeForce GT 520 and many future NVIDIA cards, and at least the Intel
> >> Ibex Peak (in my wife's laptop), have more pin widgets than converters,
> >> and each pin widget has a mux to select which converter to take the
> >> audio from.
> >>
> >> For pretty pictures, see the 4th (and 3rd) diagrams at:
> >>
> >> ftp://download.nvidia.com/XFree86/gpu-hdmi-audio-document/gpu-hdmi-audio.html#_examples
> >
> > Wow, nice to see such a good document.
> 
> Indeed!
> 
> >> I'd like to discuss how best to handle such cards.
> >>
> >> I'd expect to see a PCM device created for each pin widget. This would
> >> create a 1:1 mapping between PCM devices and attached monitors and hence
> >> ELD data (with the possible exception of DisplayPort 1.2 daisy- chaining;
> >> I have no idea yet how that plays into this).
> >>
> >> Does this seem like a reasonable approach to everyone?
> >
> > Sounds reasonable and logical to me.
> > It won't break the older chips, and it will work with new chips.
> 
> It makes PulseAudio's life harder, if that is to take into account; in

(I'll use the phrase "unsupported" below to mean the codecs with fewer
converters than pins. This is slightly false since they are supported
to some extent, but bear with the simple terminology:-)

Certainly, we should take this into account. But, I don't think it actually
changes anything for PulseAudio; see my comments below.

> the startup phase, PulseAudio usually only tries one of the pcm streams
> (which is bad here, it'll effectively rule out all ports but the first),
> if we change to all of the pcm streams, the latter two will fail since
> the former ones are already opened.

PulseAudio already needs to deal with this.

The supported NVIDIA HDA controllers have 4 codecs, each having 1
converter and 1 pin. Hence, there are 4 PCMs, 1 per codec. PCMs can't
be unified/shared/... across codecs. However, our HDA controllers only
support 2 streams, because our GPUs only support two active displays
at once. Hence, any attempt to open all 4 PCMs at the same time
(rather than in serial) will fail on the 3rd PCM.

So, the one-PCM-per-pin model for the new unsupported codecs doesn't
introduce any new issue here.

> So we're effectively breaking some of the older chips by changing the
> PCM device from hw:X to hw:X,1 for a specific pin...?

I don't believe the PCM device names would change for existing supported
codecs.

Specifically, the PCM names are:

hw:${card},${codec_index}                    (1 converter)

hw:${card},${codec_index}.${converter_index} (n converters)

Since for existing supported codecs, the number of codecs and
converters doesn't change in HW, the device names won't change.

Yes, we'll change ${converter_index} to ${pin_index} in the above
names, but again, since that's going to be 0 in both cases for
existing codecs, there will be no change.

For the unsupported codecs, yes, we'll start to see ${pin_index} be
non-zero, and end up with names like hw:0,3.0, hw:0,3.1, and hw:0,3.2.
However, even if we made a PCM-per-converter work for those codecs,
rather than switching to a PCM-per-pin like I proposed, we'd still end
up with those new names, simply due to the existence of multiple
converters within the codec.

In fact, if there are any codecs that have a 1:1 mapping between
converters and pins, yet lump everything into a single codec, we already
have those names. I don't know enough about other vendors' codecs to 
know if this setup existing in practice or not.

So, PulseAudio needs to support hw:x,P.z with z!=0 anyway.

> OTOH, the bigger change here is what's inevitable; that PulseAudio at
> some point must start to take into account that the current capabilities
> of a PCM can change during its lifetime. (And that goes regardless of
> whether have one pcm per pin or converter.) And so the upcoming question
> is, can PulseAudio detect that the PCM capabilities have changed
> somehow, and then reprobe it?

Sure, but that happens irrespective.

Right now, the PCM capabilities can change when you plug in a different
monitor at run-time. I think the Jack detection you added, and the
ability to expose the ELD as a binary kcontrol (or similar) that
Pierre-Louis mentioned, should help here.

For the unsupported codecs, I think the PCM-per-pin makes this situation
far more manageable; there will be a single ELD per PCM, rather than
one converter driving N pins, each of which has an ELD, and hence
applications would have to just pick one (which?) or merge the
capabilities, which would be complex and probably a usability nightmare.

> >> The current situation is that we only create a PCM for each converter
> >> that is mux'd to a pin by default HW initialization. On both the GeForce
> >> 520 and Ibex Peak, this means that only 1 PCM gets created, since all
> >> pin widgets' muxes point at the first converter by default. This:
> >>
> >> a) Doesn't allow usage of both converters at once, since there's only 1
> >> PCM object.
> 
> So, create a second PCM object for the second converter...?

I don't think that'd work very well; one converter driving N pins would
run in to the problems in the last paragraph I mentioned above.

If we did do one-PCM-per-converter, we'd probably need to expose the muxes
in the pins as kcontrols and allow user-space to select which PCM routes
to which pin. That would more directly represent the HW capabilities, but
would be more complex for user-space to manage. We'd need to expose the
ELD information for a pin rather than a PCM too, so we'd need some concept
of a pin at the ALSA API level (or a defacto naming standard for the ELD
kcontrols used to retrieve the data).

> >> b) Ends up with up to 4 (in NVIDIA's case) pin widgets (and hence ELD
> >> information) logically associated with the PCM object. Hence, it'll be
> >> confusing for application to know what features they can really use on
> >> the PCM. In fact, ALSA actually only enforces the ELD of the first pin
> >> associated with the converter, and ignores the rest.
> 
> Looks like they should be combined; restricted to whatever formats they
> both support in case both have valid ELD's?
> 
> >> The basic mechanics of a-PCM-per-pin would require hdmi_pcm_open to look
> >> at the list of converters, find one not in use by a stream, and then
> >> dynamically associate the converter with the stream, or fail the open if
> >> a converter was not available. We'd also need a custom close or cleanup
> >> function to "unassign" the converter from the PCM, and set
> >> hda_pcm_stream.nid to an invalid value.
> 
> Would it be more difficult for hdmi_pcm_open to open the associated
> converter and dynamically assign a pin?

Implementing that would probably be just as easy. The difficult part is
knowing which pin the user wanted to "assign" the converter to. That's the
part that having an explicit PCM-per-pin solves.

Thanks for taking the time to think the proposal; it's always good to get
feedback.

--
nvpublic

Re: HDA HDMI pin to converter mapping

[Stephen Warren]

Stephen Warren wrote at Thursday, May 26, 2011 11:45 AM:
> David Henningsson wrote at Thursday, May 26, 2011 3:52 AM:
> ...
> > So we're effectively breaking some of the older chips by changing the
> > PCM device from hw:X to hw:X,1 for a specific pin...?
> 
> I don't believe the PCM device names would change for existing supported
> codecs.
> 
> Specifically, the PCM names are:
> 
> hw:${card},${codec_index}                    (1 converter)
> 
> hw:${card},${codec_index}.${converter_index} (n converters)

Actually, I think I was confused here; looking at snd_hda_codec_build_pcms,
I think the naming is always flat, irrespective of how many converters
are within a codec:

hw:${card},${dev}

with dev being assigned in sequence 3,7,8,9 within an entire HDA bus,
which I think means within an entire HDA controller.

So, the naming scheme most likely won't change at all.

-- 
nvpublic

Re: HDA HDMI pin to converter mapping

[David Henningsson]

On 2011-05-26 19:45, Stephen Warren wrote:
> David Henningsson wrote at Thursday, May 26, 2011 3:52 AM:
>> On 2011-05-26 08:56, Takashi Iwai wrote:
>>> At Wed, 25 May 2011 11:29:35 -0700, Stephen Warren wrote:
>>>>
>>>> Most cards supported by patch_hdmi.c have the property of a 1:1 mapping
>>>> or connection between converter widgets and pin widgets. Hence,
>>>> patch_hdmi.c sets up a PCM for each converter, and occasionally performs
>>>> some operations on the pin widget as required.
>>>>
>>>> However, some new cards don't have this 1:1 mapping. In particular, the
>>>> new GeForce GT 520 and many future NVIDIA cards, and at least the Intel
>>>> Ibex Peak (in my wife's laptop), have more pin widgets than converters,
>>>> and each pin widget has a mux to select which converter to take the
>>>> audio from.
>>>>
>>>> For pretty pictures, see the 4th (and 3rd) diagrams at:
>>>>
>>>> ftp://download.nvidia.com/XFree86/gpu-hdmi-audio-document/gpu-hdmi-audio.html#_examples
>>>
>>> Wow, nice to see such a good document.
>>
>> Indeed!
>>
>>>> I'd like to discuss how best to handle such cards.
>>>>
>>>> I'd expect to see a PCM device created for each pin widget. This would
>>>> create a 1:1 mapping between PCM devices and attached monitors and hence
>>>> ELD data (with the possible exception of DisplayPort 1.2 daisy- chaining;
>>>> I have no idea yet how that plays into this).
>>>>
>>>> Does this seem like a reasonable approach to everyone?
>>>
>>> Sounds reasonable and logical to me.
>>> It won't break the older chips, and it will work with new chips.
>>
>> It makes PulseAudio's life harder, if that is to take into account; in
>
> (I'll use the phrase "unsupported" below to mean the codecs with fewer
> converters than pins. This is slightly false since they are supported
> to some extent, but bear with the simple terminology:-)
>
> Certainly, we should take this into account. But, I don't think it actually
> changes anything for PulseAudio; see my comments below.
>
>> the startup phase, PulseAudio usually only tries one of the pcm streams
>> (which is bad here, it'll effectively rule out all ports but the first),
>> if we change to all of the pcm streams, the latter two will fail since
>> the former ones are already opened.
>
> PulseAudio already needs to deal with this.

I stand corrected; I must have thought wrong somehow.

So from a conceptual view, I don't really like the one-pcm-per-pin idea, 
because it makes HDMI hda and analogue hda more different. (Or are we 
considering one-pcm-per-pin for the other codecs as well?) We risc 
fragmenting the driver further, where we should instead try to unify it.

But from a pragmatic view, for this particular problem, it seems your 
solution might be the simplest way forward.

> The supported NVIDIA HDA controllers have 4 codecs, each having 1
> converter and 1 pin. Hence, there are 4 PCMs, 1 per codec. PCMs can't
> be unified/shared/... across codecs. However, our HDA controllers only
> support 2 streams, because our GPUs only support two active displays
> at once. Hence, any attempt to open all 4 PCMs at the same time
> (rather than in serial) will fail on the 3rd PCM.
>
> So, the one-PCM-per-pin model for the new unsupported codecs doesn't
> introduce any new issue here.
>
>> So we're effectively breaking some of the older chips by changing the
>> PCM device from hw:X to hw:X,1 for a specific pin...?
>
> I don't believe the PCM device names would change for existing supported
> codecs.

If you have one converter and three pins connected to it (I think some 
intels have this?), then addressing the second and third pin would be a 
change.

> For the unsupported codecs, yes, we'll start to see ${pin_index} be
> non-zero, and end up with names like hw:0,3.0, hw:0,3.1, and hw:0,3.2.
> However, even if we made a PCM-per-converter work for those codecs,
> rather than switching to a PCM-per-pin like I proposed, we'd still end
> up with those new names, simply due to the existence of multiple
> converters within the codec.

*If* you want to use the second converter at all, which will only be in 
the relative rare case of you wanting to use two converters in parallel.

> In fact, if there are any codecs that have a 1:1 mapping between
> converters and pins, yet lump everything into a single codec, we already
> have those names. I don't know enough about other vendors' codecs to
> know if this setup existing in practice or not.
>
> So, PulseAudio needs to support hw:x,P.z with z!=0 anyway.

But you're right about this (with your correction about it's hw:x,y in 
both cases), so maybe it doesn't matter...

> Implementing that would probably be just as easy. The difficult part is
> knowing which pin the user wanted to "assign" the converter to. That's the
> part that having an explicit PCM-per-pin solves.

In the pcm-per-converter, one would just take the first one that 
supports audio. For the simplest and thus most common case (only one 
hdmi monitor connected) then pcm-per-converter would be simpler (just 
access the ,0 device, no need to figure out which device to use), for 
the more complex cases, maybe pcm-per-pin is simpler.

> Thanks for taking the time to think the proposal; it's always good to get
> feedback.

Thanks for taking the time to write it. :-)

-- 
David Henningsson, Canonical Ltd.
http://launchpad.net/~diwic

Re: HDA HDMI pin to converter mapping

[Stephen Warren]

David Henningsson wrote at Friday, May 27, 2011 2:45 AM:
> So from a conceptual view, I don't really like the one-pcm-per-pin idea,
> because it makes HDMI hda and analogue hda more different. (Or are we
> considering one-pcm-per-pin for the other codecs as well?) We risc
> fragmenting the driver further, where we should instead try to unify it.
> 
> But from a pragmatic view, for this particular problem, it seems your
> solution might be the simplest way forward.

OK. Let's discuss the alternatives.

The only alternative I see is to have 1 PCM per supported converter.
Whether we end supporting just 1 or all N converters are just different
variations on this theme.

Since in general there are fewer converters than pins, we need some way
to know (or control) which pins the converters (and hence PCMs) are
routed to.

A few options exist:

1)

Statically support 1 converter and route from the converter to all pins
that can be routed to (or all pins with valid ELDs at a given time).

The advantage of this is a single PCM, so less for applications to
enumerate and handle.

The disadvantage here is that the multiple ELDs are associated with a
single PCM. I believe this creates basically intractable issues with how
to resolve what audio formats to allow.

One approach would be to take the common denominator of formats across
all ELDs, but this would prevent multi-channel audio being sent to an
A/V receiver on one HDMI link if a simple stereo-only monitor was
connected to another HDMI link. That seems like a very bad user
experience.

Another approach would be to honor just one of the ELDs. That would
allow full use of a single device's audio capabilities. I see two issues
with this: (a) How to tell the ALSA driver which ELD to honor, since it
limits a PCM's capabilities by the union of the converter's and ELD's
capabilities. (b) That same audio gets sent to all HDMI devices. Some
may not support the format. Hopefully the devices will just ignore the
signal rather than playing e.g. static. However, I'm sure some devices
will have bugs there.

2)

Allocate a PCM per converter, but somehow dynamically connect the PCM
and converter to some ALSA-kernel-driver-determined pin when it's opened,
e.g. the first not-in-use pin with a valid ELD.

The disadvantage here is that the kernel is implementing policy; it's
deciding which pin complexes to use. If there are 3 connected active
HDMI/DP monitors, this ends up meaning that audio can never be routed to
one of them (since you could only open e.g. 2 PCMs at once, and hence
only access the first two pins, since this model gives user-space no
control over the routing selection, and the ALSA driver would presumably
be consistent in its routing selection)

Again, that seems like a very bad user experience. I believe some AMD
graphics cards support 6 displays at once. I'm not sure if they support
audio on all 6 displays too, but if they do, we could actually run in
to this situation very quickly.

3)

Allocate one PCM per converter. Implement a kcontrol for the mux on the
front of each pin, so user-space can determine the routing policy; it
will open one PCM, flip the kcontrols to setup the routing to the
appropriate pin, and go.

This case starts to sound a lot like ASoC audio configuration.

This approach is more complex for user-space applications; they now need
a way to know there's routing involved, configure that routing, etc. I'm
not familiar with analog codecs, so perhaps this is completely normal?
Alternatively, this might end up needing UCM (Use-Case Manager) applied
to HD-audio controllers in addition to ASoC-based audio cards. While UCM
is good and necessary for ASoC-based cards, I'm not convinced we should
push it at situations that can be solved by simpler means.

My inclination is that this approach just dumps a lot of complexity onto
user-space, without actually achieving anything more than a PCM-per-pin
setup would.

Off the top of my head, I can't think of any other alternatives. Do let
me know if there's something better.

As an aside, I think I've got the PCM-per-pin approach coded up at least
for codecs that use the "generic" setup in patch_hdmi.c. I'll start
testing out how it works today.

More below.

> > The supported NVIDIA HDA controllers have 4 codecs, each having 1
> > converter and 1 pin. Hence, there are 4 PCMs, 1 per codec. PCMs can't
> > be unified/shared/... across codecs. However, our HDA controllers only
> > support 2 streams, because our GPUs only support two active displays
> > at once. Hence, any attempt to open all 4 PCMs at the same time
> > (rather than in serial) will fail on the 3rd PCM.
> >
> > So, the one-PCM-per-pin model for the new unsupported codecs doesn't
> > introduce any new issue here.
> >
> >> So we're effectively breaking some of the older chips by changing the
> >> PCM device from hw:X to hw:X,1 for a specific pin...?
> >
> > I don't believe the PCM device names would change for existing supported
> > codecs.
> 
> If you have one converter and three pins connected to it (I think some
> intels have this?), then addressing the second and third pin would be a
> change.

Sure, although I think some way of accessing the other 2 pins is required,
and I don't think that changes any existing device names, just adds some
new device names.

> > For the unsupported codecs, yes, we'll start to see ${pin_index} be
> > non-zero, and end up with names like hw:0,3.0, hw:0,3.1, and hw:0,3.2.
> > However, even if we made a PCM-per-converter work for those codecs,
> > rather than switching to a PCM-per-pin like I proposed, we'd still end
> > up with those new names, simply due to the existence of multiple
> > converters within the codec.
> 
> *If* you want to use the second converter at all, which will only be in
> the relative rare case of you wanting to use two converters in parallel.

Well, I do that often when testing:-)

I assume that our marketing department determined that would be a
worthwhile use-case to support, or we probably wouldn't have burned
the hardware on multiple converters.

As an aside, I believe Windows exposes the general model of PCM-per-pin.
I have no idea about Mac. Not that Linux has to follow other vendors, but
just a note that others consider the use-case worth supporting.

-- 
nvpublic