[RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Ricardo Neri]

Hello!

I have been working on prototypes for the ASoC OMAP HDMI audio driver to 
propagate events from the HDMI output (e.g., display getting 
enabled/disabled/suspended). This for the users of the driver to react 
to such events. For instance, if the display is disabled or disconected, 
audio could be stopped, rerouted or whatever other decision the user 
makes. This is needed because, if, for instance, the  HDMI IP goes off, 
audio will stall and the audio users will only see a "playback write 
error (DMA or IRQ trouble?)"

In my prototypes I have used snd_soc_jack for this purpose and I have 
some questions:

*I see snd_soc_jack is used mostly for headsets and microphones with 
actual external mechanical connections. Strictly, in my case I propagate 
events originated by the OMAP display driver (changes in the power 
state), and not from external events. Some of these events are generated 
from an actual HDMI cable connection/disconnection, though.

*Maybe the event should be propagated by omapdss/omapdrm/drm and the 
entity in charge of the audio policy should listen those events instead.

*I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it is 
feasible for an audio driver to report events from an AV output.

I was wondering about how much sense does it make to you guys use a 
snd_soc_jack in this case?

Thanks in advance for your comments!

Ricardo

Re: [alsa-devel] [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Takashi Iwai]

At Mon, 20 Aug 2012 20:47:17 -0500,
Ricardo Neri wrote:
> 
> Hello!
> 
> I have been working on prototypes for the ASoC OMAP HDMI audio driver to 
> propagate events from the HDMI output (e.g., display getting 
> enabled/disabled/suspended). This for the users of the driver to react 
> to such events. For instance, if the display is disabled or disconected, 
> audio could be stopped, rerouted or whatever other decision the user 
> makes. This is needed because, if, for instance, the  HDMI IP goes off, 
> audio will stall and the audio users will only see a "playback write 
> error (DMA or IRQ trouble?)"
> 
> In my prototypes I have used snd_soc_jack for this purpose and I have 
> some questions:
> 
> *I see snd_soc_jack is used mostly for headsets and microphones with 
> actual external mechanical connections. Strictly, in my case I propagate 
> events originated by the OMAP display driver (changes in the power 
> state), and not from external events. Some of these events are generated 
> from an actual HDMI cable connection/disconnection, though.
> 
> *Maybe the event should be propagated by omapdss/omapdrm/drm and the 
> entity in charge of the audio policy should listen those events instead.
> 
> *I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it is 
> feasible for an audio driver to report events from an AV output.
> 
> I was wondering about how much sense does it make to you guys use a 
> snd_soc_jack in this case?

HD-audio already uses the generic jack event for the HDMI/DP
connection change notification as well, so I think it would make sense
in general.


Takashi

Re: [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Tomi Valkeinen]

[-- Attachment #1.1: Type: text/plain, Size: 1875 bytes --]

On Mon, 2012-08-20 at 20:47 -0500, Ricardo Neri wrote:
> Hello!
> 
> I have been working on prototypes for the ASoC OMAP HDMI audio driver to 
> propagate events from the HDMI output (e.g., display getting 
> enabled/disabled/suspended). This for the users of the driver to react 
> to such events. For instance, if the display is disabled or disconected, 
> audio could be stopped, rerouted or whatever other decision the user 
> makes. This is needed because, if, for instance, the  HDMI IP goes off, 
> audio will stall and the audio users will only see a "playback write 
> error (DMA or IRQ trouble?)"
> 
> In my prototypes I have used snd_soc_jack for this purpose and I have 
> some questions:
> 
> *I see snd_soc_jack is used mostly for headsets and microphones with 
> actual external mechanical connections. Strictly, in my case I propagate 
> events originated by the OMAP display driver (changes in the power 
> state), and not from external events. Some of these events are generated 
> from an actual HDMI cable connection/disconnection, though.
> 
> *Maybe the event should be propagated by omapdss/omapdrm/drm and the 
> entity in charge of the audio policy should listen those events instead.
> 
> *I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it is 
> feasible for an audio driver to report events from an AV output.
> 
> I was wondering about how much sense does it make to you guys use a 
> snd_soc_jack in this case?

How does DRM handle audio? I made a quick grep, but I see the drm
drivers only enabling the audio in the HW, nothing else.

If there's a common generic way to handle this, we should obviously use
that. But if we need to choose between doing something custom or doing
it in omapdrm driver, I think we should go for drm the only solution and
forget about audio with omapfb.

 Tomi


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 1566 bytes --]

On Tue, Aug 21, 2012 at 07:28:34AM +0200, Takashi Iwai wrote:
> Ricardo Neri wrote:

> > I was wondering about how much sense does it make to you guys use a 
> > snd_soc_jack in this case?

> HD-audio already uses the generic jack event for the HDMI/DP
> connection change notification as well, so I think it would make sense
> in general.

The whole problem here is that we don't *have* a generic jack interface.
We've got:

 - sound/core/jack.c which was written to be a generic API and is used
   by everything that does jack support currently.

 - sound/core/ctljack.c which was added later and provides separate
   in-kernel and userspace APIs and is currently only used by HDA.

 - extcon which does have a good reason to be a separate API since that
   it's not audio specific (and is likely to be picked up by Android as
   the code was originally taken from there); it's currently not
   supported by the frameworks in ALSA.  I'd suggest Pulse should be using
   it too.

This is a complete shambles for both driver authors and userspace, the
ABI varies randomly with drivers and in theory driver authors have to
implement everything three times which is just nuts.

What I'd like to see happening is that we merge ctljack into jack (since
only HDA is going to be affected by that change it seems like the right
direction to make the merge) and also add extcon support, I have looked
at the extcon support.

Short term for drivers used on embedded systems I'd have to recommend
extcon rather than anything ALSA-specific.

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [alsa-devel] [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[David Henningsson]

On 08/21/2012 02:05 PM, Mark Brown wrote:
> On Tue, Aug 21, 2012 at 07:28:34AM +0200, Takashi Iwai wrote:
>> Ricardo Neri wrote:
>
>>> I was wondering about how much sense does it make to you guys use a
>>> snd_soc_jack in this case?
>
>> HD-audio already uses the generic jack event for the HDMI/DP
>> connection change notification as well, so I think it would make sense
>> in general.
>
> The whole problem here is that we don't *have* a generic jack interface.
> We've got:
>
>   - sound/core/jack.c which was written to be a generic API and is used
>     by everything that does jack support currently.
>
>   - sound/core/ctljack.c which was added later and provides separate
>     in-kernel and userspace APIs and is currently only used by HDA.

This is wrong. PulseAudio uses the new ctljack API. I started out with 
an implementation which used the input device API (sound/core/jack.c), 
but since some people did not like this API, the ctljack API was 
designed and implemented for PA to use, which it does (since PulseAudio 
2.0 - the input device API implementation in PulseAudio was never merged 
upstream).

>   - extcon which does have a good reason to be a separate API since that
>     it's not audio specific (and is likely to be picked up by Android as
>     the code was originally taken from there); it's currently not
>     supported by the frameworks in ALSA.  I'd suggest Pulse should be using
>     it too.
>
> This is a complete shambles for both driver authors and userspace, the
> ABI varies randomly with drivers and in theory driver authors have to
> implement everything three times which is just nuts.
>
> What I'd like to see happening is that we merge ctljack into jack (since
> only HDA is going to be affected by that change it seems like the right
> direction to make the merge) and also add extcon support, I have looked
> at the extcon support.

I don't know either why we have two different interfaces for jack 
detection (not counting extcon) for driver writers. I think we should 
merge jack and ctljack, as long as that does not mean you break anything 
I'm relying on in ctljack.

Maybe something for discussion at Plumbers?

> Short term for drivers used on embedded systems I'd have to recommend
> extcon rather than anything ALSA-specific.

I thought that when Takashi did the new ctljack interface, that would 
effectively deprecate the old input device interface, and that ctljack 
was the new recommendation.


-- 
David Henningsson, Canonical Ltd.
https://launchpad.net/~diwic

Re: [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Clark, Rob]

On Tue, Aug 21, 2012 at 1:01 AM, Tomi Valkeinen <tomi.valkeinen@ti.com> wrote:
> On Mon, 2012-08-20 at 20:47 -0500, Ricardo Neri wrote:
>> Hello!
>>
>> I have been working on prototypes for the ASoC OMAP HDMI audio driver to
>> propagate events from the HDMI output (e.g., display getting
>> enabled/disabled/suspended). This for the users of the driver to react
>> to such events. For instance, if the display is disabled or disconected,
>> audio could be stopped, rerouted or whatever other decision the user
>> makes. This is needed because, if, for instance, the  HDMI IP goes off,
>> audio will stall and the audio users will only see a "playback write
>> error (DMA or IRQ trouble?)"
>>
>> In my prototypes I have used snd_soc_jack for this purpose and I have
>> some questions:
>>
>> *I see snd_soc_jack is used mostly for headsets and microphones with
>> actual external mechanical connections. Strictly, in my case I propagate
>> events originated by the OMAP display driver (changes in the power
>> state), and not from external events. Some of these events are generated
>> from an actual HDMI cable connection/disconnection, though.
>>
>> *Maybe the event should be propagated by omapdss/omapdrm/drm and the
>> entity in charge of the audio policy should listen those events instead.
>>
>> *I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it is
>> feasible for an audio driver to report events from an AV output.
>>
>> I was wondering about how much sense does it make to you guys use a
>> snd_soc_jack in this case?
>
> How does DRM handle audio? I made a quick grep, but I see the drm
> drivers only enabling the audio in the HW, nothing else.

I confess to not knowing too much about audio/alsa, but what does
audio driver need from hdmi?  Just hotplug events?

>From a quick look, it seems most of what the existing drm drivers are
doing is detecting if the display supports audio, and then turning
on/off the hw.. (and some infoframe stuff in some cases).

Does ASoC support 'hotplug' of audio devices?  If so, maybe it makes
some sense to have some support in drm core.  At least all the edid
parsing stuff to determine if the display supports audio should be
generic and not driver specific.

BR,
-R

> If there's a common generic way to handle this, we should obviously use
> that. But if we need to choose between doing something custom or doing
> it in omapdrm driver, I think we should go for drm the only solution and
> forget about audio with omapfb.
>
>  Tomi
>

Re: [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 2274 bytes --]

On Tue, Aug 21, 2012 at 02:30:51PM +0200, David Henningsson wrote:
> On 08/21/2012 02:05 PM, Mark Brown wrote:

> >  - sound/core/ctljack.c which was added later and provides separate
> >    in-kernel and userspace APIs and is currently only used by HDA.

> This is wrong. PulseAudio uses the new ctljack API. I started out
> with an implementation which used the input device API
> (sound/core/jack.c), but since some people did not like this API,
> the ctljack API was designed and implemented for PA to use, which it
> does (since PulseAudio 2.0 - the input device API implementation in
> PulseAudio was never merged upstream).

...and the only thing using this API to generate events is HDA which is
what I was talking about here given that this is a question from a
driver point of view.

> >What I'd like to see happening is that we merge ctljack into jack (since
> >only HDA is going to be affected by that change it seems like the right
> >direction to make the merge) and also add extcon support, I have looked
> >at the extcon support.

> I don't know either why we have two different interfaces for jack
> detection (not counting extcon) for driver writers. I think we
> should merge jack and ctljack, as long as that does not mean you
> break anything I'm relying on in ctljack.

> Maybe something for discussion at Plumbers?

We discussed this last time we all met and came to the above conclusion :/

> >Short term for drivers used on embedded systems I'd have to recommend
> >extcon rather than anything ALSA-specific.

> I thought that when Takashi did the new ctljack interface, that
> would effectively deprecate the old input device interface, and that
> ctljack was the new recommendation.

Android currently uses the out of tree version of the extcon ABI (like I
say it's where the code originated), and some OSs do use the input ABI
also but realistically if you've got to pick one Android is where the
market is at.

To my knowledge no embedded OS is using the control jacks, it's possible
that there's something using them as a function of using PulseAudio but
the ones I'm famililar with currently ignore PulseAudio routing and just
use the mixing facilities.  Given the dependency on alsa-lib it'd be an
inconvenient ABI to adopt for most of them.

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 498 bytes --]

On Tue, Aug 21, 2012 at 07:39:55AM -0500, Clark, Rob wrote:

> Does ASoC support 'hotplug' of audio devices?  If so, maybe it makes
> some sense to have some support in drm core.  At least all the edid
> parsing stuff to determine if the display supports audio should be
> generic and not driver specific.

Not really (except on a card level), and it'd probably confuse most
userspaces.  What I'd expect to happen currently is that the current
state of the connector would get reported via extcon.

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Ricardo Neri]

On 08/21/2012 07:39 AM, Clark, Rob wrote:
> On Tue, Aug 21, 2012 at 1:01 AM, Tomi Valkeinen <tomi.valkeinen@ti.com> wrote:
>> On Mon, 2012-08-20 at 20:47 -0500, Ricardo Neri wrote:
>>> Hello!
>>>
>>> I have been working on prototypes for the ASoC OMAP HDMI audio driver to
>>> propagate events from the HDMI output (e.g., display getting
>>> enabled/disabled/suspended). This for the users of the driver to react
>>> to such events. For instance, if the display is disabled or disconected,
>>> audio could be stopped, rerouted or whatever other decision the user
>>> makes. This is needed because, if, for instance, the  HDMI IP goes off,
>>> audio will stall and the audio users will only see a "playback write
>>> error (DMA or IRQ trouble?)"
>>>
>>> In my prototypes I have used snd_soc_jack for this purpose and I have
>>> some questions:
>>>
>>> *I see snd_soc_jack is used mostly for headsets and microphones with
>>> actual external mechanical connections. Strictly, in my case I propagate
>>> events originated by the OMAP display driver (changes in the power
>>> state), and not from external events. Some of these events are generated
>>> from an actual HDMI cable connection/disconnection, though.
>>>
>>> *Maybe the event should be propagated by omapdss/omapdrm/drm and the
>>> entity in charge of the audio policy should listen those events instead.
>>>
>>> *I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it is
>>> feasible for an audio driver to report events from an AV output.
>>>
>>> I was wondering about how much sense does it make to you guys use a
>>> snd_soc_jack in this case?
>>
>> How does DRM handle audio? I made a quick grep, but I see the drm
>> drivers only enabling the audio in the HW, nothing else.
>
> I confess to not knowing too much about audio/alsa, but what does
> audio driver need from hdmi?  Just hotplug events?

At least for the case of the ASoC HDMI audio driver (but hopefully for 
other audio drivers as well), it needs to detect whether an HDMI output 
is available, if the display's current configuration supports audio 
(e.g., a 1080p display configured as VGA should be reported as not 
supporting audio). It also needs interfaces to 
configure/prepare/start/stop audio. Also, of course, needs to know if 
the display is off/on/suspended and when transitions occur. For OMAP, 
omapdss provide an interface for this functionality for ALSA or any 
other interested user.
>
>  From a quick look, it seems most of what the existing drm drivers are
> doing is detecting if the display supports audio, and then turning
> on/off the hw.. (and some infoframe stuff in some cases).

Yes, it seems to me that every driver makes its own audio 
implementation, mainly focused on configuration. I could not find any 
audio common interface so that users like ALSA can take advantage of.

Also, I could not see any ALSA driver using functionality provided by a 
drm driver.

Maybe the lack of audio support in drm is because the audio users should 
not talk to drm directly but to a lower level component (omapdrm, 
omapdss?). However, today there exists video technology supports audio 
as well, such as DisplayPort or HDMI. Could it make more sense now to 
provide audio support?

>
> Does ASoC support 'hotplug' of audio devices?  If so, maybe it makes
> some sense to have some support in drm core.  At least all the edid
> parsing stuff to determine if the display supports audio should be
> generic and not driver specific.

drm already supports generic edid parsing to check for audio support. 
This is a good example of how an audio driver could just use that 
functionality to probe audio support.

Ricardo
>
> BR,
> -R
>
>> If there's a common generic way to handle this, we should obviously use
>> that. But if we need to choose between doing something custom or doing
>> it in omapdrm driver, I think we should go for drm the only solution and
>> forget about audio with omapfb.
>>
>>   Tomi
>>

Re: [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Ricardo Neri]

On 08/21/2012 08:16 AM, Mark Brown wrote:
> On Tue, Aug 21, 2012 at 02:30:51PM +0200, David Henningsson wrote:
>> On 08/21/2012 02:05 PM, Mark Brown wrote:
>
>>>   - sound/core/ctljack.c which was added later and provides separate
>>>     in-kernel and userspace APIs and is currently only used by HDA.
>
>> This is wrong. PulseAudio uses the new ctljack API. I started out
>> with an implementation which used the input device API
>> (sound/core/jack.c), but since some people did not like this API,
>> the ctljack API was designed and implemented for PA to use, which it
>> does (since PulseAudio 2.0 - the input device API implementation in
>> PulseAudio was never merged upstream).
>
> ...and the only thing using this API to generate events is HDA which is
> what I was talking about here given that this is a question from a
> driver point of view.
>
>>> What I'd like to see happening is that we merge ctljack into jack (since
>>> only HDA is going to be affected by that change it seems like the right
>>> direction to make the merge) and also add extcon support, I have looked
>>> at the extcon support.
>
>> I don't know either why we have two different interfaces for jack
>> detection (not counting extcon) for driver writers. I think we
>> should merge jack and ctljack, as long as that does not mean you
>> break anything I'm relying on in ctljack.
>
>> Maybe something for discussion at Plumbers?
>
> We discussed this last time we all met and came to the above conclusion :/
>
>>> Short term for drivers used on embedded systems I'd have to recommend
>>> extcon rather than anything ALSA-specific.
>
>> I thought that when Takashi did the new ctljack interface, that
>> would effectively deprecate the old input device interface, and that
>> ctljack was the new recommendation.
>
> Android currently uses the out of tree version of the extcon ABI (like I
> say it's where the code originated), and some OSs do use the input ABI
> also but realistically if you've got to pick one Android is where the
> market is at.

So, it seems that the way to go is extcon. I guess that ALSA will use 
extcon just like today snd_jack uses the input driver. is this correct? 
Is there any chance that ctrljack will propagate the events through 
extcon? Is there any early implementation that I could look at? I am 
asking to know how feasible is to use ctljack today and be compatible 
with extcon in the future.

Thanks!

Ricardo

>
> To my knowledge no embedded OS is using the control jacks, it's possible
> that there's something using them as a function of using PulseAudio but
> the ones I'm famililar with currently ignore PulseAudio routing and just
> use the mixing facilities.  Given the dependency on alsa-lib it'd be an
> inconvenient ABI to adopt for most of them.
>

Re: [alsa-devel] [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Takashi Iwai]

At Tue, 21 Aug 2012 19:58:02 -0500,
Ricardo Neri wrote:
> 
> 
> 
> On 08/21/2012 07:39 AM, Clark, Rob wrote:
> > On Tue, Aug 21, 2012 at 1:01 AM, Tomi Valkeinen <tomi.valkeinen@ti.com> wrote:
> >> On Mon, 2012-08-20 at 20:47 -0500, Ricardo Neri wrote:
> >>> Hello!
> >>>
> >>> I have been working on prototypes for the ASoC OMAP HDMI audio driver to
> >>> propagate events from the HDMI output (e.g., display getting
> >>> enabled/disabled/suspended). This for the users of the driver to react
> >>> to such events. For instance, if the display is disabled or disconected,
> >>> audio could be stopped, rerouted or whatever other decision the user
> >>> makes. This is needed because, if, for instance, the  HDMI IP goes off,
> >>> audio will stall and the audio users will only see a "playback write
> >>> error (DMA or IRQ trouble?)"
> >>>
> >>> In my prototypes I have used snd_soc_jack for this purpose and I have
> >>> some questions:
> >>>
> >>> *I see snd_soc_jack is used mostly for headsets and microphones with
> >>> actual external mechanical connections. Strictly, in my case I propagate
> >>> events originated by the OMAP display driver (changes in the power
> >>> state), and not from external events. Some of these events are generated
> >>> from an actual HDMI cable connection/disconnection, though.
> >>>
> >>> *Maybe the event should be propagated by omapdss/omapdrm/drm and the
> >>> entity in charge of the audio policy should listen those events instead.
> >>>
> >>> *I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it is
> >>> feasible for an audio driver to report events from an AV output.
> >>>
> >>> I was wondering about how much sense does it make to you guys use a
> >>> snd_soc_jack in this case?
> >>
> >> How does DRM handle audio? I made a quick grep, but I see the drm
> >> drivers only enabling the audio in the HW, nothing else.
> >
> > I confess to not knowing too much about audio/alsa, but what does
> > audio driver need from hdmi?  Just hotplug events?
> 
> At least for the case of the ASoC HDMI audio driver (but hopefully for 
> other audio drivers as well), it needs to detect whether an HDMI output 
> is available, if the display's current configuration supports audio 
> (e.g., a 1080p display configured as VGA should be reported as not 
> supporting audio). It also needs interfaces to 
> configure/prepare/start/stop audio. Also, of course, needs to know if 
> the display is off/on/suspended and when transitions occur. For OMAP, 
> omapdss provide an interface for this functionality for ALSA or any 
> other interested user.
> >
> >  From a quick look, it seems most of what the existing drm drivers are
> > doing is detecting if the display supports audio, and then turning
> > on/off the hw.. (and some infoframe stuff in some cases).
> 
> Yes, it seems to me that every driver makes its own audio 
> implementation, mainly focused on configuration. I could not find any 
> audio common interface so that users like ALSA can take advantage of.
> 
> Also, I could not see any ALSA driver using functionality provided by a 
> drm driver.
> 
> Maybe the lack of audio support in drm is because the audio users should 
> not talk to drm directly but to a lower level component (omapdrm, 
> omapdss?). However, today there exists video technology supports audio 
> as well, such as DisplayPort or HDMI. Could it make more sense now to 
> provide audio support?

The reason is that the audio and video handling are already separated
in the hardware level, at least, for desktop graphics.

The audio infoframe is passed via ELD to the audio controller part
upon plug/unplugging via HD-audio unsolicited event, and the audio
driver sets up the stuff according to the given ELD.  Thus no extra
interface between drm and ALSA was required in the kernel API level,
so far.


Takashi

Re: [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 675 bytes --]

On Tue, Aug 21, 2012 at 08:24:45PM -0500, Ricardo Neri wrote:

> So, it seems that the way to go is extcon. I guess that ALSA will
> use extcon just like today snd_jack uses the input driver. is this
> correct? Is there any chance that ctrljack will propagate the events
> through extcon? Is there any early implementation that I could look
> at? I am asking to know how feasible is to use ctljack today and be
> compatible with extcon in the future.

The ctljack API doesn't really map onto the other jack APIs at the
minute since the in kernel representation is that it exposes the
booleans directly to the callers rather than grouping them together into
the overall jack.

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Ricardo Neri]

Hi Takashi,

On 08/22/2012 02:55 AM, Takashi Iwai wrote:
> At Tue, 21 Aug 2012 19:58:02 -0500,
> Ricardo Neri wrote:
>>
>>
>>
>> On 08/21/2012 07:39 AM, Clark, Rob wrote:
>>> On Tue, Aug 21, 2012 at 1:01 AM, Tomi Valkeinen <tomi.valkeinen@ti.com> wrote:
>>>> On Mon, 2012-08-20 at 20:47 -0500, Ricardo Neri wrote:
>>>>> Hello!
>>>>>
>>>>> I have been working on prototypes for the ASoC OMAP HDMI audio driver to
>>>>> propagate events from the HDMI output (e.g., display getting
>>>>> enabled/disabled/suspended). This for the users of the driver to react
>>>>> to such events. For instance, if the display is disabled or disconected,
>>>>> audio could be stopped, rerouted or whatever other decision the user
>>>>> makes. This is needed because, if, for instance, the  HDMI IP goes off,
>>>>> audio will stall and the audio users will only see a "playback write
>>>>> error (DMA or IRQ trouble?)"
>>>>>
>>>>> In my prototypes I have used snd_soc_jack for this purpose and I have
>>>>> some questions:
>>>>>
>>>>> *I see snd_soc_jack is used mostly for headsets and microphones with
>>>>> actual external mechanical connections. Strictly, in my case I propagate
>>>>> events originated by the OMAP display driver (changes in the power
>>>>> state), and not from external events. Some of these events are generated
>>>>> from an actual HDMI cable connection/disconnection, though.
>>>>>
>>>>> *Maybe the event should be propagated by omapdss/omapdrm/drm and the
>>>>> entity in charge of the audio policy should listen those events instead.
>>>>>
>>>>> *I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it ie · · Il y a 12 minutes ·

s
>>>>> feasible for an audio driver to report events from an AV output.
>>>>>
>>>>> I was wondering about how much sense does it make to you guys use a
>>>>> snd_soc_jack in this case?
>>>>
>>>> How does DRM handle audio? I made a quick grep, but I see the drm
>>>> drivers only enabling the audio in the HW, nothing else.
>>>
>>> I confess to not knowing too much about audio/alsa, but what does
>>> audio driver need from hdmi?  Just hotplug events?
>>
>> At least for the case of the ASoC HDMI audio driver (but hopefully for
>> other audio drivers as well), it needs to detect whether an HDMI output
>> is available, if the display's current configuration supports audio
>> (e.g., a 1080p display configured as VGA should be reported as not
>> supporting audio). It also needs interfaces to
>> configure/prepare/start/stop audio. Also, of course, needs to know if
>> the display is off/on/suspended and when transitions occur. For OMAP,
>> omapdss provide an interface for this functionality for ALSA or any
>> other interested user.
>>>
>>>   From a quick look, it seems most of what the existing drm drivers are
>>> doing is detecting if the display supports audio, and then turning
>>> on/off the hw.. (and some infoframe stuff in some cases).
>>
>> Yes, it seems to me that every driver makes its own audio
>> implementation, mainly focused on configuration. I could not find any
>> audio common interface so that users like ALSA can take advantage of.
>>
>> Also, I could not see any ALSA driver using functionality provided by a
>> drm driver.
>>
>> Maybe the lack of audio support in drm is because the audio users should
>> not talk to drm directly but to a lower level component (omapdrm,
>> omapdss?). However, today there exists video technology supports audio
>> as well, such as DisplayPort or HDMI. Could it make more sense now to
>> provide audio support?
>
> The reason is that the audio and video handling are already separated
> in the hardware level, at least, for desktop graphics.

Separated in what sense? Do they have separate register banks in 
independent power domains? Can audio an video work with complete 
independence. What happens if someone decides to power off video. Is the 
audio able to continue if required?
>
> The audio infoframe is passed via ELD to the audio controller part
> upon plug/unplugging via HD-audio unsolicited event, and the audio
> driver sets up the stuff according to the given ELD.  Thus no extra
> interface between drm and ALSA was required in the kernel API level,
> so far.

I see that the unsolicited event is used only to parse the EDID, 
correct? It also notifies about the jack status. Hence, if there the 
cable is disconnected the application will know and act accordingly. Is 
this understanding correct?

Also, I see that hda_pcm_ops does not have a trigger or start/stop 
functions, how does it know when to start/stop audio. Maybe with 
azx_pcm_trigger?

Sorry, I am not expert in HD-audio :)

Ricardo

>
>
> Takashi
>

Re: [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Stephen Warren]

On 08/23/2012 07:44 PM, Ricardo Neri wrote:
> On 08/22/2012 02:55 AM, Takashi Iwai wrote:
>> At Tue, 21 Aug 2012 19:58:02 -0500,
>> Ricardo Neri wrote:
...
>>> Maybe the lack of audio support in drm is because the audio users should
>>> not talk to drm directly but to a lower level component (omapdrm,
>>> omapdss?). However, today there exists video technology supports audio
>>> as well, such as DisplayPort or HDMI. Could it make more sense now to
>>> provide audio support?
>>
>> The reason is that the audio and video handling are already separated
>> in the hardware level, at least, for desktop graphics.
> 
> Separated in what sense? Do they have separate register banks in

For NVIDIA desktop GPUs, this is certainly true, and I think so for any
Intel Azalia/HDA controller. The separate register banks are in
different PCI functions on the chip. The Intel Azalia/HDA spec also
architects specific ways that the audio and video parts interact (i.e.
ELD representation of EDID data, unsolicited response messages when the
video state changes, etc.) Oh, I see Takashi mentioned this below.

> independent power domains?

Most likely yes.

> Can audio an video work with complete
> independence. What happens if someone decides to power off video. Is the
> audio able to continue if required?

I believe audio DMA isn't affect by the video state, but I'm not 100%
sure of that.

>> The audio infoframe is passed via ELD to the audio controller part
>> upon plug/unplugging via HD-audio unsolicited event, and the audio
>> driver sets up the stuff according to the given ELD.  Thus no extra
>> interface between drm and ALSA was required in the kernel API level,
>> so far.
> 
> I see that the unsolicited event is used only to parse the EDID,
> correct? It also notifies about the jack status. Hence, if there the
> cable is disconnected the application will know and act accordingly. Is
> this understanding correct?

The kernel will know, and then passes the information on to user-space.

Re: [alsa-devel] [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Takashi Iwai]

At Thu, 23 Aug 2012 20:44:32 -0500,
Ricardo Neri wrote:
> 
> Hi Takashi,
> 
> On 08/22/2012 02:55 AM, Takashi Iwai wrote:
> > At Tue, 21 Aug 2012 19:58:02 -0500,
> > Ricardo Neri wrote:
> >>
> >>
> >>
> >> On 08/21/2012 07:39 AM, Clark, Rob wrote:
> >>> On Tue, Aug 21, 2012 at 1:01 AM, Tomi Valkeinen <tomi.valkeinen@ti.com> wrote:
> >>>> On Mon, 2012-08-20 at 20:47 -0500, Ricardo Neri wrote:
> >>>>> Hello!
> >>>>>
> >>>>> I have been working on prototypes for the ASoC OMAP HDMI audio driver to
> >>>>> propagate events from the HDMI output (e.g., display getting
> >>>>> enabled/disabled/suspended). This for the users of the driver to react
> >>>>> to such events. For instance, if the display is disabled or disconected,
> >>>>> audio could be stopped, rerouted or whatever other decision the user
> >>>>> makes. This is needed because, if, for instance, the  HDMI IP goes off,
> >>>>> audio will stall and the audio users will only see a "playback write
> >>>>> error (DMA or IRQ trouble?)"
> >>>>>
> >>>>> In my prototypes I have used snd_soc_jack for this purpose and I have
> >>>>> some questions:
> >>>>>
> >>>>> *I see snd_soc_jack is used mostly for headsets and microphones with
> >>>>> actual external mechanical connections. Strictly, in my case I propagate
> >>>>> events originated by the OMAP display driver (changes in the power
> >>>>> state), and not from external events. Some of these events are generated
> >>>>> from an actual HDMI cable connection/disconnection, though.
> >>>>>
> >>>>> *Maybe the event should be propagated by omapdss/omapdrm/drm and the
> >>>>> entity in charge of the audio policy should listen those events instead.
> >>>>>
> >>>>> *I do see SND_JACK_VIDEOOUT and SND_JACK_AVOUT types so maybe it ie · · Il y a 12 minutes ·
> 
> s
> >>>>> feasible for an audio driver to report events from an AV output.
> >>>>>
> >>>>> I was wondering about how much sense does it make to you guys use a
> >>>>> snd_soc_jack in this case?
> >>>>
> >>>> How does DRM handle audio? I made a quick grep, but I see the drm
> >>>> drivers only enabling the audio in the HW, nothing else.
> >>>
> >>> I confess to not knowing too much about audio/alsa, but what does
> >>> audio driver need from hdmi?  Just hotplug events?
> >>
> >> At least for the case of the ASoC HDMI audio driver (but hopefully for
> >> other audio drivers as well), it needs to detect whether an HDMI output
> >> is available, if the display's current configuration supports audio
> >> (e.g., a 1080p display configured as VGA should be reported as not
> >> supporting audio). It also needs interfaces to
> >> configure/prepare/start/stop audio. Also, of course, needs to know if
> >> the display is off/on/suspended and when transitions occur. For OMAP,
> >> omapdss provide an interface for this functionality for ALSA or any
> >> other interested user.
> >>>
> >>>   From a quick look, it seems most of what the existing drm drivers are
> >>> doing is detecting if the display supports audio, and then turning
> >>> on/off the hw.. (and some infoframe stuff in some cases).
> >>
> >> Yes, it seems to me that every driver makes its own audio
> >> implementation, mainly focused on configuration. I could not find any
> >> audio common interface so that users like ALSA can take advantage of.
> >>
> >> Also, I could not see any ALSA driver using functionality provided by a
> >> drm driver.
> >>
> >> Maybe the lack of audio support in drm is because the audio users should
> >> not talk to drm directly but to a lower level component (omapdrm,
> >> omapdss?). However, today there exists video technology supports audio
> >> as well, such as DisplayPort or HDMI. Could it make more sense now to
> >> provide audio support?
> >
> > The reason is that the audio and video handling are already separated
> > in the hardware level, at least, for desktop graphics.
> 
> Separated in what sense? Do they have separate register banks in 
> independent power domains? Can audio an video work with complete 
> independence. What happens if someone decides to power off video. Is the 
> audio able to continue if required?

As Stephen already pointed, the functionality is independent in the
PCI hardware level.  The audio part even accepts and DMA is working
without connection to the actual output.

> > The audio infoframe is passed via ELD to the audio controller part
> > upon plug/unplugging via HD-audio unsolicited event, and the audio
> > driver sets up the stuff according to the given ELD.  Thus no extra
> > interface between drm and ALSA was required in the kernel API level,
> > so far.
> 
> I see that the unsolicited event is used only to parse the EDID, 
> correct? It also notifies about the jack status. Hence, if there the 
> cable is disconnected the application will know and act accordingly. Is 
> this understanding correct?

Correct.

> Also, I see that hda_pcm_ops does not have a trigger or start/stop 
> functions, how does it know when to start/stop audio. Maybe with 
> azx_pcm_trigger?

The start/stop isn't issued in the driver level because DMA is still
working.  For HD-audio, it's really just like a normal jack
plug/unplug.


Takashi
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [alsa-devel] [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Arun Raghavan]

On Tue, 2012-08-21 at 13:05 +0100, Mark Brown wrote:
> On Tue, Aug 21, 2012 at 07:28:34AM +0200, Takashi Iwai wrote:
> > Ricardo Neri wrote:
> 
> > > I was wondering about how much sense does it make to you guys use a 
> > > snd_soc_jack in this case?
> 
> > HD-audio already uses the generic jack event for the HDMI/DP
> > connection change notification as well, so I think it would make sense
> > in general.
> 
> The whole problem here is that we don't *have* a generic jack interface.
> We've got:
> 
>  - sound/core/jack.c which was written to be a generic API and is used
>    by everything that does jack support currently.
> 
>  - sound/core/ctljack.c which was added later and provides separate
>    in-kernel and userspace APIs and is currently only used by HDA.
> 
>  - extcon which does have a good reason to be a separate API since that
>    it's not audio specific (and is likely to be picked up by Android as
>    the code was originally taken from there); it's currently not
>    supported by the frameworks in ALSA.  I'd suggest Pulse should be using
>    it too.
> 
> This is a complete shambles for both driver authors and userspace, the
> ABI varies randomly with drivers and in theory driver authors have to
> implement everything three times which is just nuts.
> 
> What I'd like to see happening is that we merge ctljack into jack (since
> only HDA is going to be affected by that change it seems like the right
> direction to make the merge) and also add extcon support, I have looked
> at the extcon support.
> 
> Short term for drivers used on embedded systems I'd have to recommend
> extcon rather than anything ALSA-specific.

It's been in shambles for an absurdly long time. Would be good to
actually try to tackle it again at Plumbers or sth.

-- Arun

Re: [RFC] ASoC: snd_soc_jack for HDMI audio: does it make sense?

[Mark Brown]

On Fri, Aug 24, 2012 at 12:40:37PM +0530, Arun Raghavan wrote:

> It's been in shambles for an absurdly long time. Would be good to
> actually try to tackle it again at Plumbers or sth.

I think all this really needs is someone with the time to do the work -
it's more a coding problem with unifying the in kernel API than an issue
with working out what to do.