alsa-devel.alsa-project.org archive mirror
 help / color / mirror / Atom feed
* More Generic Audio Graph Sound Card idea
@ 2020-08-21  4:15 Kuninori Morimoto
  2020-08-21  5:26 ` Sameer Pujar
                   ` (4 more replies)
  0 siblings, 5 replies; 31+ messages in thread
From: Kuninori Morimoto @ 2020-08-21  4:15 UTC (permalink / raw)
  To: Mark Brown; +Cc: Sameer Pujar, Pierre-Louis Bossart, Linux-ALSA


Hi Mark
Cc Pierre-Louis, Sameer

Current audio-graph-card driver has DPCM support,
but it is limited (= Mix/Mux/TDM-split/rate-convert/channel-convert).
It was expanded forcibly expanded.

Because of it, the connection judgement for normal vs DPCM is tricky.
I know Pierre-Louis want to use it for SOF, but something is missing,
thus, can't use (?).
I know now Sameer is currently trying to expand it now.
It is OK for me, but, more expansion in the future will be difficult/buggy.
For example supporting many *generic* DPCM and normal connection in the
same time will be difficult.

I'm thinking below style as new audio-graph-card2.
It is not tricky, thus connection judgement for normal vs DSP is easy.
Then, code can be more readable (= no guaranty :P) ?

I guss "audio-graph-card" and "audio-graph-card2" can have some (= not full)
compatible for normal connection. But has no compatible for DPCM connection.

I think I need to keep as-is for current audio-graph-card,
and create new audio-graph-card2 driver.
Because there is users, and card2 implementation will have huge different.
Keeping compatible and update will be very difficult or buggy.

But what do you think ?

Note 1 is that this is just idea for now.
I'm very happy to implement it, but it will be after that
my all remaining patch was posted/accepted.

Note 2 is that I'm not full DPCM analyst.
Thus, maybe audio-graph-card2 can solve current connection issue,
but might can't for unknown connection case.
In such came we need ver3 (?).

  ------------ image ---------------------------

  | Front End PCMs    |  SoC DSP  | Back End DAIs | Audio devices |
  
                      *************
  PCM0 <------------> *           * <----DAI0-----> Codec Headset
                      *           *
  PCM1 <------------> *           * <----DAI1-----> Codec Speakers
                      *   DSP     *
  PCM2 <------------> *           * <----DAI2-----> MODEM
                      *           *
  PCM3 <------------> *           * <----DAI3-----> BT
                      *           *
                      *           * <----DAI4-----> DMIC
                      *           *
                      *           * <----DAI5-----> FM
                      *************
  ------------ DT sample ---------------------------

	sound {
		compatible = "audio-graph-card2";

		dais = <&pcm0,
			&pcm1,
			&pcm2,
			&pcm3>;
	};

	front-end {
		ports {
			pcm0: port@0 { pcm0_endpoint: endpoint { remote-endpoint = <&dsp_f0>; }; };
			pcm1: port@1 { pcm1_endpoint: endpoint { remote-endpoint = <&dsp_f1>; }; };
			pcm2: port@2 { pcm2_endpoint: endpoint { remote-endpoint = <&dsp_f2>; }; };
			pcm3: port@3 { pcm3_endpoint: endpoint { remote-endpoint = <&dsp_f3>; }; };
		};
	};

	dsp {
		compatible = "audio-graph-card2-dsp";

		ports {
			/* Front End side */
			port@0 { dsp_f0: endpoint { remote-endpoint = <&pcm0>; }; };
			port@1 { dsp_f1: endpoint { remote-endpoint = <&pcm1>; }; };
			port@2 { dsp_f2: endpoint { remote-endpoint = <&pcm2>; }; };
			port@3 { dsp_f3: endpoint { remote-endpoint = <&pcm3>; }; };

			/* Back End side */
			port@4 { dsp_b0: endpoint { remote-endpoint = <&dai0>; }; };
			port@5 { dsp_b1: endpoint { remote-endpoint = <&dai1>; }; };
			port@6 { dsp_b2: endpoint { remote-endpoint = <&dai2>; }; };
			port@7 { dsp_b3: endpoint { remote-endpoint = <&dai3>; }; };
			port@8 { dsp_b4: endpoint { remote-endpoint = <&dai4>; }; };
			port@9 { dsp_b5: endpoint { remote-endpoint = <&dai5>; }; };
		};
	};

	back-end {
		ports {
			port@0 { dai0: endpoint { remote-endpoint = <&dsp_b0>; }; };
			port@1 { dai1: endpoint { remote-endpoint = <&dsp_b1>; }; };
			port@2 { dai2: endpoint { remote-endpoint = <&dsp_b2>; }; };
			port@3 { dai3: endpoint { remote-endpoint = <&dsp_b3>; }; };
			port@4 { dai4: endpoint { remote-endpoint = <&dsp_b4>; }; };
			port@5 { dai5: endpoint { remote-endpoint = <&dsp_b5>; }; };
		};
	};


^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-21  4:15 More Generic Audio Graph Sound Card idea Kuninori Morimoto
@ 2020-08-21  5:26 ` Sameer Pujar
  2020-08-21  7:14   ` Kuninori Morimoto
  2020-08-21  7:11 ` Daniel Baluta
                   ` (3 subsequent siblings)
  4 siblings, 1 reply; 31+ messages in thread
From: Sameer Pujar @ 2020-08-21  5:26 UTC (permalink / raw)
  To: Kuninori Morimoto, Mark Brown; +Cc: Linux-ALSA, Pierre-Louis Bossart, jonathanh


Hi Kuninori,

CC Jon

Some of my thoughts on below.

On 8/21/2020 9:45 AM, Kuninori Morimoto wrote:
> External email: Use caution opening links or attachments
>
>
> Hi Mark
> Cc Pierre-Louis, Sameer
>
> Current audio-graph-card driver has DPCM support,
> but it is limited (= Mix/Mux/TDM-split/rate-convert/channel-convert).
> It was expanded forcibly expanded.

Yes it appears to be so. Right now it does not seem to be supporting 
'demux' kind of use case.

>
> Because of it, the connection judgement for normal vs DPCM is tricky.

Please note that asoc_simple_init_dai_link_params() makes the link 
codec-to-codec if any component involved has 'non_legacy_dai_naming' 
flag enabled. It is used by both audio-graph and simple-card. So it is a 
combination of three (DPCM, normal and codec-to-codec). To avoid all 
this complication, I am treating all links as DPCM in the series [0] I sent.

> I know Pierre-Louis want to use it for SOF, but something is missing,
> thus, can't use (?).

> I know now Sameer is currently trying to expand it now.
> It is OK for me, but, more expansion in the future will be difficult/buggy.
> For example supporting many *generic* DPCM and normal connection in the
> same time will be difficult.

If we encode use case information in DT, it would become regid and is 
not flexible when the HW is giving us the all flexibility (at least in 
my case). Thus may lead to complications. If there is a way to configure 
PCM parameters from the user space it would help to simplify things. 
Then DT can just describe the HW links.

>
> I'm thinking below style as new audio-graph-card2.
> It is not tricky, thus connection judgement for normal vs DSP is easy.
> Then, code can be more readable (= no guaranty :P) ?
>
> I guss "audio-graph-card" and "audio-graph-card2" can have some (= not full)
> compatible for normal connection. But has no compatible for DPCM connection.
>
> I think I need to keep as-is for current audio-graph-card,
> and create new audio-graph-card2 driver.
> Because there is users, and card2 implementation will have huge different.
> Keeping compatible and update will be very difficult or buggy.
>
> But what do you think ?
>
> Note 1 is that this is just idea for now.
> I'm very happy to implement it, but it will be after that
> my all remaining patch was posted/accepted.
>
> Note 2 is that I'm not full DPCM analyst.
> Thus, maybe audio-graph-card2 can solve current connection issue,
> but might can't for unknown connection case.
> In such came we need ver3 (?).

>    ------------ image ---------------------------
>
>    | Front End PCMs    |  SoC DSP  | Back End DAIs | Audio devices |
>
>                        *************
>    PCM0 <------------> *           * <----DAI0-----> Codec Headset
>                        *           *
>    PCM1 <------------> *           * <----DAI1-----> Codec Speakers
>                        *   DSP     *
>    PCM2 <------------> *           * <----DAI2-----> MODEM
>                        *           *
>    PCM3 <------------> *           * <----DAI3-----> BT
>                        *           *
>                        *           * <----DAI4-----> DMIC
>                        *           *
>                        *           * <----DAI5-----> FM
>                        *************
>    ------------ DT sample ---------------------------
>
>          sound {
>                  compatible = "audio-graph-card2";
>
>                  dais = <&pcm0,
>                          &pcm1,
>                          &pcm2,
>                          &pcm3>;
>          };
>
>          front-end {
>                  ports {
>                          pcm0: port@0 { pcm0_endpoint: endpoint { remote-endpoint = <&dsp_f0>; }; };
>                          pcm1: port@1 { pcm1_endpoint: endpoint { remote-endpoint = <&dsp_f1>; }; };
>                          pcm2: port@2 { pcm2_endpoint: endpoint { remote-endpoint = <&dsp_f2>; }; };
>                          pcm3: port@3 { pcm3_endpoint: endpoint { remote-endpoint = <&dsp_f3>; }; };
>                  };
>          };
>
>          dsp {
>                  compatible = "audio-graph-card2-dsp";
>
>                  ports {
>                          /* Front End side */
>                          port@0 { dsp_f0: endpoint { remote-endpoint = <&pcm0>; }; };
>                          port@1 { dsp_f1: endpoint { remote-endpoint = <&pcm1>; }; };
>                          port@2 { dsp_f2: endpoint { remote-endpoint = <&pcm2>; }; };
>                          port@3 { dsp_f3: endpoint { remote-endpoint = <&pcm3>; }; };
>
>                          /* Back End side */
>                          port@4 { dsp_b0: endpoint { remote-endpoint = <&dai0>; }; };
>                          port@5 { dsp_b1: endpoint { remote-endpoint = <&dai1>; }; };
>                          port@6 { dsp_b2: endpoint { remote-endpoint = <&dai2>; }; };
>                          port@7 { dsp_b3: endpoint { remote-endpoint = <&dai3>; }; };
>                          port@8 { dsp_b4: endpoint { remote-endpoint = <&dai4>; }; };
>                          port@9 { dsp_b5: endpoint { remote-endpoint = <&dai5>; }; };
>                  };
>          };
>
>          back-end {
>                  ports {
>                          port@0 { dai0: endpoint { remote-endpoint = <&dsp_b0>; }; };
>                          port@1 { dai1: endpoint { remote-endpoint = <&dsp_b1>; }; };
>                          port@2 { dai2: endpoint { remote-endpoint = <&dsp_b2>; }; };
>                          port@3 { dai3: endpoint { remote-endpoint = <&dsp_b3>; }; };
>                          port@4 { dai4: endpoint { remote-endpoint = <&dsp_b4>; }; };
>                          port@5 { dai5: endpoint { remote-endpoint = <&dsp_b5>; }; };
>                  };
>          };

In my series [0], I am trying to achieve the similar as above though the 
description may look little different in my case, but that is what HW 
describes I believe.

[0] https://lkml.org/lkml/2020/8/5/42


^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-21  4:15 More Generic Audio Graph Sound Card idea Kuninori Morimoto
  2020-08-21  5:26 ` Sameer Pujar
@ 2020-08-21  7:11 ` Daniel Baluta
  2020-08-21  7:25   ` Kuninori Morimoto
  2020-08-21 12:18 ` Mark Brown
                   ` (2 subsequent siblings)
  4 siblings, 1 reply; 31+ messages in thread
From: Daniel Baluta @ 2020-08-21  7:11 UTC (permalink / raw)
  To: Kuninori Morimoto
  Cc: Sameer Pujar, Mark Brown, Pierre-Louis Bossart, Linux-ALSA

On Fri, Aug 21, 2020 at 7:16 AM Kuninori Morimoto
<kuninori.morimoto.gx@renesas.com> wrote:
>
>
> Hi Mark
> Cc Pierre-Louis, Sameer
>
> Current audio-graph-card driver has DPCM support,
> but it is limited (= Mix/Mux/TDM-split/rate-convert/channel-convert).
> It was expanded forcibly expanded.
>
> Because of it, the connection judgement for normal vs DPCM is tricky.
> I know Pierre-Louis want to use it for SOF, but something is missing,
> thus, can't use (?).

Hi Morimoto-san,

Thanks for having a look at this. I also tried to use generic machine
driver (simple-card)
with SOF but I had some trouble creating desired DPCM links.

Main limitation of simple-card is that there is no direct way of
specifying DPCM links.

I had an attempt to introduce a more flexible approach for that here:
https://lkml.org/lkml/2019/10/13/123

but obviously we can do better than that.

For now, I'm using only normal links with simple-audio card and it
works fine for
me with SOF, but in the future I think we might need to expand that.

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-21  5:26 ` Sameer Pujar
@ 2020-08-21  7:14   ` Kuninori Morimoto
  2020-08-21  8:28     ` Sameer Pujar
  2020-09-25  1:43     ` Kuninori Morimoto
  0 siblings, 2 replies; 31+ messages in thread
From: Kuninori Morimoto @ 2020-08-21  7:14 UTC (permalink / raw)
  To: Sameer Pujar; +Cc: Linux-ALSA, Mark Brown, Pierre-Louis Bossart, jonathanh


Hi Sameer
Cc Mark

Thank you for your feedback

> > Current audio-graph-card driver has DPCM support,
> > but it is limited (= Mix/Mux/TDM-split/rate-convert/channel-convert).
> > It was expanded forcibly expanded.
> 
> Yes it appears to be so. Right now it does not seem to be supporting
> 'demux' kind of use case.

I'm not sure how to do demux.
By DPCM ? by multi-Codec ?
I'm happy to consider it if there was sample code.

> Please note that asoc_simple_init_dai_link_params() makes the link
> codec-to-codec if any component involved has 'non_legacy_dai_naming'
> flag enabled. It is used by both audio-graph and simple-card. So it is
> a combination of three (DPCM, normal and codec-to-codec). To avoid all
> this complication, I am treating all links as DPCM in the series [0] I
> sent.

Ahh, we want to consider codec-to-codec..
And I noticed multi-CPU / multi-Codec case too (= for normal connection).
Do we have sample code ? > Mark

> If we encode use case information in DT, it would become regid and is
> not flexible when the HW is giving us the all flexibility (at least in
> my case). Thus may lead to complications. If there is a way to
> configure PCM parameters from the user space it would help to simplify
> things. Then DT can just describe the HW links.

What kind of PCM parameters you want to get from user-space ?



Mark, this is question from me.
In my understanding,
CPU - Codec are merging each parameter automatically at runtime,
for example rates / channels / formats (= snd_soc_runtime_calc_hw()).
And system-clock / TDM-slot / DAI-format are specified via Sound Card.

I can understand that system-clock is specified via Sound Card,
but TDM-slot / DAI-format can be adjusted from CPU / Codec driver
automatically same as rates / channels / formats ?
I'm not sure it is possible, or not, but I've been wondering for a long time.

Thank you for your help !!

Best regards
---
Kuninori Morimoto

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-21  7:11 ` Daniel Baluta
@ 2020-08-21  7:25   ` Kuninori Morimoto
  2020-08-21  7:33     ` Daniel Baluta
  2020-08-21 11:47     ` Mark Brown
  0 siblings, 2 replies; 31+ messages in thread
From: Kuninori Morimoto @ 2020-08-21  7:25 UTC (permalink / raw)
  To: Daniel Baluta; +Cc: Sameer Pujar, Mark Brown, Pierre-Louis Bossart, Linux-ALSA


Hi Daniel

Thank you for your feedback

> > Because of it, the connection judgement for normal vs DPCM is tricky.
> > I know Pierre-Louis want to use it for SOF, but something is missing,
> > thus, can't use (?).
(snip)
> Thanks for having a look at this. I also tried to use generic machine
> driver (simple-card)
> with SOF but I had some trouble creating desired DPCM links.
> 
> Main limitation of simple-card is that there is no direct way of
> specifying DPCM links.
> 
> I had an attempt to introduce a more flexible approach for that here:
> https://lkml.org/lkml/2019/10/13/123
> 
> but obviously we can do better than that.
> 
> For now, I'm using only normal links with simple-audio card and it
> works fine for
> me with SOF, but in the future I think we might need to expand that.

OK, the issue is also normal vs DPCM judgement.
Maybe having "dsp" can solve it, same as audio-graph-card ?
I'm not sure we need to update simple-audio, too > Mark ?

Thank you for your help !!

Best regards
---
Kuninori Morimoto

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-21  7:25   ` Kuninori Morimoto
@ 2020-08-21  7:33     ` Daniel Baluta
  2020-08-21 11:47     ` Mark Brown
  1 sibling, 0 replies; 31+ messages in thread
From: Daniel Baluta @ 2020-08-21  7:33 UTC (permalink / raw)
  To: Kuninori Morimoto
  Cc: Sameer Pujar, Mark Brown, Pierre-Louis Bossart, Linux-ALSA

On Fri, Aug 21, 2020 at 10:25 AM Kuninori Morimoto
<kuninori.morimoto.gx@renesas.com> wrote:
>
>
> Hi Daniel
>
> Thank you for your feedback
>
> > > Because of it, the connection judgement for normal vs DPCM is tricky.
> > > I know Pierre-Louis want to use it for SOF, but something is missing,
> > > thus, can't use (?).
> (snip)
> > Thanks for having a look at this. I also tried to use generic machine
> > driver (simple-card)
> > with SOF but I had some trouble creating desired DPCM links.
> >
> > Main limitation of simple-card is that there is no direct way of
> > specifying DPCM links.
> >
> > I had an attempt to introduce a more flexible approach for that here:
> > https://lkml.org/lkml/2019/10/13/123
> >
> > but obviously we can do better than that.
> >
> > For now, I'm using only normal links with simple-audio card and it
> > works fine for
> > me with SOF, but in the future I think we might need to expand that.
>
> OK, the issue is also normal vs DPCM judgement.
> Maybe having "dsp" can solve it, same as audio-graph-card ?
> I'm not sure we need to update simple-audio, too > Mark ?

Yes, the condition choosing normal vs dpcm is fairly similar
for simple-card and audio-graph-card:

if (dpcm_selectable &&
    ((of_get_child_count(codec_port) > 1) ||
     adata.convert_rate || adata.convert_channels)

Can you expand a little bit more about what having 'DSP' means?

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-21  7:14   ` Kuninori Morimoto
@ 2020-08-21  8:28     ` Sameer Pujar
  2020-08-21 13:02       ` Mark Brown
  2020-09-25  1:43     ` Kuninori Morimoto
  1 sibling, 1 reply; 31+ messages in thread
From: Sameer Pujar @ 2020-08-21  8:28 UTC (permalink / raw)
  To: Kuninori Morimoto; +Cc: Linux-ALSA, Mark Brown, Pierre-Louis Bossart, jonathanh

Hi Morimoto-san,

...

>> If we encode use case information in DT, it would become regid and is
>> not flexible when the HW is giving us the all flexibility (at least in
>> my case). Thus may lead to complications. If there is a way to
>> configure PCM parameters from the user space it would help to simplify
>> things. Then DT can just describe the HW links.
> What kind of PCM parameters you want to get from user-space ?

I was referring to channels, rate, sample size for PCM parameters which 
are part of hw_params(). Having these strictly defined in DT would limit 
from using the same audio path for different configurations. So far used 
mixer controls for overriding this info in hw_param(), but this idea is 
dropped as per Mark's suggestion earlier. The requirement here is, some 
components have the ability to alter these parameters. Subsequent 
components in the audio path should me made aware of this.

For example, SRC (sample rate converter) can change 'rate' info coming 
from hw_param(). Similarly 'mux/demux' can change channel info. Fixing 
one set of configuration in DT will limit the usage. If this is 
configurable from user space, then it would be lot more easier.

. . .

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-21  7:25   ` Kuninori Morimoto
  2020-08-21  7:33     ` Daniel Baluta
@ 2020-08-21 11:47     ` Mark Brown
  1 sibling, 0 replies; 31+ messages in thread
From: Mark Brown @ 2020-08-21 11:47 UTC (permalink / raw)
  To: Kuninori Morimoto
  Cc: Daniel Baluta, Sameer Pujar, Linux-ALSA, Pierre-Louis Bossart

[-- Attachment #1: Type: text/plain, Size: 253 bytes --]

On Fri, Aug 21, 2020 at 04:25:53PM +0900, Kuninori Morimoto wrote:

> I'm not sure we need to update simple-audio, too > Mark ?

I think it's fine to not update simple-audio-card for new stuff, new
things are supposed to be using the graph card anyway.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-21  4:15 More Generic Audio Graph Sound Card idea Kuninori Morimoto
  2020-08-21  5:26 ` Sameer Pujar
  2020-08-21  7:11 ` Daniel Baluta
@ 2020-08-21 12:18 ` Mark Brown
  2020-08-24  0:25   ` Kuninori Morimoto
  2020-08-21 15:49 ` Pierre-Louis Bossart
  2020-10-13  4:50 ` Kuninori Morimoto
  4 siblings, 1 reply; 31+ messages in thread
From: Mark Brown @ 2020-08-21 12:18 UTC (permalink / raw)
  To: Kuninori Morimoto; +Cc: Sameer Pujar, Pierre-Louis Bossart, Linux-ALSA

[-- Attachment #1: Type: text/plain, Size: 1517 bytes --]

On Fri, Aug 21, 2020 at 01:15:41PM +0900, Kuninori Morimoto wrote:

> I'm thinking below style as new audio-graph-card2.
> It is not tricky, thus connection judgement for normal vs DSP is easy.
> Then, code can be more readable (= no guaranty :P) ?

Having an audio-graph-card2 isn't ideal but may be required at least
during development :/  Ideally we'd be able to have the new driver parse
both binding formats (or rather, have the new binding format be new use
cases for the same binding format) and only use -card2 while it's in
development.

>   | Front End PCMs    |  SoC DSP  | Back End DAIs | Audio devices |
>   
>                       *************
>   PCM0 <------------> *           * <----DAI0-----> Codec Headset
>                       *           *
>   PCM1 <------------> *           * <----DAI1-----> Codec Speakers
>                       *   DSP     *
>   PCM2 <------------> *           * <----DAI2-----> MODEM
>                       *           *
>   PCM3 <------------> *           * <----DAI3-----> BT
>                       *           *
>                       *           * <----DAI4-----> DMIC
>                       *           *
>                       *           * <----DAI5-----> FM
>                       *************

I think your plan with this explicit representation of the DSP is where
we want to get to, the issue is mapping this onto DPCM as it currently
stands.  The binding you proposed underneath looked sensible on first
readthrough.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-21  8:28     ` Sameer Pujar
@ 2020-08-21 13:02       ` Mark Brown
  0 siblings, 0 replies; 31+ messages in thread
From: Mark Brown @ 2020-08-21 13:02 UTC (permalink / raw)
  To: Sameer Pujar
  Cc: Linux-ALSA, Pierre-Louis Bossart, Kuninori Morimoto, jonathanh

[-- Attachment #1: Type: text/plain, Size: 1096 bytes --]

On Fri, Aug 21, 2020 at 01:58:15PM +0530, Sameer Pujar wrote:

> For example, SRC (sample rate converter) can change 'rate' info coming from
> hw_param(). Similarly 'mux/demux' can change channel info. Fixing one set of
> configuration in DT will limit the usage. If this is configurable from user
> space, then it would be lot more easier.

We shouldn't be fixing the configuration in DT unless it's fixed in the
hardware, in which case we should be able to figure this out through
constraints at runtime anyway and not need to expose anything to
userspace.  This covers an awful lot of SRC and other conversion use
cases, they're often there because there's some fixed DAI that means
there's only one possible setting on one side of the convertor.  A lot
of the other cases where there's more flexibility can probably be
handled in a similar fashion with something other than the SRC itself
being configured - normally the goal is to fix a link in a particular
configuration through software policy and we can work back to setting up
the SRC in the same way as if it was a hardware constraint.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-21  4:15 More Generic Audio Graph Sound Card idea Kuninori Morimoto
                   ` (2 preceding siblings ...)
  2020-08-21 12:18 ` Mark Brown
@ 2020-08-21 15:49 ` Pierre-Louis Bossart
  2020-10-13  4:50 ` Kuninori Morimoto
  4 siblings, 0 replies; 31+ messages in thread
From: Pierre-Louis Bossart @ 2020-08-21 15:49 UTC (permalink / raw)
  To: Kuninori Morimoto, Mark Brown; +Cc: Sameer Pujar, Linux-ALSA

Hi Morimoto-san,

 > I know Pierre-Louis want to use it for SOF, but something is missing,
 > thus, can't use (?).

Here's a bit more background.

For SoundWire, we could have the following examples for amplifier 
connections, where in some cases we have two amplifiers located on 
separate links (top diagram) and in the second they are on the same link.

| FE PCMs    |  SoC DSP  | BE DAIs   | Audio devices |

              *************
   PCM0 <---> *           * DAI0 <---> Codec Headset
              *           *
   PCM1 <---> *           * DAI1 <---> Amp Left
              *   DSP     *
   PCM2 <---> *           * DAI2 <---> Amp Right
              *           *
   PCM3 <---> *           * DAI3 <---> BT
              *           *
              *           * DAI4 <---> DMIC
              *           *
              *           * DAI5 <---> FM
              *************

              *************
   PCM0 <---> *           * DAI0 <---> Codec Headset
              *           *
   PCM1 <---> *           * DAI1 <---> Amp Left
              *   DSP     *        |
   PCM2 <---> *           * DAI2/  --> Amp Right
              *           *
   PCM3 <---> *           * DAI3 <---> BT
              *           *
              *           * DAI4 <---> DMIC
              *           *
              *           * DAI5 <---> FM
              *************

We'd need a means to express that the two amplifiers are really supposed 
to operate concurrently and be synchronized, and let the drivers know 
which configuration to use (multi-cpu/multi-codec or single 
CPU/multi-codec).

Currently we do this manually by hard-coding an 'endpoint group', see 
e.g. 
https://elixir.bootlin.com/linux/latest/source/sound/soc/intel/common/soc-acpi-intel-cml-match.c#L69, 
and use this information in the machine driver to know which of the 
configurations to use.

At some point we'd like to read this information from platform firmware 
(will be ACPI for Intel but that's a detail) and create the dais in the 
right way by grouping endpoints on the same dailink when they are 
connected, and create separate dailink otherwise.

Note that we only want the connection between BEs and audio devices to 
be described in platform firmware, the topology inside the SOC DSP is 
handled with a topology file that can be changed at will to e.g. 
add/remove processing.

We really need to make sure the audio graph can be described in two 
parts, the bottom part related to hardware routing and layout, and the 
DSP part. Of course the topology part would have to be constrained to 
use the DAIs used in the lower level.

Hope this helps explain what I am looking for.

Thank you for starting this thread!

Regards

-Pierre

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-21 12:18 ` Mark Brown
@ 2020-08-24  0:25   ` Kuninori Morimoto
  2020-08-24  6:25     ` Sameer Pujar
  2020-09-03 23:51     ` Kuninori Morimoto
  0 siblings, 2 replies; 31+ messages in thread
From: Kuninori Morimoto @ 2020-08-24  0:25 UTC (permalink / raw)
  To: Mark Brown; +Cc: Sameer Pujar, Pierre-Louis Bossart, Linux-ALSA


Hi Mark
Cc Sameer

> Having an audio-graph-card2 isn't ideal but may be required at least
> during development :/  Ideally we'd be able to have the new driver parse
> both binding formats (or rather, have the new binding format be new use
> cases for the same binding format) and only use -card2 while it's in
> development.

If you want to update current audio-graph-card without creating new
audio-graph-card2, I'm OK about it. But need adjusting / agreement.

Current audio-graph-card "DSP" user is only me, and I'm using it only locally.
Thus upstream doesn't get damage if I removed "audio-graph-scu-card"
(= DSP use case) support.

OTOH, Sameer is posting patch for "-cc-" support. If it was accepted
and he used it on upstream (= on tegra),
keeping compatibility will be very difficult and/or code will be very confusable.

If Sameer can OK and wait new audio-graph-card, maybe we have no compatibility issue.
But in such case, 1st version of new audio-graph-card might be not enough for him.
Sameer need to waiting / testing / adjusting / etc, etc...


Thank you for your help !!

Best regards
---
Kuninori Morimoto

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-24  0:25   ` Kuninori Morimoto
@ 2020-08-24  6:25     ` Sameer Pujar
  2020-08-25  0:59       ` Kuninori Morimoto
  2020-09-03 23:51     ` Kuninori Morimoto
  1 sibling, 1 reply; 31+ messages in thread
From: Sameer Pujar @ 2020-08-24  6:25 UTC (permalink / raw)
  To: Kuninori Morimoto, Mark Brown; +Cc: Linux-ALSA, Pierre-Louis Bossart

Hi Morimoto-san,

On 8/24/2020 5:55 AM, Kuninori Morimoto wrote:
> External email: Use caution opening links or attachments
>
>
> Hi Mark
> Cc Sameer
>
>> Having an audio-graph-card2 isn't ideal but may be required at least
>> during development :/  Ideally we'd be able to have the new driver parse
>> both binding formats (or rather, have the new binding format be new use
>> cases for the same binding format) and only use -card2 while it's in
>> development.
> If you want to update current audio-graph-card without creating new
> audio-graph-card2, I'm OK about it. But need adjusting / agreement.
>
> Current audio-graph-card "DSP" user is only me, and I'm using it only locally.
> Thus upstream doesn't get damage if I removed "audio-graph-scu-card"
> (= DSP use case) support.

> OTOH, Sameer is posting patch for "-cc-" support. If it was accepted
> and he used it on upstream (= on tegra),
> keeping compatibility will be very difficult and/or code will be very confusable.
>
> If Sameer can OK and wait new audio-graph-card, maybe we have no compatibility issue.
> But in such case, 1st version of new audio-graph-card might be not enough for him.
> Sameer need to waiting / testing / adjusting / etc, etc...

The series [0] introduces small deltas to resolve issues I am facing. As 
you see, most of the implementation is unchanged for the graph-card 
driver. Hence I am not sure if we need a new driver now. If at all it 
gets complicated in future, the "-cc-" compatible can be moved to new 
driver? Please note that the new "-cc-" compatibility is added to 
address following and some of these are discussed in [1].
- DPCM usage with component model (where there can be N number of 
components available and M (<= N) of them can be connected together to 
form an audio path). For example the path would be like,
FE -> BE_1 -> BE_2 -> ... -> BE_M.
- I am extending dpcm_path_get() for this reason and DAI ops get called 
for all connected components.

[0] https://lkml.org/lkml/2020/8/5/42
[1] https://lkml.org/lkml/2020/4/30/519
>
> Thank you for your help !!
>
> Best regards
> ---
> Kuninori Morimoto


^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-24  6:25     ` Sameer Pujar
@ 2020-08-25  0:59       ` Kuninori Morimoto
  2020-08-25  3:11         ` Sameer Pujar
  0 siblings, 1 reply; 31+ messages in thread
From: Kuninori Morimoto @ 2020-08-25  0:59 UTC (permalink / raw)
  To: Sameer Pujar; +Cc: Linux-ALSA, Mark Brown, Pierre-Louis Bossart


Hi Sameer

> The series [0] introduces small deltas to resolve issues I am
> facing. As you see, most of the implementation is unchanged for the
> graph-card driver. Hence I am not sure if we need a new driver now.

Yes, maybe it is not needed *for now*, but will be issue in the future,
because I can't have normal-link and DPCM-link in the same time, right ?

> at all it gets complicated in future, the "-cc-" compatible can be
> moved to new driver? Please note that the new "-cc-" compatibility is
> added to address following and some of these are discussed in [1].
> - DPCM usage with component model (where there can be N number of
> components available and M (<= N) of them can be connected together to
> form an audio path). For example the path would be like,
> FE -> BE_1 -> BE_2 -> ... -> BE_M.
> - I am extending dpcm_path_get() for this reason and DAI ops get
> called for all connected components.
> 
> [0] https://lkml.org/lkml/2020/8/5/42
> [1] https://lkml.org/lkml/2020/4/30/519

The difference between "-cc" and "card2" is DPCM link detection.
"-cc-"  will assume all are DPCM link,
"card2" will detect both normal-link and DPCM-link via DT.

But, I guess new driver 1st version is focus to
detecting normal-link and DPCM-link only.

This means it is not enough for your case,
because I can't full reproduce your board/situation.
Maybe you need some extra patch on "card2"
which "-cc-" added to soc-xxx.c

Thank you for your help !!

Best regards
---
Kuninori Morimoto

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-25  0:59       ` Kuninori Morimoto
@ 2020-08-25  3:11         ` Sameer Pujar
  2020-08-25  5:13           ` Kuninori Morimoto
  0 siblings, 1 reply; 31+ messages in thread
From: Sameer Pujar @ 2020-08-25  3:11 UTC (permalink / raw)
  To: Kuninori Morimoto
  Cc: sharadg, Linux-ALSA, Mark Brown, Pierre-Louis Bossart, jonathanh

Hi Morimoto-san,

CC Jon, Sharad

On 8/25/2020 6:29 AM, Kuninori Morimoto wrote:
> External email: Use caution opening links or attachments
>
>
> Hi Sameer
>
>> The series [0] introduces small deltas to resolve issues I am
>> facing. As you see, most of the implementation is unchanged for the
>> graph-card driver. Hence I am not sure if we need a new driver now.
> Yes, maybe it is not needed *for now*, but will be issue in the future,
> because I can't have normal-link and DPCM-link in the same time, right ?

Yes I am forcing usage of DPCM for all links. May be the compatible 
"-cc-" should reflect this. The reason for doing so is, wanted to use 
DPCM interface with the component model as previously discussed with 
Mark in [1]. The existing detection mechanism did not work because, in 
my case, the HW links are one-to-one and the DT is described that way in 
[0].

>
>> at all it gets complicated in future, the "-cc-" compatible can be
>> moved to new driver? Please note that the new "-cc-" compatibility is
>> added to address following and some of these are discussed in [1].
>> - DPCM usage with component model (where there can be N number of
>> components available and M (<= N) of them can be connected together to
>> form an audio path). For example the path would be like,
>> FE -> BE_1 -> BE_2 -> ... -> BE_M.
>> - I am extending dpcm_path_get() for this reason and DAI ops get
>> called for all connected components.
>>
>> [0] https://lkml.org/lkml/2020/8/5/42
>> [1] https://lkml.org/lkml/2020/4/30/519
> The difference between "-cc" and "card2" is DPCM link detection.
> "-cc-"  will assume all are DPCM link,
> "card2" will detect both normal-link and DPCM-link via DT.
>
> But, I guess new driver 1st version is focus to
> detecting normal-link and DPCM-link only.

Do you plan to propose something like enum for card2 and has scope for 
extension, where link type can be normal/DPCM/codec-to-codec/etc., ?
Since there are so many board variants and may have some specific 
requirements, I am wondering being able to detect link type from DT 
should be the only motivation for a new driver.

>
> This means it is not enough for your case,
> because I can't full reproduce your board/situation.
> Maybe you need some extra patch on "card2"
> which "-cc-" added to soc-xxx.c

I am afraid that even the new driver won't work as it is for my case 
unless a similar compatible and flag exposed for it. So I am not sure 
how different it would be from [0].


Thanks,
Sameer.

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-25  3:11         ` Sameer Pujar
@ 2020-08-25  5:13           ` Kuninori Morimoto
  2020-08-25  5:42             ` Sameer Pujar
  0 siblings, 1 reply; 31+ messages in thread
From: Kuninori Morimoto @ 2020-08-25  5:13 UTC (permalink / raw)
  To: Sameer Pujar
  Cc: sharadg, Linux-ALSA, Mark Brown, Pierre-Louis Bossart, jonathanh


Hi Sameer

> > But, I guess new driver 1st version is focus to
> > detecting normal-link and DPCM-link only.
> 
> Do you plan to propose something like enum for card2 and has scope for
> extension, where link type can be normal/DPCM/codec-to-codec/etc., ?
> Since there are so many board variants and may have some specific
> requirements, I am wondering being able to detect link type from DT
> should be the only motivation for a new driver.

I'm now thinking that new one can detect normal-link, DPCM-link,
multi-CPU / multi-Codec, Codec-to-Codec as normal audio-graph feature.
And has .hook callback for Eeach board specific feature.
see my previous mail which was To:Pierre-Louis
But, it is still under idea, not yet fixed.

> I am afraid that even the new driver won't work as it is for my case
> unless a similar compatible and flag exposed for it. So I am not sure
> how different it would be from [0].

Yes, that is the reason why I'm discussing how to implement new driver.
If we have "-cc-" and you use it on upsteam,
we will keep current card driver as-is, and I will create new "card2" driver.
Because keeping compatible will be difficult.
But I'm not sure what should I do.

Thank you for your help !!

Best regards
---
Kuninori Morimoto

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-25  5:13           ` Kuninori Morimoto
@ 2020-08-25  5:42             ` Sameer Pujar
  2020-08-25  6:35               ` Kuninori Morimoto
  0 siblings, 1 reply; 31+ messages in thread
From: Sameer Pujar @ 2020-08-25  5:42 UTC (permalink / raw)
  To: Kuninori Morimoto
  Cc: sharadg, Linux-ALSA, Mark Brown, Pierre-Louis Bossart, jonathanh

Hi Morimoto-san,

>>> But, I guess new driver 1st version is focus to
>>> detecting normal-link and DPCM-link only.
>> Do you plan to propose something like enum for card2 and has scope for
>> extension, where link type can be normal/DPCM/codec-to-codec/etc., ?
>> Since there are so many board variants and may have some specific
>> requirements, I am wondering being able to detect link type from DT
>> should be the only motivation for a new driver.
> I'm now thinking that new one can detect normal-link, DPCM-link,
> multi-CPU / multi-Codec, Codec-to-Codec as normal audio-graph feature.
> And has .hook callback for Eeach board specific feature.
> see my previous mail which was To:Pierre-Louis
> But, it is still under idea, not yet fixed.

If we plan to go this way, I think we need to consider board specific 
configuration at init time and one at runtime. In that case there could 
be multiple compatibles that would get added to the driver and various 
other requirements can be managed with behavioral flags instead from DT?


Thanks,
Sameer.

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-25  5:42             ` Sameer Pujar
@ 2020-08-25  6:35               ` Kuninori Morimoto
  2020-08-26  6:46                 ` Sameer Pujar
  0 siblings, 1 reply; 31+ messages in thread
From: Kuninori Morimoto @ 2020-08-25  6:35 UTC (permalink / raw)
  To: Sameer Pujar
  Cc: sharadg, Linux-ALSA, Mark Brown, Pierre-Louis Bossart, jonathanh


Hi Sameer

> If we plan to go this way, I think we need to consider board specific
> configuration at init time and one at runtime. In that case there
> could be multiple compatibles that would get added to the driver and
> various other requirements can be managed with behavioral flags
> instead from DT?

This is still just idea though...
But for example, if you want to
	1) basically, DT is almost audio-graph
	2) but want to have customized operation for some part

And if "audio-graph-card2" driver has graph_init() exported function,
you can create your own drviver, and use customized audio-graph
by using .hooks.

This is just idea/sample, but I'm not sure this is enough or not,
and/or if I can do what I want.
But do you think it can solve your issue ?

-- own driver ---
	static const struct of_device_id graph_of_match[] = {
=>		{ .compatible = "sameer-audio-graph" },
		{},
	};

	static audio_graph_hooks hooks = {
		.parse_of_hook_pre  = xxx,
=>		.parse_of_hook_post = sameer_parse_of_post,
		.dai_link_of_pre    = xxx,
		.dai_link_of_post   = xxx,
=>		.init               = sameer_init,
		...
	};

=>	int sameer_init(struct snd_soc_pcm_runtime *rtd)
	{
		/*
		 * This will be called runtime init timing.
		 * Call original asoc_simple_dai_init() first
		 * and do own init, for example.
		 */
		asoc_simple_dai_init(rtd);

		do_something_own_settings(rtd->xxx);
	}

=>	static int sameer_parse_of_post(struct asoc_simple_priv *priv)
	{
		struct sameer_priv *my_priv = graph_priv_to_my_priv(priv);

		/*
		 * This will be called after audio_graph's graph_parse_of()
		 */

		/*
		 * Customize own settings here.
		 *
		 * Special connection ?
		 * Special Setings ?
		 * Calculate something ?
		 * Overwrite something ?
		 */
	}

	static int sameer_probe(...)
	{
		struct sameer_priv *my_priv;
		struct asoc_simple_priv *graph_priv;

		my_priv = zalloc();
		graph_priv = my_priv_to_graph_priv(my_priv);
=>		graph_priv->hooks = hooks

		/*
		 * Basically, it will do same as audio_graph,
		 * but .hooks will be called if you specified
		 */
=>		return graph_init(graph_priv);
	}

--- Kconfig ----

	config SND_SAMEER_AUDIO_GRAPH_CARD
		tristate "Sameer's Audio Graph sound card support"
=>		depends on SND_AUDIO_GRAPH_CARD
		...

---- my-dt ----------

	  /*
	   * DT setting is almost same as audio_graph
	   * which is supporting normal and DPCM.
	   * You can add own property which will be handled under .hook
	   */
	                      *************
	  PCM0 <------------> *           * <----DAI0----->
	                      *  DSP      *
	                      *           * <----DAI1----->
	                      *************
	  PCM1 <------------------------------------------> DAI2

		sound {
			compatible = "sameer-audio-graph";

			dais = <&PCM0,	/* for DPCM */
				&PCM1>  /* for normal*/
		};

		front-end {
			ports {
				PCM0: port@0 { pcm0: endpoint { remote-endpoint = <&dsp_f0>; }; };
				PCM1: port@1 { pcm1: endpoint { remote-endpoint = <&dai2>; }; };
			};
		};

		dsp {
			compatible = "audio-graph-card2-dsp";

			ports {
				/* Front End side */
				port@0 { dsp_f0: endpoint { remote-endpoint = <&pcm0>; }; };

				/* Back End side */
				port@4 { dsp_b0: endpoint { remote-endpoint = <&dai0>; }; };
				port@5 { dsp_b1: endpoint { remote-endpoint = <&dai1>; }; };
			};
		};

		back-end {
			ports {
				port@0 { dai0: endpoint { remote-endpoint = <&dsp_b0>; }; };
				port@1 { dai1: endpoint { remote-endpoint = <&dsp_b1>; }; };
			};
		};

		codec {
			port { dai2: endpoint { remote-endpoint = <&pcm1>; }; };
		};


Thank you for your help !!

Best regards
---
Kuninori Morimoto

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-25  6:35               ` Kuninori Morimoto
@ 2020-08-26  6:46                 ` Sameer Pujar
  2020-08-27  1:18                   ` Kuninori Morimoto
  2020-08-27  1:36                   ` Kuninori Morimoto
  0 siblings, 2 replies; 31+ messages in thread
From: Sameer Pujar @ 2020-08-26  6:46 UTC (permalink / raw)
  To: Kuninori Morimoto
  Cc: sharadg, Linux-ALSA, Mark Brown, Pierre-Louis Bossart, jonathanh

Hi Morimoto-san,

>> If we plan to go this way, I think we need to consider board specific
>> configuration at init time and one at runtime. In that case there
>> could be multiple compatibles that would get added to the driver and
>> various other requirements can be managed with behavioral flags
>> instead from DT?
> This is still just idea though...
> But for example, if you want to
>          1) basically, DT is almost audio-graph
>          2) but want to have customized operation for some part

> And if "audio-graph-card2" driver has graph_init() exported function,
> you can create your own drviver, and use customized audio-graph
> by using .hooks.

What you are suggesting is 'audio-graph-card2' is a new restructured 
version of 'audio-graph-card' with some additional customization 
available for specific users. Do you think updating existing 
'audio-graph-card' itself, with necessary hooks, would be too 
complicated to handle?

> This is just idea/sample, but I'm not sure this is enough or not,
> and/or if I can do what I want.
> But do you think it can solve your issue ?

 From a brief overview, it may solve my issue in customizing few stuff. 
But I am not too sure if we want to go that way, because eventually we 
end up in writing a separate machine driver for Tegra (though there can 
be common stuff used from the generic graph card). The original idea was 
to use 'audio-graph-card' and people facing similar issues could use 
"-cc-" compatible.

...

>          static audio_graph_hooks hooks = {
>                  .parse_of_hook_pre  = xxx,
> =>              .parse_of_hook_post = sameer_parse_of_post,

We may end up re-parsing the whole stuff under 'hook_post' (which 
appears redundant) because I am forcing DPCM and would like func_dpcm() 
to execute for all links. If I could set a flag under 'hooks' and if 
'audio-graph-card' can use it to force DPCM, it would appear to be a 
better choice.

...

> ---- my-dt ----------
>
>            /*
>             * DT setting is almost same as audio_graph
>             * which is supporting normal and DPCM.
>             * You can add own property which will be handled under .hook
>             */
>                                *************
>            PCM0 <------------> *           * <----DAI0----->
>                                *  DSP      *
>                                *           * <----DAI1----->
>                                *************
>            PCM1 <------------------------------------------> DAI2

I think, the main issue is you want to keep both normal/DPCM link 
detections, where as I am trying to force DPCM usage. That is mainly 
because I do have N components and would like to connect M(<=N) 
components with DPCM usage. My DT would be like this.

                                *************
           PCM0 <-------------> *           * <------ DAI0 (component-0) 
------>
                     ...        *           * <------ DAI1 (component-1) 
------>
                      * Crossbar  * ...
           PCMX <-------------> *           * <------ DAIN (component-N) 
------>
                                *           *
                                *************

So at runtime my audio path could be,
at t0: PCM0 -> DAI0 -> DAI1
at t1: PCM0 -> DAI1 -> DAIN

Audio path can have ideally any combination of DAIs (and components) in 
the path. DT may look like it is similar to having normal links, but 
conceptually any PCMx can be routed to any set of DAIs. DAIs can be 
resampler, mixer, multiplexer etc., in SoC. I hope this makes clear as 
to what I am looking for.

...

>                  dsp {
>                          compatible = "audio-graph-card2-dsp";

Sorry I did not understand this. Do you intend to parse 'dsp' separately 
with some version of audio graph card? In my case 'dsp' is just a 
'crossbar' and is registered as a component exposing all routes. However 
I have described links in the DT in a similar way where my 'crossbar' is 
exposing FEs and BEs like below.

>
>                          ports {
>                                  /* Front End side */
>                                  port@0 { dsp_f0: endpoint { remote-endpoint = <&pcm0>; }; };
>
>                                  /* Back End side */
>                                  port@4 { dsp_b0: endpoint { remote-endpoint = <&dai0>; }; };
>                                  port@5 { dsp_b1: endpoint { remote-endpoint = <&dai1>; }; };
>                          };
>                  };
>
>                  back-end {
>                          ports {
>                                  port@0 { dai0: endpoint { remote-endpoint = <&dsp_b0>; }; };
>                                  port@1 { dai1: endpoint { remote-endpoint = <&dsp_b1>; }; };
>                          };
>                  };
>
>                  codec {
>                          port { dai2: endpoint { remote-endpoint = <&pcm1>; }; };
>                  };
>
>
> Thank you for your help !!
>
> Best regards
> ---
> Kuninori Morimoto


^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-26  6:46                 ` Sameer Pujar
@ 2020-08-27  1:18                   ` Kuninori Morimoto
  2020-08-27  1:36                   ` Kuninori Morimoto
  1 sibling, 0 replies; 31+ messages in thread
From: Kuninori Morimoto @ 2020-08-27  1:18 UTC (permalink / raw)
  To: Sameer Pujar
  Cc: sharadg, Linux-ALSA, Mark Brown, Pierre-Louis Bossart, jonathanh


Hi Sameer

> What you are suggesting is 'audio-graph-card2' is a new restructured
> version of 'audio-graph-card' with some additional customization
> available for specific users. Do you think updating existing
> 'audio-graph-card' itself, with necessary hooks, would be too
> complicated to handle?

It depends on how new driver was implemented.
If we need to keep current card, I will keep it as-is and do nothing anymore,
and has customize option at card2.
If we update current card, customize option will be added to it.

> From a brief overview, it may solve my issue in customizing few
> stuff. But I am not too sure if we want to go that way, because
> eventually we end up in writing a separate machine driver for Tegra
> (though there can be common stuff used from the generic graph
> card). The original idea was to use 'audio-graph-card' and people
> facing similar issues could use "-cc-" compatible.

The biggest issue on current audio-graph-card is that
DPCM feature is limited, and because of it, the link detection
is very tricky.
Your "-cc-" assumes all links are DPCM, but it is more limitation.
We want to have more flexible/generic detection.

> >                  dsp {
> >                          compatible = "audio-graph-card2-dsp";
> 
> Sorry I did not understand this. Do you intend to parse 'dsp'
> separately with some version of audio graph card? In my case 'dsp' is
> just a 'crossbar' and is registered as a component exposing all
> routes. However I have described links in the DT in a similar way
> where my 'crossbar' is exposing FEs and BEs like below.

This compatible is used just for indicating audio-graph DSP.
"audio-graph-card" will parse it if it was connected.
If you confuse it, just ignore for now.


Thank you for your help !!

Best regards
---
Kuninori Morimoto

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-26  6:46                 ` Sameer Pujar
  2020-08-27  1:18                   ` Kuninori Morimoto
@ 2020-08-27  1:36                   ` Kuninori Morimoto
  1 sibling, 0 replies; 31+ messages in thread
From: Kuninori Morimoto @ 2020-08-27  1:36 UTC (permalink / raw)
  To: Sameer Pujar
  Cc: sharadg, Linux-ALSA, Mark Brown, Pierre-Louis Bossart, jonathanh


Hi Sameer

> >          static audio_graph_hooks hooks = {
> >                  .parse_of_hook_pre  = xxx,
> > =>              .parse_of_hook_post = sameer_parse_of_post,
> 
> We may end up re-parsing the whole stuff under 'hook_post' (which
> appears redundant) because I am forcing DPCM and would like
> func_dpcm() to execute for all links. If I could set a flag under
> 'hooks' and if 'audio-graph-card' can use it to force DPCM, it would
> appear to be a better choice.

Asking how to judge link to driver is maybe nice idea.

I didn't mentioned in order not to not diverge,
but I'm also thinking that some overwrite function itself
as you mentioned.

	if (priv->hooks->func_pre)
		priv->hooks->func_pre(...);

	if (priv->hooks->func)
		priv->hooks->func(...);	/* driver's function */
	else
		graph_func(...);	/* audio-graph function */
		
	if (priv->hooks->func_post)
		priv->hooks->func_post(...);


Thank you for your help !!

Best regards
---
Kuninori Morimoto

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-24  0:25   ` Kuninori Morimoto
  2020-08-24  6:25     ` Sameer Pujar
@ 2020-09-03 23:51     ` Kuninori Morimoto
  2020-09-09 11:33       ` Sameer Pujar
  1 sibling, 1 reply; 31+ messages in thread
From: Kuninori Morimoto @ 2020-09-03 23:51 UTC (permalink / raw)
  To: Mark Brown; +Cc: Sameer Pujar, Pierre-Louis Bossart, Linux-ALSA


Hi Mark
Cc Sameer

> > Having an audio-graph-card2 isn't ideal but may be required at least
> > during development :/  Ideally we'd be able to have the new driver parse
> > both binding formats (or rather, have the new binding format be new use
> > cases for the same binding format) and only use -card2 while it's in
> > development.
> 
> If you want to update current audio-graph-card without creating new
> audio-graph-card2, I'm OK about it. But need adjusting / agreement.
> 
> Current audio-graph-card "DSP" user is only me, and I'm using it only locally.
> Thus upstream doesn't get damage if I removed "audio-graph-scu-card"
> (= DSP use case) support.
> 
> OTOH, Sameer is posting patch for "-cc-" support. If it was accepted
> and he used it on upstream (= on tegra),
> keeping compatibility will be very difficult and/or code will be very confusable.
> 
> If Sameer can OK and wait new audio-graph-card, maybe we have no compatibility issue.
> But in such case, 1st version of new audio-graph-card might be not enough for him.
> Sameer need to waiting / testing / adjusting / etc, etc...

The problem I'm feeling is that new card (= let's call it as card2 here) and
Sameer's posting "-cc-" are conflicted / no-compatibility.
It expands card1 DPCM detection.

The good things having "-cc-" is that he can use it immediately.
the bad things is that it is dificult to keep compatibility between "card1 + -cc-" and "card2".

Creating card2 for development purpose is very welcome for me, and card1 user :)
But after that, if we want to merge card1 and card2, we need to drop "-cc-" support unfortunately.
If we want to keep "-cc-", we need to have both card1 / card2.

So, how to handle it ?

Thank you for your help !!

Best regards
---
Kuninori Morimoto

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-09-03 23:51     ` Kuninori Morimoto
@ 2020-09-09 11:33       ` Sameer Pujar
  0 siblings, 0 replies; 31+ messages in thread
From: Sameer Pujar @ 2020-09-09 11:33 UTC (permalink / raw)
  To: Kuninori Morimoto, Mark Brown; +Cc: Linux-ALSA, Pierre-Louis Bossart

Hi Mark, Morimoto-san,


>>> Having an audio-graph-card2 isn't ideal but may be required at least
>>> during development :/  Ideally we'd be able to have the new driver parse
>>> both binding formats (or rather, have the new binding format be new use
>>> cases for the same binding format) and only use -card2 while it's in
>>> development.
>> If you want to update current audio-graph-card without creating new
>> audio-graph-card2, I'm OK about it. But need adjusting / agreement.
>>
>> Current audio-graph-card "DSP" user is only me, and I'm using it only locally.
>> Thus upstream doesn't get damage if I removed "audio-graph-scu-card"
>> (= DSP use case) support.
>>
>> OTOH, Sameer is posting patch for "-cc-" support. If it was accepted
>> and he used it on upstream (= on tegra),
>> keeping compatibility will be very difficult and/or code will be very confusable.
>>
>> If Sameer can OK and wait new audio-graph-card, maybe we have no compatibility issue.
>> But in such case, 1st version of new audio-graph-card might be not enough for him.
>> Sameer need to waiting / testing / adjusting / etc, etc...
> The problem I'm feeling is that new card (= let's call it as card2 here) and
> Sameer's posting "-cc-" are conflicted / no-compatibility.
> It expands card1 DPCM detection.
>
> The good things having "-cc-" is that he can use it immediately.
> the bad things is that it is dificult to keep compatibility between "card1 + -cc-" and "card2".
>
> Creating card2 for development purpose is very welcome for me, and card1 user :)
> But after that, if we want to merge card1 and card2, we need to drop "-cc-" support unfortunately.
> If we want to keep "-cc-", we need to have both card1 / card2.
>
> So, how to handle it ?
>
>
Need your help for clarity on above. Currently I cannot publish v3 for 
[0] unless above is resolved.

[0] https://lkml.org/lkml/2020/8/5/42

Thanks,
Sameer.

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-21  7:14   ` Kuninori Morimoto
  2020-08-21  8:28     ` Sameer Pujar
@ 2020-09-25  1:43     ` Kuninori Morimoto
  2020-09-25 19:22       ` Mark Brown
  1 sibling, 1 reply; 31+ messages in thread
From: Kuninori Morimoto @ 2020-09-25  1:43 UTC (permalink / raw)
  To: Sameer Pujar; +Cc: Linux-ALSA, Mark Brown, Pierre-Louis Bossart, jonathanh


Hi

For more generic Auido Graph Sound Card, I'm thinking about
Multi-CPU/Codec now.
(Need to think about Codec-to-Codec later, but it is next task)
I want to know your idea, opinion, etc, etc...

> > Please note that asoc_simple_init_dai_link_params() makes the link
> > codec-to-codec if any component involved has 'non_legacy_dai_naming'
> > flag enabled. It is used by both audio-graph and simple-card. So it is
> > a combination of three (DPCM, normal and codec-to-codec). To avoid all
> > this complication, I am treating all links as DPCM in the series [0] I
> > sent.
> 
> Ahh, we want to consider codec-to-codec..
> And I noticed multi-CPU / multi-Codec case too (= for normal connection).
> Do we have sample code ? > Mark

First idea is using "ports".
In such case, I think multi-CPU side is not big issue,
because "dais" selects "port" (and "ports") instead of "endpoint".
(Note: here have no compatible to "-scu-" of audio-graph)

	CPU0-1 <------> Codec0
	CPU0-2 <---/
	CPU0-3 <---/

	sound {
		compatible = "audio-graph-card2";

=>		dais = <&cpu0>;
	};

	cpu-device {
		/*
		 * use "ports" for Multi-CPU
		 */
=>		cpu0: ports {
			port@0 { cpu0_1: endpoint { remote-endpoint = <&codec0_1>; }; };
			port@1 { cpu0_2: endpoint { remote-endpoint = <&codec0_2>; }; };
			port@2 { cpu0_3: endpoint { remote-endpoint = <&codec0_3>; }; };
		};

		/*
		 * According to graph.txt, 
		 * we can use other "port" and/or "ports" for non multi-CPU.
		 */
		port@x { };
		port@x { };
		ports {
			port { };
			...
		};
	};

	codec-device {
		port {	codec0_1: endpoint@0 { remote-endpoint = <&cpu0_1>; };
			codec0_2: endpoint@0 { remote-endpoint = <&cpu0_2>; }; 
			codec0_3: endpoint@0 { remote-endpoint = <&cpu0_3>; }; };
	};


But multi-Codec side is difficult.
Because it is selected via "endpoint" via CPU.
No way to select it via "port" and/or "ports".

	CPU1 <------> Codec1-1
	        \---> Codec1-2
	        \---> Codec1-3

	sound {
		compatible = "audio-graph-card2";

		dais = <&cpu1>;
	};

	cpu-device {
		/*
		 * CPU selects Codec by "endpoints".
		 */
		cpu1: port {	cpu1_1: endpoint { remote-endpoint = <&codec1_1>; };
				cpu1_2: endpoint { remote-endpoint = <&codec1_2>; };
				cpu1_3: endpoint { remote-endpoint = <&codec1_3>; };
		};
	};

	codec-device {
		/*
		 * Codec is selected via CPU.
		 * It is best if we can select "codec1" (= port / ports) from CPU.
		 * But is selected via "endpoint" in reality,
		 * using "ports" is not enough for Multi-Codec.
		 * because see below
		 */
		codec1: ports {
			port@0 { codec1_1: endpoint { remote-endpoint = <&dai0_0>; }; };
			port@1 { codec1_2: endpoint { remote-endpoint = <&dai0_1>; }; };
			port@2 { codec1_3: endpoint { remote-endpoint = <&dai0_2>; }; };
		};

		/*
		 * Because "ports" is used to for non Multi-xxx too.
		 * How to know the "ports" was for Multi-xxx or not ?
		 */
		ports {
			port@1 { for device1 };
			port@2 { for device2 };
			port@3 { for device3 };
		};
	};

We might want to select Multi-CPU/Codec by using multi deivces ?
in such case, using "ports" idea is not enough.

Using extra device like DSP can be more generic ?

	<--- multi-CPU --->
	            *******
	CPU0-1 <--> *     * <--> Codec0
	CPU0-2 <--> *     *
	CPU0-3 <--> *     *
	            *******

	sound {
		compatible = "audio-graph-card2";

		dais = <&cpu0>;
	};

	cpu-device {
		ports {
			/* for multi-CPU */
			port@0 { cpu0_1: endpoint { remote-endpoint = <&multi_cpu_1>; }; };
			port@1 { cpu0_2: endpoint { remote-endpoint = <&multi_cpu_2>; }; };
			port@2 { cpu0_3: endpoint { remote-endpoint = <&multi_cpu_3>; }; };

			/* for others */
			port@3 { ... };
		};
	};

	multi-cpu {
		compatible = "audio-graph-card2-multi";

		/* CPU side */
		cpu0: ports@0 {
			port@0 { multi_cpu_1: endpoint { remote-endpoint = <&cpu0_1>; }; };
			port@1 { multi_cpu_2: endpoint { remote-endpoint = <&cpu0_2>; }; };
			port@2 { multi_cpu_3: endpoint { remote-endpoint = <&cpu0_3>; }; };
		};

		/* Codec side */
		ports@1 {
			port@0 { multi_cpu0: endpoint { remote-endpoint = <&codec0>; }; };
		};
	};

	codec-device {
		ports {
			port@0 { codec0: endpoint { remote-endpoint = <&multi_cpu0>; }; };
			port@1 { ... };
		};
	};


	          <--- multi-Codec --->
	          *******
	CPU1 <--> *     * <--> Codec1-1
	          *     * <--> Codec1-2
	          *     * <--> Codec1-3
	          *******

	sound {
		compatible = "audio-graph-card2";

		dais = <&cpu1>;
	};

	cpu-device {
		ports {
			cpu1: port@0 { cpu1_0: endpoint { remote-endpoint = <&multi_codec1>; }; };
			      port@1 { ... };
		};
	};

	multi-cpu {
		compatible = "audio-graph-card2-multi";

		/* Front End */
		ports@0 {
			port { multi_codec1: endpoint { remote-endpoint = <&cpu1_0>; }; };
		};

		/* Back End */
		ports@1 {
			port@0 { multi_codec1_1: endpoint { remote-endpoint = <&codec1_1>; }; };
			port@1 { multi_codec1_2: endpoint { remote-endpoint = <&codec1_2>; }; };
			port@2 { multi_codec1_3: endpoint { remote-endpoint = <&codec1_3>; }; };
		};
	};

	codec-device {
		ports {
			port@0 { codec1_1: endpoint { remote-endpoint = <&multi_codec1_1>; }; };
			port@0 { codec1_2: endpoint { remote-endpoint = <&multi_codec1_2>; }; };
			port@0 { codec1_3: endpoint { remote-endpoint = <&multi_codec1_3>; }; };
			port@1 { ... };
		};
	};



Thank you for your help !!

Best regards
---
Kuninori Morimoto

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-09-25  1:43     ` Kuninori Morimoto
@ 2020-09-25 19:22       ` Mark Brown
  2020-09-25 20:04         ` Pierre-Louis Bossart
  0 siblings, 1 reply; 31+ messages in thread
From: Mark Brown @ 2020-09-25 19:22 UTC (permalink / raw)
  To: Kuninori Morimoto
  Cc: Sameer Pujar, Pierre-Louis Bossart, Linux-ALSA, jonathanh

[-- Attachment #1: Type: text/plain, Size: 1422 bytes --]

On Fri, Sep 25, 2020 at 10:43:59AM +0900, Kuninori Morimoto wrote:

> But multi-Codec side is difficult.
> Because it is selected via "endpoint" via CPU.
> No way to select it via "port" and/or "ports".

Indeed.

> We might want to select Multi-CPU/Codec by using multi deivces ?
> in such case, using "ports" idea is not enough.

> Using extra device like DSP can be more generic ?

> 	<--- multi-CPU --->
> 	            *******
> 	CPU0-1 <--> *     * <--> Codec0
> 	CPU0-2 <--> *     *
> 	CPU0-3 <--> *     *
> 	            *******

I think this is what we want for SoCs, represent the DSPs explicitly and
then have the FEs and BEs all be ports on the DSP.  I think a similar
thing would also work for legacy (I2S & so on) DAIs where we've got more
endpoints on the DAI - if we define slots on the DAI then from the point
of view of the DT bindings it's just a very, very inflexible DSP:

        CPU1 <--> DAI slot A <--> Codec1-1
              \-> DAI slot B <--> Codec1-2
        CPU2 <--> DAI slot C <--> Codec1-3

or whatever.  This doesn't allow for really complex setups that change
the slot mapping at runtime (TBH those probably need custom cards
anyway) but I think it should support most cases where TDM causes
difficulties today.  I'm not sure if we need this for more modern buses
like SoundWire, I'd hope we can dynamically assign slots at runtime more
easily, but ICBW.

Thanks for all your work on this!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-09-25 19:22       ` Mark Brown
@ 2020-09-25 20:04         ` Pierre-Louis Bossart
  2020-09-25 20:10           ` Mark Brown
  0 siblings, 1 reply; 31+ messages in thread
From: Pierre-Louis Bossart @ 2020-09-25 20:04 UTC (permalink / raw)
  To: Mark Brown, Kuninori Morimoto; +Cc: Sameer Pujar, Linux-ALSA, jonathanh



On 9/25/20 2:22 PM, Mark Brown wrote:
> On Fri, Sep 25, 2020 at 10:43:59AM +0900, Kuninori Morimoto wrote:
> 
>> But multi-Codec side is difficult.
>> Because it is selected via "endpoint" via CPU.
>> No way to select it via "port" and/or "ports".
> 
> Indeed.
> 
>> We might want to select Multi-CPU/Codec by using multi deivces ?
>> in such case, using "ports" idea is not enough.
> 
>> Using extra device like DSP can be more generic ?
> 
>> 	<--- multi-CPU --->
>> 	            *******
>> 	CPU0-1 <--> *     * <--> Codec0
>> 	CPU0-2 <--> *     *
>> 	CPU0-3 <--> *     *
>> 	            *******
> 
> I think this is what we want for SoCs, represent the DSPs explicitly and
> then have the FEs and BEs all be ports on the DSP.  I think a similar
> thing would also work for legacy (I2S & so on) DAIs where we've got more
> endpoints on the DAI - if we define slots on the DAI then from the point
> of view of the DT bindings it's just a very, very inflexible DSP:
> 
>          CPU1 <--> DAI slot A <--> Codec1-1
>                \-> DAI slot B <--> Codec1-2
>          CPU2 <--> DAI slot C <--> Codec1-3
> 
> or whatever.  This doesn't allow for really complex setups that change
> the slot mapping at runtime (TBH those probably need custom cards
> anyway) but I think it should support most cases where TDM causes
> difficulties today.  I'm not sure if we need this for more modern buses
> like SoundWire, I'd hope we can dynamically assign slots at runtime more
> easily, but ICBW.

SoundWire doesn't have a notion of 'slot'. Instead you program the data 
ports for the type of audio data to be transmitted/received.

See some pointers at 
https://mipi.org/sites/default/files/MIPI-SoundWire-webinar-20150121-final.pdf
Pages 42-47 describe the main concepts.

The actual bit allocation can be done in different ways. On the Intel 
side, we use a dynamic allocation. It's my understanding that Qualcomm 
have a static allocation for their amplifier links.

In most cases, a sink port receives exactly what it needs, but for 
playback we have cases where all amplifiers receive the same data (we 
call this 'mirror mode', and each amplifier will be configured to render 
a specific channel from the data received. This is useful to deal with 
orientation/posture changes where the data transmitted on the wires 
doesn't need to be changed. This avoid dynamic re-configurations on the 
DSP + bus sides, only the amplifier settings need to be modified - 
typically via controls.

That said, the mapping of data ports between CPU and codec sides is 
rather static, mostly because devices typically dedicate specific data 
ports to specific functionality. SDCA will not change this, quite the 
opposite, the mapping between ports and audio functionality behind the 
port will be defined in platform firmware.

It's a bit of a stretch but conceptually there is some level of overlap 
between SoundWire data ports and TDM slots, e.g. if in a TDM link you 
used slots 4,5 for headset playback, you might use data port 2 on a 
SoundWire link. It's however a 'logical' mapping, the actual position of 
the bits in the frame is handled by the bit allocation.

Hope this helps!
-Pierre


^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-09-25 20:04         ` Pierre-Louis Bossart
@ 2020-09-25 20:10           ` Mark Brown
  2020-09-25 20:50             ` Pierre-Louis Bossart
  0 siblings, 1 reply; 31+ messages in thread
From: Mark Brown @ 2020-09-25 20:10 UTC (permalink / raw)
  To: Pierre-Louis Bossart
  Cc: Sameer Pujar, Linux-ALSA, Kuninori Morimoto, jonathanh

[-- Attachment #1: Type: text/plain, Size: 1794 bytes --]

On Fri, Sep 25, 2020 at 03:04:37PM -0500, Pierre-Louis Bossart wrote:
> On 9/25/20 2:22 PM, Mark Brown wrote:

> > difficulties today.  I'm not sure if we need this for more modern buses
> > like SoundWire, I'd hope we can dynamically assign slots at runtime more
> > easily, but ICBW.

> SoundWire doesn't have a notion of 'slot'. Instead you program the data
> ports for the type of audio data to be transmitted/received.

...

> In most cases, a sink port receives exactly what it needs, but for playback
> we have cases where all amplifiers receive the same data (we call this
> 'mirror mode', and each amplifier will be configured to render a specific
> channel from the data received. This is useful to deal with

Channels are essentially the same as timeslots on a TDM bus TBH.

> That said, the mapping of data ports between CPU and codec sides is rather
> static, mostly because devices typically dedicate specific data ports to
> specific functionality. SDCA will not change this, quite the opposite, the
> mapping between ports and audio functionality behind the port will be
> defined in platform firmware.

> It's a bit of a stretch but conceptually there is some level of overlap
> between SoundWire data ports and TDM slots, e.g. if in a TDM link you used
> slots 4,5 for headset playback, you might use data port 2 on a SoundWire
> link. It's however a 'logical' mapping, the actual position of the bits in
> the frame is handled by the bit allocation.

If a device is hard limited to particular slots we can presumably
discover that (either through the spec or by keying off the ID
registers) and do the right thing?  In any case if we need a firmware
mapping for DT systems it sounds like something that works for TDM
should be mappable onto SoundWire channels easily enough.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-09-25 20:10           ` Mark Brown
@ 2020-09-25 20:50             ` Pierre-Louis Bossart
  0 siblings, 0 replies; 31+ messages in thread
From: Pierre-Louis Bossart @ 2020-09-25 20:50 UTC (permalink / raw)
  To: Mark Brown; +Cc: Sameer Pujar, Linux-ALSA, Kuninori Morimoto, jonathanh




> If a device is hard limited to particular slots we can presumably
> discover that (either through the spec or by keying off the ID
> registers) and do the right thing?  In any case if we need a firmware
> mapping for DT systems it sounds like something that works for TDM
> should be mappable onto SoundWire channels easily enough.

yes, if we can define a notion of logical TDM 'slot group' (e.g. 
headphone, mic, streamA, <add tag here>, etc), then the mapping with 
SoundWire data ports is straightforward. The mapping would typically 
come from platform firmware, not by checking device registers.

The actual value of a TDM slot which defines the transport position is 
however irrelevant for SoundWire.

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-08-21  4:15 More Generic Audio Graph Sound Card idea Kuninori Morimoto
                   ` (3 preceding siblings ...)
  2020-08-21 15:49 ` Pierre-Louis Bossart
@ 2020-10-13  4:50 ` Kuninori Morimoto
  2020-10-15 14:32   ` Mark Brown
  4 siblings, 1 reply; 31+ messages in thread
From: Kuninori Morimoto @ 2020-10-13  4:50 UTC (permalink / raw)
  To: Mark Brown; +Cc: Sameer Pujar, Pierre-Louis Bossart, Linux-ALSA


Hi ALSA ML


> I'm thinking below style as new audio-graph-card2.
> It is not tricky, thus connection judgement for normal vs DSP is easy.
> Then, code can be more readable (= no guaranty :P) ?

I have proposed DPCM connection idea for audio-card2 before,
and proposed Multi-CPU/Codec connection idea later.
I'm not yet implemented these, but I want to merge / reuse these
idea to keep simple code.

By this mail, I want to propose Codec2Codec idea.
It is almost same idea with DPCM, Multi-CPU/Codec,
and is last parts for missing feature.

I think this idea can be generic and easy to update
in the future (if needed).

But, I want feedback, opinion.
I wonder does it works for your use-case ?

-- Image ---

Codec2Codec

Codec1 <--> Codec2

-- DT Image --

   +--+
=> |  |<--> Codec1
   |  |<--> Codec2
   +--+

-- DT Sample --

	sound {
		compatible = "audio-graph-card2";

		dais = <&codec2codec>;
	};

	CODEC2CODEC {
		compatible = "audio-graph-card2-codec2codec";

		ports {
			codec2codec: port@0 { fe_ep: endpoint { remote-endpoint = <&codec1_ep>; }; };
			             port@1 { be_ep: endpoint { remote-endpoint = <&codec2_ep>; }; };
		};
	};

 	Codec1 {
		port@0 { codec1_ep: endpoint { remote-endpoint = <&fe_ep>; }; };
	};

 	Codec2 {
		port@0 { codec2_ep: endpoint { remote-endpoint = <&be_ep>; }; };
	};


-- Image ---

          *******
PCM0 <--> *     * <--> DAI0: Codec Headset
PCM1 <--> *     * <--> DAI1: Codec Speakers
PCM2 <--> * DSP * <--> DAI2: MODEM
PCM3 <--> *     * <--> DAI3: BT
          *     * <--> DAI4: DMIC
          *     * <--> DAI5: FM
          *******

-- DT Image --

   +--+                ****                 +--+
=> |  |<--> PCM0       *  * <-- (dummy) --> |  | <=
   |  |<-- (dummy) --> *  *       DAI0 <--> |  |
   +--+                *  *                 +--+
                       *  *            
   +--+                *  *                 +--+
=> |  |<--> PCM1       *  * <-- (dummy) --> |  | <=
   |  |<-- (dummy) --> *  *       DAI1 <--> |  |
   +--+                *  *                 +--+
   ...                 ...                 ...


-- DT Sample --

	sound {
		compatible = "audio-graph-card2";

		/* indicate all Front-End, Back-End in DPCM case */
		dais = <&dsp_fe0, &dsp_fe1, &dsp_fe2, &dsp_fe3,
		        &dsp_be0, &dsp_be1, &dsp_be2, &dsp_be3, &dsp_be4, &dsp_be5>;
	};

	DSP {
		compatible = "audio-graph-card2-dsp";

		ports@0 {
			/* Front-End is pcm0_ep, Back-End is dummy */
			dsp_fe0: port@0 { dsp_fe0_ep: endpoint { remote-endpoint = <&pcm0_ep>; }; };
			         port@1 { };
		};

		...

		ports@4 {
			/* Front-End is dummy, Back-End is dai_ep */
			dsp_be0: port@0 { };
			         port@1 { dsp_be0_ep: endpoint { remote-endpoint = <&dai0_ep>; }; };
		};

		...

	};

 	CPU {
		ports {
			port@0 { pcm0_ep: endpoint { remote-endpoint = <&dsp_fe0_ep>; }; };
			port@1 { pcm1_ep: endpoint { remote-endpoint = <&dsp_fe1_ep>; }; };
			port@2 { pcm2_ep: endpoint { remote-endpoint = <&dsp_fe2_ep>; }; };
			port@3 { pcm3_ep: endpoint { remote-endpoint = <&dsp_fe3_ep>; }; };
			...
		};
	};

	Codec {
		ports {
			port@0 { dai0_ep: endpoint { remote-endpoint = <&dsp_be0_ep>; }; };
			port@1 { dai1_ep: endpoint { remote-endpoint = <&dsp_be1_ep>; }; };
			port@2 { dai2_ep: endpoint { remote-endpoint = <&dsp_be2_ep>; }; };
			port@3 { dai3_ep: endpoint { remote-endpoint = <&dsp_be3_ep>; }; };
			port@4 { dai4_ep: endpoint { remote-endpoint = <&dsp_be4_ep>; }; };
			port@5 { dai5_ep: endpoint { remote-endpoint = <&dsp_be5_ep>; }; };
			...
		};
	};


-- Image ---

As Sample, Normal Audio-Graph

CPU <---> Codec (Same as before)

-- DT Sample --

	sound {
		compatible = "audio-graph-card2";

		dais = <&cpu>;
	};

	CPU {
		cpu: port { cpu_ep: endpoint { remote-endpoint = <&codec_ep>; }; };
	};

	Codec {
		codec: port { codec_ep: endpoint { remote-endpoint = <&cpu_ep>; }; };
	};


-- Image ---

Multi-CPU/Codec
(complex connection as sample)

<- multi_CPU_0 ->
           ******
CPU01 <--> *    * <--> Codec0
       \-> *    *
CPU02 <--> *    *
           ******

           <-- multi_codec_1 -->
          ******
CPU1 <--> *    * <--> Codec11
          *    * <-/
          *    * <--> Codec12
          ******

-- DT Image --

   +--+                     +--+
=> |  | <--> CPU01_1     => |  | <--> CPU1
   |  | <--> CPU01_2        |  |
   |  | <--> CPU02          |  | <--> Codec11_1
   |  |                     |  | <--> Codec11_2
   |  | <--> Codec0         |  | <--> Codec12
   +--+                     +--+

-- DT Sample --

	sound {
		compatible = "audio-graph-card2";

		dais = <&cpu_0,
		        &cpu_1>;
	};

	MULTI0 {
		compatible = "audio-graph-card2-multi";

		cpu_0: ports@0 {
			port@0 { fe0_1_ep: endpoint { remote-endpoint = <&cpu01_1_ep>; }; };
			port@1 { fe0_2_ep: endpoint { remote-endpoint = <&cpu01_2_ep>; }; };
			port@2 { fe0_3_ep: endpoint { remote-endpoint = <&cpu02_ep>; }; };
		};
		ports@1 {
			port@0 { be0_ep: endpoint { remote-endpoint = <&codec0_ep>; }; };
		};
	};

	MULTI1 {
		compatible = "audio-graph-card2-multi";

		cpu_1: ports@0 {
			port@0 { fe1_ep: endpoint { remote-endpoint = <&cpu1_ep>; }; };
		};
		ports@1 {
			port@0 { be1_1_ep: endpoint { remote-endpoint = <&codec11_1_ep>; }; };
			port@1 { be1_2_ep: endpoint { remote-endpoint = <&codec11_2_ep>; }; };
			port@2 { be1_3_ep: endpoint { remote-endpoint = <&codec12_ep>; }; };
		};
	};

	CPU01 {
		ports {
			port@0 { cpu01_1_ep: endpoint { remote-endpoint = <&fe0_1_ep>; }; };
			port@1 { cpu01_2_ep: endpoint { remote-endpoint = <&fe0_2_ep>; }; };
			...
		};
	};

	CPU02 {
		ports {
			port@0 { cpu02_ep: endpoint { remote-endpoint = <&fe0_3_ep>; }; };
			...
		};
	};

	CPU1 {
		ports {
			port@0 { cpu1_ep: endpoint { remote-endpoint = <&fe1_ep>; }; };
			...
		};
	};

	Codec0 {
		ports {
			port@0 { codec0_ep: endpoint { remote-endpoint = <&be0_ep>; }; };
			port@1 { ... };
		};
	};

	Codec11 {
		ports {
			port@0 { codec11_1_ep: endpoint { remote-endpoint = <&be1_1_ep>; }; };
			port@1 { codec11_2_ep: endpoint { remote-endpoint = <&be1_2_ep>; }; };
			port@2 { ... };
		};
	};

	Codec12 {
		ports {
			port@0 { codec12_ep: endpoint { remote-endpoint = <&be1_3_ep>; }; };
			port@1 { ... };
		};
	};

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-10-13  4:50 ` Kuninori Morimoto
@ 2020-10-15 14:32   ` Mark Brown
  2020-10-15 23:04     ` Kuninori Morimoto
  0 siblings, 1 reply; 31+ messages in thread
From: Mark Brown @ 2020-10-15 14:32 UTC (permalink / raw)
  To: Kuninori Morimoto; +Cc: Sameer Pujar, Pierre-Louis Bossart, Linux-ALSA

[-- Attachment #1: Type: text/plain, Size: 2208 bytes --]

On Tue, Oct 13, 2020 at 01:50:25PM +0900, Kuninori Morimoto wrote:

> > I'm thinking below style as new audio-graph-card2.
> > It is not tricky, thus connection judgement for normal vs DSP is easy.
> > Then, code can be more readable (= no guaranty :P) ?

> I have proposed DPCM connection idea for audio-card2 before,
> and proposed Multi-CPU/Codec connection idea later.
> I'm not yet implemented these, but I want to merge / reuse these
> idea to keep simple code.

> By this mail, I want to propose Codec2Codec idea.
> It is almost same idea with DPCM, Multi-CPU/Codec,
> and is last parts for missing feature.

> I think this idea can be generic and easy to update
> in the future (if needed).

> But, I want feedback, opinion.
> I wonder does it works for your use-case ?

> -- Image ---
> 
> Codec2Codec
> 
> Codec1 <--> Codec2
> 
> -- DT Image --
> 
>    +--+
> => |  |<--> Codec1
>    |  |<--> Codec2
>    +--+
> 
> -- DT Sample --
> 
> 	sound {
> 		compatible = "audio-graph-card2";
> 
> 		dais = <&codec2codec>;
> 	};
> 
> 	CODEC2CODEC {
> 		compatible = "audio-graph-card2-codec2codec";
> 
> 		ports {
> 			codec2codec: port@0 { fe_ep: endpoint { remote-endpoint = <&codec1_ep>; }; };
> 			             port@1 { be_ep: endpoint { remote-endpoint = <&codec2_ep>; }; };
> 		};
> 	};

I'm a bit confused about the use of FE and BE here since the two CODECs
are peers rather than this being DPCM - I think that's just a bit
cut'n'paste rather than anything else though?  The other examples look
sensible I think but this one feels odd.

> -- Image ---
> 
>           *******
> PCM0 <--> *     * <--> DAI0: Codec Headset
> PCM1 <--> *     * <--> DAI1: Codec Speakers
> PCM2 <--> * DSP * <--> DAI2: MODEM
> PCM3 <--> *     * <--> DAI3: BT
>           *     * <--> DAI4: DMIC
>           *     * <--> DAI5: FM
>           *******

> 	DSP {
> 		compatible = "audio-graph-card2-dsp";
> 
> 		ports@0 {
> 			/* Front-End is pcm0_ep, Back-End is dummy */
> 			dsp_fe0: port@0 { dsp_fe0_ep: endpoint { remote-endpoint = <&pcm0_ep>; }; };
> 			         port@1 { };
> 		};

I'm not clear why we need to represent the dummy port here?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: More Generic Audio Graph Sound Card idea
  2020-10-15 14:32   ` Mark Brown
@ 2020-10-15 23:04     ` Kuninori Morimoto
  0 siblings, 0 replies; 31+ messages in thread
From: Kuninori Morimoto @ 2020-10-15 23:04 UTC (permalink / raw)
  To: Mark Brown; +Cc: Sameer Pujar, Pierre-Louis Bossart, Linux-ALSA


Hi Mark

Thank you for feedback

> > -- DT Image --
> > 
> >    +--+
> > => |  |<--> Codec1
> >    |  |<--> Codec2
> >    +--+
> > 
> > -- DT Sample --
> > 
> > 	sound {
> > 		compatible = "audio-graph-card2";
> > 
> > 		dais = <&codec2codec>;
> > 	};
> > 
> > 	CODEC2CODEC {
> > 		compatible = "audio-graph-card2-codec2codec";
> > 
> > 		ports {
> > 			codec2codec: port@0 { fe_ep: endpoint { remote-endpoint = <&codec1_ep>; }; };
> > 			             port@1 { be_ep: endpoint { remote-endpoint = <&codec2_ep>; }; };
> > 		};
> > 	};
> 
> I'm a bit confused about the use of FE and BE here since the two CODECs
> are peers rather than this being DPCM - I think that's just a bit
> cut'n'paste rather than anything else though?  The other examples look
> sensible I think but this one feels odd.

FE here is indicating CPU part's Codec,
BE is for Codec part's Codec.

> > -- Image ---
> > 
> >           *******
> > PCM0 <--> *     * <--> DAI0: Codec Headset
> > PCM1 <--> *     * <--> DAI1: Codec Speakers
> > PCM2 <--> * DSP * <--> DAI2: MODEM
> > PCM3 <--> *     * <--> DAI3: BT
> >           *     * <--> DAI4: DMIC
> >           *     * <--> DAI5: FM
> >           *******
> 
> > 	DSP {
> > 		compatible = "audio-graph-card2-dsp";
> > 
> > 		ports@0 {
> > 			/* Front-End is pcm0_ep, Back-End is dummy */
> > 			dsp_fe0: port@0 { dsp_fe0_ep: endpoint { remote-endpoint = <&pcm0_ep>; }; };
> > 			         port@1 { };
> > 		};
> 
> I'm not clear why we need to represent the dummy port here?

I'm thining that
port@0 is IN (= CPU), port@1 is OUT (= Codec).
Empty endpoint means not yet connected (= dummy).

It is difficut to know it was CPU-dummy or dummy-Codec without it.

Thank you for your help !!

Best regards
---
Kuninori Morimoto

^ permalink raw reply	[flat|nested] 31+ messages in thread

end of thread, other threads:[~2020-10-15 23:05 UTC | newest]

Thread overview: 31+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-08-21  4:15 More Generic Audio Graph Sound Card idea Kuninori Morimoto
2020-08-21  5:26 ` Sameer Pujar
2020-08-21  7:14   ` Kuninori Morimoto
2020-08-21  8:28     ` Sameer Pujar
2020-08-21 13:02       ` Mark Brown
2020-09-25  1:43     ` Kuninori Morimoto
2020-09-25 19:22       ` Mark Brown
2020-09-25 20:04         ` Pierre-Louis Bossart
2020-09-25 20:10           ` Mark Brown
2020-09-25 20:50             ` Pierre-Louis Bossart
2020-08-21  7:11 ` Daniel Baluta
2020-08-21  7:25   ` Kuninori Morimoto
2020-08-21  7:33     ` Daniel Baluta
2020-08-21 11:47     ` Mark Brown
2020-08-21 12:18 ` Mark Brown
2020-08-24  0:25   ` Kuninori Morimoto
2020-08-24  6:25     ` Sameer Pujar
2020-08-25  0:59       ` Kuninori Morimoto
2020-08-25  3:11         ` Sameer Pujar
2020-08-25  5:13           ` Kuninori Morimoto
2020-08-25  5:42             ` Sameer Pujar
2020-08-25  6:35               ` Kuninori Morimoto
2020-08-26  6:46                 ` Sameer Pujar
2020-08-27  1:18                   ` Kuninori Morimoto
2020-08-27  1:36                   ` Kuninori Morimoto
2020-09-03 23:51     ` Kuninori Morimoto
2020-09-09 11:33       ` Sameer Pujar
2020-08-21 15:49 ` Pierre-Louis Bossart
2020-10-13  4:50 ` Kuninori Morimoto
2020-10-15 14:32   ` Mark Brown
2020-10-15 23:04     ` Kuninori Morimoto

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).