Re: Best representation for TDM devices

Re: Best representation for TDM devices

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 910 bytes --]

On Thu, Aug 01, 2019 at 01:43:06PM +0200, Ali Burak Parım wrote:
> On Thu, Jul 25, 2019 at 03:01:12PM +0200, Maxime Ripard wrote:

> > What is the actual hardware - what do the microphones physically look
> > like in the system, what functional role do they play?  My initial guess

> Hardware is a custom board I designed with 4 PDM output microphones
> and 2 adau7002 devices as the codec for PDM-to-PCM. We want to do
> signal processing with this board. Therefore having separate streams for
> each microphone is crucial to the application though I am not sure where
> we should implement this exactly.

What is this processing - are the streams from these microphones
logically related in any way (eg, is this a microphone array)?  There's
probably going to be some overlap in the input signals at least.  If you
need to for example correlate different microphones then that's
relevant.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Best representation for TDM devices

[Ali Burak Parım]

Hi,

On Thu, 1 Aug 2019 at 13:48, Mark Brown <broonie@kernel.org> wrote:
>
> On Thu, Aug 01, 2019 at 01:43:06PM +0200, Ali Burak Parım wrote:
> > On Thu, Jul 25, 2019 at 03:01:12PM +0200, Maxime Ripard wrote:
>
> > > What is the actual hardware - what do the microphones physically look
> > > like in the system, what functional role do they play?  My initial guess
>
> > Hardware is a custom board I designed with 4 PDM output microphones
> > and 2 adau7002 devices as the codec for PDM-to-PCM. We want to do
> > signal processing with this board. Therefore having separate streams for
> > each microphone is crucial to the application though I am not sure where
> > we should implement this exactly.
>
> What is this processing - are the streams from these microphones
> logically related in any way (eg, is this a microphone array)?  There's
Yes, it is a microphone array application for speech enhancement. Thus,
signal levels and physical time delays are important.
> probably going to be some overlap in the input signals at least.  If you
> need to for example correlate different microphones then that's
> relevant.
Yes, we correlate different microphone signals in some of our algorithms.

P.S.: I am so sorry for late reply, the thread went to my junk box somehow.

Best regards,
Burak


--

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Best representation for TDM devices

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 1458 bytes --]

On Wed, Aug 14, 2019 at 12:53:24PM +0200, Ali Burak Parım wrote:
> On Thu, 1 Aug 2019 at 13:48, Mark Brown <broonie@kernel.org> wrote:
> > On Thu, Aug 01, 2019 at 01:43:06PM +0200, Ali Burak Parım wrote:
> > > On Thu, Jul 25, 2019 at 03:01:12PM +0200, Maxime Ripard wrote:

> > > Hardware is a custom board I designed with 4 PDM output microphones
> > > and 2 adau7002 devices as the codec for PDM-to-PCM. We want to do
> > > signal processing with this board. Therefore having separate streams for
> > > each microphone is crucial to the application though I am not sure where
> > > we should implement this exactly.

> > What is this processing - are the streams from these microphones
> > logically related in any way (eg, is this a microphone array)?  There's

> Yes, it is a microphone array application for speech enhancement. Thus,
> signal levels and physical time delays are important.

> > probably going to be some overlap in the input signals at least.  If you
> > need to for example correlate different microphones then that's
> > relevant.

> Yes, we correlate different microphone signals in some of our algorithms.

OK, in that case I'd recommend providing them to userspace as a single
four channel stream - keeping everything bundled together as long as
possible to make it easier to keep the processing synced up.

> P.S.: I am so sorry for late reply, the thread went to my junk box somehow.

No problem.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Best representation for TDM devices

[Maxime Ripard]

[-- Attachment #1.1: Type: text/plain, Size: 2015 bytes --]

Hi Mark,

On Wed, Aug 14, 2019 at 05:02:34PM +0100, Mark Brown wrote:
> On Wed, Aug 14, 2019 at 12:53:24PM +0200, Ali Burak Parım wrote:
> > On Thu, 1 Aug 2019 at 13:48, Mark Brown <broonie@kernel.org> wrote:
> > > On Thu, Aug 01, 2019 at 01:43:06PM +0200, Ali Burak Parım wrote:
> > > > On Thu, Jul 25, 2019 at 03:01:12PM +0200, Maxime Ripard wrote:
>
> > > > Hardware is a custom board I designed with 4 PDM output microphones
> > > > and 2 adau7002 devices as the codec for PDM-to-PCM. We want to do
> > > > signal processing with this board. Therefore having separate streams for
> > > > each microphone is crucial to the application though I am not sure where
> > > > we should implement this exactly.
>
> > > What is this processing - are the streams from these microphones
> > > logically related in any way (eg, is this a microphone array)?  There's
>
> > Yes, it is a microphone array application for speech enhancement. Thus,
> > signal levels and physical time delays are important.
>
> > > probably going to be some overlap in the input signals at least.  If you
> > > need to for example correlate different microphones then that's
> > > relevant.
>
> > Yes, we correlate different microphone signals in some of our algorithms.
>
> OK, in that case I'd recommend providing them to userspace as a single
> four channel stream - keeping everything bundled together as long as
> possible to make it easier to keep the processing synced up.

Ok, that's what I had in mind as well :)

However, it looks like we can only capture as many channels as the max
being exposed by the codec on the link?

Any attempt at capturing something with 4 channels here using arecord
was either reduced to two channels (the amount of channels provided by
the adau7002 driver), or just refused by the ALSA core.

Is there anything that we need to configure / work on to enable this?

Maxime

--
Maxime Ripard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Best representation for TDM devices

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 977 bytes --]

On Mon, Aug 19, 2019 at 10:41:41AM +0200, Maxime Ripard wrote:
> On Wed, Aug 14, 2019 at 05:02:34PM +0100, Mark Brown wrote:

> > OK, in that case I'd recommend providing them to userspace as a single
> > four channel stream - keeping everything bundled together as long as
> > possible to make it easier to keep the processing synced up.

> Ok, that's what I had in mind as well :)

> However, it looks like we can only capture as many channels as the max
> being exposed by the codec on the link?

> Any attempt at capturing something with 4 channels here using arecord
> was either reduced to two channels (the amount of channels provided by
> the adau7002 driver), or just refused by the ALSA core.

> Is there anything that we need to configure / work on to enable this?

I'd expect there's some issue in the capability/constraint matching code
that needs fixing - that should work but I can totally see it not
working unless someone goes in and does something to fix it.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Best representation for TDM devices

[Maxime Ripard]

[-- Attachment #1.1: Type: text/plain, Size: 1197 bytes --]

On Mon, Aug 19, 2019 at 04:45:14PM +0100, Mark Brown wrote:
> On Mon, Aug 19, 2019 at 10:41:41AM +0200, Maxime Ripard wrote:
> > On Wed, Aug 14, 2019 at 05:02:34PM +0100, Mark Brown wrote:
>
> > > OK, in that case I'd recommend providing them to userspace as a single
> > > four channel stream - keeping everything bundled together as long as
> > > possible to make it easier to keep the processing synced up.
>
> > Ok, that's what I had in mind as well :)
>
> > However, it looks like we can only capture as many channels as the max
> > being exposed by the codec on the link?
>
> > Any attempt at capturing something with 4 channels here using arecord
> > was either reduced to two channels (the amount of channels provided by
> > the adau7002 driver), or just refused by the ALSA core.
>
> > Is there anything that we need to configure / work on to enable this?
>
> I'd expect there's some issue in the capability/constraint matching code
> that needs fixing - that should work but I can totally see it not
> working unless someone goes in and does something to fix it.

Ok, I'll have a look, thanks!
Maxime


--
Maxime Ripard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Best representation for TDM devices

[Maxime Ripard]

[-- Attachment #1.1: Type: text/plain, Size: 1084 bytes --]

Hi,

I'm currently working on a device that has two Analog's ADAU7002
connected to an I2S/TDM Controller (sound/soc/sunxi/sun4i-i2s.c, with
some modifications), over the slots 1/2 and 3/4.

The capture itself is working fine, however I'm not quite sure about
the way to expose those streams to the userspace.

Since I have two codecs, I have two links (described through
simple-card) and thus two subdevices. However, the microphones
connected are really separate streams, so ideally (I guess?) we should
have 1 subdevice per microphone.

An alternative would be that we capture the whole 4 channels and use
dsnoop to demux them, but trying to use the device directly (using
arecord -D hw:0) results in the ioctl to change the number of channels
being rejected, I'm assuming because each codec only has 2 channels?

So I'm a bit lost at this point about how I should go and expose
this. I couldn't really find any similar setup either in the
kernel. Any preferred way of doing this?

Thanks!
Maxime

--
Maxime Ripard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: Best representation for TDM devices

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 1091 bytes --]

On Thu, Jul 25, 2019 at 03:01:12PM +0200, Maxime Ripard wrote:

> Since I have two codecs, I have two links (described through
> simple-card) and thus two subdevices. However, the microphones
> connected are really separate streams, so ideally (I guess?) we should
> have 1 subdevice per microphone.

> An alternative would be that we capture the whole 4 channels and use
> dsnoop to demux them, but trying to use the device directly (using
> arecord -D hw:0) results in the ioctl to change the number of channels
> being rejected, I'm assuming because each codec only has 2 channels?

> So I'm a bit lost at this point about how I should go and expose
> this. I couldn't really find any similar setup either in the
> kernel. Any preferred way of doing this?

What is the actual hardware - what do the microphones physically look
like in the system, what functional role do they play?  My initial guess
would be to combine them into a single PCM for userspace, there's some
support for that but it's not surprising if that's broken somewhere
along the line as it's very infrequently tested.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel