Linux-ARM-Kernel Archive on lore.kernel.org
 help / color / Atom feed
* Re: Best representation for TDM devices
       [not found] <CAMTEBE0mPY=44txPC2G+Cb=NqeH6Cqesar9Vkq0afXBv4dUWHQ@mail.gmail.com>
@ 2019-08-01 11:48 ` Mark Brown
  2019-08-14 10:53   ` Ali Burak Parım
  0 siblings, 1 reply; 8+ messages in thread
From: Mark Brown @ 2019-08-01 11:48 UTC (permalink / raw)
  To: Ali Burak Parım; +Cc: maxime.ripard, linux-arm-kernel

[-- Attachment #1.1: Type: text/plain, Size: 910 bytes --]

On Thu, Aug 01, 2019 at 01:43:06PM +0200, Ali Burak Parım wrote:
> On Thu, Jul 25, 2019 at 03:01:12PM +0200, Maxime Ripard wrote:

> > What is the actual hardware - what do the microphones physically look
> > like in the system, what functional role do they play?  My initial guess

> Hardware is a custom board I designed with 4 PDM output microphones
> and 2 adau7002 devices as the codec for PDM-to-PCM. We want to do
> signal processing with this board. Therefore having separate streams for
> each microphone is crucial to the application though I am not sure where
> we should implement this exactly.

What is this processing - are the streams from these microphones
logically related in any way (eg, is this a microphone array)?  There's
probably going to be some overlap in the input signals at least.  If you
need to for example correlate different microphones then that's
relevant.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Best representation for TDM devices
  2019-08-01 11:48 ` Best representation for TDM devices Mark Brown
@ 2019-08-14 10:53   ` Ali Burak Parım
  2019-08-14 16:02     ` Mark Brown
  0 siblings, 1 reply; 8+ messages in thread
From: Ali Burak Parım @ 2019-08-14 10:53 UTC (permalink / raw)
  To: Mark Brown; +Cc: maxime.ripard, linux-arm-kernel

Hi,

On Thu, 1 Aug 2019 at 13:48, Mark Brown <broonie@kernel.org> wrote:
>
> On Thu, Aug 01, 2019 at 01:43:06PM +0200, Ali Burak Parım wrote:
> > On Thu, Jul 25, 2019 at 03:01:12PM +0200, Maxime Ripard wrote:
>
> > > What is the actual hardware - what do the microphones physically look
> > > like in the system, what functional role do they play?  My initial guess
>
> > Hardware is a custom board I designed with 4 PDM output microphones
> > and 2 adau7002 devices as the codec for PDM-to-PCM. We want to do
> > signal processing with this board. Therefore having separate streams for
> > each microphone is crucial to the application though I am not sure where
> > we should implement this exactly.
>
> What is this processing - are the streams from these microphones
> logically related in any way (eg, is this a microphone array)?  There's
Yes, it is a microphone array application for speech enhancement. Thus,
signal levels and physical time delays are important.
> probably going to be some overlap in the input signals at least.  If you
> need to for example correlate different microphones then that's
> relevant.
Yes, we correlate different microphone signals in some of our algorithms.

P.S.: I am so sorry for late reply, the thread went to my junk box somehow.

Best regards,
Burak


--

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Best representation for TDM devices
  2019-08-14 10:53   ` Ali Burak Parım
@ 2019-08-14 16:02     ` Mark Brown
  2019-08-19  8:41       ` Maxime Ripard
  0 siblings, 1 reply; 8+ messages in thread
From: Mark Brown @ 2019-08-14 16:02 UTC (permalink / raw)
  To: Ali Burak Parım; +Cc: maxime.ripard, linux-arm-kernel

[-- Attachment #1.1: Type: text/plain, Size: 1458 bytes --]

On Wed, Aug 14, 2019 at 12:53:24PM +0200, Ali Burak Parım wrote:
> On Thu, 1 Aug 2019 at 13:48, Mark Brown <broonie@kernel.org> wrote:
> > On Thu, Aug 01, 2019 at 01:43:06PM +0200, Ali Burak Parım wrote:
> > > On Thu, Jul 25, 2019 at 03:01:12PM +0200, Maxime Ripard wrote:

> > > Hardware is a custom board I designed with 4 PDM output microphones
> > > and 2 adau7002 devices as the codec for PDM-to-PCM. We want to do
> > > signal processing with this board. Therefore having separate streams for
> > > each microphone is crucial to the application though I am not sure where
> > > we should implement this exactly.

> > What is this processing - are the streams from these microphones
> > logically related in any way (eg, is this a microphone array)?  There's

> Yes, it is a microphone array application for speech enhancement. Thus,
> signal levels and physical time delays are important.

> > probably going to be some overlap in the input signals at least.  If you
> > need to for example correlate different microphones then that's
> > relevant.

> Yes, we correlate different microphone signals in some of our algorithms.

OK, in that case I'd recommend providing them to userspace as a single
four channel stream - keeping everything bundled together as long as
possible to make it easier to keep the processing synced up.

> P.S.: I am so sorry for late reply, the thread went to my junk box somehow.

No problem.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Best representation for TDM devices
  2019-08-14 16:02     ` Mark Brown
@ 2019-08-19  8:41       ` Maxime Ripard
  2019-08-19 15:45         ` Mark Brown
  0 siblings, 1 reply; 8+ messages in thread
From: Maxime Ripard @ 2019-08-19  8:41 UTC (permalink / raw)
  To: Mark Brown; +Cc: Ali Burak Parım, linux-arm-kernel

[-- Attachment #1.1: Type: text/plain, Size: 2015 bytes --]

Hi Mark,

On Wed, Aug 14, 2019 at 05:02:34PM +0100, Mark Brown wrote:
> On Wed, Aug 14, 2019 at 12:53:24PM +0200, Ali Burak Parım wrote:
> > On Thu, 1 Aug 2019 at 13:48, Mark Brown <broonie@kernel.org> wrote:
> > > On Thu, Aug 01, 2019 at 01:43:06PM +0200, Ali Burak Parım wrote:
> > > > On Thu, Jul 25, 2019 at 03:01:12PM +0200, Maxime Ripard wrote:
>
> > > > Hardware is a custom board I designed with 4 PDM output microphones
> > > > and 2 adau7002 devices as the codec for PDM-to-PCM. We want to do
> > > > signal processing with this board. Therefore having separate streams for
> > > > each microphone is crucial to the application though I am not sure where
> > > > we should implement this exactly.
>
> > > What is this processing - are the streams from these microphones
> > > logically related in any way (eg, is this a microphone array)?  There's
>
> > Yes, it is a microphone array application for speech enhancement. Thus,
> > signal levels and physical time delays are important.
>
> > > probably going to be some overlap in the input signals at least.  If you
> > > need to for example correlate different microphones then that's
> > > relevant.
>
> > Yes, we correlate different microphone signals in some of our algorithms.
>
> OK, in that case I'd recommend providing them to userspace as a single
> four channel stream - keeping everything bundled together as long as
> possible to make it easier to keep the processing synced up.

Ok, that's what I had in mind as well :)

However, it looks like we can only capture as many channels as the max
being exposed by the codec on the link?

Any attempt at capturing something with 4 channels here using arecord
was either reduced to two channels (the amount of channels provided by
the adau7002 driver), or just refused by the ALSA core.

Is there anything that we need to configure / work on to enable this?

Maxime

--
Maxime Ripard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Best representation for TDM devices
  2019-08-19  8:41       ` Maxime Ripard
@ 2019-08-19 15:45         ` Mark Brown
  2019-08-20 14:06           ` Maxime Ripard
  0 siblings, 1 reply; 8+ messages in thread
From: Mark Brown @ 2019-08-19 15:45 UTC (permalink / raw)
  To: Maxime Ripard; +Cc: Ali Burak Parım, linux-arm-kernel

[-- Attachment #1.1: Type: text/plain, Size: 977 bytes --]

On Mon, Aug 19, 2019 at 10:41:41AM +0200, Maxime Ripard wrote:
> On Wed, Aug 14, 2019 at 05:02:34PM +0100, Mark Brown wrote:

> > OK, in that case I'd recommend providing them to userspace as a single
> > four channel stream - keeping everything bundled together as long as
> > possible to make it easier to keep the processing synced up.

> Ok, that's what I had in mind as well :)

> However, it looks like we can only capture as many channels as the max
> being exposed by the codec on the link?

> Any attempt at capturing something with 4 channels here using arecord
> was either reduced to two channels (the amount of channels provided by
> the adau7002 driver), or just refused by the ALSA core.

> Is there anything that we need to configure / work on to enable this?

I'd expect there's some issue in the capability/constraint matching code
that needs fixing - that should work but I can totally see it not
working unless someone goes in and does something to fix it.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Best representation for TDM devices
  2019-08-19 15:45         ` Mark Brown
@ 2019-08-20 14:06           ` Maxime Ripard
  0 siblings, 0 replies; 8+ messages in thread
From: Maxime Ripard @ 2019-08-20 14:06 UTC (permalink / raw)
  To: Mark Brown; +Cc: Ali Burak Parım, linux-arm-kernel

[-- Attachment #1.1: Type: text/plain, Size: 1197 bytes --]

On Mon, Aug 19, 2019 at 04:45:14PM +0100, Mark Brown wrote:
> On Mon, Aug 19, 2019 at 10:41:41AM +0200, Maxime Ripard wrote:
> > On Wed, Aug 14, 2019 at 05:02:34PM +0100, Mark Brown wrote:
>
> > > OK, in that case I'd recommend providing them to userspace as a single
> > > four channel stream - keeping everything bundled together as long as
> > > possible to make it easier to keep the processing synced up.
>
> > Ok, that's what I had in mind as well :)
>
> > However, it looks like we can only capture as many channels as the max
> > being exposed by the codec on the link?
>
> > Any attempt at capturing something with 4 channels here using arecord
> > was either reduced to two channels (the amount of channels provided by
> > the adau7002 driver), or just refused by the ALSA core.
>
> > Is there anything that we need to configure / work on to enable this?
>
> I'd expect there's some issue in the capability/constraint matching code
> that needs fixing - that should work but I can totally see it not
> working unless someone goes in and does something to fix it.

Ok, I'll have a look, thanks!
Maxime


--
Maxime Ripard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: Best representation for TDM devices
  2019-07-25 13:01 Maxime Ripard
@ 2019-07-25 17:44 ` Mark Brown
  0 siblings, 0 replies; 8+ messages in thread
From: Mark Brown @ 2019-07-25 17:44 UTC (permalink / raw)
  To: Maxime Ripard
  Cc: alsa-devel, Alexandre Belloni, Takashi Iwai, Liam Girdwood,
	Chen-Yu Tsai, Thomas Petazzoni, Jaroslav Kysela,
	linux-arm-kernel

[-- Attachment #1.1: Type: text/plain, Size: 1091 bytes --]

On Thu, Jul 25, 2019 at 03:01:12PM +0200, Maxime Ripard wrote:

> Since I have two codecs, I have two links (described through
> simple-card) and thus two subdevices. However, the microphones
> connected are really separate streams, so ideally (I guess?) we should
> have 1 subdevice per microphone.

> An alternative would be that we capture the whole 4 channels and use
> dsnoop to demux them, but trying to use the device directly (using
> arecord -D hw:0) results in the ioctl to change the number of channels
> being rejected, I'm assuming because each codec only has 2 channels?

> So I'm a bit lost at this point about how I should go and expose
> this. I couldn't really find any similar setup either in the
> kernel. Any preferred way of doing this?

What is the actual hardware - what do the microphones physically look
like in the system, what functional role do they play?  My initial guess
would be to combine them into a single PCM for userspace, there's some
support for that but it's not surprising if that's broken somewhere
along the line as it's very infrequently tested.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Best representation for TDM devices
@ 2019-07-25 13:01 Maxime Ripard
  2019-07-25 17:44 ` Mark Brown
  0 siblings, 1 reply; 8+ messages in thread
From: Maxime Ripard @ 2019-07-25 13:01 UTC (permalink / raw)
  To: alsa-devel
  Cc: Alexandre Belloni, Takashi Iwai, Liam Girdwood, Chen-Yu Tsai,
	Mark Brown, Thomas Petazzoni, Jaroslav Kysela, linux-arm-kernel

[-- Attachment #1.1: Type: text/plain, Size: 1084 bytes --]

Hi,

I'm currently working on a device that has two Analog's ADAU7002
connected to an I2S/TDM Controller (sound/soc/sunxi/sun4i-i2s.c, with
some modifications), over the slots 1/2 and 3/4.

The capture itself is working fine, however I'm not quite sure about
the way to expose those streams to the userspace.

Since I have two codecs, I have two links (described through
simple-card) and thus two subdevices. However, the microphones
connected are really separate streams, so ideally (I guess?) we should
have 1 subdevice per microphone.

An alternative would be that we capture the whole 4 channels and use
dsnoop to demux them, but trying to use the device directly (using
arecord -D hw:0) results in the ioctl to change the number of channels
being rejected, I'm assuming because each codec only has 2 channels?

So I'm a bit lost at this point about how I should go and expose
this. I couldn't really find any similar setup either in the
kernel. Any preferred way of doing this?

Thanks!
Maxime

--
Maxime Ripard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, back to index

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <CAMTEBE0mPY=44txPC2G+Cb=NqeH6Cqesar9Vkq0afXBv4dUWHQ@mail.gmail.com>
2019-08-01 11:48 ` Best representation for TDM devices Mark Brown
2019-08-14 10:53   ` Ali Burak Parım
2019-08-14 16:02     ` Mark Brown
2019-08-19  8:41       ` Maxime Ripard
2019-08-19 15:45         ` Mark Brown
2019-08-20 14:06           ` Maxime Ripard
2019-07-25 13:01 Maxime Ripard
2019-07-25 17:44 ` Mark Brown

Linux-ARM-Kernel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-arm-kernel/0 linux-arm-kernel/git/0.git
	git clone --mirror https://lore.kernel.org/linux-arm-kernel/1 linux-arm-kernel/git/1.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-arm-kernel linux-arm-kernel/ https://lore.kernel.org/linux-arm-kernel \
		linux-arm-kernel@lists.infradead.org infradead-linux-arm-kernel@archiver.kernel.org
	public-inbox-index linux-arm-kernel


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.infradead.lists.linux-arm-kernel


AGPL code for this site: git clone https://public-inbox.org/ public-inbox