RFC: usb: gadget: u_audio: Notifying gadget that host started playback/capture?

RFC: usb: gadget: u_audio: Notifying gadget that host started playback/capture?

[Pavel Hofman]

Hi,

The current audio gadget has no way to inform the gadget side that the 
host side has started playback/capture and that gadget-side alsa 
processes should be started.

Playback/capture processes on the host side do not get stuck without the 
gadget side consuming/producing data (OUT requests are ignored in 
u_audio_iso_complete, IN ones send initial zeros in their req->buf).

However, playback/capture processes on the gadget side get stuck without 
the host side sending playback OUT packets or capture IN requests and 
time out with error. If there was a way to inform the gadget side that 
playback/capture has started on the host side, the gadget clients could 
react accordingly.

I have been trying to investigate the packet flow/behavior in 
u_audio:u_audio_iso_complete, u_audio_start_capture/playback, 
u_audio_stop_capture/playback but could not find a pattern which would 
recognize that playback or capture has started. I see incoming OUT and 
IN requests shortly after gadget enumeration. Also before starting 
playback/capture both _start_ and _stop_ method are called. I guess the 
actual sequence will be specific for every UAC2, maybe even for every 
player/capture app on the host.

Technically a boolean alsa ctrl could be used for each direction 
(Capture Running/Playback Running, resp. specific rate or zero in 
Capture Rate/Playback Rate in Julian's multiple rates patches I am 
preparing), with the gadget client registering for notifications.But 
detecting when to set the ctrl value is the key.

Thanks a lot for ideas and recommendations.

Pavel.

Re: RFC: usb: gadget: u_audio: Notifying gadget that host started playback/capture?

[Pavel Hofman]

[-- Attachment #1: Type: text/plain, Size: 2097 bytes --]

Hi,

Dne 08. 09. 21 v 10:21 Pavel Hofman napsal(a):
> Hi,
> 
> The current audio gadget has no way to inform the gadget side that the 
> host side has started playback/capture and that gadget-side alsa 
> processes should be started.
> 
> Playback/capture processes on the host side do not get stuck without the 
> gadget side consuming/producing data (OUT requests are ignored in 
> u_audio_iso_complete, IN ones send initial zeros in their req->buf).
> 
> However, playback/capture processes on the gadget side get stuck without 
> the host side sending playback OUT packets or capture IN requests and 
> time out with error. If there was a way to inform the gadget side that 
> playback/capture has started on the host side, the gadget clients could 
> react accordingly.
> 

I drafted a simple patch for u_audio.c which defines read-only boolean 
ctl elems "Capture Requested" and "Playback Requested". Their values are 
set/reset in methods u_audio_start_capture/playback and 
u_audio_stop_capture/playback, i.e. at changes of respective altsettings 
from 0 to 1 and back. Every ctl elem value change sends notification via 
snd_ctl_notify. The principle works OK for capture/playback start/stop 
on the host, as monitored by alsactl:

pi@raspberrypi:~ $ alsactl monitor hw:UAC2Gadget
node hw:UAC2Gadget, #4 (3,0,0,Capture Requested,0) VALUE
node hw:UAC2Gadget, #4 (3,0,0,Capture Requested,0) VALUE
node hw:UAC2Gadget, #3 (3,0,0,Playback Requested,0) VALUE
node hw:UAC2Gadget, #3 (3,0,0,Playback Requested,0) VALUE

However at enumeration the USB host switches both playback and capture 
altsettings repeatedly, generating "fake" events from the gadget side 
POW. The host even sends regular-sized EP-OUT packets filled with zeros 
during enumeration (tested on linux only for now).

Please is there any way to "detect" the enumeration stage to mask out 
the "fake" playback/capture start/stop events?

The attached patch does not apply cleanly to mainline u_audio.c because 
it's rebased on other patches not submitted yet but it's only a 
discussion inducer for now.

Thanks a lot,

Pavel.

[-- Attachment #2: patch.diff --]
[-- Type: text/x-patch, Size: 5116 bytes --]

diff --git a/drivers/usb/gadget/function/u_audio.c b/drivers/usb/gadget/function/u_audio.c
index e395be32275c..69aef4e3677d 100644
--- a/drivers/usb/gadget/function/u_audio.c
+++ b/drivers/usb/gadget/function/u_audio.c
@@ -34,6 +34,8 @@ enum {
 	UAC_VOLUME_CTRL,
 	UAC_CAPTURE_RATE_CTRL,
 	UAC_PLAYBACK_RATE_CTRL,
+	UAC_CAPTURE_REQ_CTRL,
+	UAC_PLAYBACK_REQ_CTRL,
 };
 
 /* Runtime data params for one stream */
@@ -63,6 +65,9 @@ struct uac_rtd_params {
   s16 volume_min, volume_max, volume_res;
   s16 volume;
   int mute;
+  /* playback/capture_is_requested and their state */
+  struct snd_kcontrol *snd_kctl_is_req;
+  int is_requested;
 
   spinlock_t lock; /* lock for control transfers */
 
@@ -542,6 +547,8 @@ int u_audio_set_playback_srate(struct g_audio *audio_dev, int srate)
 }
 EXPORT_SYMBOL_GPL(u_audio_set_playback_srate);
 
+static void update_is_requested(struct uac_rtd_params *prm, unsigned int val);
+
 int u_audio_start_capture(struct g_audio *audio_dev)
 {
 	struct snd_uac_chip *uac = audio_dev->uac;
@@ -553,6 +560,7 @@ int u_audio_start_capture(struct g_audio *audio_dev)
 	struct uac_params *params = &audio_dev->params;
 	int req_len, i;
 
+	update_is_requested(&uac->c_prm, 1);
 	ep = audio_dev->out_ep;
 	prm = &uac->c_prm;
 	config_ep_by_speed(gadget, &audio_dev->func, ep);
@@ -625,6 +633,7 @@ void u_audio_stop_capture(struct g_audio *audio_dev)
 {
 	struct snd_uac_chip *uac = audio_dev->uac;
 
+	update_is_requested(&uac->c_prm, 0);
 	if (audio_dev->in_ep_fback)
 		free_ep_fback(&uac->c_prm, audio_dev->in_ep_fback);
 	free_ep(&uac->c_prm, audio_dev->out_ep);
@@ -645,6 +654,7 @@ int u_audio_start_playback(struct g_audio *audio_dev)
 	int req_len, i;
 	unsigned int p_interval, p_pktsize, p_pktsize_residue;
 
+	update_is_requested(&uac->p_prm, 1);
 	dev_dbg(dev, "start playback with rate %d\n", params->p_srate_active);
 	ep = audio_dev->in_ep;
 	prm = &uac->p_prm;
@@ -711,6 +721,7 @@ void u_audio_stop_playback(struct g_audio *audio_dev)
 {
 	struct snd_uac_chip *uac = audio_dev->uac;
 
+	update_is_requested(&uac->p_prm, 0);
 	free_ep(&uac->p_prm, audio_dev->in_ep);
 }
 EXPORT_SYMBOL_GPL(u_audio_stop_playback);
@@ -1027,6 +1038,27 @@ static int uac_pcm_rate_get(struct snd_kcontrol *kcontrol,
 	return 0;
 }
 
+static int u_audio_stream_requested_info(struct snd_kcontrol *kcontrol,
+				   struct snd_ctl_elem_info *uinfo)
+{
+	uinfo->type = SNDRV_CTL_ELEM_TYPE_BOOLEAN;
+	uinfo->count = 1;
+	uinfo->value.integer.min = 0;
+	uinfo->value.integer.max = 1;
+	uinfo->value.integer.step = 1;
+
+	return 0;
+}
+
+static int u_audio_stream_requested_get(struct snd_kcontrol *kcontrol,
+				   struct snd_ctl_elem_value *ucontrol)
+{
+	struct uac_rtd_params *prm = snd_kcontrol_chip(kcontrol);
+	ucontrol->value.integer.value[0] = prm->is_requested;
+	return 0;
+}
+
+
 static struct snd_kcontrol_new u_audio_controls[]  = {
   [UAC_FBACK_CTRL] {
     .iface =        SNDRV_CTL_ELEM_IFACE_PCM,
@@ -1072,8 +1104,35 @@ static struct snd_kcontrol_new u_audio_controls[]  = {
 		.get = uac_pcm_rate_get,
 		.private_value = SNDRV_PCM_STREAM_PLAYBACK,
 		},
+  [UAC_CAPTURE_REQ_CTRL] {
+    .iface =        SNDRV_CTL_ELEM_IFACE_PCM,
+    .name =         "Capture Requested",
+    .access = SNDRV_CTL_ELEM_ACCESS_READ | SNDRV_CTL_ELEM_ACCESS_VOLATILE,
+    .info =         u_audio_stream_requested_info,
+    .get =          u_audio_stream_requested_get,
+    .private_value = SNDRV_PCM_STREAM_CAPTURE,
+  },
+  [UAC_PLAYBACK_REQ_CTRL] {
+    .iface =        SNDRV_CTL_ELEM_IFACE_PCM,
+    .name =         "Playback Requested",
+    .access = SNDRV_CTL_ELEM_ACCESS_READ | SNDRV_CTL_ELEM_ACCESS_VOLATILE,
+    .info =         u_audio_stream_requested_info,
+    .get =          u_audio_stream_requested_get,
+    .private_value = SNDRV_PCM_STREAM_PLAYBACK,
+  },
 };
 
+static void update_is_requested(struct uac_rtd_params *prm, unsigned int val) {
+	struct snd_kcontrol *kctl = prm->snd_kctl_is_req;
+
+	if (prm->is_requested != val) {
+		prm->is_requested = val;
+		snd_ctl_notify(prm->uac->card, SNDRV_CTL_EVENT_MASK_VALUE,
+				&kctl->id);
+		pr_debug("Setting '%s' to %d", kctl->id.name, val);
+	}
+}
+
 int g_audio_setup(struct g_audio *g_audio, const char *pcm_name,
 					const char *card_name)
 {
@@ -1209,6 +1268,38 @@ int g_audio_setup(struct g_audio *g_audio, const char *pcm_name,
 		err = snd_ctl_add(card, kctl);
 		if (err < 0)
 			goto snd_fail;
+
+		kctl = snd_ctl_new1(&u_audio_controls[UAC_PLAYBACK_REQ_CTRL],
+						&uac->p_prm);
+		if (!kctl) {
+			err = -ENOMEM;
+			goto snd_fail;
+		}
+
+		kctl->id.device = pcm->device;
+		kctl->id.subdevice = 0;
+
+		err = snd_ctl_add(card, kctl);
+		if (err < 0)
+			goto snd_fail;
+		(&uac->p_prm)->snd_kctl_is_req = kctl;
+	}
+
+	if (c_chmask) {
+		kctl = snd_ctl_new1(&u_audio_controls[UAC_CAPTURE_REQ_CTRL],
+					&uac->c_prm);
+		if (!kctl) {
+			err = -ENOMEM;
+			goto snd_fail;
+		}
+
+		kctl->id.device = pcm->device;
+		kctl->id.subdevice = 0;
+
+		err = snd_ctl_add(card, kctl);
+		if (err < 0)
+			goto snd_fail;
+		(&uac->c_prm)->snd_kctl_is_req = kctl;
 	}
 
 	for (i = 0; i <= SNDRV_PCM_STREAM_LAST; i++) {

Re: RFC: usb: gadget: u_audio: Notifying gadget that host started playback/capture?

[Pavel Hofman]

[-- Attachment #1: Type: text/plain, Size: 2097 bytes --]

Hi,

Dne 08. 09. 21 v 10:21 Pavel Hofman napsal(a):
> Hi,
> 
> The current audio gadget has no way to inform the gadget side that the 
> host side has started playback/capture and that gadget-side alsa 
> processes should be started.
> 
> Playback/capture processes on the host side do not get stuck without the 
> gadget side consuming/producing data (OUT requests are ignored in 
> u_audio_iso_complete, IN ones send initial zeros in their req->buf).
> 
> However, playback/capture processes on the gadget side get stuck without 
> the host side sending playback OUT packets or capture IN requests and 
> time out with error. If there was a way to inform the gadget side that 
> playback/capture has started on the host side, the gadget clients could 
> react accordingly.
> 

I drafted a simple patch for u_audio.c which defines read-only boolean 
ctl elems "Capture Requested" and "Playback Requested". Their values are 
set/reset in methods u_audio_start_capture/playback and 
u_audio_stop_capture/playback, i.e. at changes of respective altsettings 
from 0 to 1 and back. Every ctl elem value change sends notification via 
snd_ctl_notify. The principle works OK for capture/playback start/stop 
on the host, as monitored by alsactl:

pi@raspberrypi:~ $ alsactl monitor hw:UAC2Gadget
node hw:UAC2Gadget, #4 (3,0,0,Capture Requested,0) VALUE
node hw:UAC2Gadget, #4 (3,0,0,Capture Requested,0) VALUE
node hw:UAC2Gadget, #3 (3,0,0,Playback Requested,0) VALUE
node hw:UAC2Gadget, #3 (3,0,0,Playback Requested,0) VALUE

However at enumeration the USB host switches both playback and capture 
altsettings repeatedly, generating "fake" events from the gadget side 
POW. The host even sends regular-sized EP-OUT packets filled with zeros 
during enumeration (tested on linux only for now).

Please is there any way to "detect" the enumeration stage to mask out 
the "fake" playback/capture start/stop events?

The attached patch does not apply cleanly to mainline u_audio.c because 
it's rebased on other patches not submitted yet but it's only a 
discussion inducer for now.

Thanks a lot,

Pavel.

[-- Attachment #2: patch.diff --]
[-- Type: text/x-patch, Size: 5116 bytes --]

diff --git a/drivers/usb/gadget/function/u_audio.c b/drivers/usb/gadget/function/u_audio.c
index e395be32275c..69aef4e3677d 100644
--- a/drivers/usb/gadget/function/u_audio.c
+++ b/drivers/usb/gadget/function/u_audio.c
@@ -34,6 +34,8 @@ enum {
 	UAC_VOLUME_CTRL,
 	UAC_CAPTURE_RATE_CTRL,
 	UAC_PLAYBACK_RATE_CTRL,
+	UAC_CAPTURE_REQ_CTRL,
+	UAC_PLAYBACK_REQ_CTRL,
 };
 
 /* Runtime data params for one stream */
@@ -63,6 +65,9 @@ struct uac_rtd_params {
   s16 volume_min, volume_max, volume_res;
   s16 volume;
   int mute;
+  /* playback/capture_is_requested and their state */
+  struct snd_kcontrol *snd_kctl_is_req;
+  int is_requested;
 
   spinlock_t lock; /* lock for control transfers */
 
@@ -542,6 +547,8 @@ int u_audio_set_playback_srate(struct g_audio *audio_dev, int srate)
 }
 EXPORT_SYMBOL_GPL(u_audio_set_playback_srate);
 
+static void update_is_requested(struct uac_rtd_params *prm, unsigned int val);
+
 int u_audio_start_capture(struct g_audio *audio_dev)
 {
 	struct snd_uac_chip *uac = audio_dev->uac;
@@ -553,6 +560,7 @@ int u_audio_start_capture(struct g_audio *audio_dev)
 	struct uac_params *params = &audio_dev->params;
 	int req_len, i;
 
+	update_is_requested(&uac->c_prm, 1);
 	ep = audio_dev->out_ep;
 	prm = &uac->c_prm;
 	config_ep_by_speed(gadget, &audio_dev->func, ep);
@@ -625,6 +633,7 @@ void u_audio_stop_capture(struct g_audio *audio_dev)
 {
 	struct snd_uac_chip *uac = audio_dev->uac;
 
+	update_is_requested(&uac->c_prm, 0);
 	if (audio_dev->in_ep_fback)
 		free_ep_fback(&uac->c_prm, audio_dev->in_ep_fback);
 	free_ep(&uac->c_prm, audio_dev->out_ep);
@@ -645,6 +654,7 @@ int u_audio_start_playback(struct g_audio *audio_dev)
 	int req_len, i;
 	unsigned int p_interval, p_pktsize, p_pktsize_residue;
 
+	update_is_requested(&uac->p_prm, 1);
 	dev_dbg(dev, "start playback with rate %d\n", params->p_srate_active);
 	ep = audio_dev->in_ep;
 	prm = &uac->p_prm;
@@ -711,6 +721,7 @@ void u_audio_stop_playback(struct g_audio *audio_dev)
 {
 	struct snd_uac_chip *uac = audio_dev->uac;
 
+	update_is_requested(&uac->p_prm, 0);
 	free_ep(&uac->p_prm, audio_dev->in_ep);
 }
 EXPORT_SYMBOL_GPL(u_audio_stop_playback);
@@ -1027,6 +1038,27 @@ static int uac_pcm_rate_get(struct snd_kcontrol *kcontrol,
 	return 0;
 }
 
+static int u_audio_stream_requested_info(struct snd_kcontrol *kcontrol,
+				   struct snd_ctl_elem_info *uinfo)
+{
+	uinfo->type = SNDRV_CTL_ELEM_TYPE_BOOLEAN;
+	uinfo->count = 1;
+	uinfo->value.integer.min = 0;
+	uinfo->value.integer.max = 1;
+	uinfo->value.integer.step = 1;
+
+	return 0;
+}
+
+static int u_audio_stream_requested_get(struct snd_kcontrol *kcontrol,
+				   struct snd_ctl_elem_value *ucontrol)
+{
+	struct uac_rtd_params *prm = snd_kcontrol_chip(kcontrol);
+	ucontrol->value.integer.value[0] = prm->is_requested;
+	return 0;
+}
+
+
 static struct snd_kcontrol_new u_audio_controls[]  = {
   [UAC_FBACK_CTRL] {
     .iface =        SNDRV_CTL_ELEM_IFACE_PCM,
@@ -1072,8 +1104,35 @@ static struct snd_kcontrol_new u_audio_controls[]  = {
 		.get = uac_pcm_rate_get,
 		.private_value = SNDRV_PCM_STREAM_PLAYBACK,
 		},
+  [UAC_CAPTURE_REQ_CTRL] {
+    .iface =        SNDRV_CTL_ELEM_IFACE_PCM,
+    .name =         "Capture Requested",
+    .access = SNDRV_CTL_ELEM_ACCESS_READ | SNDRV_CTL_ELEM_ACCESS_VOLATILE,
+    .info =         u_audio_stream_requested_info,
+    .get =          u_audio_stream_requested_get,
+    .private_value = SNDRV_PCM_STREAM_CAPTURE,
+  },
+  [UAC_PLAYBACK_REQ_CTRL] {
+    .iface =        SNDRV_CTL_ELEM_IFACE_PCM,
+    .name =         "Playback Requested",
+    .access = SNDRV_CTL_ELEM_ACCESS_READ | SNDRV_CTL_ELEM_ACCESS_VOLATILE,
+    .info =         u_audio_stream_requested_info,
+    .get =          u_audio_stream_requested_get,
+    .private_value = SNDRV_PCM_STREAM_PLAYBACK,
+  },
 };
 
+static void update_is_requested(struct uac_rtd_params *prm, unsigned int val) {
+	struct snd_kcontrol *kctl = prm->snd_kctl_is_req;
+
+	if (prm->is_requested != val) {
+		prm->is_requested = val;
+		snd_ctl_notify(prm->uac->card, SNDRV_CTL_EVENT_MASK_VALUE,
+				&kctl->id);
+		pr_debug("Setting '%s' to %d", kctl->id.name, val);
+	}
+}
+
 int g_audio_setup(struct g_audio *g_audio, const char *pcm_name,
 					const char *card_name)
 {
@@ -1209,6 +1268,38 @@ int g_audio_setup(struct g_audio *g_audio, const char *pcm_name,
 		err = snd_ctl_add(card, kctl);
 		if (err < 0)
 			goto snd_fail;
+
+		kctl = snd_ctl_new1(&u_audio_controls[UAC_PLAYBACK_REQ_CTRL],
+						&uac->p_prm);
+		if (!kctl) {
+			err = -ENOMEM;
+			goto snd_fail;
+		}
+
+		kctl->id.device = pcm->device;
+		kctl->id.subdevice = 0;
+
+		err = snd_ctl_add(card, kctl);
+		if (err < 0)
+			goto snd_fail;
+		(&uac->p_prm)->snd_kctl_is_req = kctl;
+	}
+
+	if (c_chmask) {
+		kctl = snd_ctl_new1(&u_audio_controls[UAC_CAPTURE_REQ_CTRL],
+					&uac->c_prm);
+		if (!kctl) {
+			err = -ENOMEM;
+			goto snd_fail;
+		}
+
+		kctl->id.device = pcm->device;
+		kctl->id.subdevice = 0;
+
+		err = snd_ctl_add(card, kctl);
+		if (err < 0)
+			goto snd_fail;
+		(&uac->c_prm)->snd_kctl_is_req = kctl;
 	}
 
 	for (i = 0; i <= SNDRV_PCM_STREAM_LAST; i++) {

Re: RFC: usb: gadget: u_audio: Notifying gadget that host started playback/capture?

[Arun Raghavan]

Hi folks,

On Fri, 1 Oct 2021, at 8:38 AM, Pavel Hofman wrote:
> Hi,
>
> Dne 08. 09. 21 v 10:21 Pavel Hofman napsal(a):
>> Hi,
>> 
>> The current audio gadget has no way to inform the gadget side that the 
>> host side has started playback/capture and that gadget-side alsa 
>> processes should be started.
>> 
>> Playback/capture processes on the host side do not get stuck without the 
>> gadget side consuming/producing data (OUT requests are ignored in 
>> u_audio_iso_complete, IN ones send initial zeros in their req->buf).
>> 
>> However, playback/capture processes on the gadget side get stuck without 
>> the host side sending playback OUT packets or capture IN requests and 
>> time out with error. If there was a way to inform the gadget side that 
>> playback/capture has started on the host side, the gadget clients could 
>> react accordingly.
>> 
>
> I drafted a simple patch for u_audio.c which defines read-only boolean 
> ctl elems "Capture Requested" and "Playback Requested". Their values are 
> set/reset in methods u_audio_start_capture/playback and 
> u_audio_stop_capture/playback, i.e. at changes of respective altsettings 
> from 0 to 1 and back. Every ctl elem value change sends notification via 
> snd_ctl_notify. The principle works OK for capture/playback start/stop 
> on the host, as monitored by alsactl:
>
> pi@raspberrypi:~ $ alsactl monitor hw:UAC2Gadget
> node hw:UAC2Gadget, #4 (3,0,0,Capture Requested,0) VALUE
> node hw:UAC2Gadget, #4 (3,0,0,Capture Requested,0) VALUE
> node hw:UAC2Gadget, #3 (3,0,0,Playback Requested,0) VALUE
> node hw:UAC2Gadget, #3 (3,0,0,Playback Requested,0) VALUE
>
> However at enumeration the USB host switches both playback and capture 
> altsettings repeatedly, generating "fake" events from the gadget side 
> POW. The host even sends regular-sized EP-OUT packets filled with zeros 
> during enumeration (tested on linux only for now).
>
> Please is there any way to "detect" the enumeration stage to mask out 
> the "fake" playback/capture start/stop events?
>
> The attached patch does not apply cleanly to mainline u_audio.c because 
> it's rebased on other patches not submitted yet but it's only a 
> discussion inducer for now.

Resurrecting this one -- is there any input on how we want to deal wit letting UAC gadgets know when the host is sending/receiving data?

Cheers,
Arun

Re: RFC: usb: gadget: u_audio: Notifying gadget that host started playback/capture?

[Pavel Hofman]

Dne 21. 09. 23 v 3:30 Arun Raghavan napsal(a):
> Hi folks,
> 
> On Fri, 1 Oct 2021, at 8:38 AM, Pavel Hofman wrote:
>> Hi,
>>
>> Dne 08. 09. 21 v 10:21 Pavel Hofman napsal(a):
>>> Hi,
>>>
>>> The current audio gadget has no way to inform the gadget side that the
>>> host side has started playback/capture and that gadget-side alsa
>>> processes should be started.
>>>
>>> Playback/capture processes on the host side do not get stuck without the
>>> gadget side consuming/producing data (OUT requests are ignored in
>>> u_audio_iso_complete, IN ones send initial zeros in their req->buf).
>>>
>>> However, playback/capture processes on the gadget side get stuck without
>>> the host side sending playback OUT packets or capture IN requests and
>>> time out with error. If there was a way to inform the gadget side that
>>> playback/capture has started on the host side, the gadget clients could
>>> react accordingly.
>>>
>>
>> I drafted a simple patch for u_audio.c which defines read-only boolean
>> ctl elems "Capture Requested" and "Playback Requested". Their values are
>> set/reset in methods u_audio_start_capture/playback and
>> u_audio_stop_capture/playback, i.e. at changes of respective altsettings
>> from 0 to 1 and back. Every ctl elem value change sends notification via
>> snd_ctl_notify. The principle works OK for capture/playback start/stop
>> on the host, as monitored by alsactl:
>>
>> pi@raspberrypi:~ $ alsactl monitor hw:UAC2Gadget
>> node hw:UAC2Gadget, #4 (3,0,0,Capture Requested,0) VALUE
>> node hw:UAC2Gadget, #4 (3,0,0,Capture Requested,0) VALUE
>> node hw:UAC2Gadget, #3 (3,0,0,Playback Requested,0) VALUE
>> node hw:UAC2Gadget, #3 (3,0,0,Playback Requested,0) VALUE
>>
>> However at enumeration the USB host switches both playback and capture
>> altsettings repeatedly, generating "fake" events from the gadget side
>> POW. The host even sends regular-sized EP-OUT packets filled with zeros
>> during enumeration (tested on linux only for now).
>>
>> Please is there any way to "detect" the enumeration stage to mask out
>> the "fake" playback/capture start/stop events?
>>
>> The attached patch does not apply cleanly to mainline u_audio.c because
>> it's rebased on other patches not submitted yet but it's only a
>> discussion inducer for now.
> 
> Resurrecting this one -- is there any input on how we want to deal wit letting UAC gadgets know when the host is sending/receiving data?

The current version uses the Playback/Capture Rate alsa ctls with 
notifications 
https://lore.kernel.org/all/20220121155308.48794-8-pavel.hofman@ivitera.com/

Example of handling is e.g. https://github.com/pavhofman/gaudio_ctl , 
the controller is being used in a number of projects, mostly DIY.

Recently Qualcomm devs have submitted patches for alternative approach 
using uevents 
https://lore.kernel.org/lkml/2023050801-handshake-refusing-0367@gregkh/T/#mcd6b346f3ddab6ab34792be0141633bb362d168f 
and later versions. The detection is identical, monitoring change in 
altsetting from 0 to non zero and back (methods 
u_audio_[start/stop]_[capture/playback]), just a different means of 
communicating the events to userspace.

Both methods (using the same principle) suffer from not knowing what's 
going on the host side and cannot differentiate between player really 
starting playback vs. UAC2 host driver or Pulseaudio shortly checking 
device availability. That's why the gaudio_ctl controller can debounce 
the playback/capture start 
https://github.com/pavhofman/gaudio_ctl#debouncing . But that is just an 
ugly workaround...

With regards,

Pavel.

Re: RFC: usb: gadget: u_audio: Notifying gadget that host started playback/capture?

[Arun Raghavan]

On Fri, 22 Sep 2023, at 3:09 AM, Pavel Hofman wrote:
> Dne 21. 09. 23 v 3:30 Arun Raghavan napsal(a):
>> Hi folks,
>> 
>> On Fri, 1 Oct 2021, at 8:38 AM, Pavel Hofman wrote:
>>> Hi,
>>>
>>> Dne 08. 09. 21 v 10:21 Pavel Hofman napsal(a):
>>>> Hi,
>>>>
>>>> The current audio gadget has no way to inform the gadget side that the
>>>> host side has started playback/capture and that gadget-side alsa
>>>> processes should be started.
>>>>
>>>> Playback/capture processes on the host side do not get stuck without the
>>>> gadget side consuming/producing data (OUT requests are ignored in
>>>> u_audio_iso_complete, IN ones send initial zeros in their req->buf).
>>>>
>>>> However, playback/capture processes on the gadget side get stuck without
>>>> the host side sending playback OUT packets or capture IN requests and
>>>> time out with error. If there was a way to inform the gadget side that
>>>> playback/capture has started on the host side, the gadget clients could
>>>> react accordingly.
>>>>
>>>
>>> I drafted a simple patch for u_audio.c which defines read-only boolean
>>> ctl elems "Capture Requested" and "Playback Requested". Their values are
>>> set/reset in methods u_audio_start_capture/playback and
>>> u_audio_stop_capture/playback, i.e. at changes of respective altsettings
>>> from 0 to 1 and back. Every ctl elem value change sends notification via
>>> snd_ctl_notify. The principle works OK for capture/playback start/stop
>>> on the host, as monitored by alsactl:
>>>
>>> pi@raspberrypi:~ $ alsactl monitor hw:UAC2Gadget
>>> node hw:UAC2Gadget, #4 (3,0,0,Capture Requested,0) VALUE
>>> node hw:UAC2Gadget, #4 (3,0,0,Capture Requested,0) VALUE
>>> node hw:UAC2Gadget, #3 (3,0,0,Playback Requested,0) VALUE
>>> node hw:UAC2Gadget, #3 (3,0,0,Playback Requested,0) VALUE
>>>
>>> However at enumeration the USB host switches both playback and capture
>>> altsettings repeatedly, generating "fake" events from the gadget side
>>> POW. The host even sends regular-sized EP-OUT packets filled with zeros
>>> during enumeration (tested on linux only for now).
>>>
>>> Please is there any way to "detect" the enumeration stage to mask out
>>> the "fake" playback/capture start/stop events?
>>>
>>> The attached patch does not apply cleanly to mainline u_audio.c because
>>> it's rebased on other patches not submitted yet but it's only a
>>> discussion inducer for now.
>> 
>> Resurrecting this one -- is there any input on how we want to deal wit letting UAC gadgets know when the host is sending/receiving data?
>
> The current version uses the Playback/Capture Rate alsa ctls with 
> notifications 
> https://lore.kernel.org/all/20220121155308.48794-8-pavel.hofman@ivitera.com/
>
> Example of handling is e.g. https://github.com/pavhofman/gaudio_ctl , 
> the controller is being used in a number of projects, mostly DIY.
>
> Recently Qualcomm devs have submitted patches for alternative approach 
> using uevents 
> https://lore.kernel.org/lkml/2023050801-handshake-refusing-0367@gregkh/T/#mcd6b346f3ddab6ab34792be0141633bb362d168f 
> and later versions. The detection is identical, monitoring change in 
> altsetting from 0 to non zero and back (methods 
> u_audio_[start/stop]_[capture/playback]), just a different means of 
> communicating the events to userspace.
>
> Both methods (using the same principle) suffer from not knowing what's 
> going on the host side and cannot differentiate between player really 
> starting playback vs. UAC2 host driver or Pulseaudio shortly checking 
> device availability. That's why the gaudio_ctl controller can debounce 
> the playback/capture start 
> https://github.com/pavhofman/gaudio_ctl#debouncing . But that is just an 
> ugly workaround...

Thank you for the links, Pavel! This all makes sense.

I guess the uevent mechanism is more "general" than the ALSA ctl for clients that want to plug in, listen and do $something. Not sure if there are other pros/cons of either approach.

I wonder if it might not be good to have some debouncing in the kernel rather than having every client have to implement this.

Cheers,
Arun

Re: RFC: usb: gadget: u_audio: Notifying gadget that host started playback/capture?

[Pavel Hofman]

Dne 05. 10. 23 v 1:15 Arun Raghavan napsal(a):
> On Fri, 22 Sep 2023, at 3:09 AM, Pavel Hofman wrote:
>> Dne 21. 09. 23 v 3:30 Arun Raghavan napsal(a):
>>> Hi folks,
>>>
>>> On Fri, 1 Oct 2021, at 8:38 AM, Pavel Hofman wrote:
>>>> Hi,
>>>>
>>>
>>> Resurrecting this one -- is there any input on how we want to deal wit letting UAC gadgets know when the host is sending/receiving data?
>>
>> The current version uses the Playback/Capture Rate alsa ctls with
>> notifications
>> https://lore.kernel.org/all/20220121155308.48794-8-pavel.hofman@ivitera.com/
>>
>> Example of handling is e.g. https://github.com/pavhofman/gaudio_ctl ,
>> the controller is being used in a number of projects, mostly DIY.
>>
>> Recently Qualcomm devs have submitted patches for alternative approach
>> using uevents
>> https://lore.kernel.org/lkml/2023050801-handshake-refusing-0367@gregkh/T/#mcd6b346f3ddab6ab34792be0141633bb362d168f
>> and later versions. The detection is identical, monitoring change in
>> altsetting from 0 to non zero and back (methods
>> u_audio_[start/stop]_[capture/playback]), just a different means of
>> communicating the events to userspace.
>>
>> Both methods (using the same principle) suffer from not knowing what's
>> going on the host side and cannot differentiate between player really
>> starting playback vs. UAC2 host driver or Pulseaudio shortly checking
>> device availability. That's why the gaudio_ctl controller can debounce
>> the playback/capture start
>> https://github.com/pavhofman/gaudio_ctl#debouncing . But that is just an
>> ugly workaround...
> 
> Thank you for the links, Pavel! This all makes sense.
> 
> I guess the uevent mechanism is more "general" than the ALSA ctl for clients that want to plug in, listen and do $something. Not sure if there are other pros/cons of either approach.

If the gadget defines multiple samplerates, the client must look at the 
RATE alsa ctls for learn the actual rate requested by the host. The ctls 
provide both the rate and notification of playback/capture start/stop at 
the same time.

For gadgets with a single fixed samplerate, the alsa ctl vs. uevents 
methods are equivalent. Both may find their users and can be in the 
gadget code, IMO.

> 
> I wonder if it might not be good to have some debouncing in the kernel rather than having every client have to implement this.

I am afraid this would be a large feature (debouncing requires extra 
threads), I have not even tried to push it through. Much better would be 
having some nice solution instead of a workaround :-)

With regards,

Pavel.

Re: RFC: usb: gadget: u_audio: Notifying gadget that host started playback/capture?

[Arun Raghavan]

On Thu, 5 Oct 2023, at 10:30 AM, Pavel Hofman wrote:
> Dne 05. 10. 23 v 1:15 Arun Raghavan napsal(a):
>> On Fri, 22 Sep 2023, at 3:09 AM, Pavel Hofman wrote:
>>> Dne 21. 09. 23 v 3:30 Arun Raghavan napsal(a):
>>>> Hi folks,
>>>>
>>>> On Fri, 1 Oct 2021, at 8:38 AM, Pavel Hofman wrote:
>>>>> Hi,
>>>>>
>>>>
>>>> Resurrecting this one -- is there any input on how we want to deal wit letting UAC gadgets know when the host is sending/receiving data?
>>>
>>> The current version uses the Playback/Capture Rate alsa ctls with
>>> notifications
>>> https://lore.kernel.org/all/20220121155308.48794-8-pavel.hofman@ivitera.com/
>>>
>>> Example of handling is e.g. https://github.com/pavhofman/gaudio_ctl ,
>>> the controller is being used in a number of projects, mostly DIY.
>>>
>>> Recently Qualcomm devs have submitted patches for alternative approach
>>> using uevents
>>> https://lore.kernel.org/lkml/2023050801-handshake-refusing-0367@gregkh/T/#mcd6b346f3ddab6ab34792be0141633bb362d168f
>>> and later versions. The detection is identical, monitoring change in
>>> altsetting from 0 to non zero and back (methods
>>> u_audio_[start/stop]_[capture/playback]), just a different means of
>>> communicating the events to userspace.
>>>
>>> Both methods (using the same principle) suffer from not knowing what's
>>> going on the host side and cannot differentiate between player really
>>> starting playback vs. UAC2 host driver or Pulseaudio shortly checking
>>> device availability. That's why the gaudio_ctl controller can debounce
>>> the playback/capture start
>>> https://github.com/pavhofman/gaudio_ctl#debouncing . But that is just an
>>> ugly workaround...
>> 
>> Thank you for the links, Pavel! This all makes sense.
>> 
[...]
>> I wonder if it might not be good to have some debouncing in the kernel rather than having every client have to implement this.
>
> I am afraid this would be a large feature (debouncing requires extra 
> threads), I have not even tried to push it through. Much better would be 
> having some nice solution instead of a workaround :-)

Makes sense.

Maybe a silly question, but what is the status of these patches -- I see them as "Accepted" on patchwork but they've not actually landed upstream yet?

Regards,
Arun

Re: RFC: usb: gadget: u_audio: Notifying gadget that host started playback/capture?

[Arun Raghavan]

On Wed, 25 Oct 2023, at 12:33 PM, Arun Raghavan wrote:
[...]
> Maybe a silly question, but what is the status of these patches -- I 
> see them as "Accepted" on patchwork but they've not actually landed 
> upstream yet?

Definitely a silly question, this has been upstream since 5.17, just missed that somehow. Sorry about the noise, all.

Thanks!
Arun