[RFC PATCH 00/14] Introduce QC USB SND audio offloading support

[RFC PATCH 00/14] Introduce QC USB SND audio offloading support

[Wesley Cheng]

Several Qualcomm based chipsets can support USB audio offloading to a
dedicated audio DSP, which can take over issuing transfers to the USB
host controller.  The intention is to reduce the load on the main
processors in the SoC, and allow them to be placed into lower power modes.
There are several parts to this design:
  1. Adding ASoC binding layer
  2. Create a USB backend for Q6DSP
  3. Introduce XHCI interrupter support
  4. Create vendor ops for the USB SND driver

Adding ASoC binding layer:
soc-usb: Intention is to treat a USB port similar to a headphone jack.
The port is always present on the device, but cable/pin status can be
enabled/disabled.  Expose mechanisms for USB backend ASoC drivers to
communicate with USB SND.

Create a USB backend for Q6DSP:
q6usb: Basic backend driver that will be responsible for maintaining the
resources needed to initiate a playback stream using the Q6DSP.  Will
be the entity that checks to make sure the connected USB audio device
supports the requested PCM format.  If it does not, the PCM open call will
fail, and userpsace ALSA can take action accordingly.

Introduce XHCI interrupter support:
XHCI HCD supports multiple interrupters, which allows for events to be routed
to different event rings.  This is determined by "Interrupter Target" field
specified in Section "6.4.1.1 Normal TRB" of the XHCI specification.

Events in the offloading case will be routed to an event ring that is assigned
to the audio DSP.

Create vendor ops for the USB SND driver:
qc_audio_offload: This particular driver has several components associated
with it:
- QMI stream request handler
- XHCI interrupter and resource management
- audio DSP memory management

When the audio DSP wants to enable a playback stream, the request is first
received by the ASoC platform sound card.  Depending on the selected route,
ASoC will bring up the individual DAIs in the path.  The Q6USB backend DAI
will send an AFE port start command (with enabling the USB playback path), and
the audio DSP will handle the request accordingly.

Part of the AFE USB port start handling will have an exchange of control
messages using the QMI protocol.  The qc_audio_offload driver will populate the
buffer information:
- Event ring base address
- EP transfer ring base address

and pass it along to the audio DSP.  All endpoint management will now be handed
over to the DSP, and the main processor is not involved in transfers.

Overall, implementing this feature will still expose separate sound card and PCM
devices for both the platorm card and USB audio device:
 0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D
                      SM8250-MTP-WCD9380-WSA8810-VA-DMIC
 1 [Audio          ]: USB-Audio - USB Audio
                      Generic USB Audio at usb-xhci-hcd.1.auto-1.4, high speed

This is to ensure that userspace ALSA entities can decide which route to take
when executing the audio playback.  In the above, if card#1 is selected, then
USB audio data will take the legacy path over the USB PCM drivers, etc...

This feature was validated using:
- tinymix: set/enable the multimedia path to route to USB backend
- tinyplay: issue playback on platform card

Wesley Cheng (14):
  ASoC: Add SOC USB APIs for adding an USB backend
  ASoC: qcom: qdsp6: Introduce USB AFE port to q6dsp
  ASoC: qcom: Add USB backend ASoC driver for Q6
  sound: usb: card: Introduce USB SND vendor op callbacks
  sound: usb: Export USB SND APIs for modules
  usb: core: hcd: Introduce USB HCD APIs for interrupter management
  usb: host: xhci: Add XHCI secondary interrupter support
  usb: dwc3: Add DT parameter to specify maximum number of interrupters
  sound: usb: Introduce QC USB SND offloading support
  sound: usb: card: Check for support for requested audio format
  sound: soc: soc-usb: Add PCM format check API for USB backend
  sound: soc: qcom: qusb6: Ensure PCM format is supported by USB audio
    device
  ASoC: dt-bindings: Add Q6USB backend bindings
  ASoC: dt-bindings: Update example for enabling USB offload on SM8250

 .../bindings/sound/qcom,q6usb-dais.yaml       |   55 +
 .../bindings/sound/qcom,sm8250.yaml           |   13 +
 drivers/usb/core/hcd.c                        |   86 +
 drivers/usb/dwc3/core.c                       |   12 +
 drivers/usb/dwc3/core.h                       |    2 +
 drivers/usb/dwc3/host.c                       |    5 +-
 drivers/usb/host/xhci-mem.c                   |  219 ++-
 drivers/usb/host/xhci-plat.c                  |    2 +
 drivers/usb/host/xhci.c                       |  169 +-
 drivers/usb/host/xhci.h                       |   15 +
 .../sound/qcom,q6dsp-lpass-ports.h            |    1 +
 include/linux/usb.h                           |    7 +
 include/linux/usb/hcd.h                       |   16 +-
 include/sound/q6usboffload.h                  |   20 +
 include/sound/soc-usb.h                       |   34 +
 sound/soc/Makefile                            |    2 +-
 sound/soc/qcom/Kconfig                        |    4 +
 sound/soc/qcom/qdsp6/Makefile                 |    1 +
 sound/soc/qcom/qdsp6/q6afe-dai.c              |   47 +
 sound/soc/qcom/qdsp6/q6afe.c                  |  183 ++
 sound/soc/qcom/qdsp6/q6afe.h                  |   46 +-
 sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c      |   23 +
 sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h      |    1 +
 sound/soc/qcom/qdsp6/q6routing.c              |    8 +
 sound/soc/qcom/qdsp6/q6usb.c                  |  239 +++
 sound/soc/soc-usb.c                           |   79 +
 sound/usb/Kconfig                             |   14 +
 sound/usb/Makefile                            |    2 +-
 sound/usb/card.c                              |   43 +
 sound/usb/card.h                              |   10 +
 sound/usb/endpoint.c                          |    2 +
 sound/usb/helper.c                            |    1 +
 sound/usb/pcm.c                               |    9 +-
 sound/usb/pcm.h                               |   12 +
 sound/usb/qcom/Makefile                       |    2 +
 sound/usb/qcom/qc_audio_offload.c             | 1610 +++++++++++++++++
 sound/usb/qcom/usb_audio_qmi_v01.c            |  892 +++++++++
 sound/usb/qcom/usb_audio_qmi_v01.h            |  162 ++
 38 files changed, 3998 insertions(+), 50 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml
 create mode 100644 include/sound/q6usboffload.h
 create mode 100644 include/sound/soc-usb.h
 create mode 100644 sound/soc/qcom/qdsp6/q6usb.c
 create mode 100644 sound/soc/soc-usb.c
 create mode 100644 sound/usb/qcom/Makefile
 create mode 100644 sound/usb/qcom/qc_audio_offload.c
 create mode 100644 sound/usb/qcom/usb_audio_qmi_v01.c
 create mode 100644 sound/usb/qcom/usb_audio_qmi_v01.h

[RFC PATCH 01/14] ASoC: Add SOC USB APIs for adding an USB backend

[Wesley Cheng]

Some platforms may want to register its USB port to be handled by the ASoC
framework.  Audio playback/capture support is also handled entirely by the
vendor ASoC drivers.

Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
---
 include/sound/soc-usb.h | 31 +++++++++++++++++++
 sound/soc/Makefile      |  2 +-
 sound/soc/soc-usb.c     | 66 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 98 insertions(+), 1 deletion(-)
 create mode 100644 include/sound/soc-usb.h
 create mode 100644 sound/soc/soc-usb.c

diff --git a/include/sound/soc-usb.h b/include/sound/soc-usb.h
new file mode 100644
index 000000000000..7d52e5d2371c
--- /dev/null
+++ b/include/sound/soc-usb.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0
+ *
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef __LINUX_SND_SOC_USB_H
+#define __LINUX_SND_SOC_USB_H
+
+/**
+ * struct snd_soc_usb
+ * @component - Reference to DAPM component
+ * @connection_status_cb - callback to notify connection events
+ * @priv_data - vendor data
+ **/
+struct snd_soc_usb {
+	struct snd_soc_component *component;
+	int (*connection_status_cb)(struct snd_soc_usb *usb, int card_idx,
+				int connected);
+	void *priv_data;
+};
+
+int snd_soc_usb_connect(int card_idx);
+int snd_soc_usb_disconnect(void);
+void snd_soc_usb_set_priv_data(void *priv);
+void *snd_soc_usb_get_priv_data(void);
+
+struct snd_soc_usb *snd_soc_usb_add_port(struct device *dev,
+			int (*connection_cb)(struct snd_soc_usb *usb, int card_idx,
+			int connected));
+int snd_soc_usb_remove_port(void);
+#endif
diff --git a/sound/soc/Makefile b/sound/soc/Makefile
index 507eaed1d6a1..3305ceb59d84 100644
--- a/sound/soc/Makefile
+++ b/sound/soc/Makefile
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0
-snd-soc-core-objs := soc-core.o soc-dapm.o soc-jack.o soc-utils.o soc-dai.o soc-component.o
+snd-soc-core-objs := soc-core.o soc-dapm.o soc-jack.o soc-usb.o soc-utils.o soc-dai.o soc-component.o
 snd-soc-core-objs += soc-pcm.o soc-devres.o soc-ops.o soc-link.o soc-card.o
 snd-soc-core-$(CONFIG_SND_SOC_COMPRESS) += soc-compress.o
 
diff --git a/sound/soc/soc-usb.c b/sound/soc/soc-usb.c
new file mode 100644
index 000000000000..c6c376960e4d
--- /dev/null
+++ b/sound/soc/soc-usb.c
@@ -0,0 +1,66 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+#include <linux/usb.h>
+#include <sound/soc.h>
+#include <sound/soc-usb.h>
+#include "../usb/card.h"
+
+struct snd_soc_usb *ctx;
+
+void *snd_soc_usb_get_priv_data(void)
+{
+	if (ctx)
+		return ctx->priv_data;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(snd_soc_usb_get_priv_data);
+
+void snd_soc_usb_set_priv_data(void *priv)
+{
+	if (ctx)
+		ctx->priv_data = priv;
+}
+EXPORT_SYMBOL_GPL(snd_soc_usb_set_priv_data);
+
+struct snd_soc_usb *snd_soc_usb_add_port(struct device *dev,
+			int (*connection_cb)(struct snd_soc_usb *usb, int card_idx,
+			int connected))
+{
+	struct snd_soc_usb *usb;
+
+	usb = devm_kzalloc(dev, sizeof(*usb), GFP_KERNEL);
+	if (!usb)
+		return ERR_PTR(-ENOMEM);
+
+	usb->connection_status_cb = connection_cb;
+	ctx = usb;
+
+	return usb;
+}
+EXPORT_SYMBOL_GPL(snd_soc_usb_add_port);
+
+int snd_soc_usb_remove_port(void)
+{
+	ctx = NULL;
+	return 0;
+}
+EXPORT_SYMBOL_GPL(snd_soc_usb_remove_port);
+
+int snd_soc_usb_connect(int card_idx)
+{
+	if (ctx && ctx->connection_status_cb)
+		ctx->connection_status_cb(ctx, card_idx, 1);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(snd_soc_usb_connect);
+
+int snd_soc_usb_disconnect(void)
+{
+	if (ctx && ctx->connection_status_cb)
+		ctx->connection_status_cb(ctx, -1, 0);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(snd_soc_usb_disconnect);

[RFC PATCH 02/14] ASoC: qcom: qdsp6: Introduce USB AFE port to q6dsp

[Wesley Cheng]

The QC ADSP is able to support USB playback and capture, so that the
main application processor can be placed into lower CPU power modes.  This
adds the required AFE port configurations and port start command to start
an audio session.

Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
---
 .../sound/qcom,q6dsp-lpass-ports.h            |   1 +
 sound/soc/qcom/qdsp6/q6afe-dai.c              |  47 +++++
 sound/soc/qcom/qdsp6/q6afe.c                  | 183 ++++++++++++++++++
 sound/soc/qcom/qdsp6/q6afe.h                  |  46 ++++-
 sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c      |  23 +++
 sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h      |   1 +
 sound/soc/qcom/qdsp6/q6routing.c              |   8 +
 7 files changed, 308 insertions(+), 1 deletion(-)

diff --git a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h
index 9f7c5103bc82..746bc462bb2e 100644
--- a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h
+++ b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h
@@ -131,6 +131,7 @@
 #define RX_CODEC_DMA_RX_7	126
 #define QUINARY_MI2S_RX		127
 #define QUINARY_MI2S_TX		128
+#define USB_RX				129
 
 #define LPASS_CLK_ID_PRI_MI2S_IBIT	1
 #define LPASS_CLK_ID_PRI_MI2S_EBIT	2
diff --git a/sound/soc/qcom/qdsp6/q6afe-dai.c b/sound/soc/qcom/qdsp6/q6afe-dai.c
index 8bb7452b8f18..7fdee8cbd4de 100644
--- a/sound/soc/qcom/qdsp6/q6afe-dai.c
+++ b/sound/soc/qcom/qdsp6/q6afe-dai.c
@@ -111,6 +111,40 @@ static int q6hdmi_hw_params(struct snd_pcm_substream *substream,
 	return 0;
 }
 
+static int q6usb_hw_params(struct snd_pcm_substream *substream,
+			   struct snd_pcm_hw_params *params,
+			   struct snd_soc_dai *dai)
+{
+	struct q6afe_dai_data *dai_data = dev_get_drvdata(dai->dev);
+	int channels = params_channels(params);
+	int rate = params_rate(params);
+	struct q6afe_usb_cfg *usb = &dai_data->port_config[dai->id].usb_audio;
+
+	usb->sample_rate = rate;
+	usb->num_channels = channels;
+
+	switch (params_format(params)) {
+	case SNDRV_PCM_FORMAT_U16_LE:
+	case SNDRV_PCM_FORMAT_S16_LE:
+	case SNDRV_PCM_FORMAT_SPECIAL:
+		usb->bit_width = 16;
+		break;
+	case SNDRV_PCM_FORMAT_S24_LE:
+	case SNDRV_PCM_FORMAT_S24_3LE:
+		usb->bit_width = 24;
+		break;
+	case SNDRV_PCM_FORMAT_S32_LE:
+		usb->bit_width = 32;
+		break;
+	default:
+		dev_err(dai->dev, "%s: invalid format %d\n",
+			__func__, params_format(params));
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
 static int q6i2s_hw_params(struct snd_pcm_substream *substream,
 			   struct snd_pcm_hw_params *params,
 			   struct snd_soc_dai *dai)
@@ -411,6 +445,10 @@ static int q6afe_dai_prepare(struct snd_pcm_substream *substream,
 		q6afe_cdc_dma_port_prepare(dai_data->port[dai->id],
 					   &dai_data->port_config[dai->id].dma_cfg);
 		break;
+	case USB_RX:
+		q6afe_usb_port_prepare(dai_data->port[dai->id],
+					   &dai_data->port_config[dai->id].usb_audio);
+		break;
 	default:
 		return -EINVAL;
 	}
@@ -495,6 +533,7 @@ static int q6afe_mi2s_set_sysclk(struct snd_soc_dai *dai,
 }
 
 static const struct snd_soc_dapm_route q6afe_dapm_routes[] = {
+	{"USB Playback", NULL, "USB_RX"},
 	{"HDMI Playback", NULL, "HDMI_RX"},
 	{"Display Port Playback", NULL, "DISPLAY_PORT_RX"},
 	{"Slimbus Playback", NULL, "SLIMBUS_0_RX"},
@@ -639,6 +678,12 @@ static const struct snd_soc_dapm_route q6afe_dapm_routes[] = {
 	{"RX_CODEC_DMA_RX_7 Playback", NULL, "RX_CODEC_DMA_RX_7"},
 };
 
+static const struct snd_soc_dai_ops q6usb_ops = {
+	.prepare	= q6afe_dai_prepare,
+	.hw_params	= q6usb_hw_params,
+	.shutdown	= q6afe_dai_shutdown,
+};
+
 static const struct snd_soc_dai_ops q6hdmi_ops = {
 	.prepare	= q6afe_dai_prepare,
 	.hw_params	= q6hdmi_hw_params,
@@ -703,6 +748,7 @@ static int msm_dai_q6_dai_remove(struct snd_soc_dai *dai)
 }
 
 static const struct snd_soc_dapm_widget q6afe_dai_widgets[] = {
+	SND_SOC_DAPM_AIF_IN("USB_RX", NULL, 0, SND_SOC_NOPM, 0, 0),
 	SND_SOC_DAPM_AIF_IN("HDMI_RX", NULL, 0, SND_SOC_NOPM, 0, 0),
 	SND_SOC_DAPM_AIF_IN("SLIMBUS_0_RX", NULL, 0, SND_SOC_NOPM, 0, 0),
 	SND_SOC_DAPM_AIF_IN("SLIMBUS_1_RX", NULL, 0, SND_SOC_NOPM, 0, 0),
@@ -1068,6 +1114,7 @@ static int q6afe_dai_dev_probe(struct platform_device *pdev)
 	cfg.q6i2s_ops = &q6i2s_ops;
 	cfg.q6tdm_ops = &q6tdm_ops;
 	cfg.q6dma_ops = &q6dma_ops;
+	cfg.q6usb_ops = &q6usb_ops;
 	dais = q6dsp_audio_ports_set_config(dev, &cfg, &num_dais);
 
 	return devm_snd_soc_register_component(dev, &q6afe_dai_component, dais, num_dais);
diff --git a/sound/soc/qcom/qdsp6/q6afe.c b/sound/soc/qcom/qdsp6/q6afe.c
index 919e326b9462..2054f5723e03 100644
--- a/sound/soc/qcom/qdsp6/q6afe.c
+++ b/sound/soc/qcom/qdsp6/q6afe.c
@@ -34,6 +34,8 @@
 #define AFE_MODULE_TDM			0x0001028A
 
 #define AFE_PARAM_ID_CDC_SLIMBUS_SLAVE_CFG 0x00010235
+#define AFE_PARAM_ID_USB_AUDIO_DEV_PARAMS    0x000102A5
+#define AFE_PARAM_ID_USB_AUDIO_DEV_LPCM_FMT 0x000102AA
 
 #define AFE_PARAM_ID_LPAIF_CLK_CONFIG	0x00010238
 #define AFE_PARAM_ID_INT_DIGITAL_CDC_CLK_CONFIG	0x00010239
@@ -43,6 +45,7 @@
 #define AFE_PARAM_ID_TDM_CONFIG	0x0001029D
 #define AFE_PARAM_ID_PORT_SLOT_MAPPING_CONFIG	0x00010297
 #define AFE_PARAM_ID_CODEC_DMA_CONFIG	0x000102B8
+#define AFE_PARAM_ID_USB_AUDIO_CONFIG    0x000102A4
 #define AFE_CMD_REMOTE_LPASS_CORE_HW_VOTE_REQUEST	0x000100f4
 #define AFE_CMD_RSP_REMOTE_LPASS_CORE_HW_VOTE_REQUEST   0x000100f5
 #define AFE_CMD_REMOTE_LPASS_CORE_HW_DEVOTE_REQUEST	0x000100f6
@@ -71,12 +74,16 @@
 #define AFE_PORT_CONFIG_I2S_WS_SRC_INTERNAL	0x1
 #define AFE_LINEAR_PCM_DATA				0x0
 
+#define AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG 0x1
 
 /* Port IDs */
 #define AFE_API_VERSION_HDMI_CONFIG	0x1
 #define AFE_PORT_ID_MULTICHAN_HDMI_RX	0x100E
 #define AFE_PORT_ID_HDMI_OVER_DP_RX	0x6020
 
+/* USB AFE port */
+#define AFE_PORT_ID_USB_RX                       0x7000
+
 #define AFE_API_VERSION_SLIMBUS_CONFIG 0x1
 /* Clock set API version */
 #define AFE_API_VERSION_CLOCK_SET 1
@@ -512,12 +519,109 @@ struct afe_param_id_cdc_dma_cfg {
 	u16	active_channels_mask;
 } __packed;
 
+struct afe_param_id_usb_cfg {
+/* Minor version used for tracking USB audio device configuration.
+ * Supported values: AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG
+ */
+	u32                  cfg_minor_version;
+/* Sampling rate of the port.
+ * Supported values:
+ * - AFE_PORT_SAMPLE_RATE_8K
+ * - AFE_PORT_SAMPLE_RATE_11025
+ * - AFE_PORT_SAMPLE_RATE_12K
+ * - AFE_PORT_SAMPLE_RATE_16K
+ * - AFE_PORT_SAMPLE_RATE_22050
+ * - AFE_PORT_SAMPLE_RATE_24K
+ * - AFE_PORT_SAMPLE_RATE_32K
+ * - AFE_PORT_SAMPLE_RATE_44P1K
+ * - AFE_PORT_SAMPLE_RATE_48K
+ * - AFE_PORT_SAMPLE_RATE_96K
+ * - AFE_PORT_SAMPLE_RATE_192K
+ */
+	u32                  sample_rate;
+/* Bit width of the sample.
+ * Supported values: 16, 24
+ */
+	u16                  bit_width;
+/* Number of channels.
+ * Supported values: 1 and 2
+ */
+	u16                  num_channels;
+/* Data format supported by the USB. The supported value is
+ * 0 (#AFE_USB_AUDIO_DATA_FORMAT_LINEAR_PCM).
+ */
+	u16                  data_format;
+/* this field must be 0 */
+	u16                  reserved;
+/* device token of actual end USB aduio device */
+	u32                  dev_token;
+/* endianness of this interface */
+	u32                   endian;
+/* service interval */
+	u32                  service_interval;
+} __packed;
+
+/**
+ * struct afe_param_id_usb_audio_dev_params
+ * @cfg_minor_version: Minor version used for tracking USB audio device
+ * configuration.
+ * Supported values:
+ *     AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG
+ * @dev_token: device token of actual end USB aduio device
+ **/
+struct afe_param_id_usb_audio_dev_params {
+	u32	cfg_minor_version;
+	u32	dev_token;
+} __packed;
+
+/**
+ * struct afe_param_id_usb_audio_dev_lpcm_fmt
+ * @cfg_minor_version: Minor version used for tracking USB audio device
+ * configuration.
+ * Supported values:
+ *     AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG
+ * @endian: endianness of this interface
+ **/
+struct afe_param_id_usb_audio_dev_lpcm_fmt {
+	u32	cfg_minor_version;
+	u32	endian;
+} __packed;
+
+/**
+ * struct afe_param_id_usb_audio_dev_latency_mode
+ * @cfg_minor_version: Minor version used for tracking USB audio device
+ * configuration.
+ * Supported values:
+ *     AFE_API_MINOR_VERSION_USB_AUDIO_LATENCY_MODE
+ * @mode: latency mode for the USB audio device
+ **/
+struct afe_param_id_usb_audio_dev_latency_mode {
+	u32	minor_version;
+	u32	mode;
+} __packed;
+
+#define AFE_PARAM_ID_USB_AUDIO_SVC_INTERVAL     0x000102B7
+
+/**
+ * struct afe_param_id_usb_audio_svc_interval
+ * @cfg_minor_version: Minor version used for tracking USB audio device
+ * configuration.
+ * Supported values:
+ *     AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG
+ * @svc_interval: service interval
+ **/
+struct afe_param_id_usb_audio_svc_interval {
+	u32	cfg_minor_version;
+	u32	svc_interval;
+} __packed;
+
 union afe_port_config {
 	struct afe_param_id_hdmi_multi_chan_audio_cfg hdmi_multi_ch;
 	struct afe_param_id_slimbus_cfg           slim_cfg;
 	struct afe_param_id_i2s_cfg	i2s_cfg;
 	struct afe_param_id_tdm_cfg	tdm_cfg;
 	struct afe_param_id_cdc_dma_cfg	dma_cfg;
+	struct afe_param_id_usb_cfg usb_cfg;
 } __packed;
 
 
@@ -577,6 +681,7 @@ struct afe_port_map {
  */
 
 static struct afe_port_map port_maps[AFE_PORT_MAX] = {
+	[USB_RX] = { AFE_PORT_ID_USB_RX, USB_RX, 1, 1},
 	[HDMI_RX] = { AFE_PORT_ID_MULTICHAN_HDMI_RX, HDMI_RX, 1, 1},
 	[SLIMBUS_0_RX] = { AFE_PORT_ID_SLIMBUS_MULTI_CHAN_0_RX,
 				SLIMBUS_0_RX, 1, 1},
@@ -1289,6 +1394,82 @@ void q6afe_tdm_port_prepare(struct q6afe_port *port,
 }
 EXPORT_SYMBOL_GPL(q6afe_tdm_port_prepare);
 
+int afe_port_send_usb_dev_param(struct q6afe_port *port, struct q6afe_usb_cfg *cfg)
+{
+	union afe_port_config *pcfg = &port->port_cfg;
+	struct afe_param_id_usb_audio_dev_params usb_dev;
+	struct afe_param_id_usb_audio_dev_lpcm_fmt lpcm_fmt;
+	struct afe_param_id_usb_audio_svc_interval svc_int;
+	int ret = 0;
+
+	if (!pcfg) {
+		pr_err("%s: Error, no configuration data\n", __func__);
+		ret = -EINVAL;
+		goto exit;
+	}
+
+	memset(&usb_dev, 0, sizeof(usb_dev));
+	memset(&lpcm_fmt, 0, sizeof(lpcm_fmt));
+
+	usb_dev.cfg_minor_version = AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG;
+	q6afe_port_set_param_v2(port, &usb_dev,
+					AFE_PARAM_ID_USB_AUDIO_DEV_PARAMS,
+					AFE_MODULE_AUDIO_DEV_INTERFACE, sizeof(usb_dev));
+	if (ret) {
+		pr_err("%s: AFE device param cmd failed %d\n",
+			__func__, ret);
+		goto exit;
+	}
+
+	lpcm_fmt.cfg_minor_version = AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG;
+	lpcm_fmt.endian = pcfg->usb_cfg.endian;
+	ret = q6afe_port_set_param_v2(port, &lpcm_fmt,
+					AFE_PARAM_ID_USB_AUDIO_DEV_LPCM_FMT,
+					AFE_MODULE_AUDIO_DEV_INTERFACE, sizeof(lpcm_fmt));
+	if (ret) {
+		pr_err("%s: AFE device param cmd LPCM_FMT failed %d\n",
+			__func__, ret);
+		goto exit;
+	}
+
+	svc_int.cfg_minor_version =
+		AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG;
+	svc_int.svc_interval = pcfg->usb_cfg.service_interval;
+	ret = q6afe_port_set_param_v2(port, &svc_int,
+					AFE_PARAM_ID_USB_AUDIO_SVC_INTERVAL,
+					AFE_MODULE_AUDIO_DEV_INTERFACE, sizeof(svc_int));
+	if (ret) {
+		pr_err("%s: AFE device param cmd svc_interval failed %d\n",
+			__func__, ret);
+		ret = -EINVAL;
+		goto exit;
+	}
+exit:
+	return ret;
+}
+
+/**
+ * q6afe_usb_port_prepare() - Prepare usb afe port.
+ *
+ * @port: Instance of afe port
+ * @cfg: USB configuration for the afe port
+ *
+ */
+void q6afe_usb_port_prepare(struct q6afe_port *port,
+			     struct q6afe_usb_cfg *cfg)
+{
+	union afe_port_config *pcfg = &port->port_cfg;
+
+	pcfg->usb_cfg.cfg_minor_version =
+					AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG;
+	pcfg->usb_cfg.sample_rate = cfg->sample_rate;
+	pcfg->usb_cfg.num_channels = cfg->num_channels;
+	pcfg->usb_cfg.bit_width = cfg->bit_width;
+
+	afe_port_send_usb_dev_param(port, cfg);
+}
+EXPORT_SYMBOL_GPL(q6afe_usb_port_prepare);
+
 /**
  * q6afe_hdmi_port_prepare() - Prepare hdmi afe port.
  *
@@ -1611,6 +1792,8 @@ struct q6afe_port *q6afe_port_get_from_id(struct device *dev, int id)
 		break;
 	case AFE_PORT_ID_WSA_CODEC_DMA_RX_0 ... AFE_PORT_ID_RX_CODEC_DMA_RX_7:
 		cfg_type = AFE_PARAM_ID_CODEC_DMA_CONFIG;
+	case AFE_PORT_ID_USB_RX:
+		cfg_type = AFE_PARAM_ID_USB_AUDIO_CONFIG;
 	break;
 	default:
 		dev_err(dev, "Invalid port id 0x%x\n", port_id);
diff --git a/sound/soc/qcom/qdsp6/q6afe.h b/sound/soc/qcom/qdsp6/q6afe.h
index 30fd77e2f458..88550a08e57d 100644
--- a/sound/soc/qcom/qdsp6/q6afe.h
+++ b/sound/soc/qcom/qdsp6/q6afe.h
@@ -5,7 +5,7 @@
 
 #include <dt-bindings/sound/qcom,q6afe.h>
 
-#define AFE_PORT_MAX		129
+#define AFE_PORT_MAX		130
 
 #define MSM_AFE_PORT_TYPE_RX 0
 #define MSM_AFE_PORT_TYPE_TX 1
@@ -205,6 +205,47 @@ struct q6afe_cdc_dma_cfg {
 	u16	active_channels_mask;
 };
 
+/**
+ * struct q6afe_usb_cfg
+ * @cfg_minor_version: Minor version used for tracking USB audio device
+ * configuration.
+ * Supported values:
+ *     AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG
+ * @sample_rate: Sampling rate of the port
+ *    Supported values:
+ *      AFE_PORT_SAMPLE_RATE_8K
+ *      AFE_PORT_SAMPLE_RATE_11025
+ *      AFE_PORT_SAMPLE_RATE_12K
+ *      AFE_PORT_SAMPLE_RATE_16K
+ *      AFE_PORT_SAMPLE_RATE_22050
+ *      AFE_PORT_SAMPLE_RATE_24K
+ *      AFE_PORT_SAMPLE_RATE_32K
+ *      AFE_PORT_SAMPLE_RATE_44P1K
+ *      AFE_PORT_SAMPLE_RATE_48K
+ *      AFE_PORT_SAMPLE_RATE_96K
+ *      AFE_PORT_SAMPLE_RATE_192K
+ * @bit_width: Bit width of the sample.
+ *    Supported values: 16, 24
+ * @num_channels: Number of channels
+ *    Supported values: 1, 2
+ * @data_format: Data format supported by the USB
+ *    Supported values: 0
+ * @reserved: this field must be 0
+ * @dev_token: device token of actual end USB aduio device
+ * @endian: endianness of this interface
+ * @service_interval: service interval
+ **/
+struct q6afe_usb_cfg {
+	u32	cfg_minor_version;
+	u32     sample_rate;
+	u16	bit_width;
+	u16	num_channels;
+	u16	data_format;
+	u16	reserved;
+	u32	dev_token;
+	u32	endian;
+	u32	service_interval;
+};
 
 struct q6afe_port_config {
 	struct q6afe_hdmi_cfg hdmi;
@@ -212,6 +253,7 @@ struct q6afe_port_config {
 	struct q6afe_i2s_cfg i2s_cfg;
 	struct q6afe_tdm_cfg tdm;
 	struct q6afe_cdc_dma_cfg dma_cfg;
+	struct q6afe_usb_cfg usb_audio;
 };
 
 struct q6afe_port;
@@ -221,6 +263,8 @@ int q6afe_port_start(struct q6afe_port *port);
 int q6afe_port_stop(struct q6afe_port *port);
 void q6afe_port_put(struct q6afe_port *port);
 int q6afe_get_port_id(int index);
+void q6afe_usb_port_prepare(struct q6afe_port *port,
+			     struct q6afe_usb_cfg *cfg);
 void q6afe_hdmi_port_prepare(struct q6afe_port *port,
 			    struct q6afe_hdmi_cfg *cfg);
 void q6afe_slim_port_prepare(struct q6afe_port *port,
diff --git a/sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c b/sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c
index f67c16fd90b9..39719c3f1767 100644
--- a/sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c
+++ b/sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c
@@ -81,6 +81,26 @@
 
 
 static struct snd_soc_dai_driver q6dsp_audio_fe_dais[] = {
+	{
+		.playback = {
+			.stream_name = "USB Playback",
+			.rates = SNDRV_PCM_RATE_8000 | SNDRV_PCM_RATE_11025 |
+					SNDRV_PCM_RATE_16000 | SNDRV_PCM_RATE_22050 |
+					SNDRV_PCM_RATE_32000 | SNDRV_PCM_RATE_44100 |
+					SNDRV_PCM_RATE_48000 | SNDRV_PCM_RATE_96000 |
+					SNDRV_PCM_RATE_192000,
+			.formats = SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S16_BE |
+					SNDRV_PCM_FMTBIT_U16_LE | SNDRV_PCM_FMTBIT_U16_BE |
+					SNDRV_PCM_FMTBIT_S24_LE | SNDRV_PCM_FMTBIT_S24_BE |
+					SNDRV_PCM_FMTBIT_U24_LE | SNDRV_PCM_FMTBIT_U24_BE,
+			.channels_min = 1,
+			.channels_max = 2,
+			.rate_min =	8000,
+			.rate_max = 192000,
+		},
+		.id = USB_RX,
+		.name = "USB_RX",
+	},
 	{
 		.playback = {
 			.stream_name = "HDMI Playback",
@@ -616,6 +636,9 @@ struct snd_soc_dai_driver *q6dsp_audio_ports_set_config(struct device *dev,
 		case WSA_CODEC_DMA_RX_0 ... RX_CODEC_DMA_RX_7:
 			q6dsp_audio_fe_dais[i].ops = cfg->q6dma_ops;
 			break;
+		case USB_RX:
+			q6dsp_audio_fe_dais[i].ops = cfg->q6usb_ops;
+			break;
 		default:
 			break;
 		}
diff --git a/sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h b/sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h
index 7f052c8a1257..d8dde6dd0aca 100644
--- a/sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h
+++ b/sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h
@@ -11,6 +11,7 @@ struct q6dsp_audio_port_dai_driver_config {
 	const struct snd_soc_dai_ops *q6i2s_ops;
 	const struct snd_soc_dai_ops *q6tdm_ops;
 	const struct snd_soc_dai_ops *q6dma_ops;
+	const struct snd_soc_dai_ops *q6usb_ops;
 };
 
 struct snd_soc_dai_driver *q6dsp_audio_ports_set_config(struct device *dev,
diff --git a/sound/soc/qcom/qdsp6/q6routing.c b/sound/soc/qcom/qdsp6/q6routing.c
index 928fd23e2c27..683ae2ae8e50 100644
--- a/sound/soc/qcom/qdsp6/q6routing.c
+++ b/sound/soc/qcom/qdsp6/q6routing.c
@@ -514,6 +514,9 @@ static int msm_routing_put_audio_mixer(struct snd_kcontrol *kcontrol,
 	return 1;
 }
 
+static const struct snd_kcontrol_new usb_mixer_controls[] = {
+	Q6ROUTING_RX_MIXERS(USB_RX) };
+
 static const struct snd_kcontrol_new hdmi_mixer_controls[] = {
 	Q6ROUTING_RX_MIXERS(HDMI_RX) };
 
@@ -733,6 +736,10 @@ static const struct snd_kcontrol_new mmul8_mixer_controls[] = {
 
 static const struct snd_soc_dapm_widget msm_qdsp6_widgets[] = {
 	/* Mixer definitions */
+	SND_SOC_DAPM_MIXER("USB Mixer", SND_SOC_NOPM, 0, 0,
+			   usb_mixer_controls,
+			   ARRAY_SIZE(usb_mixer_controls)),
+
 	SND_SOC_DAPM_MIXER("HDMI Mixer", SND_SOC_NOPM, 0, 0,
 			   hdmi_mixer_controls,
 			   ARRAY_SIZE(hdmi_mixer_controls)),
@@ -952,6 +959,7 @@ static const struct snd_soc_dapm_widget msm_qdsp6_widgets[] = {
 };
 
 static const struct snd_soc_dapm_route intercon[] = {
+	Q6ROUTING_RX_DAPM_ROUTE("USB Mixer", "USB_RX"),
 	Q6ROUTING_RX_DAPM_ROUTE("HDMI Mixer", "HDMI_RX"),
 	Q6ROUTING_RX_DAPM_ROUTE("DISPLAY_PORT_RX Audio Mixer",
 				"DISPLAY_PORT_RX"),

[RFC PATCH 03/14] ASoC: qcom: Add USB backend ASoC driver for Q6

[Wesley Cheng]

Create a USB BE component that will register a new USB port to the ASoC USB
framework.  This will handle determination on if the requested audio
profile is supported by the USB device currently selected.

Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
---
 include/sound/q6usboffload.h  |  20 +++
 sound/soc/qcom/Kconfig        |   4 +
 sound/soc/qcom/qdsp6/Makefile |   1 +
 sound/soc/qcom/qdsp6/q6usb.c  | 232 ++++++++++++++++++++++++++++++++++
 4 files changed, 257 insertions(+)
 create mode 100644 include/sound/q6usboffload.h
 create mode 100644 sound/soc/qcom/qdsp6/q6usb.c

diff --git a/include/sound/q6usboffload.h b/include/sound/q6usboffload.h
new file mode 100644
index 000000000000..e576808901d9
--- /dev/null
+++ b/include/sound/q6usboffload.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0
+ *
+ * linux/sound/q6usboffload.h -- QDSP6 USB offload
+ *
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+/**
+ * struct q6usb_offload
+ * @dev - dev handle to usb be
+ * @sid - streamID for iommu
+ * @intr_num - usb interrupter number
+ * @domain - allocated iommu domain
+ **/
+struct q6usb_offload {
+	struct device *dev;
+	u32 sid;
+	u32 intr_num;
+	struct iommu_domain *domain;
+};
diff --git a/sound/soc/qcom/Kconfig b/sound/soc/qcom/Kconfig
index 8c7398bc1ca8..d65c365116e5 100644
--- a/sound/soc/qcom/Kconfig
+++ b/sound/soc/qcom/Kconfig
@@ -111,6 +111,9 @@ config SND_SOC_QDSP6_APM
 config SND_SOC_QDSP6_PRM_LPASS_CLOCKS
 	tristate
 
+config SND_SOC_QDSP6_USB
+	tristate
+
 config SND_SOC_QDSP6_PRM
 	tristate
 	select SND_SOC_QDSP6_PRM_LPASS_CLOCKS
@@ -131,6 +134,7 @@ config SND_SOC_QDSP6
 	select SND_SOC_TOPOLOGY
 	select SND_SOC_QDSP6_APM
 	select SND_SOC_QDSP6_PRM
+	select SND_SOC_QDSP6_USB
 	help
 	 To add support for MSM QDSP6 Soc Audio.
 	 This will enable sound soc platform specific
diff --git a/sound/soc/qcom/qdsp6/Makefile b/sound/soc/qcom/qdsp6/Makefile
index 3963bf234664..c9457ee898d0 100644
--- a/sound/soc/qcom/qdsp6/Makefile
+++ b/sound/soc/qcom/qdsp6/Makefile
@@ -17,3 +17,4 @@ obj-$(CONFIG_SND_SOC_QDSP6_APM_DAI) += q6apm-dai.o
 obj-$(CONFIG_SND_SOC_QDSP6_APM_LPASS_DAI) += q6apm-lpass-dais.o
 obj-$(CONFIG_SND_SOC_QDSP6_PRM) += q6prm.o
 obj-$(CONFIG_SND_SOC_QDSP6_PRM_LPASS_CLOCKS) += q6prm-clocks.o
+obj-$(CONFIG_SND_SOC_QDSP6_USB) += q6usb.o
diff --git a/sound/soc/qcom/qdsp6/q6usb.c b/sound/soc/qcom/qdsp6/q6usb.c
new file mode 100644
index 000000000000..a9da6dec6c6f
--- /dev/null
+++ b/sound/soc/qcom/qdsp6/q6usb.c
@@ -0,0 +1,232 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include <linux/err.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+#include <linux/iommu.h>
+#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
+
+#include <sound/pcm.h>
+#include <sound/soc.h>
+#include <sound/soc-usb.h>
+#include <sound/pcm_params.h>
+#include <sound/asound.h>
+#include <sound/q6usboffload.h>
+
+#include "q6dsp-lpass-ports.h"
+#include "q6afe.h"
+
+struct q6usb_port_data {
+	struct q6afe_usb_cfg usb_cfg;
+	struct snd_soc_usb *usb;
+	struct q6usb_offload priv;
+	int active_idx;
+};
+
+static const struct snd_soc_dapm_widget q6usb_dai_widgets[] = {
+	SND_SOC_DAPM_HP("USB_RX_BE", NULL),
+};
+
+static const struct snd_soc_dapm_route q6usb_dapm_routes[] = {
+	{"USB Playback", NULL, "USB_RX_BE"},
+};
+
+static int q6usb_hw_params(struct snd_pcm_substream *substream,
+			   struct snd_pcm_hw_params *params,
+			   struct snd_soc_dai *dai)
+{
+	return 0;
+}
+static const struct snd_soc_dai_ops q6usb_ops = {
+	.hw_params	= q6usb_hw_params,
+};
+
+static struct snd_soc_dai_driver q6usb_be_dais[] = {
+	{
+		.playback = {
+			.stream_name = "USB BE RX",
+			.rates = SNDRV_PCM_RATE_8000 | SNDRV_PCM_RATE_11025 |
+					SNDRV_PCM_RATE_16000 | SNDRV_PCM_RATE_22050 |
+					SNDRV_PCM_RATE_32000 | SNDRV_PCM_RATE_44100 |
+					SNDRV_PCM_RATE_48000 | SNDRV_PCM_RATE_96000 |
+					SNDRV_PCM_RATE_192000,
+			.formats = SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S16_BE |
+					SNDRV_PCM_FMTBIT_U16_LE | SNDRV_PCM_FMTBIT_U16_BE |
+					SNDRV_PCM_FMTBIT_S24_LE | SNDRV_PCM_FMTBIT_S24_BE |
+					SNDRV_PCM_FMTBIT_U24_LE | SNDRV_PCM_FMTBIT_U24_BE,
+			.channels_min = 1,
+			.channels_max = 2,
+			.rate_max =     192000,
+			.rate_min =	8000,
+		},
+		.id = USB_RX,
+		.name = "USB_RX_BE",
+		.ops = &q6usb_ops,
+	},
+};
+
+int q6usb_audio_ports_of_xlate_dai_name(struct snd_soc_component *component,
+					const struct of_phandle_args *args,
+					const char **dai_name)
+{
+	int id = args->args[0];
+	int ret = -EINVAL;
+	int i;
+
+	for (i = 0; i  < ARRAY_SIZE(q6usb_be_dais); i++) {
+		if (q6usb_be_dais[i].id == id) {
+			*dai_name = q6usb_be_dais[i].name;
+			ret = 0;
+			break;
+		}
+	}
+
+	return ret;
+}
+
+static int q6usb_component_probe(struct snd_soc_component *component)
+{
+	struct q6usb_port_data *data = dev_get_drvdata(component->dev);
+	struct snd_soc_dapm_context *dapm = snd_soc_component_get_dapm(component);
+
+	data->usb->component = component;
+
+	snd_soc_dapm_disable_pin(dapm, "USB_RX_BE");
+	snd_soc_dapm_sync(dapm);
+
+	return 0;
+}
+
+static const struct snd_soc_component_driver q6usb_dai_component = {
+	.probe		= q6usb_component_probe,
+	.name		= "q6usb-dai-component",
+	.dapm_widgets = q6usb_dai_widgets,
+	.num_dapm_widgets = ARRAY_SIZE(q6usb_dai_widgets),
+	.dapm_routes = q6usb_dapm_routes,
+	.num_dapm_routes = ARRAY_SIZE(q6usb_dapm_routes),
+	.of_xlate_dai_name = q6usb_audio_ports_of_xlate_dai_name,
+};
+
+int q6usb_alsa_connection_cb(struct snd_soc_usb *usb, int card_idx,
+			int connected)
+{
+	struct snd_soc_dapm_context *dapm;
+	struct q6usb_port_data *data;
+
+	if (!usb->component)
+		return 0;
+
+	dapm = snd_soc_component_get_dapm(usb->component);
+	data = dev_get_drvdata(usb->component->dev);
+
+	if (connected) {
+		snd_soc_dapm_enable_pin(dapm, "USB_RX_BE");
+		/* We only track the latest USB headset plugged in */
+		data->active_idx = card_idx;
+	} else {
+		snd_soc_dapm_disable_pin(dapm, "USB_RX_BE");
+	}
+	snd_soc_dapm_sync(dapm);
+
+	return 0;
+}
+
+static int q6usb_dai_dev_probe(struct platform_device *pdev)
+{
+	struct device_node *node = pdev->dev.of_node;
+	struct q6usb_port_data *data;
+	struct device *dev = &pdev->dev;
+	int ret;
+
+	data = devm_kzalloc(dev, sizeof(*data), GFP_KERNEL);
+	if (!data)
+		return -ENOMEM;
+
+	ret = of_property_read_u32(node, "qcom,usb-audio-stream-id",
+				&data->priv.sid);
+	if (ret) {
+		dev_err(&pdev->dev, "failed to read sid.\n");
+		return -ENODEV;
+	}
+
+	ret = of_property_read_u32(node, "qcom,usb-audio-intr-num",
+				&data->priv.intr_num);
+	if (ret) {
+		dev_err(&pdev->dev, "failed to read intr num.\n");
+		return -ENODEV;
+	}
+
+	data->priv.domain = iommu_domain_alloc(pdev->dev.bus);
+	if (!data->priv.domain) {
+		dev_err(&pdev->dev, "failed to allocate iommu domain\n");
+		return -ENODEV;
+	}
+
+	/* attach to external processor iommu */
+	ret = iommu_attach_device(data->priv.domain, &pdev->dev);
+	if (ret) {
+		dev_err(&pdev->dev, "failed to attach device ret = %d\n", ret);
+		goto free_domain;
+	}
+
+	data->usb = snd_soc_usb_add_port(dev, q6usb_alsa_connection_cb);
+	if (IS_ERR(data->usb)) {
+		dev_err(&pdev->dev, "failed to add usb port\n");
+		goto detach_device;
+	}
+
+	data->priv.dev = dev;
+	dev_set_drvdata(dev, data);
+	devm_snd_soc_register_component(dev, &q6usb_dai_component,
+							q6usb_be_dais, ARRAY_SIZE(q6usb_be_dais));
+	snd_soc_usb_set_priv_data(&data->priv);
+
+	return 0;
+
+detach_device:
+	iommu_detach_device(data->priv.domain, &pdev->dev);
+free_domain:
+	iommu_domain_free(data->priv.domain);
+
+	return ret;
+}
+
+static int q6usb_dai_dev_remove(struct platform_device *pdev)
+{
+	struct q6usb_port_data *data = platform_get_drvdata(pdev);
+
+	iommu_detach_device(data->priv.domain, &pdev->dev);
+	iommu_domain_free(data->priv.domain);
+
+	snd_soc_usb_remove_port();
+
+	return 0;
+}
+
+#ifdef CONFIG_OF
+static const struct of_device_id q6usb_dai_device_id[] = {
+	{ .compatible = "qcom,q6usb-dais" },
+	{},
+};
+MODULE_DEVICE_TABLE(of, q6afe_dai_device_id);
+#endif
+
+static struct platform_driver q6usb_dai_platform_driver = {
+	.driver = {
+		.name = "q6usb-dai",
+		.of_match_table = of_match_ptr(q6usb_dai_device_id),
+	},
+	.probe = q6usb_dai_dev_probe,
+	.remove = q6usb_dai_dev_remove,
+};
+module_platform_driver(q6usb_dai_platform_driver);
+
+MODULE_DESCRIPTION("Q6 USB backend dai driver");
+MODULE_LICENSE("GPL");

[RFC PATCH 04/14] sound: usb: card: Introduce USB SND vendor op callbacks

[Wesley Cheng]

Allow for different vendors to be notified on USB SND connect/disconnect
seqeunces.  This allows for vendor USB SND modules to properly initialize
and populate internal structures with references to the USB SND chip
device.

Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
---
 sound/usb/card.c | 22 ++++++++++++++++++++++
 sound/usb/card.h |  7 +++++++
 2 files changed, 29 insertions(+)

diff --git a/sound/usb/card.c b/sound/usb/card.c
index 26268ffb8274..212f55a7683c 100644
--- a/sound/usb/card.c
+++ b/sound/usb/card.c
@@ -117,6 +117,21 @@ MODULE_PARM_DESC(skip_validation, "Skip unit descriptor validation (default: no)
 static DEFINE_MUTEX(register_mutex);
 static struct snd_usb_audio *usb_chip[SNDRV_CARDS];
 static struct usb_driver usb_audio_driver;
+static struct snd_usb_vendor_ops *vendor_ops;
+
+int snd_usb_register_vendor_ops(struct snd_usb_vendor_ops *ops)
+{
+	vendor_ops = ops;
+	return 0;
+}
+EXPORT_SYMBOL_GPL(snd_usb_register_vendor_ops);
+
+int snd_usb_unregister_vendor_ops(void)
+{
+	vendor_ops = NULL;
+	return 0;
+}
+EXPORT_SYMBOL_GPL(snd_usb_unregister_vendor_ops);
 
 /*
  * disconnect streams
@@ -910,6 +925,10 @@ static int usb_audio_probe(struct usb_interface *intf,
 	usb_set_intfdata(intf, chip);
 	atomic_dec(&chip->active);
 	mutex_unlock(&register_mutex);
+
+	if (vendor_ops->connect_cb)
+		vendor_ops->connect_cb(intf, chip);
+
 	return 0;
 
  __error:
@@ -943,6 +962,9 @@ static void usb_audio_disconnect(struct usb_interface *intf)
 	if (chip == USB_AUDIO_IFACE_UNUSED)
 		return;
 
+	if (vendor_ops->disconnect_cb)
+		vendor_ops->disconnect_cb(intf);
+
 	card = chip->card;
 
 	mutex_lock(&register_mutex);
diff --git a/sound/usb/card.h b/sound/usb/card.h
index 40061550105a..a785bb256b0d 100644
--- a/sound/usb/card.h
+++ b/sound/usb/card.h
@@ -206,4 +206,11 @@ struct snd_usb_stream {
 	struct list_head list;
 };
 
+struct snd_usb_vendor_ops {
+	void (*connect_cb)(struct usb_interface *intf, struct snd_usb_audio *chip);
+	void (*disconnect_cb)(struct usb_interface *intf);
+};
+
+int snd_usb_register_vendor_ops(struct snd_usb_vendor_ops *ops);
+int snd_usb_unregister_vendor_ops(void);
 #endif /* __USBAUDIO_CARD_H */

[RFC PATCH 05/14] sound: usb: Export USB SND APIs for modules

[Wesley Cheng]

Some vendor modules will utilize useful parsing and endpoint management
APIs to start audio playback/capture.

Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
---
 sound/usb/card.c     |  2 ++
 sound/usb/endpoint.c |  2 ++
 sound/usb/helper.c   |  1 +
 sound/usb/pcm.c      |  9 ++++++---
 sound/usb/pcm.h      | 12 ++++++++++++
 5 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/sound/usb/card.c b/sound/usb/card.c
index 212f55a7683c..396e5a34e23b 100644
--- a/sound/usb/card.c
+++ b/sound/usb/card.c
@@ -1068,6 +1068,7 @@ int snd_usb_autoresume(struct snd_usb_audio *chip)
 	}
 	return 0;
 }
+EXPORT_SYMBOL(snd_usb_autoresume);
 
 void snd_usb_autosuspend(struct snd_usb_audio *chip)
 {
@@ -1081,6 +1082,7 @@ void snd_usb_autosuspend(struct snd_usb_audio *chip)
 	for (i = 0; i < chip->num_interfaces; i++)
 		usb_autopm_put_interface(chip->intf[i]);
 }
+EXPORT_SYMBOL(snd_usb_autosuspend);
 
 static int usb_audio_suspend(struct usb_interface *intf, pm_message_t message)
 {
diff --git a/sound/usb/endpoint.c b/sound/usb/endpoint.c
index 310cd6fb0038..1663f9e9cb4b 100644
--- a/sound/usb/endpoint.c
+++ b/sound/usb/endpoint.c
@@ -858,6 +858,7 @@ snd_usb_endpoint_open(struct snd_usb_audio *chip,
 	mutex_unlock(&chip->mutex);
 	return ep;
 }
+EXPORT_SYMBOL_GPL(snd_usb_endpoint_open);
 
 /*
  * snd_usb_endpoint_set_sync: Link data and sync endpoints
@@ -1525,6 +1526,7 @@ int snd_usb_endpoint_get_clock_rate(struct snd_usb_audio *chip, int clock)
 	mutex_unlock(&chip->mutex);
 	return rate;
 }
+EXPORT_SYMBOL(snd_usb_endpoint_prepare);
 
 /**
  * snd_usb_endpoint_start: start an snd_usb_endpoint
diff --git a/sound/usb/helper.c b/sound/usb/helper.c
index a4410267bf70..b4ed9ef3eeb3 100644
--- a/sound/usb/helper.c
+++ b/sound/usb/helper.c
@@ -62,6 +62,7 @@ void *snd_usb_find_csint_desc(void *buffer, int buflen, void *after, u8 dsubtype
 	}
 	return NULL;
 }
+EXPORT_SYMBOL_GPL(snd_usb_find_csint_desc);
 
 /*
  * Wrapper for usb_control_msg().
diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c
index 8ed165f036a0..624f555c6657 100644
--- a/sound/usb/pcm.c
+++ b/sound/usb/pcm.c
@@ -87,7 +87,7 @@ static snd_pcm_uframes_t snd_usb_pcm_pointer(struct snd_pcm_substream *substream
 /*
  * find a matching audio format
  */
-static const struct audioformat *
+const struct audioformat *
 find_format(struct list_head *fmt_list_head, snd_pcm_format_t format,
 	    unsigned int rate, unsigned int channels, bool strict_match,
 	    struct snd_usb_substream *subs)
@@ -147,8 +147,9 @@ find_format(struct list_head *fmt_list_head, snd_pcm_format_t format,
 	}
 	return found;
 }
+EXPORT_SYMBOL(find_format);
 
-static const struct audioformat *
+const struct audioformat *
 find_substream_format(struct snd_usb_substream *subs,
 		      const struct snd_pcm_hw_params *params)
 {
@@ -156,6 +157,7 @@ find_substream_format(struct snd_usb_substream *subs,
 			   params_rate(params), params_channels(params),
 			   true, subs);
 }
+EXPORT_SYMBOL(find_substream_format);
 
 static int init_pitch_v1(struct snd_usb_audio *chip, int ep)
 {
@@ -418,7 +420,7 @@ int snd_usb_pcm_resume(struct snd_usb_stream *as)
 	return 0;
 }
 
-static void close_endpoints(struct snd_usb_audio *chip,
+void close_endpoints(struct snd_usb_audio *chip,
 			    struct snd_usb_substream *subs)
 {
 	if (subs->data_endpoint) {
@@ -432,6 +434,7 @@ static void close_endpoints(struct snd_usb_audio *chip,
 		subs->sync_endpoint = NULL;
 	}
 }
+EXPORT_SYMBOL(close_endpoints);
 
 /*
  * hw_params callback
diff --git a/sound/usb/pcm.h b/sound/usb/pcm.h
index 493a4e34d78d..43a4a03dfce7 100644
--- a/sound/usb/pcm.h
+++ b/sound/usb/pcm.h
@@ -13,4 +13,16 @@ void snd_usb_preallocate_buffer(struct snd_usb_substream *subs);
 int snd_usb_audioformat_set_sync_ep(struct snd_usb_audio *chip,
 				    struct audioformat *fmt);
 
+void close_endpoints(struct snd_usb_audio *chip,
+			    struct snd_usb_substream *subs);
+int configure_endpoints(struct snd_usb_audio *chip,
+			       struct snd_usb_substream *subs);
+
+const struct audioformat *
+find_format(struct list_head *fmt_list_head, snd_pcm_format_t format,
+	    unsigned int rate, unsigned int channels, bool strict_match,
+	    struct snd_usb_substream *subs);
+const struct audioformat *
+find_substream_format(struct snd_usb_substream *subs,
+		      const struct snd_pcm_hw_params *params);
 #endif /* __USBAUDIO_PCM_H */

[RFC PATCH 06/14] usb: core: hcd: Introduce USB HCD APIs for interrupter management

[Wesley Cheng]

For USB HCDs that can support multiple USB interrupters, expose functions
that class drivers can utilize for setting up secondary interrupters.
Class drivers can pass this information to its respective clients, i.e.
a dedicated DSP.

Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
---
 drivers/usb/core/hcd.c  | 86 +++++++++++++++++++++++++++++++++++++++++
 include/linux/usb.h     |  7 ++++
 include/linux/usb/hcd.h | 16 +++++++-
 3 files changed, 108 insertions(+), 1 deletion(-)

diff --git a/drivers/usb/core/hcd.c b/drivers/usb/core/hcd.c
index 8300baedafd2..90ead90faf1d 100644
--- a/drivers/usb/core/hcd.c
+++ b/drivers/usb/core/hcd.c
@@ -1579,6 +1579,92 @@ int usb_hcd_submit_urb (struct urb *urb, gfp_t mem_flags)
 	return status;
 }
 
+/**
+ * usb_free_interrupter - free an interrupter
+ * @udev: usb device which requested the interrupter
+ * @intr_num: interrupter number to free
+ *
+ * Release the USB HCD interrupter that was reserved by a USB class driver.
+ **/
+int usb_free_interrupter(struct usb_device *udev, int intr_num)
+{
+	struct usb_hcd *hcd = bus_to_hcd(udev->bus);
+	int ret = 0;
+
+	if (hcd->driver->free_interrupter)
+		ret = hcd->driver->free_interrupter(hcd, intr_num);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(usb_free_interrupter);
+
+/**
+ * usb_set_interruper - Reserve an interrupter
+ * @udev: usb device which requested the interrupter
+ * @intr_num: interrupter number to reserve
+ * @dma: iova address to event ring
+ *
+ * Request for a specific interrupter to be reserved for a USB class driver.
+ * This will return the base address to the event ring that was allocated to
+ * the specific interrupter.
+ **/
+phys_addr_t usb_set_interruper(struct usb_device *udev, int intr_num,
+							dma_addr_t *dma)
+{
+	struct usb_hcd *hcd;
+	phys_addr_t pa = 0;
+
+	hcd = bus_to_hcd(udev->bus);
+	if (hcd->driver->update_interrupter)
+		pa = hcd->driver->update_interrupter(hcd, intr_num, dma);
+
+	return pa;
+}
+EXPORT_SYMBOL_GPL(usb_set_interruper);
+
+/**
+ * usb_hcd_get_transfer_resource - Retrieve USB EP resource information
+ * @udev: usb device
+ * @ep: usb ep to retrieve resource information
+ * @dma: iova address to transfer resource
+ *
+ * Request for USB EP transfer resource which is used for submitting
+ * transfers to the USB HCD.
+ **/
+phys_addr_t usb_hcd_get_transfer_resource(struct usb_device *udev,
+				struct usb_host_endpoint *ep, dma_addr_t *dma)
+{
+	struct usb_hcd *hcd;
+	phys_addr_t pa = 0;
+
+	hcd = bus_to_hcd(udev->bus);
+	if (hcd->driver->get_transfer_resource)
+		pa = hcd->driver->get_transfer_resource(udev, ep, dma);
+
+	return pa;
+}
+EXPORT_SYMBOL_GPL(usb_hcd_get_transfer_resource);
+
+/**
+ * usb_hcd_stop_endpoint - Halt USB EP transfers
+ * @udev: usb device
+ * @ep: usb ep to stop
+ *
+ * Stop pending transfers on a specific USB endpoint.
+ **/
+int usb_hcd_stop_endpoint(struct usb_device *udev,
+					struct usb_host_endpoint *ep)
+{
+	struct usb_hcd *hcd = bus_to_hcd(udev->bus);
+	int ret = 0;
+
+	if (hcd->driver->stop_endpoint)
+		ret = hcd->driver->stop_endpoint(hcd, udev, ep);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(usb_hcd_stop_endpoint);
+
 /*-------------------------------------------------------------------------*/
 
 /* this makes the hcd giveback() the urb more quickly, by kicking it
diff --git a/include/linux/usb.h b/include/linux/usb.h
index d4afeeec1e1a..2f71cd4fb6e0 100644
--- a/include/linux/usb.h
+++ b/include/linux/usb.h
@@ -1724,6 +1724,13 @@ static inline void usb_fill_int_urb(struct urb *urb,
 	urb->start_frame = -1;
 }
 
+extern int usb_free_interrupter(struct usb_device *udev, int intr_num);
+extern phys_addr_t usb_set_interruper(struct usb_device *udev,
+						int intr_num, dma_addr_t *dma);
+extern int usb_hcd_stop_endpoint(struct usb_device *udev,
+						struct usb_host_endpoint *ep);
+extern phys_addr_t usb_hcd_get_transfer_resource(struct usb_device *udev,
+						struct usb_host_endpoint *ep, dma_addr_t *dma);
 extern void usb_init_urb(struct urb *urb);
 extern struct urb *usb_alloc_urb(int iso_packets, gfp_t mem_flags);
 extern void usb_free_urb(struct urb *urb);
diff --git a/include/linux/usb/hcd.h b/include/linux/usb/hcd.h
index b51c07111729..f5bce80b2e40 100644
--- a/include/linux/usb/hcd.h
+++ b/include/linux/usb/hcd.h
@@ -343,7 +343,21 @@ struct hc_driver {
 	int	(*free_streams)(struct usb_hcd *hcd, struct usb_device *udev,
 		struct usb_host_endpoint **eps, unsigned int num_eps,
 		gfp_t mem_flags);
-
+	/* Frees an interrupter from the current client, and makes it available
+	 * for use.
+	 */
+	int (*free_interrupter)(struct usb_hcd *hcd, int intr_num);
+	/* Request an interrupter from the current allocated pool.  Will provide
+	 * the address to the allocated ring.
+	 */
+	phys_addr_t (*update_interrupter)(struct usb_hcd *hcd, int intr_num,
+		dma_addr_t *dma);
+	/* Fetch transfer/ep ring address */
+	phys_addr_t (*get_transfer_resource)(struct usb_device *udev,
+		struct usb_host_endpoint *ep, dma_addr_t *dma);
+	/* Stop transfers on a particular endpoint */
+	int	(*stop_endpoint)(struct usb_hcd *hcd, struct usb_device *udev,
+		struct usb_host_endpoint *ep);
 	/* Bandwidth computation functions */
 	/* Note that add_endpoint() can only be called once per endpoint before
 	 * check_bandwidth() or reset_bandwidth() must be called.

[RFC PATCH 07/14] usb: host: xhci: Add XHCI secondary interrupter support

[Wesley Cheng]

Implement the XHCI operations for allocating and requesting for a secondary
interrupter.  The secondary interrupter can allow for events for a
particular endpoint to be routed to a separate event ring.  The event
routing is defined when submitting a transfer descriptor to the USB HW.
There is a specific field which denotes which interrupter ring to route the
event to when the transfer is completed.

An example use case, such as audio packet offloading can utilize a separate
event ring, so that these events can be routed to a different processor
within the system.  The processor would be able to independently submit
transfers and handle its completions without intervention from the main
processor.

Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
---
 drivers/usb/host/xhci-mem.c  | 219 ++++++++++++++++++++++++++++-------
 drivers/usb/host/xhci-plat.c |   2 +
 drivers/usb/host/xhci.c      | 169 ++++++++++++++++++++++++++-
 drivers/usb/host/xhci.h      |  15 +++
 4 files changed, 363 insertions(+), 42 deletions(-)

diff --git a/drivers/usb/host/xhci-mem.c b/drivers/usb/host/xhci-mem.c
index 81ca2bc1f0be..d5cb4b82ad3d 100644
--- a/drivers/usb/host/xhci-mem.c
+++ b/drivers/usb/host/xhci-mem.c
@@ -1835,6 +1835,7 @@ void xhci_free_erst(struct xhci_hcd *xhci, struct xhci_erst *erst)
 void xhci_mem_cleanup(struct xhci_hcd *xhci)
 {
 	struct device	*dev = xhci_to_hcd(xhci)->self.sysdev;
+	struct xhci_sec *sec, *tmp;
 	int i, j, num_ports;
 
 	cancel_delayed_work_sync(&xhci->cmd_timer);
@@ -1846,6 +1847,16 @@ void xhci_mem_cleanup(struct xhci_hcd *xhci)
 	xhci->event_ring = NULL;
 	xhci_dbg_trace(xhci, trace_xhci_dbg_init, "Freed event ring");
 
+	list_for_each_entry_safe(sec, tmp, &xhci->xhci_sec_list, list) {
+		list_del(&sec->list);
+		if (sec->event_ring) {
+			xhci_ring_free(xhci, sec->event_ring);
+			xhci_dbg_trace(xhci, trace_xhci_dbg_init,
+						"Freed secondary ring %d", sec->intr_num);
+		}
+		kfree(sec);
+	}
+
 	if (xhci->cmd_ring)
 		xhci_ring_free(xhci, xhci->cmd_ring);
 	xhci->cmd_ring = NULL;
@@ -2087,18 +2098,18 @@ static int xhci_check_trb_in_td_math(struct xhci_hcd *xhci)
 	return 0;
 }
 
-static void xhci_set_hc_event_deq(struct xhci_hcd *xhci)
+static void xhci_set_hc_event_deq(struct xhci_hcd *xhci, struct xhci_ring *er,
+				struct xhci_intr_reg __iomem *ir_set)
 {
 	u64 temp;
 	dma_addr_t deq;
 
-	deq = xhci_trb_virt_to_dma(xhci->event_ring->deq_seg,
-			xhci->event_ring->dequeue);
+	deq = xhci_trb_virt_to_dma(er->deq_seg, er->dequeue);
 	if (!deq)
 		xhci_warn(xhci, "WARN something wrong with SW event ring "
 				"dequeue ptr.\n");
 	/* Update HC event ring dequeue pointer */
-	temp = xhci_read_64(xhci, &xhci->ir_set->erst_dequeue);
+	temp = xhci_read_64(xhci, &ir_set->erst_dequeue);
 	temp &= ERST_PTR_MASK;
 	/* Don't clear the EHB bit (which is RW1C) because
 	 * there might be more events to service.
@@ -2108,7 +2119,7 @@ static void xhci_set_hc_event_deq(struct xhci_hcd *xhci)
 			"// Write event ring dequeue pointer, "
 			"preserving EHB bit");
 	xhci_write_64(xhci, ((u64) deq & (u64) ~ERST_PTR_MASK) | temp,
-			&xhci->ir_set->erst_dequeue);
+			&ir_set->erst_dequeue);
 }
 
 static void xhci_add_in_port(struct xhci_hcd *xhci, unsigned int num_ports,
@@ -2375,10 +2386,159 @@ static int xhci_setup_port_arrays(struct xhci_hcd *xhci, gfp_t flags)
 	return 0;
 }
 
+static void xhci_reset_evt_ring_base(struct xhci_hcd *xhci, struct xhci_ring *er,
+		struct xhci_erst *erst, struct xhci_intr_reg __iomem *ir_set)
+{
+	u64	val_64;
+
+	xhci_dbg_trace(xhci, trace_xhci_dbg_init,
+			"// Set ERST entries to point to event ring.");
+	/* set the segment table base address */
+	xhci_dbg_trace(xhci, trace_xhci_dbg_init,
+			"// Set ERST base address for ir_set 0 = 0x%llx",
+			(unsigned long long)erst->erst_dma_addr);
+	val_64 = xhci_read_64(xhci, &ir_set->erst_base);
+	val_64 &= ERST_PTR_MASK;
+	pr_err("after clearing ptr = 0x%llx\n", val_64);
+	val_64 |= (erst->erst_dma_addr & (u64) ~ERST_PTR_MASK);
+		pr_err("after clearing ptr = 0x%llx\n", val_64);
+
+	xhci_write_64(xhci, val_64, &ir_set->erst_base);
+	xhci_dbg_trace(xhci, trace_xhci_dbg_init,
+			"done.");
+	/* Set the event ring dequeue address */
+	xhci_set_hc_event_deq(xhci, er, ir_set);
+	xhci_dbg_trace(xhci, trace_xhci_dbg_init,
+			"Wrote ERST address to ir_set 0.");
+}
+
+void xhci_handle_sec_intr_events(struct xhci_hcd *xhci, struct xhci_ring *ring,
+	struct xhci_intr_reg __iomem *ir_set, struct xhci_sec *sec)
+{
+	union xhci_trb *erdp_trb, *current_trb;
+	struct xhci_segment	*seg;
+	u64 erdp_reg;
+	u32 iman_reg;
+	dma_addr_t deq;
+	unsigned long segment_offset;
+	struct xhci_erst *erst = &sec->erst;
+
+	/* disable irq, ack pending interrupt and ack all pending events */
+
+	iman_reg = readl_relaxed(&ir_set->irq_pending);
+	iman_reg &= ~IMAN_IE;
+	writel_relaxed(iman_reg, &ir_set->irq_pending);
+	iman_reg = readl_relaxed(&ir_set->irq_pending);
+	if (iman_reg & IMAN_IP)
+		writel_relaxed(iman_reg, &ir_set->irq_pending);
+
+	/* last acked event trb is in erdp reg  */
+	erdp_reg = xhci_read_64(xhci, &ir_set->erst_dequeue);
+	deq = (dma_addr_t)(erdp_reg & ~ERST_PTR_MASK);
+	if (!deq) {
+		pr_debug("%s: event ring handling not required\n", __func__);
+		return;
+	}
+
+	seg = ring->first_seg;
+	segment_offset = deq - seg->dma;
+
+	/* find out virtual address of the last acked event trb */
+	erdp_trb = current_trb = &seg->trbs[0] +
+				(segment_offset/sizeof(*current_trb));
+
+	/* read cycle state of the last acked trb to find out CCS */
+	ring->cycle_state = le32_to_cpu(current_trb->event_cmd.flags) & TRB_CYCLE;
+
+	while (1) {
+		/* last trb of the event ring: toggle cycle state */
+		if (current_trb == &seg->trbs[TRBS_PER_SEGMENT - 1]) {
+			ring->cycle_state ^= 1;
+			current_trb = &seg->trbs[0];
+		} else {
+			current_trb++;
+		}
+
+		/* cycle state transition */
+		if ((le32_to_cpu(current_trb->event_cmd.flags) & TRB_CYCLE) !=
+		    ring->cycle_state)
+			break;
+	}
+
+	if (erdp_trb != current_trb) {
+		deq = xhci_trb_virt_to_dma(ring->deq_seg, current_trb);
+		if (deq == 0)
+			xhci_warn(xhci,
+				"WARN invalid SW event ring dequeue ptr.\n");
+		/* Update HC event ring dequeue pointer */
+		erdp_reg &= ERST_PTR_MASK;
+		erdp_reg |= ((u64) deq & (u64) ~ERST_PTR_MASK);
+	}
+
+	/* Clear the event handler busy flag (RW1C); event ring is empty. */
+	erdp_reg |= ERST_EHB;
+	xhci_write_64(xhci, erdp_reg, &ir_set->erst_dequeue);
+
+	xhci_reset_evt_ring_base(xhci, ring, erst, ir_set);
+}
+
+static int xhci_event_ring_setup(struct xhci_hcd *xhci, struct xhci_ring **er,
+	struct xhci_erst *erst,	struct xhci_intr_reg __iomem *ir_set,
+	unsigned int intr_num, gfp_t flags)
+{
+	int ret;
+	unsigned int	val;
+
+	/*
+	 * Event ring setup: Allocate a normal ring, but also setup
+	 * the event ring segment table (ERST).  Section 4.9.3.
+	 */
+	xhci_dbg_trace(xhci, trace_xhci_dbg_init, "// Allocating event ring");
+	*er = xhci_ring_alloc(xhci, ERST_NUM_SEGS, 1, TYPE_EVENT,
+					0, flags);
+	if (!*er)
+		return -ENOMEM;
+
+	ret = xhci_alloc_erst(xhci, *er, erst, flags);
+	if (ret)
+		return ret;
+
+	/* set ERST count with the number of entries in the segment table */
+	val = readl(&ir_set->erst_size);
+	val &= ERST_SIZE_MASK;
+	val |= ERST_NUM_SEGS;
+	xhci_dbg_trace(xhci, trace_xhci_dbg_init,
+			"// Write ERST size = %i to ir_set 0 (some bits preserved)",
+			val);
+	writel(val, &ir_set->erst_size);
+
+	xhci_reset_evt_ring_base(xhci, *er, erst, ir_set);
+
+	return 0;
+}
+
+int xhci_mem_sec_init(struct xhci_hcd *xhci, struct xhci_sec *sec)
+{
+	int ret;
+
+	sec->ir_set = &xhci->run_regs->ir_set[sec->intr_num];
+	ret = xhci_event_ring_setup(xhci, &sec->event_ring,
+				&sec->erst, sec->ir_set, sec->intr_num, GFP_KERNEL);
+	if (ret) {
+		xhci_err(xhci, "sec event ring setup failed inter#%d\n",
+			sec->intr_num);
+		return ret;
+	}
+	sec->xhci = 0;
+
+	return 0;
+}
+
 int xhci_mem_init(struct xhci_hcd *xhci, gfp_t flags)
 {
 	dma_addr_t	dma;
 	struct device	*dev = xhci_to_hcd(xhci)->self.sysdev;
+	struct xhci_sec *sec;
 	unsigned int	val, val2;
 	u64		val_64;
 	u32		page_size, temp;
@@ -2497,46 +2657,23 @@ int xhci_mem_init(struct xhci_hcd *xhci, gfp_t flags)
 	/* Set ir_set to interrupt register set 0 */
 	xhci->ir_set = &xhci->run_regs->ir_set[0];
 
-	/*
-	 * Event ring setup: Allocate a normal ring, but also setup
-	 * the event ring segment table (ERST).  Section 4.9.3.
-	 */
-	xhci_dbg_trace(xhci, trace_xhci_dbg_init, "// Allocating event ring");
-	xhci->event_ring = xhci_ring_alloc(xhci, ERST_NUM_SEGS, 1, TYPE_EVENT,
-					0, flags);
-	if (!xhci->event_ring)
-		goto fail;
-	if (xhci_check_trb_in_td_math(xhci) < 0)
+	ret = xhci_event_ring_setup(xhci, &xhci->event_ring, &xhci->erst, xhci->ir_set, 0, flags);
+	if (ret < 0)
 		goto fail;
 
-	ret = xhci_alloc_erst(xhci, xhci->event_ring, &xhci->erst, flags);
-	if (ret)
+	if (xhci_check_trb_in_td_math(xhci) < 0)
 		goto fail;
 
-	/* set ERST count with the number of entries in the segment table */
-	val = readl(&xhci->ir_set->erst_size);
-	val &= ERST_SIZE_MASK;
-	val |= ERST_NUM_SEGS;
-	xhci_dbg_trace(xhci, trace_xhci_dbg_init,
-			"// Write ERST size = %i to ir_set 0 (some bits preserved)",
-			val);
-	writel(val, &xhci->ir_set->erst_size);
-
-	xhci_dbg_trace(xhci, trace_xhci_dbg_init,
-			"// Set ERST entries to point to event ring.");
-	/* set the segment table base address */
-	xhci_dbg_trace(xhci, trace_xhci_dbg_init,
-			"// Set ERST base address for ir_set 0 = 0x%llx",
-			(unsigned long long)xhci->erst.erst_dma_addr);
-	val_64 = xhci_read_64(xhci, &xhci->ir_set->erst_base);
-	val_64 &= ERST_PTR_MASK;
-	val_64 |= (xhci->erst.erst_dma_addr & (u64) ~ERST_PTR_MASK);
-	xhci_write_64(xhci, val_64, &xhci->ir_set->erst_base);
-
-	/* Set the event ring dequeue address */
-	xhci_set_hc_event_deq(xhci);
-	xhci_dbg_trace(xhci, trace_xhci_dbg_init,
-			"Wrote ERST address to ir_set 0.");
+	/* Setup secondary interrupters (if any) */
+	INIT_LIST_HEAD(&xhci->xhci_sec_list);
+	for (i = 0; i < xhci->max_interrupters; i++) {
+		sec = kzalloc(sizeof(struct xhci_sec), GFP_KERNEL);
+		if (sec) {
+			sec->intr_num = i + 1;
+			xhci_mem_sec_init(xhci, sec);
+			list_add_tail(&sec->list, &xhci->xhci_sec_list);
+		}
+	}
 
 	xhci->isoc_bei_interval = AVOID_BEI_INTERVAL_MAX;
 
diff --git a/drivers/usb/host/xhci-plat.c b/drivers/usb/host/xhci-plat.c
index 5fb55bf19493..a1b6c17ecf74 100644
--- a/drivers/usb/host/xhci-plat.c
+++ b/drivers/usb/host/xhci-plat.c
@@ -303,6 +303,8 @@ static int xhci_plat_probe(struct platform_device *pdev)
 
 		device_property_read_u32(tmpdev, "imod-interval-ns",
 					 &xhci->imod_interval);
+		device_property_read_u8(tmpdev, "num-hc-interrupters",
+					 &xhci->max_interrupters);
 	}
 
 	hcd->usb_phy = devm_usb_get_phy_by_phandle(sysdev, "usb-phy", 0);
diff --git a/drivers/usb/host/xhci.c b/drivers/usb/host/xhci.c
index 79d7931c048a..353b06df2000 100644
--- a/drivers/usb/host/xhci.c
+++ b/drivers/usb/host/xhci.c
@@ -2101,6 +2101,64 @@ int xhci_add_endpoint(struct usb_hcd *hcd, struct usb_device *udev,
 }
 EXPORT_SYMBOL_GPL(xhci_add_endpoint);
 
+static int xhci_stop_endpoint(struct usb_hcd *hcd, struct usb_device *udev,
+						struct usb_host_endpoint *ep)
+{
+	struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+	unsigned int ep_index;
+	struct xhci_virt_device *virt_dev;
+	struct xhci_command *cmd;
+	unsigned long flags;
+	int ret = 0;
+
+	ret = xhci_check_args(hcd, udev, ep, 1, true, __func__);
+	if (ret <= 0)
+		return ret;
+
+	cmd = xhci_alloc_command(xhci, true, GFP_NOIO);
+	if (!cmd)
+		return -ENOMEM;
+
+	spin_lock_irqsave(&xhci->lock, flags);
+	virt_dev = xhci->devs[udev->slot_id];
+	if (!virt_dev) {
+		ret = -ENODEV;
+		goto err;
+	}
+
+	ep_index = xhci_get_endpoint_index(&ep->desc);
+	if (virt_dev->eps[ep_index].ring &&
+			virt_dev->eps[ep_index].ring->dequeue) {
+		ret = xhci_queue_stop_endpoint(xhci, cmd, udev->slot_id,
+				ep_index, 0);
+		if (ret)
+			goto err;
+
+		xhci_ring_cmd_db(xhci);
+		spin_unlock_irqrestore(&xhci->lock, flags);
+
+		/* Wait for stop endpoint command to finish */
+		wait_for_completion(cmd->completion);
+
+		if (cmd->status == COMP_COMMAND_ABORTED ||
+				cmd->status == COMP_STOPPED) {
+			xhci_warn(xhci,
+				"stop endpoint command timeout for ep%d%s\n",
+				usb_endpoint_num(&ep->desc),
+				usb_endpoint_dir_in(&ep->desc) ? "in" : "out");
+			ret = -ETIME;
+				}
+		goto free_cmd;
+	}
+
+err:
+	spin_unlock_irqrestore(&xhci->lock, flags);
+free_cmd:
+	xhci_free_command(xhci, cmd);
+
+	return ret;
+}
+
 static void xhci_zero_in_ctx(struct xhci_hcd *xhci, struct xhci_virt_device *virt_dev)
 {
 	struct xhci_input_control_ctx *ctrl_ctx;
@@ -5278,6 +5336,110 @@ static void xhci_hcd_init_usb3_data(struct xhci_hcd *xhci, struct usb_hcd *hcd)
 	xhci->usb3_rhub.hcd = hcd;
 }
 
+/*
+ * Free a XHCI interrupter that was previously allocated.  Ensure that the
+ * event ring which was used is reset to the proper state, and recycle the
+ * interrupter for next use by clearing the XHCI reference.
+ */
+static int xhci_free_interrupter(struct usb_hcd *hcd, int intr_num)
+{
+	struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+	struct xhci_sec *sec;
+
+	mutex_lock(&xhci->mutex);
+	list_for_each_entry(sec, &xhci->xhci_sec_list, list) {
+		if (sec->intr_num == intr_num) {
+			xhci_handle_sec_intr_events(xhci, sec->event_ring, sec->ir_set, sec);
+			sec->xhci = 0;
+		}
+	}
+	mutex_unlock(&xhci->mutex);
+
+	return 0;
+}
+
+/*
+ * Reserve a XHCI interrupter, and pass the base address of the event ring for
+ * this pariticular interrupter back to the client.
+ */
+static phys_addr_t xhci_update_interrupter(struct usb_hcd *hcd, int intr_num,
+								dma_addr_t *dma)
+{
+	struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+	struct device *dev = hcd->self.sysdev;
+	struct sg_table sgt;
+	phys_addr_t pa;
+	struct xhci_sec *sec;
+
+	if (!HCD_RH_RUNNING(hcd) ||
+			(xhci->xhc_state & XHCI_STATE_HALTED))
+		return 0;
+
+	mutex_lock(&xhci->mutex);
+	list_for_each_entry(sec, &xhci->xhci_sec_list, list) {
+		if (sec->intr_num == intr_num) {
+			dma_get_sgtable(dev, &sgt, sec->event_ring->first_seg->trbs,
+				sec->event_ring->first_seg->dma, TRB_SEGMENT_SIZE);
+
+			*dma = sec->event_ring->first_seg->dma;
+
+			pa = page_to_phys(sg_page(sgt.sgl));
+			sg_free_table(&sgt);
+			sec->xhci = xhci;
+			mutex_unlock(&xhci->mutex);
+
+			return pa;
+		}
+	}
+	mutex_unlock(&xhci->mutex);
+
+	return 0;
+}
+
+/* Retrieve the transfer ring base address for a specific endpoint. */
+static phys_addr_t xhci_get_xfer_resource(struct usb_device *udev,
+					struct usb_host_endpoint *ep, dma_addr_t *dma)
+{
+	struct usb_hcd *hcd = bus_to_hcd(udev->bus);
+	struct device *dev = hcd->self.sysdev;
+	struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+	struct sg_table sgt;
+	phys_addr_t pa;
+	int ret;
+	unsigned int ep_index;
+	struct xhci_virt_device *virt_dev;
+
+	if (!HCD_RH_RUNNING(hcd))
+		return 0;
+
+	ret = xhci_check_args(hcd, udev, ep, 1, true, __func__);
+	if (ret <= 0) {
+		xhci_err(xhci, "%s: invalid args\n", __func__);
+		return 0;
+	}
+
+	virt_dev = xhci->devs[udev->slot_id];
+	ep_index = xhci_get_endpoint_index(&ep->desc);
+
+	if (virt_dev->eps[ep_index].ring &&
+		virt_dev->eps[ep_index].ring->first_seg) {
+
+		dma_get_sgtable(dev, &sgt,
+			virt_dev->eps[ep_index].ring->first_seg->trbs,
+			virt_dev->eps[ep_index].ring->first_seg->dma,
+			TRB_SEGMENT_SIZE);
+
+		*dma = virt_dev->eps[ep_index].ring->first_seg->dma;
+
+		pa = page_to_phys(sg_page(sgt.sgl));
+		sg_free_table(&sgt);
+
+		return pa;
+	}
+
+	return 0;
+}
+
 int xhci_gen_setup(struct usb_hcd *hcd, xhci_get_quirks_t get_quirks)
 {
 	struct xhci_hcd		*xhci;
@@ -5321,6 +5483,8 @@ int xhci_gen_setup(struct usb_hcd *hcd, xhci_get_quirks_t get_quirks)
 		xhci->hcc_params2 = readl(&xhci->cap_regs->hcc_params2);
 
 	xhci->quirks |= quirks;
+	xhci->max_interrupters = min_t(u32, HCS_MAX_INTRS(xhci->hcs_params1),
+		      xhci->max_interrupters);
 
 	get_quirks(dev, xhci);
 
@@ -5454,7 +5618,10 @@ static const struct hc_driver xhci_hc_driver = {
 	.enable_device =	xhci_enable_device,
 	.update_hub_device =	xhci_update_hub_device,
 	.reset_device =		xhci_discover_or_reset_device,
-
+	.update_interrupter =	xhci_update_interrupter,
+	.free_interrupter =	xhci_free_interrupter,
+	.get_transfer_resource =	xhci_get_xfer_resource,
+	.stop_endpoint =	xhci_stop_endpoint,
 	/*
 	 * scheduling support
 	 */
diff --git a/drivers/usb/host/xhci.h b/drivers/usb/host/xhci.h
index c9f06c5e4e9d..2e694686c849 100644
--- a/drivers/usb/host/xhci.h
+++ b/drivers/usb/host/xhci.h
@@ -1926,6 +1926,7 @@ struct xhci_hcd {
 	struct dentry		*debugfs_root;
 	struct dentry		*debugfs_slots;
 	struct list_head	regset_list;
+	struct list_head	xhci_sec_list;
 
 	void			*dbc;
 	/* platform-specific data -- must come last */
@@ -1945,6 +1946,17 @@ struct xhci_driver_overrides {
 	void (*reset_bandwidth)(struct usb_hcd *, struct usb_device *);
 };
 
+struct xhci_sec {
+	struct xhci_ring	*event_ring;
+	struct xhci_erst	erst;
+	/* secondary interrupter */
+	struct xhci_intr_reg __iomem *ir_set;
+	struct xhci_hcd		*xhci;
+	int			intr_num;
+
+	struct list_head	list;
+};
+
 #define	XHCI_CFC_DELAY		10
 
 /* convert between an HCD pointer and the corresponding EHCI_HCD */
@@ -2034,6 +2046,9 @@ void xhci_dbg_trace(struct xhci_hcd *xhci, void (*trace)(struct va_format *),
 /* xHCI memory management */
 void xhci_mem_cleanup(struct xhci_hcd *xhci);
 int xhci_mem_init(struct xhci_hcd *xhci, gfp_t flags);
+void xhci_handle_sec_intr_events(struct xhci_hcd *xhci,
+		struct xhci_ring *ring, struct xhci_intr_reg __iomem *ir_set, struct xhci_sec *sec);
+int xhci_mem_sec_init(struct xhci_hcd *xhci, struct xhci_sec *sec);
 void xhci_free_virt_device(struct xhci_hcd *xhci, int slot_id);
 int xhci_alloc_virt_device(struct xhci_hcd *xhci, int slot_id, struct usb_device *udev, gfp_t flags);
 int xhci_setup_addressable_virt_dev(struct xhci_hcd *xhci, struct usb_device *udev);

[RFC PATCH 08/14] usb: dwc3: Add DT parameter to specify maximum number of interrupters

[Wesley Cheng]

Allow for the DWC3 host driver to pass along a XHCI property that defines
how many interrupters to allocate.  This is in relation for the number of
event rings that can be potentially used by other processors within the
system.

Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
---
 drivers/usb/dwc3/core.c | 12 ++++++++++++
 drivers/usb/dwc3/core.h |  2 ++
 drivers/usb/dwc3/host.c |  5 ++++-
 3 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c
index 476b63618511..67d6f0ae81d2 100644
--- a/drivers/usb/dwc3/core.c
+++ b/drivers/usb/dwc3/core.c
@@ -1446,6 +1446,7 @@ static void dwc3_get_properties(struct dwc3 *dwc)
 	u8			tx_thr_num_pkt_prd = 0;
 	u8			tx_max_burst_prd = 0;
 	u8			tx_fifo_resize_max_num;
+	u8			num_hc_interrupters;
 	const char		*usb_psy_name;
 	int			ret;
 
@@ -1468,6 +1469,9 @@ static void dwc3_get_properties(struct dwc3 *dwc)
 	 */
 	tx_fifo_resize_max_num = 6;
 
+	/* default to a single XHCI interrupter */
+	num_hc_interrupters = 1;
+
 	dwc->maximum_speed = usb_get_maximum_speed(dev);
 	dwc->max_ssp_rate = usb_get_maximum_ssp_rate(dev);
 	dwc->dr_mode = usb_get_dr_mode(dev);
@@ -1511,6 +1515,12 @@ static void dwc3_get_properties(struct dwc3 *dwc)
 				&tx_thr_num_pkt_prd);
 	device_property_read_u8(dev, "snps,tx-max-burst-prd",
 				&tx_max_burst_prd);
+	device_property_read_u8(dev, "snps,num-hc-interrupters",
+				&num_hc_interrupters);
+	/* DWC3 core allowed to have a max of 8 interrupters */
+	if (num_hc_interrupters > 8)
+		num_hc_interrupters = 8;
+
 	dwc->do_fifo_resize = device_property_read_bool(dev,
 							"tx-fifo-resize");
 	if (dwc->do_fifo_resize)
@@ -1589,6 +1599,8 @@ static void dwc3_get_properties(struct dwc3 *dwc)
 	dwc->imod_interval = 0;
 
 	dwc->tx_fifo_resize_max_num = tx_fifo_resize_max_num;
+
+	dwc->num_hc_interrupters = num_hc_interrupters;
 }
 
 /* check whether the core supports IMOD */
diff --git a/drivers/usb/dwc3/core.h b/drivers/usb/dwc3/core.h
index 8f9959ba9fd4..09037299da53 100644
--- a/drivers/usb/dwc3/core.h
+++ b/drivers/usb/dwc3/core.h
@@ -1050,6 +1050,7 @@ struct dwc3_scratchpad_array {
  * @tx_max_burst_prd: max periodic ESS transmit burst size
  * @tx_fifo_resize_max_num: max number of fifos allocated during txfifo resize
  * @clear_stall_protocol: endpoint number that requires a delayed status phase
+ * @num_hc_interrupters: number of host controller interrupters
  * @hsphy_interface: "utmi" or "ulpi"
  * @connected: true when we're connected to a host, false otherwise
  * @softconnect: true when gadget connect is called, false when disconnect runs
@@ -1275,6 +1276,7 @@ struct dwc3 {
 	u8			tx_max_burst_prd;
 	u8			tx_fifo_resize_max_num;
 	u8			clear_stall_protocol;
+	u8			num_hc_interrupters;
 
 	const char		*hsphy_interface;
 
diff --git a/drivers/usb/dwc3/host.c b/drivers/usb/dwc3/host.c
index f6f13e7f1ba1..52a284fdd704 100644
--- a/drivers/usb/dwc3/host.c
+++ b/drivers/usb/dwc3/host.c
@@ -66,7 +66,7 @@ static int dwc3_host_get_irq(struct dwc3 *dwc)
 
 int dwc3_host_init(struct dwc3 *dwc)
 {
-	struct property_entry	props[4];
+	struct property_entry	props[5];
 	struct platform_device	*xhci;
 	int			ret, irq;
 	int			prop_idx = 0;
@@ -112,6 +112,9 @@ int dwc3_host_init(struct dwc3 *dwc)
 	if (DWC3_VER_IS_WITHIN(DWC3, ANY, 300A))
 		props[prop_idx++] = PROPERTY_ENTRY_BOOL("quirk-broken-port-ped");
 
+	props[prop_idx++] = PROPERTY_ENTRY_U8("num-hc-interrupters",
+								dwc->num_hc_interrupters);
+
 	if (prop_idx) {
 		ret = device_create_managed_software_node(&xhci->dev, props, NULL);
 		if (ret) {

[RFC PATCH 09/14] sound: usb: Introduce QC USB SND offloading support

[Wesley Cheng]

Several Qualcomm SoCs have a dedicated audio DSP, which has the ability to
support USB sound devices.  This vendor driver will implement the required
handshaking with the DSP, in order to pass along required resources that
will be utilized by the DSP's USB SW.  The communication channel used for
this handshaking will be using the QMI protocol.  Required resources
include:
- Allocated secondary event ring address
- EP transfer ring address
- Interrupter number

The above information will allow for the audio DSP to execute USB transfers
over the USB bus.  It will also be able to support devices that have an
implicit feedback and sync endpoint as well.  Offloading these data
transfers will allow the main/applications processor to enter lower CPU
power modes, and sustain a longer duration in those modes.

Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
---
 sound/usb/Kconfig                  |   14 +
 sound/usb/Makefile                 |    2 +-
 sound/usb/qcom/Makefile            |    2 +
 sound/usb/qcom/qc_audio_offload.c  | 1610 ++++++++++++++++++++++++++++
 sound/usb/qcom/usb_audio_qmi_v01.c |  892 +++++++++++++++
 sound/usb/qcom/usb_audio_qmi_v01.h |  162 +++
 6 files changed, 2681 insertions(+), 1 deletion(-)
 create mode 100644 sound/usb/qcom/Makefile
 create mode 100644 sound/usb/qcom/qc_audio_offload.c
 create mode 100644 sound/usb/qcom/usb_audio_qmi_v01.c
 create mode 100644 sound/usb/qcom/usb_audio_qmi_v01.h

diff --git a/sound/usb/Kconfig b/sound/usb/Kconfig
index 059242f15d75..18d65a0d905a 100644
--- a/sound/usb/Kconfig
+++ b/sound/usb/Kconfig
@@ -165,6 +165,20 @@ config SND_BCD2000
 	  To compile this driver as a module, choose M here: the module
 	  will be called snd-bcd2000.
 
+config QC_USB_AUDIO_OFFLOAD
+	tristate "Qualcomm Audio Offload driver"
+	select SND_PCM
+	help
+	  Say Y here to enable the Qualcomm USB audio offloading feature
+
+	  This module sets up the required QMI stream enable/disable
+	  responses to requests generated by the audio DSP.  It passes the
+	  USB transfer resource references, so that the audio DSP can issue
+	  USB transfers to the host controller.
+
+	  To compile this driver as a module, choose M here: the module
+	  will be called qc-audio-offload.
+
 source "sound/usb/line6/Kconfig"
 
 endif	# SND_USB
diff --git a/sound/usb/Makefile b/sound/usb/Makefile
index 9ccb21a4ff8a..2243ae333ec9 100644
--- a/sound/usb/Makefile
+++ b/sound/usb/Makefile
@@ -33,5 +33,5 @@ obj-$(CONFIG_SND_USB_UA101) += snd-usbmidi-lib.o
 obj-$(CONFIG_SND_USB_USX2Y) += snd-usbmidi-lib.o
 obj-$(CONFIG_SND_USB_US122L) += snd-usbmidi-lib.o
 
-obj-$(CONFIG_SND) += misc/ usx2y/ caiaq/ 6fire/ hiface/ bcd2000/
+obj-$(CONFIG_SND) += misc/ usx2y/ caiaq/ 6fire/ hiface/ bcd2000/ qcom/
 obj-$(CONFIG_SND_USB_LINE6)	+= line6/
diff --git a/sound/usb/qcom/Makefile b/sound/usb/qcom/Makefile
new file mode 100644
index 000000000000..d27d39beb8ce
--- /dev/null
+++ b/sound/usb/qcom/Makefile
@@ -0,0 +1,2 @@
+snd-usb-audio-qmi-objs := usb_audio_qmi_v01.o qc_audio_offload.o
+obj-$(CONFIG_QC_USB_AUDIO_OFFLOAD) += snd-usb-audio-qmi.o
\ No newline at end of file
diff --git a/sound/usb/qcom/qc_audio_offload.c b/sound/usb/qcom/qc_audio_offload.c
new file mode 100644
index 000000000000..a3a91c72c684
--- /dev/null
+++ b/sound/usb/qcom/qc_audio_offload.c
@@ -0,0 +1,1610 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include <linux/ctype.h>
+#include <linux/moduleparam.h>
+#include <linux/module.h>
+#include <linux/usb.h>
+#include <linux/init.h>
+#include <linux/usb/quirks.h>
+#include <linux/usb/audio.h>
+#include <linux/usb/audio-v2.h>
+#include <linux/usb/audio-v3.h>
+#include <linux/soc/qcom/qmi.h>
+#include <linux/iommu.h>
+#include <linux/dma-mapping.h>
+#include <linux/dma-map-ops.h>
+#include <sound/q6usboffload.h>
+
+#include <sound/control.h>
+#include <sound/core.h>
+#include <sound/info.h>
+#include <sound/pcm.h>
+#include <sound/pcm_params.h>
+#include <sound/initval.h>
+
+#include <sound/soc.h>
+#include <sound/soc-usb.h>
+#include "../usbaudio.h"
+#include "../card.h"
+#include "../midi.h"
+#include "../mixer.h"
+#include "../proc.h"
+#include "../quirks.h"
+#include "../endpoint.h"
+#include "../helper.h"
+#include "../pcm.h"
+#include "../format.h"
+#include "../power.h"
+#include "../stream.h"
+#include "../media.h"
+#include "usb_audio_qmi_v01.h"
+
+#define DEV_RELEASE_WAIT_TIMEOUT 10000 /* in ms */
+
+#define BUS_INTERVAL_FULL_SPEED 1000 /* in us */
+#define BUS_INTERVAL_HIGHSPEED_AND_ABOVE 125 /* in us */
+#define MAX_BINTERVAL_ISOC_EP 16
+
+#define SND_PCM_CARD_NUM_MASK 0xffff0000
+#define SND_PCM_DEV_NUM_MASK 0xff00
+#define SND_PCM_STREAM_DIRECTION 0xff
+
+#define PREPEND_SID_TO_IOVA(iova, sid) ((u64)(((u64)(iova)) | \
+					(((u64)sid) << 32)))
+
+/*  event ring iova base address */
+#define IOVA_BASE 0x1000
+
+#define IOVA_XFER_RING_BASE (IOVA_BASE + PAGE_SIZE * (SNDRV_CARDS + 1))
+#define IOVA_XFER_BUF_BASE (IOVA_XFER_RING_BASE + PAGE_SIZE * SNDRV_CARDS * 32)
+#define IOVA_XFER_RING_MAX (IOVA_XFER_BUF_BASE - PAGE_SIZE)
+#define IOVA_XFER_BUF_MAX (0xfffff000 - PAGE_SIZE)
+
+#define MAX_XFER_BUFF_LEN (24 * PAGE_SIZE)
+
+struct iova_info {
+	struct list_head list;
+	unsigned long start_iova;
+	size_t size;
+	bool in_use;
+};
+
+struct intf_info {
+	unsigned long data_xfer_ring_va;
+	size_t data_xfer_ring_size;
+	unsigned long sync_xfer_ring_va;
+	size_t sync_xfer_ring_size;
+	unsigned long xfer_buf_va;
+	size_t xfer_buf_size;
+	phys_addr_t xfer_buf_pa;
+	unsigned int data_ep_pipe;
+	unsigned int sync_ep_pipe;
+	u8 *xfer_buf;
+	u8 intf_num;
+	u8 pcm_card_num;
+	u8 pcm_dev_num;
+	u8 direction;
+	bool in_use;
+};
+
+struct uaudio_qmi_dev {
+	struct device *dev;
+	u32 sid;
+	u32 intr_num;
+	struct xhci_ring *sec_ring;
+	struct iommu_domain *domain;
+
+	/* list to keep track of available iova */
+	struct list_head xfer_ring_list;
+	size_t xfer_ring_iova_size;
+	unsigned long curr_xfer_ring_iova;
+	struct list_head xfer_buf_list;
+	size_t xfer_buf_iova_size;
+	unsigned long curr_xfer_buf_iova;
+
+	/* bit fields representing pcm card enabled */
+	unsigned long card_slot;
+	/* indicate event ring mapped or not */
+	bool er_mapped;
+	/* reference count to number of possible consumers */
+	atomic_t qdev_in_use;
+	/* idx to last udev card number plugged in */
+	unsigned int last_card_num;
+};
+
+struct uaudio_dev {
+	struct usb_device *udev;
+	/* audio control interface */
+	struct usb_host_interface *ctrl_intf;
+	unsigned int card_num;
+	unsigned int usb_core_id;
+	atomic_t in_use;
+	struct kref kref;
+	wait_queue_head_t disconnect_wq;
+
+	/* interface specific */
+	int num_intf;
+	struct intf_info *info;
+	struct snd_usb_audio *chip;
+};
+
+static struct uaudio_dev uadev[SNDRV_CARDS];
+static struct uaudio_qmi_dev *uaudio_qdev;
+static struct uaudio_qmi_svc *uaudio_svc;
+
+struct uaudio_qmi_svc {
+	struct qmi_handle *uaudio_svc_hdl;
+	struct work_struct qmi_disconnect_work;
+	struct workqueue_struct *uaudio_wq;
+	struct sockaddr_qrtr client_sq;
+	bool client_connected;
+};
+
+enum mem_type {
+	MEM_EVENT_RING,
+	MEM_XFER_RING,
+	MEM_XFER_BUF,
+};
+
+enum usb_qmi_audio_format {
+	USB_QMI_PCM_FORMAT_S8 = 0,
+	USB_QMI_PCM_FORMAT_U8,
+	USB_QMI_PCM_FORMAT_S16_LE,
+	USB_QMI_PCM_FORMAT_S16_BE,
+	USB_QMI_PCM_FORMAT_U16_LE,
+	USB_QMI_PCM_FORMAT_U16_BE,
+	USB_QMI_PCM_FORMAT_S24_LE,
+	USB_QMI_PCM_FORMAT_S24_BE,
+	USB_QMI_PCM_FORMAT_U24_LE,
+	USB_QMI_PCM_FORMAT_U24_BE,
+	USB_QMI_PCM_FORMAT_S24_3LE,
+	USB_QMI_PCM_FORMAT_S24_3BE,
+	USB_QMI_PCM_FORMAT_U24_3LE,
+	USB_QMI_PCM_FORMAT_U24_3BE,
+	USB_QMI_PCM_FORMAT_S32_LE,
+	USB_QMI_PCM_FORMAT_S32_BE,
+	USB_QMI_PCM_FORMAT_U32_LE,
+	USB_QMI_PCM_FORMAT_U32_BE,
+};
+
+static void uaudio_iommu_unmap(enum mem_type mtype, unsigned long va,
+	size_t iova_size, size_t mapped_iova_size);
+static void uaudio_dev_cleanup(struct uaudio_dev *dev);
+static void disable_audio_stream(struct snd_usb_substream *subs);
+static struct snd_usb_substream *find_substream(unsigned int card_num,
+	unsigned int pcm_idx, unsigned int direction);
+
+static void qmi_disconnect_work(struct work_struct *w)
+{
+	struct intf_info *info;
+	int idx, if_idx;
+	struct snd_usb_substream *subs;
+	struct snd_usb_audio *chip;
+
+	/* find all active intf for set alt 0 and cleanup usb audio dev */
+	for (idx = 0; idx < SNDRV_CARDS; idx++) {
+		if (!atomic_read(&uadev[idx].in_use))
+			continue;
+
+		for (if_idx = 0; if_idx < uadev[idx].num_intf; if_idx++) {
+			if (!uadev[idx].info || !uadev[idx].info[if_idx].in_use)
+				continue;
+			info = &uadev[idx].info[if_idx];
+			subs = find_substream(info->pcm_card_num,
+						info->pcm_dev_num,
+						info->direction);
+			chip = uadev[idx].chip;
+			if (!subs || !chip || atomic_read(&chip->shutdown)) {
+				pr_err("no subs for c#%u, dev#%u dir%u\n",
+						info->pcm_card_num,
+						info->pcm_dev_num,
+						info->direction);
+				continue;
+			}
+			disable_audio_stream(subs);
+		}
+		atomic_set(&uadev[idx].in_use, 0);
+		uaudio_dev_cleanup(&uadev[idx]);
+	}
+}
+
+static void qmi_bye_cb(struct qmi_handle *handle, unsigned int node)
+{
+	struct uaudio_qmi_svc *svc = uaudio_svc;
+
+	if (svc->uaudio_svc_hdl != handle) {
+		pr_err("handle mismatch\n");
+		return;
+	}
+
+	if (svc->client_connected && svc->client_sq.sq_node == node) {
+		pr_err("node: %d\n", node);
+		queue_work(svc->uaudio_wq, &svc->qmi_disconnect_work);
+		svc->client_sq.sq_node = 0;
+		svc->client_sq.sq_port = 0;
+		svc->client_sq.sq_family = 0;
+		svc->client_connected = false;
+	}
+}
+
+static void qmi_svc_disconnect_cb(struct qmi_handle *handle,
+				  unsigned int node, unsigned int port)
+{
+	struct uaudio_qmi_svc *svc;
+
+	if (uaudio_svc == NULL)
+		return;
+
+	svc = uaudio_svc;
+	if (svc->uaudio_svc_hdl != handle) {
+		pr_err("handle mismatch\n");
+		return;
+	}
+
+	if (svc->client_connected && svc->client_sq.sq_node == node &&
+			svc->client_sq.sq_port == port) {
+		pr_debug("client node:%x port:%x\n", node, port);
+		queue_work(svc->uaudio_wq, &svc->qmi_disconnect_work);
+		svc->client_sq.sq_node = 0;
+		svc->client_sq.sq_port = 0;
+		svc->client_sq.sq_family = 0;
+		svc->client_connected = false;
+	}
+}
+
+static struct qmi_ops uaudio_svc_ops_options = {
+	.bye = qmi_bye_cb,
+	.del_client = qmi_svc_disconnect_cb,
+};
+
+static enum usb_audio_device_speed_enum_v01
+get_speed_info(enum usb_device_speed udev_speed)
+{
+	switch (udev_speed) {
+	case USB_SPEED_LOW:
+		return USB_AUDIO_DEVICE_SPEED_LOW_V01;
+	case USB_SPEED_FULL:
+		return USB_AUDIO_DEVICE_SPEED_FULL_V01;
+	case USB_SPEED_HIGH:
+		return USB_AUDIO_DEVICE_SPEED_HIGH_V01;
+	case USB_SPEED_SUPER:
+		return USB_AUDIO_DEVICE_SPEED_SUPER_V01;
+	case USB_SPEED_SUPER_PLUS:
+		return USB_AUDIO_DEVICE_SPEED_SUPER_PLUS_V01;
+	default:
+		pr_err("udev speed %d\n", udev_speed);
+		return USB_AUDIO_DEVICE_SPEED_INVALID_V01;
+	}
+}
+
+static unsigned long uaudio_get_iova(unsigned long *curr_iova,
+	size_t *curr_iova_size, struct list_head *head, size_t size)
+{
+	struct iova_info *info, *new_info = NULL;
+	struct list_head *curr_head;
+	unsigned long va = 0;
+	size_t tmp_size = size;
+	bool found = false;
+
+	if (size % PAGE_SIZE) {
+		pr_err("size %zu is not page size multiple\n", size);
+		goto done;
+	}
+
+	if (size > *curr_iova_size) {
+		pr_err("size %zu > curr size %zu\n", size, *curr_iova_size);
+		goto done;
+	}
+	if (*curr_iova_size == 0) {
+		pr_err("iova mapping is full\n");
+		goto done;
+	}
+
+	list_for_each_entry(info, head, list) {
+		/* exact size iova_info */
+		if (!info->in_use && info->size == size) {
+			info->in_use = true;
+			va = info->start_iova;
+			*curr_iova_size -= size;
+			found = true;
+			pr_debug("exact size: %zu found\n", size);
+			goto done;
+		} else if (!info->in_use && tmp_size >= info->size) {
+			if (!new_info)
+				new_info = info;
+			pr_debug("partial size: %zu found\n", info->size);
+			tmp_size -= info->size;
+			if (tmp_size)
+				continue;
+
+			va = new_info->start_iova;
+			for (curr_head = &new_info->list; curr_head !=
+			&info->list; curr_head = curr_head->next) {
+				new_info = list_entry(curr_head, struct
+						iova_info, list);
+				new_info->in_use = true;
+			}
+			info->in_use = true;
+			*curr_iova_size -= size;
+			found = true;
+			goto done;
+		} else {
+			/* iova region in use */
+			new_info = NULL;
+			tmp_size = size;
+		}
+	}
+
+	info = kzalloc(sizeof(struct iova_info), GFP_KERNEL);
+	if (!info) {
+		va = 0;
+		goto done;
+	}
+
+	va = info->start_iova = *curr_iova;
+	info->size = size;
+	info->in_use = true;
+	*curr_iova += size;
+	*curr_iova_size -= size;
+	found = true;
+	list_add_tail(&info->list, head);
+
+done:
+	if (!found)
+		pr_err("unable to find %zu size iova\n", size);
+	else
+		pr_debug("va:0x%08lx curr_iova:0x%08lx curr_iova_size:%zu\n",
+				va, *curr_iova, *curr_iova_size);
+
+	return va;
+}
+
+static unsigned long uaudio_iommu_map(enum mem_type mtype, bool dma_coherent,
+		phys_addr_t pa, size_t size, struct sg_table *sgt)
+{
+	unsigned long va_sg, va = 0;
+	bool map = true;
+	int i, ret;
+	size_t sg_len, total_len = 0;
+	struct scatterlist *sg;
+	phys_addr_t pa_sg;
+	int prot = IOMMU_READ | IOMMU_WRITE;
+
+	if (dma_coherent)
+		prot |= IOMMU_CACHE;
+
+	switch (mtype) {
+	case MEM_EVENT_RING:
+		va = IOVA_BASE;
+		/* er already mapped */
+		if (uaudio_qdev->er_mapped)
+			map = false;
+		break;
+	case MEM_XFER_RING:
+		va = uaudio_get_iova(&uaudio_qdev->curr_xfer_ring_iova,
+		&uaudio_qdev->xfer_ring_iova_size, &uaudio_qdev->xfer_ring_list,
+		size);
+		break;
+	case MEM_XFER_BUF:
+		va = uaudio_get_iova(&uaudio_qdev->curr_xfer_buf_iova,
+		&uaudio_qdev->xfer_buf_iova_size, &uaudio_qdev->xfer_buf_list,
+		size);
+		break;
+	default:
+		pr_err("unknown mem type %d\n", mtype);
+	}
+
+	if (!va || !map)
+		goto done;
+
+	if (!sgt)
+		goto skip_sgt_map;
+
+	va_sg = va;
+	for_each_sg(sgt->sgl, sg, sgt->nents, i) {
+		sg_len = PAGE_ALIGN(sg->offset + sg->length);
+		pa_sg = page_to_phys(sg_page(sg));
+		ret = iommu_map(uaudio_qdev->domain, va_sg, pa_sg, sg_len,
+								prot);
+		if (ret) {
+			pr_err("mapping failed ret%d\n", ret);
+			pr_err("type:%d, pa:%pa iova:0x%08lx sg_len:%zu\n",
+				mtype, &pa_sg, va_sg, sg_len);
+			uaudio_iommu_unmap(MEM_XFER_BUF, va, size, total_len);
+			va = 0;
+			goto done;
+		}
+		pr_debug("type:%d map pa:%pa to iova:0x%08lx len:%zu offset:%u\n",
+				mtype, &pa_sg, va_sg, sg_len, sg->offset);
+		va_sg += sg_len;
+		total_len += sg_len;
+	}
+
+	if (size != total_len) {
+		pr_err("iova size %zu != mapped iova size %zu\n", size,
+				total_len);
+		uaudio_iommu_unmap(MEM_XFER_BUF, va, size, total_len);
+		va = 0;
+	}
+	return va;
+
+skip_sgt_map:
+	pr_debug("type:%d map pa:%pa to iova:0x%08lx size:%zu\n", mtype, &pa,
+			va, size);
+
+	ret = iommu_map(uaudio_qdev->domain, va, pa, size, prot);
+	if (ret)
+		pr_err("failed to map pa:%pa iova:0x%lx type:%d ret:%d\n",
+				&pa, va, mtype, ret);
+done:
+	return va;
+}
+
+static void uaudio_put_iova(unsigned long va, size_t size, struct list_head
+	*head, size_t *curr_iova_size)
+{
+	struct iova_info *info;
+	size_t tmp_size = size;
+	bool found = false;
+
+	list_for_each_entry(info, head, list) {
+		if (info->start_iova == va) {
+			if (!info->in_use) {
+				pr_err("va %lu is not in use\n", va);
+				return;
+			}
+			found = true;
+			info->in_use = false;
+			if (info->size == size)
+				goto done;
+		}
+
+		if (found && tmp_size >= info->size) {
+			info->in_use = false;
+			tmp_size -= info->size;
+			if (!tmp_size)
+				goto done;
+		}
+	}
+
+	if (!found) {
+		pr_err("unable to find the va %lu\n", va);
+		return;
+	}
+done:
+	*curr_iova_size += size;
+	pr_debug("curr_iova_size %zu\n", *curr_iova_size);
+}
+
+static void uaudio_iommu_unmap(enum mem_type mtype, unsigned long va,
+	size_t iova_size, size_t mapped_iova_size)
+{
+	size_t umap_size;
+	bool unmap = true;
+
+	if (!va || !iova_size)
+		return;
+
+	switch (mtype) {
+	case MEM_EVENT_RING:
+		if (uaudio_qdev->er_mapped)
+			uaudio_qdev->er_mapped = false;
+		else
+			unmap = false;
+		break;
+
+	case MEM_XFER_RING:
+		uaudio_put_iova(va, iova_size, &uaudio_qdev->xfer_ring_list,
+		&uaudio_qdev->xfer_ring_iova_size);
+		break;
+	case MEM_XFER_BUF:
+		uaudio_put_iova(va, iova_size, &uaudio_qdev->xfer_buf_list,
+		&uaudio_qdev->xfer_buf_iova_size);
+		break;
+	default:
+		pr_err("unknown mem type %d\n", mtype);
+		unmap = false;
+	}
+
+	if (!unmap || !mapped_iova_size)
+		return;
+
+	pr_debug("type %d: unmap iova 0x%08lx size %zu\n", mtype, va,
+			mapped_iova_size);
+
+	umap_size = iommu_unmap(uaudio_qdev->domain, va, mapped_iova_size);
+	if (umap_size != mapped_iova_size)
+		pr_err("unmapped size %zu for iova 0x%08lx of mapped size %zu\n",
+				umap_size, va, mapped_iova_size);
+}
+
+/* looks up alias, if any, for controller DT node and returns the index */
+static int usb_get_controller_id(struct usb_device *udev)
+{
+	if (!udev || !udev->bus->sysdev || !udev->bus->sysdev->of_node)
+		pr_err("UDEV NULL\n");
+
+	if (udev->bus->sysdev && udev->bus->sysdev->of_node)
+		return of_alias_get_id(udev->bus->sysdev->of_node, "usb");
+
+	return -ENODEV;
+}
+
+static void uaudio_dev_intf_cleanup(struct usb_device *udev,
+	struct intf_info *info)
+{
+	uaudio_iommu_unmap(MEM_XFER_RING, info->data_xfer_ring_va,
+		info->data_xfer_ring_size, info->data_xfer_ring_size);
+	info->data_xfer_ring_va = 0;
+	info->data_xfer_ring_size = 0;
+
+	uaudio_iommu_unmap(MEM_XFER_RING, info->sync_xfer_ring_va,
+		info->sync_xfer_ring_size, info->sync_xfer_ring_size);
+	info->sync_xfer_ring_va = 0;
+	info->sync_xfer_ring_size = 0;
+
+	uaudio_iommu_unmap(MEM_XFER_BUF, info->xfer_buf_va,
+		info->xfer_buf_size, info->xfer_buf_size);
+	info->xfer_buf_va = 0;
+
+	usb_free_coherent(udev, info->xfer_buf_size,
+		info->xfer_buf, info->xfer_buf_pa);
+	info->xfer_buf_size = 0;
+	info->xfer_buf = NULL;
+	info->xfer_buf_pa = 0;
+
+	info->in_use = false;
+}
+
+static void uaudio_event_ring_cleanup_free(struct uaudio_dev *dev)
+{
+	clear_bit(dev->card_num, &uaudio_qdev->card_slot);
+	/* all audio devices are disconnected */
+	if (!uaudio_qdev->card_slot) {
+		uaudio_iommu_unmap(MEM_EVENT_RING, IOVA_BASE, PAGE_SIZE,
+			PAGE_SIZE);
+		usb_free_interrupter(dev->udev, uaudio_qdev->intr_num);
+		pr_debug("all audio devices disconnected\n");
+	}
+}
+
+static void uaudio_dev_release(struct kref *kref)
+{
+	struct uaudio_dev *dev = container_of(kref, struct uaudio_dev, kref);
+
+	uaudio_event_ring_cleanup_free(dev);
+	atomic_set(&dev->in_use, 0);
+	wake_up(&dev->disconnect_wq);
+}
+
+static struct snd_usb_substream *find_substream(unsigned int card_num,
+	unsigned int pcm_idx, unsigned int direction)
+{
+	struct snd_usb_stream *as;
+	struct snd_usb_substream *subs = NULL;
+	struct snd_usb_audio *chip;
+
+	chip = uadev[card_num].chip;
+	if (!chip || atomic_read(&chip->shutdown)) {
+		pr_debug("%s: instance of usb card # %d does not exist\n",
+			__func__, card_num);
+		goto done;
+	}
+
+	if (pcm_idx >= chip->pcm_devs) {
+		pr_err("%s: invalid pcm dev number %u > %d\n", __func__,
+			pcm_idx, chip->pcm_devs);
+		goto done;
+	}
+
+	if (direction > SNDRV_PCM_STREAM_CAPTURE) {
+		pr_err("%s: invalid direction %u\n", __func__, direction);
+		goto done;
+	}
+
+	list_for_each_entry(as, &chip->pcm_list, list) {
+		if (as->pcm_index == pcm_idx) {
+			subs = &as->substream[direction];
+			goto done;
+		}
+	}
+
+done:
+	if (!subs)
+		pr_err("%s: substream instance not found\n", __func__);
+	return subs;
+}
+
+static int info_idx_from_ifnum(int card_num, int intf_num, bool enable)
+{
+	int i;
+
+	/*
+	 * default index 0 is used when info is allocated upon
+	 * first enable audio stream req for a pcm device
+	 */
+	if (enable && !uadev[card_num].info)
+		return 0;
+
+	for (i = 0; i < uadev[card_num].num_intf; i++) {
+		if (enable && !uadev[card_num].info[i].in_use)
+			return i;
+		else if (!enable &&
+				uadev[card_num].info[i].intf_num == intf_num)
+			return i;
+	}
+
+	return -EINVAL;
+}
+
+static int get_data_interval_from_si(struct snd_usb_substream *subs,
+	u32 service_interval)
+{
+	unsigned int bus_intval, bus_intval_mult, binterval;
+
+	if (subs->dev->speed >= USB_SPEED_HIGH)
+		bus_intval = BUS_INTERVAL_HIGHSPEED_AND_ABOVE;
+	else
+		bus_intval = BUS_INTERVAL_FULL_SPEED;
+
+	if (service_interval % bus_intval)
+		return -EINVAL;
+
+	bus_intval_mult = service_interval / bus_intval;
+	binterval = ffs(bus_intval_mult);
+	if (!binterval || binterval > MAX_BINTERVAL_ISOC_EP)
+		return -EINVAL;
+
+	/* check if another bit is set then bail out */
+	bus_intval_mult = bus_intval_mult >> binterval;
+	if (bus_intval_mult)
+		return -EINVAL;
+
+	return (binterval - 1);
+}
+
+/* maps audio format received over QMI to asound.h based pcm format */
+static snd_pcm_format_t map_pcm_format(enum usb_qmi_audio_format fmt_received)
+{
+	switch (fmt_received) {
+	case USB_QMI_PCM_FORMAT_S8:
+		return SNDRV_PCM_FORMAT_S8;
+	case USB_QMI_PCM_FORMAT_U8:
+		return SNDRV_PCM_FORMAT_U8;
+	case USB_QMI_PCM_FORMAT_S16_LE:
+		return SNDRV_PCM_FORMAT_S16_LE;
+	case USB_QMI_PCM_FORMAT_S16_BE:
+		return SNDRV_PCM_FORMAT_S16_BE;
+	case USB_QMI_PCM_FORMAT_U16_LE:
+		return SNDRV_PCM_FORMAT_U16_LE;
+	case USB_QMI_PCM_FORMAT_U16_BE:
+		return SNDRV_PCM_FORMAT_U16_BE;
+	case USB_QMI_PCM_FORMAT_S24_LE:
+		return SNDRV_PCM_FORMAT_S24_LE;
+	case USB_QMI_PCM_FORMAT_S24_BE:
+		return SNDRV_PCM_FORMAT_S24_BE;
+	case USB_QMI_PCM_FORMAT_U24_LE:
+		return SNDRV_PCM_FORMAT_U24_LE;
+	case USB_QMI_PCM_FORMAT_U24_BE:
+		return SNDRV_PCM_FORMAT_U24_BE;
+	case USB_QMI_PCM_FORMAT_S24_3LE:
+		return SNDRV_PCM_FORMAT_S24_3LE;
+	case USB_QMI_PCM_FORMAT_S24_3BE:
+		return SNDRV_PCM_FORMAT_S24_3BE;
+	case USB_QMI_PCM_FORMAT_U24_3LE:
+		return SNDRV_PCM_FORMAT_U24_3LE;
+	case USB_QMI_PCM_FORMAT_U24_3BE:
+		return SNDRV_PCM_FORMAT_U24_3BE;
+	case USB_QMI_PCM_FORMAT_S32_LE:
+		return SNDRV_PCM_FORMAT_S32_LE;
+	case USB_QMI_PCM_FORMAT_S32_BE:
+		return SNDRV_PCM_FORMAT_S32_BE;
+	case USB_QMI_PCM_FORMAT_U32_LE:
+		return SNDRV_PCM_FORMAT_U32_LE;
+	case USB_QMI_PCM_FORMAT_U32_BE:
+		return SNDRV_PCM_FORMAT_U32_BE;
+	default:
+		/*
+		 * We expect the caller to do input validation so we should
+		 * never hit this. But we do have to return a proper
+		 * snd_pcm_format_t value due to the __bitwise attribute; so
+		 * just return the equivalent of 0 in case of bad input.
+		 */
+		return SNDRV_PCM_FORMAT_S8;
+	}
+}
+
+static int snd_interval_refine_set(struct snd_interval *i, unsigned int val)
+{
+	struct snd_interval t;
+
+	t.empty = 0;
+	t.min = t.max = val;
+	t.openmin = t.openmax = 0;
+	t.integer = 1;
+	return snd_interval_refine(i, &t);
+}
+
+static int _snd_pcm_hw_param_set(struct snd_pcm_hw_params *params,
+				 snd_pcm_hw_param_t var, unsigned int val,
+				 int dir)
+{
+	int changed;
+
+	if (hw_is_mask(var)) {
+		struct snd_mask *m = hw_param_mask(params, var);
+
+		if (val == 0 && dir < 0) {
+			changed = -EINVAL;
+			snd_mask_none(m);
+		} else {
+			if (dir > 0)
+				val++;
+			else if (dir < 0)
+				val--;
+			changed = snd_mask_refine_set(
+					hw_param_mask(params, var), val);
+		}
+	} else if (hw_is_interval(var)) {
+		struct snd_interval *i = hw_param_interval(params, var);
+
+		if (val == 0 && dir < 0) {
+			changed = -EINVAL;
+			snd_interval_none(i);
+		} else if (dir == 0)
+			changed = snd_interval_refine_set(i, val);
+		else {
+			struct snd_interval t;
+
+			t.openmin = 1;
+			t.openmax = 1;
+			t.empty = 0;
+			t.integer = 0;
+			if (dir < 0) {
+				t.min = val - 1;
+				t.max = val;
+			} else {
+				t.min = val;
+				t.max = val+1;
+			}
+			changed = snd_interval_refine(i, &t);
+		}
+	} else
+		return -EINVAL;
+	if (changed) {
+		params->cmask |= 1 << var;
+		params->rmask |= 1 << var;
+	}
+	return changed;
+}
+
+static void disable_audio_stream(struct snd_usb_substream *subs)
+{
+	struct snd_usb_audio *chip = subs->stream->chip;
+
+	if (subs->data_endpoint || subs->sync_endpoint) {
+		close_endpoints(chip, subs);
+
+		mutex_lock(&chip->mutex);
+		subs->cur_audiofmt = NULL;
+		mutex_unlock(&chip->mutex);
+	}
+
+	snd_usb_autosuspend(chip);
+}
+
+static int enable_audio_stream(struct snd_usb_substream *subs,
+				snd_pcm_format_t pcm_format,
+				unsigned int channels, unsigned int cur_rate,
+				int datainterval)
+{
+	struct snd_usb_audio *chip = subs->stream->chip;
+	struct snd_pcm_hw_params params;
+	const struct audioformat *fmt;
+	int ret;
+
+	_snd_pcm_hw_params_any(&params);
+	_snd_pcm_hw_param_set(&params, SNDRV_PCM_HW_PARAM_FORMAT,
+			pcm_format, 0);
+	_snd_pcm_hw_param_set(&params, SNDRV_PCM_HW_PARAM_CHANNELS,
+			channels, 0);
+	_snd_pcm_hw_param_set(&params, SNDRV_PCM_HW_PARAM_RATE,
+			cur_rate, 0);
+
+	pm_runtime_barrier(&chip->intf[0]->dev);
+	snd_usb_autoresume(chip);
+
+	fmt = find_format(&subs->fmt_list, pcm_format, cur_rate,
+			channels, datainterval, subs);
+	if (!fmt) {
+		dev_err(&subs->dev->dev, "cannot find format: format = %#x, rate = %d, channels = %d\n",
+			   pcm_format, cur_rate, channels);
+		return -EINVAL;
+	}
+
+	if (atomic_read(&chip->shutdown)) {
+		pr_err("chip already shutdown\n");
+		ret = -ENODEV;
+	} else {
+		if (subs->data_endpoint)
+			close_endpoints(chip, subs);
+
+		subs->data_endpoint = snd_usb_endpoint_open(chip, fmt,
+				&params, false);
+		if (!subs->data_endpoint) {
+			pr_err("failed to open data endpoint\n");
+			return -EINVAL;
+		}
+
+		if (fmt->sync_ep) {
+			subs->sync_endpoint = snd_usb_endpoint_open(chip,
+					fmt, &params, true);
+			if (!subs->sync_endpoint) {
+				pr_err("failed to open sync endpoint\n");
+				return -EINVAL;
+			}
+
+			subs->data_endpoint->sync_source = subs->sync_endpoint;
+		}
+
+		mutex_lock(&chip->mutex);
+		subs->cur_audiofmt = fmt;
+		mutex_unlock(&chip->mutex);
+
+		if (subs->sync_endpoint) {
+			ret = snd_usb_endpoint_prepare(chip, subs->sync_endpoint);
+			if (ret < 0)
+				return ret;
+		}
+
+		ret = snd_usb_endpoint_prepare(chip, subs->data_endpoint);
+		if (ret < 0)
+			return ret;
+
+		pr_info("selected %s iface:%d altsetting:%d datainterval:%dus\n",
+				subs->direction ? "capture" : "playback",
+				fmt->iface, fmt->altsetting,
+				(1 << fmt->datainterval) *
+				(subs->dev->speed >= USB_SPEED_HIGH ?
+				 BUS_INTERVAL_HIGHSPEED_AND_ABOVE :
+				 BUS_INTERVAL_FULL_SPEED));
+	}
+
+	return 0;
+}
+
+static int prepare_qmi_response(struct snd_usb_substream *subs,
+		struct qmi_uaudio_stream_req_msg_v01 *req_msg,
+		struct qmi_uaudio_stream_resp_msg_v01 *resp, int info_idx)
+{
+	struct usb_interface *iface;
+	struct usb_host_interface *alts;
+	struct usb_interface_descriptor *altsd;
+	struct usb_interface_assoc_descriptor *assoc;
+	struct usb_host_endpoint *ep;
+	struct uac_format_type_i_continuous_descriptor *fmt;
+	struct uac_format_type_i_discrete_descriptor *fmt_v1;
+	struct uac_format_type_i_ext_descriptor *fmt_v2;
+	struct uac1_as_header_descriptor *as;
+	int ret;
+	int protocol, card_num, pcm_dev_num;
+	void *hdr_ptr;
+	u8 *xfer_buf;
+	unsigned int data_ep_pipe = 0, sync_ep_pipe = 0;
+	u32 len, mult, remainder, xfer_buf_len;
+	unsigned long va, tr_data_va = 0, tr_sync_va = 0;
+	phys_addr_t xhci_pa, xfer_buf_pa, tr_data_pa = 0, tr_sync_pa = 0;
+	dma_addr_t dma;
+	struct sg_table sgt;
+	bool dma_coherent;
+
+	iface = usb_ifnum_to_if(subs->dev, subs->cur_audiofmt->iface);
+	if (!iface) {
+		pr_err("interface # %d does not exist\n", subs->cur_audiofmt->iface);
+		ret = -ENODEV;
+		goto err;
+	}
+
+	assoc = iface->intf_assoc;
+	pcm_dev_num = (req_msg->usb_token & SND_PCM_DEV_NUM_MASK) >> 8;
+	xfer_buf_len = req_msg->xfer_buff_size;
+	card_num = uaudio_qdev->last_card_num;
+
+	alts = &iface->altsetting[subs->cur_audiofmt->altset_idx];
+	altsd = get_iface_desc(alts);
+	protocol = altsd->bInterfaceProtocol;
+
+	/* get format type */
+	if (protocol != UAC_VERSION_3) {
+		fmt = snd_usb_find_csint_desc(alts->extra, alts->extralen, NULL,
+				UAC_FORMAT_TYPE);
+		if (!fmt) {
+			pr_err("%u:%d : no UAC_FORMAT_TYPE desc\n",
+					subs->cur_audiofmt->iface,
+					subs->cur_audiofmt->altset_idx);
+			ret = -ENODEV;
+			goto err;
+		}
+	}
+
+	if (!uadev[card_num].ctrl_intf) {
+		pr_err("audio ctrl intf info not cached\n");
+		ret = -ENODEV;
+		goto err;
+	}
+
+	if (protocol != UAC_VERSION_3) {
+		hdr_ptr = snd_usb_find_csint_desc(uadev[card_num].ctrl_intf->extra,
+				uadev[card_num].ctrl_intf->extralen, NULL,
+				UAC_HEADER);
+		if (!hdr_ptr) {
+			pr_err("no UAC_HEADER desc\n");
+			ret = -ENODEV;
+			goto err;
+		}
+	}
+
+	if (protocol == UAC_VERSION_1) {
+		struct uac1_ac_header_descriptor *uac1_hdr = hdr_ptr;
+
+		as = snd_usb_find_csint_desc(alts->extra, alts->extralen, NULL,
+			UAC_AS_GENERAL);
+		if (!as) {
+			pr_err("%u:%d : no UAC_AS_GENERAL desc\n",
+					subs->cur_audiofmt->iface,
+					subs->cur_audiofmt->altset_idx);
+			ret = -ENODEV;
+			goto err;
+		}
+		resp->data_path_delay = as->bDelay;
+		resp->data_path_delay_valid = 1;
+		fmt_v1 = (struct uac_format_type_i_discrete_descriptor *)fmt;
+		resp->usb_audio_subslot_size = fmt_v1->bSubframeSize;
+		resp->usb_audio_subslot_size_valid = 1;
+
+		resp->usb_audio_spec_revision = le16_to_cpu(uac1_hdr->bcdADC);
+		resp->usb_audio_spec_revision_valid = 1;
+	} else if (protocol == UAC_VERSION_2) {
+		struct uac2_ac_header_descriptor *uac2_hdr = hdr_ptr;
+
+		fmt_v2 = (struct uac_format_type_i_ext_descriptor *)fmt;
+		resp->usb_audio_subslot_size = fmt_v2->bSubslotSize;
+		resp->usb_audio_subslot_size_valid = 1;
+
+		resp->usb_audio_spec_revision = le16_to_cpu(uac2_hdr->bcdADC);
+		resp->usb_audio_spec_revision_valid = 1;
+	} else if (protocol == UAC_VERSION_3) {
+		if (assoc->bFunctionSubClass ==
+					UAC3_FUNCTION_SUBCLASS_FULL_ADC_3_0) {
+			pr_err("full adc is not supported\n");
+			ret = -EINVAL;
+		}
+
+		switch (le16_to_cpu(get_endpoint(alts, 0)->wMaxPacketSize)) {
+		case UAC3_BADD_EP_MAXPSIZE_SYNC_MONO_16:
+		case UAC3_BADD_EP_MAXPSIZE_SYNC_STEREO_16:
+		case UAC3_BADD_EP_MAXPSIZE_ASYNC_MONO_16:
+		case UAC3_BADD_EP_MAXPSIZE_ASYNC_STEREO_16: {
+			resp->usb_audio_subslot_size = 0x2;
+			break;
+		}
+
+		case UAC3_BADD_EP_MAXPSIZE_SYNC_MONO_24:
+		case UAC3_BADD_EP_MAXPSIZE_SYNC_STEREO_24:
+		case UAC3_BADD_EP_MAXPSIZE_ASYNC_MONO_24:
+		case UAC3_BADD_EP_MAXPSIZE_ASYNC_STEREO_24: {
+			resp->usb_audio_subslot_size = 0x3;
+			break;
+		}
+
+		default:
+			pr_err("%d: %u: Invalid wMaxPacketSize\n",
+					subs->cur_audiofmt->iface,
+					subs->cur_audiofmt->altset_idx);
+			ret = -EINVAL;
+			goto err;
+		}
+		resp->usb_audio_subslot_size_valid = 1;
+	} else {
+		pr_err("unknown protocol version %x\n", protocol);
+		ret = -ENODEV;
+		goto err;
+	}
+
+	resp->slot_id = subs->dev->slot_id;
+	resp->slot_id_valid = 1;
+
+	memcpy(&resp->std_as_opr_intf_desc, &alts->desc, sizeof(alts->desc));
+	resp->std_as_opr_intf_desc_valid = 1;
+
+	ep = usb_pipe_endpoint(subs->dev, subs->data_endpoint->pipe);
+	if (!ep) {
+		pr_err("data ep # %d context is null\n",
+				subs->data_endpoint->ep_num);
+		ret = -ENODEV;
+		goto err;
+	}
+	data_ep_pipe = subs->data_endpoint->pipe;
+	memcpy(&resp->std_as_data_ep_desc, &ep->desc, sizeof(ep->desc));
+	resp->std_as_data_ep_desc_valid = 1;
+
+	tr_data_pa = usb_hcd_get_transfer_resource(subs->dev, ep, &dma);
+	if (!tr_data_pa) {
+		pr_err("failed to get data ep ring dma address\n");
+		ret = -ENODEV;
+		goto err;
+	}
+	resp->xhci_mem_info.tr_data.pa = dma;
+
+	if (subs->sync_endpoint) {
+		ep = usb_pipe_endpoint(subs->dev, subs->sync_endpoint->pipe);
+		if (!ep) {
+			pr_err("implicit fb on data ep\n");
+			goto skip_sync_ep;
+		}
+		sync_ep_pipe = subs->sync_endpoint->pipe;
+		memcpy(&resp->std_as_sync_ep_desc, &ep->desc, sizeof(ep->desc));
+		resp->std_as_sync_ep_desc_valid = 1;
+
+		tr_sync_pa = usb_hcd_get_transfer_resource(subs->dev, ep, &dma);
+		if (!tr_sync_pa) {
+			pr_err("failed to get sync ep ring dma address\n");
+			ret = -ENODEV;
+			goto err;
+		}
+		resp->xhci_mem_info.tr_sync.pa = dma;
+	}
+
+skip_sync_ep:
+	resp->interrupter_num = uaudio_qdev->intr_num;
+	resp->interrupter_num_valid = 1;
+	resp->controller_num_valid = 0;
+	ret = usb_get_controller_id(subs->dev);
+	if (ret >= 0) {
+		resp->controller_num = ret;
+		resp->controller_num_valid = 1;
+	}
+	/* map xhci data structures PA memory to iova */
+	dma_coherent = dev_is_dma_coherent(subs->dev->bus->sysdev);
+
+	/* event ring */
+	xhci_pa = usb_set_interruper(subs->dev, resp->interrupter_num, &dma);
+	if (!xhci_pa) {
+		pr_err("failed to get sec event ring dma address\n");
+		ret = -ENODEV;
+		goto free_sec_ring;
+	}
+
+	va = uaudio_iommu_map(MEM_EVENT_RING, dma_coherent, xhci_pa, PAGE_SIZE,
+			NULL);
+	if (!va) {
+		ret = -ENOMEM;
+		goto free_sec_ring;
+	}
+
+	resp->xhci_mem_info.evt_ring.va = PREPEND_SID_TO_IOVA(va,
+						uaudio_qdev->sid);
+	resp->xhci_mem_info.evt_ring.pa = dma;
+	resp->xhci_mem_info.evt_ring.size = PAGE_SIZE;
+	uaudio_qdev->er_mapped = true;
+
+	resp->speed_info = get_speed_info(subs->dev->speed);
+	if (resp->speed_info == USB_AUDIO_DEVICE_SPEED_INVALID_V01) {
+		ret = -ENODEV;
+		goto unmap_er;
+	}
+
+	resp->speed_info_valid = 1;
+
+	/* data transfer ring */
+	va = uaudio_iommu_map(MEM_XFER_RING, dma_coherent, tr_data_pa,
+			PAGE_SIZE, NULL);
+	if (!va) {
+		ret = -ENOMEM;
+		goto unmap_er;
+	}
+
+	tr_data_va = va;
+	resp->xhci_mem_info.tr_data.va = PREPEND_SID_TO_IOVA(va,
+						uaudio_qdev->sid);
+	resp->xhci_mem_info.tr_data.size = PAGE_SIZE;
+
+	/* sync transfer ring */
+	if (!resp->xhci_mem_info.tr_sync.pa)
+		goto skip_sync;
+
+	xhci_pa = resp->xhci_mem_info.tr_sync.pa;
+	va = uaudio_iommu_map(MEM_XFER_RING, dma_coherent, tr_sync_pa,
+			PAGE_SIZE, NULL);
+	if (!va) {
+		ret = -ENOMEM;
+		goto unmap_data;
+	}
+
+	tr_sync_va = va;
+	resp->xhci_mem_info.tr_sync.va = PREPEND_SID_TO_IOVA(va,
+						uaudio_qdev->sid);
+	resp->xhci_mem_info.tr_sync.size = PAGE_SIZE;
+
+skip_sync:
+	/* xfer buffer, multiple of 4K only */
+	if (!xfer_buf_len)
+		xfer_buf_len = PAGE_SIZE;
+
+	mult = xfer_buf_len / PAGE_SIZE;
+	remainder = xfer_buf_len % PAGE_SIZE;
+	len = mult * PAGE_SIZE;
+	len += remainder ? PAGE_SIZE : 0;
+
+	if (len > MAX_XFER_BUFF_LEN) {
+		pr_err("req buf len %d > max buf len %lu, setting %lu\n",
+				len, MAX_XFER_BUFF_LEN, MAX_XFER_BUFF_LEN);
+		len = MAX_XFER_BUFF_LEN;
+	}
+
+	xfer_buf = usb_alloc_coherent(subs->dev, len, GFP_KERNEL, &xfer_buf_pa);
+	if (!xfer_buf) {
+		ret = -ENOMEM;
+		goto unmap_sync;
+	}
+
+	dma_get_sgtable(subs->dev->bus->sysdev, &sgt, xfer_buf, xfer_buf_pa,
+			len);
+	va = uaudio_iommu_map(MEM_XFER_BUF, dma_coherent, xfer_buf_pa, len,
+			&sgt);
+	if (!va) {
+		ret = -ENOMEM;
+		goto unmap_sync;
+	}
+
+	resp->xhci_mem_info.xfer_buff.pa = xfer_buf_pa;
+	resp->xhci_mem_info.xfer_buff.size = len;
+
+	resp->xhci_mem_info.xfer_buff.va = PREPEND_SID_TO_IOVA(va,
+						uaudio_qdev->sid);
+
+	resp->xhci_mem_info_valid = 1;
+
+	sg_free_table(&sgt);
+
+	if (!atomic_read(&uadev[card_num].in_use)) {
+		kref_init(&uadev[card_num].kref);
+		init_waitqueue_head(&uadev[card_num].disconnect_wq);
+		uadev[card_num].num_intf =
+			subs->dev->config->desc.bNumInterfaces;
+		uadev[card_num].info = kcalloc(uadev[card_num].num_intf,
+			sizeof(struct intf_info), GFP_KERNEL);
+		if (!uadev[card_num].info) {
+			ret = -ENOMEM;
+			goto unmap_sync;
+		}
+		uadev[card_num].udev = subs->dev;
+		atomic_set(&uadev[card_num].in_use, 1);
+	} else {
+		kref_get(&uadev[card_num].kref);
+	}
+
+	uadev[card_num].card_num = card_num;
+	uadev[card_num].usb_core_id = resp->controller_num;
+
+	/* cache intf specific info to use it for unmap and free xfer buf */
+	uadev[card_num].info[info_idx].data_xfer_ring_va = tr_data_va;
+	uadev[card_num].info[info_idx].data_xfer_ring_size = PAGE_SIZE;
+	uadev[card_num].info[info_idx].sync_xfer_ring_va = tr_sync_va;
+	uadev[card_num].info[info_idx].sync_xfer_ring_size = PAGE_SIZE;
+	uadev[card_num].info[info_idx].xfer_buf_va = va;
+	uadev[card_num].info[info_idx].xfer_buf_pa = xfer_buf_pa;
+	uadev[card_num].info[info_idx].xfer_buf_size = len;
+	uadev[card_num].info[info_idx].data_ep_pipe = data_ep_pipe;
+	uadev[card_num].info[info_idx].sync_ep_pipe = sync_ep_pipe;
+	uadev[card_num].info[info_idx].xfer_buf = xfer_buf;
+	uadev[card_num].info[info_idx].pcm_card_num = card_num;
+	uadev[card_num].info[info_idx].pcm_dev_num = pcm_dev_num;
+	uadev[card_num].info[info_idx].direction = subs->direction;
+	uadev[card_num].info[info_idx].intf_num = subs->cur_audiofmt->iface;
+	uadev[card_num].info[info_idx].in_use = true;
+
+	set_bit(card_num, &uaudio_qdev->card_slot);
+
+	return 0;
+
+unmap_sync:
+	usb_free_coherent(subs->dev, len, xfer_buf, xfer_buf_pa);
+	uaudio_iommu_unmap(MEM_XFER_RING, tr_sync_va, PAGE_SIZE, PAGE_SIZE);
+unmap_data:
+	uaudio_iommu_unmap(MEM_XFER_RING, tr_data_va, PAGE_SIZE, PAGE_SIZE);
+unmap_er:
+	uaudio_iommu_unmap(MEM_EVENT_RING, IOVA_BASE, PAGE_SIZE, PAGE_SIZE);
+free_sec_ring:
+	usb_free_interrupter(subs->dev, uaudio_qdev->intr_num);
+err:
+	return ret;
+}
+
+static void handle_uaudio_stream_req(struct qmi_handle *handle,
+			struct sockaddr_qrtr *sq,
+			struct qmi_txn *txn,
+			const void *decoded_msg)
+{
+	struct qmi_uaudio_stream_req_msg_v01 *req_msg;
+	struct qmi_uaudio_stream_resp_msg_v01 resp = {{0}, 0};
+	struct snd_usb_substream *subs;
+	struct snd_usb_audio *chip = NULL;
+	struct uaudio_qmi_svc *svc = uaudio_svc;
+	struct intf_info *info;
+	struct usb_host_endpoint *ep;
+	u8 pcm_card_num, pcm_dev_num, direction;
+	int info_idx = -EINVAL, datainterval = -EINVAL, ret = 0;
+
+	if (!svc->client_connected) {
+		svc->client_sq = *sq;
+		svc->client_connected = true;
+	}
+
+	req_msg = (struct qmi_uaudio_stream_req_msg_v01 *)decoded_msg;
+	if (!req_msg->audio_format_valid || !req_msg->bit_rate_valid ||
+	!req_msg->number_of_ch_valid || !req_msg->xfer_buff_size_valid) {
+		pr_err("invalid request msg\n");
+		ret = -EINVAL;
+		goto response;
+	}
+
+	direction = (req_msg->usb_token & SND_PCM_STREAM_DIRECTION);
+	pcm_dev_num = (req_msg->usb_token & SND_PCM_DEV_NUM_MASK) >> 8;
+	pcm_card_num = req_msg->enable ? uaudio_qdev->last_card_num :
+				ffs(uaudio_qdev->card_slot) - 1;
+
+	pr_info("card#:%d dev#:%d dir:%d en:%d fmt:%d rate:%d #ch:%d\n",
+			pcm_card_num, pcm_dev_num, (req_msg->usb_token & SND_PCM_STREAM_DIRECTION),
+			req_msg->enable, req_msg->audio_format, req_msg->bit_rate,
+			req_msg->number_of_ch);
+
+	if (pcm_card_num >= SNDRV_CARDS) {
+		pr_err("invalid card # %u", pcm_card_num);
+		ret = -EINVAL;
+		goto response;
+	}
+
+	if (req_msg->audio_format > USB_QMI_PCM_FORMAT_U32_BE) {
+		pr_err("unsupported pcm format received %d\n",
+				req_msg->audio_format);
+		ret = -EINVAL;
+		goto response;
+	}
+
+	subs = find_substream(pcm_card_num, pcm_dev_num, direction);
+	chip = uadev[pcm_card_num].chip;
+	if (!subs || !chip || atomic_read(&chip->shutdown)) {
+		pr_err("can't find substream for card# %u, dev# %u dir%u\n",
+				pcm_card_num, pcm_dev_num, direction);
+		ret = -ENODEV;
+		goto response;
+	}
+
+	info_idx = info_idx_from_ifnum(pcm_card_num, subs->cur_audiofmt ?
+			subs->cur_audiofmt->iface : -1, req_msg->enable);
+	if (atomic_read(&chip->shutdown) || !subs->stream || !subs->stream->pcm
+			|| !subs->stream->chip) {
+		pr_err("chip or sub not available: shutdown:%d stream:%p pcm:%p chip:%p\n",
+				atomic_read(&chip->shutdown), subs->stream,
+				subs->stream->pcm, subs->stream->chip);
+		ret = -ENODEV;
+		goto response;
+	}
+
+	if (req_msg->enable) {
+		if (info_idx < 0) {
+			pr_err("interface# %d already in use card# %d\n",
+					subs->cur_audiofmt->iface, pcm_card_num);
+			ret = -EBUSY;
+			goto response;
+		}
+	}
+
+	if (req_msg->service_interval_valid) {
+		ret = get_data_interval_from_si(subs,
+						req_msg->service_interval);
+		if (ret == -EINVAL) {
+			pr_err("invalid service interval %u\n",
+					req_msg->service_interval);
+			goto response;
+		}
+
+		datainterval = ret;
+	}
+
+	uadev[pcm_card_num].ctrl_intf = chip->ctrl_intf;
+
+	if (req_msg->enable) {
+		ret = enable_audio_stream(subs,
+				map_pcm_format(req_msg->audio_format),
+				req_msg->number_of_ch, req_msg->bit_rate,
+				datainterval);
+
+		if (!ret)
+			ret = prepare_qmi_response(subs, req_msg, &resp,
+					info_idx);
+	} else {
+		info = &uadev[pcm_card_num].info[info_idx];
+		if (info->data_ep_pipe) {
+			ep = usb_pipe_endpoint(uadev[pcm_card_num].udev,
+						info->data_ep_pipe);
+			if (!ep)
+				pr_err("no data ep\n");
+			else
+				usb_hcd_stop_endpoint(uadev[pcm_card_num].udev,
+						ep);
+			info->data_ep_pipe = 0;
+		}
+
+		if (info->sync_ep_pipe) {
+			ep = usb_pipe_endpoint(uadev[pcm_card_num].udev,
+						info->sync_ep_pipe);
+			if (!ep)
+				pr_err("no sync ep\n");
+			else
+				usb_hcd_stop_endpoint(uadev[pcm_card_num].udev,
+						ep);
+			info->sync_ep_pipe = 0;
+		}
+
+		disable_audio_stream(subs);
+	}
+
+response:
+	if (!req_msg->enable && ret != -EINVAL && ret != -ENODEV) {
+		mutex_lock(&chip->mutex);
+		if (info_idx >= 0) {
+			info = &uadev[pcm_card_num].info[info_idx];
+			uaudio_dev_intf_cleanup(
+					uadev[pcm_card_num].udev,
+					info);
+			pr_info("release resources: intf# %d card# %d\n",
+					info->intf_num, pcm_card_num);
+		}
+		if (atomic_read(&uadev[pcm_card_num].in_use))
+			kref_put(&uadev[pcm_card_num].kref,
+					uaudio_dev_release);
+		mutex_unlock(&chip->mutex);
+	}
+
+	resp.usb_token = req_msg->usb_token;
+	resp.usb_token_valid = 1;
+	resp.internal_status = ret;
+	resp.internal_status_valid = 1;
+	resp.status = ret ? USB_AUDIO_STREAM_REQ_FAILURE_V01 : ret;
+	resp.status_valid = 1;
+	ret = qmi_send_response(svc->uaudio_svc_hdl, sq, txn,
+			QMI_UAUDIO_STREAM_RESP_V01,
+			QMI_UAUDIO_STREAM_RESP_MSG_V01_MAX_MSG_LEN,
+			qmi_uaudio_stream_resp_msg_v01_ei, &resp);
+}
+
+static struct qmi_msg_handler uaudio_stream_req_handlers = {
+	.type = QMI_REQUEST,
+	.msg_id = QMI_UAUDIO_STREAM_REQ_V01,
+	.ei = qmi_uaudio_stream_req_msg_v01_ei,
+	.decoded_size = QMI_UAUDIO_STREAM_REQ_MSG_V01_MAX_MSG_LEN,
+	.fn = handle_uaudio_stream_req,
+};
+
+int qc_usb_audio_offload_init_qmi_dev(void)
+{
+	struct q6usb_offload *data;
+
+	uaudio_qdev = kzalloc(sizeof(struct uaudio_qmi_dev),
+		GFP_KERNEL);
+	if (!uaudio_qdev)
+		return -ENOMEM;
+
+	/* initialize xfer ring and xfer buf iova list */
+	INIT_LIST_HEAD(&uaudio_qdev->xfer_ring_list);
+	uaudio_qdev->curr_xfer_ring_iova = IOVA_XFER_RING_BASE;
+	uaudio_qdev->xfer_ring_iova_size =
+			IOVA_XFER_RING_MAX - IOVA_XFER_RING_BASE;
+
+	INIT_LIST_HEAD(&uaudio_qdev->xfer_buf_list);
+	uaudio_qdev->curr_xfer_buf_iova = IOVA_XFER_BUF_BASE;
+	uaudio_qdev->xfer_buf_iova_size =
+		IOVA_XFER_BUF_MAX - IOVA_XFER_BUF_BASE;
+
+	data = snd_soc_usb_get_priv_data();
+	if (data) {
+		uaudio_qdev->domain = data->domain;
+		uaudio_qdev->intr_num = data->intr_num;
+		uaudio_qdev->sid = data->sid;
+		uaudio_qdev->dev = data->dev;
+	}
+
+	return 0;
+}
+
+void qc_usb_audio_offload_probe(struct usb_interface *intf, struct snd_usb_audio *chip)
+{
+	if (!uaudio_qdev)
+		qc_usb_audio_offload_init_qmi_dev();
+
+	atomic_inc(&uaudio_qdev->qdev_in_use);
+	uadev[chip->card->number].chip = chip;
+	uaudio_qdev->last_card_num = chip->card->number;
+	snd_soc_usb_connect(chip->index);
+}
+
+static void uaudio_dev_cleanup(struct uaudio_dev *dev)
+{
+	int if_idx;
+
+	if (!dev->udev) {
+		pr_info("USB audio device memory is already freed.\n");
+		return;
+	}
+
+	/* free xfer buffer and unmap xfer ring and buf per interface */
+	for (if_idx = 0; if_idx < dev->num_intf; if_idx++) {
+		if (!dev->info[if_idx].in_use)
+			continue;
+		uaudio_dev_intf_cleanup(dev->udev, &dev->info[if_idx]);
+		pr_debug("release resources: intf# %d card# %d\n",
+				dev->info[if_idx].intf_num, dev->card_num);
+	}
+
+	dev->num_intf = 0;
+
+	/* free interface info */
+	kfree(dev->info);
+	dev->info = NULL;
+	uaudio_event_ring_cleanup_free(dev);
+	dev->udev = NULL;
+}
+
+static void qc_usb_audio_cleanup_qmi_dev(void)
+{
+	kfree(uaudio_qdev);
+	uaudio_qdev = NULL;
+}
+
+void qc_usb_audio_offload_disconnect(struct usb_interface *intf)
+{
+	int ret;
+	struct snd_usb_audio *chip = usb_get_intfdata(intf);
+	struct uaudio_dev *dev;
+	int card_num;
+	struct uaudio_qmi_svc *svc = uaudio_svc;
+	struct qmi_uaudio_stream_ind_msg_v01 disconnect_ind = {0};
+
+	if (!chip) {
+		pr_err("chip is NULL\n");
+		return;
+	}
+
+	card_num = chip->card->number;
+	pr_debug("intf: %s: %p chip: %p card: %d\n", dev_name(&intf->dev),
+			intf, chip, card_num);
+
+	if (card_num >= SNDRV_CARDS) {
+		pr_err("invalid card number\n");
+		return;
+	}
+
+	dev = &uadev[card_num];
+
+	/* clean up */
+	if (!dev->udev) {
+		pr_debug("no clean up required\n");
+		goto done;
+	}
+
+	if (atomic_read(&dev->in_use)) {
+		pr_debug("sending qmi indication disconnect\n");
+		pr_debug("sq->sq_family:%x sq->sq_node:%x sq->sq_port:%x\n",
+				svc->client_sq.sq_family,
+				svc->client_sq.sq_node, svc->client_sq.sq_port);
+		disconnect_ind.dev_event = USB_AUDIO_DEV_DISCONNECT_V01;
+		disconnect_ind.slot_id = dev->udev->slot_id;
+		disconnect_ind.controller_num = dev->usb_core_id;
+		disconnect_ind.controller_num_valid = 1;
+		ret = qmi_send_indication(svc->uaudio_svc_hdl, &svc->client_sq,
+				QMI_UAUDIO_STREAM_IND_V01,
+				QMI_UAUDIO_STREAM_IND_MSG_V01_MAX_MSG_LEN,
+				qmi_uaudio_stream_ind_msg_v01_ei,
+				&disconnect_ind);
+		if (ret < 0)
+			pr_err("qmi send failed with err: %d\n", ret);
+
+		ret = wait_event_interruptible_timeout(dev->disconnect_wq,
+				!atomic_read(&dev->in_use),
+				msecs_to_jiffies(DEV_RELEASE_WAIT_TIMEOUT));
+		if (!ret) {
+			pr_err("timeout while waiting for dev_release\n");
+			atomic_set(&dev->in_use, 0);
+		} else if (ret < 0) {
+			pr_err("failed with ret %d\n", ret);
+			atomic_set(&dev->in_use, 0);
+		}
+	}
+
+	uaudio_dev_cleanup(dev);
+done:
+	uadev[card_num].chip = NULL;
+
+	atomic_dec(&uaudio_qdev->qdev_in_use);
+	if (!atomic_read(&uaudio_qdev->qdev_in_use)) {
+		snd_soc_usb_disconnect();
+		qc_usb_audio_cleanup_qmi_dev();
+	}
+}
+
+struct snd_usb_vendor_ops offload_ops = {
+	.connect_cb = qc_usb_audio_offload_probe,
+	.disconnect_cb = qc_usb_audio_offload_disconnect,
+};
+
+static int __init qc_usb_audio_offload_init(void)
+{
+	struct uaudio_qmi_svc *svc;
+	int ret;
+
+	ret = snd_usb_register_vendor_ops(&offload_ops);
+	if (ret < 0)
+		return ret;
+
+	svc = kzalloc(sizeof(struct uaudio_qmi_svc), GFP_KERNEL);
+	if (!svc) {
+		ret = -ENOMEM;
+		goto unreg_ops;
+	}
+
+	svc->uaudio_wq = create_singlethread_workqueue("uaudio_svc");
+	if (!svc->uaudio_wq) {
+		ret = -ENOMEM;
+		goto free_svc;
+	}
+
+	svc->uaudio_svc_hdl = kzalloc(sizeof(struct qmi_handle), GFP_KERNEL);
+	if (!svc->uaudio_svc_hdl) {
+		ret = -ENOMEM;
+		goto free_wq;
+	}
+
+	ret = qmi_handle_init(svc->uaudio_svc_hdl,
+				QMI_UAUDIO_STREAM_REQ_MSG_V01_MAX_MSG_LEN,
+				&uaudio_svc_ops_options,
+				&uaudio_stream_req_handlers);
+	ret = qmi_add_server(svc->uaudio_svc_hdl, UAUDIO_STREAM_SERVICE_ID_V01,
+					UAUDIO_STREAM_SERVICE_VERS_V01, 0);
+
+	INIT_WORK(&svc->qmi_disconnect_work, qmi_disconnect_work);
+	uaudio_svc = svc;
+
+	return 0;
+
+free_wq:
+	destroy_workqueue(svc->uaudio_wq);
+free_svc:
+	kfree(svc);
+unreg_ops:
+	snd_usb_unregister_vendor_ops();
+
+	return ret;
+}
+
+static void __exit qc_usb_audio_offload_exit(void)
+{
+	struct uaudio_qmi_svc *svc = uaudio_svc;
+
+	qmi_handle_release(svc->uaudio_svc_hdl);
+	flush_workqueue(svc->uaudio_wq);
+	destroy_workqueue(svc->uaudio_wq);
+	kfree(svc);
+	uaudio_svc = NULL;
+	snd_usb_unregister_vendor_ops();
+}
+
+module_init(qc_usb_audio_offload_init);
+module_exit(qc_usb_audio_offload_exit);
+
+MODULE_DESCRIPTION("QC USB Audio Offloading");
+MODULE_LICENSE("GPL");
diff --git a/sound/usb/qcom/usb_audio_qmi_v01.c b/sound/usb/qcom/usb_audio_qmi_v01.c
new file mode 100644
index 000000000000..95ae434f0a41
--- /dev/null
+++ b/sound/usb/qcom/usb_audio_qmi_v01.c
@@ -0,0 +1,892 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#include <linux/soc/qcom/qmi.h>
+
+#include "usb_audio_qmi_v01.h"
+
+static struct qmi_elem_info mem_info_v01_ei[] = {
+	{
+		.data_type	= QMI_UNSIGNED_8_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u64),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct mem_info_v01, va),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_8_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u64),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct mem_info_v01, pa),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_4_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u32),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct mem_info_v01, size),
+	},
+	{
+		.data_type	= QMI_EOTI,
+		.array_type	= NO_ARRAY,
+		.tlv_type	= QMI_COMMON_TLV_TYPE,
+	},
+};
+
+static struct qmi_elem_info apps_mem_info_v01_ei[] = {
+	{
+		.data_type	= QMI_STRUCT,
+		.elem_len	= 1,
+		.elem_size	= sizeof(struct mem_info_v01),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct apps_mem_info_v01, evt_ring),
+		.ei_array	= mem_info_v01_ei,
+	},
+	{
+		.data_type	= QMI_STRUCT,
+		.elem_len	= 1,
+		.elem_size	= sizeof(struct mem_info_v01),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct apps_mem_info_v01, tr_data),
+		.ei_array	= mem_info_v01_ei,
+	},
+	{
+		.data_type	= QMI_STRUCT,
+		.elem_len	= 1,
+		.elem_size	= sizeof(struct mem_info_v01),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct apps_mem_info_v01, tr_sync),
+		.ei_array	= mem_info_v01_ei,
+	},
+	{
+		.data_type	= QMI_STRUCT,
+		.elem_len	= 1,
+		.elem_size	= sizeof(struct mem_info_v01),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct apps_mem_info_v01, xfer_buff),
+		.ei_array	= mem_info_v01_ei,
+	},
+	{
+		.data_type	= QMI_STRUCT,
+		.elem_len	= 1,
+		.elem_size	= sizeof(struct mem_info_v01),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct apps_mem_info_v01, dcba),
+		.ei_array	= mem_info_v01_ei,
+	},
+	{
+		.data_type	= QMI_EOTI,
+		.array_type	= NO_ARRAY,
+		.tlv_type	= QMI_COMMON_TLV_TYPE,
+	},
+};
+
+static struct qmi_elem_info usb_endpoint_descriptor_v01_ei[] = {
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct usb_endpoint_descriptor_v01,
+						bLength),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct usb_endpoint_descriptor_v01,
+						bDescriptorType),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct usb_endpoint_descriptor_v01,
+						bEndpointAddress),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct usb_endpoint_descriptor_v01,
+						bmAttributes),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_2_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u16),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct usb_endpoint_descriptor_v01,
+						wMaxPacketSize),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct usb_endpoint_descriptor_v01,
+						bInterval),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct usb_endpoint_descriptor_v01,
+						bRefresh),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct usb_endpoint_descriptor_v01,
+						bSynchAddress),
+	},
+	{
+		.data_type	= QMI_EOTI,
+		.array_type	= NO_ARRAY,
+		.tlv_type	= QMI_COMMON_TLV_TYPE,
+	},
+};
+
+static struct qmi_elem_info usb_interface_descriptor_v01_ei[] = {
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct usb_interface_descriptor_v01,
+						bLength),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct usb_interface_descriptor_v01,
+						bDescriptorType),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct usb_interface_descriptor_v01,
+						bInterfaceNumber),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct usb_interface_descriptor_v01,
+						bAlternateSetting),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct usb_interface_descriptor_v01,
+						bNumEndpoints),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct usb_interface_descriptor_v01,
+						bInterfaceClass),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct usb_interface_descriptor_v01,
+						bInterfaceSubClass),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct usb_interface_descriptor_v01,
+						bInterfaceProtocol),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0,
+		.offset		= offsetof(struct usb_interface_descriptor_v01,
+						iInterface),
+	},
+	{
+		.data_type	= QMI_EOTI,
+		.array_type	= NO_ARRAY,
+		.tlv_type	= QMI_COMMON_TLV_TYPE,
+	},
+};
+
+struct qmi_elem_info qmi_uaudio_stream_req_msg_v01_ei[] = {
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x01,
+		.offset		= offsetof(struct qmi_uaudio_stream_req_msg_v01,
+						enable),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_4_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u32),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x02,
+		.offset		= offsetof(struct qmi_uaudio_stream_req_msg_v01,
+						usb_token),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x10,
+		.offset		= offsetof(struct qmi_uaudio_stream_req_msg_v01,
+						audio_format_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_4_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u32),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x10,
+		.offset		= offsetof(struct qmi_uaudio_stream_req_msg_v01,
+						audio_format),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x11,
+		.offset		= offsetof(struct qmi_uaudio_stream_req_msg_v01,
+						number_of_ch_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_4_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u32),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x11,
+		.offset		= offsetof(struct qmi_uaudio_stream_req_msg_v01,
+						number_of_ch),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x12,
+		.offset		= offsetof(struct qmi_uaudio_stream_req_msg_v01,
+						bit_rate_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_4_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u32),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x12,
+		.offset		= offsetof(struct qmi_uaudio_stream_req_msg_v01,
+						bit_rate),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x13,
+		.offset		= offsetof(struct qmi_uaudio_stream_req_msg_v01,
+						xfer_buff_size_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_4_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u32),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x13,
+		.offset		= offsetof(struct qmi_uaudio_stream_req_msg_v01,
+						xfer_buff_size),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x14,
+		.offset		= offsetof(struct qmi_uaudio_stream_req_msg_v01,
+						service_interval_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_4_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u32),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x14,
+		.offset		= offsetof(struct qmi_uaudio_stream_req_msg_v01,
+						service_interval),
+	},
+	{
+		.data_type	= QMI_EOTI,
+		.array_type	= NO_ARRAY,
+		.tlv_type	= QMI_COMMON_TLV_TYPE,
+	},
+};
+
+struct qmi_elem_info qmi_uaudio_stream_resp_msg_v01_ei[] = {
+	{
+		.data_type	= QMI_STRUCT,
+		.elem_len	= 1,
+		.elem_size	= sizeof(struct qmi_response_type_v01),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x02,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					resp),
+		.ei_array	= qmi_response_type_v01_ei,
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x10,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					status_valid),
+	},
+	{
+		.data_type	= QMI_SIGNED_4_BYTE_ENUM,
+		.elem_len	= 1,
+		.elem_size	= sizeof(enum usb_audio_stream_status_enum_v01),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x10,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					status),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x11,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					internal_status_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_4_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u32),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x11,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					internal_status),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x12,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					slot_id_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_4_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u32),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x12,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					slot_id),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x13,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					usb_token_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_4_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u32),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x13,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					usb_token),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x14,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					std_as_opr_intf_desc_valid),
+	},
+	{
+		.data_type	= QMI_STRUCT,
+		.elem_len	= 1,
+		.elem_size	= sizeof(struct usb_interface_descriptor_v01),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x14,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					std_as_opr_intf_desc),
+		.ei_array	= usb_interface_descriptor_v01_ei,
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x15,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					std_as_data_ep_desc_valid),
+	},
+	{
+		.data_type	= QMI_STRUCT,
+		.elem_len	= 1,
+		.elem_size	= sizeof(struct usb_endpoint_descriptor_v01),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x15,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					std_as_data_ep_desc),
+		.ei_array	= usb_endpoint_descriptor_v01_ei,
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x16,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					std_as_sync_ep_desc_valid),
+	},
+	{
+		.data_type	= QMI_STRUCT,
+		.elem_len	= 1,
+		.elem_size	= sizeof(struct usb_endpoint_descriptor_v01),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x16,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					std_as_sync_ep_desc),
+		.ei_array	= usb_endpoint_descriptor_v01_ei,
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x17,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					usb_audio_spec_revision_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_2_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u16),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x17,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					usb_audio_spec_revision),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x18,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					data_path_delay_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x18,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					data_path_delay),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x19,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					usb_audio_subslot_size_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x19,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					usb_audio_subslot_size),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x1A,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					xhci_mem_info_valid),
+	},
+	{
+		.data_type	= QMI_STRUCT,
+		.elem_len	= 1,
+		.elem_size	= sizeof(struct apps_mem_info_v01),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x1A,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					xhci_mem_info),
+		.ei_array	= apps_mem_info_v01_ei,
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x1B,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					interrupter_num_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x1B,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					interrupter_num),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x1C,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					speed_info_valid),
+	},
+	{
+		.data_type	= QMI_SIGNED_4_BYTE_ENUM,
+		.elem_len	= 1,
+		.elem_size	= sizeof(enum usb_audio_device_speed_enum_v01),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x1C,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					speed_info),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x1D,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					controller_num_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x1D,
+		.offset		= offsetof(
+					struct qmi_uaudio_stream_resp_msg_v01,
+					controller_num),
+	},
+	{
+		.data_type	= QMI_EOTI,
+		.array_type	= NO_ARRAY,
+		.tlv_type	= QMI_COMMON_TLV_TYPE,
+	},
+};
+
+struct qmi_elem_info qmi_uaudio_stream_ind_msg_v01_ei[] = {
+	{
+		.data_type	= QMI_SIGNED_4_BYTE_ENUM,
+		.elem_len	= 1,
+		.elem_size	= sizeof(
+				enum usb_audio_device_indication_enum_v01),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x01,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						dev_event),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_4_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u32),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x02,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						slot_id),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x10,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						usb_token_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_4_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u32),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x10,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						usb_token),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x11,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						std_as_opr_intf_desc_valid),
+	},
+	{
+		.data_type	= QMI_STRUCT,
+		.elem_len	= 1,
+		.elem_size	= sizeof(struct usb_interface_descriptor_v01),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x11,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						std_as_opr_intf_desc),
+		.ei_array	= usb_interface_descriptor_v01_ei,
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x12,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						std_as_data_ep_desc_valid),
+	},
+	{
+		.data_type	= QMI_STRUCT,
+		.elem_len	= 1,
+		.elem_size	= sizeof(struct usb_endpoint_descriptor_v01),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x12,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						std_as_data_ep_desc),
+		.ei_array	= usb_endpoint_descriptor_v01_ei,
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x13,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						std_as_sync_ep_desc_valid),
+	},
+	{
+		.data_type	= QMI_STRUCT,
+		.elem_len	= 1,
+		.elem_size	= sizeof(struct usb_endpoint_descriptor_v01),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x13,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						std_as_sync_ep_desc),
+		.ei_array	= usb_endpoint_descriptor_v01_ei,
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x14,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						usb_audio_spec_revision_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_2_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u16),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x14,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						usb_audio_spec_revision),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x15,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						data_path_delay_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x15,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						data_path_delay),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x16,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						usb_audio_subslot_size_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x16,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						usb_audio_subslot_size),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x17,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						xhci_mem_info_valid),
+	},
+	{
+		.data_type	= QMI_STRUCT,
+		.elem_len	= 1,
+		.elem_size	= sizeof(struct apps_mem_info_v01),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x17,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						xhci_mem_info),
+		.ei_array	= apps_mem_info_v01_ei,
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x18,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						interrupter_num_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x18,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						interrupter_num),
+	},
+	{
+		.data_type	= QMI_OPT_FLAG,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x19,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						controller_num_valid),
+	},
+	{
+		.data_type	= QMI_UNSIGNED_1_BYTE,
+		.elem_len	= 1,
+		.elem_size	= sizeof(u8),
+		.array_type	= NO_ARRAY,
+		.tlv_type	= 0x19,
+		.offset		= offsetof(struct qmi_uaudio_stream_ind_msg_v01,
+						controller_num),
+	},
+	{
+		.data_type	= QMI_EOTI,
+		.array_type	= NO_ARRAY,
+		.tlv_type	= QMI_COMMON_TLV_TYPE,
+	},
+};
diff --git a/sound/usb/qcom/usb_audio_qmi_v01.h b/sound/usb/qcom/usb_audio_qmi_v01.h
new file mode 100644
index 000000000000..4e9b5f0bcddf
--- /dev/null
+++ b/sound/usb/qcom/usb_audio_qmi_v01.h
@@ -0,0 +1,162 @@
+/* SPDX-License-Identifier: GPL-2.0
+ *
+ * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
+ */
+
+#ifndef USB_QMI_V01_H
+#define USB_QMI_V01_H
+
+#define UAUDIO_STREAM_SERVICE_ID_V01 0x41D
+#define UAUDIO_STREAM_SERVICE_VERS_V01 0x01
+
+#define QMI_UAUDIO_STREAM_RESP_V01 0x0001
+#define QMI_UAUDIO_STREAM_REQ_V01 0x0001
+#define QMI_UAUDIO_STREAM_IND_V01 0x0001
+
+
+struct mem_info_v01 {
+	u64 va;
+	u64 pa;
+	u32 size;
+};
+
+struct apps_mem_info_v01 {
+	struct mem_info_v01 evt_ring;
+	struct mem_info_v01 tr_data;
+	struct mem_info_v01 tr_sync;
+	struct mem_info_v01 xfer_buff;
+	struct mem_info_v01 dcba;
+};
+
+struct usb_endpoint_descriptor_v01 {
+	u8 bLength;
+	u8 bDescriptorType;
+	u8 bEndpointAddress;
+	u8 bmAttributes;
+	u16 wMaxPacketSize;
+	u8 bInterval;
+	u8 bRefresh;
+	u8 bSynchAddress;
+};
+
+struct usb_interface_descriptor_v01 {
+	u8 bLength;
+	u8 bDescriptorType;
+	u8 bInterfaceNumber;
+	u8 bAlternateSetting;
+	u8 bNumEndpoints;
+	u8 bInterfaceClass;
+	u8 bInterfaceSubClass;
+	u8 bInterfaceProtocol;
+	u8 iInterface;
+};
+
+enum usb_audio_stream_status_enum_v01 {
+	USB_AUDIO_STREAM_STATUS_ENUM_MIN_VAL_V01 = INT_MIN,
+	USB_AUDIO_STREAM_REQ_SUCCESS_V01 = 0,
+	USB_AUDIO_STREAM_REQ_FAILURE_V01 = 1,
+	USB_AUDIO_STREAM_REQ_FAILURE_NOT_FOUND_V01 = 2,
+	USB_AUDIO_STREAM_REQ_FAILURE_INVALID_PARAM_V01 = 3,
+	USB_AUDIO_STREAM_REQ_FAILURE_MEMALLOC_V01 = 4,
+	USB_AUDIO_STREAM_STATUS_ENUM_MAX_VAL_V01 = INT_MAX,
+};
+
+enum usb_audio_device_indication_enum_v01 {
+	USB_AUDIO_DEVICE_INDICATION_ENUM_MIN_VAL_V01 = INT_MIN,
+	USB_AUDIO_DEV_CONNECT_V01 = 0,
+	USB_AUDIO_DEV_DISCONNECT_V01 = 1,
+	USB_AUDIO_DEV_SUSPEND_V01 = 2,
+	USB_AUDIO_DEV_RESUME_V01 = 3,
+	USB_AUDIO_DEVICE_INDICATION_ENUM_MAX_VAL_V01 = INT_MAX,
+};
+
+enum usb_audio_device_speed_enum_v01 {
+	USB_AUDIO_DEVICE_SPEED_ENUM_MIN_VAL_V01 = INT_MIN,
+	USB_AUDIO_DEVICE_SPEED_INVALID_V01 = 0,
+	USB_AUDIO_DEVICE_SPEED_LOW_V01 = 1,
+	USB_AUDIO_DEVICE_SPEED_FULL_V01 = 2,
+	USB_AUDIO_DEVICE_SPEED_HIGH_V01 = 3,
+	USB_AUDIO_DEVICE_SPEED_SUPER_V01 = 4,
+	USB_AUDIO_DEVICE_SPEED_SUPER_PLUS_V01 = 5,
+	USB_AUDIO_DEVICE_SPEED_ENUM_MAX_VAL_V01 = INT_MAX,
+};
+
+struct qmi_uaudio_stream_req_msg_v01 {
+	u8 enable;
+	u32 usb_token;
+	u8 audio_format_valid;
+	u32 audio_format;
+	u8 number_of_ch_valid;
+	u32 number_of_ch;
+	u8 bit_rate_valid;
+	u32 bit_rate;
+	u8 xfer_buff_size_valid;
+	u32 xfer_buff_size;
+	u8 service_interval_valid;
+	u32 service_interval;
+};
+#define QMI_UAUDIO_STREAM_REQ_MSG_V01_MAX_MSG_LEN 46
+extern struct qmi_elem_info qmi_uaudio_stream_req_msg_v01_ei[];
+
+struct qmi_uaudio_stream_resp_msg_v01 {
+	struct qmi_response_type_v01 resp;
+	u8 status_valid;
+	enum usb_audio_stream_status_enum_v01 status;
+	u8 internal_status_valid;
+	u32 internal_status;
+	u8 slot_id_valid;
+	u32 slot_id;
+	u8 usb_token_valid;
+	u32 usb_token;
+	u8 std_as_opr_intf_desc_valid;
+	struct usb_interface_descriptor_v01 std_as_opr_intf_desc;
+	u8 std_as_data_ep_desc_valid;
+	struct usb_endpoint_descriptor_v01 std_as_data_ep_desc;
+	u8 std_as_sync_ep_desc_valid;
+	struct usb_endpoint_descriptor_v01 std_as_sync_ep_desc;
+	u8 usb_audio_spec_revision_valid;
+	u16 usb_audio_spec_revision;
+	u8 data_path_delay_valid;
+	u8 data_path_delay;
+	u8 usb_audio_subslot_size_valid;
+	u8 usb_audio_subslot_size;
+	u8 xhci_mem_info_valid;
+	struct apps_mem_info_v01 xhci_mem_info;
+	u8 interrupter_num_valid;
+	u8 interrupter_num;
+	u8 speed_info_valid;
+	enum usb_audio_device_speed_enum_v01 speed_info;
+	u8 controller_num_valid;
+	u8 controller_num;
+};
+#define QMI_UAUDIO_STREAM_RESP_MSG_V01_MAX_MSG_LEN 202
+extern struct qmi_elem_info qmi_uaudio_stream_resp_msg_v01_ei[];
+
+struct qmi_uaudio_stream_ind_msg_v01 {
+	enum usb_audio_device_indication_enum_v01 dev_event;
+	u32 slot_id;
+	u8 usb_token_valid;
+	u32 usb_token;
+	u8 std_as_opr_intf_desc_valid;
+	struct usb_interface_descriptor_v01 std_as_opr_intf_desc;
+	u8 std_as_data_ep_desc_valid;
+	struct usb_endpoint_descriptor_v01 std_as_data_ep_desc;
+	u8 std_as_sync_ep_desc_valid;
+	struct usb_endpoint_descriptor_v01 std_as_sync_ep_desc;
+	u8 usb_audio_spec_revision_valid;
+	u16 usb_audio_spec_revision;
+	u8 data_path_delay_valid;
+	u8 data_path_delay;
+	u8 usb_audio_subslot_size_valid;
+	u8 usb_audio_subslot_size;
+	u8 xhci_mem_info_valid;
+	struct apps_mem_info_v01 xhci_mem_info;
+	u8 interrupter_num_valid;
+	u8 interrupter_num;
+	u8 controller_num_valid;
+	u8 controller_num;
+};
+#define QMI_UAUDIO_STREAM_IND_MSG_V01_MAX_MSG_LEN 181
+extern struct qmi_elem_info qmi_uaudio_stream_ind_msg_v01_ei[];
+
+#endif

[RFC PATCH 10/14] sound: usb: card: Check for support for requested audio format

[Wesley Cheng]

Allow for checks on a specific USB audio device to see if a requested PCM
format is supported.  This is needed for support for when playback is
initiated by the ASoC USB backend path.

Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
---
 sound/usb/card.c | 19 +++++++++++++++++++
 sound/usb/card.h |  3 +++
 2 files changed, 22 insertions(+)

diff --git a/sound/usb/card.c b/sound/usb/card.c
index 396e5a34e23b..9b8d2ed308c8 100644
--- a/sound/usb/card.c
+++ b/sound/usb/card.c
@@ -133,6 +133,25 @@ int snd_usb_unregister_vendor_ops(void)
 }
 EXPORT_SYMBOL_GPL(snd_usb_unregister_vendor_ops);
 
+struct snd_usb_stream *snd_usb_find_suppported_substream(int card_idx,
+			struct snd_pcm_hw_params *params, int direction)
+{
+	struct snd_usb_stream *as;
+	struct snd_usb_substream *subs = NULL;
+	const struct audioformat *fmt;
+
+	if (usb_chip[card_idx] && enable[card_idx]) {
+		list_for_each_entry(as, &usb_chip[card_idx]->pcm_list, list) {
+			subs = &as->substream[direction];
+			fmt = find_substream_format(subs, params);
+			if (fmt)
+				return as;
+		}
+	}
+
+	return 0;
+}
+
 /*
  * disconnect streams
  * called from usb_audio_disconnect()
diff --git a/sound/usb/card.h b/sound/usb/card.h
index a785bb256b0d..d4936b6054fc 100644
--- a/sound/usb/card.h
+++ b/sound/usb/card.h
@@ -213,4 +213,7 @@ struct snd_usb_vendor_ops {
 
 int snd_usb_register_vendor_ops(struct snd_usb_vendor_ops *ops);
 int snd_usb_unregister_vendor_ops(void);
+
+struct snd_usb_stream *snd_usb_find_suppported_substream(int card_idx,
+			struct snd_pcm_hw_params *params, int direction);
 #endif /* __USBAUDIO_CARD_H */

[RFC PATCH 11/14] sound: soc: soc-usb: Add PCM format check API for USB backend

[Wesley Cheng]

Introduce a check for if a particular PCM format is supported by the USB
audio device connected.  If the USB audio device does not have an audio
profile which can support the requested format, then notify the USB
backend.

Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
---
 include/sound/soc-usb.h |  3 +++
 sound/soc/soc-usb.c     | 13 +++++++++++++
 2 files changed, 16 insertions(+)

diff --git a/include/sound/soc-usb.h b/include/sound/soc-usb.h
index 7d52e5d2371c..36df6f4fe093 100644
--- a/include/sound/soc-usb.h
+++ b/include/sound/soc-usb.h
@@ -19,6 +19,9 @@ struct snd_soc_usb {
 	void *priv_data;
 };
 
+int snd_soc_usb_find_format(int card_idx, struct snd_pcm_hw_params *params,
+			int direction);
+
 int snd_soc_usb_connect(int card_idx);
 int snd_soc_usb_disconnect(void);
 void snd_soc_usb_set_priv_data(void *priv);
diff --git a/sound/soc/soc-usb.c b/sound/soc/soc-usb.c
index c6c376960e4d..021380a022bc 100644
--- a/sound/soc/soc-usb.c
+++ b/sound/soc/soc-usb.c
@@ -25,6 +25,19 @@ void snd_soc_usb_set_priv_data(void *priv)
 }
 EXPORT_SYMBOL_GPL(snd_soc_usb_set_priv_data);
 
+int snd_soc_usb_find_format(int card_idx, struct snd_pcm_hw_params *params,
+			int direction)
+{
+	struct snd_usb_stream *as;
+
+	as = snd_usb_find_suppported_substream(card_idx, params, direction);
+	if (!as)
+		return -EOPNOTSUPP;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(snd_soc_usb_find_format);
+
 struct snd_soc_usb *snd_soc_usb_add_port(struct device *dev,
 			int (*connection_cb)(struct snd_soc_usb *usb, int card_idx,
 			int connected))

[RFC PATCH 12/14] sound: soc: qcom: qusb6: Ensure PCM format is supported by USB audio device

[Wesley Cheng]

Check for if the PCM format is supported during the hw_params callback.  If
the profile is not supported then the userspace ALSA entity will receive an
error, and can take further action.

Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
---
 sound/soc/qcom/qdsp6/q6usb.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/sound/soc/qcom/qdsp6/q6usb.c b/sound/soc/qcom/qdsp6/q6usb.c
index a9da6dec6c6f..128e0974db4e 100644
--- a/sound/soc/qcom/qdsp6/q6usb.c
+++ b/sound/soc/qcom/qdsp6/q6usb.c
@@ -42,7 +42,14 @@ static int q6usb_hw_params(struct snd_pcm_substream *substream,
 			   struct snd_pcm_hw_params *params,
 			   struct snd_soc_dai *dai)
 {
-	return 0;
+	struct q6usb_port_data *data = dev_get_drvdata(dai->dev);
+	int direction = substream->stream;
+	int ret;
+
+	ret = snd_soc_usb_find_format(data->active_idx, params, direction);
+
+	return ret;
+
 }
 static const struct snd_soc_dai_ops q6usb_ops = {
 	.hw_params	= q6usb_hw_params,

[RFC PATCH 13/14] ASoC: dt-bindings: Add Q6USB backend bindings

[Wesley Cheng]

Add a dt-binding to describe the definition of enabling the Q6 USB backend
device for audio offloading.  The node carries information, which is passed
along to the QC USB SND class driver counterpart.  These parameters will be
utilized during QMI stream enable requests.

Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
---
 .../bindings/sound/qcom,q6usb-dais.yaml       | 55 +++++++++++++++++++
 1 file changed, 55 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml

diff --git a/Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml b/Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml
new file mode 100644
index 000000000000..e24b4d52fa7e
--- /dev/null
+++ b/Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml
@@ -0,0 +1,55 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/sound/qcom,q6usb-dais.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm ASoC USB backend DAI
+
+maintainers:
+  - Wesley Cheng <quic_wcheng@quicinc.com>
+
+description:
+  The Q6USB backend is a supported AFE port on the Q6DSP. This backend
+  driver will communicate the required settings to the QC USB SND class
+  driver for properly enabling the audio stream.  Parameters defined
+  under this node will carry settings, which will be passed along during
+  the QMI stream enable request.
+
+properties:
+  compatible:
+    enum:
+      - qcom,q6usb-dais
+
+  iommus:
+    maxItems: 1
+
+  "#sound-dai-cells":
+    const: 1
+
+  qcom,usb-audio-stream-id:
+    description:
+      SID for the Q6DSP processor for IOMMU mapping.
+    $ref: /schemas/types.yaml#/definitions/uint32
+
+  qcom,usb-audio-intr-num:
+    description:
+      Desired XHCI interrupter number to use.
+    $ref: /schemas/types.yaml#/definitions/uint32
+
+required:
+  - compatible
+  - '#sound-dai-cells'
+  - qcom,usb-audio-intr-num
+
+additionalProperties: false
+
+examples:
+  - |
+    usbdai: usbd {
+      compatible = "qcom,q6usb-dais";
+      #sound-dai-cells = <1>;
+      iommus = <&apps_smmu 0x180f 0x0>;
+      qcom,usb-audio-stream-id = <0xf>;
+      qcom,usb-audio-intr-num = <2>;
+    };

[RFC PATCH 14/14] ASoC: dt-bindings: Update example for enabling USB offload on SM8250

[Wesley Cheng]

Add an example on enabling of USB offload for the Q6DSP.  The routing can
be done by the mixer, which can pass the multimedia stream to the USB
backend.

Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
---
 .../devicetree/bindings/sound/qcom,sm8250.yaml      | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/Documentation/devicetree/bindings/sound/qcom,sm8250.yaml b/Documentation/devicetree/bindings/sound/qcom,sm8250.yaml
index 70080d04ddc9..60cd84e6727a 100644
--- a/Documentation/devicetree/bindings/sound/qcom,sm8250.yaml
+++ b/Documentation/devicetree/bindings/sound/qcom,sm8250.yaml
@@ -216,6 +216,19 @@ examples:
                 sound-dai = <&vamacro 0>;
             };
         };
+
+        usb-dai-link {
+            link-name = "USB Playback";
+            cpu {
+                sound-dai = <&q6afedai USB_RX>;
+            };
+            codec {
+                sound-dai = <&usbdai USB_RX>;
+            };
+            platform {
+                sound-dai = <&q6routing>;
+            };
+        };
     };
 
   - |

Re: [RFC PATCH 00/14] Introduce QC USB SND audio offloading support

[Greg KH]

On Fri, Dec 23, 2022 at 03:31:46PM -0800, Wesley Cheng wrote:
> Several Qualcomm based chipsets can support USB audio offloading to a
> dedicated audio DSP, which can take over issuing transfers to the USB
> host controller.  The intention is to reduce the load on the main
> processors in the SoC, and allow them to be placed into lower power modes.
> There are several parts to this design:
>   1. Adding ASoC binding layer
>   2. Create a USB backend for Q6DSP
>   3. Introduce XHCI interrupter support
>   4. Create vendor ops for the USB SND driver
> 
> Adding ASoC binding layer:
> soc-usb: Intention is to treat a USB port similar to a headphone jack.
> The port is always present on the device, but cable/pin status can be
> enabled/disabled.  Expose mechanisms for USB backend ASoC drivers to
> communicate with USB SND.
> 
> Create a USB backend for Q6DSP:
> q6usb: Basic backend driver that will be responsible for maintaining the
> resources needed to initiate a playback stream using the Q6DSP.  Will
> be the entity that checks to make sure the connected USB audio device
> supports the requested PCM format.  If it does not, the PCM open call will
> fail, and userpsace ALSA can take action accordingly.
> 
> Introduce XHCI interrupter support:
> XHCI HCD supports multiple interrupters, which allows for events to be routed
> to different event rings.  This is determined by "Interrupter Target" field
> specified in Section "6.4.1.1 Normal TRB" of the XHCI specification.
> 
> Events in the offloading case will be routed to an event ring that is assigned
> to the audio DSP.
> 
> Create vendor ops for the USB SND driver:
> qc_audio_offload: This particular driver has several components associated
> with it:
> - QMI stream request handler
> - XHCI interrupter and resource management
> - audio DSP memory management
> 
> When the audio DSP wants to enable a playback stream, the request is first
> received by the ASoC platform sound card.  Depending on the selected route,
> ASoC will bring up the individual DAIs in the path.  The Q6USB backend DAI
> will send an AFE port start command (with enabling the USB playback path), and
> the audio DSP will handle the request accordingly.
> 
> Part of the AFE USB port start handling will have an exchange of control
> messages using the QMI protocol.  The qc_audio_offload driver will populate the
> buffer information:
> - Event ring base address
> - EP transfer ring base address
> 
> and pass it along to the audio DSP.  All endpoint management will now be handed
> over to the DSP, and the main processor is not involved in transfers.
> 
> Overall, implementing this feature will still expose separate sound card and PCM
> devices for both the platorm card and USB audio device:
>  0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D
>                       SM8250-MTP-WCD9380-WSA8810-VA-DMIC
>  1 [Audio          ]: USB-Audio - USB Audio
>                       Generic USB Audio at usb-xhci-hcd.1.auto-1.4, high speed
> 
> This is to ensure that userspace ALSA entities can decide which route to take
> when executing the audio playback.  In the above, if card#1 is selected, then
> USB audio data will take the legacy path over the USB PCM drivers, etc...
> 
> This feature was validated using:
> - tinymix: set/enable the multimedia path to route to USB backend
> - tinyplay: issue playback on platform card

This looks to duplicate a bunch of the same things that a number of
different google developers have posted recently.  Please work with them
to come up with a unified set of patches that you all can agree with,
AND get them to sign off on the changes before resubmitting them.

This uncoordinated drip of patches from different people doing the same
thing is almost impossible to review from our side, as I'm sure you can
imagine.

That being said, thank you finally for at least submitting all of the
needed changes together as one patch set.  That's a first, and something
we had been asking for for years.

Have a good holiday break,

greg k-h

Re: [RFC PATCH 01/14] ASoC: Add SOC USB APIs for adding an USB backend

[Greg KH]

On Fri, Dec 23, 2022 at 03:31:47PM -0800, Wesley Cheng wrote:
> diff --git a/sound/soc/soc-usb.c b/sound/soc/soc-usb.c
> new file mode 100644
> index 000000000000..c6c376960e4d
> --- /dev/null
> +++ b/sound/soc/soc-usb.c
> @@ -0,0 +1,66 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +#include <linux/usb.h>
> +#include <sound/soc.h>
> +#include <sound/soc-usb.h>
> +#include "../usb/card.h"
> +
> +struct snd_soc_usb *ctx;

Note, this will not work.  You can not only have "one" state for a
system like this.  That just broke any system with more than one
controller, of which we have millions.

This has to be dynamic for any number of controllers in the system, like
the sound and USB core can handle.  Any requirement of "there can be
only one!" will obviously never be acceptable as that is not how Linux
works.

thanks,

greg k-h

Re: [RFC PATCH 05/14] sound: usb: Export USB SND APIs for modules

[Greg KH]

On Fri, Dec 23, 2022 at 03:31:51PM -0800, Wesley Cheng wrote:
> +EXPORT_SYMBOL(snd_usb_autoresume);

EXPORT_SYMBOL_GPL()?  I have to ask...

Same for the other ones here.

> +EXPORT_SYMBOL(find_format);

That is not a valid global symbol name to use, you know better than
this...

thanks,

greg k-h

Re: [RFC PATCH 12/14] sound: soc: qcom: qusb6: Ensure PCM format is supported by USB audio device

[Sergey Shtylyov]

Hello!

On 12/24/22 2:31 AM, Wesley Cheng wrote:

> Check for if the PCM format is supported during the hw_params callback.  If
> the profile is not supported then the userspace ALSA entity will receive an
> error, and can take further action.
> 
> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
> ---
>  sound/soc/qcom/qdsp6/q6usb.c | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/sound/soc/qcom/qdsp6/q6usb.c b/sound/soc/qcom/qdsp6/q6usb.c
> index a9da6dec6c6f..128e0974db4e 100644
> --- a/sound/soc/qcom/qdsp6/q6usb.c
> +++ b/sound/soc/qcom/qdsp6/q6usb.c
> @@ -42,7 +42,14 @@ static int q6usb_hw_params(struct snd_pcm_substream *substream,
>  			   struct snd_pcm_hw_params *params,
>  			   struct snd_soc_dai *dai)
>  {
> -	return 0;
> +	struct q6usb_port_data *data = dev_get_drvdata(dai->dev);
> +	int direction = substream->stream;
> +	int ret;

   You don't seem to need this variable, just use *return*
snd_soc_usb_find_format(...).

> +
> +	ret = snd_soc_usb_find_format(data->active_idx, params, direction);
> +
> +	return ret;
> +
>  }
>  static const struct snd_soc_dai_ops q6usb_ops = {
>  	.hw_params	= q6usb_hw_params,

MBR, Sergey

Re: [RFC PATCH 00/14] Introduce QC USB SND audio offloading support

[Wesley Cheng]

Hi Greg,

On 12/23/2022 10:45 PM, Greg KH wrote:
> On Fri, Dec 23, 2022 at 03:31:46PM -0800, Wesley Cheng wrote:
>> Several Qualcomm based chipsets can support USB audio offloading to a
>> dedicated audio DSP, which can take over issuing transfers to the USB
>> host controller.  The intention is to reduce the load on the main
>> processors in the SoC, and allow them to be placed into lower power modes.
>> There are several parts to this design:
>>    1. Adding ASoC binding layer
>>    2. Create a USB backend for Q6DSP
>>    3. Introduce XHCI interrupter support
>>    4. Create vendor ops for the USB SND driver
>>
>> Adding ASoC binding layer:
>> soc-usb: Intention is to treat a USB port similar to a headphone jack.
>> The port is always present on the device, but cable/pin status can be
>> enabled/disabled.  Expose mechanisms for USB backend ASoC drivers to
>> communicate with USB SND.
>>
>> Create a USB backend for Q6DSP:
>> q6usb: Basic backend driver that will be responsible for maintaining the
>> resources needed to initiate a playback stream using the Q6DSP.  Will
>> be the entity that checks to make sure the connected USB audio device
>> supports the requested PCM format.  If it does not, the PCM open call will
>> fail, and userpsace ALSA can take action accordingly.
>>
>> Introduce XHCI interrupter support:
>> XHCI HCD supports multiple interrupters, which allows for events to be routed
>> to different event rings.  This is determined by "Interrupter Target" field
>> specified in Section "6.4.1.1 Normal TRB" of the XHCI specification.
>>
>> Events in the offloading case will be routed to an event ring that is assigned
>> to the audio DSP.
>>
>> Create vendor ops for the USB SND driver:
>> qc_audio_offload: This particular driver has several components associated
>> with it:
>> - QMI stream request handler
>> - XHCI interrupter and resource management
>> - audio DSP memory management
>>
>> When the audio DSP wants to enable a playback stream, the request is first
>> received by the ASoC platform sound card.  Depending on the selected route,
>> ASoC will bring up the individual DAIs in the path.  The Q6USB backend DAI
>> will send an AFE port start command (with enabling the USB playback path), and
>> the audio DSP will handle the request accordingly.
>>
>> Part of the AFE USB port start handling will have an exchange of control
>> messages using the QMI protocol.  The qc_audio_offload driver will populate the
>> buffer information:
>> - Event ring base address
>> - EP transfer ring base address
>>
>> and pass it along to the audio DSP.  All endpoint management will now be handed
>> over to the DSP, and the main processor is not involved in transfers.
>>
>> Overall, implementing this feature will still expose separate sound card and PCM
>> devices for both the platorm card and USB audio device:
>>   0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D
>>                        SM8250-MTP-WCD9380-WSA8810-VA-DMIC
>>   1 [Audio          ]: USB-Audio - USB Audio
>>                        Generic USB Audio at usb-xhci-hcd.1.auto-1.4, high speed
>>
>> This is to ensure that userspace ALSA entities can decide which route to take
>> when executing the audio playback.  In the above, if card#1 is selected, then
>> USB audio data will take the legacy path over the USB PCM drivers, etc...
>>
>> This feature was validated using:
>> - tinymix: set/enable the multimedia path to route to USB backend
>> - tinyplay: issue playback on platform card
> 
> This looks to duplicate a bunch of the same things that a number of
> different google developers have posted recently.  Please work with them
> to come up with a unified set of patches that you all can agree with,
> AND get them to sign off on the changes before resubmitting them.
> 
> This uncoordinated drip of patches from different people doing the same
> thing is almost impossible to review from our side, as I'm sure you can
> imagine.

I saw some of the Google patchsets submitted awhile back, but didn't 
really get a chance to look at them in detail.  Let me reach out to 
Albert Wang to see if we can come to a solution that works for both 
implementations.

 From the looks of it (at least from the XHCI HCD changes), it seems 
that a different set of resources is required for the Google 
implementation to work.  I'll need to ask for a bit more details before 
I can comment further...

> 
> That being said, thank you finally for at least submitting all of the
> needed changes together as one patch set.  That's a first, and something
> we had been asking for for years.
> 
> Have a good holiday break,

Thanks for the quick in-depth review, and the feedback.  Gives me some 
more things to think about improving over the break :).  Happy holidays!

Thanks
Wesley Cheng

Re: [RFC PATCH 12/14] sound: soc: qcom: qusb6: Ensure PCM format is supported by USB audio device

[Wesley Cheng]

Hi Sergey,

On 12/24/2022 12:19 AM, Sergey Shtylyov wrote:
> Hello!
> 
> On 12/24/22 2:31 AM, Wesley Cheng wrote:
> 
>> Check for if the PCM format is supported during the hw_params callback.  If
>> the profile is not supported then the userspace ALSA entity will receive an
>> error, and can take further action.
>>
>> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
>> ---
>>   sound/soc/qcom/qdsp6/q6usb.c | 9 ++++++++-
>>   1 file changed, 8 insertions(+), 1 deletion(-)
>>
>> diff --git a/sound/soc/qcom/qdsp6/q6usb.c b/sound/soc/qcom/qdsp6/q6usb.c
>> index a9da6dec6c6f..128e0974db4e 100644
>> --- a/sound/soc/qcom/qdsp6/q6usb.c
>> +++ b/sound/soc/qcom/qdsp6/q6usb.c
>> @@ -42,7 +42,14 @@ static int q6usb_hw_params(struct snd_pcm_substream *substream,
>>   			   struct snd_pcm_hw_params *params,
>>   			   struct snd_soc_dai *dai)
>>   {
>> -	return 0;
>> +	struct q6usb_port_data *data = dev_get_drvdata(dai->dev);
>> +	int direction = substream->stream;
>> +	int ret;
> 
>     You don't seem to need this variable, just use *return*
> snd_soc_usb_find_format(...) >

Thanks for catching this... Will fix it in the next submission I make. 
Happy holidays!

Thanks
Wesley Cheng

Re: [RFC PATCH 06/14] usb: core: hcd: Introduce USB HCD APIs for interrupter management

[Greg KH]

On Fri, Dec 23, 2022 at 03:31:52PM -0800, Wesley Cheng wrote:
> For USB HCDs that can support multiple USB interrupters, expose functions
> that class drivers can utilize for setting up secondary interrupters.
> Class drivers can pass this information to its respective clients, i.e.
> a dedicated DSP.

Where is the locking here that seems to be required when a hcd is
removed from the system and you have data in flight?  What am I missing
here in the design of this?

And yes, HCDs get removed all the time, and will happen more and more in
the future with the design of more systems moving to Thunderbolt/PCIe
designs due to the simplicity of it.

> +/**
> + * usb_set_interruper - Reserve an interrupter

Where is an "interrupter" defined?  I don't know what this term means
sorry, is this in the USB specification somewhere?


> + * @udev: usb device which requested the interrupter
> + * @intr_num: interrupter number to reserve
> + * @dma: iova address to event ring
> + *
> + * Request for a specific interrupter to be reserved for a USB class driver.
> + * This will return the base address to the event ring that was allocated to
> + * the specific interrupter.
> + **/
> +phys_addr_t usb_set_interruper(struct usb_device *udev, int intr_num,
> +							dma_addr_t *dma)
> +{
> +	struct usb_hcd *hcd;
> +	phys_addr_t pa = 0;
> +
> +	hcd = bus_to_hcd(udev->bus);
> +	if (hcd->driver->update_interrupter)
> +		pa = hcd->driver->update_interrupter(hcd, intr_num, dma);
> +
> +	return pa;

Wait, you return a physical address?  What are you going to do with
that?  And what guarantees that the address is valid after you return it
(again, remember memory and devices can be removed at any point in time.

thanks,

greg k-h

Re: [RFC PATCH 07/14] usb: host: xhci: Add XHCI secondary interrupter support

[Greg KH]

On Fri, Dec 23, 2022 at 03:31:53PM -0800, Wesley Cheng wrote:
> Implement the XHCI operations for allocating and requesting for a secondary
> interrupter.  The secondary interrupter can allow for events for a
> particular endpoint to be routed to a separate event ring.  The event
> routing is defined when submitting a transfer descriptor to the USB HW.
> There is a specific field which denotes which interrupter ring to route the
> event to when the transfer is completed.
> 
> An example use case, such as audio packet offloading can utilize a separate
> event ring, so that these events can be routed to a different processor
> within the system.  The processor would be able to independently submit
> transfers and handle its completions without intervention from the main
> processor.
> 
> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
> ---
>  drivers/usb/host/xhci-mem.c  | 219 ++++++++++++++++++++++++++++-------
>  drivers/usb/host/xhci-plat.c |   2 +
>  drivers/usb/host/xhci.c      | 169 ++++++++++++++++++++++++++-
>  drivers/usb/host/xhci.h      |  15 +++
>  4 files changed, 363 insertions(+), 42 deletions(-)
> 
> diff --git a/drivers/usb/host/xhci-mem.c b/drivers/usb/host/xhci-mem.c
> index 81ca2bc1f0be..d5cb4b82ad3d 100644
> --- a/drivers/usb/host/xhci-mem.c
> +++ b/drivers/usb/host/xhci-mem.c
> @@ -1835,6 +1835,7 @@ void xhci_free_erst(struct xhci_hcd *xhci, struct xhci_erst *erst)
>  void xhci_mem_cleanup(struct xhci_hcd *xhci)
>  {
>  	struct device	*dev = xhci_to_hcd(xhci)->self.sysdev;
> +	struct xhci_sec *sec, *tmp;
>  	int i, j, num_ports;
>  
>  	cancel_delayed_work_sync(&xhci->cmd_timer);
> @@ -1846,6 +1847,16 @@ void xhci_mem_cleanup(struct xhci_hcd *xhci)
>  	xhci->event_ring = NULL;
>  	xhci_dbg_trace(xhci, trace_xhci_dbg_init, "Freed event ring");
>  
> +	list_for_each_entry_safe(sec, tmp, &xhci->xhci_sec_list, list) {
> +		list_del(&sec->list);
> +		if (sec->event_ring) {
> +			xhci_ring_free(xhci, sec->event_ring);
> +			xhci_dbg_trace(xhci, trace_xhci_dbg_init,
> +						"Freed secondary ring %d", sec->intr_num);

Odd indentation :(

Re: [RFC PATCH 10/14] sound: usb: card: Check for support for requested audio format

[Greg KH]

On Fri, Dec 23, 2022 at 03:31:56PM -0800, Wesley Cheng wrote:
> Allow for checks on a specific USB audio device to see if a requested PCM
> format is supported.  This is needed for support for when playback is
> initiated by the ASoC USB backend path.
> 
> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
> ---
>  sound/usb/card.c | 19 +++++++++++++++++++
>  sound/usb/card.h |  3 +++
>  2 files changed, 22 insertions(+)
> 
> diff --git a/sound/usb/card.c b/sound/usb/card.c
> index 396e5a34e23b..9b8d2ed308c8 100644
> --- a/sound/usb/card.c
> +++ b/sound/usb/card.c
> @@ -133,6 +133,25 @@ int snd_usb_unregister_vendor_ops(void)
>  }
>  EXPORT_SYMBOL_GPL(snd_usb_unregister_vendor_ops);
>  
> +struct snd_usb_stream *snd_usb_find_suppported_substream(int card_idx,
> +			struct snd_pcm_hw_params *params, int direction)
> +{
> +	struct snd_usb_stream *as;
> +	struct snd_usb_substream *subs = NULL;
> +	const struct audioformat *fmt;
> +
> +	if (usb_chip[card_idx] && enable[card_idx]) {
> +		list_for_each_entry(as, &usb_chip[card_idx]->pcm_list, list) {
> +			subs = &as->substream[direction];
> +			fmt = find_substream_format(subs, params);
> +			if (fmt)
> +				return as;
> +		}
> +	}

Where is the locking here?  How can you walk a list that can be changed
as you walk it?

And what about reference counting?  You are returning a pointer to a
structure, who now "owns" it?  What happens if it is removed from the
system after you return it?

> +	return 0;

Didn't sparse complain about this?  You can't return "0" as a pointer,
it should be NULL.

Always run basic tools like sparse on code before submitting it so that
we don't have to find errors like this.

thanks,

greg k-h

Re: [RFC PATCH 03/14] ASoC: qcom: Add USB backend ASoC driver for Q6

[Greg KH]

On Fri, Dec 23, 2022 at 03:31:49PM -0800, Wesley Cheng wrote:
> Create a USB BE component that will register a new USB port to the ASoC USB
> framework.  This will handle determination on if the requested audio
> profile is supported by the USB device currently selected.
> 
> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
> ---
>  include/sound/q6usboffload.h  |  20 +++
>  sound/soc/qcom/Kconfig        |   4 +
>  sound/soc/qcom/qdsp6/Makefile |   1 +
>  sound/soc/qcom/qdsp6/q6usb.c  | 232 ++++++++++++++++++++++++++++++++++
>  4 files changed, 257 insertions(+)
>  create mode 100644 include/sound/q6usboffload.h
>  create mode 100644 sound/soc/qcom/qdsp6/q6usb.c
> 
> diff --git a/include/sound/q6usboffload.h b/include/sound/q6usboffload.h
> new file mode 100644
> index 000000000000..e576808901d9
> --- /dev/null
> +++ b/include/sound/q6usboffload.h
> @@ -0,0 +1,20 @@
> +/* SPDX-License-Identifier: GPL-2.0
> + *
> + * linux/sound/q6usboffload.h -- QDSP6 USB offload
> + *
> + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
> + */
> +
> +/**
> + * struct q6usb_offload
> + * @dev - dev handle to usb be

"be"?  What is that?

> + * @sid - streamID for iommu
> + * @intr_num - usb interrupter number
> + * @domain - allocated iommu domain
> + **/
> +struct q6usb_offload {
> +	struct device *dev;

Do you properly reference count this?

> +	u32 sid;
> +	u32 intr_num;
> +	struct iommu_domain *domain;
> +};

What is the lifetime of this structure?  Who owns it?  Where is the lock
for accessing it?

> diff --git a/sound/soc/qcom/Kconfig b/sound/soc/qcom/Kconfig
> index 8c7398bc1ca8..d65c365116e5 100644
> --- a/sound/soc/qcom/Kconfig
> +++ b/sound/soc/qcom/Kconfig
> @@ -111,6 +111,9 @@ config SND_SOC_QDSP6_APM
>  config SND_SOC_QDSP6_PRM_LPASS_CLOCKS
>  	tristate
>  
> +config SND_SOC_QDSP6_USB
> +	tristate
> +
>  config SND_SOC_QDSP6_PRM
>  	tristate
>  	select SND_SOC_QDSP6_PRM_LPASS_CLOCKS
> @@ -131,6 +134,7 @@ config SND_SOC_QDSP6
>  	select SND_SOC_TOPOLOGY
>  	select SND_SOC_QDSP6_APM
>  	select SND_SOC_QDSP6_PRM
> +	select SND_SOC_QDSP6_USB
>  	help
>  	 To add support for MSM QDSP6 Soc Audio.
>  	 This will enable sound soc platform specific
> diff --git a/sound/soc/qcom/qdsp6/Makefile b/sound/soc/qcom/qdsp6/Makefile
> index 3963bf234664..c9457ee898d0 100644
> --- a/sound/soc/qcom/qdsp6/Makefile
> +++ b/sound/soc/qcom/qdsp6/Makefile
> @@ -17,3 +17,4 @@ obj-$(CONFIG_SND_SOC_QDSP6_APM_DAI) += q6apm-dai.o
>  obj-$(CONFIG_SND_SOC_QDSP6_APM_LPASS_DAI) += q6apm-lpass-dais.o
>  obj-$(CONFIG_SND_SOC_QDSP6_PRM) += q6prm.o
>  obj-$(CONFIG_SND_SOC_QDSP6_PRM_LPASS_CLOCKS) += q6prm-clocks.o
> +obj-$(CONFIG_SND_SOC_QDSP6_USB) += q6usb.o
> diff --git a/sound/soc/qcom/qdsp6/q6usb.c b/sound/soc/qcom/qdsp6/q6usb.c
> new file mode 100644
> index 000000000000..a9da6dec6c6f
> --- /dev/null
> +++ b/sound/soc/qcom/qdsp6/q6usb.c
> @@ -0,0 +1,232 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.

All of the code in this patch series is older than 2022 as I know it has
been in shipping devices for many years.  Please use the proper
copyright year to make your lawyers happy...

> + */
> +
> +#include <linux/err.h>
> +#include <linux/init.h>
> +#include <linux/module.h>
> +#include <linux/device.h>
> +#include <linux/platform_device.h>
> +#include <linux/slab.h>
> +#include <linux/iommu.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/dma-map-ops.h>
> +
> +#include <sound/pcm.h>
> +#include <sound/soc.h>
> +#include <sound/soc-usb.h>
> +#include <sound/pcm_params.h>
> +#include <sound/asound.h>
> +#include <sound/q6usboffload.h>
> +
> +#include "q6dsp-lpass-ports.h"
> +#include "q6afe.h"
> +
> +struct q6usb_port_data {
> +	struct q6afe_usb_cfg usb_cfg;
> +	struct snd_soc_usb *usb;
> +	struct q6usb_offload priv;
> +	int active_idx;
> +};
> +
> +static const struct snd_soc_dapm_widget q6usb_dai_widgets[] = {
> +	SND_SOC_DAPM_HP("USB_RX_BE", NULL),
> +};
> +
> +static const struct snd_soc_dapm_route q6usb_dapm_routes[] = {
> +	{"USB Playback", NULL, "USB_RX_BE"},
> +};

No terminating entry?  How does this not break?  Why do you need to
specify the size of the array, that feels like a design bug somewhere.


> +
> +static int q6usb_hw_params(struct snd_pcm_substream *substream,
> +			   struct snd_pcm_hw_params *params,
> +			   struct snd_soc_dai *dai)
> +{
> +	return 0;
> +}
> +static const struct snd_soc_dai_ops q6usb_ops = {
> +	.hw_params	= q6usb_hw_params,
> +};
> +
> +static struct snd_soc_dai_driver q6usb_be_dais[] = {
> +	{
> +		.playback = {
> +			.stream_name = "USB BE RX",
> +			.rates = SNDRV_PCM_RATE_8000 | SNDRV_PCM_RATE_11025 |
> +					SNDRV_PCM_RATE_16000 | SNDRV_PCM_RATE_22050 |
> +					SNDRV_PCM_RATE_32000 | SNDRV_PCM_RATE_44100 |
> +					SNDRV_PCM_RATE_48000 | SNDRV_PCM_RATE_96000 |
> +					SNDRV_PCM_RATE_192000,
> +			.formats = SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S16_BE |
> +					SNDRV_PCM_FMTBIT_U16_LE | SNDRV_PCM_FMTBIT_U16_BE |
> +					SNDRV_PCM_FMTBIT_S24_LE | SNDRV_PCM_FMTBIT_S24_BE |
> +					SNDRV_PCM_FMTBIT_U24_LE | SNDRV_PCM_FMTBIT_U24_BE,
> +			.channels_min = 1,
> +			.channels_max = 2,
> +			.rate_max =     192000,
> +			.rate_min =	8000,
> +		},
> +		.id = USB_RX,
> +		.name = "USB_RX_BE",
> +		.ops = &q6usb_ops,
> +	},
> +};
> +
> +int q6usb_audio_ports_of_xlate_dai_name(struct snd_soc_component *component,
> +					const struct of_phandle_args *args,
> +					const char **dai_name)
> +{
> +	int id = args->args[0];
> +	int ret = -EINVAL;
> +	int i;
> +
> +	for (i = 0; i  < ARRAY_SIZE(q6usb_be_dais); i++) {
> +		if (q6usb_be_dais[i].id == id) {
> +			*dai_name = q6usb_be_dais[i].name;
> +			ret = 0;
> +			break;
> +		}
> +	}
> +
> +	return ret;
> +}
> +
> +static int q6usb_component_probe(struct snd_soc_component *component)
> +{
> +	struct q6usb_port_data *data = dev_get_drvdata(component->dev);
> +	struct snd_soc_dapm_context *dapm = snd_soc_component_get_dapm(component);
> +
> +	data->usb->component = component;
> +
> +	snd_soc_dapm_disable_pin(dapm, "USB_RX_BE");
> +	snd_soc_dapm_sync(dapm);
> +
> +	return 0;
> +}
> +
> +static const struct snd_soc_component_driver q6usb_dai_component = {
> +	.probe		= q6usb_component_probe,
> +	.name		= "q6usb-dai-component",
> +	.dapm_widgets = q6usb_dai_widgets,
> +	.num_dapm_widgets = ARRAY_SIZE(q6usb_dai_widgets),
> +	.dapm_routes = q6usb_dapm_routes,
> +	.num_dapm_routes = ARRAY_SIZE(q6usb_dapm_routes),
> +	.of_xlate_dai_name = q6usb_audio_ports_of_xlate_dai_name,
> +};
> +
> +int q6usb_alsa_connection_cb(struct snd_soc_usb *usb, int card_idx,
> +			int connected)
> +{
> +	struct snd_soc_dapm_context *dapm;
> +	struct q6usb_port_data *data;
> +
> +	if (!usb->component)
> +		return 0;

How can this happen?

Why is this not an error?

> +
> +	dapm = snd_soc_component_get_dapm(usb->component);
> +	data = dev_get_drvdata(usb->component->dev);
> +
> +	if (connected) {
> +		snd_soc_dapm_enable_pin(dapm, "USB_RX_BE");
> +		/* We only track the latest USB headset plugged in */
> +		data->active_idx = card_idx;
> +	} else {
> +		snd_soc_dapm_disable_pin(dapm, "USB_RX_BE");
> +	}
> +	snd_soc_dapm_sync(dapm);
> +
> +	return 0;
> +}
> +
> +static int q6usb_dai_dev_probe(struct platform_device *pdev)
> +{
> +	struct device_node *node = pdev->dev.of_node;
> +	struct q6usb_port_data *data;
> +	struct device *dev = &pdev->dev;
> +	int ret;
> +
> +	data = devm_kzalloc(dev, sizeof(*data), GFP_KERNEL);
> +	if (!data)
> +		return -ENOMEM;
> +
> +	ret = of_property_read_u32(node, "qcom,usb-audio-stream-id",
> +				&data->priv.sid);
> +	if (ret) {
> +		dev_err(&pdev->dev, "failed to read sid.\n");
> +		return -ENODEV;
> +	}
> +
> +	ret = of_property_read_u32(node, "qcom,usb-audio-intr-num",
> +				&data->priv.intr_num);
> +	if (ret) {
> +		dev_err(&pdev->dev, "failed to read intr num.\n");
> +		return -ENODEV;
> +	}
> +
> +	data->priv.domain = iommu_domain_alloc(pdev->dev.bus);
> +	if (!data->priv.domain) {
> +		dev_err(&pdev->dev, "failed to allocate iommu domain\n");
> +		return -ENODEV;
> +	}
> +
> +	/* attach to external processor iommu */
> +	ret = iommu_attach_device(data->priv.domain, &pdev->dev);
> +	if (ret) {
> +		dev_err(&pdev->dev, "failed to attach device ret = %d\n", ret);
> +		goto free_domain;
> +	}
> +
> +	data->usb = snd_soc_usb_add_port(dev, q6usb_alsa_connection_cb);
> +	if (IS_ERR(data->usb)) {
> +		dev_err(&pdev->dev, "failed to add usb port\n");
> +		goto detach_device;
> +	}
> +
> +	data->priv.dev = dev;
> +	dev_set_drvdata(dev, data);
> +	devm_snd_soc_register_component(dev, &q6usb_dai_component,
> +							q6usb_be_dais, ARRAY_SIZE(q6usb_be_dais));

Very odd indentation.  Please do this properly everywhere.


> +	snd_soc_usb_set_priv_data(&data->priv);
> +
> +	return 0;
> +
> +detach_device:
> +	iommu_detach_device(data->priv.domain, &pdev->dev);
> +free_domain:
> +	iommu_domain_free(data->priv.domain);
> +
> +	return ret;
> +}
> +
> +static int q6usb_dai_dev_remove(struct platform_device *pdev)
> +{
> +	struct q6usb_port_data *data = platform_get_drvdata(pdev);
> +
> +	iommu_detach_device(data->priv.domain, &pdev->dev);
> +	iommu_domain_free(data->priv.domain);
> +
> +	snd_soc_usb_remove_port();
> +
> +	return 0;
> +}
> +
> +#ifdef CONFIG_OF

Is this really needed still?

thanks,

greg k-h

Re: [RFC PATCH 04/14] sound: usb: card: Introduce USB SND vendor op callbacks

[Dmitry Baryshkov]

Hi,

On Sat, 24 Dec 2022 at 01:33, Wesley Cheng <quic_wcheng@quicinc.com> wrote:
>
> Allow for different vendors to be notified on USB SND connect/disconnect
> seqeunces.  This allows for vendor USB SND modules to properly initialize
> and populate internal structures with references to the USB SND chip
> device.

The commit message definitely needs some improvement. We do not notify
vendors on SND connect/disconnect events.


>
> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
> ---
>  sound/usb/card.c | 22 ++++++++++++++++++++++
>  sound/usb/card.h |  7 +++++++
>  2 files changed, 29 insertions(+)
>
> diff --git a/sound/usb/card.c b/sound/usb/card.c
> index 26268ffb8274..212f55a7683c 100644
> --- a/sound/usb/card.c
> +++ b/sound/usb/card.c
> @@ -117,6 +117,21 @@ MODULE_PARM_DESC(skip_validation, "Skip unit descriptor validation (default: no)
>  static DEFINE_MUTEX(register_mutex);
>  static struct snd_usb_audio *usb_chip[SNDRV_CARDS];
>  static struct usb_driver usb_audio_driver;
> +static struct snd_usb_vendor_ops *vendor_ops;
> +
> +int snd_usb_register_vendor_ops(struct snd_usb_vendor_ops *ops)

platform ops?

> +{
> +       vendor_ops = ops;
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(snd_usb_register_vendor_ops);

What happens if several platforms try to register different ops? I saw
from the patch 09/14 that you register these ops unconditionally. If
other devices follow your approach there is an obvious conflict.

> +
> +int snd_usb_unregister_vendor_ops(void)
> +{
> +       vendor_ops = NULL;
> +       return 0;
> +}
> +EXPORT_SYMBOL_GPL(snd_usb_unregister_vendor_ops);
>
>  /*
>   * disconnect streams
> @@ -910,6 +925,10 @@ static int usb_audio_probe(struct usb_interface *intf,
>         usb_set_intfdata(intf, chip);
>         atomic_dec(&chip->active);
>         mutex_unlock(&register_mutex);
> +
> +       if (vendor_ops->connect_cb)
> +               vendor_ops->connect_cb(intf, chip);
> +
>         return 0;
>
>   __error:
> @@ -943,6 +962,9 @@ static void usb_audio_disconnect(struct usb_interface *intf)
>         if (chip == USB_AUDIO_IFACE_UNUSED)
>                 return;
>
> +       if (vendor_ops->disconnect_cb)
> +               vendor_ops->disconnect_cb(intf);
> +
>         card = chip->card;
>
>         mutex_lock(&register_mutex);
> diff --git a/sound/usb/card.h b/sound/usb/card.h
> index 40061550105a..a785bb256b0d 100644
> --- a/sound/usb/card.h
> +++ b/sound/usb/card.h
> @@ -206,4 +206,11 @@ struct snd_usb_stream {
>         struct list_head list;
>  };
>
> +struct snd_usb_vendor_ops {
> +       void (*connect_cb)(struct usb_interface *intf, struct snd_usb_audio *chip);
> +       void (*disconnect_cb)(struct usb_interface *intf);
> +};
> +
> +int snd_usb_register_vendor_ops(struct snd_usb_vendor_ops *ops);
> +int snd_usb_unregister_vendor_ops(void);
>  #endif /* __USBAUDIO_CARD_H */



--
With best wishes

Dmitry

Re: [RFC PATCH 08/14] usb: dwc3: Add DT parameter to specify maximum number of interrupters

[Dmitry Baryshkov]

On Sat, 24 Dec 2022 at 01:33, Wesley Cheng <quic_wcheng@quicinc.com> wrote:
>
> Allow for the DWC3 host driver to pass along a XHCI property that defines
> how many interrupters to allocate.  This is in relation for the number of
> event rings that can be potentially used by other processors within the
> system.
>
> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
> ---
>  drivers/usb/dwc3/core.c | 12 ++++++++++++
>  drivers/usb/dwc3/core.h |  2 ++
>  drivers/usb/dwc3/host.c |  5 ++++-
>  3 files changed, 18 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c
> index 476b63618511..67d6f0ae81d2 100644
> --- a/drivers/usb/dwc3/core.c
> +++ b/drivers/usb/dwc3/core.c
> @@ -1446,6 +1446,7 @@ static void dwc3_get_properties(struct dwc3 *dwc)
>         u8                      tx_thr_num_pkt_prd = 0;
>         u8                      tx_max_burst_prd = 0;
>         u8                      tx_fifo_resize_max_num;
> +       u8                      num_hc_interrupters;
>         const char              *usb_psy_name;
>         int                     ret;
>
> @@ -1468,6 +1469,9 @@ static void dwc3_get_properties(struct dwc3 *dwc)
>          */
>         tx_fifo_resize_max_num = 6;
>
> +       /* default to a single XHCI interrupter */
> +       num_hc_interrupters = 1;
> +
>         dwc->maximum_speed = usb_get_maximum_speed(dev);
>         dwc->max_ssp_rate = usb_get_maximum_ssp_rate(dev);
>         dwc->dr_mode = usb_get_dr_mode(dev);
> @@ -1511,6 +1515,12 @@ static void dwc3_get_properties(struct dwc3 *dwc)
>                                 &tx_thr_num_pkt_prd);
>         device_property_read_u8(dev, "snps,tx-max-burst-prd",
>                                 &tx_max_burst_prd);
> +       device_property_read_u8(dev, "snps,num-hc-interrupters",
> +                               &num_hc_interrupters);

bindings change?

> +       /* DWC3 core allowed to have a max of 8 interrupters */
> +       if (num_hc_interrupters > 8)
> +               num_hc_interrupters = 8;
> +
>         dwc->do_fifo_resize = device_property_read_bool(dev,
>                                                         "tx-fifo-resize");
>         if (dwc->do_fifo_resize)
> @@ -1589,6 +1599,8 @@ static void dwc3_get_properties(struct dwc3 *dwc)
>         dwc->imod_interval = 0;
>
>         dwc->tx_fifo_resize_max_num = tx_fifo_resize_max_num;
> +
> +       dwc->num_hc_interrupters = num_hc_interrupters;
>  }
>
>  /* check whether the core supports IMOD */
> diff --git a/drivers/usb/dwc3/core.h b/drivers/usb/dwc3/core.h
> index 8f9959ba9fd4..09037299da53 100644
> --- a/drivers/usb/dwc3/core.h
> +++ b/drivers/usb/dwc3/core.h
> @@ -1050,6 +1050,7 @@ struct dwc3_scratchpad_array {
>   * @tx_max_burst_prd: max periodic ESS transmit burst size
>   * @tx_fifo_resize_max_num: max number of fifos allocated during txfifo resize
>   * @clear_stall_protocol: endpoint number that requires a delayed status phase
> + * @num_hc_interrupters: number of host controller interrupters
>   * @hsphy_interface: "utmi" or "ulpi"
>   * @connected: true when we're connected to a host, false otherwise
>   * @softconnect: true when gadget connect is called, false when disconnect runs
> @@ -1275,6 +1276,7 @@ struct dwc3 {
>         u8                      tx_max_burst_prd;
>         u8                      tx_fifo_resize_max_num;
>         u8                      clear_stall_protocol;
> +       u8                      num_hc_interrupters;
>
>         const char              *hsphy_interface;
>
> diff --git a/drivers/usb/dwc3/host.c b/drivers/usb/dwc3/host.c
> index f6f13e7f1ba1..52a284fdd704 100644
> --- a/drivers/usb/dwc3/host.c
> +++ b/drivers/usb/dwc3/host.c
> @@ -66,7 +66,7 @@ static int dwc3_host_get_irq(struct dwc3 *dwc)
>
>  int dwc3_host_init(struct dwc3 *dwc)
>  {
> -       struct property_entry   props[4];
> +       struct property_entry   props[5];
>         struct platform_device  *xhci;
>         int                     ret, irq;
>         int                     prop_idx = 0;
> @@ -112,6 +112,9 @@ int dwc3_host_init(struct dwc3 *dwc)
>         if (DWC3_VER_IS_WITHIN(DWC3, ANY, 300A))
>                 props[prop_idx++] = PROPERTY_ENTRY_BOOL("quirk-broken-port-ped");
>
> +       props[prop_idx++] = PROPERTY_ENTRY_U8("num-hc-interrupters",
> +                                                               dwc->num_hc_interrupters);
> +
>         if (prop_idx) {
>                 ret = device_create_managed_software_node(&xhci->dev, props, NULL);
>                 if (ret) {



-- 
With best wishes
Dmitry

Re: [RFC PATCH 06/14] usb: core: hcd: Introduce USB HCD APIs for interrupter management

[Alan Stern]

On Fri, Dec 23, 2022 at 03:31:52PM -0800, Wesley Cheng wrote:
> For USB HCDs that can support multiple USB interrupters, expose functions
> that class drivers can utilize for setting up secondary interrupters.
> Class drivers can pass this information to its respective clients, i.e.
> a dedicated DSP.
> 
> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
> ---
>  drivers/usb/core/hcd.c  | 86 +++++++++++++++++++++++++++++++++++++++++
>  include/linux/usb.h     |  7 ++++
>  include/linux/usb/hcd.h | 16 +++++++-
>  3 files changed, 108 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/usb/core/hcd.c b/drivers/usb/core/hcd.c
> index 8300baedafd2..90ead90faf1d 100644
> --- a/drivers/usb/core/hcd.c
> +++ b/drivers/usb/core/hcd.c

> +/**
> + * usb_hcd_stop_endpoint - Halt USB EP transfers
> + * @udev: usb device
> + * @ep: usb ep to stop
> + *
> + * Stop pending transfers on a specific USB endpoint.
> + **/
> +int usb_hcd_stop_endpoint(struct usb_device *udev,
> +					struct usb_host_endpoint *ep)
> +{
> +	struct usb_hcd *hcd = bus_to_hcd(udev->bus);
> +	int ret = 0;
> +
> +	if (hcd->driver->stop_endpoint)
> +		ret = hcd->driver->stop_endpoint(hcd, udev, ep);
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(usb_hcd_stop_endpoint);

You know, there already is a function that does this.  It's named 
usb_hcd_flush_endpoint().  No need to add another function that does the 
same thing.

Alan Stern

Re: [RFC PATCH 13/14] ASoC: dt-bindings: Add Q6USB backend bindings

[Krzysztof Kozlowski]

On 24/12/2022 00:31, Wesley Cheng wrote:
> Add a dt-binding to describe the definition of enabling the Q6 USB backend
> device for audio offloading.  The node carries information, which is passed
> along to the QC USB SND class driver counterpart.  These parameters will be
> utilized during QMI stream enable requests.
> 
> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
> ---
>  .../bindings/sound/qcom,q6usb-dais.yaml       | 55 +++++++++++++++++++
>  1 file changed, 55 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml
> 
> diff --git a/Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml b/Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml
> new file mode 100644
> index 000000000000..e24b4d52fa7e
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml
> @@ -0,0 +1,55 @@
> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/sound/qcom,q6usb-dais.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Qualcomm ASoC USB backend DAI

What is "ASoC"? Does not look like Qualcomm name for hardware, but
rather Linux, so should be dropped or changed to SoC or ADSP etc.

> +
> +maintainers:
> +  - Wesley Cheng <quic_wcheng@quicinc.com>
> +
> +description:
> +  The Q6USB backend is a supported AFE port on the Q6DSP. This backend

What does "supported AFE port" mean? Can an AFE port be unsupported?

> +  driver will communicate the required settings to the QC USB SND class

If you mean Linux driver, then please drop entirely all references to
drivers. In other case, please explain more.

> +  driver for properly enabling the audio stream.  Parameters defined
> +  under this node will carry settings, which will be passed along during
> +  the QMI stream enable request.
> +
> +properties:
> +  compatible:
> +    enum:
> +      - qcom,q6usb-dais
> +
> +  iommus:
> +    maxItems: 1
> +
> +  "#sound-dai-cells":
> +    const: 1
> +
> +  qcom,usb-audio-stream-id:
> +    description:
> +      SID for the Q6DSP processor for IOMMU mapping.

Why this is not part of regular iommus property? It's the first time
something like this appears in Qualcomm hardware, so I wonder why this
DAI is special?

> +    $ref: /schemas/types.yaml#/definitions/uint32
> +
> +  qcom,usb-audio-intr-num:
> +    description:
> +      Desired XHCI interrupter number to use.

What does it mean? Interrupts are defined with "interrupts" property, so
you need to explain more.

> +    $ref: /schemas/types.yaml#/definitions/uint32
> +
> +required:
> +  - compatible
> +  - '#sound-dai-cells'

Keep consistent quotes - either ' or "

> +  - qcom,usb-audio-intr-num
> +
> +additionalProperties: false
> +
> +examples:
> +  - |
> +    usbdai: usbd {

Generic node names, so dai or dais


> +      compatible = "qcom,q6usb-dais";
> +      #sound-dai-cells = <1>;
> +      iommus = <&apps_smmu 0x180f 0x0>;
> +      qcom,usb-audio-stream-id = <0xf>;
> +      qcom,usb-audio-intr-num = <2>;
> +    };

Best regards,
Krzysztof

Re: [RFC PATCH 14/14] ASoC: dt-bindings: Update example for enabling USB offload on SM8250

[Krzysztof Kozlowski]

On 24/12/2022 00:32, Wesley Cheng wrote:
> Add an example on enabling of USB offload for the Q6DSP.  The routing can
> be done by the mixer, which can pass the multimedia stream to the USB
> backend.
> 
> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
> ---
>  .../devicetree/bindings/sound/qcom,sm8250.yaml      | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
> 
> diff --git a/Documentation/devicetree/bindings/sound/qcom,sm8250.yaml b/Documentation/devicetree/bindings/sound/qcom,sm8250.yaml
> index 70080d04ddc9..60cd84e6727a 100644
> --- a/Documentation/devicetree/bindings/sound/qcom,sm8250.yaml
> +++ b/Documentation/devicetree/bindings/sound/qcom,sm8250.yaml
> @@ -216,6 +216,19 @@ examples:
>                  sound-dai = <&vamacro 0>;
>              };
>          };
> +
> +        usb-dai-link {
> +            link-name = "USB Playback";
> +            cpu {
> +                sound-dai = <&q6afedai USB_RX>;

Hmm, that makes me wonder if you really tested the bindings before
sending? If yes, where is the USB_RX defined?

Best regards,
Krzysztof

Re: [RFC PATCH 08/14] usb: dwc3: Add DT parameter to specify maximum number of interrupters

[Krzysztof Kozlowski]

On 24/12/2022 12:13, Dmitry Baryshkov wrote:
>> @@ -1468,6 +1469,9 @@ static void dwc3_get_properties(struct dwc3 *dwc)
>>          */
>>         tx_fifo_resize_max_num = 6;
>>
>> +       /* default to a single XHCI interrupter */
>> +       num_hc_interrupters = 1;
>> +
>>         dwc->maximum_speed = usb_get_maximum_speed(dev);
>>         dwc->max_ssp_rate = usb_get_maximum_ssp_rate(dev);
>>         dwc->dr_mode = usb_get_dr_mode(dev);
>> @@ -1511,6 +1515,12 @@ static void dwc3_get_properties(struct dwc3 *dwc)
>>                                 &tx_thr_num_pkt_prd);
>>         device_property_read_u8(dev, "snps,tx-max-burst-prd",
>>                                 &tx_max_burst_prd);
>> +       device_property_read_u8(dev, "snps,num-hc-interrupters",
>> +                               &num_hc_interrupters);
> 
> bindings change?

Undocumented bindings change :(

Best regards,
Krzysztof

Re: [RFC PATCH 03/14] ASoC: qcom: Add USB backend ASoC driver for Q6

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1292 bytes --]

On Sat, Dec 24, 2022 at 10:02:59AM +0100, Greg KH wrote:
> On Fri, Dec 23, 2022 at 03:31:49PM -0800, Wesley Cheng wrote:

> > + * struct q6usb_offload
> > + * @dev - dev handle to usb be

> "be"?  What is that?

Back end.  This is a concept in DPCM which should be reasonably
discoverable to people working on the audio portions of this code.

> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.

> All of the code in this patch series is older than 2022 as I know it has
> been in shipping devices for many years.  Please use the proper
> copyright year to make your lawyers happy...

Are you *positive* about this.  Based on some preparatory discussions
the Qualcomm people had with Takashi and I it seemed like this was a new
version of existing concepts.  I'm sure they had something already but
it's not obvious to me that they're just posting the same code.

> > +static const struct snd_soc_dapm_route q6usb_dapm_routes[] = {
> > +	{"USB Playback", NULL, "USB_RX_BE"},
> > +};

> No terminating entry?  How does this not break?  Why do you need to
> specify the size of the array, that feels like a design bug somewhere.

It's how the interface works, the number of entries is passed in when
adding routes.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [RFC PATCH 03/14] ASoC: qcom: Add USB backend ASoC driver for Q6

[Greg KH]

On Tue, Dec 27, 2022 at 01:04:55PM +0000, Mark Brown wrote:
> On Sat, Dec 24, 2022 at 10:02:59AM +0100, Greg KH wrote:
> > On Fri, Dec 23, 2022 at 03:31:49PM -0800, Wesley Cheng wrote:
> 
> > > + * struct q6usb_offload
> > > + * @dev - dev handle to usb be
> 
> > "be"?  What is that?
> 
> Back end.  This is a concept in DPCM which should be reasonably
> discoverable to people working on the audio portions of this code.

Ok, then how is the reference counting logic handled here?  USB devices
can be removed from the system at any point in time...

> > > +// SPDX-License-Identifier: GPL-2.0
> > > +/*
> > > + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
> 
> > All of the code in this patch series is older than 2022 as I know it has
> > been in shipping devices for many years.  Please use the proper
> > copyright year to make your lawyers happy...
> 
> Are you *positive* about this.  Based on some preparatory discussions
> the Qualcomm people had with Takashi and I it seemed like this was a new
> version of existing concepts.  I'm sure they had something already but
> it's not obvious to me that they're just posting the same code.

I thought that this same code has been shipping in devices for a few
years now in the last few Samsung phone models.  Is this not the same
code that is in those devices?

If not, why not, what happened to that codebase that makes it not worthy
of being submitted upstream?

> > > +static const struct snd_soc_dapm_route q6usb_dapm_routes[] = {
> > > +	{"USB Playback", NULL, "USB_RX_BE"},
> > > +};
> 
> > No terminating entry?  How does this not break?  Why do you need to
> > specify the size of the array, that feels like a design bug somewhere.
> 
> It's how the interface works, the number of entries is passed in when
> adding routes.

Ah, you all might want to change that to make it simpler :)

thanks,

greg k-h

Re: [RFC PATCH 03/14] ASoC: qcom: Add USB backend ASoC driver for Q6

[Takashi Iwai]

On Tue, 27 Dec 2022 14:45:13 +0100,
Greg KH wrote:
> 
> On Tue, Dec 27, 2022 at 01:04:55PM +0000, Mark Brown wrote:
> > On Sat, Dec 24, 2022 at 10:02:59AM +0100, Greg KH wrote:
> > > On Fri, Dec 23, 2022 at 03:31:49PM -0800, Wesley Cheng wrote:
> > 
> > > > + * struct q6usb_offload
> > > > + * @dev - dev handle to usb be
> > 
> > > "be"?  What is that?
> > 
> > Back end.  This is a concept in DPCM which should be reasonably
> > discoverable to people working on the audio portions of this code.
> 
> Ok, then how is the reference counting logic handled here?  USB devices
> can be removed from the system at any point in time...

The whole picture is fairly complex, and this patch is a part
belonging to the ASoC machine driver -- that is, it's bound to the
Qualcomm host, and there can be only one on a system.  

OTOH, USB audio devices are still managed by the existing USB audio
driver as is, and they can be multiple and any devices.  The basic
idea here is a hijack of the USB data processing in USB audio driver
with the offloading mechanism by this ASoC driver (only if the
condition met).


thanks,

Takashi

Re: [RFC PATCH 03/14] ASoC: qcom: Add USB backend ASoC driver for Q6

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1164 bytes --]

On Tue, Dec 27, 2022 at 03:02:31PM +0100, Takashi Iwai wrote:
> Greg KH wrote:
> > On Tue, Dec 27, 2022 at 01:04:55PM +0000, Mark Brown wrote:
> > > On Sat, Dec 24, 2022 at 10:02:59AM +0100, Greg KH wrote:

> > > > "be"?  What is that?

> > > Back end.  This is a concept in DPCM which should be reasonably
> > > discoverable to people working on the audio portions of this code.

> > Ok, then how is the reference counting logic handled here?  USB devices
> > can be removed from the system at any point in time...

> The whole picture is fairly complex, and this patch is a part
> belonging to the ASoC machine driver -- that is, it's bound to the
> Qualcomm host, and there can be only one on a system.  

> OTOH, USB audio devices are still managed by the existing USB audio
> driver as is, and they can be multiple and any devices.  The basic
> idea here is a hijack of the USB data processing in USB audio driver
> with the offloading mechanism by this ASoC driver (only if the
> condition met).

Right.  I haven't even begun to look at the actual code here, just
triaging my inbox, so I've got no thoughts on if things work or not.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [RFC PATCH 00/14] Introduce QC USB SND audio offloading support

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1356 bytes --]

On Sat, Dec 24, 2022 at 07:45:38AM +0100, Greg KH wrote:
> On Fri, Dec 23, 2022 at 03:31:46PM -0800, Wesley Cheng wrote:

> > soc-usb: Intention is to treat a USB port similar to a headphone jack.
> > The port is always present on the device, but cable/pin status can be
> > enabled/disabled.  Expose mechanisms for USB backend ASoC drivers to
> > communicate with USB SND.

> > Create a USB backend for Q6DSP:
> > q6usb: Basic backend driver that will be responsible for maintaining the
> > resources needed to initiate a playback stream using the Q6DSP.  Will

> This looks to duplicate a bunch of the same things that a number of
> different google developers have posted recently.  Please work with them
> to come up with a unified set of patches that you all can agree with,
> AND get them to sign off on the changes before resubmitting them.

> This uncoordinated drip of patches from different people doing the same
> thing is almost impossible to review from our side, as I'm sure you can
> imagine.

I have to say this is the first I've heard of any such patches other
than from the Qualcomm people and I can't immediately see anything that
was on the list either, though I might be missing something since I
don't have the subject or anything.  If other people send things again
it's probably good to suggest they copy in audio people and lists.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [RFC PATCH 03/14] ASoC: qcom: Add USB backend ASoC driver for Q6

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1628 bytes --]

On Tue, Dec 27, 2022 at 02:45:13PM +0100, Greg KH wrote:
> On Tue, Dec 27, 2022 at 01:04:55PM +0000, Mark Brown wrote:
> > On Sat, Dec 24, 2022 at 10:02:59AM +0100, Greg KH wrote:

> > > > + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.

> > > All of the code in this patch series is older than 2022 as I know it has
> > > been in shipping devices for many years.  Please use the proper
> > > copyright year to make your lawyers happy...

> > Are you *positive* about this.  Based on some preparatory discussions
> > the Qualcomm people had with Takashi and I it seemed like this was a new
> > version of existing concepts.  I'm sure they had something already but
> > it's not obvious to me that they're just posting the same code.

> I thought that this same code has been shipping in devices for a few
> years now in the last few Samsung phone models.  Is this not the same
> code that is in those devices?

> If not, why not, what happened to that codebase that makes it not worthy
> of being submitted upstream?

I don't specifically know anything about that code but I'd expect that
for out of tree code breaking new ground like this there'd be a strong
likelyhood that there'd be design level issues and that's what the pre
submission discussions were all about - how to fit the concept into the
kernel subsystems in a way that might be maintainable.  There's also
been the whole transition to their new DSP firmware going on.  It's
possible that what was shipped was implemented in the same way with the
same code but I'd not assume that this is the case without actually
comparing the two.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [RFC PATCH 03/14] ASoC: qcom: Add USB backend ASoC driver for Q6

[Wesley Cheng]

Hi Mark/Greg,

On 12/27/2022 7:11 AM, Mark Brown wrote:
> On Tue, Dec 27, 2022 at 02:45:13PM +0100, Greg KH wrote:
>> On Tue, Dec 27, 2022 at 01:04:55PM +0000, Mark Brown wrote:
>>> On Sat, Dec 24, 2022 at 10:02:59AM +0100, Greg KH wrote:
> 
>>>>> + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
> 
>>>> All of the code in this patch series is older than 2022 as I know it has
>>>> been in shipping devices for many years.  Please use the proper
>>>> copyright year to make your lawyers happy...
> 
>>> Are you *positive* about this.  Based on some preparatory discussions
>>> the Qualcomm people had with Takashi and I it seemed like this was a new
>>> version of existing concepts.  I'm sure they had something already but
>>> it's not obvious to me that they're just posting the same code.
> 
>> I thought that this same code has been shipping in devices for a few
>> years now in the last few Samsung phone models.  Is this not the same
>> code that is in those devices?
> 
>> If not, why not, what happened to that codebase that makes it not worthy
>> of being submitted upstream?
> 
> I don't specifically know anything about that code but I'd expect that
> for out of tree code breaking new ground like this there'd be a strong
> likelyhood that there'd be design level issues and that's what the pre
> submission discussions were all about - how to fit the concept into the
> kernel subsystems in a way that might be maintainable.  There's also
> been the whole transition to their new DSP firmware going on.  It's
> possible that what was shipped was implemented in the same way with the
> same code but I'd not assume that this is the case without actually
> comparing the two.

It's correct that all the ASoC related patches are new, and didn't exist 
previously in QC products.  It is due to the fact that Android has a 
different ALSA userspace concept which allowed for a lot of the Q6 audio 
front end (AFE) communication to be done from userspace, I believe. 
This is the reason that we had to introduce this new ASoC based design.

I can't comment much more about the Android ALSA design, maybe @Patrick 
Lai can.

Thanks
Wesley Cheng

Re: [RFC PATCH 08/14] usb: dwc3: Add DT parameter to specify maximum number of interrupters

[Wesley Cheng]

Hi Dmitry,

On 12/24/2022 3:13 AM, Dmitry Baryshkov wrote:
> On Sat, 24 Dec 2022 at 01:33, Wesley Cheng <quic_wcheng@quicinc.com> wrote:
>>
>> Allow for the DWC3 host driver to pass along a XHCI property that defines
>> how many interrupters to allocate.  This is in relation for the number of
>> event rings that can be potentially used by other processors within the
>> system.
>>
>> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
>> ---
>>   drivers/usb/dwc3/core.c | 12 ++++++++++++
>>   drivers/usb/dwc3/core.h |  2 ++
>>   drivers/usb/dwc3/host.c |  5 ++++-
>>   3 files changed, 18 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c
>> index 476b63618511..67d6f0ae81d2 100644
>> --- a/drivers/usb/dwc3/core.c
>> +++ b/drivers/usb/dwc3/core.c
>> @@ -1446,6 +1446,7 @@ static void dwc3_get_properties(struct dwc3 *dwc)
>>          u8                      tx_thr_num_pkt_prd = 0;
>>          u8                      tx_max_burst_prd = 0;
>>          u8                      tx_fifo_resize_max_num;
>> +       u8                      num_hc_interrupters;
>>          const char              *usb_psy_name;
>>          int                     ret;
>>
>> @@ -1468,6 +1469,9 @@ static void dwc3_get_properties(struct dwc3 *dwc)
>>           */
>>          tx_fifo_resize_max_num = 6;
>>
>> +       /* default to a single XHCI interrupter */
>> +       num_hc_interrupters = 1;
>> +
>>          dwc->maximum_speed = usb_get_maximum_speed(dev);
>>          dwc->max_ssp_rate = usb_get_maximum_ssp_rate(dev);
>>          dwc->dr_mode = usb_get_dr_mode(dev);
>> @@ -1511,6 +1515,12 @@ static void dwc3_get_properties(struct dwc3 *dwc)
>>                                  &tx_thr_num_pkt_prd);
>>          device_property_read_u8(dev, "snps,tx-max-burst-prd",
>>                                  &tx_max_burst_prd);
>> +       device_property_read_u8(dev, "snps,num-hc-interrupters",
>> +                               &num_hc_interrupters);
> 
> bindings change?
> 

Will add one.  Thanks!

Thanks
Wesley Cheng

Re: [RFC PATCH 04/14] sound: usb: card: Introduce USB SND vendor op callbacks

[Wesley Cheng]

Hi Dmitry,

On 12/24/2022 3:03 AM, Dmitry Baryshkov wrote:
> Hi,
> 
> On Sat, 24 Dec 2022 at 01:33, Wesley Cheng <quic_wcheng@quicinc.com> wrote:
>>
>> Allow for different vendors to be notified on USB SND connect/disconnect
>> seqeunces.  This allows for vendor USB SND modules to properly initialize
>> and populate internal structures with references to the USB SND chip
>> device.
> 
> The commit message definitely needs some improvement. We do not notify
> vendors on SND connect/disconnect events.
> 
> 
>>
>> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
>> ---
>>   sound/usb/card.c | 22 ++++++++++++++++++++++
>>   sound/usb/card.h |  7 +++++++
>>   2 files changed, 29 insertions(+)
>>
>> diff --git a/sound/usb/card.c b/sound/usb/card.c
>> index 26268ffb8274..212f55a7683c 100644
>> --- a/sound/usb/card.c
>> +++ b/sound/usb/card.c
>> @@ -117,6 +117,21 @@ MODULE_PARM_DESC(skip_validation, "Skip unit descriptor validation (default: no)
>>   static DEFINE_MUTEX(register_mutex);
>>   static struct snd_usb_audio *usb_chip[SNDRV_CARDS];
>>   static struct usb_driver usb_audio_driver;
>> +static struct snd_usb_vendor_ops *vendor_ops;
>> +
>> +int snd_usb_register_vendor_ops(struct snd_usb_vendor_ops *ops)
> 
> platform ops?
> 

Will change it.

>> +{
>> +       vendor_ops = ops;
>> +       return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(snd_usb_register_vendor_ops);
> 
> What happens if several platforms try to register different ops? I saw
> from the patch 09/14 that you register these ops unconditionally. If
> other devices follow your approach there is an obvious conflict.
> 

Thank you for the review.

That is true.  I don't think there is a proper need to have multiple 
vendor ops being registered, so maybe just returning an error for if ops 
are already registered is sufficient.

Thanks
Wesley Cheng

Re: [RFC PATCH 03/14] ASoC: qcom: Add USB backend ASoC driver for Q6

[Wesley Cheng]

Hi Greg,

On 12/24/2022 1:02 AM, Greg KH wrote:
> On Fri, Dec 23, 2022 at 03:31:49PM -0800, Wesley Cheng wrote:
>> Create a USB BE component that will register a new USB port to the ASoC USB
>> framework.  This will handle determination on if the requested audio
>> profile is supported by the USB device currently selected.
>>
>> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
>> ---
>>   include/sound/q6usboffload.h  |  20 +++
>>   sound/soc/qcom/Kconfig        |   4 +
>>   sound/soc/qcom/qdsp6/Makefile |   1 +
>>   sound/soc/qcom/qdsp6/q6usb.c  | 232 ++++++++++++++++++++++++++++++++++
>>   4 files changed, 257 insertions(+)
>>   create mode 100644 include/sound/q6usboffload.h
>>   create mode 100644 sound/soc/qcom/qdsp6/q6usb.c
>>
>> diff --git a/include/sound/q6usboffload.h b/include/sound/q6usboffload.h
>> new file mode 100644
>> index 000000000000..e576808901d9
>> --- /dev/null
>> +++ b/include/sound/q6usboffload.h
>> @@ -0,0 +1,20 @@
>> +/* SPDX-License-Identifier: GPL-2.0
>> + *
>> + * linux/sound/q6usboffload.h -- QDSP6 USB offload
>> + *
>> + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
>> + */
>> +
>> +/**
>> + * struct q6usb_offload
>> + * @dev - dev handle to usb be
> 
> "be"?  What is that?
> 
>> + * @sid - streamID for iommu
>> + * @intr_num - usb interrupter number
>> + * @domain - allocated iommu domain
>> + **/
>> +struct q6usb_offload {
>> +	struct device *dev;
> 
> Do you properly reference count this?
> 
>> +	u32 sid;
>> +	u32 intr_num;
>> +	struct iommu_domain *domain;
>> +};
> 
> What is the lifetime of this structure?  Who owns it?  Where is the lock
> for accessing it?
> 

Owner of this structure is the USB backend.  If the USB backend is 
removed, then the qc_audio_offload driver would never receive the QMI 
request to enable the audio stream path from the audio dsp.  (where this 
is referenced) It will exist as long as the USB BE device exists.

One thing I will need to follow up on is if an ASoC backend device is 
removed while an audio playback is happening how it would handle it.

>> diff --git a/sound/soc/qcom/Kconfig b/sound/soc/qcom/Kconfig
>> index 8c7398bc1ca8..d65c365116e5 100644
>> --- a/sound/soc/qcom/Kconfig
>> +++ b/sound/soc/qcom/Kconfig
>> @@ -111,6 +111,9 @@ config SND_SOC_QDSP6_APM
>>   config SND_SOC_QDSP6_PRM_LPASS_CLOCKS
>>   	tristate
>>   
>> +config SND_SOC_QDSP6_USB
>> +	tristate
>> +
>>   config SND_SOC_QDSP6_PRM
>>   	tristate
>>   	select SND_SOC_QDSP6_PRM_LPASS_CLOCKS
>> @@ -131,6 +134,7 @@ config SND_SOC_QDSP6
>>   	select SND_SOC_TOPOLOGY
>>   	select SND_SOC_QDSP6_APM
>>   	select SND_SOC_QDSP6_PRM
>> +	select SND_SOC_QDSP6_USB
>>   	help
>>   	 To add support for MSM QDSP6 Soc Audio.
>>   	 This will enable sound soc platform specific
>> diff --git a/sound/soc/qcom/qdsp6/Makefile b/sound/soc/qcom/qdsp6/Makefile
>> index 3963bf234664..c9457ee898d0 100644
>> --- a/sound/soc/qcom/qdsp6/Makefile
>> +++ b/sound/soc/qcom/qdsp6/Makefile
>> @@ -17,3 +17,4 @@ obj-$(CONFIG_SND_SOC_QDSP6_APM_DAI) += q6apm-dai.o
>>   obj-$(CONFIG_SND_SOC_QDSP6_APM_LPASS_DAI) += q6apm-lpass-dais.o
>>   obj-$(CONFIG_SND_SOC_QDSP6_PRM) += q6prm.o
>>   obj-$(CONFIG_SND_SOC_QDSP6_PRM_LPASS_CLOCKS) += q6prm-clocks.o
>> +obj-$(CONFIG_SND_SOC_QDSP6_USB) += q6usb.o
>> diff --git a/sound/soc/qcom/qdsp6/q6usb.c b/sound/soc/qcom/qdsp6/q6usb.c
>> new file mode 100644
>> index 000000000000..a9da6dec6c6f
>> --- /dev/null
>> +++ b/sound/soc/qcom/qdsp6/q6usb.c
>> @@ -0,0 +1,232 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/*
>> + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
> 
> All of the code in this patch series is older than 2022 as I know it has
> been in shipping devices for many years.  Please use the proper
> copyright year to make your lawyers happy...
> 
>> + */
>> +
>> +#include <linux/err.h>
>> +#include <linux/init.h>
>> +#include <linux/module.h>
>> +#include <linux/device.h>
>> +#include <linux/platform_device.h>
>> +#include <linux/slab.h>
>> +#include <linux/iommu.h>
>> +#include <linux/dma-mapping.h>
>> +#include <linux/dma-map-ops.h>
>> +
>> +#include <sound/pcm.h>
>> +#include <sound/soc.h>
>> +#include <sound/soc-usb.h>
>> +#include <sound/pcm_params.h>
>> +#include <sound/asound.h>
>> +#include <sound/q6usboffload.h>
>> +
>> +#include "q6dsp-lpass-ports.h"
>> +#include "q6afe.h"
>> +
>> +struct q6usb_port_data {
>> +	struct q6afe_usb_cfg usb_cfg;
>> +	struct snd_soc_usb *usb;
>> +	struct q6usb_offload priv;
>> +	int active_idx;
>> +};
>> +
>> +static const struct snd_soc_dapm_widget q6usb_dai_widgets[] = {
>> +	SND_SOC_DAPM_HP("USB_RX_BE", NULL),
>> +};
>> +
>> +static const struct snd_soc_dapm_route q6usb_dapm_routes[] = {
>> +	{"USB Playback", NULL, "USB_RX_BE"},
>> +};
> 
> No terminating entry?  How does this not break?  Why do you need to
> specify the size of the array, that feels like a design bug somewhere.
> 
> 
>> +
>> +static int q6usb_hw_params(struct snd_pcm_substream *substream,
>> +			   struct snd_pcm_hw_params *params,
>> +			   struct snd_soc_dai *dai)
>> +{
>> +	return 0;
>> +}
>> +static const struct snd_soc_dai_ops q6usb_ops = {
>> +	.hw_params	= q6usb_hw_params,
>> +};
>> +
>> +static struct snd_soc_dai_driver q6usb_be_dais[] = {
>> +	{
>> +		.playback = {
>> +			.stream_name = "USB BE RX",
>> +			.rates = SNDRV_PCM_RATE_8000 | SNDRV_PCM_RATE_11025 |
>> +					SNDRV_PCM_RATE_16000 | SNDRV_PCM_RATE_22050 |
>> +					SNDRV_PCM_RATE_32000 | SNDRV_PCM_RATE_44100 |
>> +					SNDRV_PCM_RATE_48000 | SNDRV_PCM_RATE_96000 |
>> +					SNDRV_PCM_RATE_192000,
>> +			.formats = SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S16_BE |
>> +					SNDRV_PCM_FMTBIT_U16_LE | SNDRV_PCM_FMTBIT_U16_BE |
>> +					SNDRV_PCM_FMTBIT_S24_LE | SNDRV_PCM_FMTBIT_S24_BE |
>> +					SNDRV_PCM_FMTBIT_U24_LE | SNDRV_PCM_FMTBIT_U24_BE,
>> +			.channels_min = 1,
>> +			.channels_max = 2,
>> +			.rate_max =     192000,
>> +			.rate_min =	8000,
>> +		},
>> +		.id = USB_RX,
>> +		.name = "USB_RX_BE",
>> +		.ops = &q6usb_ops,
>> +	},
>> +};
>> +
>> +int q6usb_audio_ports_of_xlate_dai_name(struct snd_soc_component *component,
>> +					const struct of_phandle_args *args,
>> +					const char **dai_name)
>> +{
>> +	int id = args->args[0];
>> +	int ret = -EINVAL;
>> +	int i;
>> +
>> +	for (i = 0; i  < ARRAY_SIZE(q6usb_be_dais); i++) {
>> +		if (q6usb_be_dais[i].id == id) {
>> +			*dai_name = q6usb_be_dais[i].name;
>> +			ret = 0;
>> +			break;
>> +		}
>> +	}
>> +
>> +	return ret;
>> +}
>> +
>> +static int q6usb_component_probe(struct snd_soc_component *component)
>> +{
>> +	struct q6usb_port_data *data = dev_get_drvdata(component->dev);
>> +	struct snd_soc_dapm_context *dapm = snd_soc_component_get_dapm(component);
>> +
>> +	data->usb->component = component;
>> +
>> +	snd_soc_dapm_disable_pin(dapm, "USB_RX_BE");
>> +	snd_soc_dapm_sync(dapm);
>> +
>> +	return 0;
>> +}
>> +
>> +static const struct snd_soc_component_driver q6usb_dai_component = {
>> +	.probe		= q6usb_component_probe,
>> +	.name		= "q6usb-dai-component",
>> +	.dapm_widgets = q6usb_dai_widgets,
>> +	.num_dapm_widgets = ARRAY_SIZE(q6usb_dai_widgets),
>> +	.dapm_routes = q6usb_dapm_routes,
>> +	.num_dapm_routes = ARRAY_SIZE(q6usb_dapm_routes),
>> +	.of_xlate_dai_name = q6usb_audio_ports_of_xlate_dai_name,
>> +};
>> +
>> +int q6usb_alsa_connection_cb(struct snd_soc_usb *usb, int card_idx,
>> +			int connected)
>> +{
>> +	struct snd_soc_dapm_context *dapm;
>> +	struct q6usb_port_data *data;
>> +
>> +	if (!usb->component)
>> +		return 0;
> 
> How can this happen?
> 
> Why is this not an error?
> 

If the ASoC platform card that this USB BE is a child of, is not yet 
registered then the component probe hasn't happened.  However, that is 
independent of when the USB connect/disconnect events can happen.

>> +
>> +	dapm = snd_soc_component_get_dapm(usb->component);
>> +	data = dev_get_drvdata(usb->component->dev);
>> +
>> +	if (connected) {
>> +		snd_soc_dapm_enable_pin(dapm, "USB_RX_BE");
>> +		/* We only track the latest USB headset plugged in */
>> +		data->active_idx = card_idx;
>> +	} else {
>> +		snd_soc_dapm_disable_pin(dapm, "USB_RX_BE");
>> +	}
>> +	snd_soc_dapm_sync(dapm);
>> +
>> +	return 0;
>> +}
>> +
>> +static int q6usb_dai_dev_probe(struct platform_device *pdev)
>> +{
>> +	struct device_node *node = pdev->dev.of_node;
>> +	struct q6usb_port_data *data;
>> +	struct device *dev = &pdev->dev;
>> +	int ret;
>> +
>> +	data = devm_kzalloc(dev, sizeof(*data), GFP_KERNEL);
>> +	if (!data)
>> +		return -ENOMEM;
>> +
>> +	ret = of_property_read_u32(node, "qcom,usb-audio-stream-id",
>> +				&data->priv.sid);
>> +	if (ret) {
>> +		dev_err(&pdev->dev, "failed to read sid.\n");
>> +		return -ENODEV;
>> +	}
>> +
>> +	ret = of_property_read_u32(node, "qcom,usb-audio-intr-num",
>> +				&data->priv.intr_num);
>> +	if (ret) {
>> +		dev_err(&pdev->dev, "failed to read intr num.\n");
>> +		return -ENODEV;
>> +	}
>> +
>> +	data->priv.domain = iommu_domain_alloc(pdev->dev.bus);
>> +	if (!data->priv.domain) {
>> +		dev_err(&pdev->dev, "failed to allocate iommu domain\n");
>> +		return -ENODEV;
>> +	}
>> +
>> +	/* attach to external processor iommu */
>> +	ret = iommu_attach_device(data->priv.domain, &pdev->dev);
>> +	if (ret) {
>> +		dev_err(&pdev->dev, "failed to attach device ret = %d\n", ret);
>> +		goto free_domain;
>> +	}
>> +
>> +	data->usb = snd_soc_usb_add_port(dev, q6usb_alsa_connection_cb);
>> +	if (IS_ERR(data->usb)) {
>> +		dev_err(&pdev->dev, "failed to add usb port\n");
>> +		goto detach_device;
>> +	}
>> +
>> +	data->priv.dev = dev;
>> +	dev_set_drvdata(dev, data);
>> +	devm_snd_soc_register_component(dev, &q6usb_dai_component,
>> +							q6usb_be_dais, ARRAY_SIZE(q6usb_be_dais));
> 
> Very odd indentation.  Please do this properly everywhere.
> 
> 

Got it...will fix everywhere

>> +	snd_soc_usb_set_priv_data(&data->priv);
>> +
>> +	return 0;
>> +
>> +detach_device:
>> +	iommu_detach_device(data->priv.domain, &pdev->dev);
>> +free_domain:
>> +	iommu_domain_free(data->priv.domain);
>> +
>> +	return ret;
>> +}
>> +
>> +static int q6usb_dai_dev_remove(struct platform_device *pdev)
>> +{
>> +	struct q6usb_port_data *data = platform_get_drvdata(pdev);
>> +
>> +	iommu_detach_device(data->priv.domain, &pdev->dev);
>> +	iommu_domain_free(data->priv.domain);
>> +
>> +	snd_soc_usb_remove_port();
>> +
>> +	return 0;
>> +}
>> +
>> +#ifdef CONFIG_OF
> 
> Is this really needed still?
> 

Will remove this

Thanks
Wesley Cheng

Re: [RFC PATCH 06/14] usb: core: hcd: Introduce USB HCD APIs for interrupter management

[Wesley Cheng]

Hi Alan,

On 12/24/2022 7:29 AM, Alan Stern wrote:
> On Fri, Dec 23, 2022 at 03:31:52PM -0800, Wesley Cheng wrote:
>> For USB HCDs that can support multiple USB interrupters, expose functions
>> that class drivers can utilize for setting up secondary interrupters.
>> Class drivers can pass this information to its respective clients, i.e.
>> a dedicated DSP.
>>
>> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
>> ---
>>   drivers/usb/core/hcd.c  | 86 +++++++++++++++++++++++++++++++++++++++++
>>   include/linux/usb.h     |  7 ++++
>>   include/linux/usb/hcd.h | 16 +++++++-
>>   3 files changed, 108 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/usb/core/hcd.c b/drivers/usb/core/hcd.c
>> index 8300baedafd2..90ead90faf1d 100644
>> --- a/drivers/usb/core/hcd.c
>> +++ b/drivers/usb/core/hcd.c
> 
>> +/**
>> + * usb_hcd_stop_endpoint - Halt USB EP transfers
>> + * @udev: usb device
>> + * @ep: usb ep to stop
>> + *
>> + * Stop pending transfers on a specific USB endpoint.
>> + **/
>> +int usb_hcd_stop_endpoint(struct usb_device *udev,
>> +					struct usb_host_endpoint *ep)
>> +{
>> +	struct usb_hcd *hcd = bus_to_hcd(udev->bus);
>> +	int ret = 0;
>> +
>> +	if (hcd->driver->stop_endpoint)
>> +		ret = hcd->driver->stop_endpoint(hcd, udev, ep);
>> +
>> +	return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(usb_hcd_stop_endpoint);
> 
> You know, there already is a function that does this.  It's named
> usb_hcd_flush_endpoint().  No need to add another function that does the
> same thing.
> 

Thanks for the suggestion and review.

Hmmm...maybe I should change the name of the API then to avoid the 
confusion.  Yes, usb_hcd_flush_endpoint() does ensure that URBs 
submitted to the EP are stopped.  However, with this offloading concept, 
we aren't actually submitting URBs from the main processor, so the 
ep->urb_list will be empty.

This means the usb_hcd_flush_endpoint() API won't actually do anything. 
  What we need is to ensure that we send a XHCI stop ep command to the 
controller.

Thanks
Wesley Cheng

Re: [RFC PATCH 10/14] sound: usb: card: Check for support for requested audio format

[Wesley Cheng]

Hi Greg,

On 12/24/2022 12:59 AM, Greg KH wrote:
> On Fri, Dec 23, 2022 at 03:31:56PM -0800, Wesley Cheng wrote:
>> Allow for checks on a specific USB audio device to see if a requested PCM
>> format is supported.  This is needed for support for when playback is
>> initiated by the ASoC USB backend path.
>>
>> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
>> ---
>>   sound/usb/card.c | 19 +++++++++++++++++++
>>   sound/usb/card.h |  3 +++
>>   2 files changed, 22 insertions(+)
>>
>> diff --git a/sound/usb/card.c b/sound/usb/card.c
>> index 396e5a34e23b..9b8d2ed308c8 100644
>> --- a/sound/usb/card.c
>> +++ b/sound/usb/card.c
>> @@ -133,6 +133,25 @@ int snd_usb_unregister_vendor_ops(void)
>>   }
>>   EXPORT_SYMBOL_GPL(snd_usb_unregister_vendor_ops);
>>   
>> +struct snd_usb_stream *snd_usb_find_suppported_substream(int card_idx,
>> +			struct snd_pcm_hw_params *params, int direction)
>> +{
>> +	struct snd_usb_stream *as;
>> +	struct snd_usb_substream *subs = NULL;
>> +	const struct audioformat *fmt;
>> +
>> +	if (usb_chip[card_idx] && enable[card_idx]) {
>> +		list_for_each_entry(as, &usb_chip[card_idx]->pcm_list, list) {
>> +			subs = &as->substream[direction];
>> +			fmt = find_substream_format(subs, params);
>> +			if (fmt)
>> +				return as;
>> +		}
>> +	}
> 
> Where is the locking here?  How can you walk a list that can be changed
> as you walk it?
> 
> And what about reference counting?  You are returning a pointer to a
> structure, who now "owns" it?  What happens if it is removed from the
> system after you return it?
> 
>> +	return 0;
> 
> Didn't sparse complain about this?  You can't return "0" as a pointer,
> it should be NULL.
> 
> Always run basic tools like sparse on code before submitting it so that
> we don't have to find errors like this.
> 

Got it...I didn't get a chance to run that, but will do it on future 
submissions.  Will also address the locking and pointer reference you 
mentioned.

Thanks
Wesley Cheng

Re: [RFC PATCH 06/14] usb: core: hcd: Introduce USB HCD APIs for interrupter management

[Wesley Cheng]

Hi Greg,

On 12/24/2022 12:54 AM, Greg KH wrote:
> On Fri, Dec 23, 2022 at 03:31:52PM -0800, Wesley Cheng wrote:
>> For USB HCDs that can support multiple USB interrupters, expose functions
>> that class drivers can utilize for setting up secondary interrupters.
>> Class drivers can pass this information to its respective clients, i.e.
>> a dedicated DSP.
> 
> Where is the locking here that seems to be required when a hcd is
> removed from the system and you have data in flight?  What am I missing
> here in the design of this?

The XHCI driver is the one that maintains the list of interrupters that 
are available, so the locking was placed in the XHCI driver versus 
adding it in the core hcd layer.

> 
> And yes, HCDs get removed all the time, and will happen more and more in
> the future with the design of more systems moving to Thunderbolt/PCIe
> designs due to the simplicity of it.
> 

As part of the HCD removal, it has to first ensure that class driver 
interfaces, and the connected udevs are removed first.  qc_audio_offload 
will first handle the transfer cleanup and stopping of the audio stream 
before returning from the disconnect callback. (this includes ensuring 
that the interrupter is released)

This concept is how all usb class drivers are currently implemented. 
When the HCD remove occurs, the class drivers are the ones responsible 
for ensuring that all URBs are stopped, and removed before it returns 
from its respective disconnect callback.

>> +/**
>> + * usb_set_interruper - Reserve an interrupter
> 
> Where is an "interrupter" defined?  I don't know what this term means
> sorry, is this in the USB specification somewhere?
> 

Interrupter is defined in the XHCI spec, refer to "section 4.17 
Interrupters"

> 
>> + * @udev: usb device which requested the interrupter
>> + * @intr_num: interrupter number to reserve
>> + * @dma: iova address to event ring
>> + *
>> + * Request for a specific interrupter to be reserved for a USB class driver.
>> + * This will return the base address to the event ring that was allocated to
>> + * the specific interrupter.
>> + **/
>> +phys_addr_t usb_set_interruper(struct usb_device *udev, int intr_num,
>> +							dma_addr_t *dma)
>> +{
>> +	struct usb_hcd *hcd;
>> +	phys_addr_t pa = 0;
>> +
>> +	hcd = bus_to_hcd(udev->bus);
>> +	if (hcd->driver->update_interrupter)
>> +		pa = hcd->driver->update_interrupter(hcd, intr_num, dma);
>> +
>> +	return pa;
> 
> Wait, you return a physical address?  What are you going to do with
> that?  And what guarantees that the address is valid after you return it
> (again, remember memory and devices can be removed at any point in time.
> 

The interrupter is basically another event ring which is the buffer 
allocated for the controller to copy events into.  Since the audio dsp 
now takes over handling of the endpoint events, then it needs to know 
where the buffer resides.  Will fix the interruper typo as well.

The allocation and freeing of this event ring follows how XHCI allocates 
and frees the main event ring as well.  This API just reserves the 
interrupter for the class driver, and return the previously allocated 
(during XHCI init) memory address.

Thanks
Wesley Cheng

Re: [RFC PATCH 04/14] sound: usb: card: Introduce USB SND vendor op callbacks

[Dmitry Baryshkov]

On 27/12/2022 23:07, Wesley Cheng wrote:
> Hi Dmitry,
> 
> On 12/24/2022 3:03 AM, Dmitry Baryshkov wrote:
>> Hi,
>>
>> On Sat, 24 Dec 2022 at 01:33, Wesley Cheng <quic_wcheng@quicinc.com> 
>> wrote:
>>>
>>> Allow for different vendors to be notified on USB SND connect/disconnect
>>> seqeunces.  This allows for vendor USB SND modules to properly 
>>> initialize
>>> and populate internal structures with references to the USB SND chip
>>> device.
>>
>> The commit message definitely needs some improvement. We do not notify
>> vendors on SND connect/disconnect events.
>>
>>
>>>
>>> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
>>> ---
>>>   sound/usb/card.c | 22 ++++++++++++++++++++++
>>>   sound/usb/card.h |  7 +++++++
>>>   2 files changed, 29 insertions(+)
>>>
>>> diff --git a/sound/usb/card.c b/sound/usb/card.c
>>> index 26268ffb8274..212f55a7683c 100644
>>> --- a/sound/usb/card.c
>>> +++ b/sound/usb/card.c
>>> @@ -117,6 +117,21 @@ MODULE_PARM_DESC(skip_validation, "Skip unit 
>>> descriptor validation (default: no)
>>>   static DEFINE_MUTEX(register_mutex);
>>>   static struct snd_usb_audio *usb_chip[SNDRV_CARDS];
>>>   static struct usb_driver usb_audio_driver;
>>> +static struct snd_usb_vendor_ops *vendor_ops;
>>> +
>>> +int snd_usb_register_vendor_ops(struct snd_usb_vendor_ops *ops)
>>
>> platform ops?
>>
> 
> Will change it.
> 
>>> +{
>>> +       vendor_ops = ops;
>>> +       return 0;
>>> +}
>>> +EXPORT_SYMBOL_GPL(snd_usb_register_vendor_ops);
>>
>> What happens if several platforms try to register different ops? I saw
>> from the patch 09/14 that you register these ops unconditionally. If
>> other devices follow your approach there is an obvious conflict.
>>
> 
> Thank you for the review.
> 
> That is true.  I don't think there is a proper need to have multiple 
> vendor ops being registered, so maybe just returning an error for if ops 
> are already registered is sufficient.

This would be a required step. And also you have to check the running 
platform before registering your ops unconditionally. Ideally this 
should be done as a part of the device's driver, so that we can control 
registration of the platform ops using the usual interface.

> 
> Thanks
> Wesley Cheng

-- 
With best wishes
Dmitry

Re: [RFC PATCH 06/14] usb: core: hcd: Introduce USB HCD APIs for interrupter management

[Oliver Neukum]

On 27.12.22 22:07, Wesley Cheng wrote:

> 
> Hmmm...maybe I should change the name of the API then to avoid the confusion.  Yes, usb_hcd_flush_endpoint() does ensure that URBs submitted to the EP are stopped.  However, with this offloading concept, we aren't actually submitting URBs from the main processor, so the ep->urb_list will be empty.
> 
> This means the usb_hcd_flush_endpoint() API won't actually do anything.  What we need is to ensure that we send a XHCI stop ep command to the controller.

That is a concept specific to XHCI, yet you are adding a generic
API. The namin should reflect that. usb_quiesce_endpoint() ?

	Regards
		Oliver

Re: [RFC PATCH 06/14] usb: core: hcd: Introduce USB HCD APIs for interrupter management

[Alan Stern]

On Wed, Dec 28, 2022 at 09:59:03AM +0100, Oliver Neukum wrote:
> 
> 
> On 27.12.22 22:07, Wesley Cheng wrote:
> 
> > 
> > Hmmm...maybe I should change the name of the API then to avoid the confusion.  Yes, usb_hcd_flush_endpoint() does ensure that URBs submitted to the EP are stopped.  However, with this offloading concept, we aren't actually submitting URBs from the main processor, so the ep->urb_list will be empty.
> > 
> > This means the usb_hcd_flush_endpoint() API won't actually do anything.  What we need is to ensure that we send a XHCI stop ep command to the controller.
> 
> That is a concept specific to XHCI, yet you are adding a generic
> API. The namin should reflect that. usb_quiesce_endpoint() ?

Or even xhci_send_stop_ep_cmd(), which is what the routine is intended 
to do.

Alan Stern

Re: [RFC PATCH 07/14] usb: host: xhci: Add XHCI secondary interrupter support

[Mathias Nyman]

On 24.12.2022 1.31, Wesley Cheng wrote:
> Implement the XHCI operations for allocating and requesting for a secondary
> interrupter.  The secondary interrupter can allow for events for a
> particular endpoint to be routed to a separate event ring.  The event
> routing is defined when submitting a transfer descriptor to the USB HW.
> There is a specific field which denotes which interrupter ring to route the
> event to when the transfer is completed.
> 
> An example use case, such as audio packet offloading can utilize a separate
> event ring, so that these events can be routed to a different processor
> within the system.  The processor would be able to independently submit
> transfers and handle its completions without intervention from the main
> processor.
> 

Adding support for more xHCI interrupters than just the primary one make sense for
both the offloading and virtualization cases.

xHCI support for several interrupters was probably added to support virtualization,
to hand over usb devices to virtual machines and give them their own event ring and
MSI/MSI-X vector.

In this offloading case you probably want to avoid xHC interrupts from this device
completely, making sure it doesn't wake up the main CPU unnecessarily.

So is the idea here to let xhci driver set up the new interrupter, its event ring,
and the endpoint transfer rings. Then pass the address of the endpoint transfer rings
and the new event ring to the separate processor.

This separate processor then both polls the event ring for new events, sets its dequeue
pointer, clears EHB bit, and queues new TRBs on the transfer ring.

so xhci driver does not handle any events for the audio part, and no audio data URBs
are sent to usb core?

How about the control part?
Is the control endpoint for this device still handled normally by usb core/xhci?

For the xhci parts I think we should start start by adding generic support for several
interrupters, then add parts needed for offloading.

Thanks
Mathias

Re: [RFC PATCH 06/14] usb: core: hcd: Introduce USB HCD APIs for interrupter management

[Wesley Cheng]

Hi Alan,

On 12/28/2022 7:16 AM, Alan Stern wrote:
> On Wed, Dec 28, 2022 at 09:59:03AM +0100, Oliver Neukum wrote:
>>
>>
>> On 27.12.22 22:07, Wesley Cheng wrote:
>>
>>>
>>> Hmmm...maybe I should change the name of the API then to avoid the confusion.  Yes, usb_hcd_flush_endpoint() does ensure that URBs submitted to the EP are stopped.  However, with this offloading concept, we aren't actually submitting URBs from the main processor, so the ep->urb_list will be empty.
>>>
>>> This means the usb_hcd_flush_endpoint() API won't actually do anything.  What we need is to ensure that we send a XHCI stop ep command to the controller.
>>
>> That is a concept specific to XHCI, yet you are adding a generic
>> API. The namin should reflect that. usb_quiesce_endpoint() ?
> 
> Or even xhci_send_stop_ep_cmd(), which is what the routine is intended
> to do.
> 

Just to clarify, you're talking about renaming the API that was added in 
the XHCI driver, correct?

Thanks
Wesley Cheng

Re: [RFC PATCH 06/14] usb: core: hcd: Introduce USB HCD APIs for interrupter management

[Alan Stern]

On Wed, Dec 28, 2022 at 12:31:16PM -0800, Wesley Cheng wrote:
> Hi Alan,
> 
> On 12/28/2022 7:16 AM, Alan Stern wrote:
> > On Wed, Dec 28, 2022 at 09:59:03AM +0100, Oliver Neukum wrote:
> > > 
> > > 
> > > On 27.12.22 22:07, Wesley Cheng wrote:
> > > 
> > > > 
> > > > Hmmm...maybe I should change the name of the API then to avoid the confusion.  Yes, usb_hcd_flush_endpoint() does ensure that URBs submitted to the EP are stopped.  However, with this offloading concept, we aren't actually submitting URBs from the main processor, so the ep->urb_list will be empty.
> > > > 
> > > > This means the usb_hcd_flush_endpoint() API won't actually do anything.  What we need is to ensure that we send a XHCI stop ep command to the controller.
> > > 
> > > That is a concept specific to XHCI, yet you are adding a generic
> > > API. The namin should reflect that. usb_quiesce_endpoint() ?
> > 
> > Or even xhci_send_stop_ep_cmd(), which is what the routine is intended
> > to do.
> > 
> 
> Just to clarify, you're talking about renaming the API that was added in the
> XHCI driver, correct?

To be precise, we're talking about renaming your usb_hcd_stop_endpoint() 
function, although similar arguments probably apply to your 
usb_free_interrupter(), usb_set_interrupter(), and 
usb_hcd_get_transfer_resource() routines.

You wrote earlier:

	The XHCI driver is the one that maintains the list of 
	interrupters that are available, so the locking was placed in 
	the XHCI driver versus adding it in the core hcd layer.

The "stop ep" functionality and other interrupter management things you 
want to add seem a lot like this locking stuff.  Since you decided to 
put the locking in the xhci-hcd driver instead of the core HCD layer, it 
would be logical to do the same with the "stop ep" and other routines.  
Which means there shouldn't be any need to make changes to hcd.c or 
include/linux/usb/hcd.h.

Alan Stern

Re: [RFC PATCH 04/14] sound: usb: card: Introduce USB SND vendor op callbacks

[Oliver Neukum]

On 24.12.22 00:31, Wesley Cheng wrote:
> Allow for different vendors to be notified on USB SND connect/disconnect
> seqeunces.  This allows for vendor USB SND modules to properly initialize
> and populate internal structures with references to the USB SND chip
> device.

Hi,

this raises a design question. If the system is suspending or, worse,
hibernating, how do you make sure the offloader and the device are
suspended in the correct order?
And what happens if you need to go into reset_resume() when resuming?

	Regards
		Oliver

Re: [RFC PATCH 04/14] sound: usb: card: Introduce USB SND vendor op callbacks

[Takashi Iwai]

On Thu, 29 Dec 2022 14:49:21 +0100,
Oliver Neukum wrote:
> 
> 
> 
> On 24.12.22 00:31, Wesley Cheng wrote:
> > Allow for different vendors to be notified on USB SND connect/disconnect
> > seqeunces.  This allows for vendor USB SND modules to properly initialize
> > and populate internal structures with references to the USB SND chip
> > device.
> 
> Hi,
> 
> this raises a design question. If the system is suspending or, worse,
> hibernating, how do you make sure the offloader and the device are
> suspended in the correct order?
> And what happens if you need to go into reset_resume() when resuming?

I guess we'd need to establish a device link when the binding from the
offload driver is done.  Then the PM order will be assured.


Takashi

Re: [RFC PATCH 07/14] usb: host: xhci: Add XHCI secondary interrupter support

[Wesley Cheng]

Hi Mathias,

On 12/28/2022 7:47 AM, Mathias Nyman wrote:
> On 24.12.2022 1.31, Wesley Cheng wrote:
>> Implement the XHCI operations for allocating and requesting for a 
>> secondary
>> interrupter.  The secondary interrupter can allow for events for a
>> particular endpoint to be routed to a separate event ring.  The event
>> routing is defined when submitting a transfer descriptor to the USB HW.
>> There is a specific field which denotes which interrupter ring to 
>> route the
>> event to when the transfer is completed.
>>
>> An example use case, such as audio packet offloading can utilize a 
>> separate
>> event ring, so that these events can be routed to a different processor
>> within the system.  The processor would be able to independently submit
>> transfers and handle its completions without intervention from the main
>> processor.
>>
> 
> Adding support for more xHCI interrupters than just the primary one make 
> sense for
> both the offloading and virtualization cases.
> 
> xHCI support for several interrupters was probably added to support 
> virtualization,
> to hand over usb devices to virtual machines and give them their own 
> event ring and
> MSI/MSI-X vector.
> 
> In this offloading case you probably want to avoid xHC interrupts from 
> this device
> completely, making sure it doesn't wake up the main CPU unnecessarily.
> 
> So is the idea here to let xhci driver set up the new interrupter, its 
> event ring,
> and the endpoint transfer rings. Then pass the address of the endpoint 
> transfer rings
> and the new event ring to the separate processor.
> 
> This separate processor then both polls the event ring for new events, 
> sets its dequeue
> pointer, clears EHB bit, and queues new TRBs on the transfer ring.
> 
> so xhci driver does not handle any events for the audio part, and no 
> audio data URBs
> are sent to usb core?

Your entire description is correct.  To clarify, the interfaces which 
are non-audio will still be handled by the main processor.  For example, 
a USB headset can have a HID interface as well for volume control.  The 
HID interface will still be handled by the main processor, and events 
routed to the main event ring.

> 
> How about the control part?
> Is the control endpoint for this device still handled normally by usb 
> core/xhci?
> 

Control transfers are always handled on the main processor.  Only audio 
interface's endpoints.

> For the xhci parts I think we should start start by adding generic 
> support for several
> interrupters, then add parts needed for offloading.

I can split up the patchsets to add interrupters first, then adding the 
offloading APIs in a separate patch.

Thanks
Wesley Cheng

Re: [RFC PATCH 04/14] sound: usb: card: Introduce USB SND vendor op callbacks

[Wesley Cheng]

Hi,

On 12/29/2022 6:20 AM, Takashi Iwai wrote:
> On Thu, 29 Dec 2022 14:49:21 +0100,
> Oliver Neukum wrote:
>>
>>
>>
>> On 24.12.22 00:31, Wesley Cheng wrote:
>>> Allow for different vendors to be notified on USB SND connect/disconnect
>>> seqeunces.  This allows for vendor USB SND modules to properly initialize
>>> and populate internal structures with references to the USB SND chip
>>> device.
>>
>> Hi,
>>
>> this raises a design question. If the system is suspending or, worse,
>> hibernating, how do you make sure the offloader and the device are
>> suspended in the correct order?
>> And what happens if you need to go into reset_resume() when resuming?

It may depend on how the offloading is implemented, but we do have a 
mechanism to force the audio stream off from the qc_usb_audio_offload. 
Regardless of if the UDEV is suspended first, or the USB backend, as 
long as we ensure that the offloading is disabled before entering 
suspend, I think that should be sufficient.  I would need to add some 
suspend handling in the offload driver to issue the command to stop the 
offloading.

As for the resume path, is there a concern if either device is resumed 
first?  The only scenario where maybe it could cause some mishandling is 
if the USB backend is resumed before the offload driver is 
connected/resumed.  This means that the userspace ALSA would have access 
to the platform sound card, and could potentially attempt to route audio 
streams to it.  I think in worst case, if we were going through a 
reset_resume() we would end up rejecting that request coming from the 
audio DSP to enable the stream.  However, userspace entities would be 
resumed/unfrozen last, so not sure if that would ever be a problem.

The reset_resume() path is fine.  Bus reset is going to cause a 
disconnect() callback in the offload driver, in which we already have 
the proper handling for ensuring the offload path is halted, and we 
reject any incoming stream start requests.

Thanks
Wesley Cheng

Re: [RFC PATCH 07/14] usb: host: xhci: Add XHCI secondary interrupter support

[Mathias Nyman]

On 29.12.2022 23.14, Wesley Cheng wrote:
> Hi Mathias,
> 
> On 12/28/2022 7:47 AM, Mathias Nyman wrote:
>> On 24.12.2022 1.31, Wesley Cheng wrote:
>>> Implement the XHCI operations for allocating and requesting for a secondary
>>> interrupter.  The secondary interrupter can allow for events for a
>>> particular endpoint to be routed to a separate event ring.  The event
>>> routing is defined when submitting a transfer descriptor to the USB HW.
>>> There is a specific field which denotes which interrupter ring to route the
>>> event to when the transfer is completed.
>>>
>>> An example use case, such as audio packet offloading can utilize a separate
>>> event ring, so that these events can be routed to a different processor
>>> within the system.  The processor would be able to independently submit
>>> transfers and handle its completions without intervention from the main
>>> processor.
>>>
>>
>> Adding support for more xHCI interrupters than just the primary one make sense for
>> both the offloading and virtualization cases.
>>
>> xHCI support for several interrupters was probably added to support virtualization,
>> to hand over usb devices to virtual machines and give them their own event ring and
>> MSI/MSI-X vector.
>>
>> In this offloading case you probably want to avoid xHC interrupts from this device
>> completely, making sure it doesn't wake up the main CPU unnecessarily.
>>
>> So is the idea here to let xhci driver set up the new interrupter, its event ring,
>> and the endpoint transfer rings. Then pass the address of the endpoint transfer rings
>> and the new event ring to the separate processor.
>>
>> This separate processor then both polls the event ring for new events, sets its dequeue
>> pointer, clears EHB bit, and queues new TRBs on the transfer ring.
>>
>> so xhci driver does not handle any events for the audio part, and no audio data URBs
>> are sent to usb core?
> 
> Your entire description is correct.  To clarify, the interfaces which are non-audio will still be handled by the main processor.  For example, a USB headset can have a HID interface as well for volume control.  The HID interface will still be handled by the main processor, and events routed to the main event ring.
> 
>>
>> How about the control part?
>> Is the control endpoint for this device still handled normally by usb core/xhci?
>>
> 
> Control transfers are always handled on the main processor.  Only audio interface's endpoints.

Good to know, that means interrupter should be chosen per endpoint, not per device.

> 
>> For the xhci parts I think we should start start by adding generic support for several
>> interrupters, then add parts needed for offloading.
> 
> I can split up the patchsets to add interrupters first, then adding the offloading APIs in a separate patch.


I started looking at supporting secondary interrupters myself.
Let me work on that part a bit first. We have a bit different end goals.
I want to handle interrupts from a secondary interrupter, while this audio offload
really just wants to mask some interrupts.

Thanks
Mathias

Re: [RFC PATCH 09/14] sound: usb: Introduce QC USB SND offloading support

[Takashi Iwai]

On Sat, 24 Dec 2022 00:31:55 +0100,
Wesley Cheng wrote:
> 
> Several Qualcomm SoCs have a dedicated audio DSP, which has the ability to
> support USB sound devices.  This vendor driver will implement the required
> handshaking with the DSP, in order to pass along required resources that
> will be utilized by the DSP's USB SW.  The communication channel used for
> this handshaking will be using the QMI protocol.  Required resources
> include:
> - Allocated secondary event ring address
> - EP transfer ring address
> - Interrupter number
> 
> The above information will allow for the audio DSP to execute USB transfers
> over the USB bus.  It will also be able to support devices that have an
> implicit feedback and sync endpoint as well.  Offloading these data
> transfers will allow the main/applications processor to enter lower CPU
> power modes, and sustain a longer duration in those modes.
> 
> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>

Hmm, this must be the main part that works to bypass the normal USB
packet handling in USB audio driver but hooks to the own offload one,
but there is no description how to take over and manage.
A missing "big picture" makes it difficult to understand and review.

Also, since both drivers are asynchronous, we may need some proper
locking.

More on the code change:

> +static int snd_interval_refine_set(struct snd_interval *i, unsigned int val)
> +{
> +	struct snd_interval t;
> +
> +	t.empty = 0;
> +	t.min = t.max = val;
> +	t.openmin = t.openmax = 0;
> +	t.integer = 1;
> +	return snd_interval_refine(i, &t);
> +}
> +
> +static int _snd_pcm_hw_param_set(struct snd_pcm_hw_params *params,
> +				 snd_pcm_hw_param_t var, unsigned int val,
> +				 int dir)
> +{
> +	int changed;
> +
> +	if (hw_is_mask(var)) {
> +		struct snd_mask *m = hw_param_mask(params, var);
> +
> +		if (val == 0 && dir < 0) {
> +			changed = -EINVAL;
> +			snd_mask_none(m);
> +		} else {
> +			if (dir > 0)
> +				val++;
> +			else if (dir < 0)
> +				val--;
> +			changed = snd_mask_refine_set(
> +					hw_param_mask(params, var), val);
> +		}
> +	} else if (hw_is_interval(var)) {
> +		struct snd_interval *i = hw_param_interval(params, var);
> +
> +		if (val == 0 && dir < 0) {
> +			changed = -EINVAL;
> +			snd_interval_none(i);
> +		} else if (dir == 0)
> +			changed = snd_interval_refine_set(i, val);
> +		else {
> +			struct snd_interval t;
> +
> +			t.openmin = 1;
> +			t.openmax = 1;
> +			t.empty = 0;
> +			t.integer = 0;
> +			if (dir < 0) {
> +				t.min = val - 1;
> +				t.max = val;
> +			} else {
> +				t.min = val;
> +				t.max = val+1;
> +			}
> +			changed = snd_interval_refine(i, &t);
> +		}
> +	} else
> +		return -EINVAL;
> +	if (changed) {
> +		params->cmask |= 1 << var;
> +		params->rmask |= 1 << var;
> +	}
> +	return changed;
> +}

Those are taken from sound/core/oss/pcm_oss.c?  We may put to the
common PCM helper instead of duplication.

> +static void disable_audio_stream(struct snd_usb_substream *subs)
> +{
> +	struct snd_usb_audio *chip = subs->stream->chip;
> +
> +	if (subs->data_endpoint || subs->sync_endpoint) {
> +		close_endpoints(chip, subs);
> +
> +		mutex_lock(&chip->mutex);
> +		subs->cur_audiofmt = NULL;
> +		mutex_unlock(&chip->mutex);
> +	}
> +
> +	snd_usb_autosuspend(chip);
> +}
> +
> +static int enable_audio_stream(struct snd_usb_substream *subs,
> +				snd_pcm_format_t pcm_format,
> +				unsigned int channels, unsigned int cur_rate,
> +				int datainterval)
> +{
> +	struct snd_usb_audio *chip = subs->stream->chip;
> +	struct snd_pcm_hw_params params;
> +	const struct audioformat *fmt;
> +	int ret;
> +
> +	_snd_pcm_hw_params_any(&params);
> +	_snd_pcm_hw_param_set(&params, SNDRV_PCM_HW_PARAM_FORMAT,
> +			pcm_format, 0);
> +	_snd_pcm_hw_param_set(&params, SNDRV_PCM_HW_PARAM_CHANNELS,
> +			channels, 0);
> +	_snd_pcm_hw_param_set(&params, SNDRV_PCM_HW_PARAM_RATE,
> +			cur_rate, 0);

What about other parameters like period / buffer sizes?

> +struct qmi_uaudio_stream_req_msg_v01 {
> +	u8 enable;
> +	u32 usb_token;
> +	u8 audio_format_valid;
> +	u32 audio_format;
> +	u8 number_of_ch_valid;
> +	u32 number_of_ch;
> +	u8 bit_rate_valid;
> +	u32 bit_rate;
> +	u8 xfer_buff_size_valid;
> +	u32 xfer_buff_size;
> +	u8 service_interval_valid;
> +	u32 service_interval;
> +};

Are this and the other structs a part of DSP ABI?
Or is it a definition only used in kernel?  I'm asking because
__packed attribute is required for most of ABI definitions with
different field types.


thanks,

Takashi

Re: [RFC PATCH 04/14] sound: usb: card: Introduce USB SND vendor op callbacks

[Oliver Neukum]

On 30.12.22 08:10, Wesley Cheng wrote:

> It may depend on how the offloading is implemented, but we do have a mechanism to force the audio stream off from the qc_usb_audio_offload. Regardless of if the UDEV is suspended first, or the USB backend, as long as we ensure that the offloading is disabled before entering suspend, I think that should be sufficient.

You would presumably output garbage, if the UDEV is asleep but the backend is not.

  
> The reset_resume() path is fine.  Bus reset is going to cause a disconnect() callback in the offload driver, in which we already have the proper handling for ensuring the offload path is halted, and we reject any incoming stream start requests.

How? If we go the reset_resume() code path, we find that usb-audio does not make
a difference between regular resume() and reset_resume()

	Regards
		Oliver

Re: [RFC PATCH 04/14] sound: usb: card: Introduce USB SND vendor op callbacks

[Takashi Iwai]

On Tue, 03 Jan 2023 13:20:48 +0100,
Oliver Neukum wrote:
> 
> 
> 
> On 30.12.22 08:10, Wesley Cheng wrote:
> 
> > It may depend on how the offloading is implemented, but we do have a mechanism to force the audio stream off from the qc_usb_audio_offload. Regardless of if the UDEV is suspended first, or the USB backend, as long as we ensure that the offloading is disabled before entering suspend, I think that should be sufficient.
> 
> You would presumably output garbage, if the UDEV is asleep but the backend is not.
> 
>  
> > The reset_resume() path is fine.  Bus reset is going to cause a disconnect() callback in the offload driver, in which we already have the proper handling for ensuring the offload path is halted, and we reject any incoming stream start requests.
> 
> How? If we go the reset_resume() code path, we find that usb-audio does not make
> a difference between regular resume() and reset_resume()

Note that, for USB audio, there is no much difference between resume()
and reset_resume(), especially about the PCM stream handling that is
the main target for the offload (the mixer isn't handled there).
And for the PCM, we just set the power state for UAC3, and that's
all.  All the rest is handled by the PCM core handler as usual.


Takashi

Re: [RFC PATCH 12/14] sound: soc: qcom: qusb6: Ensure PCM format is supported by USB audio device

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 575 bytes --]

On Fri, Dec 23, 2022 at 03:31:58PM -0800, Wesley Cheng wrote:

> Check for if the PCM format is supported during the hw_params callback.  If
> the profile is not supported then the userspace ALSA entity will receive an
> error, and can take further action.

Ideally we'd wire up constraints for this but that gets complicated with
DPCM so it's probably disproportionate effort.  Otherwise other than the
subject lines not using ASoC on this and the previous change I don't
have any issues that other people didn't raise, but then most of the
complication is in the USB bits.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [RFC PATCH 14/14] ASoC: dt-bindings: Update example for enabling USB offload on SM8250

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 407 bytes --]

On Mon, Dec 26, 2022 at 01:27:21PM +0100, Krzysztof Kozlowski wrote:
> On 24/12/2022 00:32, Wesley Cheng wrote:

> > +            link-name = "USB Playback";
> > +            cpu {
> > +                sound-dai = <&q6afedai USB_RX>;

> Hmm, that makes me wonder if you really tested the bindings before
> sending? If yes, where is the USB_RX defined?

It was added in patch 2, it's in include/dt-bindings.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [RFC PATCH 04/14] sound: usb: card: Introduce USB SND vendor op callbacks

[Wesley Cheng]

Hi Oliver,

On 1/3/2023 4:49 AM, Takashi Iwai wrote:
> On Tue, 03 Jan 2023 13:20:48 +0100,
> Oliver Neukum wrote:
>>
>>
>>
>> On 30.12.22 08:10, Wesley Cheng wrote:
>>
>>> It may depend on how the offloading is implemented, but we do have a mechanism to force the audio stream off from the qc_usb_audio_offload. Regardless of if the UDEV is suspended first, or the USB backend, as long as we ensure that the offloading is disabled before entering suspend, I think that should be sufficient.
>>
>> You would presumably output garbage, if the UDEV is asleep but the backend is not.
>>

As long as the stream is halted, i.e. the audio DSP doesn't execute 
further transfers on the bus, there shouldn't be any noise/static that 
will continue to be outputted.  When I mentioned that we have a 
mechanism to force for the offloading to be disabled to the audio DSP 
side, it will no longer submit any audio data to the USB bus.

>>   
>>> The reset_resume() path is fine.  Bus reset is going to cause a disconnect() callback in the offload driver, in which we already have the proper handling for ensuring the offload path is halted, and we reject any incoming stream start requests.
>>
>> How? If we go the reset_resume() code path, we find that usb-audio does not make
>> a difference between regular resume() and reset_resume()
> 
> Note that, for USB audio, there is no much difference between resume()
> and reset_resume(), especially about the PCM stream handling that is
> the main target for the offload (the mixer isn't handled there).
> And for the PCM, we just set the power state for UAC3, and that's
> all.  All the rest is handled by the PCM core handler as usual.
> 

Sorry, I was under the impression that the USB SND class driver didn't 
register a reset_resume() callback, which Takashi helped clarify that it 
does indeed do.  (if no callback is registered, then USB interface is 
re-binded in the resume path)  However, it doesn't explicitly treat the 
reset_resume differently than a normal resume.

For the offload path, we don't need to do anything special either - if 
we have ensured that the stream was stopped in the suspend path. (to be 
added) It would be up to the userspace to restart the ASoC PCM stream, 
which would cause another stream request enable QMI command handshake to 
happen before any transfers would start.

One thing that I could add to protect the situation where the USB ASoC 
backend is resumed before UDEV, is to check the chip->system_suspend 
state.  Normally, the offload driver needs to ensure the bus is in U0 
before starting the audio stream, but it is done using runtime PM:

static int enable_audio_stream(struct snd_usb_substream *subs,
				snd_pcm_format_t pcm_format,
				unsigned int channels, unsigned int cur_rate,
				int datainterval)
{
...
	pm_runtime_barrier(&chip->intf[0]->dev);
	snd_usb_autoresume(chip);

In case we're in the PM resume path, I don't believe PM runtime can be 
triggered to resume a device.  In this case, the snd_usb_autoresume() 
would return w/o ensuring the USB SND device is fully exited from PM 
suspend.  Although, this situation would be a corner case, as userspace 
entities (userspace ALSA) are going to be unfrozen after kernel devices 
are resumed, so most likely there should be no request to enable the 
audio stream if kernel devices are still resuming.

I don't see an issue with the sequence of UDEV being resumed before USB 
backend.  In this case, the USB bus is ready to go, and able to handle 
stream enable requests.

Thanks
Wesley Cheng

Re: [RFC PATCH 14/14] ASoC: dt-bindings: Update example for enabling USB offload on SM8250

[Rob Herring]

On Fri, 23 Dec 2022 15:32:00 -0800, Wesley Cheng wrote:
> Add an example on enabling of USB offload for the Q6DSP.  The routing can
> be done by the mixer, which can pass the multimedia stream to the USB
> backend.
> 
> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
> ---
>  .../devicetree/bindings/sound/qcom,sm8250.yaml      | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
> 

Acked-by: Rob Herring <robh@kernel.org>

Re: [RFC PATCH 09/14] sound: usb: Introduce QC USB SND offloading support

[Wesley Cheng]

Hi Takashi,

On 1/2/2023 9:28 AM, Takashi Iwai wrote:
> On Sat, 24 Dec 2022 00:31:55 +0100,
> Wesley Cheng wrote:
>>
>> Several Qualcomm SoCs have a dedicated audio DSP, which has the ability to
>> support USB sound devices.  This vendor driver will implement the required
>> handshaking with the DSP, in order to pass along required resources that
>> will be utilized by the DSP's USB SW.  The communication channel used for
>> this handshaking will be using the QMI protocol.  Required resources
>> include:
>> - Allocated secondary event ring address
>> - EP transfer ring address
>> - Interrupter number
>>
>> The above information will allow for the audio DSP to execute USB transfers
>> over the USB bus.  It will also be able to support devices that have an
>> implicit feedback and sync endpoint as well.  Offloading these data
>> transfers will allow the main/applications processor to enter lower CPU
>> power modes, and sustain a longer duration in those modes.
>>
>> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
> 
> Hmm, this must be the main part that works to bypass the normal USB
> packet handling in USB audio driver but hooks to the own offload one,
> but there is no description how to take over and manage.
> A missing "big picture" makes it difficult to understand and review.
> 

Technically, we are not taking over the functionality of the USB SND, as 
we still want the normal path to be accessible in case there is an audio 
profile/format that can't be supported by the audio DSP.  I can add some 
more information on how this offload driver co-exists with the USB SND.

> Also, since both drivers are asynchronous, we may need some proper
> locking.
> 

Yes, I think locking is needed in some places.  Will add that in the 
next revision.

> More on the code change:
> 
>> +static int snd_interval_refine_set(struct snd_interval *i, unsigned int val)
>> +{
>> +	struct snd_interval t;
>> +
>> +	t.empty = 0;
>> +	t.min = t.max = val;
>> +	t.openmin = t.openmax = 0;
>> +	t.integer = 1;
>> +	return snd_interval_refine(i, &t);
>> +}
>> +
>> +static int _snd_pcm_hw_param_set(struct snd_pcm_hw_params *params,
>> +				 snd_pcm_hw_param_t var, unsigned int val,
>> +				 int dir)
>> +{
>> +	int changed;
>> +
>> +	if (hw_is_mask(var)) {
>> +		struct snd_mask *m = hw_param_mask(params, var);
>> +
>> +		if (val == 0 && dir < 0) {
>> +			changed = -EINVAL;
>> +			snd_mask_none(m);
>> +		} else {
>> +			if (dir > 0)
>> +				val++;
>> +			else if (dir < 0)
>> +				val--;
>> +			changed = snd_mask_refine_set(
>> +					hw_param_mask(params, var), val);
>> +		}
>> +	} else if (hw_is_interval(var)) {
>> +		struct snd_interval *i = hw_param_interval(params, var);
>> +
>> +		if (val == 0 && dir < 0) {
>> +			changed = -EINVAL;
>> +			snd_interval_none(i);
>> +		} else if (dir == 0)
>> +			changed = snd_interval_refine_set(i, val);
>> +		else {
>> +			struct snd_interval t;
>> +
>> +			t.openmin = 1;
>> +			t.openmax = 1;
>> +			t.empty = 0;
>> +			t.integer = 0;
>> +			if (dir < 0) {
>> +				t.min = val - 1;
>> +				t.max = val;
>> +			} else {
>> +				t.min = val;
>> +				t.max = val+1;
>> +			}
>> +			changed = snd_interval_refine(i, &t);
>> +		}
>> +	} else
>> +		return -EINVAL;
>> +	if (changed) {
>> +		params->cmask |= 1 << var;
>> +		params->rmask |= 1 << var;
>> +	}
>> +	return changed;
>> +}
> 
> Those are taken from sound/core/oss/pcm_oss.c?  We may put to the
> common PCM helper instead of duplication.
> 

Sure, I can do that.

>> +static void disable_audio_stream(struct snd_usb_substream *subs)
>> +{
>> +	struct snd_usb_audio *chip = subs->stream->chip;
>> +
>> +	if (subs->data_endpoint || subs->sync_endpoint) {
>> +		close_endpoints(chip, subs);
>> +
>> +		mutex_lock(&chip->mutex);
>> +		subs->cur_audiofmt = NULL;
>> +		mutex_unlock(&chip->mutex);
>> +	}
>> +
>> +	snd_usb_autosuspend(chip);
>> +}
>> +
>> +static int enable_audio_stream(struct snd_usb_substream *subs,
>> +				snd_pcm_format_t pcm_format,
>> +				unsigned int channels, unsigned int cur_rate,
>> +				int datainterval)
>> +{
>> +	struct snd_usb_audio *chip = subs->stream->chip;
>> +	struct snd_pcm_hw_params params;
>> +	const struct audioformat *fmt;
>> +	int ret;
>> +
>> +	_snd_pcm_hw_params_any(&params);
>> +	_snd_pcm_hw_param_set(&params, SNDRV_PCM_HW_PARAM_FORMAT,
>> +			pcm_format, 0);
>> +	_snd_pcm_hw_param_set(&params, SNDRV_PCM_HW_PARAM_CHANNELS,
>> +			channels, 0);
>> +	_snd_pcm_hw_param_set(&params, SNDRV_PCM_HW_PARAM_RATE,
>> +			cur_rate, 0);
> 
> What about other parameters like period / buffer sizes?
> 

I don't think we will need those parameters on the audio DSP.  The 
"params" here is used to pass the pcm format into the qmi response.

>> +struct qmi_uaudio_stream_req_msg_v01 {
>> +	u8 enable;
>> +	u32 usb_token;
>> +	u8 audio_format_valid;
>> +	u32 audio_format;
>> +	u8 number_of_ch_valid;
>> +	u32 number_of_ch;
>> +	u8 bit_rate_valid;
>> +	u32 bit_rate;
>> +	u8 xfer_buff_size_valid;
>> +	u32 xfer_buff_size;
>> +	u8 service_interval_valid;
>> +	u32 service_interval;
>> +};
> 
> Are this and the other structs a part of DSP ABI?
> Or is it a definition only used in kernel?  I'm asking because
> __packed attribute is required for most of ABI definitions with
> different field types.
> 

This would be in the kernel only.

Thanks
Wesley Cheng

Re: [RFC PATCH 00/14] Introduce QC USB SND audio offloading support

[Pierre-Louis Bossart]

On 12/23/22 17:31, Wesley Cheng wrote:
> Several Qualcomm based chipsets can support USB audio offloading to a
> dedicated audio DSP, which can take over issuing transfers to the USB
> host controller.  The intention is to reduce the load on the main
> processors in the SoC, and allow them to be placed into lower power modes.

It would be nice to clarify what you want to offload
a) audio data transfers for isoc ports
b) control for e.g. volume settings (those go to endpoint 0 IIRC)
c) Both?

This has a lot of implications on the design. ASoC/DPCM is mainly
intended for audio data transfers, control is a separate problem with
configurations handled with register settings or bus-specific commands.

> There are several parts to this design:
>   1. Adding ASoC binding layer
>   2. Create a USB backend for Q6DSP
>   3. Introduce XHCI interrupter support
>   4. Create vendor ops for the USB SND driver
> 
> Adding ASoC binding layer:
> soc-usb: Intention is to treat a USB port similar to a headphone jack.
> The port is always present on the device, but cable/pin status can be
> enabled/disabled.  Expose mechanisms for USB backend ASoC drivers to
> communicate with USB SND.
> 
> Create a USB backend for Q6DSP:
> q6usb: Basic backend driver that will be responsible for maintaining the
> resources needed to initiate a playback stream using the Q6DSP.  Will
> be the entity that checks to make sure the connected USB audio device
> supports the requested PCM format.  If it does not, the PCM open call will
> fail, and userpsace ALSA can take action accordingly.
> 
> Introduce XHCI interrupter support:
> XHCI HCD supports multiple interrupters, which allows for events to be routed
> to different event rings.  This is determined by "Interrupter Target" field
> specified in Section "6.4.1.1 Normal TRB" of the XHCI specification.
> 
> Events in the offloading case will be routed to an event ring that is assigned
> to the audio DSP.
> 
> Create vendor ops for the USB SND driver:
> qc_audio_offload: This particular driver has several components associated
> with it:
> - QMI stream request handler
> - XHCI interrupter and resource management
> - audio DSP memory management
> 
> When the audio DSP wants to enable a playback stream, the request is first
> received by the ASoC platform sound card.  Depending on the selected route,
> ASoC will bring up the individual DAIs in the path.  The Q6USB backend DAI
> will send an AFE port start command (with enabling the USB playback path), and
> the audio DSP will handle the request accordingly.
> 
> Part of the AFE USB port start handling will have an exchange of control
> messages using the QMI protocol.  The qc_audio_offload driver will populate the
> buffer information:
> - Event ring base address
> - EP transfer ring base address
> 
> and pass it along to the audio DSP.  All endpoint management will now be handed
> over to the DSP, and the main processor is not involved in transfers.
> 
> Overall, implementing this feature will still expose separate sound card and PCM
> devices for both the platorm card and USB audio device:
>  0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D
>                       SM8250-MTP-WCD9380-WSA8810-VA-DMIC
>  1 [Audio          ]: USB-Audio - USB Audio
>                       Generic USB Audio at usb-xhci-hcd.1.auto-1.4, high speed
> 
> This is to ensure that userspace ALSA entities can decide which route to take
> when executing the audio playback.  In the above, if card#1 is selected, then
> USB audio data will take the legacy path over the USB PCM drivers, etc...

You would still need some sort of mutual exclusion to make sure the isoc
endpoints are not used concurrently by the two cards. Relying on
userspace intelligence to enforce that exclusion is not safe IMHO.

Intel looked at this sort of offload support a while ago and our
directions were very different - for a variety of reasons USB offload is
enabled on Windows platforms but remains a TODO for Linux. Rather than
having two cards, you could have a single card and addition subdevices
that expose the paths through the DSP. The benefits were that there was
a single set of controls that userspace needed to know about, and volume
settings were the same no matter which path you used (legacy or
DSP-optimized paths). That's consistent with the directions to use 'Deep
Buffer' PCM paths for local playback, it's the same idea of reducing
power consumption with optimized routing.

Another point is that there may be cases where the DSP paths are not
available if the DSP memory and MCPS budget is exceeded. In those cases,
the DSP parts needs the ability to notify userspace that the legacy path
should be used.

Another case to handle is that some USB devices can handle way more data
than DSPs can chew, for example Pro audio boxes that can deal with 8ch
192kHz will typically use the legacy paths. Some also handle specific
formats such as DSD over PCM. So it's quite likely that PCM devices for
card0 and card1 above do NOT expose support for the same formats, or put
differently that only a subset of the USB device capabilities are
handled through the DSP.

And last, power optimizations with DSPs typically come from additional
latency helping put the SoC in low-power modes. That's not necessarily
ideal for all usages, e.g. for music recording and mixing I am not
convinced the DSP path would help at all.

> This feature was validated using:
> - tinymix: set/enable the multimedia path to route to USB backend
> - tinyplay: issue playback on platform card

Re: [RFC PATCH 02/14] ASoC: qcom: qdsp6: Introduce USB AFE port to q6dsp

[Pierre-Louis Bossart]

On 12/23/22 17:31, Wesley Cheng wrote:
> The QC ADSP is able to support USB playback and capture, so that the
> main application processor can be placed into lower CPU power modes.  This
> adds the required AFE port configurations and port start command to start
> an audio session.

It would be good to clarify what sort of endpoints can be supported. I
presume the SOF-synchronous case is handled, but how would you deal with
async endpoints with feedback (be it explicit or implicit)?

Note that it's very hard to make the decision not to support async
endpoints, there are quite a few devices that are exposed as async to
work around an obscure legacy issue on Windows but are really
sof-synchronous endpoints that never ask for any change of pace.

> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
> ---
>  .../sound/qcom,q6dsp-lpass-ports.h            |   1 +
>  sound/soc/qcom/qdsp6/q6afe-dai.c              |  47 +++++
>  sound/soc/qcom/qdsp6/q6afe.c                  | 183 ++++++++++++++++++
>  sound/soc/qcom/qdsp6/q6afe.h                  |  46 ++++-
>  sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c      |  23 +++
>  sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h      |   1 +
>  sound/soc/qcom/qdsp6/q6routing.c              |   8 +
>  7 files changed, 308 insertions(+), 1 deletion(-)
> 
> diff --git a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h
> index 9f7c5103bc82..746bc462bb2e 100644
> --- a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h
> +++ b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h
> @@ -131,6 +131,7 @@
>  #define RX_CODEC_DMA_RX_7	126
>  #define QUINARY_MI2S_RX		127
>  #define QUINARY_MI2S_TX		128
> +#define USB_RX				129

the commit message says the DSP can support Playback and capture, but
here there's capture only ...


>  
>  static const struct snd_soc_dapm_route q6afe_dapm_routes[] = {
> +	{"USB Playback", NULL, "USB_RX"},

... but here RX means playback?

I am not sure I get the convention on directions and what is actually
supported?

> +struct afe_param_id_usb_cfg {
> +/* Minor version used for tracking USB audio device configuration.
> + * Supported values: AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG
> + */
> +	u32                  cfg_minor_version;
> +/* Sampling rate of the port.
> + * Supported values:
> + * - AFE_PORT_SAMPLE_RATE_8K
> + * - AFE_PORT_SAMPLE_RATE_11025
> + * - AFE_PORT_SAMPLE_RATE_12K
> + * - AFE_PORT_SAMPLE_RATE_16K
> + * - AFE_PORT_SAMPLE_RATE_22050
> + * - AFE_PORT_SAMPLE_RATE_24K
> + * - AFE_PORT_SAMPLE_RATE_32K
> + * - AFE_PORT_SAMPLE_RATE_44P1K
> + * - AFE_PORT_SAMPLE_RATE_48K
> + * - AFE_PORT_SAMPLE_RATE_96K
> + * - AFE_PORT_SAMPLE_RATE_192K
> + */
> +	u32                  sample_rate;
> +/* Bit width of the sample.
> + * Supported values: 16, 24
> + */
> +	u16                  bit_width;
> +/* Number of channels.
> + * Supported values: 1 and 2

that aligns with my feedback on the cover letter, if you connect a
device that can support from than 2 channels should the DSP even expose
this DSP-optimized path?

Oh and I forgot, what happens if there are multiple audio devices
connected, can the DSP deal with all of them? If not, how is this handled?

> + */
> +	u16                  num_channels;
> +/* Data format supported by the USB. The supported value is
> + * 0 (#AFE_USB_AUDIO_DATA_FORMAT_LINEAR_PCM).
> + */
> +	u16                  data_format;
> +/* this field must be 0 */
> +	u16                  reserved;
> +/* device token of actual end USB aduio device */

typo: audio

> +	u32                  dev_token;
> +/* endianness of this interface */
> +	u32                   endian;

Is this a USB concept? I can't recall having seen any parts of the USB
audio class spec that the data can be big or little endian?

> +/* service interval */
> +	u32                  service_interval;
> +} __packed;

> +int afe_port_send_usb_dev_param(struct q6afe_port *port, struct q6afe_usb_cfg *cfg)
> +{
> +	union afe_port_config *pcfg = &port->port_cfg;
> +	struct afe_param_id_usb_audio_dev_params usb_dev;
> +	struct afe_param_id_usb_audio_dev_lpcm_fmt lpcm_fmt;
> +	struct afe_param_id_usb_audio_svc_interval svc_int;
> +	int ret = 0;
> +
> +	if (!pcfg) {
> +		pr_err("%s: Error, no configuration data\n", __func__);

can you use a dev_err() here and the rest of the code?

> +		ret = -EINVAL;
> +		goto exit;
> +	}
> +
> +	memset(&usb_dev, 0, sizeof(usb_dev));
> +	memset(&lpcm_fmt, 0, sizeof(lpcm_fmt));
> +
> +	usb_dev.cfg_minor_version = AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG;
> +	q6afe_port_set_param_v2(port, &usb_dev,
> +					AFE_PARAM_ID_USB_AUDIO_DEV_PARAMS,
> +					AFE_MODULE_AUDIO_DEV_INTERFACE, sizeof(usb_dev));
> +	if (ret) {
> +		pr_err("%s: AFE device param cmd failed %d\n",
> +			__func__, ret);
> +		goto exit;
> +	}
> +
> +	lpcm_fmt.cfg_minor_version = AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG;
> +	lpcm_fmt.endian = pcfg->usb_cfg.endian;
> +	ret = q6afe_port_set_param_v2(port, &lpcm_fmt,
> +					AFE_PARAM_ID_USB_AUDIO_DEV_LPCM_FMT,
> +					AFE_MODULE_AUDIO_DEV_INTERFACE, sizeof(lpcm_fmt));
> +	if (ret) {
> +		pr_err("%s: AFE device param cmd LPCM_FMT failed %d\n",
> +			__func__, ret);
> +		goto exit;
> +	}
> +
> +	svc_int.cfg_minor_version =
> +		AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG;
> +	svc_int.svc_interval = pcfg->usb_cfg.service_interval;
> +	ret = q6afe_port_set_param_v2(port, &svc_int,
> +					AFE_PARAM_ID_USB_AUDIO_SVC_INTERVAL,
> +					AFE_MODULE_AUDIO_DEV_INTERFACE, sizeof(svc_int));
> +	if (ret) {
> +		pr_err("%s: AFE device param cmd svc_interval failed %d\n",
> +			__func__, ret);
> +		ret = -EINVAL;
> +		goto exit;
> +	}
> +exit:
> +	return ret;
> +}

Re: [RFC PATCH 03/14] ASoC: qcom: Add USB backend ASoC driver for Q6

[Pierre-Louis Bossart]

> +int q6usb_alsa_connection_cb(struct snd_soc_usb *usb, int card_idx,
> +			int connected)
> +{
> +	struct snd_soc_dapm_context *dapm;
> +	struct q6usb_port_data *data;
> +
> +	if (!usb->component)
> +		return 0;
> +
> +	dapm = snd_soc_component_get_dapm(usb->component);
> +	data = dev_get_drvdata(usb->component->dev);
> +
> +	if (connected) {
> +		snd_soc_dapm_enable_pin(dapm, "USB_RX_BE");
> +		/* We only track the latest USB headset plugged in */

that answers to my earlier question on how to deal with multiple
devices, but is this a desirable policy? This could lead to a lot of
confusion. If there are restrictions to a single device, then it might
be more interesting for userspace or the user to indicate which USB
device gets to use USB offload and all others use legacy.

> +		data->active_idx = card_idx;
> +	} else {
> +		snd_soc_dapm_disable_pin(dapm, "USB_RX_BE");
> +	}
> +	snd_soc_dapm_sync(dapm);
> +
> +	return 0;
> +}

Re: [RFC PATCH 09/14] sound: usb: Introduce QC USB SND offloading support

[Pierre-Louis Bossart]

On 12/23/22 17:31, Wesley Cheng wrote:
> Several Qualcomm SoCs have a dedicated audio DSP, which has the ability to
> support USB sound devices.  This vendor driver will implement the required
> handshaking with the DSP, in order to pass along required resources that
> will be utilized by the DSP's USB SW.  The communication channel used for
> this handshaking will be using the QMI protocol.  Required resources
> include:
> - Allocated secondary event ring address
> - EP transfer ring address
> - Interrupter number
> 
> The above information will allow for the audio DSP to execute USB transfers
> over the USB bus.  It will also be able to support devices that have an
> implicit feedback and sync endpoint as well.  Offloading these data
> transfers will allow the main/applications processor to enter lower CPU
> power modes, and sustain a longer duration in those modes.

Are you suggesting that the entire feedback loop be handled in the DSP?
It's not clear what "Offloading these data transfers" refers to, the
data part or the feedback path?

Comments are almost inexistent in this patch so it's hard to figure out
what it really does.

Re: [RFC PATCH 02/14] ASoC: qcom: qdsp6: Introduce USB AFE port to q6dsp

[Krzysztof Kozlowski]

On 24/12/2022 00:31, Wesley Cheng wrote:
> The QC ADSP is able to support USB playback and capture, so that the
> main application processor can be placed into lower CPU power modes.  This
> adds the required AFE port configurations and port start command to start
> an audio session.
> 
> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
> ---
>  .../sound/qcom,q6dsp-lpass-ports.h            |   1 +
>  sound/soc/qcom/qdsp6/q6afe-dai.c              |  47 +++++
>  sound/soc/qcom/qdsp6/q6afe.c                  | 183 ++++++++++++++++++
>  sound/soc/qcom/qdsp6/q6afe.h                  |  46 ++++-
>  sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c      |  23 +++
>  sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h      |   1 +
>  sound/soc/qcom/qdsp6/q6routing.c              |   8 +
>  7 files changed, 308 insertions(+), 1 deletion(-)
> 
> diff --git a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h
> index 9f7c5103bc82..746bc462bb2e 100644
> --- a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h
> +++ b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h
> @@ -131,6 +131,7 @@
>  #define RX_CODEC_DMA_RX_7	126
>  #define QUINARY_MI2S_RX		127
>  #define QUINARY_MI2S_TX		128
> +#define USB_RX				129
>  
>  #define LPASS_CLK_ID_PRI_MI2S_IBIT	1

Bindings are separate patches. Please split.

Best regards,
Krzysztof

Re: [RFC PATCH 14/14] ASoC: dt-bindings: Update example for enabling USB offload on SM8250

[Krzysztof Kozlowski]

On 03/01/2023 18:46, Mark Brown wrote:
> On Mon, Dec 26, 2022 at 01:27:21PM +0100, Krzysztof Kozlowski wrote:
>> On 24/12/2022 00:32, Wesley Cheng wrote:
> 
>>> +            link-name = "USB Playback";
>>> +            cpu {
>>> +                sound-dai = <&q6afedai USB_RX>;
> 
>> Hmm, that makes me wonder if you really tested the bindings before
>> sending? If yes, where is the USB_RX defined?
> 
> It was added in patch 2, it's in include/dt-bindings.

Thanks, indeed, I was looking for another bindings patch but this was
squashed with a driver.

Best regards,
Krzysztof

Re: [RFC PATCH 00/14] Introduce QC USB SND audio offloading support

[Wesley Cheng]

Hi Pierre,

On 1/4/2023 3:19 PM, Pierre-Louis Bossart wrote:
> 
> 
> On 12/23/22 17:31, Wesley Cheng wrote:
>> Several Qualcomm based chipsets can support USB audio offloading to a
>> dedicated audio DSP, which can take over issuing transfers to the USB
>> host controller.  The intention is to reduce the load on the main
>> processors in the SoC, and allow them to be placed into lower power modes.
> 
> It would be nice to clarify what you want to offload
> a) audio data transfers for isoc ports
> b) control for e.g. volume settings (those go to endpoint 0 IIRC)
> c) Both?
> 

Thanks for sharing your experience, and inputs!

It would be the audio related endpoints only, so ISOC and potentially 
feedback ep.

> This has a lot of implications on the design. ASoC/DPCM is mainly
> intended for audio data transfers, control is a separate problem with
> configurations handled with register settings or bus-specific commands.
> 

Control would still be handled by the main processor.

>> There are several parts to this design:
>>    1. Adding ASoC binding layer
>>    2. Create a USB backend for Q6DSP
>>    3. Introduce XHCI interrupter support
>>    4. Create vendor ops for the USB SND driver
>>
>> Adding ASoC binding layer:
>> soc-usb: Intention is to treat a USB port similar to a headphone jack.
>> The port is always present on the device, but cable/pin status can be
>> enabled/disabled.  Expose mechanisms for USB backend ASoC drivers to
>> communicate with USB SND.
>>
>> Create a USB backend for Q6DSP:
>> q6usb: Basic backend driver that will be responsible for maintaining the
>> resources needed to initiate a playback stream using the Q6DSP.  Will
>> be the entity that checks to make sure the connected USB audio device
>> supports the requested PCM format.  If it does not, the PCM open call will
>> fail, and userpsace ALSA can take action accordingly.
>>
>> Introduce XHCI interrupter support:
>> XHCI HCD supports multiple interrupters, which allows for events to be routed
>> to different event rings.  This is determined by "Interrupter Target" field
>> specified in Section "6.4.1.1 Normal TRB" of the XHCI specification.
>>
>> Events in the offloading case will be routed to an event ring that is assigned
>> to the audio DSP.
>>
>> Create vendor ops for the USB SND driver:
>> qc_audio_offload: This particular driver has several components associated
>> with it:
>> - QMI stream request handler
>> - XHCI interrupter and resource management
>> - audio DSP memory management
>>
>> When the audio DSP wants to enable a playback stream, the request is first
>> received by the ASoC platform sound card.  Depending on the selected route,
>> ASoC will bring up the individual DAIs in the path.  The Q6USB backend DAI
>> will send an AFE port start command (with enabling the USB playback path), and
>> the audio DSP will handle the request accordingly.
>>
>> Part of the AFE USB port start handling will have an exchange of control
>> messages using the QMI protocol.  The qc_audio_offload driver will populate the
>> buffer information:
>> - Event ring base address
>> - EP transfer ring base address
>>
>> and pass it along to the audio DSP.  All endpoint management will now be handed
>> over to the DSP, and the main processor is not involved in transfers.
>>
>> Overall, implementing this feature will still expose separate sound card and PCM
>> devices for both the platorm card and USB audio device:
>>   0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D
>>                        SM8250-MTP-WCD9380-WSA8810-VA-DMIC
>>   1 [Audio          ]: USB-Audio - USB Audio
>>                        Generic USB Audio at usb-xhci-hcd.1.auto-1.4, high speed
>>
>> This is to ensure that userspace ALSA entities can decide which route to take
>> when executing the audio playback.  In the above, if card#1 is selected, then
>> USB audio data will take the legacy path over the USB PCM drivers, etc...
> 
> You would still need some sort of mutual exclusion to make sure the isoc
> endpoints are not used concurrently by the two cards. Relying on
> userspace intelligence to enforce that exclusion is not safe IMHO.
> 

Sure, I think we can make the USB card as being used if the offloading 
path is currently being enabled.  Kernel could return an error to 
userspace when this situation happens.

> Intel looked at this sort of offload support a while ago and our
> directions were very different - for a variety of reasons USB offload is
> enabled on Windows platforms but remains a TODO for Linux. Rather than
> having two cards, you could have a single card and addition subdevices
> that expose the paths through the DSP. The benefits were that there was
> a single set of controls that userspace needed to know about, and volume
> settings were the same no matter which path you used (legacy or
> DSP-optimized paths). That's consistent with the directions to use 'Deep
> Buffer' PCM paths for local playback, it's the same idea of reducing
> power consumption with optimized routing.
> 

Volume control would still be done through the legacy path as mentioned 
above.  For example, if a USB headset w/ a HID interface exposed (for 
volume control) was connected, those HID events would be routed to 
userspace to adjust volume accordingly on the main processor. (although 
you're right about having separate controls still present - one for the 
ASoC card and another for USB card)

> Another point is that there may be cases where the DSP paths are not
> available if the DSP memory and MCPS budget is exceeded. In those cases,
> the DSP parts needs the ability to notify userspace that the legacy path
> should be used.

If we ran into this scenario, the audio DSP AFE port start command would 
fail, and this would be propagated to the userspace entity.  It could 
then potentially re-route to the legacy/non-offload path.

> 
> Another case to handle is that some USB devices can handle way more data
> than DSPs can chew, for example Pro audio boxes that can deal with 8ch
> 192kHz will typically use the legacy paths. Some also handle specific
> formats such as DSD over PCM. So it's quite likely that PCM devices for
> card0 and card1 above do NOT expose support for the same formats, or put
> differently that only a subset of the USB device capabilities are
> handled through the DSP.

Same as the above.  We have programmed the USB backend to support the 
profiles that the audio DSP can handle.  I assume if there was any other 
request, the userspace entity would fail the PCM open for that requested 
profile.

> 
> And last, power optimizations with DSPs typically come from additional
> latency helping put the SoC in low-power modes. That's not necessarily
> ideal for all usages, e.g. for music recording and mixing I am not
> convinced the DSP path would help at all.
> 

That's true.  At the same time, this feature is more for power related 
benefits, not specifically for performance. (although we haven't seen 
any performance related issues w/ this approach on the audio profiles 
the DSP supports)  I think if its an audio profile that supports a high 
sample rate and large number of channels, then the DSP wouldn't be able 
to support it anyway, and userspace could still use the legacy path. 
This would allow for those high-performance audio devices to not be 
affected.

Thanks
Wesley Cheng

Re: [RFC PATCH 02/14] ASoC: qcom: qdsp6: Introduce USB AFE port to q6dsp

[Wesley Cheng]

Hi Pierre,

On 1/4/2023 3:33 PM, Pierre-Louis Bossart wrote:
> 
> 
> On 12/23/22 17:31, Wesley Cheng wrote:
>> The QC ADSP is able to support USB playback and capture, so that the
>> main application processor can be placed into lower CPU power modes.  This
>> adds the required AFE port configurations and port start command to start
>> an audio session.
> 
> It would be good to clarify what sort of endpoints can be supported. I
> presume the SOF-synchronous case is handled, but how would you deal with
> async endpoints with feedback (be it explicit or implicit)?
> 

Sure, both types of feedback endpoints are expected to be supported by 
the audio DSP, as well as sync eps.  We have the logic there to modify 
the audio sample size accordingly.

> Note that it's very hard to make the decision not to support async
> endpoints, there are quite a few devices that are exposed as async to
> work around an obscure legacy issue on Windows but are really
> sof-synchronous endpoints that never ask for any change of pace.
> 
>> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
>> ---
>>   .../sound/qcom,q6dsp-lpass-ports.h            |   1 +
>>   sound/soc/qcom/qdsp6/q6afe-dai.c              |  47 +++++
>>   sound/soc/qcom/qdsp6/q6afe.c                  | 183 ++++++++++++++++++
>>   sound/soc/qcom/qdsp6/q6afe.h                  |  46 ++++-
>>   sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c      |  23 +++
>>   sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h      |   1 +
>>   sound/soc/qcom/qdsp6/q6routing.c              |   8 +
>>   7 files changed, 308 insertions(+), 1 deletion(-)
>>
>> diff --git a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h
>> index 9f7c5103bc82..746bc462bb2e 100644
>> --- a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h
>> +++ b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h
>> @@ -131,6 +131,7 @@
>>   #define RX_CODEC_DMA_RX_7	126
>>   #define QUINARY_MI2S_RX		127
>>   #define QUINARY_MI2S_TX		128
>> +#define USB_RX				129
> 
> the commit message says the DSP can support Playback and capture, but
> here there's capture only ...
> 
> 

Sorry I will update the commit message properly.  At the moment we've 
only verified audio playback.

>>   
>>   static const struct snd_soc_dapm_route q6afe_dapm_routes[] = {
>> +	{"USB Playback", NULL, "USB_RX"},
> 
> ... but here RX means playback?
> 
> I am not sure I get the convention on directions and what is actually
> supported?
> 

The notation is based on the direction of which the audio data is 
sourced or pushed on to the DSP.  So in playback, the DSP is receiving 
audio data to send, and capture, it is transmitting audio data received. 
(although we do not support capture yet)

>> +struct afe_param_id_usb_cfg {
>> +/* Minor version used for tracking USB audio device configuration.
>> + * Supported values: AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG
>> + */
>> +	u32                  cfg_minor_version;
>> +/* Sampling rate of the port.
>> + * Supported values:
>> + * - AFE_PORT_SAMPLE_RATE_8K
>> + * - AFE_PORT_SAMPLE_RATE_11025
>> + * - AFE_PORT_SAMPLE_RATE_12K
>> + * - AFE_PORT_SAMPLE_RATE_16K
>> + * - AFE_PORT_SAMPLE_RATE_22050
>> + * - AFE_PORT_SAMPLE_RATE_24K
>> + * - AFE_PORT_SAMPLE_RATE_32K
>> + * - AFE_PORT_SAMPLE_RATE_44P1K
>> + * - AFE_PORT_SAMPLE_RATE_48K
>> + * - AFE_PORT_SAMPLE_RATE_96K
>> + * - AFE_PORT_SAMPLE_RATE_192K
>> + */
>> +	u32                  sample_rate;
>> +/* Bit width of the sample.
>> + * Supported values: 16, 24
>> + */
>> +	u16                  bit_width;
>> +/* Number of channels.
>> + * Supported values: 1 and 2
> 
> that aligns with my feedback on the cover letter, if you connect a
> device that can support from than 2 channels should the DSP even expose
> this DSP-optimized path?
> 

My assumption is that I programmed the DAIs w/ PCM formats supported by 
the DSP, so I think the ASoC core should not allow userspace to choose 
that path if the hw params don't fit/match.

> Oh and I forgot, what happens if there are multiple audio devices
> connected, can the DSP deal with all of them? If not, how is this handled?
> 

This is one topic that we were pretty open ended on.  At least on our 
implementation, only one audio device can be supported at a time.  We 
choose the latest device that was plugged in or discovered by the USB 
SND class driver.

>> + */
>> +	u16                  num_channels;
>> +/* Data format supported by the USB. The supported value is
>> + * 0 (#AFE_USB_AUDIO_DATA_FORMAT_LINEAR_PCM).
>> + */
>> +	u16                  data_format;
>> +/* this field must be 0 */
>> +	u16                  reserved;
>> +/* device token of actual end USB aduio device */
> 
> typo: audio
> 

Thanks

>> +	u32                  dev_token;
>> +/* endianness of this interface */
>> +	u32                   endian;
> 
> Is this a USB concept? I can't recall having seen any parts of the USB
> audio class spec that the data can be big or little endian?
> 

No, this is probably just something our audio DSP uses on the AFE 
commands that it receives.

>> +/* service interval */
>> +	u32                  service_interval;
>> +} __packed;
> 
>> +int afe_port_send_usb_dev_param(struct q6afe_port *port, struct q6afe_usb_cfg *cfg)
>> +{
>> +	union afe_port_config *pcfg = &port->port_cfg;
>> +	struct afe_param_id_usb_audio_dev_params usb_dev;
>> +	struct afe_param_id_usb_audio_dev_lpcm_fmt lpcm_fmt;
>> +	struct afe_param_id_usb_audio_svc_interval svc_int;
>> +	int ret = 0;
>> +
>> +	if (!pcfg) {
>> +		pr_err("%s: Error, no configuration data\n", __func__);
> 
> can you use a dev_err() here and the rest of the code?
> 

Sure.

Thanks
Wesley Cheng

Re: [RFC PATCH 03/14] ASoC: qcom: Add USB backend ASoC driver for Q6

[Wesley Cheng]

Hi Pierre,

On 1/4/2023 3:41 PM, Pierre-Louis Bossart wrote:
> 
>> +int q6usb_alsa_connection_cb(struct snd_soc_usb *usb, int card_idx,
>> +			int connected)
>> +{
>> +	struct snd_soc_dapm_context *dapm;
>> +	struct q6usb_port_data *data;
>> +
>> +	if (!usb->component)
>> +		return 0;
>> +
>> +	dapm = snd_soc_component_get_dapm(usb->component);
>> +	data = dev_get_drvdata(usb->component->dev);
>> +
>> +	if (connected) {
>> +		snd_soc_dapm_enable_pin(dapm, "USB_RX_BE");
>> +		/* We only track the latest USB headset plugged in */
> 
> that answers to my earlier question on how to deal with multiple
> devices, but is this a desirable policy? This could lead to a lot of
> confusion. If there are restrictions to a single device, then it might
> be more interesting for userspace or the user to indicate which USB
> device gets to use USB offload and all others use legacy.
> 

Yeah, as mentioned its still pretty open ended.  I think from the 
feedback received from Mark/Takashi, this was a viable option for now. 
Would you happen to have any insight/input on how the userspace can pass 
down that selection to the ASoC framework?  Maybe some kind of PCM IOCTL 
call?

Thanks
Wesley Cheng

Re: [RFC PATCH 09/14] sound: usb: Introduce QC USB SND offloading support

[Wesley Cheng]

Hi Pierre,

On 1/4/2023 3:51 PM, Pierre-Louis Bossart wrote:
> 
> 
> On 12/23/22 17:31, Wesley Cheng wrote:
>> Several Qualcomm SoCs have a dedicated audio DSP, which has the ability to
>> support USB sound devices.  This vendor driver will implement the required
>> handshaking with the DSP, in order to pass along required resources that
>> will be utilized by the DSP's USB SW.  The communication channel used for
>> this handshaking will be using the QMI protocol.  Required resources
>> include:
>> - Allocated secondary event ring address
>> - EP transfer ring address
>> - Interrupter number
>>
>> The above information will allow for the audio DSP to execute USB transfers
>> over the USB bus.  It will also be able to support devices that have an
>> implicit feedback and sync endpoint as well.  Offloading these data
>> transfers will allow the main/applications processor to enter lower CPU
>> power modes, and sustain a longer duration in those modes.
> 
> Are you suggesting that the entire feedback loop be handled in the DSP?
> It's not clear what "Offloading these data transfers" refers to, the
> data part or the feedback path?
> 

Yes, as mentioned in the cover letter response, we'll handle the 
feedback endpoints. (feedback eps are part of the audio data interface)

> Comments are almost inexistent in this patch so it's hard to figure out
> what it really does.
> 

OK, will add some more comments.

Thanks
Wesley Cheng

Re: [RFC PATCH 02/14] ASoC: qcom: qdsp6: Introduce USB AFE port to q6dsp

[Wesley Cheng]

Hi Krzysztof,

On 1/5/2023 10:09 AM, Krzysztof Kozlowski wrote:
> On 24/12/2022 00:31, Wesley Cheng wrote:
>> The QC ADSP is able to support USB playback and capture, so that the
>> main application processor can be placed into lower CPU power modes.  This
>> adds the required AFE port configurations and port start command to start
>> an audio session.
>>
>> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com>
>> ---
>>   .../sound/qcom,q6dsp-lpass-ports.h            |   1 +
>>   sound/soc/qcom/qdsp6/q6afe-dai.c              |  47 +++++
>>   sound/soc/qcom/qdsp6/q6afe.c                  | 183 ++++++++++++++++++
>>   sound/soc/qcom/qdsp6/q6afe.h                  |  46 ++++-
>>   sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c      |  23 +++
>>   sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h      |   1 +
>>   sound/soc/qcom/qdsp6/q6routing.c              |   8 +
>>   7 files changed, 308 insertions(+), 1 deletion(-)
>>
>> diff --git a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h
>> index 9f7c5103bc82..746bc462bb2e 100644
>> --- a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h
>> +++ b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h
>> @@ -131,6 +131,7 @@
>>   #define RX_CODEC_DMA_RX_7	126
>>   #define QUINARY_MI2S_RX		127
>>   #define QUINARY_MI2S_TX		128
>> +#define USB_RX				129
>>   
>>   #define LPASS_CLK_ID_PRI_MI2S_IBIT	1
> 
> Bindings are separate patches. Please split.
> 

Thanks for catching this.  Will do.

Thanks
Wesley Cheng

Re: [RFC PATCH 00/14] Introduce QC USB SND audio offloading support

[Pierre-Louis Bossart]

>> On 12/23/22 17:31, Wesley Cheng wrote:
>>> Several Qualcomm based chipsets can support USB audio offloading to a
>>> dedicated audio DSP, which can take over issuing transfers to the USB
>>> host controller.  The intention is to reduce the load on the main
>>> processors in the SoC, and allow them to be placed into lower power
>>> modes.
>>
>> It would be nice to clarify what you want to offload
>> a) audio data transfers for isoc ports
>> b) control for e.g. volume settings (those go to endpoint 0 IIRC)
>> c) Both?
>>
> 
> Thanks for sharing your experience, and inputs!
> 
> It would be the audio related endpoints only, so ISOC and potentially
> feedback ep.

That's good, that means there's a common basis for at least two separate
hardware implementations.
>> This has a lot of implications on the design. ASoC/DPCM is mainly
>> intended for audio data transfers, control is a separate problem with
>> configurations handled with register settings or bus-specific commands.
>>
> 
> Control would still be handled by the main processor.

Excellent, one more thing in common. Maintainers like this sort of
alignment :-)

>>> There are several parts to this design:
>>>    1. Adding ASoC binding layer
>>>    2. Create a USB backend for Q6DSP
>>>    3. Introduce XHCI interrupter support
>>>    4. Create vendor ops for the USB SND driver
>>>
>>> Adding ASoC binding layer:
>>> soc-usb: Intention is to treat a USB port similar to a headphone jack.
>>> The port is always present on the device, but cable/pin status can be
>>> enabled/disabled.  Expose mechanisms for USB backend ASoC drivers to
>>> communicate with USB SND.
>>>
>>> Create a USB backend for Q6DSP:
>>> q6usb: Basic backend driver that will be responsible for maintaining the
>>> resources needed to initiate a playback stream using the Q6DSP.  Will
>>> be the entity that checks to make sure the connected USB audio device
>>> supports the requested PCM format.  If it does not, the PCM open call
>>> will
>>> fail, and userpsace ALSA can take action accordingly.
>>>
>>> Introduce XHCI interrupter support:
>>> XHCI HCD supports multiple interrupters, which allows for events to
>>> be routed
>>> to different event rings.  This is determined by "Interrupter Target"
>>> field
>>> specified in Section "6.4.1.1 Normal TRB" of the XHCI specification.
>>>
>>> Events in the offloading case will be routed to an event ring that is
>>> assigned
>>> to the audio DSP.

To the best of my knowledge this isn't needed on Intel platforms, but
that's something we would need to double-check.
>>> Create vendor ops for the USB SND driver:
>>> qc_audio_offload: This particular driver has several components
>>> associated
>>> with it:
>>> - QMI stream request handler
>>> - XHCI interrupter and resource management
>>> - audio DSP memory management
>>>
>>> When the audio DSP wants to enable a playback stream, the request is
>>> first
>>> received by the ASoC platform sound card.  Depending on the selected
>>> route,
>>> ASoC will bring up the individual DAIs in the path.  The Q6USB
>>> backend DAI
>>> will send an AFE port start command (with enabling the USB playback
>>> path), and
>>> the audio DSP will handle the request accordingly.
>>>
>>> Part of the AFE USB port start handling will have an exchange of control
>>> messages using the QMI protocol.  The qc_audio_offload driver will
>>> populate the
>>> buffer information:
>>> - Event ring base address
>>> - EP transfer ring base address
>>>
>>> and pass it along to the audio DSP.  All endpoint management will now
>>> be handed
>>> over to the DSP, and the main processor is not involved in transfers.
>>>
>>> Overall, implementing this feature will still expose separate sound
>>> card and PCM
>>> devices for both the platorm card and USB audio device:
>>>   0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D
>>>                        SM8250-MTP-WCD9380-WSA8810-VA-DMIC
>>>   1 [Audio          ]: USB-Audio - USB Audio
>>>                        Generic USB Audio at usb-xhci-hcd.1.auto-1.4,
>>> high speed
>>>
>>> This is to ensure that userspace ALSA entities can decide which route
>>> to take
>>> when executing the audio playback.  In the above, if card#1 is
>>> selected, then
>>> USB audio data will take the legacy path over the USB PCM drivers,
>>> etc...
>>
>> You would still need some sort of mutual exclusion to make sure the isoc
>> endpoints are not used concurrently by the two cards. Relying on
>> userspace intelligence to enforce that exclusion is not safe IMHO.
>>
> 
> Sure, I think we can make the USB card as being used if the offloading
> path is currently being enabled.  Kernel could return an error to
> userspace when this situation happens.

It's problematic for servers such as PipeWire/PulseAudio that open all
possible PCMs to figure out what they support in terms of formats. I am
not sure we can enforce a user-space serialization when discovering
capabilities?

> 
>> Intel looked at this sort of offload support a while ago and our
>> directions were very different - for a variety of reasons USB offload is
>> enabled on Windows platforms but remains a TODO for Linux. Rather than
>> having two cards, you could have a single card and addition subdevices
>> that expose the paths through the DSP. The benefits were that there was
>> a single set of controls that userspace needed to know about, and volume
>> settings were the same no matter which path you used (legacy or
>> DSP-optimized paths). That's consistent with the directions to use 'Deep
>> Buffer' PCM paths for local playback, it's the same idea of reducing
>> power consumption with optimized routing.
>>
> 
> Volume control would still be done through the legacy path as mentioned
> above.  For example, if a USB headset w/ a HID interface exposed (for
> volume control) was connected, those HID events would be routed to
> userspace to adjust volume accordingly on the main processor. (although
> you're right about having separate controls still present - one for the
> ASoC card and another for USB card)

The two sets of controls implied by the use of two cards is really
problematic IMHO. This adds complexity for userspace to figure out that
the controls are really the same and synchronize/mirror changes.

The premise of offload is that it should really not get in the way of
user-experience, design constructs that result in delayed starts/stop,
changed volumes or quality differences should be avoided, or
users/distros will disable this optimization.

One card with additional DSP-based PCM devices seems simpler to me in
terms of usage, but it's not without technical challenges either: with
the use of the ASoC topology framework we only know what the DSP
supports when registering a card and probing the ASoC components.

The interaction between USB audio and ASoC would also be at least as
complicated as display audio, in that it needs to work no matter what
the probe order is, and even survive the Linux device/driver model
requirement that there are no timing dependencies in the driver
bind/unbind sequences.

>> Another point is that there may be cases where the DSP paths are not
>> available if the DSP memory and MCPS budget is exceeded. In those cases,
>> the DSP parts needs the ability to notify userspace that the legacy path
>> should be used.
> 
> If we ran into this scenario, the audio DSP AFE port start command would
> fail, and this would be propagated to the userspace entity.  It could
> then potentially re-route to the legacy/non-offload path.

'start' or 'open'? This is a rather important design difference. Usually
we try to make decisions in the .open or .hw_params stage. The 'start'
or 'trigger' are usually not meant to fail due to unavailable resources
in ALSA.
>> Another case to handle is that some USB devices can handle way more data
>> than DSPs can chew, for example Pro audio boxes that can deal with 8ch
>> 192kHz will typically use the legacy paths. Some also handle specific
>> formats such as DSD over PCM. So it's quite likely that PCM devices for
>> card0 and card1 above do NOT expose support for the same formats, or put
>> differently that only a subset of the USB device capabilities are
>> handled through the DSP.
> 
> Same as the above.  We have programmed the USB backend to support the
> profiles that the audio DSP can handle.  I assume if there was any other
> request, the userspace entity would fail the PCM open for that requested
> profile.

What's not clear to me is whether there's any matching process between
the DSP capabilities and what the USB device exposes? if the USB device
is already way more complicated that what the ASoC back-end can deal
with, why expose a card?

>> And last, power optimizations with DSPs typically come from additional
>> latency helping put the SoC in low-power modes. That's not necessarily
>> ideal for all usages, e.g. for music recording and mixing I am not
>> convinced the DSP path would help at all.
>>
> 
> That's true.  At the same time, this feature is more for power related
> benefits, not specifically for performance. (although we haven't seen
> any performance related issues w/ this approach on the audio profiles
> the DSP supports)  I think if its an audio profile that supports a high
> sample rate and large number of channels, then the DSP wouldn't be able
> to support it anyway, and userspace could still use the legacy path.
> This would allow for those high-performance audio devices to not be
> affected.

ok, we are aligned as well here. Excellent. With the on-going work to
introduce 'Deep Buffer' capabilities, we'll have a need to tag PCM
devices with a usage or 'modifier', or have this information in
UCM/topology. Logic will have to be introduced in userspace to use the
best routing, I would think this work can be reused for USB cases to
indicate the offload solution is geared to power optimizations.

Re: [RFC PATCH 02/14] ASoC: qcom: qdsp6: Introduce USB AFE port to q6dsp

[Pierre-Louis Bossart]

>>> The QC ADSP is able to support USB playback and capture, so that the
>>> main application processor can be placed into lower CPU power modes. 
>>> This
>>> adds the required AFE port configurations and port start command to
>>> start
>>> an audio session.
>>
>> It would be good to clarify what sort of endpoints can be supported. I
>> presume the SOF-synchronous case is handled, but how would you deal with
>> async endpoints with feedback (be it explicit or implicit)?
>>
> 
> Sure, both types of feedback endpoints are expected to be supported by
> the audio DSP, as well as sync eps.  We have the logic there to modify
> the audio sample size accordingly.

did you mean modify samples per USB frame (or uframe), so as to change
the pace at which data is transferred? If yes it'd be the same for Intel.

>>>     static const struct snd_soc_dapm_route q6afe_dapm_routes[] = {
>>> +    {"USB Playback", NULL, "USB_RX"},
>>
>> ... but here RX means playback?
>>
>> I am not sure I get the convention on directions and what is actually
>> supported?
>>
> 
> The notation is based on the direction of which the audio data is
> sourced or pushed on to the DSP.  So in playback, the DSP is receiving
> audio data to send, and capture, it is transmitting audio data received.
> (although we do not support capture yet)

ok, it'd be good to add a comment on this convention. Usually RX/TX is
bus-centric.

> 
>>> +struct afe_param_id_usb_cfg {
>>> +/* Minor version used for tracking USB audio device configuration.
>>> + * Supported values: AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG
>>> + */
>>> +    u32                  cfg_minor_version;
>>> +/* Sampling rate of the port.
>>> + * Supported values:
>>> + * - AFE_PORT_SAMPLE_RATE_8K
>>> + * - AFE_PORT_SAMPLE_RATE_11025
>>> + * - AFE_PORT_SAMPLE_RATE_12K
>>> + * - AFE_PORT_SAMPLE_RATE_16K
>>> + * - AFE_PORT_SAMPLE_RATE_22050
>>> + * - AFE_PORT_SAMPLE_RATE_24K
>>> + * - AFE_PORT_SAMPLE_RATE_32K
>>> + * - AFE_PORT_SAMPLE_RATE_44P1K
>>> + * - AFE_PORT_SAMPLE_RATE_48K
>>> + * - AFE_PORT_SAMPLE_RATE_96K
>>> + * - AFE_PORT_SAMPLE_RATE_192K
>>> + */
>>> +    u32                  sample_rate;
>>> +/* Bit width of the sample.
>>> + * Supported values: 16, 24
>>> + */
>>> +    u16                  bit_width;
>>> +/* Number of channels.
>>> + * Supported values: 1 and 2
>>
>> that aligns with my feedback on the cover letter, if you connect a
>> device that can support from than 2 channels should the DSP even expose
>> this DSP-optimized path?
>>
> 
> My assumption is that I programmed the DAIs w/ PCM formats supported by
> the DSP, so I think the ASoC core should not allow userspace to choose
> that path if the hw params don't fit/match.

Right, but the point I was trying to make is that if the device can do
more, why create this DSP path at all?

> 
>> Oh and I forgot, what happens if there are multiple audio devices
>> connected, can the DSP deal with all of them? If not, how is this
>> handled?
>>
> 
> This is one topic that we were pretty open ended on.  At least on our
> implementation, only one audio device can be supported at a time.  We
> choose the latest device that was plugged in or discovered by the USB
> SND class driver.

Similar case for Intel. I have to revisit this, I don't recall the details.

>>> +    u32                  dev_token;
>>> +/* endianness of this interface */
>>> +    u32                   endian;
>>
>> Is this a USB concept? I can't recall having seen any parts of the USB
>> audio class spec that the data can be big or little endian?
>>
> 
> No, this is probably just something our audio DSP uses on the AFE
> commands that it receives.

ok.

Re: [RFC PATCH 03/14] ASoC: qcom: Add USB backend ASoC driver for Q6

[Pierre-Louis Bossart]

On 1/5/23 19:05, Wesley Cheng wrote:
> Hi Pierre,
> 
> On 1/4/2023 3:41 PM, Pierre-Louis Bossart wrote:
>>
>>> +int q6usb_alsa_connection_cb(struct snd_soc_usb *usb, int card_idx,
>>> +            int connected)
>>> +{
>>> +    struct snd_soc_dapm_context *dapm;
>>> +    struct q6usb_port_data *data;
>>> +
>>> +    if (!usb->component)
>>> +        return 0;
>>> +
>>> +    dapm = snd_soc_component_get_dapm(usb->component);
>>> +    data = dev_get_drvdata(usb->component->dev);
>>> +
>>> +    if (connected) {
>>> +        snd_soc_dapm_enable_pin(dapm, "USB_RX_BE");
>>> +        /* We only track the latest USB headset plugged in */
>>
>> that answers to my earlier question on how to deal with multiple
>> devices, but is this a desirable policy? This could lead to a lot of
>> confusion. If there are restrictions to a single device, then it might
>> be more interesting for userspace or the user to indicate which USB
>> device gets to use USB offload and all others use legacy.
>>
> 
> Yeah, as mentioned its still pretty open ended.  I think from the
> feedback received from Mark/Takashi, this was a viable option for now.
> Would you happen to have any insight/input on how the userspace can pass
> down that selection to the ASoC framework?  Maybe some kind of PCM IOCTL
> call?

I don't have a turn-key solution either :-)
We'd need userspace to make one device as 'preferred' or 'optimized' and
give it a priority somehow. It can't be a PCM IOCTL, it has to be at the
device level.

It's really a second-level optimization that can be parked for now, the
bulk of the work is really the interaction between USB audio and ASoC
stacks, we should probably focus on that BIG topic with a design that
can be shared across implementations.

Re: [RFC PATCH 00/14] Introduce QC USB SND audio offloading support

[Wesley Cheng]

Hi Pierre,

On 1/6/2023 7:57 AM, Pierre-Louis Bossart wrote:
> 
>>> On 12/23/22 17:31, Wesley Cheng wrote:
>>>> Several Qualcomm based chipsets can support USB audio offloading to a
>>>> dedicated audio DSP, which can take over issuing transfers to the USB
>>>> host controller.  The intention is to reduce the load on the main
>>>> processors in the SoC, and allow them to be placed into lower power
>>>> modes.
>>>
>>> It would be nice to clarify what you want to offload
>>> a) audio data transfers for isoc ports
>>> b) control for e.g. volume settings (those go to endpoint 0 IIRC)
>>> c) Both?
>>>
>>
>> Thanks for sharing your experience, and inputs!
>>
>> It would be the audio related endpoints only, so ISOC and potentially
>> feedback ep.
> 
> That's good, that means there's a common basis for at least two separate
> hardware implementations.
>>> This has a lot of implications on the design. ASoC/DPCM is mainly
>>> intended for audio data transfers, control is a separate problem with
>>> configurations handled with register settings or bus-specific commands.
>>>
>>
>> Control would still be handled by the main processor.
> 
> Excellent, one more thing in common. Maintainers like this sort of
> alignment :-)
> 
>>>> There are several parts to this design:
>>>>     1. Adding ASoC binding layer
>>>>     2. Create a USB backend for Q6DSP
>>>>     3. Introduce XHCI interrupter support
>>>>     4. Create vendor ops for the USB SND driver
>>>>
>>>> Adding ASoC binding layer:
>>>> soc-usb: Intention is to treat a USB port similar to a headphone jack.
>>>> The port is always present on the device, but cable/pin status can be
>>>> enabled/disabled.  Expose mechanisms for USB backend ASoC drivers to
>>>> communicate with USB SND.
>>>>
>>>> Create a USB backend for Q6DSP:
>>>> q6usb: Basic backend driver that will be responsible for maintaining the
>>>> resources needed to initiate a playback stream using the Q6DSP.  Will
>>>> be the entity that checks to make sure the connected USB audio device
>>>> supports the requested PCM format.  If it does not, the PCM open call
>>>> will
>>>> fail, and userpsace ALSA can take action accordingly.
>>>>
>>>> Introduce XHCI interrupter support:
>>>> XHCI HCD supports multiple interrupters, which allows for events to
>>>> be routed
>>>> to different event rings.  This is determined by "Interrupter Target"
>>>> field
>>>> specified in Section "6.4.1.1 Normal TRB" of the XHCI specification.
>>>>
>>>> Events in the offloading case will be routed to an event ring that is
>>>> assigned
>>>> to the audio DSP.
> 
> To the best of my knowledge this isn't needed on Intel platforms, but
> that's something we would need to double-check.

I think Mathias mentioned that he was looking into adding some XHCI 
secondary interrupter support as well.  However, it did have some 
slightly different requirements compared to what this offloading feature 
is trying to do.

I'll first have to split up the XHCI/HCD changes into separate parts 
(interrupter specific and offloading specific), and then I'll work with 
him to see what can be improved from there.

>>>> Create vendor ops for the USB SND driver:
>>>> qc_audio_offload: This particular driver has several components
>>>> associated
>>>> with it:
>>>> - QMI stream request handler
>>>> - XHCI interrupter and resource management
>>>> - audio DSP memory management
>>>>
>>>> When the audio DSP wants to enable a playback stream, the request is
>>>> first
>>>> received by the ASoC platform sound card.  Depending on the selected
>>>> route,
>>>> ASoC will bring up the individual DAIs in the path.  The Q6USB
>>>> backend DAI
>>>> will send an AFE port start command (with enabling the USB playback
>>>> path), and
>>>> the audio DSP will handle the request accordingly.
>>>>
>>>> Part of the AFE USB port start handling will have an exchange of control
>>>> messages using the QMI protocol.  The qc_audio_offload driver will
>>>> populate the
>>>> buffer information:
>>>> - Event ring base address
>>>> - EP transfer ring base address
>>>>
>>>> and pass it along to the audio DSP.  All endpoint management will now
>>>> be handed
>>>> over to the DSP, and the main processor is not involved in transfers.
>>>>
>>>> Overall, implementing this feature will still expose separate sound
>>>> card and PCM
>>>> devices for both the platorm card and USB audio device:
>>>>    0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D
>>>>                         SM8250-MTP-WCD9380-WSA8810-VA-DMIC
>>>>    1 [Audio          ]: USB-Audio - USB Audio
>>>>                         Generic USB Audio at usb-xhci-hcd.1.auto-1.4,
>>>> high speed
>>>>
>>>> This is to ensure that userspace ALSA entities can decide which route
>>>> to take
>>>> when executing the audio playback.  In the above, if card#1 is
>>>> selected, then
>>>> USB audio data will take the legacy path over the USB PCM drivers,
>>>> etc...
>>>
>>> You would still need some sort of mutual exclusion to make sure the isoc
>>> endpoints are not used concurrently by the two cards. Relying on
>>> userspace intelligence to enforce that exclusion is not safe IMHO.
>>>
>>
>> Sure, I think we can make the USB card as being used if the offloading
>> path is currently being enabled.  Kernel could return an error to
>> userspace when this situation happens.
> 
> It's problematic for servers such as PipeWire/PulseAudio that open all
> possible PCMs to figure out what they support in terms of formats. I am
> not sure we can enforce a user-space serialization when discovering
> capabilities?
> 

I see...I'm not too familiar yet with all the different implementations 
from userspace yet, so that is something I'll need to look up on the 
side.  Takashi, would you happen to have any inputs with regards to how 
flexible PCM device selection can be from the userspace level?  If the 
offload PCM device can't be supported, can it fallback to another PCM 
device?

>>
>>> Intel looked at this sort of offload support a while ago and our
>>> directions were very different - for a variety of reasons USB offload is
>>> enabled on Windows platforms but remains a TODO for Linux. Rather than
>>> having two cards, you could have a single card and addition subdevices
>>> that expose the paths through the DSP. The benefits were that there was
>>> a single set of controls that userspace needed to know about, and volume
>>> settings were the same no matter which path you used (legacy or
>>> DSP-optimized paths). That's consistent with the directions to use 'Deep
>>> Buffer' PCM paths for local playback, it's the same idea of reducing
>>> power consumption with optimized routing.
>>>
>>
>> Volume control would still be done through the legacy path as mentioned
>> above.  For example, if a USB headset w/ a HID interface exposed (for
>> volume control) was connected, those HID events would be routed to
>> userspace to adjust volume accordingly on the main processor. (although
>> you're right about having separate controls still present - one for the
>> ASoC card and another for USB card)
> 
> The two sets of controls implied by the use of two cards is really
> problematic IMHO. This adds complexity for userspace to figure out that
> the controls are really the same and synchronize/mirror changes.
>  > The premise of offload is that it should really not get in the way of
> user-experience, design constructs that result in delayed starts/stop,
> changed volumes or quality differences should be avoided, or
> users/distros will disable this optimization.
>

Makes sense.  I think in terms of controls, we know that for an USB 
audio device, anything will still be handled through the USB card. 
Again, since I'm not too familiar yet with all the userspace 
implementations, does it have mechanisms to treat the control and data 
interfaces separately?

> One card with additional DSP-based PCM devices seems simpler to me in
> terms of usage, but it's not without technical challenges either: with
> the use of the ASoC topology framework we only know what the DSP
> supports when registering a card and probing the ASoC components.
> 
> The interaction between USB audio and ASoC would also be at least as
> complicated as display audio, in that it needs to work no matter what
> the probe order is, and even survive the Linux device/driver model
> requirement that there are no timing dependencies in the driver
> bind/unbind sequences.
> 

Yes, this was my initial approach as well, but from the technical 
perspective it was very very messy, and potentially could have affected 
functionality on certain devices if not handled correctly.  I think the 
difficult part was that the USB SND framework itself is an independent 
entity, and it was tough to dissect the portions which created PCM/sound 
card devices.

I don't think that was something which would have gone well if 
introduced all at once.  It would require a lot of discussion before 
getting the proper implementation.  At least this series introduces the 
initial communication between ASoC and USB SND, and maybe as use cases 
become clearer we can always improve/build on top of it.

>>> Another point is that there may be cases where the DSP paths are not
>>> available if the DSP memory and MCPS budget is exceeded. In those cases,
>>> the DSP parts needs the ability to notify userspace that the legacy path
>>> should be used.
>>
>> If we ran into this scenario, the audio DSP AFE port start command would
>> fail, and this would be propagated to the userspace entity.  It could
>> then potentially re-route to the legacy/non-offload path.
> 
> 'start' or 'open'? This is a rather important design difference. Usually
> we try to make decisions in the .open or .hw_params stage. The 'start'
> or 'trigger' are usually not meant to fail due to unavailable resources
> in ALSA.

This happens during the .prepare() phase.

>>> Another case to handle is that some USB devices can handle way more data
>>> than DSPs can chew, for example Pro audio boxes that can deal with 8ch
>>> 192kHz will typically use the legacy paths. Some also handle specific
>>> formats such as DSD over PCM. So it's quite likely that PCM devices for
>>> card0 and card1 above do NOT expose support for the same formats, or put
>>> differently that only a subset of the USB device capabilities are
>>> handled through the DSP.
>>
>> Same as the above.  We have programmed the USB backend to support the
>> profiles that the audio DSP can handle.  I assume if there was any other
>> request, the userspace entity would fail the PCM open for that requested
>> profile.
> 
> What's not clear to me is whether there's any matching process between
> the DSP capabilities and what the USB device exposes? if the USB device
> is already way more complicated that what the ASoC back-end can deal
> with, why expose a card?
> 

That's something I thought was done by the ASoC core.  I can check that 
and see if that's the case.  There is a check added in hw_params of our 
ASoC component where we do query the USB audio descriptors to ensure 
that the PCM format being used is supported by the device.  I guess this 
is when the DSP capabilities are better than what the headset can 
support :).

>>> And last, power optimizations with DSPs typically come from additional
>>> latency helping put the SoC in low-power modes. That's not necessarily
>>> ideal for all usages, e.g. for music recording and mixing I am not
>>> convinced the DSP path would help at all.
>>>
>>
>> That's true.  At the same time, this feature is more for power related
>> benefits, not specifically for performance. (although we haven't seen
>> any performance related issues w/ this approach on the audio profiles
>> the DSP supports)  I think if its an audio profile that supports a high
>> sample rate and large number of channels, then the DSP wouldn't be able
>> to support it anyway, and userspace could still use the legacy path.
>> This would allow for those high-performance audio devices to not be
>> affected.
> 
> ok, we are aligned as well here. Excellent. With the on-going work to
> introduce 'Deep Buffer' capabilities, we'll have a need to tag PCM
> devices with a usage or 'modifier', or have this information in
> UCM/topology. Logic will have to be introduced in userspace to use the
> best routing, I would think this work can be reused for USB cases to
> indicate the offload solution is geared to power optimizations.

Great, I like that idea to see if we can help userspace choose the 
desired path based on what the overall system is looking for.  I wonder 
if that would also potentially help with some of the PCM device 
selection complexities you brought up as well.  If the system just wants 
best possible performance then it would just completely ignore the power 
optimized (offload) path for any device.

Thanks
Wesley Cheng

Re: [RFC PATCH 02/14] ASoC: qcom: qdsp6: Introduce USB AFE port to q6dsp

[Wesley Cheng]

Hi Pierre,

On 1/6/2023 8:09 AM, Pierre-Louis Bossart wrote:
> 
>>>> The QC ADSP is able to support USB playback and capture, so that the
>>>> main application processor can be placed into lower CPU power modes.
>>>> This
>>>> adds the required AFE port configurations and port start command to
>>>> start
>>>> an audio session.
>>>
>>> It would be good to clarify what sort of endpoints can be supported. I
>>> presume the SOF-synchronous case is handled, but how would you deal with
>>> async endpoints with feedback (be it explicit or implicit)?
>>>
>>
>> Sure, both types of feedback endpoints are expected to be supported by
>> the audio DSP, as well as sync eps.  We have the logic there to modify
>> the audio sample size accordingly.
> 
> did you mean modify samples per USB frame (or uframe), so as to change
> the pace at which data is transferred? If yes it'd be the same for Intel.
> 

Yes, sorry for not being clear.  Your understanding is correct.

>>>>      static const struct snd_soc_dapm_route q6afe_dapm_routes[] = {
>>>> +    {"USB Playback", NULL, "USB_RX"},
>>>
>>> ... but here RX means playback?
>>>
>>> I am not sure I get the convention on directions and what is actually
>>> supported?
>>>
>>
>> The notation is based on the direction of which the audio data is
>> sourced or pushed on to the DSP.  So in playback, the DSP is receiving
>> audio data to send, and capture, it is transmitting audio data received.
>> (although we do not support capture yet)
> 
> ok, it'd be good to add a comment on this convention. Usually RX/TX is
> bus-centric.
> 

Sure, will do.

>>
>>>> +struct afe_param_id_usb_cfg {
>>>> +/* Minor version used for tracking USB audio device configuration.
>>>> + * Supported values: AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG
>>>> + */
>>>> +    u32                  cfg_minor_version;
>>>> +/* Sampling rate of the port.
>>>> + * Supported values:
>>>> + * - AFE_PORT_SAMPLE_RATE_8K
>>>> + * - AFE_PORT_SAMPLE_RATE_11025
>>>> + * - AFE_PORT_SAMPLE_RATE_12K
>>>> + * - AFE_PORT_SAMPLE_RATE_16K
>>>> + * - AFE_PORT_SAMPLE_RATE_22050
>>>> + * - AFE_PORT_SAMPLE_RATE_24K
>>>> + * - AFE_PORT_SAMPLE_RATE_32K
>>>> + * - AFE_PORT_SAMPLE_RATE_44P1K
>>>> + * - AFE_PORT_SAMPLE_RATE_48K
>>>> + * - AFE_PORT_SAMPLE_RATE_96K
>>>> + * - AFE_PORT_SAMPLE_RATE_192K
>>>> + */
>>>> +    u32                  sample_rate;
>>>> +/* Bit width of the sample.
>>>> + * Supported values: 16, 24
>>>> + */
>>>> +    u16                  bit_width;
>>>> +/* Number of channels.
>>>> + * Supported values: 1 and 2
>>>
>>> that aligns with my feedback on the cover letter, if you connect a
>>> device that can support from than 2 channels should the DSP even expose
>>> this DSP-optimized path?
>>>
>>
>> My assumption is that I programmed the DAIs w/ PCM formats supported by
>> the DSP, so I think the ASoC core should not allow userspace to choose
>> that path if the hw params don't fit/match.
> 
> Right, but the point I was trying to make is that if the device can do
> more, why create this DSP path at all?
> 

Yeah, I think this brings me back to needing to understand a bit more of 
how the userspace chooses which PCM device to use.  At least for our 
current use cases, userspace would always route through the offload 
path, regardless of if the device can do more.  It will just select a 
lower audio profile if so.

>>
>>> Oh and I forgot, what happens if there are multiple audio devices
>>> connected, can the DSP deal with all of them? If not, how is this
>>> handled?
>>>
>>
>> This is one topic that we were pretty open ended on.  At least on our
>> implementation, only one audio device can be supported at a time.  We
>> choose the latest device that was plugged in or discovered by the USB
>> SND class driver.
> 
> Similar case for Intel. I have to revisit this, I don't recall the details.
> 

Got it.

Thanks
Wesley Cheng

Re: [RFC PATCH 07/14] usb: host: xhci: Add XHCI secondary interrupter support

[Wesley Cheng]

Hi Mathias,

On 1/2/2023 8:38 AM, Mathias Nyman wrote:
> On 29.12.2022 23.14, Wesley Cheng wrote:
>> Hi Mathias,
>>
>> On 12/28/2022 7:47 AM, Mathias Nyman wrote:
>>> On 24.12.2022 1.31, Wesley Cheng wrote:
>>>> Implement the XHCI operations for allocating and requesting for a 
>>>> secondary
>>>> interrupter.  The secondary interrupter can allow for events for a
>>>> particular endpoint to be routed to a separate event ring.  The event
>>>> routing is defined when submitting a transfer descriptor to the USB HW.
>>>> There is a specific field which denotes which interrupter ring to 
>>>> route the
>>>> event to when the transfer is completed.
>>>>
>>>> An example use case, such as audio packet offloading can utilize a 
>>>> separate
>>>> event ring, so that these events can be routed to a different processor
>>>> within the system.  The processor would be able to independently submit
>>>> transfers and handle its completions without intervention from the main
>>>> processor.
>>>>
>>>
>>> Adding support for more xHCI interrupters than just the primary one 
>>> make sense for
>>> both the offloading and virtualization cases.
>>>
>>> xHCI support for several interrupters was probably added to support 
>>> virtualization,
>>> to hand over usb devices to virtual machines and give them their own 
>>> event ring and
>>> MSI/MSI-X vector.
>>>
>>> In this offloading case you probably want to avoid xHC interrupts 
>>> from this device
>>> completely, making sure it doesn't wake up the main CPU unnecessarily.
>>>
>>> So is the idea here to let xhci driver set up the new interrupter, 
>>> its event ring,
>>> and the endpoint transfer rings. Then pass the address of the 
>>> endpoint transfer rings
>>> and the new event ring to the separate processor.
>>>
>>> This separate processor then both polls the event ring for new 
>>> events, sets its dequeue
>>> pointer, clears EHB bit, and queues new TRBs on the transfer ring.
>>>
>>> so xhci driver does not handle any events for the audio part, and no 
>>> audio data URBs
>>> are sent to usb core?
>>
>> Your entire description is correct.  To clarify, the interfaces which 
>> are non-audio will still be handled by the main processor.  For 
>> example, a USB headset can have a HID interface as well for volume 
>> control.  The HID interface will still be handled by the main 
>> processor, and events routed to the main event ring.
>>
>>>
>>> How about the control part?
>>> Is the control endpoint for this device still handled normally by usb 
>>> core/xhci?
>>>
>>
>> Control transfers are always handled on the main processor.  Only 
>> audio interface's endpoints.
> 
> Good to know, that means interrupter should be chosen per endpoint, not 
> per device.
> 
>>
>>> For the xhci parts I think we should start start by adding generic 
>>> support for several
>>> interrupters, then add parts needed for offloading.
>>
>> I can split up the patchsets to add interrupters first, then adding 
>> the offloading APIs in a separate patch.
> 
> 
> I started looking at supporting secondary interrupters myself.
> Let me work on that part a bit first. We have a bit different end goals.
> I want to handle interrupts from a secondary interrupter, while this 
> audio offload
> really just wants to mask some interrupts.
> 

I was looking at how we could possibly split up the XHCI secondary 
interrupter, and offloading parts.  Since the XHCI secondary interrupter 
is a feature that is defined in the XHCI spec (and we aren't doing 
anything outside of what is defined), I was thinking of having a 
separate XHCI driver (ie xhci-sec.c/h) that can be used to define all 
APIs related to setting up the event ring and ring management. 
(interrupt support can be added here)  This aligns a bit with what Alan 
suggested, and removing the APIs in the USB HCD, since this is XHCI 
specific stuff. ( 
https://lore.kernel.org/linux-usb/Y6zwZOquZOTZfnvP@rowland.harvard.edu/ )

For the offloading part, I think this is a bit more dependent on how 
different platforms implement it.  To use more of a generic approach 
like how Albert suggested here:

https://patchwork.kernel.org/project/linux-usb/list/?series=704174

Basically to give vendors the ability to define their own 
sequences/callbacks, and from which the XHCI driver will call into. (if 
needed)  These would need to be a separate set of XHCI drivers as well.

Do you think this is a proper model for us to go with, so that we can 
allow for vendors to easily add functionality?  Appreciate the inputs.

Thanks
Wesley Cheng

Re: [RFC PATCH 07/14] usb: host: xhci: Add XHCI secondary interrupter support

[Mathias Nyman]

On 9.1.2023 22.24, Wesley Cheng wrote:
> Hi Mathias,
> 
> On 1/2/2023 8:38 AM, Mathias Nyman wrote:
>> On 29.12.2022 23.14, Wesley Cheng wrote:
>>> Hi Mathias,
>>>
>>> On 12/28/2022 7:47 AM, Mathias Nyman wrote:
>>>> On 24.12.2022 1.31, Wesley Cheng wrote:
>>>>> Implement the XHCI operations for allocating and requesting for a secondary
>>>>> interrupter.  The secondary interrupter can allow for events for a
>>>>> particular endpoint to be routed to a separate event ring.  The event
>>>>> routing is defined when submitting a transfer descriptor to the USB HW.
>>>>> There is a specific field which denotes which interrupter ring to route the
>>>>> event to when the transfer is completed.
>>>>>
>>>>> An example use case, such as audio packet offloading can utilize a separate
>>>>> event ring, so that these events can be routed to a different processor
>>>>> within the system.  The processor would be able to independently submit
>>>>> transfers and handle its completions without intervention from the main
>>>>> processor.
>>>>>
>>>>
>>>> Adding support for more xHCI interrupters than just the primary one make sense for
>>>> both the offloading and virtualization cases.
>>>>
>>>> xHCI support for several interrupters was probably added to support virtualization,
>>>> to hand over usb devices to virtual machines and give them their own event ring and
>>>> MSI/MSI-X vector.
>>>>
>>>> In this offloading case you probably want to avoid xHC interrupts from this device
>>>> completely, making sure it doesn't wake up the main CPU unnecessarily.
>>>>
>>>> So is the idea here to let xhci driver set up the new interrupter, its event ring,
>>>> and the endpoint transfer rings. Then pass the address of the endpoint transfer rings
>>>> and the new event ring to the separate processor.
>>>>
>>>> This separate processor then both polls the event ring for new events, sets its dequeue
>>>> pointer, clears EHB bit, and queues new TRBs on the transfer ring.
>>>>
>>>> so xhci driver does not handle any events for the audio part, and no audio data URBs
>>>> are sent to usb core?
>>>
>>> Your entire description is correct.  To clarify, the interfaces which are non-audio will still be handled by the main processor.  For example, a USB headset can have a HID interface as well for volume control.  The HID interface will still be handled by the main processor, and events routed to the main event ring.
>>>
>>>>
>>>> How about the control part?
>>>> Is the control endpoint for this device still handled normally by usb core/xhci?
>>>>
>>>
>>> Control transfers are always handled on the main processor.  Only audio interface's endpoints.
>>
>> Good to know, that means interrupter should be chosen per endpoint, not per device.
>>
>>>
>>>> For the xhci parts I think we should start start by adding generic support for several
>>>> interrupters, then add parts needed for offloading.
>>>
>> I can split up the patchsets to add interrupters first, then adding the offloading APIs in a separate patch.
>>
>>
>> I started looking at supporting secondary interrupters myself.
>> Let me work on that part a bit first. We have a bit different end goals.
>> I want to handle interrupts from a secondary interrupter, while this audio offload
>> really just wants to mask some interrupts.
>>
> 
> I was looking at how we could possibly split up the XHCI secondary interrupter, and offloading parts.  Since the XHCI secondary interrupter is a feature that is defined in the XHCI spec (and we aren't doing anything outside of what is defined), I was thinking of having a separate XHCI driver (ie xhci-sec.c/h) that can be used to define all APIs related to setting up the event ring and ring management. (interrupt support can be added here)  This aligns a bit with what Alan suggested, and removing the APIs in the USB HCD, since this is XHCI specific stuff. ( https://lore.kernel.org/linux-usb/Y6zwZOquZOTZfnvP@rowland.harvard.edu/ )

Already started working on the interrupter, that part fits well into current driver.

Code (untested, will be randomly rebased etc) can be found in my feature_interrupters branch:
git://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git feature_interrupters
https://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git/log/?h=feature_interrupters

First step turns current event ring into a primary interrupter.
last patch is a test implementation for creating and freeing new secondary interrupters.

> 
> For the offloading part, I think this is a bit more dependent on how different platforms implement it.  To use more of a generic approach like how Albert suggested here:
> 
> https://patchwork.kernel.org/project/linux-usb/list/?series=704174
> 
> Basically to give vendors the ability to define their own sequences/callbacks, and from which the XHCI driver will call into. (if needed)  These would need to be a separate set of XHCI drivers as well.
> 
> Do you think this is a proper model for us to go with, so that we can allow for vendors to easily add functionality?  Appreciate the inputs.

I'm not convinced that overriding different xhci memory allocation functions is the best solution.
I think xhci driver will need to know which endpoints are offloaded.
maybe usb class driver could register an "offloader" with xhci for a usb device.

Trying to figure out what this xhci offload API would look like.
The dsp needs at least dma address of an event ring, and offloaded endpoint rings.
Is there anything else that the dsp would directly need to take care of, or can
we just export some xhci functions for starting/stopping endpoints, and update event deq?

Thanks
-Mathias

Re: [RFC PATCH 07/14] usb: host: xhci: Add XHCI secondary interrupter support

[Wesley Cheng]

Hi Mathias,

On 1/10/2023 11:47 AM, Mathias Nyman wrote:
> On 9.1.2023 22.24, Wesley Cheng wrote:
>> Hi Mathias,
>>
>> On 1/2/2023 8:38 AM, Mathias Nyman wrote:
>>> On 29.12.2022 23.14, Wesley Cheng wrote:
>>>> Hi Mathias,
>>>>
>>>> On 12/28/2022 7:47 AM, Mathias Nyman wrote:
>>>>> On 24.12.2022 1.31, Wesley Cheng wrote:
>>>>>> Implement the XHCI operations for allocating and requesting for a 
>>>>>> secondary
>>>>>> interrupter.  The secondary interrupter can allow for events for a
>>>>>> particular endpoint to be routed to a separate event ring.  The event
>>>>>> routing is defined when submitting a transfer descriptor to the 
>>>>>> USB HW.
>>>>>> There is a specific field which denotes which interrupter ring to 
>>>>>> route the
>>>>>> event to when the transfer is completed.
>>>>>>
>>>>>> An example use case, such as audio packet offloading can utilize a 
>>>>>> separate
>>>>>> event ring, so that these events can be routed to a different 
>>>>>> processor
>>>>>> within the system.  The processor would be able to independently 
>>>>>> submit
>>>>>> transfers and handle its completions without intervention from the 
>>>>>> main
>>>>>> processor.
>>>>>>
>>>>>
>>>>> Adding support for more xHCI interrupters than just the primary one 
>>>>> make sense for
>>>>> both the offloading and virtualization cases.
>>>>>
>>>>> xHCI support for several interrupters was probably added to support 
>>>>> virtualization,
>>>>> to hand over usb devices to virtual machines and give them their 
>>>>> own event ring and
>>>>> MSI/MSI-X vector.
>>>>>
>>>>> In this offloading case you probably want to avoid xHC interrupts 
>>>>> from this device
>>>>> completely, making sure it doesn't wake up the main CPU unnecessarily.
>>>>>
>>>>> So is the idea here to let xhci driver set up the new interrupter, 
>>>>> its event ring,
>>>>> and the endpoint transfer rings. Then pass the address of the 
>>>>> endpoint transfer rings
>>>>> and the new event ring to the separate processor.
>>>>>
>>>>> This separate processor then both polls the event ring for new 
>>>>> events, sets its dequeue
>>>>> pointer, clears EHB bit, and queues new TRBs on the transfer ring.
>>>>>
>>>>> so xhci driver does not handle any events for the audio part, and 
>>>>> no audio data URBs
>>>>> are sent to usb core?
>>>>
>>>> Your entire description is correct.  To clarify, the interfaces 
>>>> which are non-audio will still be handled by the main processor.  
>>>> For example, a USB headset can have a HID interface as well for 
>>>> volume control.  The HID interface will still be handled by the main 
>>>> processor, and events routed to the main event ring.
>>>>
>>>>>
>>>>> How about the control part?
>>>>> Is the control endpoint for this device still handled normally by 
>>>>> usb core/xhci?
>>>>>
>>>>
>>>> Control transfers are always handled on the main processor.  Only 
>>>> audio interface's endpoints.
>>>
>>> Good to know, that means interrupter should be chosen per endpoint, 
>>> not per device.
>>>
>>>>
>>>>> For the xhci parts I think we should start start by adding generic 
>>>>> support for several
>>>>> interrupters, then add parts needed for offloading.
>>>>
>>> I can split up the patchsets to add interrupters first, then adding 
>>> the offloading APIs in a separate patch.
>>>
>>>
>>> I started looking at supporting secondary interrupters myself.
>>> Let me work on that part a bit first. We have a bit different end goals.
>>> I want to handle interrupts from a secondary interrupter, while this 
>>> audio offload
>>> really just wants to mask some interrupts.
>>>
>>
>> I was looking at how we could possibly split up the XHCI secondary 
>> interrupter, and offloading parts.  Since the XHCI secondary 
>> interrupter is a feature that is defined in the XHCI spec (and we 
>> aren't doing anything outside of what is defined), I was thinking of 
>> having a separate XHCI driver (ie xhci-sec.c/h) that can be used to 
>> define all APIs related to setting up the event ring and ring 
>> management. (interrupt support can be added here)  This aligns a bit 
>> with what Alan suggested, and removing the APIs in the USB HCD, since 
>> this is XHCI specific stuff. ( 
>> https://lore.kernel.org/linux-usb/Y6zwZOquZOTZfnvP@rowland.harvard.edu/ )
> 
> Already started working on the interrupter, that part fits well into 
> current driver.
> 
> Code (untested, will be randomly rebased etc) can be found in my 
> feature_interrupters branch:
> git://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git 
> feature_interrupters
> https://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git/log/?h=feature_interrupters 
> 

Oh perfect, let me take a look.  Thanks for this!

> 
> First step turns current event ring into a primary interrupter.
> last patch is a test implementation for creating and freeing new 
> secondary interrupters.
> 
>>
>> For the offloading part, I think this is a bit more dependent on how 
>> different platforms implement it.  To use more of a generic approach 
>> like how Albert suggested here:
>>
>> https://patchwork.kernel.org/project/linux-usb/list/?series=704174
>>
>> Basically to give vendors the ability to define their own 
>> sequences/callbacks, and from which the XHCI driver will call into. 
>> (if needed)  These would need to be a separate set of XHCI drivers as 
>> well.
>>
>> Do you think this is a proper model for us to go with, so that we can 
>> allow for vendors to easily add functionality?  Appreciate the inputs.
> 
> I'm not convinced that overriding different xhci memory allocation 
> functions is the best solution.
> I think xhci driver will need to know which endpoints are offloaded.
> maybe usb class driver could register an "offloader" with xhci for a usb 
> device.
> 
> Trying to figure out what this xhci offload API would look like.
> The dsp needs at least dma address of an event ring, and offloaded 
> endpoint rings.
> Is there anything else that the dsp would directly need to take care of, 
> or can
> we just export some xhci functions for starting/stopping endpoints, and 
> update event deq?
> 

I'm still working it out with Albert as well to see what they actually 
require, and if "hooking" into these XHCI apis is really necessary.  I 
wouldn't follow the path of hooking into existing XHCI APIs either, 
since I'm sure most of it will be duplicate code that is already done by 
the XHCI drivers.  I'll follow up a bit more to get a better 
understanding, or if Albert wants to chime in here that would be helpful 
also, so everyone is on the same page.

In the QC implementation, we only need the transfer and event ring 
address, skipping pending events in the secondary event ring (similar to 
update event deq), and starting/stopping eps.  Exporting APIs would work 
for us.

Thanks
Wesley Cheng

Re: [RFC PATCH 07/14] usb: host: xhci: Add XHCI secondary interrupter support

[Wesley Cheng]

Hi Mathias,

On 1/10/2023 12:03 PM, Wesley Cheng wrote:
> Hi Mathias,
> 
> On 1/10/2023 11:47 AM, Mathias Nyman wrote:
>> On 9.1.2023 22.24, Wesley Cheng wrote:
>>> Hi Mathias,
>>>
>>> On 1/2/2023 8:38 AM, Mathias Nyman wrote:
>>>> On 29.12.2022 23.14, Wesley Cheng wrote:
>>>>> Hi Mathias,
>>>>>
>>>>> On 12/28/2022 7:47 AM, Mathias Nyman wrote:
>>>>>> On 24.12.2022 1.31, Wesley Cheng wrote:
>>>>>>> Implement the XHCI operations for allocating and requesting for a 
>>>>>>> secondary
>>>>>>> interrupter.  The secondary interrupter can allow for events for a
>>>>>>> particular endpoint to be routed to a separate event ring.  The 
>>>>>>> event
>>>>>>> routing is defined when submitting a transfer descriptor to the 
>>>>>>> USB HW.
>>>>>>> There is a specific field which denotes which interrupter ring to 
>>>>>>> route the
>>>>>>> event to when the transfer is completed.
>>>>>>>
>>>>>>> An example use case, such as audio packet offloading can utilize 
>>>>>>> a separate
>>>>>>> event ring, so that these events can be routed to a different 
>>>>>>> processor
>>>>>>> within the system.  The processor would be able to independently 
>>>>>>> submit
>>>>>>> transfers and handle its completions without intervention from 
>>>>>>> the main
>>>>>>> processor.
>>>>>>>
>>>>>>
>>>>>> Adding support for more xHCI interrupters than just the primary 
>>>>>> one make sense for
>>>>>> both the offloading and virtualization cases.
>>>>>>
>>>>>> xHCI support for several interrupters was probably added to 
>>>>>> support virtualization,
>>>>>> to hand over usb devices to virtual machines and give them their 
>>>>>> own event ring and
>>>>>> MSI/MSI-X vector.
>>>>>>
>>>>>> In this offloading case you probably want to avoid xHC interrupts 
>>>>>> from this device
>>>>>> completely, making sure it doesn't wake up the main CPU 
>>>>>> unnecessarily.
>>>>>>
>>>>>> So is the idea here to let xhci driver set up the new interrupter, 
>>>>>> its event ring,
>>>>>> and the endpoint transfer rings. Then pass the address of the 
>>>>>> endpoint transfer rings
>>>>>> and the new event ring to the separate processor.
>>>>>>
>>>>>> This separate processor then both polls the event ring for new 
>>>>>> events, sets its dequeue
>>>>>> pointer, clears EHB bit, and queues new TRBs on the transfer ring.
>>>>>>
>>>>>> so xhci driver does not handle any events for the audio part, and 
>>>>>> no audio data URBs
>>>>>> are sent to usb core?
>>>>>
>>>>> Your entire description is correct.  To clarify, the interfaces 
>>>>> which are non-audio will still be handled by the main processor. 
>>>>> For example, a USB headset can have a HID interface as well for 
>>>>> volume control.  The HID interface will still be handled by the 
>>>>> main processor, and events routed to the main event ring.
>>>>>
>>>>>>
>>>>>> How about the control part?
>>>>>> Is the control endpoint for this device still handled normally by 
>>>>>> usb core/xhci?
>>>>>>
>>>>>
>>>>> Control transfers are always handled on the main processor.  Only 
>>>>> audio interface's endpoints.
>>>>
>>>> Good to know, that means interrupter should be chosen per endpoint, 
>>>> not per device.
>>>>
>>>>>
>>>>>> For the xhci parts I think we should start start by adding generic 
>>>>>> support for several
>>>>>> interrupters, then add parts needed for offloading.
>>>>>
>>>> I can split up the patchsets to add interrupters first, then adding 
>>>> the offloading APIs in a separate patch.
>>>>
>>>>
>>>> I started looking at supporting secondary interrupters myself.
>>>> Let me work on that part a bit first. We have a bit different end 
>>>> goals.
>>>> I want to handle interrupts from a secondary interrupter, while this 
>>>> audio offload
>>>> really just wants to mask some interrupts.
>>>>
>>>
>>> I was looking at how we could possibly split up the XHCI secondary 
>>> interrupter, and offloading parts.  Since the XHCI secondary 
>>> interrupter is a feature that is defined in the XHCI spec (and we 
>>> aren't doing anything outside of what is defined), I was thinking of 
>>> having a separate XHCI driver (ie xhci-sec.c/h) that can be used to 
>>> define all APIs related to setting up the event ring and ring 
>>> management. (interrupt support can be added here)  This aligns a bit 
>>> with what Alan suggested, and removing the APIs in the USB HCD, since 
>>> this is XHCI specific stuff. ( 
>>> https://lore.kernel.org/linux-usb/Y6zwZOquZOTZfnvP@rowland.harvard.edu/ 
>>> )
>>
>> Already started working on the interrupter, that part fits well into 
>> current driver.
>>
>> Code (untested, will be randomly rebased etc) can be found in my 
>> feature_interrupters branch:
>> git://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git 
>> feature_interrupters
>> https://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git/log/?h=feature_interrupters 
>>
> 
> Oh perfect, let me take a look.  Thanks for this!
> 

I actually tried to see if I could get our audio offloading to work with 
your current series.  (I understand its still work in progress)  I did 
have to make some changes to expose the APIs to our class driver, but I 
wanted to let you know about one of the issues I saw when developing my 
implementation, because I am seeing the same behavior w/ yours. (and 
there's a discrepancy w/ what's stated in the XHCI spec :))

So the reason why my initial submission did the event ring allocation 
and set up before the run/stop bit was set, is that I found that when 
writing to the ir_set->erst_base in this scenario (for the secondary 
interrupter), it lead to a SMMU fault from the DWC3 controller.  One 
thing I noticed, was that the SMMU fault address was the lower 32 bits 
of the segment table base address allocated.  The XHCI driver utilizes 
the xhci_write_64() api which first writes the lower 32 bits then the 
upper 32 bits.  The XHCI spec states that:

Table 5-41: Event Ring Segment Table Base Address Register Bit 
Definitions (ERSTBA)

"Event Ring Segment Table Base Address Register – RW. Default = ‘0’. 
This field defines the
high order bits of the start address of the Event Ring Segment Table.
Writing this register sets the Event Ring State Machine:EREP Advancement 
to the Start state.
Refer to Figure 4-12 for more information.
**For Secondary Interrupters: This field may be modified at any time.**"

I'm not sure if this is an issue with the specific controller we're 
using, so maybe I will wait until you can give this a try on your set 
up.  However, it doesn't seem to be true that we can write the ERSTBA 
any time we want to.  My assumption is that once I made the lower 32 bit 
write, the controller attempted to enable the Event Ring State machine 
(Figure 4-12), and this led to a SMMU fault, since the upper 64 bits 
haven't been written.  I also did some bit banging manually as well 
(using devmem) and any time I write to the secondary ring ERSTBA 
register it generates a fault. (before any offloading has started)

Thanks
Wesley Cheng

Re: [RFC PATCH 07/14] usb: host: xhci: Add XHCI secondary interrupter support

[Mathias Nyman]

On 11.1.2023 5.11, Wesley Cheng wrote:
> Hi Mathias,
> 
> On 1/10/2023 12:03 PM, Wesley Cheng wrote:
>> Hi Mathias,
>>
>> On 1/10/2023 11:47 AM, Mathias Nyman wrote:
>>> On 9.1.2023 22.24, Wesley Cheng wrote:
>>>> Hi Mathias,
>>>>
>>>> On 1/2/2023 8:38 AM, Mathias Nyman wrote:
>>>>> On 29.12.2022 23.14, Wesley Cheng wrote:
>>>>>> Hi Mathias,
>>>>>>
>>>>>> On 12/28/2022 7:47 AM, Mathias Nyman wrote:
>>>>>>> On 24.12.2022 1.31, Wesley Cheng wrote:
>>>>>>>> Implement the XHCI operations for allocating and requesting for a secondary
>>>>>>>> interrupter.  The secondary interrupter can allow for events for a
>>>>>>>> particular endpoint to be routed to a separate event ring.  The event
>>>>>>>> routing is defined when submitting a transfer descriptor to the USB HW.
>>>>>>>> There is a specific field which denotes which interrupter ring to route the
>>>>>>>> event to when the transfer is completed.
>>>>>>>>
>>>>>>>> An example use case, such as audio packet offloading can utilize a separate
>>>>>>>> event ring, so that these events can be routed to a different processor
>>>>>>>> within the system.  The processor would be able to independently submit
>>>>>>>> transfers and handle its completions without intervention from the main
>>>>>>>> processor.
>>>>>>>>
>>>>>>>
>>>>>>> Adding support for more xHCI interrupters than just the primary one make sense for
>>>>>>> both the offloading and virtualization cases.
>>>>>>>
>>>>>>> xHCI support for several interrupters was probably added to support virtualization,
>>>>>>> to hand over usb devices to virtual machines and give them their own event ring and
>>>>>>> MSI/MSI-X vector.
>>>>>>>
>>>>>>> In this offloading case you probably want to avoid xHC interrupts from this device
>>>>>>> completely, making sure it doesn't wake up the main CPU unnecessarily.
>>>>>>>
>>>>>>> So is the idea here to let xhci driver set up the new interrupter, its event ring,
>>>>>>> and the endpoint transfer rings. Then pass the address of the endpoint transfer rings
>>>>>>> and the new event ring to the separate processor.
>>>>>>>
>>>>>>> This separate processor then both polls the event ring for new events, sets its dequeue
>>>>>>> pointer, clears EHB bit, and queues new TRBs on the transfer ring.
>>>>>>>
>>>>>>> so xhci driver does not handle any events for the audio part, and no audio data URBs
>>>>>>> are sent to usb core?
>>>>>>
>>>>>> Your entire description is correct.  To clarify, the interfaces which are non-audio will still be handled by the main processor. For example, a USB headset can have a HID interface as well for volume control.  The HID interface will still be handled by the main processor, and events routed to the main event ring.
>>>>>>
>>>>>>>
>>>>>>> How about the control part?
>>>>>>> Is the control endpoint for this device still handled normally by usb core/xhci?
>>>>>>>
>>>>>>
>>>>>> Control transfers are always handled on the main processor.  Only audio interface's endpoints.
>>>>>
>>>>> Good to know, that means interrupter should be chosen per endpoint, not per device.
>>>>>
>>>>>>
>>>>>>> For the xhci parts I think we should start start by adding generic support for several
>>>>>>> interrupters, then add parts needed for offloading.
>>>>>>
>>>>> I can split up the patchsets to add interrupters first, then adding the offloading APIs in a separate patch.
>>>>>
>>>>>
>>>>> I started looking at supporting secondary interrupters myself.
>>>>> Let me work on that part a bit first. We have a bit different end goals.
>>>>> I want to handle interrupts from a secondary interrupter, while this audio offload
>>>>> really just wants to mask some interrupts.
>>>>>
>>>>
>>>> I was looking at how we could possibly split up the XHCI secondary interrupter, and offloading parts.  Since the XHCI secondary interrupter is a feature that is defined in the XHCI spec (and we aren't doing anything outside of what is defined), I was thinking of having a separate XHCI driver (ie xhci-sec.c/h) that can be used to define all APIs related to setting up the event ring and ring management. (interrupt support can be added here)  This aligns a bit with what Alan suggested, and removing the APIs in the USB HCD, since this is XHCI specific stuff. ( https://lore.kernel.org/linux-usb/Y6zwZOquZOTZfnvP@rowland.harvard.edu/ )
>>>
>>> Already started working on the interrupter, that part fits well into current driver.
>>>
>>> Code (untested, will be randomly rebased etc) can be found in my feature_interrupters branch:
>>> git://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git feature_interrupters
>>> https://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git/log/?h=feature_interrupters
>>
>> Oh perfect, let me take a look.  Thanks for this!
>>
> 
> I actually tried to see if I could get our audio offloading to work with your current series.  (I understand its still work in progress)  I did have to make some changes to expose the APIs to our class driver, but I wanted to let you know about one of the issues I saw when developing my implementation, because I am seeing the same behavior w/ yours. (and there's a discrepancy w/ what's stated in the XHCI spec :))
> 
> So the reason why my initial submission did the event ring allocation and set up before the run/stop bit was set, is that I found that when writing to the ir_set->erst_base in this scenario (for the secondary interrupter), it lead to a SMMU fault from the DWC3 controller.  One thing I noticed, was that the SMMU fault address was the lower 32 bits of the segment table base address allocated.  The XHCI driver utilizes the xhci_write_64() api which first writes the lower 32 bits then the upper 32 bits.  The XHCI spec states that:
> 
> Table 5-41: Event Ring Segment Table Base Address Register Bit Definitions (ERSTBA)
> 
> "Event Ring Segment Table Base Address Register – RW. Default = ‘0’. This field defines the
> high order bits of the start address of the Event Ring Segment Table.
> Writing this register sets the Event Ring State Machine:EREP Advancement to the Start state.
> Refer to Figure 4-12 for more information.
> **For Secondary Interrupters: This field may be modified at any time.**"
> 
> I'm not sure if this is an issue with the specific controller we're using, so maybe I will wait until you can give this a try on your set up.  However, it doesn't seem to be true that we can write the ERSTBA any time we want to.  My assumption is that once I made the lower 32 bit write, the controller attempted to enable the Event Ring State machine (Figure 4-12), and this led to a SMMU fault, since the upper 64 bits haven't been written.  I also did some bit banging manually as well (using devmem) and any time I write to the secondary ring ERSTBA register it generates a fault. (before any offloading has started)

Tried on an Intel host and it seems to work fine.
I created a few secondary interrupters while xHC was running without issues.
DMA mask is 64 bits.
Only created the interrupters, no events on those new event rings, and didn't actually
check that the values written to ERSTBA were 64 bit.

Does temporarily setting DMA mask to 32 bits while allocating erst help in your case?

Thanks
Mathias

Re: [RFC PATCH 07/14] usb: host: xhci: Add XHCI secondary interrupter support

[Wesley Cheng]

Hi Mathias,

On 1/12/2023 1:24 AM, Mathias Nyman wrote:
> On 11.1.2023 5.11, Wesley Cheng wrote:
>> Hi Mathias,
>>
>> On 1/10/2023 12:03 PM, Wesley Cheng wrote:
>>> Hi Mathias,
>>>
>>> On 1/10/2023 11:47 AM, Mathias Nyman wrote:
>>>> On 9.1.2023 22.24, Wesley Cheng wrote:
>>>>> Hi Mathias,
>>>>>
>>>>> On 1/2/2023 8:38 AM, Mathias Nyman wrote:
>>>>>> On 29.12.2022 23.14, Wesley Cheng wrote:
>>>>>>> Hi Mathias,
>>>>>>>
>>>>>>> On 12/28/2022 7:47 AM, Mathias Nyman wrote:
>>>>>>>> On 24.12.2022 1.31, Wesley Cheng wrote:
>>>>>>>>> Implement the XHCI operations for allocating and requesting for 
>>>>>>>>> a secondary
>>>>>>>>> interrupter.  The secondary interrupter can allow for events for a
>>>>>>>>> particular endpoint to be routed to a separate event ring.  The 
>>>>>>>>> event
>>>>>>>>> routing is defined when submitting a transfer descriptor to the 
>>>>>>>>> USB HW.
>>>>>>>>> There is a specific field which denotes which interrupter ring 
>>>>>>>>> to route the
>>>>>>>>> event to when the transfer is completed.
>>>>>>>>>
>>>>>>>>> An example use case, such as audio packet offloading can 
>>>>>>>>> utilize a separate
>>>>>>>>> event ring, so that these events can be routed to a different 
>>>>>>>>> processor
>>>>>>>>> within the system.  The processor would be able to 
>>>>>>>>> independently submit
>>>>>>>>> transfers and handle its completions without intervention from 
>>>>>>>>> the main
>>>>>>>>> processor.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Adding support for more xHCI interrupters than just the primary 
>>>>>>>> one make sense for
>>>>>>>> both the offloading and virtualization cases.
>>>>>>>>
>>>>>>>> xHCI support for several interrupters was probably added to 
>>>>>>>> support virtualization,
>>>>>>>> to hand over usb devices to virtual machines and give them their 
>>>>>>>> own event ring and
>>>>>>>> MSI/MSI-X vector.
>>>>>>>>
>>>>>>>> In this offloading case you probably want to avoid xHC 
>>>>>>>> interrupts from this device
>>>>>>>> completely, making sure it doesn't wake up the main CPU 
>>>>>>>> unnecessarily.
>>>>>>>>
>>>>>>>> So is the idea here to let xhci driver set up the new 
>>>>>>>> interrupter, its event ring,
>>>>>>>> and the endpoint transfer rings. Then pass the address of the 
>>>>>>>> endpoint transfer rings
>>>>>>>> and the new event ring to the separate processor.
>>>>>>>>
>>>>>>>> This separate processor then both polls the event ring for new 
>>>>>>>> events, sets its dequeue
>>>>>>>> pointer, clears EHB bit, and queues new TRBs on the transfer ring.
>>>>>>>>
>>>>>>>> so xhci driver does not handle any events for the audio part, 
>>>>>>>> and no audio data URBs
>>>>>>>> are sent to usb core?
>>>>>>>
>>>>>>> Your entire description is correct.  To clarify, the interfaces 
>>>>>>> which are non-audio will still be handled by the main processor. 
>>>>>>> For example, a USB headset can have a HID interface as well for 
>>>>>>> volume control.  The HID interface will still be handled by the 
>>>>>>> main processor, and events routed to the main event ring.
>>>>>>>
>>>>>>>>
>>>>>>>> How about the control part?
>>>>>>>> Is the control endpoint for this device still handled normally 
>>>>>>>> by usb core/xhci?
>>>>>>>>
>>>>>>>
>>>>>>> Control transfers are always handled on the main processor.  Only 
>>>>>>> audio interface's endpoints.
>>>>>>
>>>>>> Good to know, that means interrupter should be chosen per 
>>>>>> endpoint, not per device.
>>>>>>
>>>>>>>
>>>>>>>> For the xhci parts I think we should start start by adding 
>>>>>>>> generic support for several
>>>>>>>> interrupters, then add parts needed for offloading.
>>>>>>>
>>>>>> I can split up the patchsets to add interrupters first, then 
>>>>>> adding the offloading APIs in a separate patch.
>>>>>>
>>>>>>
>>>>>> I started looking at supporting secondary interrupters myself.
>>>>>> Let me work on that part a bit first. We have a bit different end 
>>>>>> goals.
>>>>>> I want to handle interrupts from a secondary interrupter, while 
>>>>>> this audio offload
>>>>>> really just wants to mask some interrupts.
>>>>>>
>>>>>
>>>>> I was looking at how we could possibly split up the XHCI secondary 
>>>>> interrupter, and offloading parts.  Since the XHCI secondary 
>>>>> interrupter is a feature that is defined in the XHCI spec (and we 
>>>>> aren't doing anything outside of what is defined), I was thinking 
>>>>> of having a separate XHCI driver (ie xhci-sec.c/h) that can be used 
>>>>> to define all APIs related to setting up the event ring and ring 
>>>>> management. (interrupt support can be added here)  This aligns a 
>>>>> bit with what Alan suggested, and removing the APIs in the USB HCD, 
>>>>> since this is XHCI specific stuff. ( 
>>>>> https://lore.kernel.org/linux-usb/Y6zwZOquZOTZfnvP@rowland.harvard.edu/ 
>>>>> )
>>>>
>>>> Already started working on the interrupter, that part fits well into 
>>>> current driver.
>>>>
>>>> Code (untested, will be randomly rebased etc) can be found in my 
>>>> feature_interrupters branch:
>>>> git://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git 
>>>> feature_interrupters
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git/log/?h=feature_interrupters 
>>>>
>>>
>>> Oh perfect, let me take a look.  Thanks for this!
>>>
>>
>> I actually tried to see if I could get our audio offloading to work 
>> with your current series.  (I understand its still work in progress)  
>> I did have to make some changes to expose the APIs to our class 
>> driver, but I wanted to let you know about one of the issues I saw 
>> when developing my implementation, because I am seeing the same 
>> behavior w/ yours. (and there's a discrepancy w/ what's stated in the 
>> XHCI spec :))
>>
>> So the reason why my initial submission did the event ring allocation 
>> and set up before the run/stop bit was set, is that I found that when 
>> writing to the ir_set->erst_base in this scenario (for the secondary 
>> interrupter), it lead to a SMMU fault from the DWC3 controller.  One 
>> thing I noticed, was that the SMMU fault address was the lower 32 bits 
>> of the segment table base address allocated.  The XHCI driver utilizes 
>> the xhci_write_64() api which first writes the lower 32 bits then the 
>> upper 32 bits.  The XHCI spec states that:
>>
>> Table 5-41: Event Ring Segment Table Base Address Register Bit 
>> Definitions (ERSTBA)
>>
>> "Event Ring Segment Table Base Address Register – RW. Default = ‘0’. 
>> This field defines the
>> high order bits of the start address of the Event Ring Segment Table.
>> Writing this register sets the Event Ring State Machine:EREP 
>> Advancement to the Start state.
>> Refer to Figure 4-12 for more information.
>> **For Secondary Interrupters: This field may be modified at any time.**"
>>
>> I'm not sure if this is an issue with the specific controller we're 
>> using, so maybe I will wait until you can give this a try on your set 
>> up.  However, it doesn't seem to be true that we can write the ERSTBA 
>> any time we want to.  My assumption is that once I made the lower 32 
>> bit write, the controller attempted to enable the Event Ring State 
>> machine (Figure 4-12), and this led to a SMMU fault, since the upper 
>> 64 bits haven't been written.  I also did some bit banging manually as 
>> well (using devmem) and any time I write to the secondary ring ERSTBA 
>> register it generates a fault. (before any offloading has started)
> 
> Tried on an Intel host and it seems to work fine.
> I created a few secondary interrupters while xHC was running without 
> issues.
> DMA mask is 64 bits.
> Only created the interrupters, no events on those new event rings, and 
> didn't actually
> check that the values written to ERSTBA were 64 bit.
> 
> Does temporarily setting DMA mask to 32 bits while allocating erst help 
> in your case?
> 

Yes, that works fine.  Another thing I found which works is that, the 
XHCI spec mentions that we should always write the lower 32 bits first 
before the upper bits for a 64 bit register.  Just as an experiment, I 
flipped the write order (upper first, lower second), and that seemed to 
not trigger the event ring state machine, and I was able to verify that 
our audio offloading was working.

At the moment, I exposed/exported the 
xhci_remove_secondary_interrupter() and 
xhci_create_secondary_interrupter() into a header file 
(include/linux/usb/xhci-intr.h), along with some other APIs to fetch the 
transfer resources. (same mechanism I have in my patchset)  This allowed 
me to only reference the required APIs from the USB audio offload class 
driver w/o having to include all of xhci.h (where some of the things are 
currently defined in your changes)

Thanks
Wesley Cheng