[PATCH v9 0/4] Media Device Allocator API

[PATCH v9 0/4] Media Device Allocator API

[shuah]

From: Shuah Khan <shuah@kernel.org>

Media Device Allocator API to allows multiple drivers share a media device.
This API solves a very common use-case for media devices where one physical
device (an USB stick) provides both audio and video. When such media device
exposes a standard USB Audio class, a proprietary Video class, two or more
independent drivers will share a single physical USB bridge. In such cases,
it is necessary to coordinate access to the shared resource.

Using this API, drivers can allocate a media device with the shared struct
device as the key. Once the media device is allocated by a driver, other
drivers can get a reference to it. The media device is released when all
the references are released.

- Tested sharing resources with kaffeine, vlc, xawtv, tvtime, and
  arecord. When analog is streaming, digital and audio user-space
  applications detect that the tuner is busy and exit. When digital
  is streaming, analog and audio applications detect that the tuner is
  busy and exit. When arecord is owns the tuner, digital and analog
  detect that the tuner is busy and exit.
- Tested media device allocator API with bind/unbind testing on
  snd-usb-audio and au0828 drivers to make sure /dev/mediaX is released
  only when the last driver is unbound.
- This patch series is tested on 4.20-rc6
- Addressed review comments from Hans on the RFC v8 (rebased on 4.19)
- Updated change log to describe the use-case more clearly.
- No changes to 0001,0002 code since the v7 referenced below.
- 0003 is a new patch to enable ALSA defines that have been
  disabled for kernel between 4.9 and 4.19.
- Minor merge conflict resolution in 0004.
- Added SPDX to new files.

References:
https://lkml.org/lkml/2018/11/2/169
https://www.mail-archive.com/linux-media@vger.kernel.org/msg105854.html

Shuah Khan (4):
  media: Media Device Allocator API
  media: change au0828 to use Media Device Allocator API
  media: media.h: Enable ALSA MEDIA_INTF_T* interface types
  sound/usb: Use Media Controller API to share media resources

 Documentation/media/kapi/mc-core.rst   |  41 ++++
 drivers/media/Makefile                 |   4 +
 drivers/media/media-dev-allocator.c    | 132 ++++++++++
 drivers/media/usb/au0828/au0828-core.c |  12 +-
 drivers/media/usb/au0828/au0828.h      |   1 +
 include/media/media-dev-allocator.h    |  53 ++++
 include/uapi/linux/media.h             |  25 +-
 sound/usb/Kconfig                      |   4 +
 sound/usb/Makefile                     |   2 +
 sound/usb/card.c                       |  14 ++
 sound/usb/card.h                       |   3 +
 sound/usb/media.c                      | 321 +++++++++++++++++++++++++
 sound/usb/media.h                      |  74 ++++++
 sound/usb/mixer.h                      |   3 +
 sound/usb/pcm.c                        |  29 ++-
 sound/usb/quirks-table.h               |   1 +
 sound/usb/stream.c                     |   2 +
 sound/usb/usbaudio.h                   |   6 +
 18 files changed, 705 insertions(+), 22 deletions(-)
 create mode 100644 drivers/media/media-dev-allocator.c
 create mode 100644 include/media/media-dev-allocator.h
 create mode 100644 sound/usb/media.c
 create mode 100644 sound/usb/media.h

-- 
2.17.1

[PATCH v9 1/4] media: Media Device Allocator API

[shuah]

From: Shuah Khan <shuah@kernel.org>

Media Device Allocator API to allows multiple drivers share a media device.
This API solves a very common use-case for media devices where one physical
device (an USB stick) provides both audio and video. When such media device
exposes a standard USB Audio class, a proprietary Video class, two or more
independent drivers will share a single physical USB bridge. In such cases,
it is necessary to coordinate access to the shared resource.

Using this API, drivers can allocate a media device with the shared struct
device as the key. Once the media device is allocated by a driver, other
drivers can get a reference to it. The media device is released when all
the references are released.

Signed-off-by: Shuah Khan <shuah@kernel.org>
---
 Documentation/media/kapi/mc-core.rst |  41 +++++++++
 drivers/media/Makefile               |   4 +
 drivers/media/media-dev-allocator.c  | 132 +++++++++++++++++++++++++++
 include/media/media-dev-allocator.h  |  53 +++++++++++
 4 files changed, 230 insertions(+)
 create mode 100644 drivers/media/media-dev-allocator.c
 create mode 100644 include/media/media-dev-allocator.h

diff --git a/Documentation/media/kapi/mc-core.rst b/Documentation/media/kapi/mc-core.rst
index 69362b3135c2..022381fc1f66 100644
--- a/Documentation/media/kapi/mc-core.rst
+++ b/Documentation/media/kapi/mc-core.rst
@@ -257,6 +257,45 @@ Subsystems should facilitate link validation by providing subsystem specific
 helper functions to provide easy access for commonly needed information, and
 in the end provide a way to use driver-specific callbacks.
 
+Media Controller Device Allocator API
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When the media device belongs to more than one driver, the shared media
+device is allocated with the shared struct device as the key for look ups.
+
+The shared media device should stay in registered state until the last
+driver unregisters it. In addition, the media device should be released when
+all the references are released. Each driver gets a reference to the media
+device during probe, when it allocates the media device. If media device is
+already allocated, the allocate API bumps up the refcount and returns the
+existing media device. The driver puts the reference back in its disconnect
+routine when it calls :c:func:`media_device_delete()`.
+
+The media device is unregistered and cleaned up from the kref put handler to
+ensure that the media device stays in registered state until the last driver
+unregisters the media device.
+
+**Driver Usage**
+
+Drivers should use the appropriate media-core routines to manage the shared
+media device life-time handling the two states:
+1. allocate -> register -> delete
+2. get reference to already registered device -> delete
+
+call :c:func:`media_device_delete()` routine to make sure the shared media
+device delete is handled correctly.
+
+**driver probe:**
+Call :c:func:`media_device_usb_allocate()` to allocate or get a reference
+Call :c:func:`media_device_register()`, if media devnode isn't registered
+
+**driver disconnect:**
+Call :c:func:`media_device_delete()` to free the media_device. Freeing is
+handled by the kref put handler.
+
+API Definitions
+^^^^^^^^^^^^^^^
+
 .. kernel-doc:: include/media/media-device.h
 
 .. kernel-doc:: include/media/media-devnode.h
@@ -264,3 +303,5 @@ in the end provide a way to use driver-specific callbacks.
 .. kernel-doc:: include/media/media-entity.h
 
 .. kernel-doc:: include/media/media-request.h
+
+.. kernel-doc:: include/media/media-dev-allocator.h
diff --git a/drivers/media/Makefile b/drivers/media/Makefile
index 985d35ec6b29..1d7653318af6 100644
--- a/drivers/media/Makefile
+++ b/drivers/media/Makefile
@@ -6,6 +6,10 @@
 media-objs	:= media-device.o media-devnode.o media-entity.o \
 		   media-request.o
 
+ifeq ($(CONFIG_USB),y)
+	media-objs += media-dev-allocator.o
+endif
+
 #
 # I2C drivers should come before other drivers, otherwise they'll fail
 # when compiled as builtin drivers
diff --git a/drivers/media/media-dev-allocator.c b/drivers/media/media-dev-allocator.c
new file mode 100644
index 000000000000..82b450cbc998
--- /dev/null
+++ b/drivers/media/media-dev-allocator.c
@@ -0,0 +1,132 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * media-dev-allocator.c - Media Controller Device Allocator API
+ *
+ * Copyright (c) 2018 Shuah Khan <shuah@kernel.org>
+ *
+ * Credits: Suggested by Laurent Pinchart <laurent.pinchart@ideasonboard.com>
+ */
+
+/*
+ * This file adds a global refcounted Media Controller Device Instance API.
+ * A system wide global media device list is managed and each media device
+ * includes a kref count. The last put on the media device releases the media
+ * device instance.
+ *
+ */
+
+#include <linux/kref.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/usb.h>
+
+#include <media/media-device.h>
+
+static LIST_HEAD(media_device_list);
+static DEFINE_MUTEX(media_device_lock);
+
+struct media_device_instance {
+	struct media_device mdev;
+	struct module *owner;
+	struct list_head list;
+	struct kref refcount;
+};
+
+static inline struct media_device_instance *
+to_media_device_instance(struct media_device *mdev)
+{
+	return container_of(mdev, struct media_device_instance, mdev);
+}
+
+static void media_device_instance_release(struct kref *kref)
+{
+	struct media_device_instance *mdi =
+		container_of(kref, struct media_device_instance, refcount);
+
+	dev_dbg(mdi->mdev.dev, "%s: mdev=%p\n", __func__, &mdi->mdev);
+
+	mutex_lock(&media_device_lock);
+
+	media_device_unregister(&mdi->mdev);
+	media_device_cleanup(&mdi->mdev);
+
+	list_del(&mdi->list);
+	mutex_unlock(&media_device_lock);
+
+	kfree(mdi);
+}
+
+/* Callers should hold media_device_lock when calling this function */
+static struct media_device *__media_device_get(struct device *dev,
+					       const char *module_name)
+{
+	struct media_device_instance *mdi;
+
+	list_for_each_entry(mdi, &media_device_list, list) {
+		if (mdi->mdev.dev != dev)
+			continue;
+
+		kref_get(&mdi->refcount);
+		/* get module reference for the media_device owner */
+		if (find_module(module_name) != mdi->owner &&
+		    !try_module_get(mdi->owner))
+			dev_err(dev, "%s: try_module_get() error\n", __func__);
+		dev_dbg(dev, "%s: get mdev=%p module_name %s\n",
+			__func__, &mdi->mdev, module_name);
+		return &mdi->mdev;
+	}
+
+	mdi = kzalloc(sizeof(*mdi), GFP_KERNEL);
+	if (!mdi)
+		return NULL;
+
+	mdi->owner = find_module(module_name);
+	kref_init(&mdi->refcount);
+	list_add_tail(&mdi->list, &media_device_list);
+
+	dev_dbg(dev, "%s: alloc mdev=%p module_name %s\n", __func__,
+		&mdi->mdev, module_name);
+	return &mdi->mdev;
+}
+
+#if IS_ENABLED(CONFIG_USB)
+struct media_device *media_device_usb_allocate(struct usb_device *udev,
+					       const char *module_name)
+{
+	struct media_device *mdev;
+
+	mutex_lock(&media_device_lock);
+	mdev = __media_device_get(&udev->dev, module_name);
+	if (!mdev) {
+		mutex_unlock(&media_device_lock);
+		return ERR_PTR(-ENOMEM);
+	}
+
+	/* check if media device is already initialized */
+	if (!mdev->dev)
+		__media_device_usb_init(mdev, udev, udev->product,
+					module_name);
+	mutex_unlock(&media_device_lock);
+	return mdev;
+}
+EXPORT_SYMBOL_GPL(media_device_usb_allocate);
+#endif
+
+void media_device_delete(struct media_device *mdev, const char *module_name)
+{
+	struct media_device_instance *mdi = to_media_device_instance(mdev);
+
+	dev_dbg(mdi->mdev.dev, "%s: mdev=%p module_name %s\n",
+		__func__, &mdi->mdev, module_name);
+
+	mutex_lock(&media_device_lock);
+	/* put module reference if media_device owner is not THIS_MODULE */
+	if (mdi->owner != find_module(module_name)) {
+		module_put(mdi->owner);
+		dev_dbg(mdi->mdev.dev,
+			"%s decremented owner module reference\n", __func__);
+	}
+	mutex_unlock(&media_device_lock);
+	kref_put(&mdi->refcount, media_device_instance_release);
+}
+EXPORT_SYMBOL_GPL(media_device_delete);
diff --git a/include/media/media-dev-allocator.h b/include/media/media-dev-allocator.h
new file mode 100644
index 000000000000..9164795e911c
--- /dev/null
+++ b/include/media/media-dev-allocator.h
@@ -0,0 +1,53 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * media-dev-allocator.h - Media Controller Device Allocator API
+ *
+ * Copyright (c) 2018 Shuah Khan <shuah@kernel.org>
+ *
+ * Credits: Suggested by Laurent Pinchart <laurent.pinchart@ideasonboard.com>
+ */
+
+/*
+ * This file adds a global ref-counted Media Controller Device Instance API.
+ * A system wide global media device list is managed and each media device
+ * includes a kref count. The last put on the media device releases the media
+ * device instance.
+ */
+
+#ifndef _MEDIA_DEV_ALLOCTOR_H
+#define _MEDIA_DEV_ALLOCTOR_H
+
+struct usb_device;
+
+#if defined(CONFIG_MEDIA_CONTROLLER) && defined(CONFIG_USB)
+/**
+ * media_device_usb_allocate() - Allocate and return struct &media device
+ *
+ * @udev:		struct &usb_device pointer
+ * @module_name:	should be filled with %KBUILD_MODNAME
+ *
+ * This interface should be called to allocate a Media Device when multiple
+ * drivers share usb_device and the media device. This interface allocates
+ * &media_device structure and calls media_device_usb_init() to initialize
+ * it.
+ *
+ */
+struct media_device *media_device_usb_allocate(struct usb_device *udev,
+					       char *module_name);
+/**
+ * media_device_delete() - Release media device. Calls kref_put().
+ *
+ * @mdev:		struct &media_device pointer
+ * @module_name:	should be filled with %KBUILD_MODNAME
+ *
+ * This interface should be called to put Media Device Instance kref.
+ */
+void media_device_delete(struct media_device *mdev, char *module_name);
+#else
+static inline struct media_device *media_device_usb_allocate(
+			struct usb_device *udev, char *module_name)
+			{ return NULL; }
+static inline void media_device_delete(
+			struct media_device *mdev, char *module_name) { }
+#endif /* CONFIG_MEDIA_CONTROLLER */
+#endif
-- 
2.17.1

[PATCH v9 2/4] media: change au0828 to use Media Device Allocator API

[shuah]

From: Shuah Khan <shuah@kernel.org>

Media Device Allocator API to allows multiple drivers share a media device.
This API solves a very common use-case for media devices where one physical
device (an USB stick) provides both audio and video. When such media device
exposes a standard USB Audio class, a proprietary Video class, two or more
independent drivers will share a single physical USB bridge. In such cases,
it is necessary to coordinate access to the shared resource.

Using this API, drivers can allocate a media device with the shared struct
device as the key. Once the media device is allocated by a driver, other
drivers can get a reference to it. The media device is released when all
the references are released.

Change au0828 to use Media Device Allocator API to allocate media device
with the parent usb struct device as the key, so it can be shared with the
snd_usb_audio driver.

Signed-off-by: Shuah Khan <shuah@kernel.org>
---
 drivers/media/usb/au0828/au0828-core.c | 12 ++++--------
 drivers/media/usb/au0828/au0828.h      |  1 +
 2 files changed, 5 insertions(+), 8 deletions(-)

diff --git a/drivers/media/usb/au0828/au0828-core.c b/drivers/media/usb/au0828/au0828-core.c
index 1fdb1601dc65..4b0a395d59aa 100644
--- a/drivers/media/usb/au0828/au0828-core.c
+++ b/drivers/media/usb/au0828/au0828-core.c
@@ -155,9 +155,7 @@ static void au0828_unregister_media_device(struct au0828_dev *dev)
 	dev->media_dev->disable_source = NULL;
 	mutex_unlock(&mdev->graph_mutex);
 
-	media_device_unregister(dev->media_dev);
-	media_device_cleanup(dev->media_dev);
-	kfree(dev->media_dev);
+	media_device_delete(dev->media_dev, KBUILD_MODNAME);
 	dev->media_dev = NULL;
 #endif
 }
@@ -210,14 +208,10 @@ static int au0828_media_device_init(struct au0828_dev *dev,
 #ifdef CONFIG_MEDIA_CONTROLLER
 	struct media_device *mdev;
 
-	mdev = kzalloc(sizeof(*mdev), GFP_KERNEL);
+	mdev = media_device_usb_allocate(udev, KBUILD_MODNAME);
 	if (!mdev)
 		return -ENOMEM;
 
-	/* check if media device is already initialized */
-	if (!mdev->dev)
-		media_device_usb_init(mdev, udev, udev->product);
-
 	dev->media_dev = mdev;
 #endif
 	return 0;
@@ -480,6 +474,8 @@ static int au0828_media_device_register(struct au0828_dev *dev,
 		/* register media device */
 		ret = media_device_register(dev->media_dev);
 		if (ret) {
+			media_device_delete(dev->media_dev, KBUILD_MODNAME);
+			dev->media_dev = NULL;
 			dev_err(&udev->dev,
 				"Media Device Register Error: %d\n", ret);
 			return ret;
diff --git a/drivers/media/usb/au0828/au0828.h b/drivers/media/usb/au0828/au0828.h
index 004eadef55c7..7dbe3db15ebe 100644
--- a/drivers/media/usb/au0828/au0828.h
+++ b/drivers/media/usb/au0828/au0828.h
@@ -31,6 +31,7 @@
 #include <media/v4l2-ctrls.h>
 #include <media/v4l2-fh.h>
 #include <media/media-device.h>
+#include <media/media-dev-allocator.h>
 
 /* DVB */
 #include <media/demux.h>
-- 
2.17.1

[PATCH v9 3/4] media: media.h: Enable ALSA MEDIA_INTF_T* interface types

[shuah]

From: Shuah Khan <shuah@kernel.org>

Move PCM_CAPTURE, PCM_PLAYBACK, and CONTROL ALSA MEDIA_INTF_T* interface
types back into __KERNEL__ scope to get ready for adding ALSA support for
these to the media controller.

Signed-off-by: Shuah Khan <shuah@kernel.org>
---
 include/uapi/linux/media.h | 25 +++++++++++++++----------
 1 file changed, 15 insertions(+), 10 deletions(-)

diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index e5d0c5c611b5..9aedb187bc48 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -262,6 +262,11 @@ struct media_links_enum {
 #define MEDIA_INTF_T_V4L_SWRADIO		(MEDIA_INTF_T_V4L_BASE + 4)
 #define MEDIA_INTF_T_V4L_TOUCH			(MEDIA_INTF_T_V4L_BASE + 5)
 
+#define MEDIA_INTF_T_ALSA_BASE			0x00000300
+#define MEDIA_INTF_T_ALSA_PCM_CAPTURE		(MEDIA_INTF_T_ALSA_BASE)
+#define MEDIA_INTF_T_ALSA_PCM_PLAYBACK		(MEDIA_INTF_T_ALSA_BASE + 1)
+#define MEDIA_INTF_T_ALSA_CONTROL		(MEDIA_INTF_T_ALSA_BASE + 2)
+
 #if defined(__KERNEL__)
 
 /*
@@ -413,19 +418,19 @@ struct media_v2_topology {
 #define MEDIA_ENT_F_DTV_DECODER			MEDIA_ENT_F_DV_DECODER
 
 /*
- * There is still no ALSA support in the media controller. These
+ * There is still no full ALSA support in the media controller. These
  * defines should not have been added and we leave them here only
  * in case some application tries to use these defines.
+ *
+ * The ALSA defines that are in use have been moved into __KERNEL__
+ * scope. As support gets added to these interface types, they should
+ * be moved into __KERNEL__ scope with the code that uses them.
  */
-#define MEDIA_INTF_T_ALSA_BASE			0x00000300
-#define MEDIA_INTF_T_ALSA_PCM_CAPTURE		(MEDIA_INTF_T_ALSA_BASE)
-#define MEDIA_INTF_T_ALSA_PCM_PLAYBACK		(MEDIA_INTF_T_ALSA_BASE + 1)
-#define MEDIA_INTF_T_ALSA_CONTROL		(MEDIA_INTF_T_ALSA_BASE + 2)
-#define MEDIA_INTF_T_ALSA_COMPRESS		(MEDIA_INTF_T_ALSA_BASE + 3)
-#define MEDIA_INTF_T_ALSA_RAWMIDI		(MEDIA_INTF_T_ALSA_BASE + 4)
-#define MEDIA_INTF_T_ALSA_HWDEP			(MEDIA_INTF_T_ALSA_BASE + 5)
-#define MEDIA_INTF_T_ALSA_SEQUENCER		(MEDIA_INTF_T_ALSA_BASE + 6)
-#define MEDIA_INTF_T_ALSA_TIMER			(MEDIA_INTF_T_ALSA_BASE + 7)
+#define MEDIA_INTF_T_ALSA_COMPRESS             (MEDIA_INTF_T_ALSA_BASE + 3)
+#define MEDIA_INTF_T_ALSA_RAWMIDI              (MEDIA_INTF_T_ALSA_BASE + 4)
+#define MEDIA_INTF_T_ALSA_HWDEP                (MEDIA_INTF_T_ALSA_BASE + 5)
+#define MEDIA_INTF_T_ALSA_SEQUENCER            (MEDIA_INTF_T_ALSA_BASE + 6)
+#define MEDIA_INTF_T_ALSA_TIMER                (MEDIA_INTF_T_ALSA_BASE + 7)
 
 /* Obsolete symbol for media_version, no longer used in the kernel */
 #define MEDIA_API_VERSION			((0 << 16) | (1 << 8) | 0)
-- 
2.17.1

[PATCH v9 4/4] sound/usb: Use Media Controller API to share media resources

[shuah]

From: Shuah Khan <shuah@kernel.org>

Media Device Allocator API to allows multiple drivers share a media device.
This API solves a very common use-case for media devices where one physical
device (an USB stick) provides both audio and video. When such media device
exposes a standard USB Audio class, a proprietary Video class, two or more
independent drivers will share a single physical USB bridge. In such cases,
it is necessary to coordinate access to the shared resource.

Using this API, drivers can allocate a media device with the shared struct
device as the key. Once the media device is allocated by a driver, other
drivers can get a reference to it. The media device is released when all
the references are released.

Change the ALSA driver to use the Media Controller API to share media
resources with DVB, and V4L2 drivers on a AU0828 media device.

The Media Controller specific initialization is done after sound card is
registered. ALSA creates Media interface and entity function graph nodes
for Control, Mixer, PCM Playback, and PCM Capture devices.

snd_usb_hw_params() will call Media Controller enable source handler
interface to request the media resource. If resource request is granted,
it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
returned.

Media specific cleanup is done in usb_audio_disconnect().

Signed-off-by: Shuah Khan <shuah@kernel.org>
---
 sound/usb/Kconfig        |   4 +
 sound/usb/Makefile       |   2 +
 sound/usb/card.c         |  14 ++
 sound/usb/card.h         |   3 +
 sound/usb/media.c        | 321 +++++++++++++++++++++++++++++++++++++++
 sound/usb/media.h        |  74 +++++++++
 sound/usb/mixer.h        |   3 +
 sound/usb/pcm.c          |  29 +++-
 sound/usb/quirks-table.h |   1 +
 sound/usb/stream.c       |   2 +
 sound/usb/usbaudio.h     |   6 +
 11 files changed, 455 insertions(+), 4 deletions(-)
 create mode 100644 sound/usb/media.c
 create mode 100644 sound/usb/media.h

diff --git a/sound/usb/Kconfig b/sound/usb/Kconfig
index f61b5662bb89..6319b544ba3a 100644
--- a/sound/usb/Kconfig
+++ b/sound/usb/Kconfig
@@ -15,6 +15,7 @@ config SND_USB_AUDIO
 	select SND_RAWMIDI
 	select SND_PCM
 	select BITREVERSE
+	select SND_USB_AUDIO_USE_MEDIA_CONTROLLER if MEDIA_CONTROLLER && (MEDIA_SUPPORT=y || MEDIA_SUPPORT=SND_USB_AUDIO)
 	help
 	  Say Y here to include support for USB audio and USB MIDI
 	  devices.
@@ -22,6 +23,9 @@ config SND_USB_AUDIO
 	  To compile this driver as a module, choose M here: the module
 	  will be called snd-usb-audio.
 
+config SND_USB_AUDIO_USE_MEDIA_CONTROLLER
+	bool
+
 config SND_USB_UA101
 	tristate "Edirol UA-101/UA-1000 driver"
 	select SND_PCM
diff --git a/sound/usb/Makefile b/sound/usb/Makefile
index d330f74c90e6..e1ce257ab705 100644
--- a/sound/usb/Makefile
+++ b/sound/usb/Makefile
@@ -18,6 +18,8 @@ snd-usb-audio-objs := 	card.o \
 			quirks.o \
 			stream.o
 
+snd-usb-audio-$(CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER) += media.o
+
 snd-usbmidi-lib-objs := midi.o
 
 # Toplevel Module Dependency
diff --git a/sound/usb/card.c b/sound/usb/card.c
index a105947eaf55..4ad075e98dc2 100644
--- a/sound/usb/card.c
+++ b/sound/usb/card.c
@@ -68,6 +68,7 @@
 #include "format.h"
 #include "power.h"
 #include "stream.h"
+#include "media.h"
 
 MODULE_AUTHOR("Takashi Iwai <tiwai@suse.de>");
 MODULE_DESCRIPTION("USB Audio");
@@ -673,6 +674,11 @@ static int usb_audio_probe(struct usb_interface *intf,
 	if (err < 0)
 		goto __error;
 
+	if (quirk && quirk->shares_media_device) {
+		/* don't want to fail when snd_media_device_create() fails */
+		snd_media_device_create(chip, intf);
+	}
+
 	usb_chip[chip->index] = chip;
 	chip->num_interfaces++;
 	usb_set_intfdata(intf, chip);
@@ -732,6 +738,14 @@ static void usb_audio_disconnect(struct usb_interface *intf)
 		list_for_each(p, &chip->midi_list) {
 			snd_usbmidi_disconnect(p);
 		}
+		/*
+		 * Nice to check quirk && quirk->shares_media_device and
+		 * then call the snd_media_device_delete(). Don't have
+		 * access to the quirk here. snd_media_device_delete()
+		 * accesses mixer_list
+		 */
+		snd_media_device_delete(chip);
+
 		/* release mixer resources */
 		list_for_each_entry(mixer, &chip->mixer_list, list) {
 			snd_usb_mixer_disconnect(mixer);
diff --git a/sound/usb/card.h b/sound/usb/card.h
index ac785d15ced4..5dd3538ed6b5 100644
--- a/sound/usb/card.h
+++ b/sound/usb/card.h
@@ -108,6 +108,8 @@ struct snd_usb_endpoint {
 	struct list_head list;
 };
 
+struct media_ctl;
+
 struct snd_usb_substream {
 	struct snd_usb_stream *stream;
 	struct usb_device *dev;
@@ -160,6 +162,7 @@ struct snd_usb_substream {
 	} dsd_dop;
 
 	bool trigger_tstamp_pending_update; /* trigger timestamp being updated from initial estimate */
+	struct media_ctl *media_ctl;
 };
 
 struct snd_usb_stream {
diff --git a/sound/usb/media.c b/sound/usb/media.c
new file mode 100644
index 000000000000..eb6fe13523c8
--- /dev/null
+++ b/sound/usb/media.c
@@ -0,0 +1,321 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * media.c - Media Controller specific ALSA driver code
+ *
+ * Copyright (c) 2018 Shuah Khan <shuah@kernel.org>
+ *
+ */
+
+/*
+ * This file adds Media Controller support to the ALSA driver
+ * to use the Media Controller API to share the tuner with DVB
+ * and V4L2 drivers that control the media device.
+ *
+ * The media device is created based on the existing quirks framework.
+ * Using this approach, the media controller API usage can be added for
+ * a specific device.
+ */
+
+#include <linux/init.h>
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/slab.h>
+#include <linux/usb.h>
+
+#include <sound/pcm.h>
+#include <sound/core.h>
+
+#include "usbaudio.h"
+#include "card.h"
+#include "mixer.h"
+#include "media.h"
+
+int snd_media_stream_init(struct snd_usb_substream *subs, struct snd_pcm *pcm,
+			  int stream)
+{
+	struct media_device *mdev;
+	struct media_ctl *mctl;
+	struct device *pcm_dev = &pcm->streams[stream].dev;
+	u32 intf_type;
+	int ret = 0;
+	u16 mixer_pad;
+	struct media_entity *entity;
+
+	mdev = subs->stream->chip->media_dev;
+	if (!mdev)
+		return 0;
+
+	if (subs->media_ctl)
+		return 0;
+
+	/* allocate media_ctl */
+	mctl = kzalloc(sizeof(*mctl), GFP_KERNEL);
+	if (!mctl)
+		return -ENOMEM;
+
+	mctl->media_dev = mdev;
+	if (stream == SNDRV_PCM_STREAM_PLAYBACK) {
+		intf_type = MEDIA_INTF_T_ALSA_PCM_PLAYBACK;
+		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_PLAYBACK;
+		mctl->media_pad.flags = MEDIA_PAD_FL_SOURCE;
+		mixer_pad = 1;
+	} else {
+		intf_type = MEDIA_INTF_T_ALSA_PCM_CAPTURE;
+		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_CAPTURE;
+		mctl->media_pad.flags = MEDIA_PAD_FL_SINK;
+		mixer_pad = 2;
+	}
+	mctl->media_entity.name = pcm->name;
+	media_entity_pads_init(&mctl->media_entity, 1, &mctl->media_pad);
+	ret =  media_device_register_entity(mctl->media_dev,
+					    &mctl->media_entity);
+	if (ret)
+		goto free_mctl;
+
+	mctl->intf_devnode = media_devnode_create(mdev, intf_type, 0,
+						  MAJOR(pcm_dev->devt),
+						  MINOR(pcm_dev->devt));
+	if (!mctl->intf_devnode) {
+		ret = -ENOMEM;
+		goto unregister_entity;
+	}
+	mctl->intf_link = media_create_intf_link(&mctl->media_entity,
+						 &mctl->intf_devnode->intf,
+						 MEDIA_LNK_FL_ENABLED);
+	if (!mctl->intf_link) {
+		ret = -ENOMEM;
+		goto devnode_remove;
+	}
+
+	/* create link between mixer and audio */
+	media_device_for_each_entity(entity, mdev) {
+		switch (entity->function) {
+		case MEDIA_ENT_F_AUDIO_MIXER:
+			ret = media_create_pad_link(entity, mixer_pad,
+						    &mctl->media_entity, 0,
+						    MEDIA_LNK_FL_ENABLED);
+			if (ret)
+				goto remove_intf_link;
+			break;
+		}
+	}
+
+	subs->media_ctl = mctl;
+	return 0;
+
+remove_intf_link:
+	media_remove_intf_link(mctl->intf_link);
+devnode_remove:
+	media_devnode_remove(mctl->intf_devnode);
+unregister_entity:
+	media_device_unregister_entity(&mctl->media_entity);
+free_mctl:
+	kfree(mctl);
+	return ret;
+}
+
+void snd_media_stream_delete(struct snd_usb_substream *subs)
+{
+	struct media_ctl *mctl = subs->media_ctl;
+
+	if (mctl) {
+		struct media_device *mdev;
+
+		mdev = mctl->media_dev;
+		if (mdev && media_devnode_is_registered(mdev->devnode)) {
+			media_devnode_remove(mctl->intf_devnode);
+			media_device_unregister_entity(&mctl->media_entity);
+			media_entity_cleanup(&mctl->media_entity);
+		}
+		kfree(mctl);
+		subs->media_ctl = NULL;
+	}
+}
+
+int snd_media_start_pipeline(struct snd_usb_substream *subs)
+{
+	struct media_ctl *mctl = subs->media_ctl;
+	int ret = 0;
+
+	if (!mctl)
+		return 0;
+
+	mutex_lock(&mctl->media_dev->graph_mutex);
+	if (mctl->media_dev->enable_source)
+		ret = mctl->media_dev->enable_source(&mctl->media_entity,
+						     &mctl->media_pipe);
+	mutex_unlock(&mctl->media_dev->graph_mutex);
+	return ret;
+}
+
+void snd_media_stop_pipeline(struct snd_usb_substream *subs)
+{
+	struct media_ctl *mctl = subs->media_ctl;
+
+	if (!mctl)
+		return;
+
+	mutex_lock(&mctl->media_dev->graph_mutex);
+	if (mctl->media_dev->disable_source)
+		mctl->media_dev->disable_source(&mctl->media_entity);
+	mutex_unlock(&mctl->media_dev->graph_mutex);
+}
+
+static int snd_media_mixer_init(struct snd_usb_audio *chip)
+{
+	struct device *ctl_dev = &chip->card->ctl_dev;
+	struct media_intf_devnode *ctl_intf;
+	struct usb_mixer_interface *mixer;
+	struct media_device *mdev = chip->media_dev;
+	struct media_mixer_ctl *mctl;
+	u32 intf_type = MEDIA_INTF_T_ALSA_CONTROL;
+	int ret;
+
+	if (!mdev)
+		return -ENODEV;
+
+	ctl_intf = chip->ctl_intf_media_devnode;
+	if (!ctl_intf) {
+		ctl_intf = media_devnode_create(mdev, intf_type, 0,
+						MAJOR(ctl_dev->devt),
+						MINOR(ctl_dev->devt));
+		if (!ctl_intf)
+			return -ENOMEM;
+		chip->ctl_intf_media_devnode = ctl_intf;
+	}
+
+	list_for_each_entry(mixer, &chip->mixer_list, list) {
+
+		if (mixer->media_mixer_ctl)
+			continue;
+
+		/* allocate media_mixer_ctl */
+		mctl = kzalloc(sizeof(*mctl), GFP_KERNEL);
+		if (!mctl)
+			return -ENOMEM;
+
+		mctl->media_dev = mdev;
+		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_MIXER;
+		mctl->media_entity.name = chip->card->mixername;
+		mctl->media_pad[0].flags = MEDIA_PAD_FL_SINK;
+		mctl->media_pad[1].flags = MEDIA_PAD_FL_SOURCE;
+		mctl->media_pad[2].flags = MEDIA_PAD_FL_SOURCE;
+		media_entity_pads_init(&mctl->media_entity, MEDIA_MIXER_PAD_MAX,
+				  mctl->media_pad);
+		ret =  media_device_register_entity(mctl->media_dev,
+						    &mctl->media_entity);
+		if (ret) {
+			kfree(mctl);
+			return ret;
+		}
+
+		mctl->intf_link = media_create_intf_link(&mctl->media_entity,
+							 &ctl_intf->intf,
+							 MEDIA_LNK_FL_ENABLED);
+		if (!mctl->intf_link) {
+			media_device_unregister_entity(&mctl->media_entity);
+			media_entity_cleanup(&mctl->media_entity);
+			kfree(mctl);
+			return -ENOMEM;
+		}
+		mctl->intf_devnode = ctl_intf;
+		mixer->media_mixer_ctl = mctl;
+	}
+	return 0;
+}
+
+static void snd_media_mixer_delete(struct snd_usb_audio *chip)
+{
+	struct usb_mixer_interface *mixer;
+	struct media_device *mdev = chip->media_dev;
+
+	if (!mdev)
+		return;
+
+	list_for_each_entry(mixer, &chip->mixer_list, list) {
+		struct media_mixer_ctl *mctl;
+
+		mctl = mixer->media_mixer_ctl;
+		if (!mixer->media_mixer_ctl)
+			continue;
+
+		if (media_devnode_is_registered(mdev->devnode)) {
+			media_device_unregister_entity(&mctl->media_entity);
+			media_entity_cleanup(&mctl->media_entity);
+		}
+		kfree(mctl);
+		mixer->media_mixer_ctl = NULL;
+	}
+	if (media_devnode_is_registered(mdev->devnode))
+		media_devnode_remove(chip->ctl_intf_media_devnode);
+	chip->ctl_intf_media_devnode = NULL;
+}
+
+int snd_media_device_create(struct snd_usb_audio *chip,
+			struct usb_interface *iface)
+{
+	struct media_device *mdev;
+	struct usb_device *usbdev = interface_to_usbdev(iface);
+	int ret;
+
+	/* usb-audio driver is probed for each usb interface, and
+	 * there are multiple interfaces per device. Avoid calling
+	 * media_device_usb_allocate() each time usb_audio_probe()
+	 * is called. Do it only once.
+	 */
+	if (chip->media_dev)
+		goto snd_mixer_init;
+
+	mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
+	if (!mdev)
+		return -ENOMEM;
+
+	if (!media_devnode_is_registered(mdev->devnode)) {
+		/* register media_device */
+		ret = media_device_register(mdev);
+		if (ret) {
+			media_device_delete(mdev, KBUILD_MODNAME);
+			dev_err(&usbdev->dev,
+				"Couldn't register media device. Error: %d\n",
+				ret);
+			return ret;
+		}
+	}
+
+	/* save media device - avoid lookups */
+	chip->media_dev = mdev;
+
+snd_mixer_init:
+	/* Create media entities for mixer and control dev */
+	ret = snd_media_mixer_init(chip);
+	if (ret) {
+		dev_err(&usbdev->dev,
+			"Couldn't create media mixer entities. Error: %d\n",
+			ret);
+
+		/* clear saved media_dev */
+		chip->media_dev = NULL;
+
+		return ret;
+	}
+	return 0;
+}
+
+void snd_media_device_delete(struct snd_usb_audio *chip)
+{
+	struct media_device *mdev = chip->media_dev;
+	struct snd_usb_stream *stream;
+
+	/* release resources */
+	list_for_each_entry(stream, &chip->pcm_list, list) {
+		snd_media_stream_delete(&stream->substream[0]);
+		snd_media_stream_delete(&stream->substream[1]);
+	}
+
+	snd_media_mixer_delete(chip);
+
+	if (mdev) {
+		media_device_delete(mdev, KBUILD_MODNAME);
+		chip->media_dev = NULL;
+	}
+}
diff --git a/sound/usb/media.h b/sound/usb/media.h
new file mode 100644
index 000000000000..0e8a4785f858
--- /dev/null
+++ b/sound/usb/media.h
@@ -0,0 +1,74 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * media.h - Media Controller specific ALSA driver code
+ *
+ * Copyright (c) 2018 Shuah Khan <shuah@kernel.org>
+ *
+ */
+
+/*
+ * This file adds Media Controller support to the ALSA driver
+ * to use the Media Controller API to share the tuner with DVB
+ * and V4L2 drivers that control the media device.
+ *
+ * The media device is created based on the existing quirks framework.
+ * Using this approach, the media controller API usage can be added for
+ * a specific device.
+ */
+#ifndef __MEDIA_H
+
+#ifdef CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER
+
+#include <linux/media.h>
+#include <media/media-device.h>
+#include <media/media-entity.h>
+#include <media/media-dev-allocator.h>
+#include <sound/asound.h>
+
+struct media_ctl {
+	struct media_device *media_dev;
+	struct media_entity media_entity;
+	struct media_intf_devnode *intf_devnode;
+	struct media_link *intf_link;
+	struct media_pad media_pad;
+	struct media_pipeline media_pipe;
+};
+
+/*
+ * One source pad each for SNDRV_PCM_STREAM_CAPTURE and
+ * SNDRV_PCM_STREAM_PLAYBACK. One for sink pad to link
+ * to AUDIO Source
+ */
+#define MEDIA_MIXER_PAD_MAX    (SNDRV_PCM_STREAM_LAST + 2)
+
+struct media_mixer_ctl {
+	struct media_device *media_dev;
+	struct media_entity media_entity;
+	struct media_intf_devnode *intf_devnode;
+	struct media_link *intf_link;
+	struct media_pad media_pad[MEDIA_MIXER_PAD_MAX];
+	struct media_pipeline media_pipe;
+};
+
+int snd_media_device_create(struct snd_usb_audio *chip,
+			    struct usb_interface *iface);
+void snd_media_device_delete(struct snd_usb_audio *chip);
+int snd_media_stream_init(struct snd_usb_substream *subs, struct snd_pcm *pcm,
+			  int stream);
+void snd_media_stream_delete(struct snd_usb_substream *subs);
+int snd_media_start_pipeline(struct snd_usb_substream *subs);
+void snd_media_stop_pipeline(struct snd_usb_substream *subs);
+#else
+static inline int snd_media_device_create(struct snd_usb_audio *chip,
+					  struct usb_interface *iface)
+						{ return 0; }
+static inline void snd_media_device_delete(struct snd_usb_audio *chip) { }
+static inline int snd_media_stream_init(struct snd_usb_substream *subs,
+					struct snd_pcm *pcm, int stream)
+						{ return 0; }
+static inline void snd_media_stream_delete(struct snd_usb_substream *subs) { }
+static inline int snd_media_start_pipeline(struct snd_usb_substream *subs)
+					{ return 0; }
+static inline void snd_media_stop_pipeline(struct snd_usb_substream *subs) { }
+#endif
+#endif /* __MEDIA_H */
diff --git a/sound/usb/mixer.h b/sound/usb/mixer.h
index 3d12af8bf191..394cd9107507 100644
--- a/sound/usb/mixer.h
+++ b/sound/usb/mixer.h
@@ -4,6 +4,8 @@
 
 #include <sound/info.h>
 
+struct media_mixer_ctl;
+
 struct usb_mixer_interface {
 	struct snd_usb_audio *chip;
 	struct usb_host_interface *hostif;
@@ -23,6 +25,7 @@ struct usb_mixer_interface {
 	struct urb *rc_urb;
 	struct usb_ctrlrequest *rc_setup_packet;
 	u8 rc_buffer[6];
+	struct media_mixer_ctl *media_mixer_ctl;
 
 	bool disconnected;
 };
diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c
index 382847154227..538ed10ab66d 100644
--- a/sound/usb/pcm.c
+++ b/sound/usb/pcm.c
@@ -35,6 +35,7 @@
 #include "pcm.h"
 #include "clock.h"
 #include "power.h"
+#include "media.h"
 
 #define SUBSTREAM_FLAG_DATA_EP_STARTED	0
 #define SUBSTREAM_FLAG_SYNC_EP_STARTED	1
@@ -776,6 +777,10 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
 	struct audioformat *fmt;
 	int ret;
 
+	ret = snd_media_start_pipeline(subs);
+	if (ret)
+		return ret;
+
 	if (snd_usb_use_vmalloc)
 		ret = snd_pcm_lib_alloc_vmalloc_buffer(substream,
 						       params_buffer_bytes(hw_params));
@@ -783,7 +788,7 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
 		ret = snd_pcm_lib_malloc_pages(substream,
 					       params_buffer_bytes(hw_params));
 	if (ret < 0)
-		return ret;
+		goto stop_pipeline;
 
 	subs->pcm_format = params_format(hw_params);
 	subs->period_bytes = params_period_bytes(hw_params);
@@ -797,12 +802,13 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
 		dev_dbg(&subs->dev->dev,
 			"cannot set format: format = %#x, rate = %d, channels = %d\n",
 			   subs->pcm_format, subs->cur_rate, subs->channels);
-		return -EINVAL;
+		ret = -EINVAL;
+		goto stop_pipeline;
 	}
 
 	ret = snd_usb_lock_shutdown(subs->stream->chip);
 	if (ret < 0)
-		return ret;
+		goto stop_pipeline;
 
 	ret = snd_usb_pcm_change_state(subs, UAC3_PD_STATE_D0);
 	if (ret < 0)
@@ -818,6 +824,12 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
 
  unlock:
 	snd_usb_unlock_shutdown(subs->stream->chip);
+	if (ret < 0)
+		goto stop_pipeline;
+	return ret;
+
+ stop_pipeline:
+	snd_media_stop_pipeline(subs);
 	return ret;
 }
 
@@ -830,6 +842,7 @@ static int snd_usb_hw_free(struct snd_pcm_substream *substream)
 {
 	struct snd_usb_substream *subs = substream->runtime->private_data;
 
+	snd_media_stop_pipeline(subs);
 	subs->cur_audiofmt = NULL;
 	subs->cur_rate = 0;
 	subs->period_bytes = 0;
@@ -1302,6 +1315,7 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream)
 	struct snd_usb_stream *as = snd_pcm_substream_chip(substream);
 	struct snd_pcm_runtime *runtime = substream->runtime;
 	struct snd_usb_substream *subs = &as->substream[direction];
+	int ret;
 
 	subs->interface = -1;
 	subs->altset_idx = 0;
@@ -1315,7 +1329,13 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream)
 	subs->dsd_dop.channel = 0;
 	subs->dsd_dop.marker = 1;
 
-	return setup_hw_info(runtime, subs);
+	ret = setup_hw_info(runtime, subs);
+	if (ret == 0) {
+		ret = snd_media_stream_init(subs, as->pcm, direction);
+		if (ret)
+			snd_usb_autosuspend(subs->stream->chip);
+	}
+	return ret;
 }
 
 static int snd_usb_pcm_close(struct snd_pcm_substream *substream)
@@ -1326,6 +1346,7 @@ static int snd_usb_pcm_close(struct snd_pcm_substream *substream)
 	int ret;
 
 	stop_endpoints(subs, true);
+	snd_media_stop_pipeline(subs);
 
 	if (!as->chip->keep_iface &&
 	    subs->interface >= 0 &&
diff --git a/sound/usb/quirks-table.h b/sound/usb/quirks-table.h
index 37fc0447c071..85c7bdaad922 100644
--- a/sound/usb/quirks-table.h
+++ b/sound/usb/quirks-table.h
@@ -2887,6 +2887,7 @@ YAMAHA_DEVICE(0x7010, "UB99"),
 		.product_name = pname, \
 		.ifnum = QUIRK_ANY_INTERFACE, \
 		.type = QUIRK_AUDIO_ALIGN_TRANSFER, \
+		.shares_media_device = 1, \
 	} \
 }
 
diff --git a/sound/usb/stream.c b/sound/usb/stream.c
index 67cf849aa16b..f19d53445f38 100644
--- a/sound/usb/stream.c
+++ b/sound/usb/stream.c
@@ -38,6 +38,7 @@
 #include "clock.h"
 #include "stream.h"
 #include "power.h"
+#include "media.h"
 
 /*
  * free a substream
@@ -55,6 +56,7 @@ static void free_substream(struct snd_usb_substream *subs)
 	}
 	kfree(subs->rate_list.list);
 	kfree(subs->str_pd);
+	snd_media_stream_delete(subs);
 }
 
 
diff --git a/sound/usb/usbaudio.h b/sound/usb/usbaudio.h
index b9faeca645fd..0968a45c8925 100644
--- a/sound/usb/usbaudio.h
+++ b/sound/usb/usbaudio.h
@@ -30,6 +30,9 @@
  *
  */
 
+struct media_device;
+struct media_intf_devnode;
+
 struct snd_usb_audio {
 	int index;
 	struct usb_device *dev;
@@ -66,6 +69,8 @@ struct snd_usb_audio {
 					 */
 
 	struct usb_host_interface *ctrl_intf;	/* the audio control interface */
+	struct media_device *media_dev;
+	struct media_intf_devnode *ctl_intf_media_devnode;
 };
 
 #define usb_audio_err(chip, fmt, args...) \
@@ -117,6 +122,7 @@ struct snd_usb_audio_quirk {
 	const char *profile_name;	/* override the card->longname */
 	int16_t ifnum;
 	uint16_t type;
+	bool shares_media_device;
 	const void *data;
 };
 
-- 
2.17.1

Re: [alsa-devel] [PATCH v9 4/4] sound/usb: Use Media Controller API to share media resources

[Takashi Iwai]

On Tue, 18 Dec 2018 18:59:39 +0100,
shuah@kernel.org wrote:
> 
> From: Shuah Khan <shuah@kernel.org>
> 
> Media Device Allocator API to allows multiple drivers share a media device.
> This API solves a very common use-case for media devices where one physical
> device (an USB stick) provides both audio and video. When such media device
> exposes a standard USB Audio class, a proprietary Video class, two or more
> independent drivers will share a single physical USB bridge. In such cases,
> it is necessary to coordinate access to the shared resource.
> 
> Using this API, drivers can allocate a media device with the shared struct
> device as the key. Once the media device is allocated by a driver, other
> drivers can get a reference to it. The media device is released when all
> the references are released.
> 
> Change the ALSA driver to use the Media Controller API to share media
> resources with DVB, and V4L2 drivers on a AU0828 media device.
> 
> The Media Controller specific initialization is done after sound card is
> registered. ALSA creates Media interface and entity function graph nodes
> for Control, Mixer, PCM Playback, and PCM Capture devices.
> 
> snd_usb_hw_params() will call Media Controller enable source handler
> interface to request the media resource. If resource request is granted,
> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
> returned.
> 
> Media specific cleanup is done in usb_audio_disconnect().
> 
> Signed-off-by: Shuah Khan <shuah@kernel.org>

Feel free to take my ack regarding the sound stuff:
  Reviewed-by: Takashi Iwai <tiwai@suse.de>


Thanks!

Takashi

Re: [alsa-devel] [PATCH v9 4/4] sound/usb: Use Media Controller API to share media resources

[shuah]

On 12/19/18 6:51 AM, Takashi Iwai wrote:
> On Tue, 18 Dec 2018 18:59:39 +0100,
> shuah@kernel.org wrote:
>>
>> From: Shuah Khan <shuah@kernel.org>
>>
>> Media Device Allocator API to allows multiple drivers share a media device.
>> This API solves a very common use-case for media devices where one physical
>> device (an USB stick) provides both audio and video. When such media device
>> exposes a standard USB Audio class, a proprietary Video class, two or more
>> independent drivers will share a single physical USB bridge. In such cases,
>> it is necessary to coordinate access to the shared resource.
>>
>> Using this API, drivers can allocate a media device with the shared struct
>> device as the key. Once the media device is allocated by a driver, other
>> drivers can get a reference to it. The media device is released when all
>> the references are released.
>>
>> Change the ALSA driver to use the Media Controller API to share media
>> resources with DVB, and V4L2 drivers on a AU0828 media device.
>>
>> The Media Controller specific initialization is done after sound card is
>> registered. ALSA creates Media interface and entity function graph nodes
>> for Control, Mixer, PCM Playback, and PCM Capture devices.
>>
>> snd_usb_hw_params() will call Media Controller enable source handler
>> interface to request the media resource. If resource request is granted,
>> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
>> returned.
>>
>> Media specific cleanup is done in usb_audio_disconnect().
>>
>> Signed-off-by: Shuah Khan <shuah@kernel.org>
> 
> Feel free to take my ack regarding the sound stuff:
>    Reviewed-by: Takashi Iwai <tiwai@suse.de>
> 
> 
> Thanks!
> 
> Takashi
> 

Hi Mauro,

Any update on this patch series?

thanks,
-- Shuah

Re: [alsa-devel] [PATCH v9 4/4] sound/usb: Use Media Controller API to share media resources

[Hans Verkuil]

On 1/11/19 3:57 PM, shuah wrote:
> On 12/19/18 6:51 AM, Takashi Iwai wrote:
>> On Tue, 18 Dec 2018 18:59:39 +0100,
>> shuah@kernel.org wrote:
>>>
>>> From: Shuah Khan <shuah@kernel.org>
>>>
>>> Media Device Allocator API to allows multiple drivers share a media device.
>>> This API solves a very common use-case for media devices where one physical
>>> device (an USB stick) provides both audio and video. When such media device
>>> exposes a standard USB Audio class, a proprietary Video class, two or more
>>> independent drivers will share a single physical USB bridge. In such cases,
>>> it is necessary to coordinate access to the shared resource.
>>>
>>> Using this API, drivers can allocate a media device with the shared struct
>>> device as the key. Once the media device is allocated by a driver, other
>>> drivers can get a reference to it. The media device is released when all
>>> the references are released.
>>>
>>> Change the ALSA driver to use the Media Controller API to share media
>>> resources with DVB, and V4L2 drivers on a AU0828 media device.
>>>
>>> The Media Controller specific initialization is done after sound card is
>>> registered. ALSA creates Media interface and entity function graph nodes
>>> for Control, Mixer, PCM Playback, and PCM Capture devices.
>>>
>>> snd_usb_hw_params() will call Media Controller enable source handler
>>> interface to request the media resource. If resource request is granted,
>>> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
>>> returned.
>>>
>>> Media specific cleanup is done in usb_audio_disconnect().
>>>
>>> Signed-off-by: Shuah Khan <shuah@kernel.org>
>>
>> Feel free to take my ack regarding the sound stuff:
>>    Reviewed-by: Takashi Iwai <tiwai@suse.de>
>>
>>
>> Thanks!
>>
>> Takashi
>>
> 
> Hi Mauro,
> 
> Any update on this patch series?

I'm planning to process this series for 5.1. Haven't gotten around to it yet,
but I expect to do this next week.

Still going through all the pending patches after the Christmas period :-)

Regards,

	Hans

Re: [alsa-devel] [PATCH v9 4/4] sound/usb: Use Media Controller API to share media resources

[shuah]

On 1/11/19 7:59 AM, Hans Verkuil wrote:
> On 1/11/19 3:57 PM, shuah wrote:
>> On 12/19/18 6:51 AM, Takashi Iwai wrote:
>>> On Tue, 18 Dec 2018 18:59:39 +0100,
>>> shuah@kernel.org wrote:
>>>>
>>>> From: Shuah Khan <shuah@kernel.org>
>>>>
>>>> Media Device Allocator API to allows multiple drivers share a media device.
>>>> This API solves a very common use-case for media devices where one physical
>>>> device (an USB stick) provides both audio and video. When such media device
>>>> exposes a standard USB Audio class, a proprietary Video class, two or more
>>>> independent drivers will share a single physical USB bridge. In such cases,
>>>> it is necessary to coordinate access to the shared resource.
>>>>
>>>> Using this API, drivers can allocate a media device with the shared struct
>>>> device as the key. Once the media device is allocated by a driver, other
>>>> drivers can get a reference to it. The media device is released when all
>>>> the references are released.
>>>>
>>>> Change the ALSA driver to use the Media Controller API to share media
>>>> resources with DVB, and V4L2 drivers on a AU0828 media device.
>>>>
>>>> The Media Controller specific initialization is done after sound card is
>>>> registered. ALSA creates Media interface and entity function graph nodes
>>>> for Control, Mixer, PCM Playback, and PCM Capture devices.
>>>>
>>>> snd_usb_hw_params() will call Media Controller enable source handler
>>>> interface to request the media resource. If resource request is granted,
>>>> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
>>>> returned.
>>>>
>>>> Media specific cleanup is done in usb_audio_disconnect().
>>>>
>>>> Signed-off-by: Shuah Khan <shuah@kernel.org>
>>>
>>> Feel free to take my ack regarding the sound stuff:
>>>     Reviewed-by: Takashi Iwai <tiwai@suse.de>
>>>
>>>
>>> Thanks!
>>>
>>> Takashi
>>>
>>
>> Hi Mauro,
>>
>> Any update on this patch series?
> 
> I'm planning to process this series for 5.1. Haven't gotten around to it yet,
> but I expect to do this next week.
> 
> Still going through all the pending patches after the Christmas period :-)
> 
> Regards,
> 
> 	Hans
> 

Hans,

Thanks for a quick reply. No worries. I am recovering from the Christmas
and vacation myself. :)

thanks,
-- Shuah

Re: [PATCH v9 4/4] sound/usb: Use Media Controller API to share media resources

[Hans Verkuil]

On 12/18/18 6:59 PM, shuah@kernel.org wrote:
> From: Shuah Khan <shuah@kernel.org>
> 
> Media Device Allocator API to allows multiple drivers share a media device.
> This API solves a very common use-case for media devices where one physical
> device (an USB stick) provides both audio and video. When such media device
> exposes a standard USB Audio class, a proprietary Video class, two or more
> independent drivers will share a single physical USB bridge. In such cases,
> it is necessary to coordinate access to the shared resource.
> 
> Using this API, drivers can allocate a media device with the shared struct
> device as the key. Once the media device is allocated by a driver, other
> drivers can get a reference to it. The media device is released when all
> the references are released.
> 
> Change the ALSA driver to use the Media Controller API to share media
> resources with DVB, and V4L2 drivers on a AU0828 media device.
> 
> The Media Controller specific initialization is done after sound card is
> registered. ALSA creates Media interface and entity function graph nodes
> for Control, Mixer, PCM Playback, and PCM Capture devices.
> 
> snd_usb_hw_params() will call Media Controller enable source handler
> interface to request the media resource. If resource request is granted,
> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
> returned.
> 
> Media specific cleanup is done in usb_audio_disconnect().
> 
> Signed-off-by: Shuah Khan <shuah@kernel.org>
> ---
>  sound/usb/Kconfig        |   4 +
>  sound/usb/Makefile       |   2 +
>  sound/usb/card.c         |  14 ++
>  sound/usb/card.h         |   3 +
>  sound/usb/media.c        | 321 +++++++++++++++++++++++++++++++++++++++
>  sound/usb/media.h        |  74 +++++++++
>  sound/usb/mixer.h        |   3 +
>  sound/usb/pcm.c          |  29 +++-
>  sound/usb/quirks-table.h |   1 +
>  sound/usb/stream.c       |   2 +
>  sound/usb/usbaudio.h     |   6 +
>  11 files changed, 455 insertions(+), 4 deletions(-)
>  create mode 100644 sound/usb/media.c
>  create mode 100644 sound/usb/media.h
> 

<snip>

> +int snd_media_device_create(struct snd_usb_audio *chip,
> +			struct usb_interface *iface)
> +{
> +	struct media_device *mdev;
> +	struct usb_device *usbdev = interface_to_usbdev(iface);
> +	int ret;
> +
> +	/* usb-audio driver is probed for each usb interface, and
> +	 * there are multiple interfaces per device. Avoid calling
> +	 * media_device_usb_allocate() each time usb_audio_probe()
> +	 * is called. Do it only once.
> +	 */
> +	if (chip->media_dev)
> +		goto snd_mixer_init;
> +
> +	mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
> +	if (!mdev)
> +		return -ENOMEM;
> +
> +	if (!media_devnode_is_registered(mdev->devnode)) {

It looks like you missed my comment for v8:

"You should first configure the media device before registering it."

In other words, first create the media entities, and only then do you
register the media device. Otherwise it will come up without any alsa
entities, which are then added. So an application that immediately
opens the media device upon creation will see a topology that is still
in flux.

> +		/* register media_device */
> +		ret = media_device_register(mdev);
> +		if (ret) {
> +			media_device_delete(mdev, KBUILD_MODNAME);
> +			dev_err(&usbdev->dev,
> +				"Couldn't register media device. Error: %d\n",
> +				ret);
> +			return ret;
> +		}
> +	}
> +
> +	/* save media device - avoid lookups */
> +	chip->media_dev = mdev;
> +
> +snd_mixer_init:
> +	/* Create media entities for mixer and control dev */
> +	ret = snd_media_mixer_init(chip);
> +	if (ret) {
> +		dev_err(&usbdev->dev,
> +			"Couldn't create media mixer entities. Error: %d\n",
> +			ret);
> +
> +		/* clear saved media_dev */
> +		chip->media_dev = NULL;
> +
> +		return ret;
> +	}
> +	return 0;
> +}

I'll do some testing of this series on Monday.

Regards,

	Hans

Re: [PATCH v9 4/4] sound/usb: Use Media Controller API to share media resources

[shuah]

On 1/18/19 1:36 AM, Hans Verkuil wrote:
> On 12/18/18 6:59 PM, shuah@kernel.org wrote:
>> From: Shuah Khan <shuah@kernel.org>
>>
>> Media Device Allocator API to allows multiple drivers share a media device.
>> This API solves a very common use-case for media devices where one physical
>> device (an USB stick) provides both audio and video. When such media device
>> exposes a standard USB Audio class, a proprietary Video class, two or more
>> independent drivers will share a single physical USB bridge. In such cases,
>> it is necessary to coordinate access to the shared resource.
>>
>> Using this API, drivers can allocate a media device with the shared struct
>> device as the key. Once the media device is allocated by a driver, other
>> drivers can get a reference to it. The media device is released when all
>> the references are released.
>>
>> Change the ALSA driver to use the Media Controller API to share media
>> resources with DVB, and V4L2 drivers on a AU0828 media device.
>>
>> The Media Controller specific initialization is done after sound card is
>> registered. ALSA creates Media interface and entity function graph nodes
>> for Control, Mixer, PCM Playback, and PCM Capture devices.
>>
>> snd_usb_hw_params() will call Media Controller enable source handler
>> interface to request the media resource. If resource request is granted,
>> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
>> returned.
>>
>> Media specific cleanup is done in usb_audio_disconnect().
>>
>> Signed-off-by: Shuah Khan <shuah@kernel.org>
>> ---
>>   sound/usb/Kconfig        |   4 +
>>   sound/usb/Makefile       |   2 +
>>   sound/usb/card.c         |  14 ++
>>   sound/usb/card.h         |   3 +
>>   sound/usb/media.c        | 321 +++++++++++++++++++++++++++++++++++++++
>>   sound/usb/media.h        |  74 +++++++++
>>   sound/usb/mixer.h        |   3 +
>>   sound/usb/pcm.c          |  29 +++-
>>   sound/usb/quirks-table.h |   1 +
>>   sound/usb/stream.c       |   2 +
>>   sound/usb/usbaudio.h     |   6 +
>>   11 files changed, 455 insertions(+), 4 deletions(-)
>>   create mode 100644 sound/usb/media.c
>>   create mode 100644 sound/usb/media.h
>>
> 
> <snip>
> 
>> +int snd_media_device_create(struct snd_usb_audio *chip,
>> +			struct usb_interface *iface)
>> +{
>> +	struct media_device *mdev;
>> +	struct usb_device *usbdev = interface_to_usbdev(iface);
>> +	int ret;
>> +
>> +	/* usb-audio driver is probed for each usb interface, and
>> +	 * there are multiple interfaces per device. Avoid calling
>> +	 * media_device_usb_allocate() each time usb_audio_probe()
>> +	 * is called. Do it only once.
>> +	 */
>> +	if (chip->media_dev)
>> +		goto snd_mixer_init;
>> +
>> +	mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
>> +	if (!mdev)
>> +		return -ENOMEM;
>> +
>> +	if (!media_devnode_is_registered(mdev->devnode)) {
> 
> It looks like you missed my comment for v8:
> 
> "You should first configure the media device before registering it."
> 
> In other words, first create the media entities, and only then do you
> register the media device. Otherwise it will come up without any alsa
> entities, which are then added. So an application that immediately
> opens the media device upon creation will see a topology that is still
> in flux.

Yes. You are right. I saw your comment and thought I got it addressed.
I will fix it. I have the logic correct in au0828, but not here.

Thanks,
-- Shuah

Re: [PATCH v9 4/4] sound/usb: Use Media Controller API to share media resources

[shuah]

On 1/18/19 2:54 PM, shuah wrote:
> On 1/18/19 1:36 AM, Hans Verkuil wrote:
>> On 12/18/18 6:59 PM, shuah@kernel.org wrote:
>>> From: Shuah Khan <shuah@kernel.org>
>>>
>>> Media Device Allocator API to allows multiple drivers share a media 
>>> device.
>>> This API solves a very common use-case for media devices where one 
>>> physical
>>> device (an USB stick) provides both audio and video. When such media 
>>> device
>>> exposes a standard USB Audio class, a proprietary Video class, two or 
>>> more
>>> independent drivers will share a single physical USB bridge. In such 
>>> cases,
>>> it is necessary to coordinate access to the shared resource.
>>>
>>> Using this API, drivers can allocate a media device with the shared 
>>> struct
>>> device as the key. Once the media device is allocated by a driver, other
>>> drivers can get a reference to it. The media device is released when all
>>> the references are released.
>>>
>>> Change the ALSA driver to use the Media Controller API to share media
>>> resources with DVB, and V4L2 drivers on a AU0828 media device.
>>>
>>> The Media Controller specific initialization is done after sound card is
>>> registered. ALSA creates Media interface and entity function graph nodes
>>> for Control, Mixer, PCM Playback, and PCM Capture devices.
>>>
>>> snd_usb_hw_params() will call Media Controller enable source handler
>>> interface to request the media resource. If resource request is granted,
>>> it will release it from snd_usb_hw_free(). If resource is busy, 
>>> -EBUSY is
>>> returned.
>>>
>>> Media specific cleanup is done in usb_audio_disconnect().
>>>
>>> Signed-off-by: Shuah Khan <shuah@kernel.org>
>>> ---
>>>   sound/usb/Kconfig        |   4 +
>>>   sound/usb/Makefile       |   2 +
>>>   sound/usb/card.c         |  14 ++
>>>   sound/usb/card.h         |   3 +
>>>   sound/usb/media.c        | 321 +++++++++++++++++++++++++++++++++++++++
>>>   sound/usb/media.h        |  74 +++++++++
>>>   sound/usb/mixer.h        |   3 +
>>>   sound/usb/pcm.c          |  29 +++-
>>>   sound/usb/quirks-table.h |   1 +
>>>   sound/usb/stream.c       |   2 +
>>>   sound/usb/usbaudio.h     |   6 +
>>>   11 files changed, 455 insertions(+), 4 deletions(-)
>>>   create mode 100644 sound/usb/media.c
>>>   create mode 100644 sound/usb/media.h
>>>
>>
>> <snip>
>>
>>> +int snd_media_device_create(struct snd_usb_audio *chip,
>>> +            struct usb_interface *iface)
>>> +{
>>> +    struct media_device *mdev;
>>> +    struct usb_device *usbdev = interface_to_usbdev(iface);
>>> +    int ret;
>>> +
>>> +    /* usb-audio driver is probed for each usb interface, and
>>> +     * there are multiple interfaces per device. Avoid calling
>>> +     * media_device_usb_allocate() each time usb_audio_probe()
>>> +     * is called. Do it only once.
>>> +     */
>>> +    if (chip->media_dev)
>>> +        goto snd_mixer_init;
>>> +
>>> +    mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
>>> +    if (!mdev)
>>> +        return -ENOMEM;
>>> +
>>> +    if (!media_devnode_is_registered(mdev->devnode)) {
>>
>> It looks like you missed my comment for v8:
>>
>> "You should first configure the media device before registering it."
>>
>> In other words, first create the media entities, and only then do you
>> register the media device. Otherwise it will come up without any alsa
>> entities, which are then added. So an application that immediately
>> opens the media device upon creation will see a topology that is still
>> in flux.
> 
> Yes. You are right. I saw your comment and thought I got it addressed.
> I will fix it. I have the logic correct in au0828, but not here.
> 

One thing to mention here is some ALSA entities get created dynamically
during PCM open when stream is initialized. This happens after media
device is registered. These get deleted when pcm close happens. There is
no way to avoid creating entities after media device register.

snd_media_device_create() could get called after media_device_register()
happens, if au0828 registers it first.

I did find a problem in snd_media_device_create() and I will send a v10
with that fix.

thanks,
-- Shuah

Re: [PATCH v9 4/4] sound/usb: Use Media Controller API to share media resources

[Hans Verkuil]

On 01/19/2019 02:03 AM, shuah wrote:
> On 1/18/19 2:54 PM, shuah wrote:
>> On 1/18/19 1:36 AM, Hans Verkuil wrote:
>>> On 12/18/18 6:59 PM, shuah@kernel.org wrote:
>>>> From: Shuah Khan <shuah@kernel.org>
>>>>
>>>> Media Device Allocator API to allows multiple drivers share a media 
>>>> device.
>>>> This API solves a very common use-case for media devices where one 
>>>> physical
>>>> device (an USB stick) provides both audio and video. When such media 
>>>> device
>>>> exposes a standard USB Audio class, a proprietary Video class, two or 
>>>> more
>>>> independent drivers will share a single physical USB bridge. In such 
>>>> cases,
>>>> it is necessary to coordinate access to the shared resource.
>>>>
>>>> Using this API, drivers can allocate a media device with the shared 
>>>> struct
>>>> device as the key. Once the media device is allocated by a driver, other
>>>> drivers can get a reference to it. The media device is released when all
>>>> the references are released.
>>>>
>>>> Change the ALSA driver to use the Media Controller API to share media
>>>> resources with DVB, and V4L2 drivers on a AU0828 media device.
>>>>
>>>> The Media Controller specific initialization is done after sound card is
>>>> registered. ALSA creates Media interface and entity function graph nodes
>>>> for Control, Mixer, PCM Playback, and PCM Capture devices.
>>>>
>>>> snd_usb_hw_params() will call Media Controller enable source handler
>>>> interface to request the media resource. If resource request is granted,
>>>> it will release it from snd_usb_hw_free(). If resource is busy, 
>>>> -EBUSY is
>>>> returned.
>>>>
>>>> Media specific cleanup is done in usb_audio_disconnect().
>>>>
>>>> Signed-off-by: Shuah Khan <shuah@kernel.org>
>>>> ---
>>>>   sound/usb/Kconfig        |   4 +
>>>>   sound/usb/Makefile       |   2 +
>>>>   sound/usb/card.c         |  14 ++
>>>>   sound/usb/card.h         |   3 +
>>>>   sound/usb/media.c        | 321 +++++++++++++++++++++++++++++++++++++++
>>>>   sound/usb/media.h        |  74 +++++++++
>>>>   sound/usb/mixer.h        |   3 +
>>>>   sound/usb/pcm.c          |  29 +++-
>>>>   sound/usb/quirks-table.h |   1 +
>>>>   sound/usb/stream.c       |   2 +
>>>>   sound/usb/usbaudio.h     |   6 +
>>>>   11 files changed, 455 insertions(+), 4 deletions(-)
>>>>   create mode 100644 sound/usb/media.c
>>>>   create mode 100644 sound/usb/media.h
>>>>
>>>
>>> <snip>
>>>
>>>> +int snd_media_device_create(struct snd_usb_audio *chip,
>>>> +            struct usb_interface *iface)
>>>> +{
>>>> +    struct media_device *mdev;
>>>> +    struct usb_device *usbdev = interface_to_usbdev(iface);
>>>> +    int ret;
>>>> +
>>>> +    /* usb-audio driver is probed for each usb interface, and
>>>> +     * there are multiple interfaces per device. Avoid calling
>>>> +     * media_device_usb_allocate() each time usb_audio_probe()
>>>> +     * is called. Do it only once.
>>>> +     */
>>>> +    if (chip->media_dev)
>>>> +        goto snd_mixer_init;
>>>> +
>>>> +    mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
>>>> +    if (!mdev)
>>>> +        return -ENOMEM;
>>>> +
>>>> +    if (!media_devnode_is_registered(mdev->devnode)) {
>>>
>>> It looks like you missed my comment for v8:
>>>
>>> "You should first configure the media device before registering it."
>>>
>>> In other words, first create the media entities, and only then do you
>>> register the media device. Otherwise it will come up without any alsa
>>> entities, which are then added. So an application that immediately
>>> opens the media device upon creation will see a topology that is still
>>> in flux.
>>
>> Yes. You are right. I saw your comment and thought I got it addressed.
>> I will fix it. I have the logic correct in au0828, but not here.
>>
> 
> One thing to mention here is some ALSA entities get created dynamically
> during PCM open when stream is initialized. This happens after media
> device is registered. These get deleted when pcm close happens. There is
> no way to avoid creating entities after media device register.

That's OK. We're missing some infrastructure (media events) to inform the
application about topology changes.

But when registering the media device for the first time it is good practice
to do that after creating all the entities that you can.

Regards,

	Hans

> snd_media_device_create() could get called after media_device_register()
> happens, if au0828 registers it first.
> 
> I did find a problem in snd_media_device_create() and I will send a v10
> with that fix.
> 
> thanks,
> -- Shuah
> 

Re: [PATCH v9 4/4] sound/usb: Use Media Controller API to share media resources

[shuah]

On 1/19/19 3:30 AM, Hans Verkuil wrote:
> On 01/19/2019 02:03 AM, shuah wrote:
>> On 1/18/19 2:54 PM, shuah wrote:
>>> On 1/18/19 1:36 AM, Hans Verkuil wrote:
>>>> On 12/18/18 6:59 PM, shuah@kernel.org wrote:
>>>>> From: Shuah Khan <shuah@kernel.org>
>>>>>
>>>>> Media Device Allocator API to allows multiple drivers share a media
>>>>> device.
>>>>> This API solves a very common use-case for media devices where one
>>>>> physical
>>>>> device (an USB stick) provides both audio and video. When such media
>>>>> device
>>>>> exposes a standard USB Audio class, a proprietary Video class, two or
>>>>> more
>>>>> independent drivers will share a single physical USB bridge. In such
>>>>> cases,
>>>>> it is necessary to coordinate access to the shared resource.
>>>>>
>>>>> Using this API, drivers can allocate a media device with the shared
>>>>> struct
>>>>> device as the key. Once the media device is allocated by a driver, other
>>>>> drivers can get a reference to it. The media device is released when all
>>>>> the references are released.
>>>>>
>>>>> Change the ALSA driver to use the Media Controller API to share media
>>>>> resources with DVB, and V4L2 drivers on a AU0828 media device.
>>>>>
>>>>> The Media Controller specific initialization is done after sound card is
>>>>> registered. ALSA creates Media interface and entity function graph nodes
>>>>> for Control, Mixer, PCM Playback, and PCM Capture devices.
>>>>>
>>>>> snd_usb_hw_params() will call Media Controller enable source handler
>>>>> interface to request the media resource. If resource request is granted,
>>>>> it will release it from snd_usb_hw_free(). If resource is busy,
>>>>> -EBUSY is
>>>>> returned.
>>>>>
>>>>> Media specific cleanup is done in usb_audio_disconnect().
>>>>>
>>>>> Signed-off-by: Shuah Khan <shuah@kernel.org>
>>>>> ---
>>>>>    sound/usb/Kconfig        |   4 +
>>>>>    sound/usb/Makefile       |   2 +
>>>>>    sound/usb/card.c         |  14 ++
>>>>>    sound/usb/card.h         |   3 +
>>>>>    sound/usb/media.c        | 321 +++++++++++++++++++++++++++++++++++++++
>>>>>    sound/usb/media.h        |  74 +++++++++
>>>>>    sound/usb/mixer.h        |   3 +
>>>>>    sound/usb/pcm.c          |  29 +++-
>>>>>    sound/usb/quirks-table.h |   1 +
>>>>>    sound/usb/stream.c       |   2 +
>>>>>    sound/usb/usbaudio.h     |   6 +
>>>>>    11 files changed, 455 insertions(+), 4 deletions(-)
>>>>>    create mode 100644 sound/usb/media.c
>>>>>    create mode 100644 sound/usb/media.h
>>>>>
>>>>
>>>> <snip>
>>>>
>>>>> +int snd_media_device_create(struct snd_usb_audio *chip,
>>>>> +            struct usb_interface *iface)
>>>>> +{
>>>>> +    struct media_device *mdev;
>>>>> +    struct usb_device *usbdev = interface_to_usbdev(iface);
>>>>> +    int ret;
>>>>> +
>>>>> +    /* usb-audio driver is probed for each usb interface, and
>>>>> +     * there are multiple interfaces per device. Avoid calling
>>>>> +     * media_device_usb_allocate() each time usb_audio_probe()
>>>>> +     * is called. Do it only once.
>>>>> +     */
>>>>> +    if (chip->media_dev)
>>>>> +        goto snd_mixer_init;
>>>>> +
>>>>> +    mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
>>>>> +    if (!mdev)
>>>>> +        return -ENOMEM;
>>>>> +
>>>>> +    if (!media_devnode_is_registered(mdev->devnode)) {
>>>>
>>>> It looks like you missed my comment for v8:
>>>>
>>>> "You should first configure the media device before registering it."
>>>>
>>>> In other words, first create the media entities, and only then do you
>>>> register the media device. Otherwise it will come up without any alsa
>>>> entities, which are then added. So an application that immediately
>>>> opens the media device upon creation will see a topology that is still
>>>> in flux.
>>>
>>> Yes. You are right. I saw your comment and thought I got it addressed.
>>> I will fix it. I have the logic correct in au0828, but not here.
>>>
>>
>> One thing to mention here is some ALSA entities get created dynamically
>> during PCM open when stream is initialized. This happens after media
>> device is registered. These get deleted when pcm close happens. There is
>> no way to avoid creating entities after media device register.
> 
> That's OK. We're missing some infrastructure (media events) to inform the
> application about topology changes.
> 
> But when registering the media device for the first time it is good practice
> to do that after creating all the entities that you can.
> 

Great. We have the same understanding. I think at least one mixer
controller entries can be created prior to registering the media
device. I am working on v10 to do that. Sorry I misread init sequence
in this patch the flow the first time around for v9.

thanks,
-- Shuah

Re: [PATCH v9 0/4] Media Device Allocator API

[Hans Verkuil]

Hi Shuah,

On 12/18/2018 06:59 PM, shuah@kernel.org wrote:
> From: Shuah Khan <shuah@kernel.org>
> 
> Media Device Allocator API to allows multiple drivers share a media device.
> This API solves a very common use-case for media devices where one physical
> device (an USB stick) provides both audio and video. When such media device
> exposes a standard USB Audio class, a proprietary Video class, two or more
> independent drivers will share a single physical USB bridge. In such cases,
> it is necessary to coordinate access to the shared resource.
> 
> Using this API, drivers can allocate a media device with the shared struct
> device as the key. Once the media device is allocated by a driver, other
> drivers can get a reference to it. The media device is released when all
> the references are released.
> 
> - Tested sharing resources with kaffeine, vlc, xawtv, tvtime, and
>   arecord. When analog is streaming, digital and audio user-space
>   applications detect that the tuner is busy and exit. When digital
>   is streaming, analog and audio applications detect that the tuner is
>   busy and exit. When arecord is owns the tuner, digital and analog
>   detect that the tuner is busy and exit.
> - Tested media device allocator API with bind/unbind testing on
>   snd-usb-audio and au0828 drivers to make sure /dev/mediaX is released
>   only when the last driver is unbound.
> - This patch series is tested on 4.20-rc6
> - Addressed review comments from Hans on the RFC v8 (rebased on 4.19)
> - Updated change log to describe the use-case more clearly.
> - No changes to 0001,0002 code since the v7 referenced below.
> - 0003 is a new patch to enable ALSA defines that have been
>   disabled for kernel between 4.9 and 4.19.
> - Minor merge conflict resolution in 0004.
> - Added SPDX to new files.
> 
> References:
> https://lkml.org/lkml/2018/11/2/169
> https://www.mail-archive.com/linux-media@vger.kernel.org/msg105854.html

When I connect my au0828 to my laptop with your v9 patch series applied I get
these warnings:

[   45.416047] xhci_hcd 0000:39:00.0: xHCI Host Controller
[   45.417882] xhci_hcd 0000:39:00.0: new USB bus registered, assigned bus number 3
[   45.419292] xhci_hcd 0000:39:00.0: hcc params 0x200077c1 hci version 0x110 quirks 0x0000000200009810
[   45.420835] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.00
[   45.420893] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   45.420899] usb usb3: Product: xHCI Host Controller
[   45.420905] usb usb3: Manufacturer: Linux 5.0.0-rc1-zen xhci-hcd
[   45.420911] usb usb3: SerialNumber: 0000:39:00.0
[   45.424290] hub 3-0:1.0: USB hub found
[   45.425274] hub 3-0:1.0: 2 ports detected
[   45.431061] xhci_hcd 0000:39:00.0: xHCI Host Controller
[   45.432436] xhci_hcd 0000:39:00.0: new USB bus registered, assigned bus number 4
[   45.432448] xhci_hcd 0000:39:00.0: Host supports USB 3.1 Enhanced SuperSpeed
[   45.433299] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.00
[   45.433354] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   45.433360] usb usb4: Product: xHCI Host Controller
[   45.433365] usb usb4: Manufacturer: Linux 5.0.0-rc1-zen xhci-hcd
[   45.433370] usb usb4: SerialNumber: 0000:39:00.0
[   45.436134] hub 4-0:1.0: USB hub found
[   45.436576] hub 4-0:1.0: 2 ports detected
[   45.750940] usb 3-1: new high-speed USB device number 2 using xhci_hcd
[   45.899927] usb 3-1: New USB device found, idVendor=2040, idProduct=721e, bcdDevice= 0.05
[   45.899949] usb 3-1: New USB device strings: Mfr=1, Product=2, SerialNumber=10
[   45.899960] usb 3-1: Product: WinTV Aero-A
[   45.899970] usb 3-1: Manufacturer: Hauppauge
[   45.899979] usb 3-1: SerialNumber: 4033622430
[   46.053476] au0828: au0828 driver loaded
[   46.053726] WARNING: CPU: 1 PID: 1824 at kernel/module.c:262 module_assert_mutex+0x20/0x30
[   46.053818] Modules linked in: au0828(+) tveeprom dvb_core v4l2_common rfcomm bnep snd_hda_codec_hdmi snd_hda_codec_realtek
snd_hda_codec_generic uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common videodev media btusb btintel bluetooth
snd_soc_skl snd_soc_skl_ipc snd_soc_sst_ipc snd_soc_sst_dsp snd_soc_acpi_intel_match snd_soc_acpi snd_hda_ext_core x86_pkg_temp_thermal
snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine snd_hda_intel iwlmvm snd_hda_codec snd_hda_core snd_hwdep snd_pcm snd_timer snd iwlwifi
i915 intel_gtt battery ac pcc_cpufreq thermal
[   46.053853] CPU: 1 PID: 1824 Comm: systemd-udevd Not tainted 5.0.0-rc1-zen #85
[   46.053856] Hardware name: ASUSTeK COMPUTER INC. UX490UA/UX490UA, BIOS UX490UA.312 04/09/2018
[   46.053862] RIP: 0010:module_assert_mutex+0x20/0x30
[   46.053867] Code: 5d c3 e8 f3 68 f5 ff 0f 1f 00 8b 05 d2 5a 7c 01 85 c0 75 01 c3 be ff ff ff ff 48 c7 c7 80 bc 86 82 e8 e4 27 fb ff 85 c0
75 ea <0f> 0b c3 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 53 48 89 fb e8 c7
[   46.053871] RSP: 0018:ffffc90002aa3ac8 EFLAGS: 00010246
[   46.053876] RAX: 0000000000000000 RBX: ffffffffa0573abf RCX: 0000000000000000
[   46.053880] RDX: 0000000000000000 RSI: ffffffff8286bc80 RDI: ffff8882a517d570
[   46.053883] RBP: ffff8882880e6000 R08: 0000000000000000 R09: ffff8882b1f5d000
[   46.053886] R10: 0000000000000001 R11: 0000000000000003 R12: ffffffffa0573abf
[   46.053890] R13: ffff8882880e60a0 R14: ffff8882880e6000 R15: ffffffffa0577200
[   46.053894] FS:  00007f0ac7d318c0(0000) GS:ffff8882b6a80000(0000) knlGS:0000000000000000
[   46.053898] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   46.053902] CR2: 0000560d354ed800 CR3: 00000002880f9003 CR4: 00000000003606e0
[   46.053905] Call Trace:
[   46.053910]  find_module+0x9/0x20
[   46.053918]  media_device_usb_allocate+0x10b/0x170 [media]
[   46.053929]  au0828_usb_probe+0x1bd/0x290 [au0828]
[   46.053954]  usb_probe_interface+0xe8/0x2a0
[   46.053960]  really_probe+0xee/0x2a0
[   46.053966]  driver_probe_device+0x4a/0xb0
[   46.053970]  __driver_attach+0xb3/0xd0
[   46.053975]  ? driver_probe_device+0xb0/0xb0
[   46.053979]  bus_for_each_dev+0x74/0xc0
[   46.053985]  bus_add_driver+0x19a/0x1e0
[   46.053990]  driver_register+0x66/0xb0
[   46.053995]  usb_register_driver+0x9a/0x150
[   46.053999]  ? 0xffffffffa0284000
[   46.054007]  au0828_init+0xb3/0x1000 [au0828]
[   46.054012]  do_one_initcall+0x61/0x2fb
[   46.054016]  ? do_init_module+0x1d/0x1e0
[   46.054021]  ? rcu_read_lock_sched_held+0x6f/0x80
[   46.054026]  ? kmem_cache_alloc_trace+0x123/0x210
[   46.054032]  do_init_module+0x55/0x1e0
[   46.054036]  load_module+0x145b/0x1710
[   46.054046]  ? vfs_read+0x128/0x150
[   46.054057]  ? __do_sys_finit_module+0xba/0xe0
[   46.054060]  __do_sys_finit_module+0xba/0xe0
[   46.054071]  do_syscall_64+0x4b/0x180
[   46.054077]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[   46.054080] RIP: 0033:0x7f0ac8993a79
[   46.054084] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08
0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d df 43 0c 00 f7 d8 64 89 01 48
[   46.054087] RSP: 002b:00007ffcbd9034f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[   46.054092] RAX: ffffffffffffffda RBX: 0000560d35494b30 RCX: 00007f0ac8993a79
[   46.054095] RDX: 0000000000000000 RSI: 00007f0ac869b0ed RDI: 0000000000000011
[   46.054098] RBP: 00007f0ac869b0ed R08: 0000000000000000 R09: 0000000000000000
[   46.054101] R10: 0000000000000011 R11: 0000000000000246 R12: 0000000000000000
[   46.054104] R13: 0000560d3547aef0 R14: 0000000000020000 R15: 0000560d35494b30
[   46.054114] irq event stamp: 6660
[   46.054119] hardirqs last  enabled at (6659): [<ffffffff81251eca>] get_page_from_freelist.part.104+0x10fa/0x15c0
[   46.054124] hardirqs last disabled at (6660): [<ffffffff81001c56>] trace_hardirqs_off_thunk+0x1a/0x1c
[   46.054129] softirqs last  enabled at (6488): [<ffffffff819f628c>] peernet2id+0x4c/0x70
[   46.054132] softirqs last disabled at (6486): [<ffffffff819f626d>] peernet2id+0x2d/0x70
[   46.054134] ---[ end trace 33cd45564590d7c7 ]---
[   46.054169] WARNING: CPU: 1 PID: 1824 at kernel/module.c:272 module_assert_mutex_or_preempt+0x29/0x30
[   46.054172] Modules linked in: au0828(+) tveeprom dvb_core v4l2_common rfcomm bnep snd_hda_codec_hdmi snd_hda_codec_realtek
snd_hda_codec_generic uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common videodev media btusb btintel bluetooth
snd_soc_skl snd_soc_skl_ipc snd_soc_sst_ipc snd_soc_sst_dsp snd_soc_acpi_intel_match snd_soc_acpi snd_hda_ext_core x86_pkg_temp_thermal
snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine snd_hda_intel iwlmvm snd_hda_codec snd_hda_core snd_hwdep snd_pcm snd_timer snd iwlwifi
i915 intel_gtt battery ac pcc_cpufreq thermal
[   46.054198] CPU: 1 PID: 1824 Comm: systemd-udevd Tainted: G        W         5.0.0-rc1-zen #85
[   46.054201] Hardware name: ASUSTeK COMPUTER INC. UX490UA/UX490UA, BIOS UX490UA.312 04/09/2018
[   46.054206] RIP: 0010:module_assert_mutex_or_preempt+0x29/0x30
[   46.054210] Code: 00 8b 05 a2 5e 7c 01 85 c0 74 09 e8 11 5b fd ff 85 c0 74 01 c3 be ff ff ff ff 48 c7 c7 80 bc 86 82 e8 ab 2b fb ff 85 c0
75 ea <0f> 0b c3 0f 1f 40 00 41 56 49 89 fe 41 55 41 89 d5 41 54 49 89 f4
[   46.054215] RSP: 0018:ffffc90002aa3aa8 EFLAGS: 00010246
[   46.054219] RAX: 0000000000000000 RBX: ffff8882b1f5d000 RCX: 0000000000000000
[   46.054222] RDX: 0000000000000000 RSI: ffffffff8286bc80 RDI: ffff8882a517d570
[   46.054225] RBP: ffff8882880e6000 R08: 0000000000000000 R09: ffff8882b1f5d000
[   46.054229] R10: 0000000000000001 R11: 0000000000000003 R12: 0000000000000006
[   46.054232] R13: 0000000000000000 R14: ffffffffa0573abf R15: ffffffffa0577200
[   46.054236] FS:  00007f0ac7d318c0(0000) GS:ffff8882b6a80000(0000) knlGS:0000000000000000
[   46.054239] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   46.054243] CR2: 0000560d354ed800 CR3: 00000002880f9003 CR4: 00000000003606e0
[   46.054246] Call Trace:
[   46.054251]  find_module_all+0x16/0x90
[   46.054260]  media_device_usb_allocate+0x10b/0x170 [media]
[   46.054269]  au0828_usb_probe+0x1bd/0x290 [au0828]
[   46.054277]  usb_probe_interface+0xe8/0x2a0
[   46.054285]  really_probe+0xee/0x2a0
[   46.054292]  driver_probe_device+0x4a/0xb0
[   46.054297]  __driver_attach+0xb3/0xd0
[   46.054304]  ? driver_probe_device+0xb0/0xb0
[   46.054311]  bus_for_each_dev+0x74/0xc0
[   46.054321]  bus_add_driver+0x19a/0x1e0
[   46.054330]  driver_register+0x66/0xb0
[   46.054337]  usb_register_driver+0x9a/0x150
[   46.054344]  ? 0xffffffffa0284000
[   46.054353]  au0828_init+0xb3/0x1000 [au0828]
[   46.054361]  do_one_initcall+0x61/0x2fb
[   46.054367]  ? do_init_module+0x1d/0x1e0
[   46.054375]  ? rcu_read_lock_sched_held+0x6f/0x80
[   46.054381]  ? kmem_cache_alloc_trace+0x123/0x210
[   46.054391]  do_init_module+0x55/0x1e0
[   46.054398]  load_module+0x145b/0x1710
[   46.054411]  ? vfs_read+0x128/0x150
[   46.054427]  ? __do_sys_finit_module+0xba/0xe0
[   46.054433]  __do_sys_finit_module+0xba/0xe0
[   46.054446]  do_syscall_64+0x4b/0x180
[   46.054454]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[   46.054461] RIP: 0033:0x7f0ac8993a79
[   46.054467] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08
0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d df 43 0c 00 f7 d8 64 89 01 48
[   46.054473] RSP: 002b:00007ffcbd9034f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[   46.054477] RAX: ffffffffffffffda RBX: 0000560d35494b30 RCX: 00007f0ac8993a79
[   46.054481] RDX: 0000000000000000 RSI: 00007f0ac869b0ed RDI: 0000000000000011
[   46.054484] RBP: 00007f0ac869b0ed R08: 0000000000000000 R09: 0000000000000000
[   46.054488] R10: 0000000000000011 R11: 0000000000000246 R12: 0000000000000000
[   46.054491] R13: 0000560d3547aef0 R14: 0000000000020000 R15: 0000560d35494b30
[   46.054502] irq event stamp: 6674
[   46.054507] hardirqs last  enabled at (6673): [<ffffffff81001c3a>] trace_hardirqs_on_thunk+0x1a/0x1c
[   46.054511] hardirqs last disabled at (6674): [<ffffffff81001c56>] trace_hardirqs_off_thunk+0x1a/0x1c
[   46.054516] softirqs last  enabled at (6672): [<ffffffff820002c0>] __do_softirq+0x2c0/0x4c0
[   46.054521] softirqs last disabled at (6663): [<ffffffff811369f9>] irq_exit+0xa9/0xc0
[   46.054524] ---[ end trace 33cd45564590d7c8 ]---
[   46.418079] au0828: i2c bus registered
[   46.464037] tveeprom: Hauppauge model 72251, rev D3F0, serial# 4033622430
[   46.464052] tveeprom: MAC address is 00:0d:fe:6c:31:9e
[   46.464061] tveeprom: tuner model is Xceive XC5000 (idx 150, type 76)
[   46.464070] tveeprom: TV standards NTSC(M) ATSC/DVB Digital (eeprom 0x88)
[   46.464077] tveeprom: audio processor is AU8522 (idx 44)
[   46.464085] tveeprom: decoder processor is AU8522 (idx 42)
[   46.464091] tveeprom: has no radio
[   46.464100] au0828: hauppauge_eeprom: hauppauge eeprom: model=72251
[   46.480275] au8522 9-0047: creating new instance
[   46.481073] au8522_decoder creating new instance...
[   46.493942] tuner: 9-0061: Tuner -1 found with type(s) Radio TV.
[   46.498807] xc5000 9-0061: creating new instance
[   46.503833] xc5000: Successfully identified at address 0x61
[   46.503840] xc5000: Firmware has not been loaded previously
[   46.673037] au8522 9-0047: attaching existing instance
[   46.675595] xc5000 9-0061: attaching existing instance
[   46.679924] xc5000: Successfully identified at address 0x61
[   46.679937] xc5000: Firmware has not been loaded previously
[   46.680182] dvbdev: DVB: registering new adapter (au0828)
[   46.680543] usb 3-1: DVB: registering adapter 0 frontend 0 (Auvitek AU8522 QAM/8VSB Frontend)...
[   46.680970] dvbdev: dvb_create_media_entity: media entity 'Auvitek AU8522 QAM/8VSB Frontend' registered.
[   46.686969] dvbdev: dvb_create_media_entity: media entity 'dvb-demux' registered.
[   46.787967] Registered IR keymap rc-hauppauge
[   46.791557] IR RC5(x/sz) protocol handler initialized
[   46.815417] rc rc0: au0828 IR (Hauppauge HVR950Q) as /devices/pci0000:00/0000:00:1c.4/0000:03:00.0/0000:04:02.0/0000:39:00.0/usb3/3-1/rc/rc0
[   46.815731] input: au0828 IR (Hauppauge HVR950Q) as
/devices/pci0000:00/0000:00:1c.4/0000:03:00.0/0000:04:02.0/0000:39:00.0/usb3/3-1/rc/rc0/input18
[   46.821468] au0828: Remote controller au0828 IR (Hauppauge HVR950Q) initialized
[   46.821479] au0828: Registered device AU0828 [Hauppauge HVR950Q]
[   46.822937] usbcore: registered new interface driver au0828
[   46.861371] WARNING: CPU: 1 PID: 1822 at kernel/module.c:262 module_assert_mutex+0x20/0x30
[   46.861490] Modules linked in: snd_usb_audio(+) snd_usbmidi_lib snd_rawmidi ir_rc5_decoder rc_hauppauge au8522_dig xc5000 tuner
au8522_decoder au8522_common au0828 tveeprom dvb_core v4l2_common rfcomm bnep snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic
uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common videodev media btusb btintel bluetooth snd_soc_skl
snd_soc_skl_ipc snd_soc_sst_ipc snd_soc_sst_dsp snd_soc_acpi_intel_match snd_soc_acpi snd_hda_ext_core x86_pkg_temp_thermal snd_soc_core
snd_compress ac97_bus snd_pcm_dmaengine snd_hda_intel iwlmvm snd_hda_codec snd_hda_core snd_hwdep snd_pcm snd_timer snd iwlwifi i915
intel_gtt battery ac pcc_cpufreq thermal
[   46.861570] CPU: 1 PID: 1822 Comm: systemd-udevd Tainted: G        W         5.0.0-rc1-zen #85
[   46.861577] Hardware name: ASUSTeK COMPUTER INC. UX490UA/UX490UA, BIOS UX490UA.312 04/09/2018
[   46.861588] RIP: 0010:module_assert_mutex+0x20/0x30
[   46.861596] Code: 5d c3 e8 f3 68 f5 ff 0f 1f 00 8b 05 d2 5a 7c 01 85 c0 75 01 c3 be ff ff ff ff 48 c7 c7 80 bc 86 82 e8 e4 27 fb ff 85 c0
75 ea <0f> 0b c3 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 53 48 89 fb e8 c7
[   46.861603] RSP: 0018:ffffc90002b87a48 EFLAGS: 00010246
[   46.861611] RAX: 0000000000000000 RBX: ffffffffa059d994 RCX: 0000000000000000
[   46.861618] RDX: ffff888288242640 RSI: ffffffff8286bc80 RDI: ffff888288242f30
[   46.861626] RBP: ffff8882880e6000 R08: ffffffff83999000 R09: 0000000000000000
[   46.861633] R10: ffffc90002b87a60 R11: 0000000000000003 R12: ffffffffa059d994
[   46.861641] R13: ffff8882880e60a0 R14: ffff88829a88ac9c R15: ffff88829ab2ec58
[   46.861651] FS:  00007f0ac7d318c0(0000) GS:ffff8882b6a80000(0000) knlGS:0000000000000000
[   46.861659] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   46.861667] CR2: 00007f0ac740dba0 CR3: 00000002ad550004 CR4: 00000000003606e0
[   46.861675] Call Trace:
[   46.861687]  find_module+0x9/0x20
[   46.861707]  media_device_usb_allocate+0x7c/0x170 [media]
[   46.861741]  snd_media_device_create+0x160/0x280 [snd_usb_audio]
[   46.861754]  ? snd_card_register+0x124/0x1c0 [snd]
[   46.861775]  usb_audio_probe+0x8a4/0xaa0 [snd_usb_audio]
[   46.861793]  usb_probe_interface+0xe8/0x2a0
[   46.861805]  really_probe+0xee/0x2a0
[   46.861815]  driver_probe_device+0x4a/0xb0
[   46.861828]  __driver_attach+0xb3/0xd0
[   46.861840]  ? driver_probe_device+0xb0/0xb0
[   46.861849]  bus_for_each_dev+0x74/0xc0
[   46.861860]  bus_add_driver+0x19a/0x1e0
[   46.861870]  driver_register+0x66/0xb0
[   46.861879]  usb_register_driver+0x9a/0x150
[   46.861886]  ? 0xffffffffa05c3000
[   46.861894]  do_one_initcall+0x61/0x2fb
[   46.861901]  ? do_init_module+0x1d/0x1e0
[   46.861910]  ? rcu_read_lock_sched_held+0x6f/0x80
[   46.861918]  ? kmem_cache_alloc_trace+0x123/0x210
[   46.861927]  do_init_module+0x55/0x1e0
[   46.861935]  load_module+0x145b/0x1710
[   46.861951]  ? vfs_read+0x128/0x150
[   46.861969]  ? __do_sys_finit_module+0xba/0xe0
[   46.861974]  __do_sys_finit_module+0xba/0xe0
[   46.861991]  do_syscall_64+0x4b/0x180
[   46.862001]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[   46.862007] RIP: 0033:0x7f0ac8993a79
[   46.862014] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08
0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d df 43 0c 00 f7 d8 64 89 01 48
[   46.862019] RSP: 002b:00007ffcbd9034f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[   46.862026] RAX: ffffffffffffffda RBX: 0000560d3548f460 RCX: 00007f0ac8993a79
[   46.862031] RDX: 0000000000000000 RSI: 0000560d354a6ab0 RDI: 0000000000000012
[   46.862036] RBP: 0000560d354a6ab0 R08: 0000000000000000 R09: 0000000000000000
[   46.862041] R10: 0000000000000012 R11: 0000000000000246 R12: 0000000000000000
[   46.862045] R13: 0000560d354846d0 R14: 0000000000020000 R15: 0000000000000000
[   46.862061] irq event stamp: 8750
[   46.862068] hardirqs last  enabled at (8749): [<ffffffff81c8340c>] _raw_spin_unlock_irqrestore+0x4c/0x60
[   46.862075] hardirqs last disabled at (8750): [<ffffffff81001c56>] trace_hardirqs_off_thunk+0x1a/0x1c
[   46.862083] softirqs last  enabled at (8722): [<ffffffff820002c0>] __do_softirq+0x2c0/0x4c0
[   46.862091] softirqs last disabled at (8709): [<ffffffff811369f9>] irq_exit+0xa9/0xc0
[   46.862096] ---[ end trace 33cd45564590d7c9 ]---
[   46.866081] usbcore: registered new interface driver snd-usb-audio
[   55.010427] IPv6: ADDRCONF(NETDEV_UP): wlp2s0: link is not ready
[   55.045345] IPv6: ADDRCONF(NETDEV_UP): wlp2s0: link is not ready
[   57.480184] IPv6: ADDRCONF(NETDEV_UP): wlp2s0: link is not ready
[   58.460287] wlp2s0: authenticate with e0:28:6d:86:46:9e
[   58.478641] wlp2s0: send auth to e0:28:6d:86:46:9e (try 1/3)
[   58.487199] wlp2s0: authenticated
[   58.489866] wlp2s0: associate with e0:28:6d:86:46:9e (try 1/3)
[   58.492757] wlp2s0: RX AssocResp from e0:28:6d:86:46:9e (capab=0x1511 status=0 aid=4)
[   58.498171] wlp2s0: associated
[   58.536392] wlp2s0: Limiting TX power to 27 (30 - 3) dBm as advertised by e0:28:6d:86:46:9e
[   58.537657] IPv6: ADDRCONF(NETDEV_CHANGE): wlp2s0: link becomes ready

And the same when I unplug:

[   74.220960] usb 3-1: USB disconnect, device number 2
[   74.323245] xhci_hcd 0000:39:00.0: remove, state 4
[   74.323424] usb usb4: USB disconnect, device number 1
[   74.327940] xhci_hcd 0000:39:00.0: USB bus 4 deregistered
[   74.329918] xhci_hcd 0000:39:00.0: xHCI host controller not responding, assume dead
[   74.329995] xhci_hcd 0000:39:00.0: remove, state 1
[   74.330075] usb usb3: USB disconnect, device number 1
[   74.338585] au8522 9-0047: destroying instance
[   74.341038] xc5000 9-0061: destroying instance
[   74.341142] WARNING: CPU: 2 PID: 46 at kernel/module.c:262 module_assert_mutex+0x20/0x30
[   74.341173] Modules linked in: snd_usb_audio snd_usbmidi_lib snd_rawmidi ir_rc5_decoder rc_hauppauge au8522_dig xc5000 tuner
au8522_decoder au8522_common au0828 tveeprom dvb_core v4l2_common rfcomm bnep snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic
uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common videodev media btusb btintel bluetooth snd_soc_skl
snd_soc_skl_ipc snd_soc_sst_ipc snd_soc_sst_dsp snd_soc_acpi_intel_match snd_soc_acpi snd_hda_ext_core x86_pkg_temp_thermal snd_soc_core
snd_compress ac97_bus snd_pcm_dmaengine snd_hda_intel iwlmvm snd_hda_codec snd_hda_core snd_hwdep snd_pcm snd_timer snd iwlwifi i915
intel_gtt battery ac pcc_cpufreq thermal
[   74.341198] CPU: 2 PID: 46 Comm: kworker/2:1 Tainted: G        W         5.0.0-rc1-zen #85
[   74.341200] Hardware name: ASUSTeK COMPUTER INC. UX490UA/UX490UA, BIOS UX490UA.312 04/09/2018
[   74.341204] Workqueue: usb_hub_wq hub_event
[   74.341207] RIP: 0010:module_assert_mutex+0x20/0x30
[   74.341209] Code: 5d c3 e8 f3 68 f5 ff 0f 1f 00 8b 05 d2 5a 7c 01 85 c0 75 01 c3 be ff ff ff ff 48 c7 c7 80 bc 86 82 e8 e4 27 fb ff 85 c0
75 ea <0f> 0b c3 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 53 48 89 fb e8 c7
[   74.341211] RSP: 0018:ffffc90000e3fb18 EFLAGS: 00010246
[   74.341213] RAX: 0000000000000000 RBX: ffffffffa0573abf RCX: 0000000000000000
[   74.341215] RDX: ffff8882b5684c80 RSI: ffffffff8286bc80 RDI: ffff8882b56855c0
[   74.341217] RBP: ffffffffa0577800 R08: ffffffff839a7d00 R09: ffffffff82b53590
[   74.341218] R10: ffffc90000e3fb30 R11: 0000000000000013 R12: ffffffffa0573abf
[   74.341220] R13: ffff8882b1f5d140 R14: ffff8882880e60a0 R15: 00000000ffffffed
[   74.341223] FS:  0000000000000000(0000) GS:ffff8882b6b00000(0000) knlGS:0000000000000000
[   74.341224] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   74.341226] CR2: 000056317f04cfe0 CR3: 000000029e7cf004 CR4: 00000000003606e0
[   74.341228] Call Trace:
[   74.341231]  find_module+0x9/0x20
[   74.341236]  media_device_delete+0x27/0x68 [media]
[   74.341241]  au0828_usb_release+0xdf/0x100 [au0828]
[   74.341248]  v4l2_device_put+0x24/0x30 [videodev]
[   74.341252]  au0828_analog_unregister+0x68/0x70 [au0828]
[   74.341255]  au0828_usb_disconnect+0x61/0x80 [au0828]
[   74.341259]  usb_unbind_interface+0x77/0x280
[   74.341263]  ? _raw_spin_unlock_irqrestore+0x39/0x60
[   74.341267]  device_release_driver_internal+0x186/0x240
[   74.341271]  bus_remove_device+0xee/0x130
[   74.341274]  device_del+0x140/0x350
[   74.341278]  usb_disable_device+0x7c/0x1d0
[   74.341281]  usb_disconnect+0xb4/0x280
[   74.341284]  hub_port_connect+0x7a/0xa10
[   74.341289]  port_event+0x4c8/0x6a0
[   74.341294]  hub_event+0x119/0x2c0
[   74.341299]  process_one_work+0x283/0x690
[   74.341304]  worker_thread+0x34/0x3d0
[   74.341308]  ? rescuer_thread+0x360/0x360
[   74.341311]  kthread+0x118/0x130
[   74.341313]  ? kthread_create_on_node+0x60/0x60
[   74.341316]  ret_from_fork+0x3a/0x50
[   74.341322] irq event stamp: 94118
[   74.341325] hardirqs last  enabled at (94117): [<ffffffff81c8340c>] _raw_spin_unlock_irqrestore+0x4c/0x60
[   74.341328] hardirqs last disabled at (94118): [<ffffffff81001c56>] trace_hardirqs_off_thunk+0x1a/0x1c
[   74.341331] softirqs last  enabled at (94092): [<ffffffff819f628c>] peernet2id+0x4c/0x70
[   74.341334] softirqs last disabled at (94090): [<ffffffff819f626d>] peernet2id+0x2d/0x70
[   74.341335] ---[ end trace 33cd45564590d7ca ]---
[   74.350561] WARNING: CPU: 2 PID: 46 at kernel/module.c:262 module_assert_mutex+0x20/0x30
[   74.350610] Modules linked in: snd_usb_audio snd_usbmidi_lib snd_rawmidi ir_rc5_decoder rc_hauppauge au8522_dig xc5000 tuner
au8522_decoder au8522_common au0828 tveeprom dvb_core v4l2_common rfcomm bnep snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic
uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common videodev media btusb btintel bluetooth snd_soc_skl
snd_soc_skl_ipc snd_soc_sst_ipc snd_soc_sst_dsp snd_soc_acpi_intel_match snd_soc_acpi snd_hda_ext_core x86_pkg_temp_thermal snd_soc_core
snd_compress ac97_bus snd_pcm_dmaengine snd_hda_intel iwlmvm snd_hda_codec snd_hda_core snd_hwdep snd_pcm snd_timer snd iwlwifi i915
intel_gtt battery ac pcc_cpufreq thermal
[   74.350644] CPU: 2 PID: 46 Comm: kworker/2:1 Tainted: G        W         5.0.0-rc1-zen #85
[   74.350647] Hardware name: ASUSTeK COMPUTER INC. UX490UA/UX490UA, BIOS UX490UA.312 04/09/2018
[   74.350652] Workqueue: usb_hub_wq hub_event
[   74.350657] RIP: 0010:module_assert_mutex+0x20/0x30
[   74.350661] Code: 5d c3 e8 f3 68 f5 ff 0f 1f 00 8b 05 d2 5a 7c 01 85 c0 75 01 c3 be ff ff ff ff 48 c7 c7 80 bc 86 82 e8 e4 27 fb ff 85 c0
75 ea <0f> 0b c3 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 53 48 89 fb e8 c7
[   74.350664] RSP: 0018:ffffc90000e3fae0 EFLAGS: 00010246
[   74.350668] RAX: 0000000000000000 RBX: ffffffffa059d994 RCX: 0000000000000000
[   74.350671] RDX: ffff8882b5684c80 RSI: ffffffff8286bc80 RDI: ffff8882b56855e8
[   74.350674] RBP: ffffffffa0577800 R08: 0000000000012a81 R09: ffffffff82554cd8
[   74.350677] R10: ffffc90000e3faf8 R11: 0000000000000006 R12: ffffffffa059d994
[   74.350680] R13: ffff8882b1f5d000 R14: ffff8882b1f5d000 R15: ffff88829ab2eda8
[   74.350683] FS:  0000000000000000(0000) GS:ffff8882b6b00000(0000) knlGS:0000000000000000
[   74.350686] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   74.350689] CR2: 0000560d354e6920 CR3: 00000002b31ef003 CR4: 00000000003606e0
[   74.350691] Call Trace:
[   74.350696]  find_module+0x9/0x20
[   74.350704]  media_device_delete+0x27/0x68 [media]
[   74.350714]  snd_media_device_delete+0x112/0x131 [snd_usb_audio]
[   74.350724]  usb_audio_disconnect+0x1bc/0x240 [snd_usb_audio]
[   74.350730]  ? __pm_runtime_resume+0x4f/0x80
[   74.350737]  usb_unbind_interface+0x77/0x280
[   74.350742]  ? _raw_spin_unlock_irqrestore+0x39/0x60
[   74.350749]  device_release_driver_internal+0x186/0x240
[   74.350754]  bus_remove_device+0xee/0x130
[   74.350758]  device_del+0x140/0x350
[   74.350765]  usb_disable_device+0x7c/0x1d0
[   74.350770]  usb_disconnect+0xb4/0x280
[   74.350775]  hub_port_connect+0x7a/0xa10
[   74.350783]  port_event+0x4c8/0x6a0
[   74.350791]  hub_event+0x119/0x2c0
[   74.350800]  process_one_work+0x283/0x690
[   74.350808]  worker_thread+0x34/0x3d0
[   74.350814]  ? rescuer_thread+0x360/0x360
[   74.350817]  kthread+0x118/0x130
[   74.350821]  ? kthread_create_on_node+0x60/0x60
[   74.350826]  ret_from_fork+0x3a/0x50
[   74.350835] irq event stamp: 94728
[   74.350840] hardirqs last  enabled at (94727): [<ffffffff812ac725>] kfree+0xc5/0x270
[   74.350845] hardirqs last disabled at (94728): [<ffffffff81001c56>] trace_hardirqs_off_thunk+0x1a/0x1c
[   74.350849] softirqs last  enabled at (94632): [<ffffffff819f628c>] peernet2id+0x4c/0x70
[   74.350853] softirqs last disabled at (94630): [<ffffffff819f626d>] peernet2id+0x2d/0x70
[   74.350856] ---[ end trace 33cd45564590d7cb ]---
[   74.360386] xhci_hcd 0000:39:00.0: Host halt failed, -19
[   74.360445] xhci_hcd 0000:39:00.0: Host not accessible, reset failed.
[   74.360976] xhci_hcd 0000:39:00.0: USB bus 3 deregistered

My .config is available upon request.

Regards,

	Hans

Re: [PATCH v9 0/4] Media Device Allocator API

[shuah]

On 1/21/19 7:46 AM, Hans Verkuil wrote:
> Hi Shuah,
> 
> On 12/18/2018 06:59 PM, shuah@kernel.org wrote:
>> From: Shuah Khan <shuah@kernel.org>
>>
>> Media Device Allocator API to allows multiple drivers share a media device.
>> This API solves a very common use-case for media devices where one physical
>> device (an USB stick) provides both audio and video. When such media device
>> exposes a standard USB Audio class, a proprietary Video class, two or more
>> independent drivers will share a single physical USB bridge. In such cases,
>> it is necessary to coordinate access to the shared resource.
>>
>> Using this API, drivers can allocate a media device with the shared struct
>> device as the key. Once the media device is allocated by a driver, other
>> drivers can get a reference to it. The media device is released when all
>> the references are released.
>>
>> - Tested sharing resources with kaffeine, vlc, xawtv, tvtime, and
>>    arecord. When analog is streaming, digital and audio user-space
>>    applications detect that the tuner is busy and exit. When digital
>>    is streaming, analog and audio applications detect that the tuner is
>>    busy and exit. When arecord is owns the tuner, digital and analog
>>    detect that the tuner is busy and exit.
>> - Tested media device allocator API with bind/unbind testing on
>>    snd-usb-audio and au0828 drivers to make sure /dev/mediaX is released
>>    only when the last driver is unbound.
>> - This patch series is tested on 4.20-rc6
>> - Addressed review comments from Hans on the RFC v8 (rebased on 4.19)
>> - Updated change log to describe the use-case more clearly.
>> - No changes to 0001,0002 code since the v7 referenced below.
>> - 0003 is a new patch to enable ALSA defines that have been
>>    disabled for kernel between 4.9 and 4.19.
>> - Minor merge conflict resolution in 0004.
>> - Added SPDX to new files.
>>
>> References:
>> https://lkml.org/lkml/2018/11/2/169
>> https://www.mail-archive.com/linux-media@vger.kernel.org/msg105854.html
> 
> When I connect my au0828 to my laptop with your v9 patch series applied I get
> these warnings:
> 
> [   45.416047] xhci_hcd 0000:39:00.0: xHCI Host Controller
> [   45.417882] xhci_hcd 0000:39:00.0: new USB bus registered, assigned bus number 3
> [   45.419292] xhci_hcd 0000:39:00.0: hcc params 0x200077c1 hci version 0x110 quirks 0x0000000200009810
> [   45.420835] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.00
> [   45.420893] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> [   45.420899] usb usb3: Product: xHCI Host Controller
> [   45.420905] usb usb3: Manufacturer: Linux 5.0.0-rc1-zen xhci-hcd
> [   45.420911] usb usb3: SerialNumber: 0000:39:00.0
> [   45.424290] hub 3-0:1.0: USB hub found
> [   45.425274] hub 3-0:1.0: 2 ports detected
> [   45.431061] xhci_hcd 0000:39:00.0: xHCI Host Controller
> [   45.432436] xhci_hcd 0000:39:00.0: new USB bus registered, assigned bus number 4
> [   45.432448] xhci_hcd 0000:39:00.0: Host supports USB 3.1 Enhanced SuperSpeed
> [   45.433299] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.00
> [   45.433354] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> [   45.433360] usb usb4: Product: xHCI Host Controller
> [   45.433365] usb usb4: Manufacturer: Linux 5.0.0-rc1-zen xhci-hcd
> [   45.433370] usb usb4: SerialNumber: 0000:39:00.0
> [   45.436134] hub 4-0:1.0: USB hub found
> [   45.436576] hub 4-0:1.0: 2 ports detected
> [   45.750940] usb 3-1: new high-speed USB device number 2 using xhci_hcd
> [   45.899927] usb 3-1: New USB device found, idVendor=2040, idProduct=721e, bcdDevice= 0.05
> [   45.899949] usb 3-1: New USB device strings: Mfr=1, Product=2, SerialNumber=10
> [   45.899960] usb 3-1: Product: WinTV Aero-A
> [   45.899970] usb 3-1: Manufacturer: Hauppauge
> [   45.899979] usb 3-1: SerialNumber: 4033622430
> [   46.053476] au0828: au0828 driver loaded
> [   46.053726] WARNING: CPU: 1 PID: 1824 at kernel/module.c:262 module_assert_mutex+0x20/0x30
> [   46.053818] Modules linked in: au0828(+) tveeprom dvb_core v4l2_common rfcomm bnep snd_hda_codec_hdmi snd_hda_codec_realtek
> snd_hda_codec_generic uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common videodev media btusb btintel bluetooth
> snd_soc_skl snd_soc_skl_ipc snd_soc_sst_ipc snd_soc_sst_dsp snd_soc_acpi_intel_match snd_soc_acpi snd_hda_ext_core x86_pkg_temp_thermal
> snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine snd_hda_intel iwlmvm snd_hda_codec snd_hda_core snd_hwdep snd_pcm snd_timer snd iwlwifi
> i915 intel_gtt battery ac pcc_cpufreq thermal
> [   46.053853] CPU: 1 PID: 1824 Comm: systemd-udevd Not tainted 5.0.0-rc1-zen #85
> [   46.053856] Hardware name: ASUSTeK COMPUTER INC. UX490UA/UX490UA, BIOS UX490UA.312 04/09/2018
> [   46.053862] RIP: 0010:module_assert_mutex+0x20/0x30
> [   46.053867] Code: 5d c3 e8 f3 68 f5 ff 0f 1f 00 8b 05 d2 5a 7c 01 85 c0 75 01 c3 be ff ff ff ff 48 c7 c7 80 bc 86 82 e8 e4 27 fb ff 85 c0
> 75 ea <0f> 0b c3 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 53 48 89 fb e8 c7
> [   46.053871] RSP: 0018:ffffc90002aa3ac8 EFLAGS: 00010246
> [   46.053876] RAX: 0000000000000000 RBX: ffffffffa0573abf RCX: 0000000000000000
> [   46.053880] RDX: 0000000000000000 RSI: ffffffff8286bc80 RDI: ffff8882a517d570
> [   46.053883] RBP: ffff8882880e6000 R08: 0000000000000000 R09: ffff8882b1f5d000
> [   46.053886] R10: 0000000000000001 R11: 0000000000000003 R12: ffffffffa0573abf
> [   46.053890] R13: ffff8882880e60a0 R14: ffff8882880e6000 R15: ffffffffa0577200
> [   46.053894] FS:  00007f0ac7d318c0(0000) GS:ffff8882b6a80000(0000) knlGS:0000000000000000
> [   46.053898] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   46.053902] CR2: 0000560d354ed800 CR3: 00000002880f9003 CR4: 00000000003606e0
> [   46.053905] Call Trace:
> [   46.053910]  find_module+0x9/0x20
> 
> My .config is available upon request.
> 

Thanks for the config.

Thanks for finding this. find_module() should be called with
module_mutex since 4.15. The media allocator patch was written
before that.

I didn't realize I don't have CONFIG_DEBUG_MUTEXES=y when I switched
to a new system.

I reproduced the problem and re-worked patch 1 and patch 4 in this
series. I will resend the v10 for just those two patches tomorrow.

Please let me know if you want me to send the entire series.

thanks,
-- Shuah

Re: [PATCH v9 0/4] Media Device Allocator API

[Hans Verkuil]

On 1/24/19 2:35 AM, shuah wrote:
> On 1/21/19 7:46 AM, Hans Verkuil wrote:
>> Hi Shuah,
>>
>> On 12/18/2018 06:59 PM, shuah@kernel.org wrote:
>>> From: Shuah Khan <shuah@kernel.org>
>>>
>>> Media Device Allocator API to allows multiple drivers share a media device.
>>> This API solves a very common use-case for media devices where one physical
>>> device (an USB stick) provides both audio and video. When such media device
>>> exposes a standard USB Audio class, a proprietary Video class, two or more
>>> independent drivers will share a single physical USB bridge. In such cases,
>>> it is necessary to coordinate access to the shared resource.
>>>
>>> Using this API, drivers can allocate a media device with the shared struct
>>> device as the key. Once the media device is allocated by a driver, other
>>> drivers can get a reference to it. The media device is released when all
>>> the references are released.
>>>
>>> - Tested sharing resources with kaffeine, vlc, xawtv, tvtime, and
>>>    arecord. When analog is streaming, digital and audio user-space
>>>    applications detect that the tuner is busy and exit. When digital
>>>    is streaming, analog and audio applications detect that the tuner is
>>>    busy and exit. When arecord is owns the tuner, digital and analog
>>>    detect that the tuner is busy and exit.
>>> - Tested media device allocator API with bind/unbind testing on
>>>    snd-usb-audio and au0828 drivers to make sure /dev/mediaX is released
>>>    only when the last driver is unbound.
>>> - This patch series is tested on 4.20-rc6
>>> - Addressed review comments from Hans on the RFC v8 (rebased on 4.19)
>>> - Updated change log to describe the use-case more clearly.
>>> - No changes to 0001,0002 code since the v7 referenced below.
>>> - 0003 is a new patch to enable ALSA defines that have been
>>>    disabled for kernel between 4.9 and 4.19.
>>> - Minor merge conflict resolution in 0004.
>>> - Added SPDX to new files.
>>>
>>> References:
>>> https://lkml.org/lkml/2018/11/2/169
>>> https://www.mail-archive.com/linux-media@vger.kernel.org/msg105854.html
>>
>> When I connect my au0828 to my laptop with your v9 patch series applied I get
>> these warnings:
>>
>> [   45.416047] xhci_hcd 0000:39:00.0: xHCI Host Controller
>> [   45.417882] xhci_hcd 0000:39:00.0: new USB bus registered, assigned bus number 3
>> [   45.419292] xhci_hcd 0000:39:00.0: hcc params 0x200077c1 hci version 0x110 quirks 0x0000000200009810
>> [   45.420835] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.00
>> [   45.420893] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
>> [   45.420899] usb usb3: Product: xHCI Host Controller
>> [   45.420905] usb usb3: Manufacturer: Linux 5.0.0-rc1-zen xhci-hcd
>> [   45.420911] usb usb3: SerialNumber: 0000:39:00.0
>> [   45.424290] hub 3-0:1.0: USB hub found
>> [   45.425274] hub 3-0:1.0: 2 ports detected
>> [   45.431061] xhci_hcd 0000:39:00.0: xHCI Host Controller
>> [   45.432436] xhci_hcd 0000:39:00.0: new USB bus registered, assigned bus number 4
>> [   45.432448] xhci_hcd 0000:39:00.0: Host supports USB 3.1 Enhanced SuperSpeed
>> [   45.433299] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.00
>> [   45.433354] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
>> [   45.433360] usb usb4: Product: xHCI Host Controller
>> [   45.433365] usb usb4: Manufacturer: Linux 5.0.0-rc1-zen xhci-hcd
>> [   45.433370] usb usb4: SerialNumber: 0000:39:00.0
>> [   45.436134] hub 4-0:1.0: USB hub found
>> [   45.436576] hub 4-0:1.0: 2 ports detected
>> [   45.750940] usb 3-1: new high-speed USB device number 2 using xhci_hcd
>> [   45.899927] usb 3-1: New USB device found, idVendor=2040, idProduct=721e, bcdDevice= 0.05
>> [   45.899949] usb 3-1: New USB device strings: Mfr=1, Product=2, SerialNumber=10
>> [   45.899960] usb 3-1: Product: WinTV Aero-A
>> [   45.899970] usb 3-1: Manufacturer: Hauppauge
>> [   45.899979] usb 3-1: SerialNumber: 4033622430
>> [   46.053476] au0828: au0828 driver loaded
>> [   46.053726] WARNING: CPU: 1 PID: 1824 at kernel/module.c:262 module_assert_mutex+0x20/0x30
>> [   46.053818] Modules linked in: au0828(+) tveeprom dvb_core v4l2_common rfcomm bnep snd_hda_codec_hdmi snd_hda_codec_realtek
>> snd_hda_codec_generic uvcvideo videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common videodev media btusb btintel bluetooth
>> snd_soc_skl snd_soc_skl_ipc snd_soc_sst_ipc snd_soc_sst_dsp snd_soc_acpi_intel_match snd_soc_acpi snd_hda_ext_core x86_pkg_temp_thermal
>> snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine snd_hda_intel iwlmvm snd_hda_codec snd_hda_core snd_hwdep snd_pcm snd_timer snd iwlwifi
>> i915 intel_gtt battery ac pcc_cpufreq thermal
>> [   46.053853] CPU: 1 PID: 1824 Comm: systemd-udevd Not tainted 5.0.0-rc1-zen #85
>> [   46.053856] Hardware name: ASUSTeK COMPUTER INC. UX490UA/UX490UA, BIOS UX490UA.312 04/09/2018
>> [   46.053862] RIP: 0010:module_assert_mutex+0x20/0x30
>> [   46.053867] Code: 5d c3 e8 f3 68 f5 ff 0f 1f 00 8b 05 d2 5a 7c 01 85 c0 75 01 c3 be ff ff ff ff 48 c7 c7 80 bc 86 82 e8 e4 27 fb ff 85 c0
>> 75 ea <0f> 0b c3 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 53 48 89 fb e8 c7
>> [   46.053871] RSP: 0018:ffffc90002aa3ac8 EFLAGS: 00010246
>> [   46.053876] RAX: 0000000000000000 RBX: ffffffffa0573abf RCX: 0000000000000000
>> [   46.053880] RDX: 0000000000000000 RSI: ffffffff8286bc80 RDI: ffff8882a517d570
>> [   46.053883] RBP: ffff8882880e6000 R08: 0000000000000000 R09: ffff8882b1f5d000
>> [   46.053886] R10: 0000000000000001 R11: 0000000000000003 R12: ffffffffa0573abf
>> [   46.053890] R13: ffff8882880e60a0 R14: ffff8882880e6000 R15: ffffffffa0577200
>> [   46.053894] FS:  00007f0ac7d318c0(0000) GS:ffff8882b6a80000(0000) knlGS:0000000000000000
>> [   46.053898] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [   46.053902] CR2: 0000560d354ed800 CR3: 00000002880f9003 CR4: 00000000003606e0
>> [   46.053905] Call Trace:
>> [   46.053910]  find_module+0x9/0x20
>>
>> My .config is available upon request.
>>
> 
> Thanks for the config.
> 
> Thanks for finding this. find_module() should be called with
> module_mutex since 4.15. The media allocator patch was written
> before that.
> 
> I didn't realize I don't have CONFIG_DEBUG_MUTEXES=y when I switched
> to a new system.
> 
> I reproduced the problem and re-worked patch 1 and patch 4 in this
> series. I will resend the v10 for just those two patches tomorrow.
> 
> Please let me know if you want me to send the entire series.

Please send the entire series. Less chance of mistakes that way.

Thanks!

	Hans