[RFC PATCH v8 0/4] Media Device Allocator API

[RFC PATCH v8 0/4] Media Device Allocator API

[shuah]

From: Shuah Khan <shuah@kernel.org>

This patch series has been on the ice for the last couple of years.
Mauro asked me to restart the discussion on this and see if we can
make progress.

I rebased the series to Linux 4.19. It has been surprisingly well
preserved with one small merge conflict resolved in 0004.

Media Device Allocator API to allows multiple drivers share a media device.
Using this API, drivers can allocate a media device with the shared struct
device as the key. Once the media device is allocated by a driver, other
drivers can get a reference to it. The media device is released when all
the references are released.

- No changes to 0001,0002 code since the v7 referenced below.
- 0003 is a new patch to enable ALSA defines that have been
  disabled for kernel between 4.9 and 4.19.
- Minor merge conflict resolution in 0004.
- Added SPDX to new files.

References:
https://www.mail-archive.com/linux-media@vger.kernel.org/msg105854.html

Please review. I am sending this as RFC even though it has been tested
several times prior to this non-event rebased v8. I ran sanity tests.

Shuah Khan (4):
  media: Media Device Allocator API
  media: change au0828 to use Media Device Allocator API
  media: media.h: Enable ALSA MEDIA_INTF_T* interface types
  sound/usb: Use Media Controller API to share media resources

 Documentation/media/kapi/mc-core.rst   |  37 +++
 drivers/media/Makefile                 |   3 +-
 drivers/media/media-dev-allocator.c    | 132 ++++++++++
 drivers/media/usb/au0828/au0828-core.c |  12 +-
 drivers/media/usb/au0828/au0828.h      |   1 +
 include/media/media-dev-allocator.h    |  53 ++++
 include/uapi/linux/media.h             |  25 +-
 sound/usb/Kconfig                      |   4 +
 sound/usb/Makefile                     |   2 +
 sound/usb/card.c                       |  14 ++
 sound/usb/card.h                       |   3 +
 sound/usb/media.c                      | 320 +++++++++++++++++++++++++
 sound/usb/media.h                      |  73 ++++++
 sound/usb/mixer.h                      |   3 +
 sound/usb/pcm.c                        |  29 ++-
 sound/usb/quirks-table.h               |   1 +
 sound/usb/stream.c                     |   2 +
 sound/usb/usbaudio.h                   |   6 +
 18 files changed, 692 insertions(+), 28 deletions(-)
 create mode 100644 drivers/media/media-dev-allocator.c
 create mode 100644 include/media/media-dev-allocator.h
 create mode 100644 sound/usb/media.c
 create mode 100644 sound/usb/media.h

-- 
2.17.0

[RFC PATCH v8 1/4] media: Media Device Allocator API

[shuah]

From: Shuah Khan <shuah@kernel.org>

Media Device Allocator API to allows multiple drivers share a media device.
Using this API, drivers can allocate a media device with the shared struct
device as the key. Once the media device is allocated by a driver, other
drivers can get a reference to it. The media device is released when all
the references are released.

Signed-off-by: Shuah Khan <shuah@kernel.org>
---
 Documentation/media/kapi/mc-core.rst |  37 ++++++++
 drivers/media/Makefile               |   3 +-
 drivers/media/media-dev-allocator.c  | 132 +++++++++++++++++++++++++++
 include/media/media-dev-allocator.h  |  53 +++++++++++
 4 files changed, 224 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/media-dev-allocator.c
 create mode 100644 include/media/media-dev-allocator.h

diff --git a/Documentation/media/kapi/mc-core.rst b/Documentation/media/kapi/mc-core.rst
index 0c05503eaf1f..d6f409598065 100644
--- a/Documentation/media/kapi/mc-core.rst
+++ b/Documentation/media/kapi/mc-core.rst
@@ -257,8 +257,45 @@ Subsystems should facilitate link validation by providing subsystem specific
 helper functions to provide easy access for commonly needed information, and
 in the end provide a way to use driver-specific callbacks.
 
+Media Controller Device Allocator API
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When media device belongs to more than one driver, the shared media device
+is allocated with the shared struct device as the key for look ups.
+
+Shared media device should stay in registered state until the last driver
+unregisters it. In addition, media device should be released when all the
+references are released. Each driver gets a reference to the media device
+during probe, when it allocates the media device. If media device is already
+allocated, allocate API bumps up the refcount and return the existing media
+device. Driver puts the reference back from its disconnect routine when it
+calls :c:func:`media_device_delete()`.
+
+Media device is unregistered and cleaned up from the kref put handler to
+ensure that the media device stays in registered state until the last driver
+unregisters the media device.
+
+**Driver Usage**
+
+Drivers should use the media-core routines to get register reference and
+call :c:func:`media_device_delete()` routine to make sure the shared media
+device delete is handled correctly.
+
+**driver probe:**
+Call :c:func:`media_device_usb_allocate()` to allocate or get a reference
+Call :c:func:`media_device_register()`, if media devnode isn't registered
+
+**driver disconnect:**
+Call :c:func:`media_device_delete()` to free the media_device. Free'ing is
+handled by the kref put handler.
+
+API Definitions
+^^^^^^^^^^^^^^^
+
 .. kernel-doc:: include/media/media-device.h
 
 .. kernel-doc:: include/media/media-devnode.h
 
 .. kernel-doc:: include/media/media-entity.h
+
+.. kernel-doc:: include/media/media-dev-allocator.h
diff --git a/drivers/media/Makefile b/drivers/media/Makefile
index 594b462ddf0e..8608f0a41dca 100644
--- a/drivers/media/Makefile
+++ b/drivers/media/Makefile
@@ -3,7 +3,8 @@
 # Makefile for the kernel multimedia device drivers.
 #
 
-media-objs	:= media-device.o media-devnode.o media-entity.o
+media-objs	:= media-device.o media-devnode.o media-entity.o \
+		   media-dev-allocator.o
 
 #
 # I2C drivers should come before other drivers, otherwise they'll fail
diff --git a/drivers/media/media-dev-allocator.c b/drivers/media/media-dev-allocator.c
new file mode 100644
index 000000000000..262d1293dc13
--- /dev/null
+++ b/drivers/media/media-dev-allocator.c
@@ -0,0 +1,132 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * media-dev-allocator.c - Media Controller Device Allocator API
+ *
+ * Copyright (c) 2018 Shuah Khan <shuah@kernel.org>
+ *
+ * Credits: Suggested by Laurent Pinchart <laurent.pinchart@ideasonboard.com>
+ */
+
+/*
+ * This file adds a global refcounted Media Controller Device Instance API.
+ * A system wide global media device list is managed and each media device
+ * includes a kref count. The last put on the media device releases the media
+ * device instance.
+ *
+ */
+
+#include <linux/kref.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/usb.h>
+
+#include <media/media-device.h>
+
+static LIST_HEAD(media_device_list);
+static DEFINE_MUTEX(media_device_lock);
+
+struct media_device_instance {
+	struct media_device mdev;
+	struct module *owner;
+	struct list_head list;
+	struct kref refcount;
+};
+
+static inline struct media_device_instance *
+to_media_device_instance(struct media_device *mdev)
+{
+	return container_of(mdev, struct media_device_instance, mdev);
+}
+
+static void media_device_instance_release(struct kref *kref)
+{
+	struct media_device_instance *mdi =
+		container_of(kref, struct media_device_instance, refcount);
+
+	dev_dbg(mdi->mdev.dev, "%s: mdev=%p\n", __func__, &mdi->mdev);
+
+	mutex_lock(&media_device_lock);
+
+	media_device_unregister(&mdi->mdev);
+	media_device_cleanup(&mdi->mdev);
+
+	list_del(&mdi->list);
+	mutex_unlock(&media_device_lock);
+
+	kfree(mdi);
+}
+
+/* Callers should hold media_device_lock when calling this function */
+static struct media_device *__media_device_get(struct device *dev,
+					       char *module_name)
+{
+	struct media_device_instance *mdi;
+
+	list_for_each_entry(mdi, &media_device_list, list) {
+		if (mdi->mdev.dev != dev)
+			continue;
+
+		kref_get(&mdi->refcount);
+		/* get module reference for the media_device owner */
+		if (find_module(module_name) != mdi->owner &&
+		    !try_module_get(mdi->owner))
+			dev_err(dev, "%s: try_module_get() error\n", __func__);
+		dev_dbg(dev, "%s: get mdev=%p module_name %s\n",
+			__func__, &mdi->mdev, module_name);
+		return &mdi->mdev;
+	}
+
+	mdi = kzalloc(sizeof(*mdi), GFP_KERNEL);
+	if (!mdi)
+		return NULL;
+
+	mdi->owner = find_module(module_name);
+	kref_init(&mdi->refcount);
+	list_add_tail(&mdi->list, &media_device_list);
+
+	dev_dbg(dev, "%s: alloc mdev=%p module_name %s\n", __func__,
+		&mdi->mdev, module_name);
+	return &mdi->mdev;
+}
+
+#if IS_ENABLED(CONFIG_USB)
+struct media_device *media_device_usb_allocate(struct usb_device *udev,
+					       char *module_name)
+{
+	struct media_device *mdev;
+
+	mutex_lock(&media_device_lock);
+	mdev = __media_device_get(&udev->dev, module_name);
+	if (!mdev) {
+		mutex_unlock(&media_device_lock);
+		return ERR_PTR(-ENOMEM);
+	}
+
+	/* check if media device is already initialized */
+	if (!mdev->dev)
+		__media_device_usb_init(mdev, udev, udev->product,
+					module_name);
+	mutex_unlock(&media_device_lock);
+	return mdev;
+}
+EXPORT_SYMBOL_GPL(media_device_usb_allocate);
+#endif
+
+void media_device_delete(struct media_device *mdev, char *module_name)
+{
+	struct media_device_instance *mdi = to_media_device_instance(mdev);
+
+	dev_dbg(mdi->mdev.dev, "%s: mdev=%p module_name %s\n",
+		__func__, &mdi->mdev, module_name);
+
+	mutex_lock(&media_device_lock);
+	/* put module reference if media_device owner is not THIS_MODULE */
+	if (mdi->owner != find_module(module_name)) {
+		module_put(mdi->owner);
+		dev_dbg(mdi->mdev.dev,
+			"%s decremented owner module reference\n", __func__);
+	}
+	mutex_unlock(&media_device_lock);
+	kref_put(&mdi->refcount, media_device_instance_release);
+}
+EXPORT_SYMBOL_GPL(media_device_delete);
diff --git a/include/media/media-dev-allocator.h b/include/media/media-dev-allocator.h
new file mode 100644
index 000000000000..8fbadff2cef8
--- /dev/null
+++ b/include/media/media-dev-allocator.h
@@ -0,0 +1,53 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * media-dev-allocator.h - Media Controller Device Allocator API
+ *
+ * Copyright (c) 2018 Shuah Khan <shuah@kernel.org>
+ *
+ * Credits: Suggested by Laurent Pinchart <laurent.pinchart@ideasonboard.com>
+ */
+
+/*
+ * This file adds a global ref-counted Media Controller Device Instance API.
+ * A system wide global media device list is managed and each media device
+ * includes a kref count. The last put on the media device releases the media
+ * device instance.
+ */
+
+#ifndef _MEDIA_DEV_ALLOCTOR_H
+#define _MEDIA_DEV_ALLOCTOR_H
+
+struct usb_device;
+
+#ifdef CONFIG_MEDIA_CONTROLLER
+/**
+ * media_device_usb_allocate() - Allocate and return struct &media device
+ *
+ * @udev:		struct &usb_device pointer
+ * @module_name:	should be filled with %KBUILD_MODNAME
+ *
+ * This interface should be called to allocate a Media Device when multiple
+ * drivers share usb_device and the media device. This interface allocates
+ * &media_device structure and calls media_device_usb_init() to initialize
+ * it.
+ *
+ */
+struct media_device *media_device_usb_allocate(struct usb_device *udev,
+					       char *module_name);
+/**
+ * media_device_delete() - Release media device. Calls kref_put().
+ *
+ * @mdev:		struct &media_device pointer
+ * @module_name:	should be filled with %KBUILD_MODNAME
+ *
+ * This interface should be called to put Media Device Instance kref.
+ */
+void media_device_delete(struct media_device *mdev, char *module_name);
+#else
+static inline struct media_device *media_device_usb_allocate(
+			struct usb_device *udev, char *module_name)
+			{ return NULL; }
+static inline void media_device_delete(
+			struct media_device *mdev, char *module_name) { }
+#endif /* CONFIG_MEDIA_CONTROLLER */
+#endif
-- 
2.17.0

[RFC PATCH v8 2/4] media: change au0828 to use Media Device Allocator API

[shuah]

From: Shuah Khan <shuah@kernel.org>

Change au0828 to use Media Device Allocator API to allocate media device
with the parent usb struct device as the key, so it can be shared with the
snd_usb_audio driver.

Signed-off-by: Shuah Khan <shuah@kernel.org>
---
 drivers/media/usb/au0828/au0828-core.c | 12 ++++--------
 drivers/media/usb/au0828/au0828.h      |  1 +
 2 files changed, 5 insertions(+), 8 deletions(-)

diff --git a/drivers/media/usb/au0828/au0828-core.c b/drivers/media/usb/au0828/au0828-core.c
index cd363a2100d4..837409d190a1 100644
--- a/drivers/media/usb/au0828/au0828-core.c
+++ b/drivers/media/usb/au0828/au0828-core.c
@@ -155,9 +155,7 @@ static void au0828_unregister_media_device(struct au0828_dev *dev)
 	dev->media_dev->disable_source = NULL;
 	mutex_unlock(&mdev->graph_mutex);
 
-	media_device_unregister(dev->media_dev);
-	media_device_cleanup(dev->media_dev);
-	kfree(dev->media_dev);
+	media_device_delete(dev->media_dev, KBUILD_MODNAME);
 	dev->media_dev = NULL;
 #endif
 }
@@ -210,14 +208,10 @@ static int au0828_media_device_init(struct au0828_dev *dev,
 #ifdef CONFIG_MEDIA_CONTROLLER
 	struct media_device *mdev;
 
-	mdev = kzalloc(sizeof(*mdev), GFP_KERNEL);
+	mdev = media_device_usb_allocate(udev, KBUILD_MODNAME);
 	if (!mdev)
 		return -ENOMEM;
 
-	/* check if media device is already initialized */
-	if (!mdev->dev)
-		media_device_usb_init(mdev, udev, udev->product);
-
 	dev->media_dev = mdev;
 #endif
 	return 0;
@@ -478,6 +472,8 @@ static int au0828_media_device_register(struct au0828_dev *dev,
 		/* register media device */
 		ret = media_device_register(dev->media_dev);
 		if (ret) {
+			media_device_delete(dev->media_dev, KBUILD_MODNAME);
+			dev->media_dev = NULL;
 			dev_err(&udev->dev,
 				"Media Device Register Error: %d\n", ret);
 			return ret;
diff --git a/drivers/media/usb/au0828/au0828.h b/drivers/media/usb/au0828/au0828.h
index 004eadef55c7..7dbe3db15ebe 100644
--- a/drivers/media/usb/au0828/au0828.h
+++ b/drivers/media/usb/au0828/au0828.h
@@ -31,6 +31,7 @@
 #include <media/v4l2-ctrls.h>
 #include <media/v4l2-fh.h>
 #include <media/media-device.h>
+#include <media/media-dev-allocator.h>
 
 /* DVB */
 #include <media/demux.h>
-- 
2.17.0

[RFC PATCH v8 3/4] media: media.h: Enable ALSA MEDIA_INTF_T* interface types

[shuah]

From: Shuah Khan <shuah@kernel.org>

Move ALSA MEDIA_INTF_T* interface types back into __KERNEL__ scope
to get ready for adding ALSA support to the media controller.

Signed-off-by: Shuah Khan <shuah@kernel.org>
---
 include/uapi/linux/media.h | 25 ++++++++++---------------
 1 file changed, 10 insertions(+), 15 deletions(-)

diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index 36f76e777ef9..07be07263597 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -262,6 +262,16 @@ struct media_links_enum {
 #define MEDIA_INTF_T_V4L_SWRADIO		(MEDIA_INTF_T_V4L_BASE + 4)
 #define MEDIA_INTF_T_V4L_TOUCH			(MEDIA_INTF_T_V4L_BASE + 5)
 
+#define MEDIA_INTF_T_ALSA_BASE			0x00000300
+#define MEDIA_INTF_T_ALSA_PCM_CAPTURE		(MEDIA_INTF_T_ALSA_BASE)
+#define MEDIA_INTF_T_ALSA_PCM_PLAYBACK		(MEDIA_INTF_T_ALSA_BASE + 1)
+#define MEDIA_INTF_T_ALSA_CONTROL		(MEDIA_INTF_T_ALSA_BASE + 2)
+#define MEDIA_INTF_T_ALSA_COMPRESS		(MEDIA_INTF_T_ALSA_BASE + 3)
+#define MEDIA_INTF_T_ALSA_RAWMIDI		(MEDIA_INTF_T_ALSA_BASE + 4)
+#define MEDIA_INTF_T_ALSA_HWDEP			(MEDIA_INTF_T_ALSA_BASE + 5)
+#define MEDIA_INTF_T_ALSA_SEQUENCER		(MEDIA_INTF_T_ALSA_BASE + 6)
+#define MEDIA_INTF_T_ALSA_TIMER			(MEDIA_INTF_T_ALSA_BASE + 7)
+
 #if defined(__KERNEL__)
 
 /*
@@ -404,21 +414,6 @@ struct media_v2_topology {
 
 #define MEDIA_ENT_F_DTV_DECODER			MEDIA_ENT_F_DV_DECODER
 
-/*
- * There is still no ALSA support in the media controller. These
- * defines should not have been added and we leave them here only
- * in case some application tries to use these defines.
- */
-#define MEDIA_INTF_T_ALSA_BASE			0x00000300
-#define MEDIA_INTF_T_ALSA_PCM_CAPTURE		(MEDIA_INTF_T_ALSA_BASE)
-#define MEDIA_INTF_T_ALSA_PCM_PLAYBACK		(MEDIA_INTF_T_ALSA_BASE + 1)
-#define MEDIA_INTF_T_ALSA_CONTROL		(MEDIA_INTF_T_ALSA_BASE + 2)
-#define MEDIA_INTF_T_ALSA_COMPRESS		(MEDIA_INTF_T_ALSA_BASE + 3)
-#define MEDIA_INTF_T_ALSA_RAWMIDI		(MEDIA_INTF_T_ALSA_BASE + 4)
-#define MEDIA_INTF_T_ALSA_HWDEP			(MEDIA_INTF_T_ALSA_BASE + 5)
-#define MEDIA_INTF_T_ALSA_SEQUENCER		(MEDIA_INTF_T_ALSA_BASE + 6)
-#define MEDIA_INTF_T_ALSA_TIMER			(MEDIA_INTF_T_ALSA_BASE + 7)
-
 /* Obsolete symbol for media_version, no longer used in the kernel */
 #define MEDIA_API_VERSION			((0 << 16) | (1 << 8) | 0)
 
-- 
2.17.0

[RFC PATCH v8 4/4] sound/usb: Use Media Controller API to share media resources

[shuah]

From: Shuah Khan <shuah@kernel.org>

Change ALSA driver to use Media Controller API to share media resources
with DVB, and V4L2 drivers on a AU0828 media device.

Media Controller specific initialization is done after sound card is
registered. ALSA creates Media interface and entity function graph
nodes for Control, Mixer, PCM Playback, and PCM Capture devices.

snd_usb_hw_params() will call Media Controller enable source handler
interface to request the media resource. If resource request is granted,
it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
returned.

Media specific cleanup is done in usb_audio_disconnect().

Signed-off-by: Shuah Khan <shuah@kernel.org>
---
 sound/usb/Kconfig        |   4 +
 sound/usb/Makefile       |   2 +
 sound/usb/card.c         |  14 ++
 sound/usb/card.h         |   3 +
 sound/usb/media.c        | 320 +++++++++++++++++++++++++++++++++++++++
 sound/usb/media.h        |  73 +++++++++
 sound/usb/mixer.h        |   3 +
 sound/usb/pcm.c          |  29 +++-
 sound/usb/quirks-table.h |   1 +
 sound/usb/stream.c       |   2 +
 sound/usb/usbaudio.h     |   6 +
 11 files changed, 453 insertions(+), 4 deletions(-)
 create mode 100644 sound/usb/media.c
 create mode 100644 sound/usb/media.h

diff --git a/sound/usb/Kconfig b/sound/usb/Kconfig
index f61b5662bb89..6319b544ba3a 100644
--- a/sound/usb/Kconfig
+++ b/sound/usb/Kconfig
@@ -15,6 +15,7 @@ config SND_USB_AUDIO
 	select SND_RAWMIDI
 	select SND_PCM
 	select BITREVERSE
+	select SND_USB_AUDIO_USE_MEDIA_CONTROLLER if MEDIA_CONTROLLER && (MEDIA_SUPPORT=y || MEDIA_SUPPORT=SND_USB_AUDIO)
 	help
 	  Say Y here to include support for USB audio and USB MIDI
 	  devices.
@@ -22,6 +23,9 @@ config SND_USB_AUDIO
 	  To compile this driver as a module, choose M here: the module
 	  will be called snd-usb-audio.
 
+config SND_USB_AUDIO_USE_MEDIA_CONTROLLER
+	bool
+
 config SND_USB_UA101
 	tristate "Edirol UA-101/UA-1000 driver"
 	select SND_PCM
diff --git a/sound/usb/Makefile b/sound/usb/Makefile
index d330f74c90e6..e1ce257ab705 100644
--- a/sound/usb/Makefile
+++ b/sound/usb/Makefile
@@ -18,6 +18,8 @@ snd-usb-audio-objs := 	card.o \
 			quirks.o \
 			stream.o
 
+snd-usb-audio-$(CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER) += media.o
+
 snd-usbmidi-lib-objs := midi.o
 
 # Toplevel Module Dependency
diff --git a/sound/usb/card.c b/sound/usb/card.c
index 2bfe4e80a6b9..d9ae331280a2 100644
--- a/sound/usb/card.c
+++ b/sound/usb/card.c
@@ -68,6 +68,7 @@
 #include "format.h"
 #include "power.h"
 #include "stream.h"
+#include "media.h"
 
 MODULE_AUTHOR("Takashi Iwai <tiwai@suse.de>");
 MODULE_DESCRIPTION("USB Audio");
@@ -673,6 +674,11 @@ static int usb_audio_probe(struct usb_interface *intf,
 	if (err < 0)
 		goto __error;
 
+	if (quirk && quirk->media_device) {
+		/* don't want to fail when media_snd_device_create() fails */
+		media_snd_device_create(chip, intf);
+	}
+
 	usb_chip[chip->index] = chip;
 	chip->num_interfaces++;
 	usb_set_intfdata(intf, chip);
@@ -729,6 +735,14 @@ static void usb_audio_disconnect(struct usb_interface *intf)
 		list_for_each(p, &chip->midi_list) {
 			snd_usbmidi_disconnect(p);
 		}
+		/*
+		 * Nice to check quirk && quirk->media_device and
+		 * then call media_snd_device_delete(). Don't have
+		 * access to quirk here. media_snd_device_delete()
+		 * acceses mixer_list
+		 */
+		media_snd_device_delete(chip);
+
 		/* release mixer resources */
 		list_for_each_entry(mixer, &chip->mixer_list, list) {
 			snd_usb_mixer_disconnect(mixer);
diff --git a/sound/usb/card.h b/sound/usb/card.h
index ac785d15ced4..5dd3538ed6b5 100644
--- a/sound/usb/card.h
+++ b/sound/usb/card.h
@@ -108,6 +108,8 @@ struct snd_usb_endpoint {
 	struct list_head list;
 };
 
+struct media_ctl;
+
 struct snd_usb_substream {
 	struct snd_usb_stream *stream;
 	struct usb_device *dev;
@@ -160,6 +162,7 @@ struct snd_usb_substream {
 	} dsd_dop;
 
 	bool trigger_tstamp_pending_update; /* trigger timestamp being updated from initial estimate */
+	struct media_ctl *media_ctl;
 };
 
 struct snd_usb_stream {
diff --git a/sound/usb/media.c b/sound/usb/media.c
new file mode 100644
index 000000000000..140a98eed294
--- /dev/null
+++ b/sound/usb/media.c
@@ -0,0 +1,320 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * media.c - Media Controller specific ALSA driver code
+ *
+ * Copyright (c) 2018 Shuah Khan <shuah@kernel.org>
+ *
+ */
+
+/*
+ * This file adds Media Controller support to ALSA driver
+ * to use the Media Controller API to share tuner with DVB
+ * and V4L2 drivers that control media device. Media device
+ * is created based on existing quirks framework. Using this
+ * approach, the media controller API usage can be added for
+ * a specific device.
+ */
+
+#include <linux/init.h>
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/slab.h>
+#include <linux/usb.h>
+
+#include <sound/pcm.h>
+#include <sound/core.h>
+
+#include "usbaudio.h"
+#include "card.h"
+#include "mixer.h"
+#include "media.h"
+
+int media_snd_stream_init(struct snd_usb_substream *subs, struct snd_pcm *pcm,
+			int stream)
+{
+	struct media_device *mdev;
+	struct media_ctl *mctl;
+	struct device *pcm_dev = &pcm->streams[stream].dev;
+	u32 intf_type;
+	int ret = 0;
+	u16 mixer_pad;
+	struct media_entity *entity;
+
+	mdev = subs->stream->chip->media_dev;
+	if (!mdev)
+		return 0;
+
+	if (subs->media_ctl)
+		return 0;
+
+	/* allocate media_ctl */
+	mctl = kzalloc(sizeof(*mctl), GFP_KERNEL);
+	if (!mctl)
+		return -ENOMEM;
+
+	mctl->media_dev = mdev;
+	if (stream == SNDRV_PCM_STREAM_PLAYBACK) {
+		intf_type = MEDIA_INTF_T_ALSA_PCM_PLAYBACK;
+		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_PLAYBACK;
+		mctl->media_pad.flags = MEDIA_PAD_FL_SOURCE;
+		mixer_pad = 1;
+	} else {
+		intf_type = MEDIA_INTF_T_ALSA_PCM_CAPTURE;
+		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_CAPTURE;
+		mctl->media_pad.flags = MEDIA_PAD_FL_SINK;
+		mixer_pad = 2;
+	}
+	mctl->media_entity.name = pcm->name;
+	media_entity_pads_init(&mctl->media_entity, 1, &mctl->media_pad);
+	ret =  media_device_register_entity(mctl->media_dev,
+					    &mctl->media_entity);
+	if (ret)
+		goto free_mctl;
+
+	mctl->intf_devnode = media_devnode_create(mdev, intf_type, 0,
+						  MAJOR(pcm_dev->devt),
+						  MINOR(pcm_dev->devt));
+	if (!mctl->intf_devnode) {
+		ret = -ENOMEM;
+		goto unregister_entity;
+	}
+	mctl->intf_link = media_create_intf_link(&mctl->media_entity,
+						 &mctl->intf_devnode->intf,
+						 MEDIA_LNK_FL_ENABLED);
+	if (!mctl->intf_link) {
+		ret = -ENOMEM;
+		goto devnode_remove;
+	}
+
+	/* create link between mixer and audio */
+	media_device_for_each_entity(entity, mdev) {
+		switch (entity->function) {
+		case MEDIA_ENT_F_AUDIO_MIXER:
+			ret = media_create_pad_link(entity, mixer_pad,
+						    &mctl->media_entity, 0,
+						    MEDIA_LNK_FL_ENABLED);
+			if (ret)
+				goto remove_intf_link;
+			break;
+		}
+	}
+
+	subs->media_ctl = mctl;
+	return 0;
+
+remove_intf_link:
+	media_remove_intf_link(mctl->intf_link);
+devnode_remove:
+	media_devnode_remove(mctl->intf_devnode);
+unregister_entity:
+	media_device_unregister_entity(&mctl->media_entity);
+free_mctl:
+	kfree(mctl);
+	return ret;
+}
+
+void media_snd_stream_delete(struct snd_usb_substream *subs)
+{
+	struct media_ctl *mctl = subs->media_ctl;
+
+	if (mctl) {
+		struct media_device *mdev;
+
+		mdev = mctl->media_dev;
+		if (mdev && media_devnode_is_registered(mdev->devnode)) {
+			media_devnode_remove(mctl->intf_devnode);
+			media_device_unregister_entity(&mctl->media_entity);
+			media_entity_cleanup(&mctl->media_entity);
+		}
+		kfree(mctl);
+		subs->media_ctl = NULL;
+	}
+}
+
+int media_snd_start_pipeline(struct snd_usb_substream *subs)
+{
+	struct media_ctl *mctl = subs->media_ctl;
+	int ret = 0;
+
+	if (!mctl)
+		return 0;
+
+	mutex_lock(&mctl->media_dev->graph_mutex);
+	if (mctl->media_dev->enable_source)
+		ret = mctl->media_dev->enable_source(&mctl->media_entity,
+						     &mctl->media_pipe);
+	mutex_unlock(&mctl->media_dev->graph_mutex);
+	return ret;
+}
+
+void media_snd_stop_pipeline(struct snd_usb_substream *subs)
+{
+	struct media_ctl *mctl = subs->media_ctl;
+
+	if (!mctl)
+		return;
+
+	mutex_lock(&mctl->media_dev->graph_mutex);
+	if (mctl->media_dev->disable_source)
+		mctl->media_dev->disable_source(&mctl->media_entity);
+	mutex_unlock(&mctl->media_dev->graph_mutex);
+}
+
+static int media_snd_mixer_init(struct snd_usb_audio *chip)
+{
+	struct device *ctl_dev = &chip->card->ctl_dev;
+	struct media_intf_devnode *ctl_intf;
+	struct usb_mixer_interface *mixer;
+	struct media_device *mdev = chip->media_dev;
+	struct media_mixer_ctl *mctl;
+	u32 intf_type = MEDIA_INTF_T_ALSA_CONTROL;
+	int ret;
+
+	if (!mdev)
+		return -ENODEV;
+
+	ctl_intf = chip->ctl_intf_media_devnode;
+	if (!ctl_intf) {
+		ctl_intf = media_devnode_create(mdev, intf_type, 0,
+						MAJOR(ctl_dev->devt),
+						MINOR(ctl_dev->devt));
+		if (!ctl_intf)
+			return -ENOMEM;
+		chip->ctl_intf_media_devnode = ctl_intf;
+	}
+
+	list_for_each_entry(mixer, &chip->mixer_list, list) {
+
+		if (mixer->media_mixer_ctl)
+			continue;
+
+		/* allocate media_mixer_ctl */
+		mctl = kzalloc(sizeof(*mctl), GFP_KERNEL);
+		if (!mctl)
+			return -ENOMEM;
+
+		mctl->media_dev = mdev;
+		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_MIXER;
+		mctl->media_entity.name = chip->card->mixername;
+		mctl->media_pad[0].flags = MEDIA_PAD_FL_SINK;
+		mctl->media_pad[1].flags = MEDIA_PAD_FL_SOURCE;
+		mctl->media_pad[2].flags = MEDIA_PAD_FL_SOURCE;
+		media_entity_pads_init(&mctl->media_entity, MEDIA_MIXER_PAD_MAX,
+				  mctl->media_pad);
+		ret =  media_device_register_entity(mctl->media_dev,
+						    &mctl->media_entity);
+		if (ret) {
+			kfree(mctl);
+			return ret;
+		}
+
+		mctl->intf_link = media_create_intf_link(&mctl->media_entity,
+							 &ctl_intf->intf,
+							 MEDIA_LNK_FL_ENABLED);
+		if (!mctl->intf_link) {
+			media_device_unregister_entity(&mctl->media_entity);
+			media_entity_cleanup(&mctl->media_entity);
+			kfree(mctl);
+			return -ENOMEM;
+		}
+		mctl->intf_devnode = ctl_intf;
+		mixer->media_mixer_ctl = mctl;
+	}
+	return 0;
+}
+
+static void media_snd_mixer_delete(struct snd_usb_audio *chip)
+{
+	struct usb_mixer_interface *mixer;
+	struct media_device *mdev = chip->media_dev;
+
+	if (!mdev)
+		return;
+
+	list_for_each_entry(mixer, &chip->mixer_list, list) {
+		struct media_mixer_ctl *mctl;
+
+		mctl = mixer->media_mixer_ctl;
+		if (!mixer->media_mixer_ctl)
+			continue;
+
+		if (media_devnode_is_registered(mdev->devnode)) {
+			media_device_unregister_entity(&mctl->media_entity);
+			media_entity_cleanup(&mctl->media_entity);
+		}
+		kfree(mctl);
+		mixer->media_mixer_ctl = NULL;
+	}
+	if (media_devnode_is_registered(mdev->devnode))
+		media_devnode_remove(chip->ctl_intf_media_devnode);
+	chip->ctl_intf_media_devnode = NULL;
+}
+
+int media_snd_device_create(struct snd_usb_audio *chip,
+			struct usb_interface *iface)
+{
+	struct media_device *mdev;
+	struct usb_device *usbdev = interface_to_usbdev(iface);
+	int ret;
+
+	/* usb-audio driver is probed for each usb interface, and
+	 * there are multiple interfaces per device. Avoid calling
+	 * media_device_usb_allocate() each time usb_audio_probe()
+	 * is called. Do it only once.
+	 */
+	if (chip->media_dev)
+		goto snd_mixer_init;
+
+	mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
+	if (!mdev)
+		return -ENOMEM;
+
+	if (!media_devnode_is_registered(mdev->devnode)) {
+		/* register media_device */
+		ret = media_device_register(mdev);
+		if (ret) {
+			media_device_delete(mdev, KBUILD_MODNAME);
+			dev_err(&usbdev->dev,
+				"Couldn't register media device. Error: %d\n",
+				ret);
+			return ret;
+		}
+	}
+
+	/* save media device - avoid lookups */
+	chip->media_dev = mdev;
+
+snd_mixer_init:
+	/* Create media entities for mixer and control dev */
+	ret = media_snd_mixer_init(chip);
+	if (ret) {
+		dev_err(&usbdev->dev,
+			"Couldn't create media mixer entities. Error: %d\n",
+			ret);
+
+		/* clear saved media_dev */
+		chip->media_dev = NULL;
+
+		return ret;
+	}
+	return 0;
+}
+
+void media_snd_device_delete(struct snd_usb_audio *chip)
+{
+	struct media_device *mdev = chip->media_dev;
+	struct snd_usb_stream *stream;
+
+	/* release resources */
+	list_for_each_entry(stream, &chip->pcm_list, list) {
+		media_snd_stream_delete(&stream->substream[0]);
+		media_snd_stream_delete(&stream->substream[1]);
+	}
+
+	media_snd_mixer_delete(chip);
+
+	if (mdev) {
+		media_device_delete(mdev, KBUILD_MODNAME);
+		chip->media_dev = NULL;
+	}
+}
diff --git a/sound/usb/media.h b/sound/usb/media.h
new file mode 100644
index 000000000000..fbefa9cad215
--- /dev/null
+++ b/sound/usb/media.h
@@ -0,0 +1,73 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+/*
+ * media.h - Media Controller specific ALSA driver code
+ *
+ * Copyright (c) 2018 Shuah Khan <shuah@kernel.org>
+ *
+ */
+
+/*
+ * This file adds Media Controller support to ALSA driver
+ * to use the Media Controller API to share tuner with DVB
+ * and V4L2 drivers that control media device. Media device
+ * is created based on existing quirks framework. Using this
+ * approach, the media controller API usage can be added for
+ * a specific device.
+ */
+#ifndef __MEDIA_H
+
+#ifdef CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER
+
+#include <linux/media.h>
+#include <media/media-device.h>
+#include <media/media-entity.h>
+#include <media/media-dev-allocator.h>
+#include <sound/asound.h>
+
+struct media_ctl {
+	struct media_device *media_dev;
+	struct media_entity media_entity;
+	struct media_intf_devnode *intf_devnode;
+	struct media_link *intf_link;
+	struct media_pad media_pad;
+	struct media_pipeline media_pipe;
+};
+
+/*
+ * One source pad each for SNDRV_PCM_STREAM_CAPTURE and
+ * SNDRV_PCM_STREAM_PLAYBACK. One for sink pad to link
+ * to AUDIO Source
+ */
+#define MEDIA_MIXER_PAD_MAX    (SNDRV_PCM_STREAM_LAST + 2)
+
+struct media_mixer_ctl {
+	struct media_device *media_dev;
+	struct media_entity media_entity;
+	struct media_intf_devnode *intf_devnode;
+	struct media_link *intf_link;
+	struct media_pad media_pad[MEDIA_MIXER_PAD_MAX];
+	struct media_pipeline media_pipe;
+};
+
+int media_snd_device_create(struct snd_usb_audio *chip,
+			    struct usb_interface *iface);
+void media_snd_device_delete(struct snd_usb_audio *chip);
+int media_snd_stream_init(struct snd_usb_substream *subs, struct snd_pcm *pcm,
+			  int stream);
+void media_snd_stream_delete(struct snd_usb_substream *subs);
+int media_snd_start_pipeline(struct snd_usb_substream *subs);
+void media_snd_stop_pipeline(struct snd_usb_substream *subs);
+#else
+static inline int media_snd_device_create(struct snd_usb_audio *chip,
+					  struct usb_interface *iface)
+						{ return 0; }
+static inline void media_snd_device_delete(struct snd_usb_audio *chip) { }
+static inline int media_snd_stream_init(struct snd_usb_substream *subs,
+					struct snd_pcm *pcm, int stream)
+						{ return 0; }
+static inline void media_snd_stream_delete(struct snd_usb_substream *subs) { }
+static inline int media_snd_start_pipeline(struct snd_usb_substream *subs)
+					{ return 0; }
+static inline void media_snd_stop_pipeline(struct snd_usb_substream *subs) { }
+#endif
+#endif /* __MEDIA_H */
diff --git a/sound/usb/mixer.h b/sound/usb/mixer.h
index 3d12af8bf191..394cd9107507 100644
--- a/sound/usb/mixer.h
+++ b/sound/usb/mixer.h
@@ -4,6 +4,8 @@
 
 #include <sound/info.h>
 
+struct media_mixer_ctl;
+
 struct usb_mixer_interface {
 	struct snd_usb_audio *chip;
 	struct usb_host_interface *hostif;
@@ -23,6 +25,7 @@ struct usb_mixer_interface {
 	struct urb *rc_urb;
 	struct usb_ctrlrequest *rc_setup_packet;
 	u8 rc_buffer[6];
+	struct media_mixer_ctl *media_mixer_ctl;
 
 	bool disconnected;
 };
diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c
index 382847154227..a1971a87c613 100644
--- a/sound/usb/pcm.c
+++ b/sound/usb/pcm.c
@@ -35,6 +35,7 @@
 #include "pcm.h"
 #include "clock.h"
 #include "power.h"
+#include "media.h"
 
 #define SUBSTREAM_FLAG_DATA_EP_STARTED	0
 #define SUBSTREAM_FLAG_SYNC_EP_STARTED	1
@@ -776,6 +777,10 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
 	struct audioformat *fmt;
 	int ret;
 
+	ret = media_snd_start_pipeline(subs);
+	if (ret)
+		return ret;
+
 	if (snd_usb_use_vmalloc)
 		ret = snd_pcm_lib_alloc_vmalloc_buffer(substream,
 						       params_buffer_bytes(hw_params));
@@ -783,7 +788,7 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
 		ret = snd_pcm_lib_malloc_pages(substream,
 					       params_buffer_bytes(hw_params));
 	if (ret < 0)
-		return ret;
+		goto stop_pipeline;
 
 	subs->pcm_format = params_format(hw_params);
 	subs->period_bytes = params_period_bytes(hw_params);
@@ -797,12 +802,13 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
 		dev_dbg(&subs->dev->dev,
 			"cannot set format: format = %#x, rate = %d, channels = %d\n",
 			   subs->pcm_format, subs->cur_rate, subs->channels);
-		return -EINVAL;
+		ret = -EINVAL;
+		goto stop_pipeline;
 	}
 
 	ret = snd_usb_lock_shutdown(subs->stream->chip);
 	if (ret < 0)
-		return ret;
+		goto stop_pipeline;
 
 	ret = snd_usb_pcm_change_state(subs, UAC3_PD_STATE_D0);
 	if (ret < 0)
@@ -818,6 +824,12 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
 
  unlock:
 	snd_usb_unlock_shutdown(subs->stream->chip);
+	if (ret < 0)
+		goto stop_pipeline;
+	return ret;
+
+ stop_pipeline:
+	media_snd_stop_pipeline(subs);
 	return ret;
 }
 
@@ -830,6 +842,7 @@ static int snd_usb_hw_free(struct snd_pcm_substream *substream)
 {
 	struct snd_usb_substream *subs = substream->runtime->private_data;
 
+	media_snd_stop_pipeline(subs);
 	subs->cur_audiofmt = NULL;
 	subs->cur_rate = 0;
 	subs->period_bytes = 0;
@@ -1302,6 +1315,7 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream)
 	struct snd_usb_stream *as = snd_pcm_substream_chip(substream);
 	struct snd_pcm_runtime *runtime = substream->runtime;
 	struct snd_usb_substream *subs = &as->substream[direction];
+	int ret;
 
 	subs->interface = -1;
 	subs->altset_idx = 0;
@@ -1315,7 +1329,13 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream)
 	subs->dsd_dop.channel = 0;
 	subs->dsd_dop.marker = 1;
 
-	return setup_hw_info(runtime, subs);
+	ret = setup_hw_info(runtime, subs);
+	if (ret == 0) {
+		ret = media_snd_stream_init(subs, as->pcm, direction);
+		if (ret)
+			snd_usb_autosuspend(subs->stream->chip);
+	}
+	return ret;
 }
 
 static int snd_usb_pcm_close(struct snd_pcm_substream *substream)
@@ -1326,6 +1346,7 @@ static int snd_usb_pcm_close(struct snd_pcm_substream *substream)
 	int ret;
 
 	stop_endpoints(subs, true);
+	media_snd_stop_pipeline(subs);
 
 	if (!as->chip->keep_iface &&
 	    subs->interface >= 0 &&
diff --git a/sound/usb/quirks-table.h b/sound/usb/quirks-table.h
index 08aa78007020..02789d1b2fec 100644
--- a/sound/usb/quirks-table.h
+++ b/sound/usb/quirks-table.h
@@ -2887,6 +2887,7 @@ YAMAHA_DEVICE(0x7010, "UB99"),
 		.product_name = pname, \
 		.ifnum = QUIRK_ANY_INTERFACE, \
 		.type = QUIRK_AUDIO_ALIGN_TRANSFER, \
+		.media_device = 1, \
 	} \
 }
 
diff --git a/sound/usb/stream.c b/sound/usb/stream.c
index 67cf849aa16b..5439dd2e6980 100644
--- a/sound/usb/stream.c
+++ b/sound/usb/stream.c
@@ -38,6 +38,7 @@
 #include "clock.h"
 #include "stream.h"
 #include "power.h"
+#include "media.h"
 
 /*
  * free a substream
@@ -55,6 +56,7 @@ static void free_substream(struct snd_usb_substream *subs)
 	}
 	kfree(subs->rate_list.list);
 	kfree(subs->str_pd);
+	media_snd_stream_delete(subs);
 }
 
 
diff --git a/sound/usb/usbaudio.h b/sound/usb/usbaudio.h
index b9faeca645fd..72bdae5ca54a 100644
--- a/sound/usb/usbaudio.h
+++ b/sound/usb/usbaudio.h
@@ -30,6 +30,9 @@
  *
  */
 
+struct media_device;
+struct media_intf_devnode;
+
 struct snd_usb_audio {
 	int index;
 	struct usb_device *dev;
@@ -66,6 +69,8 @@ struct snd_usb_audio {
 					 */
 
 	struct usb_host_interface *ctrl_intf;	/* the audio control interface */
+	struct media_device *media_dev;
+	struct media_intf_devnode *ctl_intf_media_devnode;
 };
 
 #define usb_audio_err(chip, fmt, args...) \
@@ -117,6 +122,7 @@ struct snd_usb_audio_quirk {
 	const char *profile_name;	/* override the card->longname */
 	int16_t ifnum;
 	uint16_t type;
+	bool media_device;
 	const void *data;
 };
 
-- 
2.17.0

Re: [RFC PATCH v8 1/4] media: Media Device Allocator API

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 704 bytes --]

On Thu 2018-11-01 18:31:30, shuah@kernel.org wrote:
> From: Shuah Khan <shuah@kernel.org>
> 
> Media Device Allocator API to allows multiple drivers share a media device.
> Using this API, drivers can allocate a media device with the shared struct
> device as the key. Once the media device is allocated by a driver, other
> drivers can get a reference to it. The media device is released when all
> the references are released.

Sounds like a ... bad idea?

That's what new "media control" framework is for, no?

Why do you need this?
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [RFC PATCH v8 1/4] media: Media Device Allocator API

[Hans Verkuil]

On 11/02/2018 01:31 AM, shuah@kernel.org wrote:
> From: Shuah Khan <shuah@kernel.org>
> 
> Media Device Allocator API to allows multiple drivers share a media device.
> Using this API, drivers can allocate a media device with the shared struct
> device as the key. Once the media device is allocated by a driver, other
> drivers can get a reference to it. The media device is released when all
> the references are released.
> 
> Signed-off-by: Shuah Khan <shuah@kernel.org>
> ---
>  Documentation/media/kapi/mc-core.rst |  37 ++++++++
>  drivers/media/Makefile               |   3 +-
>  drivers/media/media-dev-allocator.c  | 132 +++++++++++++++++++++++++++
>  include/media/media-dev-allocator.h  |  53 +++++++++++
>  4 files changed, 224 insertions(+), 1 deletion(-)
>  create mode 100644 drivers/media/media-dev-allocator.c
>  create mode 100644 include/media/media-dev-allocator.h
> 
> diff --git a/Documentation/media/kapi/mc-core.rst b/Documentation/media/kapi/mc-core.rst
> index 0c05503eaf1f..d6f409598065 100644
> --- a/Documentation/media/kapi/mc-core.rst
> +++ b/Documentation/media/kapi/mc-core.rst
> @@ -257,8 +257,45 @@ Subsystems should facilitate link validation by providing subsystem specific
>  helper functions to provide easy access for commonly needed information, and
>  in the end provide a way to use driver-specific callbacks.
>  
> +Media Controller Device Allocator API
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +When media device belongs to more than one driver, the shared media device

When -> When the

> +is allocated with the shared struct device as the key for look ups.
> +
> +Shared media device should stay in registered state until the last driver

Shared -> The shared

> +unregisters it. In addition, media device should be released when all the

media -> the media

> +references are released. Each driver gets a reference to the media device
> +during probe, when it allocates the media device. If media device is already
> +allocated, allocate API bumps up the refcount and return the existing media

allocate -> the allocate
return -> returns

> +device. Driver puts the reference back from its disconnect routine when it

Driver -> The driver
from -> in

> +calls :c:func:`media_device_delete()`.
> +
> +Media device is unregistered and cleaned up from the kref put handler to

Media -> The media
from -> in

> +ensure that the media device stays in registered state until the last driver
> +unregisters the media device.

What happens if an application still has the media device open and you forcibly
remove the last driver? I think it should be OK since the media_devnode struct
isn't freed until the last open filehandle closes. But it is good to test this.

> +
> +**Driver Usage**
> +
> +Drivers should use the media-core routines to get register reference and

'get register reference'? Not sure what you meant to say.

> +call :c:func:`media_device_delete()` routine to make sure the shared media
> +device delete is handled correctly.
> +
> +**driver probe:**
> +Call :c:func:`media_device_usb_allocate()` to allocate or get a reference
> +Call :c:func:`media_device_register()`, if media devnode isn't registered
> +
> +**driver disconnect:**
> +Call :c:func:`media_device_delete()` to free the media_device. Free'ing is

Free'ing -> Freeing

> +handled by the kref put handler.
> +
> +API Definitions
> +^^^^^^^^^^^^^^^
> +
>  .. kernel-doc:: include/media/media-device.h
>  
>  .. kernel-doc:: include/media/media-devnode.h
>  
>  .. kernel-doc:: include/media/media-entity.h
> +
> +.. kernel-doc:: include/media/media-dev-allocator.h
> diff --git a/drivers/media/Makefile b/drivers/media/Makefile
> index 594b462ddf0e..8608f0a41dca 100644
> --- a/drivers/media/Makefile
> +++ b/drivers/media/Makefile
> @@ -3,7 +3,8 @@
>  # Makefile for the kernel multimedia device drivers.
>  #
>  
> -media-objs	:= media-device.o media-devnode.o media-entity.o
> +media-objs	:= media-device.o media-devnode.o media-entity.o \
> +		   media-dev-allocator.o

Perhaps only add media-dev-allocator if CONFIG_USB is enabled?

>  
>  #
>  # I2C drivers should come before other drivers, otherwise they'll fail
> diff --git a/drivers/media/media-dev-allocator.c b/drivers/media/media-dev-allocator.c
> new file mode 100644
> index 000000000000..262d1293dc13
> --- /dev/null
> +++ b/drivers/media/media-dev-allocator.c
> @@ -0,0 +1,132 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * media-dev-allocator.c - Media Controller Device Allocator API
> + *
> + * Copyright (c) 2018 Shuah Khan <shuah@kernel.org>
> + *
> + * Credits: Suggested by Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> + */
> +
> +/*
> + * This file adds a global refcounted Media Controller Device Instance API.
> + * A system wide global media device list is managed and each media device
> + * includes a kref count. The last put on the media device releases the media
> + * device instance.
> + *
> + */
> +
> +#include <linux/kref.h>
> +#include <linux/module.h>
> +#include <linux/slab.h>
> +#include <linux/usb.h>
> +
> +#include <media/media-device.h>
> +
> +static LIST_HEAD(media_device_list);
> +static DEFINE_MUTEX(media_device_lock);
> +
> +struct media_device_instance {
> +	struct media_device mdev;
> +	struct module *owner;
> +	struct list_head list;
> +	struct kref refcount;
> +};
> +
> +static inline struct media_device_instance *
> +to_media_device_instance(struct media_device *mdev)
> +{
> +	return container_of(mdev, struct media_device_instance, mdev);
> +}
> +
> +static void media_device_instance_release(struct kref *kref)
> +{
> +	struct media_device_instance *mdi =
> +		container_of(kref, struct media_device_instance, refcount);
> +
> +	dev_dbg(mdi->mdev.dev, "%s: mdev=%p\n", __func__, &mdi->mdev);
> +
> +	mutex_lock(&media_device_lock);

Can't the lock be moved down to just before list_del? Or am I missing something?

> +
> +	media_device_unregister(&mdi->mdev);
> +	media_device_cleanup(&mdi->mdev);
> +
> +	list_del(&mdi->list);
> +	mutex_unlock(&media_device_lock);
> +
> +	kfree(mdi);
> +}
> +
> +/* Callers should hold media_device_lock when calling this function */
> +static struct media_device *__media_device_get(struct device *dev,
> +					       char *module_name)

const char *

> +{
> +	struct media_device_instance *mdi;
> +
> +	list_for_each_entry(mdi, &media_device_list, list) {
> +		if (mdi->mdev.dev != dev)
> +			continue;
> +
> +		kref_get(&mdi->refcount);
> +		/* get module reference for the media_device owner */
> +		if (find_module(module_name) != mdi->owner &&
> +		    !try_module_get(mdi->owner))
> +			dev_err(dev, "%s: try_module_get() error\n", __func__);
> +		dev_dbg(dev, "%s: get mdev=%p module_name %s\n",
> +			__func__, &mdi->mdev, module_name);
> +		return &mdi->mdev;
> +	}
> +
> +	mdi = kzalloc(sizeof(*mdi), GFP_KERNEL);
> +	if (!mdi)
> +		return NULL;
> +
> +	mdi->owner = find_module(module_name);
> +	kref_init(&mdi->refcount);
> +	list_add_tail(&mdi->list, &media_device_list);
> +
> +	dev_dbg(dev, "%s: alloc mdev=%p module_name %s\n", __func__,
> +		&mdi->mdev, module_name);
> +	return &mdi->mdev;
> +}
> +
> +#if IS_ENABLED(CONFIG_USB)
> +struct media_device *media_device_usb_allocate(struct usb_device *udev,
> +					       char *module_name)

const char *

> +{
> +	struct media_device *mdev;
> +
> +	mutex_lock(&media_device_lock);
> +	mdev = __media_device_get(&udev->dev, module_name);
> +	if (!mdev) {
> +		mutex_unlock(&media_device_lock);
> +		return ERR_PTR(-ENOMEM);
> +	}
> +
> +	/* check if media device is already initialized */
> +	if (!mdev->dev)
> +		__media_device_usb_init(mdev, udev, udev->product,
> +					module_name);
> +	mutex_unlock(&media_device_lock);
> +	return mdev;
> +}
> +EXPORT_SYMBOL_GPL(media_device_usb_allocate);
> +#endif
> +
> +void media_device_delete(struct media_device *mdev, char *module_name)
> +{
> +	struct media_device_instance *mdi = to_media_device_instance(mdev);
> +
> +	dev_dbg(mdi->mdev.dev, "%s: mdev=%p module_name %s\n",
> +		__func__, &mdi->mdev, module_name);
> +
> +	mutex_lock(&media_device_lock);
> +	/* put module reference if media_device owner is not THIS_MODULE */
> +	if (mdi->owner != find_module(module_name)) {
> +		module_put(mdi->owner);
> +		dev_dbg(mdi->mdev.dev,
> +			"%s decremented owner module reference\n", __func__);
> +	}
> +	mutex_unlock(&media_device_lock);
> +	kref_put(&mdi->refcount, media_device_instance_release);
> +}
> +EXPORT_SYMBOL_GPL(media_device_delete);
> diff --git a/include/media/media-dev-allocator.h b/include/media/media-dev-allocator.h
> new file mode 100644
> index 000000000000..8fbadff2cef8
> --- /dev/null
> +++ b/include/media/media-dev-allocator.h
> @@ -0,0 +1,53 @@
> +/* SPDX-License-Identifier: GPL-2.0+ */
> +/*
> + * media-dev-allocator.h - Media Controller Device Allocator API
> + *
> + * Copyright (c) 2018 Shuah Khan <shuah@kernel.org>
> + *
> + * Credits: Suggested by Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> + */
> +
> +/*
> + * This file adds a global ref-counted Media Controller Device Instance API.
> + * A system wide global media device list is managed and each media device
> + * includes a kref count. The last put on the media device releases the media
> + * device instance.
> + */
> +
> +#ifndef _MEDIA_DEV_ALLOCTOR_H
> +#define _MEDIA_DEV_ALLOCTOR_H
> +
> +struct usb_device;
> +
> +#ifdef CONFIG_MEDIA_CONTROLLER

Perhaps change this to:

#if defined(CONFIG_MEDIA_CONTROLLER) && defined(CONFIG_USB)

Since for now this only makes sense for USB devices.

> +/**
> + * media_device_usb_allocate() - Allocate and return struct &media device
> + *
> + * @udev:		struct &usb_device pointer
> + * @module_name:	should be filled with %KBUILD_MODNAME
> + *
> + * This interface should be called to allocate a Media Device when multiple
> + * drivers share usb_device and the media device. This interface allocates
> + * &media_device structure and calls media_device_usb_init() to initialize
> + * it.
> + *
> + */
> +struct media_device *media_device_usb_allocate(struct usb_device *udev,
> +					       char *module_name);
> +/**
> + * media_device_delete() - Release media device. Calls kref_put().
> + *
> + * @mdev:		struct &media_device pointer
> + * @module_name:	should be filled with %KBUILD_MODNAME
> + *
> + * This interface should be called to put Media Device Instance kref.
> + */
> +void media_device_delete(struct media_device *mdev, char *module_name);
> +#else
> +static inline struct media_device *media_device_usb_allocate(
> +			struct usb_device *udev, char *module_name)
> +			{ return NULL; }
> +static inline void media_device_delete(
> +			struct media_device *mdev, char *module_name) { }
> +#endif /* CONFIG_MEDIA_CONTROLLER */
> +#endif
> 

Regards,

	Hans

Re: [RFC PATCH v8 3/4] media: media.h: Enable ALSA MEDIA_INTF_T* interface types

[Hans Verkuil]

On 11/02/2018 01:31 AM, shuah@kernel.org wrote:
> From: Shuah Khan <shuah@kernel.org>
> 
> Move ALSA MEDIA_INTF_T* interface types back into __KERNEL__ scope
> to get ready for adding ALSA support to the media controller.
> 
> Signed-off-by: Shuah Khan <shuah@kernel.org>
> ---
>  include/uapi/linux/media.h | 25 ++++++++++---------------
>  1 file changed, 10 insertions(+), 15 deletions(-)
> 
> diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
> index 36f76e777ef9..07be07263597 100644
> --- a/include/uapi/linux/media.h
> +++ b/include/uapi/linux/media.h
> @@ -262,6 +262,16 @@ struct media_links_enum {
>  #define MEDIA_INTF_T_V4L_SWRADIO		(MEDIA_INTF_T_V4L_BASE + 4)
>  #define MEDIA_INTF_T_V4L_TOUCH			(MEDIA_INTF_T_V4L_BASE + 5)
>  
> +#define MEDIA_INTF_T_ALSA_BASE			0x00000300
> +#define MEDIA_INTF_T_ALSA_PCM_CAPTURE		(MEDIA_INTF_T_ALSA_BASE)
> +#define MEDIA_INTF_T_ALSA_PCM_PLAYBACK		(MEDIA_INTF_T_ALSA_BASE + 1)
> +#define MEDIA_INTF_T_ALSA_CONTROL		(MEDIA_INTF_T_ALSA_BASE + 2)
> +#define MEDIA_INTF_T_ALSA_COMPRESS		(MEDIA_INTF_T_ALSA_BASE + 3)
> +#define MEDIA_INTF_T_ALSA_RAWMIDI		(MEDIA_INTF_T_ALSA_BASE + 4)
> +#define MEDIA_INTF_T_ALSA_HWDEP			(MEDIA_INTF_T_ALSA_BASE + 5)
> +#define MEDIA_INTF_T_ALSA_SEQUENCER		(MEDIA_INTF_T_ALSA_BASE + 6)
> +#define MEDIA_INTF_T_ALSA_TIMER			(MEDIA_INTF_T_ALSA_BASE + 7)
> +

I would only enable those defines that you need for the next patch.

Regards,

	Hans

>  #if defined(__KERNEL__)
>  
>  /*
> @@ -404,21 +414,6 @@ struct media_v2_topology {
>  
>  #define MEDIA_ENT_F_DTV_DECODER			MEDIA_ENT_F_DV_DECODER
>  
> -/*
> - * There is still no ALSA support in the media controller. These
> - * defines should not have been added and we leave them here only
> - * in case some application tries to use these defines.
> - */
> -#define MEDIA_INTF_T_ALSA_BASE			0x00000300
> -#define MEDIA_INTF_T_ALSA_PCM_CAPTURE		(MEDIA_INTF_T_ALSA_BASE)
> -#define MEDIA_INTF_T_ALSA_PCM_PLAYBACK		(MEDIA_INTF_T_ALSA_BASE + 1)
> -#define MEDIA_INTF_T_ALSA_CONTROL		(MEDIA_INTF_T_ALSA_BASE + 2)
> -#define MEDIA_INTF_T_ALSA_COMPRESS		(MEDIA_INTF_T_ALSA_BASE + 3)
> -#define MEDIA_INTF_T_ALSA_RAWMIDI		(MEDIA_INTF_T_ALSA_BASE + 4)
> -#define MEDIA_INTF_T_ALSA_HWDEP			(MEDIA_INTF_T_ALSA_BASE + 5)
> -#define MEDIA_INTF_T_ALSA_SEQUENCER		(MEDIA_INTF_T_ALSA_BASE + 6)
> -#define MEDIA_INTF_T_ALSA_TIMER			(MEDIA_INTF_T_ALSA_BASE + 7)
> -
>  /* Obsolete symbol for media_version, no longer used in the kernel */
>  #define MEDIA_API_VERSION			((0 << 16) | (1 << 8) | 0)
>  
> 

Re: [RFC PATCH v8 4/4] sound/usb: Use Media Controller API to share media resources

[Hans Verkuil]

On 11/02/2018 01:31 AM, shuah@kernel.org wrote:
> From: Shuah Khan <shuah@kernel.org>
> 
> Change ALSA driver to use Media Controller API to share media resources
> with DVB, and V4L2 drivers on a AU0828 media device.
> 
> Media Controller specific initialization is done after sound card is
> registered. ALSA creates Media interface and entity function graph
> nodes for Control, Mixer, PCM Playback, and PCM Capture devices.
> 
> snd_usb_hw_params() will call Media Controller enable source handler
> interface to request the media resource. If resource request is granted,
> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
> returned.
> 
> Media specific cleanup is done in usb_audio_disconnect().
> 
> Signed-off-by: Shuah Khan <shuah@kernel.org>
> ---
>  sound/usb/Kconfig        |   4 +
>  sound/usb/Makefile       |   2 +
>  sound/usb/card.c         |  14 ++
>  sound/usb/card.h         |   3 +
>  sound/usb/media.c        | 320 +++++++++++++++++++++++++++++++++++++++
>  sound/usb/media.h        |  73 +++++++++
>  sound/usb/mixer.h        |   3 +
>  sound/usb/pcm.c          |  29 +++-
>  sound/usb/quirks-table.h |   1 +
>  sound/usb/stream.c       |   2 +
>  sound/usb/usbaudio.h     |   6 +
>  11 files changed, 453 insertions(+), 4 deletions(-)
>  create mode 100644 sound/usb/media.c
>  create mode 100644 sound/usb/media.h
> 
> diff --git a/sound/usb/Kconfig b/sound/usb/Kconfig
> index f61b5662bb89..6319b544ba3a 100644
> --- a/sound/usb/Kconfig
> +++ b/sound/usb/Kconfig
> @@ -15,6 +15,7 @@ config SND_USB_AUDIO
>  	select SND_RAWMIDI
>  	select SND_PCM
>  	select BITREVERSE
> +	select SND_USB_AUDIO_USE_MEDIA_CONTROLLER if MEDIA_CONTROLLER && (MEDIA_SUPPORT=y || MEDIA_SUPPORT=SND_USB_AUDIO)
>  	help
>  	  Say Y here to include support for USB audio and USB MIDI
>  	  devices.
> @@ -22,6 +23,9 @@ config SND_USB_AUDIO
>  	  To compile this driver as a module, choose M here: the module
>  	  will be called snd-usb-audio.
>  
> +config SND_USB_AUDIO_USE_MEDIA_CONTROLLER
> +	bool
> +
>  config SND_USB_UA101
>  	tristate "Edirol UA-101/UA-1000 driver"
>  	select SND_PCM
> diff --git a/sound/usb/Makefile b/sound/usb/Makefile
> index d330f74c90e6..e1ce257ab705 100644
> --- a/sound/usb/Makefile
> +++ b/sound/usb/Makefile
> @@ -18,6 +18,8 @@ snd-usb-audio-objs := 	card.o \
>  			quirks.o \
>  			stream.o
>  
> +snd-usb-audio-$(CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER) += media.o
> +
>  snd-usbmidi-lib-objs := midi.o
>  
>  # Toplevel Module Dependency
> diff --git a/sound/usb/card.c b/sound/usb/card.c
> index 2bfe4e80a6b9..d9ae331280a2 100644
> --- a/sound/usb/card.c
> +++ b/sound/usb/card.c
> @@ -68,6 +68,7 @@
>  #include "format.h"
>  #include "power.h"
>  #include "stream.h"
> +#include "media.h"
>  
>  MODULE_AUTHOR("Takashi Iwai <tiwai@suse.de>");
>  MODULE_DESCRIPTION("USB Audio");
> @@ -673,6 +674,11 @@ static int usb_audio_probe(struct usb_interface *intf,
>  	if (err < 0)
>  		goto __error;
>  
> +	if (quirk && quirk->media_device) {
> +		/* don't want to fail when media_snd_device_create() fails */
> +		media_snd_device_create(chip, intf);

I'd change the prefix to snd_media_device_create. After all, you are creating
a media_device for a sound device.

> +	}
> +
>  	usb_chip[chip->index] = chip;
>  	chip->num_interfaces++;
>  	usb_set_intfdata(intf, chip);
> @@ -729,6 +735,14 @@ static void usb_audio_disconnect(struct usb_interface *intf)
>  		list_for_each(p, &chip->midi_list) {
>  			snd_usbmidi_disconnect(p);
>  		}
> +		/*
> +		 * Nice to check quirk && quirk->media_device and
> +		 * then call media_snd_device_delete(). Don't have
> +		 * access to quirk here. media_snd_device_delete()
> +		 * acceses mixer_list

acceses -> accesses

> +		 */
> +		media_snd_device_delete(chip);
> +
>  		/* release mixer resources */
>  		list_for_each_entry(mixer, &chip->mixer_list, list) {
>  			snd_usb_mixer_disconnect(mixer);
> diff --git a/sound/usb/card.h b/sound/usb/card.h
> index ac785d15ced4..5dd3538ed6b5 100644
> --- a/sound/usb/card.h
> +++ b/sound/usb/card.h
> @@ -108,6 +108,8 @@ struct snd_usb_endpoint {
>  	struct list_head list;
>  };
>  
> +struct media_ctl;
> +
>  struct snd_usb_substream {
>  	struct snd_usb_stream *stream;
>  	struct usb_device *dev;
> @@ -160,6 +162,7 @@ struct snd_usb_substream {
>  	} dsd_dop;
>  
>  	bool trigger_tstamp_pending_update; /* trigger timestamp being updated from initial estimate */
> +	struct media_ctl *media_ctl;
>  };
>  
>  struct snd_usb_stream {
> diff --git a/sound/usb/media.c b/sound/usb/media.c
> new file mode 100644
> index 000000000000..140a98eed294
> --- /dev/null
> +++ b/sound/usb/media.c
> @@ -0,0 +1,320 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * media.c - Media Controller specific ALSA driver code
> + *
> + * Copyright (c) 2018 Shuah Khan <shuah@kernel.org>
> + *
> + */
> +
> +/*
> + * This file adds Media Controller support to ALSA driver

to -> to the

> + * to use the Media Controller API to share tuner with DVB

tuner -> the tuner

> + * and V4L2 drivers that control media device. Media device

that -> that the
Media -> The media

> + * is created based on existing quirks framework. Using this

on -> on the

> + * approach, the media controller API usage can be added for
> + * a specific device.
> + */
> +
> +#include <linux/init.h>
> +#include <linux/list.h>
> +#include <linux/mutex.h>
> +#include <linux/slab.h>
> +#include <linux/usb.h>
> +
> +#include <sound/pcm.h>
> +#include <sound/core.h>
> +
> +#include "usbaudio.h"
> +#include "card.h"
> +#include "mixer.h"
> +#include "media.h"
> +
> +int media_snd_stream_init(struct snd_usb_substream *subs, struct snd_pcm *pcm,

snd_media_stream_init makes more sense to me. Just my opinion though.

> +			int stream)
> +{
> +	struct media_device *mdev;
> +	struct media_ctl *mctl;
> +	struct device *pcm_dev = &pcm->streams[stream].dev;
> +	u32 intf_type;
> +	int ret = 0;
> +	u16 mixer_pad;
> +	struct media_entity *entity;
> +
> +	mdev = subs->stream->chip->media_dev;
> +	if (!mdev)
> +		return 0;
> +
> +	if (subs->media_ctl)
> +		return 0;
> +
> +	/* allocate media_ctl */
> +	mctl = kzalloc(sizeof(*mctl), GFP_KERNEL);
> +	if (!mctl)
> +		return -ENOMEM;
> +
> +	mctl->media_dev = mdev;
> +	if (stream == SNDRV_PCM_STREAM_PLAYBACK) {
> +		intf_type = MEDIA_INTF_T_ALSA_PCM_PLAYBACK;
> +		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_PLAYBACK;
> +		mctl->media_pad.flags = MEDIA_PAD_FL_SOURCE;
> +		mixer_pad = 1;
> +	} else {
> +		intf_type = MEDIA_INTF_T_ALSA_PCM_CAPTURE;
> +		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_CAPTURE;
> +		mctl->media_pad.flags = MEDIA_PAD_FL_SINK;
> +		mixer_pad = 2;
> +	}
> +	mctl->media_entity.name = pcm->name;
> +	media_entity_pads_init(&mctl->media_entity, 1, &mctl->media_pad);
> +	ret =  media_device_register_entity(mctl->media_dev,
> +					    &mctl->media_entity);
> +	if (ret)
> +		goto free_mctl;
> +
> +	mctl->intf_devnode = media_devnode_create(mdev, intf_type, 0,
> +						  MAJOR(pcm_dev->devt),
> +						  MINOR(pcm_dev->devt));
> +	if (!mctl->intf_devnode) {
> +		ret = -ENOMEM;
> +		goto unregister_entity;
> +	}
> +	mctl->intf_link = media_create_intf_link(&mctl->media_entity,
> +						 &mctl->intf_devnode->intf,
> +						 MEDIA_LNK_FL_ENABLED);
> +	if (!mctl->intf_link) {
> +		ret = -ENOMEM;
> +		goto devnode_remove;
> +	}
> +
> +	/* create link between mixer and audio */
> +	media_device_for_each_entity(entity, mdev) {
> +		switch (entity->function) {
> +		case MEDIA_ENT_F_AUDIO_MIXER:
> +			ret = media_create_pad_link(entity, mixer_pad,
> +						    &mctl->media_entity, 0,
> +						    MEDIA_LNK_FL_ENABLED);
> +			if (ret)
> +				goto remove_intf_link;
> +			break;
> +		}
> +	}
> +
> +	subs->media_ctl = mctl;
> +	return 0;
> +
> +remove_intf_link:
> +	media_remove_intf_link(mctl->intf_link);
> +devnode_remove:
> +	media_devnode_remove(mctl->intf_devnode);
> +unregister_entity:
> +	media_device_unregister_entity(&mctl->media_entity);
> +free_mctl:
> +	kfree(mctl);
> +	return ret;
> +}
> +
> +void media_snd_stream_delete(struct snd_usb_substream *subs)
> +{
> +	struct media_ctl *mctl = subs->media_ctl;
> +
> +	if (mctl) {
> +		struct media_device *mdev;
> +
> +		mdev = mctl->media_dev;
> +		if (mdev && media_devnode_is_registered(mdev->devnode)) {
> +			media_devnode_remove(mctl->intf_devnode);
> +			media_device_unregister_entity(&mctl->media_entity);
> +			media_entity_cleanup(&mctl->media_entity);
> +		}
> +		kfree(mctl);
> +		subs->media_ctl = NULL;
> +	}
> +}
> +
> +int media_snd_start_pipeline(struct snd_usb_substream *subs)
> +{
> +	struct media_ctl *mctl = subs->media_ctl;
> +	int ret = 0;
> +
> +	if (!mctl)
> +		return 0;
> +
> +	mutex_lock(&mctl->media_dev->graph_mutex);
> +	if (mctl->media_dev->enable_source)
> +		ret = mctl->media_dev->enable_source(&mctl->media_entity,
> +						     &mctl->media_pipe);
> +	mutex_unlock(&mctl->media_dev->graph_mutex);
> +	return ret;
> +}
> +
> +void media_snd_stop_pipeline(struct snd_usb_substream *subs)
> +{
> +	struct media_ctl *mctl = subs->media_ctl;
> +
> +	if (!mctl)
> +		return;
> +
> +	mutex_lock(&mctl->media_dev->graph_mutex);
> +	if (mctl->media_dev->disable_source)
> +		mctl->media_dev->disable_source(&mctl->media_entity);
> +	mutex_unlock(&mctl->media_dev->graph_mutex);
> +}
> +
> +static int media_snd_mixer_init(struct snd_usb_audio *chip)
> +{
> +	struct device *ctl_dev = &chip->card->ctl_dev;
> +	struct media_intf_devnode *ctl_intf;
> +	struct usb_mixer_interface *mixer;
> +	struct media_device *mdev = chip->media_dev;
> +	struct media_mixer_ctl *mctl;
> +	u32 intf_type = MEDIA_INTF_T_ALSA_CONTROL;
> +	int ret;
> +
> +	if (!mdev)
> +		return -ENODEV;
> +
> +	ctl_intf = chip->ctl_intf_media_devnode;
> +	if (!ctl_intf) {
> +		ctl_intf = media_devnode_create(mdev, intf_type, 0,
> +						MAJOR(ctl_dev->devt),
> +						MINOR(ctl_dev->devt));
> +		if (!ctl_intf)
> +			return -ENOMEM;
> +		chip->ctl_intf_media_devnode = ctl_intf;
> +	}
> +
> +	list_for_each_entry(mixer, &chip->mixer_list, list) {
> +
> +		if (mixer->media_mixer_ctl)
> +			continue;
> +
> +		/* allocate media_mixer_ctl */
> +		mctl = kzalloc(sizeof(*mctl), GFP_KERNEL);
> +		if (!mctl)
> +			return -ENOMEM;
> +
> +		mctl->media_dev = mdev;
> +		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_MIXER;
> +		mctl->media_entity.name = chip->card->mixername;
> +		mctl->media_pad[0].flags = MEDIA_PAD_FL_SINK;
> +		mctl->media_pad[1].flags = MEDIA_PAD_FL_SOURCE;
> +		mctl->media_pad[2].flags = MEDIA_PAD_FL_SOURCE;
> +		media_entity_pads_init(&mctl->media_entity, MEDIA_MIXER_PAD_MAX,
> +				  mctl->media_pad);
> +		ret =  media_device_register_entity(mctl->media_dev,
> +						    &mctl->media_entity);
> +		if (ret) {
> +			kfree(mctl);
> +			return ret;
> +		}
> +
> +		mctl->intf_link = media_create_intf_link(&mctl->media_entity,
> +							 &ctl_intf->intf,
> +							 MEDIA_LNK_FL_ENABLED);
> +		if (!mctl->intf_link) {
> +			media_device_unregister_entity(&mctl->media_entity);
> +			media_entity_cleanup(&mctl->media_entity);
> +			kfree(mctl);
> +			return -ENOMEM;
> +		}
> +		mctl->intf_devnode = ctl_intf;
> +		mixer->media_mixer_ctl = mctl;
> +	}
> +	return 0;
> +}
> +
> +static void media_snd_mixer_delete(struct snd_usb_audio *chip)
> +{
> +	struct usb_mixer_interface *mixer;
> +	struct media_device *mdev = chip->media_dev;
> +
> +	if (!mdev)
> +		return;
> +
> +	list_for_each_entry(mixer, &chip->mixer_list, list) {
> +		struct media_mixer_ctl *mctl;
> +
> +		mctl = mixer->media_mixer_ctl;
> +		if (!mixer->media_mixer_ctl)
> +			continue;
> +
> +		if (media_devnode_is_registered(mdev->devnode)) {
> +			media_device_unregister_entity(&mctl->media_entity);
> +			media_entity_cleanup(&mctl->media_entity);
> +		}
> +		kfree(mctl);
> +		mixer->media_mixer_ctl = NULL;
> +	}
> +	if (media_devnode_is_registered(mdev->devnode))
> +		media_devnode_remove(chip->ctl_intf_media_devnode);
> +	chip->ctl_intf_media_devnode = NULL;
> +}
> +
> +int media_snd_device_create(struct snd_usb_audio *chip,
> +			struct usb_interface *iface)
> +{
> +	struct media_device *mdev;
> +	struct usb_device *usbdev = interface_to_usbdev(iface);
> +	int ret;
> +
> +	/* usb-audio driver is probed for each usb interface, and
> +	 * there are multiple interfaces per device. Avoid calling
> +	 * media_device_usb_allocate() each time usb_audio_probe()
> +	 * is called. Do it only once.
> +	 */
> +	if (chip->media_dev)
> +		goto snd_mixer_init;
> +
> +	mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
> +	if (!mdev)
> +		return -ENOMEM;
> +
> +	if (!media_devnode_is_registered(mdev->devnode)) {
> +		/* register media_device */
> +		ret = media_device_register(mdev);

You should first configure the media device before registering it.

General note: since there are two drivers modifying the same media device
the topology will change as these drivers configure the media device.

This means that there is a short window where the media device is registered
but the topology is only partially up.

Work is in progress to signal topology changes to userspace, but that is not
in (or even posted) yet. I do not believe that is an issue in this particular
case.

> +		if (ret) {
> +			media_device_delete(mdev, KBUILD_MODNAME);
> +			dev_err(&usbdev->dev,
> +				"Couldn't register media device. Error: %d\n",
> +				ret);
> +			return ret;
> +		}
> +	}
> +
> +	/* save media device - avoid lookups */
> +	chip->media_dev = mdev;
> +
> +snd_mixer_init:
> +	/* Create media entities for mixer and control dev */
> +	ret = media_snd_mixer_init(chip);
> +	if (ret) {
> +		dev_err(&usbdev->dev,
> +			"Couldn't create media mixer entities. Error: %d\n",
> +			ret);
> +
> +		/* clear saved media_dev */
> +		chip->media_dev = NULL;
> +
> +		return ret;
> +	}
> +	return 0;
> +}
> +
> +void media_snd_device_delete(struct snd_usb_audio *chip)
> +{
> +	struct media_device *mdev = chip->media_dev;
> +	struct snd_usb_stream *stream;
> +
> +	/* release resources */
> +	list_for_each_entry(stream, &chip->pcm_list, list) {
> +		media_snd_stream_delete(&stream->substream[0]);
> +		media_snd_stream_delete(&stream->substream[1]);
> +	}
> +
> +	media_snd_mixer_delete(chip);
> +
> +	if (mdev) {
> +		media_device_delete(mdev, KBUILD_MODNAME);
> +		chip->media_dev = NULL;
> +	}
> +}
> diff --git a/sound/usb/media.h b/sound/usb/media.h
> new file mode 100644
> index 000000000000..fbefa9cad215
> --- /dev/null
> +++ b/sound/usb/media.h
> @@ -0,0 +1,73 @@
> +/* SPDX-License-Identifier: GPL-2.0+ */
> +/*
> + * media.h - Media Controller specific ALSA driver code
> + *
> + * Copyright (c) 2018 Shuah Khan <shuah@kernel.org>
> + *
> + */
> +
> +/*
> + * This file adds Media Controller support to ALSA driver
> + * to use the Media Controller API to share tuner with DVB
> + * and V4L2 drivers that control media device. Media device
> + * is created based on existing quirks framework. Using this
> + * approach, the media controller API usage can be added for
> + * a specific device.
> + */
> +#ifndef __MEDIA_H
> +
> +#ifdef CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER
> +
> +#include <linux/media.h>
> +#include <media/media-device.h>
> +#include <media/media-entity.h>
> +#include <media/media-dev-allocator.h>
> +#include <sound/asound.h>
> +
> +struct media_ctl {
> +	struct media_device *media_dev;
> +	struct media_entity media_entity;
> +	struct media_intf_devnode *intf_devnode;
> +	struct media_link *intf_link;
> +	struct media_pad media_pad;
> +	struct media_pipeline media_pipe;
> +};
> +
> +/*
> + * One source pad each for SNDRV_PCM_STREAM_CAPTURE and
> + * SNDRV_PCM_STREAM_PLAYBACK. One for sink pad to link
> + * to AUDIO Source
> + */
> +#define MEDIA_MIXER_PAD_MAX    (SNDRV_PCM_STREAM_LAST + 2)
> +
> +struct media_mixer_ctl {
> +	struct media_device *media_dev;
> +	struct media_entity media_entity;
> +	struct media_intf_devnode *intf_devnode;
> +	struct media_link *intf_link;
> +	struct media_pad media_pad[MEDIA_MIXER_PAD_MAX];
> +	struct media_pipeline media_pipe;
> +};
> +
> +int media_snd_device_create(struct snd_usb_audio *chip,
> +			    struct usb_interface *iface);
> +void media_snd_device_delete(struct snd_usb_audio *chip);
> +int media_snd_stream_init(struct snd_usb_substream *subs, struct snd_pcm *pcm,
> +			  int stream);
> +void media_snd_stream_delete(struct snd_usb_substream *subs);
> +int media_snd_start_pipeline(struct snd_usb_substream *subs);
> +void media_snd_stop_pipeline(struct snd_usb_substream *subs);
> +#else
> +static inline int media_snd_device_create(struct snd_usb_audio *chip,
> +					  struct usb_interface *iface)
> +						{ return 0; }
> +static inline void media_snd_device_delete(struct snd_usb_audio *chip) { }
> +static inline int media_snd_stream_init(struct snd_usb_substream *subs,
> +					struct snd_pcm *pcm, int stream)
> +						{ return 0; }
> +static inline void media_snd_stream_delete(struct snd_usb_substream *subs) { }
> +static inline int media_snd_start_pipeline(struct snd_usb_substream *subs)
> +					{ return 0; }
> +static inline void media_snd_stop_pipeline(struct snd_usb_substream *subs) { }
> +#endif
> +#endif /* __MEDIA_H */
> diff --git a/sound/usb/mixer.h b/sound/usb/mixer.h
> index 3d12af8bf191..394cd9107507 100644
> --- a/sound/usb/mixer.h
> +++ b/sound/usb/mixer.h
> @@ -4,6 +4,8 @@
>  
>  #include <sound/info.h>
>  
> +struct media_mixer_ctl;
> +
>  struct usb_mixer_interface {
>  	struct snd_usb_audio *chip;
>  	struct usb_host_interface *hostif;
> @@ -23,6 +25,7 @@ struct usb_mixer_interface {
>  	struct urb *rc_urb;
>  	struct usb_ctrlrequest *rc_setup_packet;
>  	u8 rc_buffer[6];
> +	struct media_mixer_ctl *media_mixer_ctl;
>  
>  	bool disconnected;
>  };
> diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c
> index 382847154227..a1971a87c613 100644
> --- a/sound/usb/pcm.c
> +++ b/sound/usb/pcm.c
> @@ -35,6 +35,7 @@
>  #include "pcm.h"
>  #include "clock.h"
>  #include "power.h"
> +#include "media.h"
>  
>  #define SUBSTREAM_FLAG_DATA_EP_STARTED	0
>  #define SUBSTREAM_FLAG_SYNC_EP_STARTED	1
> @@ -776,6 +777,10 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
>  	struct audioformat *fmt;
>  	int ret;
>  
> +	ret = media_snd_start_pipeline(subs);
> +	if (ret)
> +		return ret;
> +
>  	if (snd_usb_use_vmalloc)
>  		ret = snd_pcm_lib_alloc_vmalloc_buffer(substream,
>  						       params_buffer_bytes(hw_params));
> @@ -783,7 +788,7 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
>  		ret = snd_pcm_lib_malloc_pages(substream,
>  					       params_buffer_bytes(hw_params));
>  	if (ret < 0)
> -		return ret;
> +		goto stop_pipeline;
>  
>  	subs->pcm_format = params_format(hw_params);
>  	subs->period_bytes = params_period_bytes(hw_params);
> @@ -797,12 +802,13 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
>  		dev_dbg(&subs->dev->dev,
>  			"cannot set format: format = %#x, rate = %d, channels = %d\n",
>  			   subs->pcm_format, subs->cur_rate, subs->channels);
> -		return -EINVAL;
> +		ret = -EINVAL;
> +		goto stop_pipeline;
>  	}
>  
>  	ret = snd_usb_lock_shutdown(subs->stream->chip);
>  	if (ret < 0)
> -		return ret;
> +		goto stop_pipeline;
>  
>  	ret = snd_usb_pcm_change_state(subs, UAC3_PD_STATE_D0);
>  	if (ret < 0)
> @@ -818,6 +824,12 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
>  
>   unlock:
>  	snd_usb_unlock_shutdown(subs->stream->chip);
> +	if (ret < 0)
> +		goto stop_pipeline;
> +	return ret;
> +
> + stop_pipeline:
> +	media_snd_stop_pipeline(subs);
>  	return ret;
>  }
>  
> @@ -830,6 +842,7 @@ static int snd_usb_hw_free(struct snd_pcm_substream *substream)
>  {
>  	struct snd_usb_substream *subs = substream->runtime->private_data;
>  
> +	media_snd_stop_pipeline(subs);
>  	subs->cur_audiofmt = NULL;
>  	subs->cur_rate = 0;
>  	subs->period_bytes = 0;
> @@ -1302,6 +1315,7 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream)
>  	struct snd_usb_stream *as = snd_pcm_substream_chip(substream);
>  	struct snd_pcm_runtime *runtime = substream->runtime;
>  	struct snd_usb_substream *subs = &as->substream[direction];
> +	int ret;
>  
>  	subs->interface = -1;
>  	subs->altset_idx = 0;
> @@ -1315,7 +1329,13 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream)
>  	subs->dsd_dop.channel = 0;
>  	subs->dsd_dop.marker = 1;
>  
> -	return setup_hw_info(runtime, subs);
> +	ret = setup_hw_info(runtime, subs);
> +	if (ret == 0) {
> +		ret = media_snd_stream_init(subs, as->pcm, direction);
> +		if (ret)
> +			snd_usb_autosuspend(subs->stream->chip);
> +	}
> +	return ret;
>  }
>  
>  static int snd_usb_pcm_close(struct snd_pcm_substream *substream)
> @@ -1326,6 +1346,7 @@ static int snd_usb_pcm_close(struct snd_pcm_substream *substream)
>  	int ret;
>  
>  	stop_endpoints(subs, true);
> +	media_snd_stop_pipeline(subs);
>  
>  	if (!as->chip->keep_iface &&
>  	    subs->interface >= 0 &&
> diff --git a/sound/usb/quirks-table.h b/sound/usb/quirks-table.h
> index 08aa78007020..02789d1b2fec 100644
> --- a/sound/usb/quirks-table.h
> +++ b/sound/usb/quirks-table.h
> @@ -2887,6 +2887,7 @@ YAMAHA_DEVICE(0x7010, "UB99"),
>  		.product_name = pname, \
>  		.ifnum = QUIRK_ANY_INTERFACE, \
>  		.type = QUIRK_AUDIO_ALIGN_TRANSFER, \
> +		.media_device = 1, \
>  	} \
>  }
>  
> diff --git a/sound/usb/stream.c b/sound/usb/stream.c
> index 67cf849aa16b..5439dd2e6980 100644
> --- a/sound/usb/stream.c
> +++ b/sound/usb/stream.c
> @@ -38,6 +38,7 @@
>  #include "clock.h"
>  #include "stream.h"
>  #include "power.h"
> +#include "media.h"
>  
>  /*
>   * free a substream
> @@ -55,6 +56,7 @@ static void free_substream(struct snd_usb_substream *subs)
>  	}
>  	kfree(subs->rate_list.list);
>  	kfree(subs->str_pd);
> +	media_snd_stream_delete(subs);
>  }
>  
>  
> diff --git a/sound/usb/usbaudio.h b/sound/usb/usbaudio.h
> index b9faeca645fd..72bdae5ca54a 100644
> --- a/sound/usb/usbaudio.h
> +++ b/sound/usb/usbaudio.h
> @@ -30,6 +30,9 @@
>   *
>   */
>  
> +struct media_device;
> +struct media_intf_devnode;
> +
>  struct snd_usb_audio {
>  	int index;
>  	struct usb_device *dev;
> @@ -66,6 +69,8 @@ struct snd_usb_audio {
>  					 */
>  
>  	struct usb_host_interface *ctrl_intf;	/* the audio control interface */
> +	struct media_device *media_dev;
> +	struct media_intf_devnode *ctl_intf_media_devnode;
>  };
>  
>  #define usb_audio_err(chip, fmt, args...) \
> @@ -117,6 +122,7 @@ struct snd_usb_audio_quirk {
>  	const char *profile_name;	/* override the card->longname */
>  	int16_t ifnum;
>  	uint16_t type;
> +	bool media_device;

I'm not sure this is the right name for the quirk.

The quirk isn't that this audio device needs a media_device, the
quirk is that it *shares* a media device with another subsystem.

So 'shares_media_device' is a better name IMHO.

>  	const void *data;
>  };
>  
> 

Regards,

	Hans

Re: [RFC PATCH v8 1/4] media: Media Device Allocator API

[shuah]

Hi Hans,

On 11/20/18 4:20 AM, Hans Verkuil wrote:
> On 11/02/2018 01:31 AM, shuah@kernel.org wrote:
>> From: Shuah Khan <shuah@kernel.org>
>>
>> Media Device Allocator API to allows multiple drivers share a media device.
>> Using this API, drivers can allocate a media device with the shared struct
>> device as the key. Once the media device is allocated by a driver, other
>> drivers can get a reference to it. The media device is released when all
>> the references are released.
>>
>> Signed-off-by: Shuah Khan <shuah@kernel.org>

Thanks for catching the documentation corrections. Fixed them all and
working on the next revision of the patch.

>> ---
>>   Documentation/media/kapi/mc-core.rst |  37 ++++++++
>>   drivers/media/Makefile               |   3 +-
>>   drivers/media/media-dev-allocator.c  | 132 +++++++++++++++++++++++++++
>>   include/media/media-dev-allocator.h  |  53 +++++++++++
>>   4 files changed, 224 insertions(+), 1 deletion(-)
>>   create mode 100644 drivers/media/media-dev-allocator.c
>>   create mode 100644 include/media/media-dev-allocator.h
>>
>> diff --git a/Documentation/media/kapi/mc-core.rst b/Documentation/media/kapi/mc-core.rst
>> index 0c05503eaf1f..d6f409598065 100644
>> --- a/Documentation/media/kapi/mc-core.rst
>> +++ b/Documentation/media/kapi/mc-core.rst
>> @@ -257,8 +257,45 @@ Subsystems should facilitate link validation by providing subsystem specific
>>   helper functions to provide easy access for commonly needed information, and
>>   in the end provide a way to use driver-specific callbacks.
>>   
>> +Media Controller Device Allocator API
>> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> +
>> +When media device belongs to more than one driver, the shared media device
> 
> When -> When the
> 
>> +is allocated with the shared struct device as the key for look ups.
>> +
>> +Shared media device should stay in registered state until the last driver
> 
> Shared -> The shared
> 
>> +unregisters it. In addition, media device should be released when all the
> 
> media -> the media
> 
>> +references are released. Each driver gets a reference to the media device
>> +during probe, when it allocates the media device. If media device is already
>> +allocated, allocate API bumps up the refcount and return the existing media
> 
> allocate -> the allocate
> return -> returns
> 
>> +device. Driver puts the reference back from its disconnect routine when it
> 
> Driver -> The driver
> from -> in
> 
>> +calls :c:func:`media_device_delete()`.
>> +
>> +Media device is unregistered and cleaned up from the kref put handler to
> 
> Media -> The media
> from -> in
> 
>> +ensure that the media device stays in registered state until the last driver
>> +unregisters the media device.
> 
> What happens if an application still has the media device open and you forcibly
> remove the last driver? I think it should be OK since the media_devnode struct
> isn't freed until the last open filehandle closes. But it is good to test this.
> 
>> +
>> +**Driver Usage**
>> +
>> +Drivers should use the media-core routines to get register reference and
> 
> 'get register reference'? Not sure what you meant to say.
> 
>> +call :c:func:`media_device_delete()` routine to make sure the shared media
>> +device delete is handled correctly.
>> +
>> +**driver probe:**
>> +Call :c:func:`media_device_usb_allocate()` to allocate or get a reference
>> +Call :c:func:`media_device_register()`, if media devnode isn't registered
>> +
>> +**driver disconnect:**
>> +Call :c:func:`media_device_delete()` to free the media_device. Free'ing is
> 
> Free'ing -> Freeing
> 
>> +handled by the kref put handler.
>> +
>> +API Definitions
>> +^^^^^^^^^^^^^^^
>> +
>>   .. kernel-doc:: include/media/media-device.h
>>   
>>   .. kernel-doc:: include/media/media-devnode.h
>>   
>>   .. kernel-doc:: include/media/media-entity.h
>> +
>> +.. kernel-doc:: include/media/media-dev-allocator.h
>> diff --git a/drivers/media/Makefile b/drivers/media/Makefile
>> index 594b462ddf0e..8608f0a41dca 100644
>> --- a/drivers/media/Makefile
>> +++ b/drivers/media/Makefile
>> @@ -3,7 +3,8 @@
>>   # Makefile for the kernel multimedia device drivers.
>>   #
>>   
>> -media-objs	:= media-device.o media-devnode.o media-entity.o
>> +media-objs	:= media-device.o media-devnode.o media-entity.o \
>> +		   media-dev-allocator.o
> 
> Perhaps only add media-dev-allocator if CONFIG_USB is enabled?
> 
>>   
>>   #
>>   # I2C drivers should come before other drivers, otherwise they'll fail
>> diff --git a/drivers/media/media-dev-allocator.c b/drivers/media/media-dev-allocator.c
>> new file mode 100644
>> index 000000000000..262d1293dc13
>> --- /dev/null
>> +++ b/drivers/media/media-dev-allocator.c
>> @@ -0,0 +1,132 @@
>> +// SPDX-License-Identifier: GPL-2.0-or-later
>> +/*
>> + * media-dev-allocator.c - Media Controller Device Allocator API
>> + *
>> + * Copyright (c) 2018 Shuah Khan <shuah@kernel.org>
>> + *
>> + * Credits: Suggested by Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>> + */
>> +
>> +/*
>> + * This file adds a global refcounted Media Controller Device Instance API.
>> + * A system wide global media device list is managed and each media device
>> + * includes a kref count. The last put on the media device releases the media
>> + * device instance.
>> + *
>> + */
>> +
>> +#include <linux/kref.h>
>> +#include <linux/module.h>
>> +#include <linux/slab.h>
>> +#include <linux/usb.h>
>> +
>> +#include <media/media-device.h>
>> +
>> +static LIST_HEAD(media_device_list);
>> +static DEFINE_MUTEX(media_device_lock);
>> +
>> +struct media_device_instance {
>> +	struct media_device mdev;
>> +	struct module *owner;
>> +	struct list_head list;
>> +	struct kref refcount;
>> +};
>> +
>> +static inline struct media_device_instance *
>> +to_media_device_instance(struct media_device *mdev)
>> +{
>> +	return container_of(mdev, struct media_device_instance, mdev);
>> +}
>> +
>> +static void media_device_instance_release(struct kref *kref)
>> +{
>> +	struct media_device_instance *mdi =
>> +		container_of(kref, struct media_device_instance, refcount);
>> +
>> +	dev_dbg(mdi->mdev.dev, "%s: mdev=%p\n", __func__, &mdi->mdev);
>> +
>> +	mutex_lock(&media_device_lock);
> 
> Can't the lock be moved down to just before list_del? Or am I missing something?
> 

The lock needs to be held while media_device_unregister() and 
media_device_cleanup() to avoid races in the unregister path if two 
drivers call media_device_delete().

>> +
>> +	media_device_unregister(&mdi->mdev);
>> +	media_device_cleanup(&mdi->mdev);
>> +
>> +	list_del(&mdi->list);
>> +	mutex_unlock(&media_device_lock);
>> +
>> +	kfree(mdi);
>> +}
>> +
>> +/* Callers should hold media_device_lock when calling this function */
>> +static struct media_device *__media_device_get(struct device *dev,
>> +					       char *module_name)
> 
> const char *

Thanks fixed all of these.

I am running final tests and will send the next version this week.

thanks,
-- Shuah

Re: [RFC PATCH v8 3/4] media: media.h: Enable ALSA MEDIA_INTF_T* interface types

[shuah]

Hi Hans,

On 11/20/18 4:22 AM, Hans Verkuil wrote:
> On 11/02/2018 01:31 AM, shuah@kernel.org wrote:
>> From: Shuah Khan <shuah@kernel.org>
>>
>> Move ALSA MEDIA_INTF_T* interface types back into __KERNEL__ scope
>> to get ready for adding ALSA support to the media controller.
>>
>> Signed-off-by: Shuah Khan <shuah@kernel.org>
>> ---
>>   include/uapi/linux/media.h | 25 ++++++++++---------------
>>   1 file changed, 10 insertions(+), 15 deletions(-)
>>
>> diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
>> index 36f76e777ef9..07be07263597 100644
>> --- a/include/uapi/linux/media.h
>> +++ b/include/uapi/linux/media.h
>> @@ -262,6 +262,16 @@ struct media_links_enum {
>>   #define MEDIA_INTF_T_V4L_SWRADIO		(MEDIA_INTF_T_V4L_BASE + 4)
>>   #define MEDIA_INTF_T_V4L_TOUCH			(MEDIA_INTF_T_V4L_BASE + 5)
>>   
>> +#define MEDIA_INTF_T_ALSA_BASE			0x00000300
>> +#define MEDIA_INTF_T_ALSA_PCM_CAPTURE		(MEDIA_INTF_T_ALSA_BASE)
>> +#define MEDIA_INTF_T_ALSA_PCM_PLAYBACK		(MEDIA_INTF_T_ALSA_BASE + 1)
>> +#define MEDIA_INTF_T_ALSA_CONTROL		(MEDIA_INTF_T_ALSA_BASE + 2)
>> +#define MEDIA_INTF_T_ALSA_COMPRESS		(MEDIA_INTF_T_ALSA_BASE + 3)
>> +#define MEDIA_INTF_T_ALSA_RAWMIDI		(MEDIA_INTF_T_ALSA_BASE + 4)
>> +#define MEDIA_INTF_T_ALSA_HWDEP			(MEDIA_INTF_T_ALSA_BASE + 5)
>> +#define MEDIA_INTF_T_ALSA_SEQUENCER		(MEDIA_INTF_T_ALSA_BASE + 6)
>> +#define MEDIA_INTF_T_ALSA_TIMER			(MEDIA_INTF_T_ALSA_BASE + 7)
>> +
> 
> I would only enable those defines that you need for the next patch.
> 

Good plan. I fixed this to move just the ones I need for this work.

thanks,
-- Shuah

Re: [RFC PATCH v8 1/4] media: Media Device Allocator API

[shuah]

On 11/19/18 1:59 AM, Pavel Machek wrote:
> On Thu 2018-11-01 18:31:30, shuah@kernel.org wrote:
>> From: Shuah Khan <shuah@kernel.org>
>>
>> Media Device Allocator API to allows multiple drivers share a media device.
>> Using this API, drivers can allocate a media device with the shared struct
>> device as the key. Once the media device is allocated by a driver, other
>> drivers can get a reference to it. The media device is released when all
>> the references are released.
> 
> Sounds like a ... bad idea?
> 
> That's what new "media control" framework is for, no?
> 
> Why do you need this?
> 								Pavel
> 

Media control framework doesn't address this problem of ownership of the 
media device when non-media drivers have to own the pipeline. In this 
case, snd-usb owns the audio pipeline when an audio application is using 
the device. Without this work, media drivers won't be able to tell if 
snd-usb is using the tuner and owns the media pipeline.

I am going to clarify this in the commit log.

thanks,
-- Shuah

Re: [RFC PATCH v8 4/4] sound/usb: Use Media Controller API to share media resources

[shuah]

Hi Hans,

On 11/20/18 4:54 AM, Hans Verkuil wrote:
> On 11/02/2018 01:31 AM, shuah@kernel.org wrote:
>> From: Shuah Khan <shuah@kernel.org>
>>
>> Change ALSA driver to use Media Controller API to share media resources
>> with DVB, and V4L2 drivers on a AU0828 media device.
>>
>> Media Controller specific initialization is done after sound card is
>> registered. ALSA creates Media interface and entity function graph
>> nodes for Control, Mixer, PCM Playback, and PCM Capture devices.
>>
>> snd_usb_hw_params() will call Media Controller enable source handler
>> interface to request the media resource. If resource request is granted,
>> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
>> returned.
>>
>> Media specific cleanup is done in usb_audio_disconnect().
>>
>> Signed-off-by: Shuah Khan <shuah@kernel.org>

Thanks for the review. Fixing them all in the next revision.

-- Shuah

Re: [RFC PATCH v8 1/4] media: Media Device Allocator API

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1431 bytes --]

On Thu 2018-12-06 08:33:14, shuah wrote:
> On 11/19/18 1:59 AM, Pavel Machek wrote:
> >On Thu 2018-11-01 18:31:30, shuah@kernel.org wrote:
> >>From: Shuah Khan <shuah@kernel.org>
> >>
> >>Media Device Allocator API to allows multiple drivers share a media device.
> >>Using this API, drivers can allocate a media device with the shared struct
> >>device as the key. Once the media device is allocated by a driver, other
> >>drivers can get a reference to it. The media device is released when all
> >>the references are released.
> >
> >Sounds like a ... bad idea?
> >
> >That's what new "media control" framework is for, no?
> >
> >Why do you need this?
> 
> Media control framework doesn't address this problem of ownership of the
> media device when non-media drivers have to own the pipeline. In this case,
> snd-usb owns the audio pipeline when an audio application is using the
> device. Without this work, media drivers won't be able to tell if snd-usb is
> using the tuner and owns the media pipeline.
> 
> I am going to clarify this in the commit log.

I guess I'll need the explanation, yes.

How can usb soundcard use the tuner? I thought we'd always have
userspace component active and moving data between tuner and usb sound
card?

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [RFC PATCH v8 1/4] media: Media Device Allocator API

[Mauro Carvalho Chehab]

Em Sun, 9 Dec 2018 09:09:44 +0100
Pavel Machek <pavel@ucw.cz> escreveu:

> On Thu 2018-12-06 08:33:14, shuah wrote:
> > On 11/19/18 1:59 AM, Pavel Machek wrote:  
> > >On Thu 2018-11-01 18:31:30, shuah@kernel.org wrote:  
> > >>From: Shuah Khan <shuah@kernel.org>
> > >>
> > >>Media Device Allocator API to allows multiple drivers share a media device.
> > >>Using this API, drivers can allocate a media device with the shared struct
> > >>device as the key. Once the media device is allocated by a driver, other
> > >>drivers can get a reference to it. The media device is released when all
> > >>the references are released.  
> > >
> > >Sounds like a ... bad idea?
> > >
> > >That's what new "media control" framework is for, no?
> > >
> > >Why do you need this?  
> > 
> > Media control framework doesn't address this problem of ownership of the
> > media device when non-media drivers have to own the pipeline. In this case,
> > snd-usb owns the audio pipeline when an audio application is using the
> > device. Without this work, media drivers won't be able to tell if snd-usb is
> > using the tuner and owns the media pipeline.
> > 
> > I am going to clarify this in the commit log.  
> 
> I guess I'll need the explanation, yes.
> 
> How can usb soundcard use the tuner? I thought we'd always have
> userspace component active and moving data between tuner and usb sound
> card?

It sounds that the description of the patch is not 100%, as it seems
that you're not seeing the hole picture.

This is designed to solve a very common usecase for media devices
where one physical device (an USB stick) provides both audio
and video.

That's, for example, the case of cameras with microphones and 
TV USB devices. Those usually expose the audio via standard
USB Audio Class, and video either via USB Video Class or via
some proprietary vendor class.

Due to the way USB Audio Class is handled, it means that two
independent drivers will provide the pipelines for a single
physical USB bridge.

The same problem also applies to more sophisticated embedded devices,
like on  SOCs designed to be used on TVs and Set Top Boxes, where the
hardware pipeline has both audio and video components on it, logically
mapped into different drivers (using Linux DTV API, V4L2 API and ALSA).

On such kind of devices, it is important to have a way to see
and control the entire audio and video pipeline present on them 
through a single media controller device, specially if one wants
to provide a hardware pipeline within the SoC that won't be 
copying data between Kernel-userspace.

Now, if the audio is implemented on a separate device (like an
Intel HDA compatible chipset at the motherboard), it should
be exposed as a separate media controller.

So, for example, a system that has both an USB audio/video
stick and an Intel HDA-compatible chipset, both exposed via
the media controller, will have two media controller devices,
one for each physically independent device.

On the other hand, an SoC designed for TV products will likely
expose a single media controller, even if each part of the
pipeline is exposed via independent Linux device drivers.

Thanks,
Mauro

Re: [RFC PATCH v8 1/4] media: Media Device Allocator API

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1877 bytes --]

Hi!

> > On Thu 2018-12-06 08:33:14, shuah wrote:
> > > On 11/19/18 1:59 AM, Pavel Machek wrote:  
> > > >On Thu 2018-11-01 18:31:30, shuah@kernel.org wrote:  
> > > >>From: Shuah Khan <shuah@kernel.org>
> > > >>
> > > >>Media Device Allocator API to allows multiple drivers share a media device.
> > > >>Using this API, drivers can allocate a media device with the shared struct
> > > >>device as the key. Once the media device is allocated by a driver, other
> > > >>drivers can get a reference to it. The media device is released when all
> > > >>the references are released.  
> > > >
> > > >Sounds like a ... bad idea?
> > > >
> > > >That's what new "media control" framework is for, no?
> > > >
> > > >Why do you need this?  
> > > 
> > > Media control framework doesn't address this problem of ownership of the
> > > media device when non-media drivers have to own the pipeline. In this case,
> > > snd-usb owns the audio pipeline when an audio application is using the
> > > device. Without this work, media drivers won't be able to tell if snd-usb is
> > > using the tuner and owns the media pipeline.
> > > 
> > > I am going to clarify this in the commit log.  
> > 
> > I guess I'll need the explanation, yes.
> > 
> > How can usb soundcard use the tuner? I thought we'd always have
> > userspace component active and moving data between tuner and usb sound
> > card?
> 
> It sounds that the description of the patch is not 100%, as it seems
> that you're not seeing the hole picture.
> 
> This is designed to solve a very common usecase for media devices
> where one physical device (an USB stick) provides both audio
> and video.

Aha, ok, it makes sense now. Thanks!
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]