[PATCH v6 0/3] Media Device Allocator API

[PATCH v6 0/3] Media Device Allocator API

[Shuah Khan]

Media Device Allocator API to allows multiple drivers share a media device.
Using this API, drivers can allocate a media device with the shared struct
device as the key. Once the media device is allocated by a driver, other
drivers can get a reference to it. The media device is released when all
the references are released.

Patches 0001 and 0002 are rebased to 4.9-rc7. Patch 0003 for snd-usb-audio
is a rebase of the patch that was tested with the original Media Device
Allocator patch series.

snd-usb-audio patch includes the fixes found during 4.7-rc1 time in the
original snd-usb-audio patch.

Changes to patches in this series:
Changes to patch 0001 since v5: (comments from Mauro and Sakari)
- Removed struct device from media_device_instance. mdev.dev is used instead.
- Added documentation.

Changes to patch 0002:
- No changes since patch v2, applies cleanly on top of the following:
media: Protect enable_source and disable_source handler code paths
https://lkml.org/lkml/2016/11/29/1001
 
Changes to patch 0003:
- Changed to hold graph_mutex to check and call enable_source and
  disable_source handlers - to match au0828 doing the same in:
media: Protect enable_source and disable_source handler code paths 
https://lkml.org/lkml/2016/11/29/1001

Changes to patch 0001 since v4:
- Addressed Sakari's review comments with the exception of
  opting to not introduce media_device_usb_allocate() macro,
  and to not add a new routine to find media device instance
  to avoid a one line check.

Changes to patch 0001 since v3:
- Fixed undefined reference to `__media_device_usb_init compile error when
  CONFIG_USB is disabled.
- Fixed kernel paging error when accessing /dev/mediaX after rmmod of the
  module that owns the media_device. The fix bumps the reference count for
  the owner when second driver comes along to share the media_device. If
  au0828 owns the media_device, then snd_usb_audio will bump the refcount
  for au0828, so it won't get deleted and vice versa.

Changes to patch 0002 since v2:
- Updated media_device_delete() to pass in module name.

Changes to patch 0003 since the last version in 4.7-rc1:
- Included fixes to bugs found during testing. 
- Updated to use the Media Allocator API.

This patch series has been tested with au0828 and snd-usb-audio drivers.
Ran bind and unbind loop tests on each driver with mc_nextgen_test and
media_device_test app loop tests while checking lsmod and dmesg.

Please refer to tools/testing/selftests/media_tests/regression_test.txt
for testing done on this series.

Shuah Khan (3):
  media: Media Device Allocator API
  media: change au0828 to use Media Device Allocator API
  sound/usb: Use Media Controller API to share media resources

 Documentation/media/kapi/mc-core.rst   |  37 ++++
 drivers/media/Makefile                 |   3 +-
 drivers/media/media-dev-allocator.c    | 133 ++++++++++++++
 drivers/media/usb/au0828/au0828-core.c |  12 +-
 drivers/media/usb/au0828/au0828.h      |   1 +
 include/media/media-dev-allocator.h    |  54 ++++++
 sound/usb/Kconfig                      |   4 +
 sound/usb/Makefile                     |   2 +
 sound/usb/card.c                       |  14 ++
 sound/usb/card.h                       |   3 +
 sound/usb/media.c                      | 326 +++++++++++++++++++++++++++++++++
 sound/usb/media.h                      |  73 ++++++++
 sound/usb/mixer.h                      |   3 +
 sound/usb/pcm.c                        |  28 ++-
 sound/usb/quirks-table.h               |   1 +
 sound/usb/stream.c                     |   2 +
 sound/usb/usbaudio.h                   |   6 +
 17 files changed, 688 insertions(+), 14 deletions(-)
 create mode 100644 drivers/media/media-dev-allocator.c
 create mode 100644 include/media/media-dev-allocator.h
 create mode 100644 sound/usb/media.c
 create mode 100644 sound/usb/media.h

-- 
2.7.4

[PATCH v6 1/3] media: Media Device Allocator API

[Shuah Khan]

Media Device Allocator API to allows multiple drivers share a media device.
Using this API, drivers can allocate a media device with the shared struct
device as the key. Once the media device is allocated by a driver, other
drivers can get a reference to it. The media device is released when all
the references are released.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
Reviewed-by: Hans Verkuil <hans.verkuil@cisco.com>
---
Changes to patch 0001 since v5: (comments from Mauro and Sakari)
- Removed struct device from media_device_instance. mdev.dev is used instead.
- Added documentation.

 Documentation/media/kapi/mc-core.rst |  37 ++++++++++
 drivers/media/Makefile               |   3 +-
 drivers/media/media-dev-allocator.c  | 133 +++++++++++++++++++++++++++++++++++
 include/media/media-dev-allocator.h  |  54 ++++++++++++++
 4 files changed, 226 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/media-dev-allocator.c
 create mode 100644 include/media/media-dev-allocator.h

diff --git a/Documentation/media/kapi/mc-core.rst b/Documentation/media/kapi/mc-core.rst
index 1a738e5..4942cdd 100644
--- a/Documentation/media/kapi/mc-core.rst
+++ b/Documentation/media/kapi/mc-core.rst
@@ -257,8 +257,45 @@ Subsystems should facilitate link validation by providing subsystem specific
 helper functions to provide easy access for commonly needed information, and
 in the end provide a way to use driver-specific callbacks.
 
+Media Controller Device Allocator API
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When media device belongs to more than one driver, the shared media device
+is allocated with the shared struct device as the key for look ups.
+
+Shared media device should stay in registered state until the last driver
+unregisters it. In addition, media device should be released when all the
+references are released. Each driver gets a reference to the media device
+during probe, when it allocates the media device. If media device is already
+allocated, allocate API bumps up the refcount and return the existing media
+device. Driver puts the reference back from its disconnect routine when it
+calls :c:func:`media_device_delete()`.
+
+Media device is unregistered and cleaned up from the kref put handler to
+ensure that the media device stays in registered state until the last driver
+unregisters the media device.
+
+**Driver Usage**
+
+Drivers should use the media-core routines to get register reference and
+call :c:func:`media_device_delete()` routine to make sure the shared media
+device delete is handled correctly.
+
+**driver probe:**
+Call :c:func:`media_device_usb_allocate()` to allocate or get a reference
+Call :c:func:`media_device_register()`, if media devnode isn't registered
+
+**driver disconnect:**
+Call :c:func:`media_device_delete()` to free the media_device. Free'ing is
+handled by the kref put handler.
+
+API Definitions
+^^^^^^^^^^^^^^^
+
 .. kernel-doc:: include/media/media-device.h
 
 .. kernel-doc:: include/media/media-devnode.h
 
 .. kernel-doc:: include/media/media-entity.h
+
+.. kernel-doc:: include/media/media-dev-allocator.h
diff --git a/drivers/media/Makefile b/drivers/media/Makefile
index 0deaa93..7c0701d 100644
--- a/drivers/media/Makefile
+++ b/drivers/media/Makefile
@@ -6,7 +6,8 @@ ifeq ($(CONFIG_MEDIA_CEC_EDID),y)
   obj-$(CONFIG_MEDIA_SUPPORT) += cec-edid.o
 endif
 
-media-objs	:= media-device.o media-devnode.o media-entity.o
+media-objs	:= media-device.o media-devnode.o media-entity.o \
+		   media-dev-allocator.o
 
 #
 # I2C drivers should come before other drivers, otherwise they'll fail
diff --git a/drivers/media/media-dev-allocator.c b/drivers/media/media-dev-allocator.c
new file mode 100644
index 0000000..0ecf152
--- /dev/null
+++ b/drivers/media/media-dev-allocator.c
@@ -0,0 +1,133 @@
+/*
+ * media-dev-allocator.c - Media Controller Device Allocator API
+ *
+ * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
+ * Copyright (c) 2016 Samsung Electronics Co., Ltd.
+ *
+ * This file is released under the GPLv2.
+ * Credits: Suggested by Laurent Pinchart <laurent.pinchart@ideasonboard.com>
+ */
+
+/*
+ * This file adds a global refcounted Media Controller Device Instance API.
+ * A system wide global media device list is managed and each media device
+ * includes a kref count. The last put on the media device releases the media
+ * device instance.
+ *
+ */
+
+#include <linux/kref.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/usb.h>
+
+#include <media/media-device.h>
+
+static LIST_HEAD(media_device_list);
+static DEFINE_MUTEX(media_device_lock);
+
+struct media_device_instance {
+	struct media_device mdev;
+	struct module *owner;
+	struct list_head list;
+	struct kref refcount;
+};
+
+static inline struct media_device_instance *
+to_media_device_instance(struct media_device *mdev)
+{
+	return container_of(mdev, struct media_device_instance, mdev);
+}
+
+static void media_device_instance_release(struct kref *kref)
+{
+	struct media_device_instance *mdi =
+		container_of(kref, struct media_device_instance, refcount);
+
+	dev_dbg(mdi->mdev.dev, "%s: mdev=%p\n", __func__, &mdi->mdev);
+
+	mutex_lock(&media_device_lock);
+
+	media_device_unregister(&mdi->mdev);
+	media_device_cleanup(&mdi->mdev);
+
+	list_del(&mdi->list);
+	mutex_unlock(&media_device_lock);
+
+	kfree(mdi);
+}
+
+/* Callers should hold media_device_lock when calling this function */
+static struct media_device *__media_device_get(struct device *dev,
+					       char *module_name)
+{
+	struct media_device_instance *mdi;
+
+	list_for_each_entry(mdi, &media_device_list, list) {
+		if (mdi->mdev.dev != dev)
+			continue;
+
+		kref_get(&mdi->refcount);
+		/* get module reference for the media_device owner */
+		if (find_module(module_name) != mdi->owner &&
+		    !try_module_get(mdi->owner))
+			dev_err(dev, "%s: try_module_get() error\n", __func__);
+		dev_dbg(dev, "%s: get mdev=%p module_name %s\n",
+			__func__, &mdi->mdev, module_name);
+		return &mdi->mdev;
+	}
+
+	mdi = kzalloc(sizeof(*mdi), GFP_KERNEL);
+	if (!mdi)
+		return NULL;
+
+	mdi->owner = find_module(module_name);
+	kref_init(&mdi->refcount);
+	list_add_tail(&mdi->list, &media_device_list);
+
+	dev_dbg(dev, "%s: alloc mdev=%p module_name %s\n", __func__,
+		&mdi->mdev, module_name);
+	return &mdi->mdev;
+}
+
+#if IS_ENABLED(CONFIG_USB)
+struct media_device *media_device_usb_allocate(struct usb_device *udev,
+					       char *module_name)
+{
+	struct media_device *mdev;
+
+	mutex_lock(&media_device_lock);
+	mdev = __media_device_get(&udev->dev, module_name);
+	if (!mdev) {
+		mutex_unlock(&media_device_lock);
+		return ERR_PTR(-ENOMEM);
+	}
+
+	/* check if media device is already initialized */
+	if (!mdev->dev)
+		__media_device_usb_init(mdev, udev, udev->product,
+					module_name);
+	mutex_unlock(&media_device_lock);
+	return mdev;
+}
+EXPORT_SYMBOL_GPL(media_device_usb_allocate);
+#endif
+
+void media_device_delete(struct media_device *mdev, char *module_name)
+{
+	struct media_device_instance *mdi = to_media_device_instance(mdev);
+
+	dev_dbg(mdi->mdev.dev, "%s: mdev=%p module_name %s\n",
+		__func__, &mdi->mdev, module_name);
+
+	mutex_lock(&media_device_lock);
+	/* put module reference if media_device owner is not THIS_MODULE */
+	if (mdi->owner != find_module(module_name)) {
+		module_put(mdi->owner);
+		dev_dbg(mdi->mdev.dev,
+			"%s decremented owner module reference\n", __func__);
+	}
+	mutex_unlock(&media_device_lock);
+	kref_put(&mdi->refcount, media_device_instance_release);
+}
+EXPORT_SYMBOL_GPL(media_device_delete);
diff --git a/include/media/media-dev-allocator.h b/include/media/media-dev-allocator.h
new file mode 100644
index 0000000..6731d825
--- /dev/null
+++ b/include/media/media-dev-allocator.h
@@ -0,0 +1,54 @@
+/*
+ * media-dev-allocator.h - Media Controller Device Allocator API
+ *
+ * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
+ * Copyright (c) 2016 Samsung Electronics Co., Ltd.
+ *
+ * This file is released under the GPLv2.
+ * Credits: Suggested by Laurent Pinchart <laurent.pinchart@ideasonboard.com>
+ */
+
+/*
+ * This file adds a global ref-counted Media Controller Device Instance API.
+ * A system wide global media device list is managed and each media device
+ * includes a kref count. The last put on the media device releases the media
+ * device instance.
+ */
+
+#ifndef _MEDIA_DEV_ALLOCTOR_H
+#define _MEDIA_DEV_ALLOCTOR_H
+
+struct usb_device;
+
+#ifdef CONFIG_MEDIA_CONTROLLER
+/**
+ * media_device_usb_allocate() - Allocate and return struct &media device
+ *
+ * @udev:		struct &usb_device pointer
+ * @module_name:	should be filled with %KBUILD_MODNAME
+ *
+ * This interface should be called to allocate a Media Device when multiple
+ * drivers share usb_device and the media device. This interface allocates
+ * &media_device structure and calls media_device_usb_init() to initialize
+ * it.
+ *
+ */
+struct media_device *media_device_usb_allocate(struct usb_device *udev,
+					       char *module_name);
+/**
+ * media_device_delete() - Release media device. Calls kref_put().
+ *
+ * @mdev:		struct &media_device pointer
+ * @module_name:	should be filled with %KBUILD_MODNAME
+ *
+ * This interface should be called to put Media Device Instance kref.
+ */
+void media_device_delete(struct media_device *mdev, char *module_name);
+#else
+static inline struct media_device *media_device_usb_allocate(
+			struct usb_device *udev, char *module_name)
+			{ return NULL; }
+static inline void media_device_delete(
+			struct media_device *mdev, char *module_name) { }
+#endif /* CONFIG_MEDIA_CONTROLLER */
+#endif
-- 
2.7.4

[PATCH v6 2/3] media: change au0828 to use Media Device Allocator API

[Shuah Khan]

Change au0828 to use Media Device Allocator API to allocate media device
with the parent usb struct device as the key, so it can be shared with the
snd_usb_audio driver.

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
Changes to patch 0002:
- No changes since patch v2, applies cleanly on top of the following:
media: Protect enable_source and disable_source handler code paths
https://lkml.org/lkml/2016/11/29/1001

 drivers/media/usb/au0828/au0828-core.c | 12 ++++--------
 drivers/media/usb/au0828/au0828.h      |  1 +
 2 files changed, 5 insertions(+), 8 deletions(-)

diff --git a/drivers/media/usb/au0828/au0828-core.c b/drivers/media/usb/au0828/au0828-core.c
index bfd6482..ff1928e 100644
--- a/drivers/media/usb/au0828/au0828-core.c
+++ b/drivers/media/usb/au0828/au0828-core.c
@@ -159,9 +159,7 @@ static void au0828_unregister_media_device(struct au0828_dev *dev)
 	dev->media_dev->disable_source = NULL;
 	mutex_unlock(&mdev->graph_mutex);
 
-	media_device_unregister(dev->media_dev);
-	media_device_cleanup(dev->media_dev);
-	kfree(dev->media_dev);
+	media_device_delete(dev->media_dev, KBUILD_MODNAME);
 	dev->media_dev = NULL;
 #endif
 }
@@ -214,14 +212,10 @@ static int au0828_media_device_init(struct au0828_dev *dev,
 #ifdef CONFIG_MEDIA_CONTROLLER
 	struct media_device *mdev;
 
-	mdev = kzalloc(sizeof(*mdev), GFP_KERNEL);
+	mdev = media_device_usb_allocate(udev, KBUILD_MODNAME);
 	if (!mdev)
 		return -ENOMEM;
 
-	/* check if media device is already initialized */
-	if (!mdev->dev)
-		media_device_usb_init(mdev, udev, udev->product);
-
 	dev->media_dev = mdev;
 #endif
 	return 0;
@@ -482,6 +476,8 @@ static int au0828_media_device_register(struct au0828_dev *dev,
 		/* register media device */
 		ret = media_device_register(dev->media_dev);
 		if (ret) {
+			media_device_delete(dev->media_dev, KBUILD_MODNAME);
+			dev->media_dev = NULL;
 			dev_err(&udev->dev,
 				"Media Device Register Error: %d\n", ret);
 			return ret;
diff --git a/drivers/media/usb/au0828/au0828.h b/drivers/media/usb/au0828/au0828.h
index dd7b378..4bf1b0c 100644
--- a/drivers/media/usb/au0828/au0828.h
+++ b/drivers/media/usb/au0828/au0828.h
@@ -35,6 +35,7 @@
 #include <media/v4l2-ctrls.h>
 #include <media/v4l2-fh.h>
 #include <media/media-device.h>
+#include <media/media-dev-allocator.h>
 
 /* DVB */
 #include "demux.h"
-- 
2.7.4

[PATCH v6 3/3] sound/usb: Use Media Controller API to share media resources

[Shuah Khan]

Change ALSA driver to use Media Controller API to share media resources
with DVB, and V4L2 drivers on a AU0828 media device.

Media Controller specific initialization is done after sound card is
registered. ALSA creates Media interface and entity function graph
nodes for Control, Mixer, PCM Playback, and PCM Capture devices.

snd_usb_hw_params() will call Media Controller enable source handler
interface to request the media resource. If resource request is granted,
it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
returned.

Media specific cleanup is done in usb_audio_disconnect().

Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
---
Changes to patch 0003:
- Changed to hold graph_mutex to check and call enable_source and
  disable_source handlers - to match au0828 doing the same in:
media: Protect enable_source and disable_source handler code paths 
https://lkml.org/lkml/2016/11/29/1001

 sound/usb/Kconfig        |   4 +
 sound/usb/Makefile       |   2 +
 sound/usb/card.c         |  14 ++
 sound/usb/card.h         |   3 +
 sound/usb/media.c        | 326 +++++++++++++++++++++++++++++++++++++++++++++++
 sound/usb/media.h        |  73 +++++++++++
 sound/usb/mixer.h        |   3 +
 sound/usb/pcm.c          |  28 +++-
 sound/usb/quirks-table.h |   1 +
 sound/usb/stream.c       |   2 +
 sound/usb/usbaudio.h     |   6 +
 11 files changed, 457 insertions(+), 5 deletions(-)
 create mode 100644 sound/usb/media.c
 create mode 100644 sound/usb/media.h

diff --git a/sound/usb/Kconfig b/sound/usb/Kconfig
index a452ad7..d14bf41 100644
--- a/sound/usb/Kconfig
+++ b/sound/usb/Kconfig
@@ -15,6 +15,7 @@ config SND_USB_AUDIO
 	select SND_RAWMIDI
 	select SND_PCM
 	select BITREVERSE
+	select SND_USB_AUDIO_USE_MEDIA_CONTROLLER if MEDIA_CONTROLLER && (MEDIA_SUPPORT=y || MEDIA_SUPPORT=SND_USB_AUDIO)
 	help
 	  Say Y here to include support for USB audio and USB MIDI
 	  devices.
@@ -22,6 +23,9 @@ config SND_USB_AUDIO
 	  To compile this driver as a module, choose M here: the module
 	  will be called snd-usb-audio.
 
+config SND_USB_AUDIO_USE_MEDIA_CONTROLLER
+	bool
+
 config SND_USB_UA101
 	tristate "Edirol UA-101/UA-1000 driver"
 	select SND_PCM
diff --git a/sound/usb/Makefile b/sound/usb/Makefile
index 2d2d122..8dca3c4 100644
--- a/sound/usb/Makefile
+++ b/sound/usb/Makefile
@@ -15,6 +15,8 @@ snd-usb-audio-objs := 	card.o \
 			quirks.o \
 			stream.o
 
+snd-usb-audio-$(CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER) += media.o
+
 snd-usbmidi-lib-objs := midi.o
 
 # Toplevel Module Dependency
diff --git a/sound/usb/card.c b/sound/usb/card.c
index 2ddc034..570ff00 100644
--- a/sound/usb/card.c
+++ b/sound/usb/card.c
@@ -66,6 +66,7 @@
 #include "format.h"
 #include "power.h"
 #include "stream.h"
+#include "media.h"
 
 MODULE_AUTHOR("Takashi Iwai <tiwai@suse.de>");
 MODULE_DESCRIPTION("USB Audio");
@@ -616,6 +617,11 @@ static int usb_audio_probe(struct usb_interface *intf,
 	if (err < 0)
 		goto __error;
 
+	if (quirk && quirk->media_device) {
+		/* don't want to fail when media_snd_device_create() fails */
+		media_snd_device_create(chip, intf);
+	}
+
 	usb_chip[chip->index] = chip;
 	chip->num_interfaces++;
 	usb_set_intfdata(intf, chip);
@@ -672,6 +678,14 @@ static void usb_audio_disconnect(struct usb_interface *intf)
 		list_for_each(p, &chip->midi_list) {
 			snd_usbmidi_disconnect(p);
 		}
+		/*
+		 * Nice to check quirk && quirk->media_device and
+		 * then call media_snd_device_delete(). Don't have
+		 * access to quirk here. media_snd_device_delete()
+		 * acceses mixer_list
+		 */
+		media_snd_device_delete(chip);
+
 		/* release mixer resources */
 		list_for_each_entry(mixer, &chip->mixer_list, list) {
 			snd_usb_mixer_disconnect(mixer);
diff --git a/sound/usb/card.h b/sound/usb/card.h
index 111b0f0..6ef8e88 100644
--- a/sound/usb/card.h
+++ b/sound/usb/card.h
@@ -105,6 +105,8 @@ struct snd_usb_endpoint {
 	struct list_head list;
 };
 
+struct media_ctl;
+
 struct snd_usb_substream {
 	struct snd_usb_stream *stream;
 	struct usb_device *dev;
@@ -156,6 +158,7 @@ struct snd_usb_substream {
 	} dsd_dop;
 
 	bool trigger_tstamp_pending_update; /* trigger timestamp being updated from initial estimate */
+	struct media_ctl *media_ctl;
 };
 
 struct snd_usb_stream {
diff --git a/sound/usb/media.c b/sound/usb/media.c
new file mode 100644
index 0000000..40c24d3f
--- /dev/null
+++ b/sound/usb/media.c
@@ -0,0 +1,326 @@
+/*
+ * media.c - Media Controller specific ALSA driver code
+ *
+ * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
+ * Copyright (c) 2016 Samsung Electronics Co., Ltd.
+ *
+ * This file is released under the GPLv2.
+ */
+
+/*
+ * This file adds Media Controller support to ALSA driver
+ * to use the Media Controller API to share tuner with DVB
+ * and V4L2 drivers that control media device. Media device
+ * is created based on existing quirks framework. Using this
+ * approach, the media controller API usage can be added for
+ * a specific device.
+ */
+
+#include <linux/init.h>
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/slab.h>
+#include <linux/usb.h>
+
+#include <sound/pcm.h>
+#include <sound/core.h>
+
+#include "usbaudio.h"
+#include "card.h"
+#include "mixer.h"
+#include "media.h"
+
+static int media_snd_enable_source(struct media_ctl *mctl)
+{
+	int ret = 0;
+
+	if (!mctl)
+		return 0;
+
+	mutex_lock(&mctl->media_dev->graph_mutex);
+	if (mctl->media_dev->enable_source)
+		ret = mctl->media_dev->enable_source(&mctl->media_entity,
+						     &mctl->media_pipe);
+	mutex_unlock(&mctl->media_dev->graph_mutex);
+	return ret;
+}
+
+static void media_snd_disable_source(struct media_ctl *mctl)
+{
+	if (!mctl)
+		return;
+
+	mutex_lock(&mctl->media_dev->graph_mutex);
+	if (mctl->media_dev->disable_source)
+		mctl->media_dev->disable_source(&mctl->media_entity);
+	mutex_unlock(&mctl->media_dev->graph_mutex);
+}
+
+int media_snd_stream_init(struct snd_usb_substream *subs, struct snd_pcm *pcm,
+			int stream)
+{
+	struct media_device *mdev;
+	struct media_ctl *mctl;
+	struct device *pcm_dev = &pcm->streams[stream].dev;
+	u32 intf_type;
+	int ret = 0;
+	u16 mixer_pad;
+	struct media_entity *entity;
+
+	mdev = subs->stream->chip->media_dev;
+	if (!mdev)
+		return 0;
+
+	if (subs->media_ctl)
+		return 0;
+
+	/* allocate media_ctl */
+	mctl = kzalloc(sizeof(*mctl), GFP_KERNEL);
+	if (!mctl)
+		return -ENOMEM;
+
+	mctl->media_dev = mdev;
+	if (stream == SNDRV_PCM_STREAM_PLAYBACK) {
+		intf_type = MEDIA_INTF_T_ALSA_PCM_PLAYBACK;
+		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_PLAYBACK;
+		mctl->media_pad.flags = MEDIA_PAD_FL_SOURCE;
+		mixer_pad = 1;
+	} else {
+		intf_type = MEDIA_INTF_T_ALSA_PCM_CAPTURE;
+		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_CAPTURE;
+		mctl->media_pad.flags = MEDIA_PAD_FL_SINK;
+		mixer_pad = 2;
+	}
+	mctl->media_entity.name = pcm->name;
+	media_entity_pads_init(&mctl->media_entity, 1, &mctl->media_pad);
+	ret =  media_device_register_entity(mctl->media_dev,
+					    &mctl->media_entity);
+	if (ret)
+		goto free_mctl;
+
+	mctl->intf_devnode = media_devnode_create(mdev, intf_type, 0,
+						  MAJOR(pcm_dev->devt),
+						  MINOR(pcm_dev->devt));
+	if (!mctl->intf_devnode) {
+		ret = -ENOMEM;
+		goto unregister_entity;
+	}
+	mctl->intf_link = media_create_intf_link(&mctl->media_entity,
+						 &mctl->intf_devnode->intf,
+						 MEDIA_LNK_FL_ENABLED);
+	if (!mctl->intf_link) {
+		ret = -ENOMEM;
+		goto devnode_remove;
+	}
+
+	/* create link between mixer and audio */
+	media_device_for_each_entity(entity, mdev) {
+		switch (entity->function) {
+		case MEDIA_ENT_F_AUDIO_MIXER:
+			ret = media_create_pad_link(entity, mixer_pad,
+						    &mctl->media_entity, 0,
+						    MEDIA_LNK_FL_ENABLED);
+			if (ret)
+				goto remove_intf_link;
+			break;
+		}
+	}
+
+	subs->media_ctl = mctl;
+	return 0;
+
+remove_intf_link:
+	media_remove_intf_link(mctl->intf_link);
+devnode_remove:
+	media_devnode_remove(mctl->intf_devnode);
+unregister_entity:
+	media_device_unregister_entity(&mctl->media_entity);
+free_mctl:
+	kfree(mctl);
+	return ret;
+}
+
+void media_snd_stream_delete(struct snd_usb_substream *subs)
+{
+	struct media_ctl *mctl = subs->media_ctl;
+
+	if (mctl && mctl->media_dev) {
+		struct media_device *mdev;
+
+		mdev = mctl->media_dev;
+		if (mdev && media_devnode_is_registered(mdev->devnode)) {
+			media_devnode_remove(mctl->intf_devnode);
+			media_device_unregister_entity(&mctl->media_entity);
+			media_entity_cleanup(&mctl->media_entity);
+		}
+		kfree(mctl);
+		subs->media_ctl = NULL;
+	}
+}
+
+int media_snd_start_pipeline(struct snd_usb_substream *subs)
+{
+	struct media_ctl *mctl = subs->media_ctl;
+
+	if (mctl)
+		return media_snd_enable_source(mctl);
+	return 0;
+}
+
+void media_snd_stop_pipeline(struct snd_usb_substream *subs)
+{
+	struct media_ctl *mctl = subs->media_ctl;
+
+	if (mctl)
+		media_snd_disable_source(mctl);
+}
+
+static int media_snd_mixer_init(struct snd_usb_audio *chip)
+{
+	struct device *ctl_dev = &chip->card->ctl_dev;
+	struct media_intf_devnode *ctl_intf;
+	struct usb_mixer_interface *mixer;
+	struct media_device *mdev = chip->media_dev;
+	struct media_mixer_ctl *mctl;
+	u32 intf_type = MEDIA_INTF_T_ALSA_CONTROL;
+	int ret;
+
+	if (!mdev)
+		return -ENODEV;
+
+	ctl_intf = chip->ctl_intf_media_devnode;
+	if (!ctl_intf) {
+		ctl_intf = media_devnode_create(mdev, intf_type, 0,
+						MAJOR(ctl_dev->devt),
+						MINOR(ctl_dev->devt));
+		if (!ctl_intf)
+			return -ENOMEM;
+		chip->ctl_intf_media_devnode = ctl_intf;
+	}
+
+	list_for_each_entry(mixer, &chip->mixer_list, list) {
+
+		if (mixer->media_mixer_ctl)
+			continue;
+
+		/* allocate media_mixer_ctl */
+		mctl = kzalloc(sizeof(*mctl), GFP_KERNEL);
+		if (!mctl)
+			return -ENOMEM;
+
+		mctl->media_dev = mdev;
+		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_MIXER;
+		mctl->media_entity.name = chip->card->mixername;
+		mctl->media_pad[0].flags = MEDIA_PAD_FL_SINK;
+		mctl->media_pad[1].flags = MEDIA_PAD_FL_SOURCE;
+		mctl->media_pad[2].flags = MEDIA_PAD_FL_SOURCE;
+		media_entity_pads_init(&mctl->media_entity, MEDIA_MIXER_PAD_MAX,
+				  mctl->media_pad);
+		ret =  media_device_register_entity(mctl->media_dev,
+						    &mctl->media_entity);
+		if (ret) {
+			kfree(mctl);
+			return ret;
+		}
+
+		mctl->intf_link = media_create_intf_link(&mctl->media_entity,
+							 &ctl_intf->intf,
+							 MEDIA_LNK_FL_ENABLED);
+		if (!mctl->intf_link) {
+			media_device_unregister_entity(&mctl->media_entity);
+			media_entity_cleanup(&mctl->media_entity);
+			kfree(mctl);
+			return -ENOMEM;
+		}
+		mctl->intf_devnode = ctl_intf;
+		mixer->media_mixer_ctl = mctl;
+	}
+	return 0;
+}
+
+static void media_snd_mixer_delete(struct snd_usb_audio *chip)
+{
+	struct usb_mixer_interface *mixer;
+	struct media_device *mdev = chip->media_dev;
+
+	if (!mdev)
+		return;
+
+	list_for_each_entry(mixer, &chip->mixer_list, list) {
+		struct media_mixer_ctl *mctl;
+
+		mctl = mixer->media_mixer_ctl;
+		if (!mixer->media_mixer_ctl)
+			continue;
+
+		if (media_devnode_is_registered(mdev->devnode)) {
+			media_device_unregister_entity(&mctl->media_entity);
+			media_entity_cleanup(&mctl->media_entity);
+		}
+		kfree(mctl);
+		mixer->media_mixer_ctl = NULL;
+	}
+	if (media_devnode_is_registered(mdev->devnode))
+		media_devnode_remove(chip->ctl_intf_media_devnode);
+	chip->ctl_intf_media_devnode = NULL;
+}
+
+int media_snd_device_create(struct snd_usb_audio *chip,
+			struct usb_interface *iface)
+{
+	struct media_device *mdev;
+	struct usb_device *usbdev = interface_to_usbdev(iface);
+	int ret;
+
+	mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
+	if (!mdev)
+		return -ENOMEM;
+
+	if (!media_devnode_is_registered(mdev->devnode)) {
+		/* register media_device */
+		ret = media_device_register(mdev);
+		if (ret) {
+			media_device_delete(mdev, KBUILD_MODNAME);
+			dev_err(&usbdev->dev,
+				"Couldn't register media device. Error: %d\n",
+				ret);
+			return ret;
+		}
+	}
+
+	/* save media device - avoid lookups */
+	chip->media_dev = mdev;
+
+	/* Create media entities for mixer and control dev */
+	ret = media_snd_mixer_init(chip);
+	if (ret) {
+		dev_err(&usbdev->dev,
+			"Couldn't create media mixer entities. Error: %d\n",
+			ret);
+
+		/* clear saved media_dev */
+		chip->media_dev = NULL;
+
+		return ret;
+	}
+	return 0;
+}
+
+void media_snd_device_delete(struct snd_usb_audio *chip)
+{
+	struct media_device *mdev = chip->media_dev;
+	struct snd_usb_stream *stream;
+
+	/* release resources */
+	list_for_each_entry(stream, &chip->pcm_list, list) {
+		media_snd_stream_delete(&stream->substream[0]);
+		media_snd_stream_delete(&stream->substream[1]);
+	}
+
+	media_snd_mixer_delete(chip);
+
+	if (mdev) {
+		media_device_delete(mdev, KBUILD_MODNAME);
+		chip->media_dev = NULL;
+	}
+}
diff --git a/sound/usb/media.h b/sound/usb/media.h
new file mode 100644
index 0000000..d204375
--- /dev/null
+++ b/sound/usb/media.h
@@ -0,0 +1,73 @@
+/*
+ * media.h - Media Controller specific ALSA driver code
+ *
+ * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
+ * Copyright (c) 2016 Samsung Electronics Co., Ltd.
+ *
+ * This file is released under the GPLv2.
+ */
+
+/*
+ * This file adds Media Controller support to ALSA driver
+ * to use the Media Controller API to share tuner with DVB
+ * and V4L2 drivers that control media device. Media device
+ * is created based on existing quirks framework. Using this
+ * approach, the media controller API usage can be added for
+ * a specific device.
+ */
+#ifndef __MEDIA_H
+
+#ifdef CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER
+
+#include <media/media-device.h>
+#include <media/media-entity.h>
+#include <media/media-dev-allocator.h>
+#include <sound/asound.h>
+
+struct media_ctl {
+	struct media_device *media_dev;
+	struct media_entity media_entity;
+	struct media_intf_devnode *intf_devnode;
+	struct media_link *intf_link;
+	struct media_pad media_pad;
+	struct media_pipeline media_pipe;
+};
+
+/*
+ * One source pad each for SNDRV_PCM_STREAM_CAPTURE and
+ * SNDRV_PCM_STREAM_PLAYBACK. One for sink pad to link
+ * to AUDIO Source
+ */
+#define MEDIA_MIXER_PAD_MAX    (SNDRV_PCM_STREAM_LAST + 2)
+
+struct media_mixer_ctl {
+	struct media_device *media_dev;
+	struct media_entity media_entity;
+	struct media_intf_devnode *intf_devnode;
+	struct media_link *intf_link;
+	struct media_pad media_pad[MEDIA_MIXER_PAD_MAX];
+	struct media_pipeline media_pipe;
+};
+
+int media_snd_device_create(struct snd_usb_audio *chip,
+			    struct usb_interface *iface);
+void media_snd_device_delete(struct snd_usb_audio *chip);
+int media_snd_stream_init(struct snd_usb_substream *subs, struct snd_pcm *pcm,
+			  int stream);
+void media_snd_stream_delete(struct snd_usb_substream *subs);
+int media_snd_start_pipeline(struct snd_usb_substream *subs);
+void media_snd_stop_pipeline(struct snd_usb_substream *subs);
+#else
+static inline int media_snd_device_create(struct snd_usb_audio *chip,
+					  struct usb_interface *iface)
+						{ return 0; }
+static inline void media_snd_device_delete(struct snd_usb_audio *chip) { }
+static inline int media_snd_stream_init(struct snd_usb_substream *subs,
+					struct snd_pcm *pcm, int stream)
+						{ return 0; }
+static inline void media_snd_stream_delete(struct snd_usb_substream *subs) { }
+static inline int media_snd_start_pipeline(struct snd_usb_substream *subs)
+					{ return 0; }
+static inline void media_snd_stop_pipeline(struct snd_usb_substream *subs) { }
+#endif
+#endif /* __MEDIA_H */
diff --git a/sound/usb/mixer.h b/sound/usb/mixer.h
index 3417ef3..f378944 100644
--- a/sound/usb/mixer.h
+++ b/sound/usb/mixer.h
@@ -3,6 +3,8 @@
 
 #include <sound/info.h>
 
+struct media_mixer_ctl;
+
 struct usb_mixer_interface {
 	struct snd_usb_audio *chip;
 	struct usb_host_interface *hostif;
@@ -22,6 +24,7 @@ struct usb_mixer_interface {
 	struct urb *rc_urb;
 	struct usb_ctrlrequest *rc_setup_packet;
 	u8 rc_buffer[6];
+	struct media_mixer_ctl *media_mixer_ctl;
 };
 
 #define MAX_CHANNELS	16	/* max logical channels */
diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c
index 44d178e..0e4e0640 100644
--- a/sound/usb/pcm.c
+++ b/sound/usb/pcm.c
@@ -35,6 +35,7 @@
 #include "pcm.h"
 #include "clock.h"
 #include "power.h"
+#include "media.h"
 
 #define SUBSTREAM_FLAG_DATA_EP_STARTED	0
 #define SUBSTREAM_FLAG_SYNC_EP_STARTED	1
@@ -717,10 +718,14 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
 	struct audioformat *fmt;
 	int ret;
 
+	ret = media_snd_start_pipeline(subs);
+	if (ret)
+		return ret;
+
 	ret = snd_pcm_lib_alloc_vmalloc_buffer(substream,
 					       params_buffer_bytes(hw_params));
 	if (ret < 0)
-		return ret;
+		goto err_ret;
 
 	subs->pcm_format = params_format(hw_params);
 	subs->period_bytes = params_period_bytes(hw_params);
@@ -734,22 +739,27 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
 		dev_dbg(&subs->dev->dev,
 			"cannot set format: format = %#x, rate = %d, channels = %d\n",
 			   subs->pcm_format, subs->cur_rate, subs->channels);
-		return -EINVAL;
+		ret = -EINVAL;
+		goto err_ret;
 	}
 
 	ret = snd_usb_lock_shutdown(subs->stream->chip);
 	if (ret < 0)
-		return ret;
+		goto err_ret;
 	ret = set_format(subs, fmt);
 	snd_usb_unlock_shutdown(subs->stream->chip);
 	if (ret < 0)
-		return ret;
+		goto err_ret;
 
 	subs->interface = fmt->iface;
 	subs->altset_idx = fmt->altset_idx;
 	subs->need_setup_ep = true;
 
 	return 0;
+
+err_ret:
+	media_snd_stop_pipeline(subs);
+	return ret;
 }
 
 /*
@@ -761,6 +771,7 @@ static int snd_usb_hw_free(struct snd_pcm_substream *substream)
 {
 	struct snd_usb_substream *subs = substream->runtime->private_data;
 
+	media_snd_stop_pipeline(subs);
 	subs->cur_audiofmt = NULL;
 	subs->cur_rate = 0;
 	subs->period_bytes = 0;
@@ -1221,6 +1232,7 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream, int direction)
 	struct snd_usb_stream *as = snd_pcm_substream_chip(substream);
 	struct snd_pcm_runtime *runtime = substream->runtime;
 	struct snd_usb_substream *subs = &as->substream[direction];
+	int ret;
 
 	subs->interface = -1;
 	subs->altset_idx = 0;
@@ -1234,7 +1246,12 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream, int direction)
 	subs->dsd_dop.channel = 0;
 	subs->dsd_dop.marker = 1;
 
-	return setup_hw_info(runtime, subs);
+	ret = setup_hw_info(runtime, subs);
+	if (ret == 0)
+		ret = media_snd_stream_init(subs, as->pcm, direction);
+	if (ret)
+		snd_usb_autosuspend(subs->stream->chip);
+	return ret;
 }
 
 static int snd_usb_pcm_close(struct snd_pcm_substream *substream, int direction)
@@ -1243,6 +1260,7 @@ static int snd_usb_pcm_close(struct snd_pcm_substream *substream, int direction)
 	struct snd_usb_substream *subs = &as->substream[direction];
 
 	stop_endpoints(subs, true);
+	media_snd_stop_pipeline(subs);
 
 	if (subs->interface >= 0 &&
 	    !snd_usb_lock_shutdown(subs->stream->chip)) {
diff --git a/sound/usb/quirks-table.h b/sound/usb/quirks-table.h
index 8a59d47..164e249 100644
--- a/sound/usb/quirks-table.h
+++ b/sound/usb/quirks-table.h
@@ -2886,6 +2886,7 @@ YAMAHA_DEVICE(0x7010, "UB99"),
 		.product_name = pname, \
 		.ifnum = QUIRK_ANY_INTERFACE, \
 		.type = QUIRK_AUDIO_ALIGN_TRANSFER, \
+		.media_device = 1, \
 	} \
 }
 
diff --git a/sound/usb/stream.c b/sound/usb/stream.c
index 8e9548bc..6fe7f21 100644
--- a/sound/usb/stream.c
+++ b/sound/usb/stream.c
@@ -36,6 +36,7 @@
 #include "format.h"
 #include "clock.h"
 #include "stream.h"
+#include "media.h"
 
 /*
  * free a substream
@@ -52,6 +53,7 @@ static void free_substream(struct snd_usb_substream *subs)
 		kfree(fp);
 	}
 	kfree(subs->rate_list.list);
+	media_snd_stream_delete(subs);
 }
 
 
diff --git a/sound/usb/usbaudio.h b/sound/usb/usbaudio.h
index 4d5c89a..89b6853 100644
--- a/sound/usb/usbaudio.h
+++ b/sound/usb/usbaudio.h
@@ -30,6 +30,9 @@
  *
  */
 
+struct media_device;
+struct media_intf_devnode;
+
 struct snd_usb_audio {
 	int index;
 	struct usb_device *dev;
@@ -61,6 +64,8 @@ struct snd_usb_audio {
 	bool autoclock;			/* from the 'autoclock' module param */
 
 	struct usb_host_interface *ctrl_intf;	/* the audio control interface */
+	struct media_device *media_dev;
+	struct media_intf_devnode *ctl_intf_media_devnode;
 };
 
 #define usb_audio_err(chip, fmt, args...) \
@@ -111,6 +116,7 @@ struct snd_usb_audio_quirk {
 	const char *product_name;
 	int16_t ifnum;
 	uint16_t type;
+	bool media_device;
 	const void *data;
 };
 
-- 
2.7.4

Re: [PATCH v6 3/3] sound/usb: Use Media Controller API to share media resources

[Shuah Khan]

On 11/30/2016 03:01 PM, Shuah Khan wrote:
> Change ALSA driver to use Media Controller API to share media resources
> with DVB, and V4L2 drivers on a AU0828 media device.
> 
> Media Controller specific initialization is done after sound card is
> registered. ALSA creates Media interface and entity function graph
> nodes for Control, Mixer, PCM Playback, and PCM Capture devices.
> 
> snd_usb_hw_params() will call Media Controller enable source handler
> interface to request the media resource. If resource request is granted,
> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
> returned.
> 
> Media specific cleanup is done in usb_audio_disconnect().
> 
> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>

Hi Takashi,

If you are good with this patch, could you please Ack it, so Mauro
can pull it into media tree with the other two patches in this series,
when he is ready to do so.

thanks,
-- Shuah

> ---
> Changes to patch 0003:
> - Changed to hold graph_mutex to check and call enable_source and
>   disable_source handlers - to match au0828 doing the same in:
> media: Protect enable_source and disable_source handler code paths 
> https://lkml.org/lkml/2016/11/29/1001
> 
>  sound/usb/Kconfig        |   4 +
>  sound/usb/Makefile       |   2 +
>  sound/usb/card.c         |  14 ++
>  sound/usb/card.h         |   3 +
>  sound/usb/media.c        | 326 +++++++++++++++++++++++++++++++++++++++++++++++
>  sound/usb/media.h        |  73 +++++++++++
>  sound/usb/mixer.h        |   3 +
>  sound/usb/pcm.c          |  28 +++-
>  sound/usb/quirks-table.h |   1 +
>  sound/usb/stream.c       |   2 +
>  sound/usb/usbaudio.h     |   6 +
>  11 files changed, 457 insertions(+), 5 deletions(-)
>  create mode 100644 sound/usb/media.c
>  create mode 100644 sound/usb/media.h
> 
> diff --git a/sound/usb/Kconfig b/sound/usb/Kconfig
> index a452ad7..d14bf41 100644
> --- a/sound/usb/Kconfig
> +++ b/sound/usb/Kconfig
> @@ -15,6 +15,7 @@ config SND_USB_AUDIO
>  	select SND_RAWMIDI
>  	select SND_PCM
>  	select BITREVERSE
> +	select SND_USB_AUDIO_USE_MEDIA_CONTROLLER if MEDIA_CONTROLLER && (MEDIA_SUPPORT=y || MEDIA_SUPPORT=SND_USB_AUDIO)
>  	help
>  	  Say Y here to include support for USB audio and USB MIDI
>  	  devices.
> @@ -22,6 +23,9 @@ config SND_USB_AUDIO
>  	  To compile this driver as a module, choose M here: the module
>  	  will be called snd-usb-audio.
>  
> +config SND_USB_AUDIO_USE_MEDIA_CONTROLLER
> +	bool
> +
>  config SND_USB_UA101
>  	tristate "Edirol UA-101/UA-1000 driver"
>  	select SND_PCM
> diff --git a/sound/usb/Makefile b/sound/usb/Makefile
> index 2d2d122..8dca3c4 100644
> --- a/sound/usb/Makefile
> +++ b/sound/usb/Makefile
> @@ -15,6 +15,8 @@ snd-usb-audio-objs := 	card.o \
>  			quirks.o \
>  			stream.o
>  
> +snd-usb-audio-$(CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER) += media.o
> +
>  snd-usbmidi-lib-objs := midi.o
>  
>  # Toplevel Module Dependency
> diff --git a/sound/usb/card.c b/sound/usb/card.c
> index 2ddc034..570ff00 100644
> --- a/sound/usb/card.c
> +++ b/sound/usb/card.c
> @@ -66,6 +66,7 @@
>  #include "format.h"
>  #include "power.h"
>  #include "stream.h"
> +#include "media.h"
>  
>  MODULE_AUTHOR("Takashi Iwai <tiwai@suse.de>");
>  MODULE_DESCRIPTION("USB Audio");
> @@ -616,6 +617,11 @@ static int usb_audio_probe(struct usb_interface *intf,
>  	if (err < 0)
>  		goto __error;
>  
> +	if (quirk && quirk->media_device) {
> +		/* don't want to fail when media_snd_device_create() fails */
> +		media_snd_device_create(chip, intf);
> +	}
> +
>  	usb_chip[chip->index] = chip;
>  	chip->num_interfaces++;
>  	usb_set_intfdata(intf, chip);
> @@ -672,6 +678,14 @@ static void usb_audio_disconnect(struct usb_interface *intf)
>  		list_for_each(p, &chip->midi_list) {
>  			snd_usbmidi_disconnect(p);
>  		}
> +		/*
> +		 * Nice to check quirk && quirk->media_device and
> +		 * then call media_snd_device_delete(). Don't have
> +		 * access to quirk here. media_snd_device_delete()
> +		 * acceses mixer_list
> +		 */
> +		media_snd_device_delete(chip);
> +
>  		/* release mixer resources */
>  		list_for_each_entry(mixer, &chip->mixer_list, list) {
>  			snd_usb_mixer_disconnect(mixer);
> diff --git a/sound/usb/card.h b/sound/usb/card.h
> index 111b0f0..6ef8e88 100644
> --- a/sound/usb/card.h
> +++ b/sound/usb/card.h
> @@ -105,6 +105,8 @@ struct snd_usb_endpoint {
>  	struct list_head list;
>  };
>  
> +struct media_ctl;
> +
>  struct snd_usb_substream {
>  	struct snd_usb_stream *stream;
>  	struct usb_device *dev;
> @@ -156,6 +158,7 @@ struct snd_usb_substream {
>  	} dsd_dop;
>  
>  	bool trigger_tstamp_pending_update; /* trigger timestamp being updated from initial estimate */
> +	struct media_ctl *media_ctl;
>  };
>  
>  struct snd_usb_stream {
> diff --git a/sound/usb/media.c b/sound/usb/media.c
> new file mode 100644
> index 0000000..40c24d3f
> --- /dev/null
> +++ b/sound/usb/media.c
> @@ -0,0 +1,326 @@
> +/*
> + * media.c - Media Controller specific ALSA driver code
> + *
> + * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
> + * Copyright (c) 2016 Samsung Electronics Co., Ltd.
> + *
> + * This file is released under the GPLv2.
> + */
> +
> +/*
> + * This file adds Media Controller support to ALSA driver
> + * to use the Media Controller API to share tuner with DVB
> + * and V4L2 drivers that control media device. Media device
> + * is created based on existing quirks framework. Using this
> + * approach, the media controller API usage can be added for
> + * a specific device.
> + */
> +
> +#include <linux/init.h>
> +#include <linux/list.h>
> +#include <linux/mutex.h>
> +#include <linux/slab.h>
> +#include <linux/usb.h>
> +
> +#include <sound/pcm.h>
> +#include <sound/core.h>
> +
> +#include "usbaudio.h"
> +#include "card.h"
> +#include "mixer.h"
> +#include "media.h"
> +
> +static int media_snd_enable_source(struct media_ctl *mctl)
> +{
> +	int ret = 0;
> +
> +	if (!mctl)
> +		return 0;
> +
> +	mutex_lock(&mctl->media_dev->graph_mutex);
> +	if (mctl->media_dev->enable_source)
> +		ret = mctl->media_dev->enable_source(&mctl->media_entity,
> +						     &mctl->media_pipe);
> +	mutex_unlock(&mctl->media_dev->graph_mutex);
> +	return ret;
> +}
> +
> +static void media_snd_disable_source(struct media_ctl *mctl)
> +{
> +	if (!mctl)
> +		return;
> +
> +	mutex_lock(&mctl->media_dev->graph_mutex);
> +	if (mctl->media_dev->disable_source)
> +		mctl->media_dev->disable_source(&mctl->media_entity);
> +	mutex_unlock(&mctl->media_dev->graph_mutex);
> +}
> +
> +int media_snd_stream_init(struct snd_usb_substream *subs, struct snd_pcm *pcm,
> +			int stream)
> +{
> +	struct media_device *mdev;
> +	struct media_ctl *mctl;
> +	struct device *pcm_dev = &pcm->streams[stream].dev;
> +	u32 intf_type;
> +	int ret = 0;
> +	u16 mixer_pad;
> +	struct media_entity *entity;
> +
> +	mdev = subs->stream->chip->media_dev;
> +	if (!mdev)
> +		return 0;
> +
> +	if (subs->media_ctl)
> +		return 0;
> +
> +	/* allocate media_ctl */
> +	mctl = kzalloc(sizeof(*mctl), GFP_KERNEL);
> +	if (!mctl)
> +		return -ENOMEM;
> +
> +	mctl->media_dev = mdev;
> +	if (stream == SNDRV_PCM_STREAM_PLAYBACK) {
> +		intf_type = MEDIA_INTF_T_ALSA_PCM_PLAYBACK;
> +		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_PLAYBACK;
> +		mctl->media_pad.flags = MEDIA_PAD_FL_SOURCE;
> +		mixer_pad = 1;
> +	} else {
> +		intf_type = MEDIA_INTF_T_ALSA_PCM_CAPTURE;
> +		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_CAPTURE;
> +		mctl->media_pad.flags = MEDIA_PAD_FL_SINK;
> +		mixer_pad = 2;
> +	}
> +	mctl->media_entity.name = pcm->name;
> +	media_entity_pads_init(&mctl->media_entity, 1, &mctl->media_pad);
> +	ret =  media_device_register_entity(mctl->media_dev,
> +					    &mctl->media_entity);
> +	if (ret)
> +		goto free_mctl;
> +
> +	mctl->intf_devnode = media_devnode_create(mdev, intf_type, 0,
> +						  MAJOR(pcm_dev->devt),
> +						  MINOR(pcm_dev->devt));
> +	if (!mctl->intf_devnode) {
> +		ret = -ENOMEM;
> +		goto unregister_entity;
> +	}
> +	mctl->intf_link = media_create_intf_link(&mctl->media_entity,
> +						 &mctl->intf_devnode->intf,
> +						 MEDIA_LNK_FL_ENABLED);
> +	if (!mctl->intf_link) {
> +		ret = -ENOMEM;
> +		goto devnode_remove;
> +	}
> +
> +	/* create link between mixer and audio */
> +	media_device_for_each_entity(entity, mdev) {
> +		switch (entity->function) {
> +		case MEDIA_ENT_F_AUDIO_MIXER:
> +			ret = media_create_pad_link(entity, mixer_pad,
> +						    &mctl->media_entity, 0,
> +						    MEDIA_LNK_FL_ENABLED);
> +			if (ret)
> +				goto remove_intf_link;
> +			break;
> +		}
> +	}
> +
> +	subs->media_ctl = mctl;
> +	return 0;
> +
> +remove_intf_link:
> +	media_remove_intf_link(mctl->intf_link);
> +devnode_remove:
> +	media_devnode_remove(mctl->intf_devnode);
> +unregister_entity:
> +	media_device_unregister_entity(&mctl->media_entity);
> +free_mctl:
> +	kfree(mctl);
> +	return ret;
> +}
> +
> +void media_snd_stream_delete(struct snd_usb_substream *subs)
> +{
> +	struct media_ctl *mctl = subs->media_ctl;
> +
> +	if (mctl && mctl->media_dev) {
> +		struct media_device *mdev;
> +
> +		mdev = mctl->media_dev;
> +		if (mdev && media_devnode_is_registered(mdev->devnode)) {
> +			media_devnode_remove(mctl->intf_devnode);
> +			media_device_unregister_entity(&mctl->media_entity);
> +			media_entity_cleanup(&mctl->media_entity);
> +		}
> +		kfree(mctl);
> +		subs->media_ctl = NULL;
> +	}
> +}
> +
> +int media_snd_start_pipeline(struct snd_usb_substream *subs)
> +{
> +	struct media_ctl *mctl = subs->media_ctl;
> +
> +	if (mctl)
> +		return media_snd_enable_source(mctl);
> +	return 0;
> +}
> +
> +void media_snd_stop_pipeline(struct snd_usb_substream *subs)
> +{
> +	struct media_ctl *mctl = subs->media_ctl;
> +
> +	if (mctl)
> +		media_snd_disable_source(mctl);
> +}
> +
> +static int media_snd_mixer_init(struct snd_usb_audio *chip)
> +{
> +	struct device *ctl_dev = &chip->card->ctl_dev;
> +	struct media_intf_devnode *ctl_intf;
> +	struct usb_mixer_interface *mixer;
> +	struct media_device *mdev = chip->media_dev;
> +	struct media_mixer_ctl *mctl;
> +	u32 intf_type = MEDIA_INTF_T_ALSA_CONTROL;
> +	int ret;
> +
> +	if (!mdev)
> +		return -ENODEV;
> +
> +	ctl_intf = chip->ctl_intf_media_devnode;
> +	if (!ctl_intf) {
> +		ctl_intf = media_devnode_create(mdev, intf_type, 0,
> +						MAJOR(ctl_dev->devt),
> +						MINOR(ctl_dev->devt));
> +		if (!ctl_intf)
> +			return -ENOMEM;
> +		chip->ctl_intf_media_devnode = ctl_intf;
> +	}
> +
> +	list_for_each_entry(mixer, &chip->mixer_list, list) {
> +
> +		if (mixer->media_mixer_ctl)
> +			continue;
> +
> +		/* allocate media_mixer_ctl */
> +		mctl = kzalloc(sizeof(*mctl), GFP_KERNEL);
> +		if (!mctl)
> +			return -ENOMEM;
> +
> +		mctl->media_dev = mdev;
> +		mctl->media_entity.function = MEDIA_ENT_F_AUDIO_MIXER;
> +		mctl->media_entity.name = chip->card->mixername;
> +		mctl->media_pad[0].flags = MEDIA_PAD_FL_SINK;
> +		mctl->media_pad[1].flags = MEDIA_PAD_FL_SOURCE;
> +		mctl->media_pad[2].flags = MEDIA_PAD_FL_SOURCE;
> +		media_entity_pads_init(&mctl->media_entity, MEDIA_MIXER_PAD_MAX,
> +				  mctl->media_pad);
> +		ret =  media_device_register_entity(mctl->media_dev,
> +						    &mctl->media_entity);
> +		if (ret) {
> +			kfree(mctl);
> +			return ret;
> +		}
> +
> +		mctl->intf_link = media_create_intf_link(&mctl->media_entity,
> +							 &ctl_intf->intf,
> +							 MEDIA_LNK_FL_ENABLED);
> +		if (!mctl->intf_link) {
> +			media_device_unregister_entity(&mctl->media_entity);
> +			media_entity_cleanup(&mctl->media_entity);
> +			kfree(mctl);
> +			return -ENOMEM;
> +		}
> +		mctl->intf_devnode = ctl_intf;
> +		mixer->media_mixer_ctl = mctl;
> +	}
> +	return 0;
> +}
> +
> +static void media_snd_mixer_delete(struct snd_usb_audio *chip)
> +{
> +	struct usb_mixer_interface *mixer;
> +	struct media_device *mdev = chip->media_dev;
> +
> +	if (!mdev)
> +		return;
> +
> +	list_for_each_entry(mixer, &chip->mixer_list, list) {
> +		struct media_mixer_ctl *mctl;
> +
> +		mctl = mixer->media_mixer_ctl;
> +		if (!mixer->media_mixer_ctl)
> +			continue;
> +
> +		if (media_devnode_is_registered(mdev->devnode)) {
> +			media_device_unregister_entity(&mctl->media_entity);
> +			media_entity_cleanup(&mctl->media_entity);
> +		}
> +		kfree(mctl);
> +		mixer->media_mixer_ctl = NULL;
> +	}
> +	if (media_devnode_is_registered(mdev->devnode))
> +		media_devnode_remove(chip->ctl_intf_media_devnode);
> +	chip->ctl_intf_media_devnode = NULL;
> +}
> +
> +int media_snd_device_create(struct snd_usb_audio *chip,
> +			struct usb_interface *iface)
> +{
> +	struct media_device *mdev;
> +	struct usb_device *usbdev = interface_to_usbdev(iface);
> +	int ret;
> +
> +	mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
> +	if (!mdev)
> +		return -ENOMEM;
> +
> +	if (!media_devnode_is_registered(mdev->devnode)) {
> +		/* register media_device */
> +		ret = media_device_register(mdev);
> +		if (ret) {
> +			media_device_delete(mdev, KBUILD_MODNAME);
> +			dev_err(&usbdev->dev,
> +				"Couldn't register media device. Error: %d\n",
> +				ret);
> +			return ret;
> +		}
> +	}
> +
> +	/* save media device - avoid lookups */
> +	chip->media_dev = mdev;
> +
> +	/* Create media entities for mixer and control dev */
> +	ret = media_snd_mixer_init(chip);
> +	if (ret) {
> +		dev_err(&usbdev->dev,
> +			"Couldn't create media mixer entities. Error: %d\n",
> +			ret);
> +
> +		/* clear saved media_dev */
> +		chip->media_dev = NULL;
> +
> +		return ret;
> +	}
> +	return 0;
> +}
> +
> +void media_snd_device_delete(struct snd_usb_audio *chip)
> +{
> +	struct media_device *mdev = chip->media_dev;
> +	struct snd_usb_stream *stream;
> +
> +	/* release resources */
> +	list_for_each_entry(stream, &chip->pcm_list, list) {
> +		media_snd_stream_delete(&stream->substream[0]);
> +		media_snd_stream_delete(&stream->substream[1]);
> +	}
> +
> +	media_snd_mixer_delete(chip);
> +
> +	if (mdev) {
> +		media_device_delete(mdev, KBUILD_MODNAME);
> +		chip->media_dev = NULL;
> +	}
> +}
> diff --git a/sound/usb/media.h b/sound/usb/media.h
> new file mode 100644
> index 0000000..d204375
> --- /dev/null
> +++ b/sound/usb/media.h
> @@ -0,0 +1,73 @@
> +/*
> + * media.h - Media Controller specific ALSA driver code
> + *
> + * Copyright (c) 2016 Shuah Khan <shuahkh@osg.samsung.com>
> + * Copyright (c) 2016 Samsung Electronics Co., Ltd.
> + *
> + * This file is released under the GPLv2.
> + */
> +
> +/*
> + * This file adds Media Controller support to ALSA driver
> + * to use the Media Controller API to share tuner with DVB
> + * and V4L2 drivers that control media device. Media device
> + * is created based on existing quirks framework. Using this
> + * approach, the media controller API usage can be added for
> + * a specific device.
> + */
> +#ifndef __MEDIA_H
> +
> +#ifdef CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER
> +
> +#include <media/media-device.h>
> +#include <media/media-entity.h>
> +#include <media/media-dev-allocator.h>
> +#include <sound/asound.h>
> +
> +struct media_ctl {
> +	struct media_device *media_dev;
> +	struct media_entity media_entity;
> +	struct media_intf_devnode *intf_devnode;
> +	struct media_link *intf_link;
> +	struct media_pad media_pad;
> +	struct media_pipeline media_pipe;
> +};
> +
> +/*
> + * One source pad each for SNDRV_PCM_STREAM_CAPTURE and
> + * SNDRV_PCM_STREAM_PLAYBACK. One for sink pad to link
> + * to AUDIO Source
> + */
> +#define MEDIA_MIXER_PAD_MAX    (SNDRV_PCM_STREAM_LAST + 2)
> +
> +struct media_mixer_ctl {
> +	struct media_device *media_dev;
> +	struct media_entity media_entity;
> +	struct media_intf_devnode *intf_devnode;
> +	struct media_link *intf_link;
> +	struct media_pad media_pad[MEDIA_MIXER_PAD_MAX];
> +	struct media_pipeline media_pipe;
> +};
> +
> +int media_snd_device_create(struct snd_usb_audio *chip,
> +			    struct usb_interface *iface);
> +void media_snd_device_delete(struct snd_usb_audio *chip);
> +int media_snd_stream_init(struct snd_usb_substream *subs, struct snd_pcm *pcm,
> +			  int stream);
> +void media_snd_stream_delete(struct snd_usb_substream *subs);
> +int media_snd_start_pipeline(struct snd_usb_substream *subs);
> +void media_snd_stop_pipeline(struct snd_usb_substream *subs);
> +#else
> +static inline int media_snd_device_create(struct snd_usb_audio *chip,
> +					  struct usb_interface *iface)
> +						{ return 0; }
> +static inline void media_snd_device_delete(struct snd_usb_audio *chip) { }
> +static inline int media_snd_stream_init(struct snd_usb_substream *subs,
> +					struct snd_pcm *pcm, int stream)
> +						{ return 0; }
> +static inline void media_snd_stream_delete(struct snd_usb_substream *subs) { }
> +static inline int media_snd_start_pipeline(struct snd_usb_substream *subs)
> +					{ return 0; }
> +static inline void media_snd_stop_pipeline(struct snd_usb_substream *subs) { }
> +#endif
> +#endif /* __MEDIA_H */
> diff --git a/sound/usb/mixer.h b/sound/usb/mixer.h
> index 3417ef3..f378944 100644
> --- a/sound/usb/mixer.h
> +++ b/sound/usb/mixer.h
> @@ -3,6 +3,8 @@
>  
>  #include <sound/info.h>
>  
> +struct media_mixer_ctl;
> +
>  struct usb_mixer_interface {
>  	struct snd_usb_audio *chip;
>  	struct usb_host_interface *hostif;
> @@ -22,6 +24,7 @@ struct usb_mixer_interface {
>  	struct urb *rc_urb;
>  	struct usb_ctrlrequest *rc_setup_packet;
>  	u8 rc_buffer[6];
> +	struct media_mixer_ctl *media_mixer_ctl;
>  };
>  
>  #define MAX_CHANNELS	16	/* max logical channels */
> diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c
> index 44d178e..0e4e0640 100644
> --- a/sound/usb/pcm.c
> +++ b/sound/usb/pcm.c
> @@ -35,6 +35,7 @@
>  #include "pcm.h"
>  #include "clock.h"
>  #include "power.h"
> +#include "media.h"
>  
>  #define SUBSTREAM_FLAG_DATA_EP_STARTED	0
>  #define SUBSTREAM_FLAG_SYNC_EP_STARTED	1
> @@ -717,10 +718,14 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
>  	struct audioformat *fmt;
>  	int ret;
>  
> +	ret = media_snd_start_pipeline(subs);
> +	if (ret)
> +		return ret;
> +
>  	ret = snd_pcm_lib_alloc_vmalloc_buffer(substream,
>  					       params_buffer_bytes(hw_params));
>  	if (ret < 0)
> -		return ret;
> +		goto err_ret;
>  
>  	subs->pcm_format = params_format(hw_params);
>  	subs->period_bytes = params_period_bytes(hw_params);
> @@ -734,22 +739,27 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
>  		dev_dbg(&subs->dev->dev,
>  			"cannot set format: format = %#x, rate = %d, channels = %d\n",
>  			   subs->pcm_format, subs->cur_rate, subs->channels);
> -		return -EINVAL;
> +		ret = -EINVAL;
> +		goto err_ret;
>  	}
>  
>  	ret = snd_usb_lock_shutdown(subs->stream->chip);
>  	if (ret < 0)
> -		return ret;
> +		goto err_ret;
>  	ret = set_format(subs, fmt);
>  	snd_usb_unlock_shutdown(subs->stream->chip);
>  	if (ret < 0)
> -		return ret;
> +		goto err_ret;
>  
>  	subs->interface = fmt->iface;
>  	subs->altset_idx = fmt->altset_idx;
>  	subs->need_setup_ep = true;
>  
>  	return 0;
> +
> +err_ret:
> +	media_snd_stop_pipeline(subs);
> +	return ret;
>  }
>  
>  /*
> @@ -761,6 +771,7 @@ static int snd_usb_hw_free(struct snd_pcm_substream *substream)
>  {
>  	struct snd_usb_substream *subs = substream->runtime->private_data;
>  
> +	media_snd_stop_pipeline(subs);
>  	subs->cur_audiofmt = NULL;
>  	subs->cur_rate = 0;
>  	subs->period_bytes = 0;
> @@ -1221,6 +1232,7 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream, int direction)
>  	struct snd_usb_stream *as = snd_pcm_substream_chip(substream);
>  	struct snd_pcm_runtime *runtime = substream->runtime;
>  	struct snd_usb_substream *subs = &as->substream[direction];
> +	int ret;
>  
>  	subs->interface = -1;
>  	subs->altset_idx = 0;
> @@ -1234,7 +1246,12 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream, int direction)
>  	subs->dsd_dop.channel = 0;
>  	subs->dsd_dop.marker = 1;
>  
> -	return setup_hw_info(runtime, subs);
> +	ret = setup_hw_info(runtime, subs);
> +	if (ret == 0)
> +		ret = media_snd_stream_init(subs, as->pcm, direction);
> +	if (ret)
> +		snd_usb_autosuspend(subs->stream->chip);
> +	return ret;
>  }
>  
>  static int snd_usb_pcm_close(struct snd_pcm_substream *substream, int direction)
> @@ -1243,6 +1260,7 @@ static int snd_usb_pcm_close(struct snd_pcm_substream *substream, int direction)
>  	struct snd_usb_substream *subs = &as->substream[direction];
>  
>  	stop_endpoints(subs, true);
> +	media_snd_stop_pipeline(subs);
>  
>  	if (subs->interface >= 0 &&
>  	    !snd_usb_lock_shutdown(subs->stream->chip)) {
> diff --git a/sound/usb/quirks-table.h b/sound/usb/quirks-table.h
> index 8a59d47..164e249 100644
> --- a/sound/usb/quirks-table.h
> +++ b/sound/usb/quirks-table.h
> @@ -2886,6 +2886,7 @@ YAMAHA_DEVICE(0x7010, "UB99"),
>  		.product_name = pname, \
>  		.ifnum = QUIRK_ANY_INTERFACE, \
>  		.type = QUIRK_AUDIO_ALIGN_TRANSFER, \
> +		.media_device = 1, \
>  	} \
>  }
>  
> diff --git a/sound/usb/stream.c b/sound/usb/stream.c
> index 8e9548bc..6fe7f21 100644
> --- a/sound/usb/stream.c
> +++ b/sound/usb/stream.c
> @@ -36,6 +36,7 @@
>  #include "format.h"
>  #include "clock.h"
>  #include "stream.h"
> +#include "media.h"
>  
>  /*
>   * free a substream
> @@ -52,6 +53,7 @@ static void free_substream(struct snd_usb_substream *subs)
>  		kfree(fp);
>  	}
>  	kfree(subs->rate_list.list);
> +	media_snd_stream_delete(subs);
>  }
>  
>  
> diff --git a/sound/usb/usbaudio.h b/sound/usb/usbaudio.h
> index 4d5c89a..89b6853 100644
> --- a/sound/usb/usbaudio.h
> +++ b/sound/usb/usbaudio.h
> @@ -30,6 +30,9 @@
>   *
>   */
>  
> +struct media_device;
> +struct media_intf_devnode;
> +
>  struct snd_usb_audio {
>  	int index;
>  	struct usb_device *dev;
> @@ -61,6 +64,8 @@ struct snd_usb_audio {
>  	bool autoclock;			/* from the 'autoclock' module param */
>  
>  	struct usb_host_interface *ctrl_intf;	/* the audio control interface */
> +	struct media_device *media_dev;
> +	struct media_intf_devnode *ctl_intf_media_devnode;
>  };
>  
>  #define usb_audio_err(chip, fmt, args...) \
> @@ -111,6 +116,7 @@ struct snd_usb_audio_quirk {
>  	const char *product_name;
>  	int16_t ifnum;
>  	uint16_t type;
> +	bool media_device;
>  	const void *data;
>  };
>  
> 

Re: [PATCH v6 3/3] sound/usb: Use Media Controller API to share media resources

[Laurent Pinchart]

Hi Shuah,

On Monday 05 Dec 2016 15:44:30 Shuah Khan wrote:
> On 11/30/2016 03:01 PM, Shuah Khan wrote:
> > Change ALSA driver to use Media Controller API to share media resources
> > with DVB, and V4L2 drivers on a AU0828 media device.
> > 
> > Media Controller specific initialization is done after sound card is
> > registered. ALSA creates Media interface and entity function graph
> > nodes for Control, Mixer, PCM Playback, and PCM Capture devices.
> > 
> > snd_usb_hw_params() will call Media Controller enable source handler
> > interface to request the media resource. If resource request is granted,
> > it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
> > returned.
> > 
> > Media specific cleanup is done in usb_audio_disconnect().
> > 
> > Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
> 
> Hi Takashi,
> 
> If you are good with this patch, could you please Ack it, so Mauro
> can pull it into media tree with the other two patches in this series,
> when he is ready to do so.

I *really* want to address the concerns raised by Sakari before pulling more 
code that makes fixing the race conditions more difficult. Please, let's all 
work on fixing the core code to build a stable base on which we can build 
additional features. V4L2 and MC need teamwork, it's time to give the 
subsystem the love it deserves.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 3/3] sound/usb: Use Media Controller API to share media resources

[Shuah Khan]

On 12/05/2016 04:21 PM, Laurent Pinchart wrote:
> Hi Shuah,
> 
> On Monday 05 Dec 2016 15:44:30 Shuah Khan wrote:
>> On 11/30/2016 03:01 PM, Shuah Khan wrote:
>>> Change ALSA driver to use Media Controller API to share media resources
>>> with DVB, and V4L2 drivers on a AU0828 media device.
>>>
>>> Media Controller specific initialization is done after sound card is
>>> registered. ALSA creates Media interface and entity function graph
>>> nodes for Control, Mixer, PCM Playback, and PCM Capture devices.
>>>
>>> snd_usb_hw_params() will call Media Controller enable source handler
>>> interface to request the media resource. If resource request is granted,
>>> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
>>> returned.
>>>
>>> Media specific cleanup is done in usb_audio_disconnect().
>>>
>>> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
>>
>> Hi Takashi,
>>
>> If you are good with this patch, could you please Ack it, so Mauro
>> can pull it into media tree with the other two patches in this series,
>> when he is ready to do so.
> 
> I *really* want to address the concerns raised by Sakari before pulling more 
> code that makes fixing the race conditions more difficult. Please, let's all 
> work on fixing the core code to build a stable base on which we can build 
> additional features. V4L2 and MC need teamwork, it's time to give the 
> subsystem the love it deserves.
> 

Hi Laurent,

The issue Sakari brought up is specific to using devm for video_device in
omap3 and vsp1. I tried reproducing the problem on two different drivers
and couldn't on Linux 4.9-rc7.

After sharing that with Sakari, I suggested to Sakari to pull up his patch
that removes the devm usage and see if he still needs all the patches in his
patch series. He didn't back to me on that. I also requested him to rebase on
top of media dev allocator because the allocator routines he has don't address
the shared media device need.

He also didn't respond to my response regarding the reasons for choosing
graph_mutex to protect enable_source and disable_source handlers.

So I am not sure how to move forward at the moment without a concrete plan
for Sakari's RFC series. Sakari's patch series is still RFC and doesn't address
shared media_device and requires all drivers to change.

thanks,
-- Shuah

Re: [PATCH v6 3/3] sound/usb: Use Media Controller API to share media resources

[Takashi Iwai]

On Wed, 30 Nov 2016 23:01:16 +0100,
Shuah Khan wrote:
> 
> --- a/sound/usb/card.c
> +++ b/sound/usb/card.c
(snip)
> @@ -616,6 +617,11 @@ static int usb_audio_probe(struct usb_interface *intf,
>  	if (err < 0)
>  		goto __error;
>  
> +	if (quirk && quirk->media_device) {
> +		/* don't want to fail when media_snd_device_create() fails */
> +		media_snd_device_create(chip, intf);

Note that the usb-audio driver is probed for each usb interface, and
there are multiple interfaces per device.  That is, usb_audio_probe()
may be called multiple times per device, and at the second or the
later calls, it appends the stuff onto the previously created card
object.

So, you'd have to make this call also conditional (e.g. check
chip->num_interfaces == 0, indicating the very first one), or allow to
be called multiple times.

In the former case, it'd be better to split media_snd_device_create()
and media_snd_mixer_init().  The *_device_create() needs to be called
only once, while *_mixer_init() still has to be called for each time
because the new mixer element may be added for each interface.


> +	}
> +
>  	usb_chip[chip->index] = chip;
>  	chip->num_interfaces++;
>  	usb_set_intfdata(intf, chip);
> @@ -672,6 +678,14 @@ static void usb_audio_disconnect(struct usb_interface *intf)
>  		list_for_each(p, &chip->midi_list) {
>  			snd_usbmidi_disconnect(p);
>  		}
> +		/*
> +		 * Nice to check quirk && quirk->media_device and
> +		 * then call media_snd_device_delete(). Don't have
> +		 * access to quirk here. media_snd_device_delete()
> +		 * acceses mixer_list
> +		 */
> +		media_snd_device_delete(chip);

... meanwhile this is OK, as it's called only once.

(BTW, is it OK to call media_* stuff while the device is still in use?
 The disconnect callback gets called for hot-unplug.)


> --- /dev/null
> +++ b/sound/usb/media.c
(snip)
> +void media_snd_stream_delete(struct snd_usb_substream *subs)
> +{
> +	struct media_ctl *mctl = subs->media_ctl;
> +
> +	if (mctl && mctl->media_dev) {

mctl->media_dev NULL check here is superfluous, as it's checked
mctl->below.

> +		struct media_device *mdev;
> +
> +		mdev = mctl->media_dev;
> +		if (mdev && media_devnode_is_registered(mdev->devnode)) {
> +			media_devnode_remove(mctl->intf_devnode);
> +			media_device_unregister_entity(&mctl->media_entity);
> +			media_entity_cleanup(&mctl->media_entity);
> +		}
> +		kfree(mctl);
> +		subs->media_ctl = NULL;
> +	}
> +}
> +
> +int media_snd_start_pipeline(struct snd_usb_substream *subs)
> +{
> +	struct media_ctl *mctl = subs->media_ctl;
> +
> +	if (mctl)
> +		return media_snd_enable_source(mctl);
> +	return 0;
> +}

It's merely a wrapper, and media_snd_enable_source() itself checks
NULL mctl.  So we can replace media_snd_enable_source() with
media_snd_start_pipeline().

> +void media_snd_stop_pipeline(struct snd_usb_substream *subs)
> +{
> +	struct media_ctl *mctl = subs->media_ctl;
> +
> +	if (mctl)
> +		media_snd_disable_source(mctl);
> +}

Ditto.

> diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c
> index 44d178e..0e4e0640 100644
> --- a/sound/usb/pcm.c
> +++ b/sound/usb/pcm.c
(snip)
> @@ -717,10 +718,14 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
>  	struct audioformat *fmt;
>  	int ret;
>  
> +	ret = media_snd_start_pipeline(subs);
> +	if (ret)
> +		return ret;

It's an open question at which timing we should call
media_snd_start_pipeline().  The hw_params is mostly OK, while the
real timing where the stream might be started is the prepare
callback.  I guess we can keep as is for now.

Also, one more thing to be considered is whether
media_snd_start_pipeline() can be called multiple times.  hw_params
and prepare callbacks may be called multiple times.  I suppose it's
OK, but just to be sure.


> @@ -1234,7 +1246,12 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream, int direction)
>  	subs->dsd_dop.channel = 0;
>  	subs->dsd_dop.marker = 1;
>  
> -	return setup_hw_info(runtime, subs);
> +	ret = setup_hw_info(runtime, subs);
> +	if (ret == 0)
> +		ret = media_snd_stream_init(subs, as->pcm, direction);
> +	if (ret)
> +		snd_usb_autosuspend(subs->stream->chip);
> +	return ret;

This leads to the PM refcount unbalance.  The call of
snd_usb_autosuspend() must be in the former if block,

	ret = setup_hw_info(runtime, subs);
	if (ret == 0) {
		ret = media_snd_stream_init(subs, as->pcm, direction);
		if (ret)
			snd_usb_autosuspend(subs->stream->chip);
	}
	return ret;


thanks,

Takashi

Re: [PATCH v6 3/3] sound/usb: Use Media Controller API to share media resources

[Takashi Iwai]

On Mon, 05 Dec 2016 23:44:30 +0100,
Shuah Khan wrote:
> 
> On 11/30/2016 03:01 PM, Shuah Khan wrote:
> > Change ALSA driver to use Media Controller API to share media resources
> > with DVB, and V4L2 drivers on a AU0828 media device.
> > 
> > Media Controller specific initialization is done after sound card is
> > registered. ALSA creates Media interface and entity function graph
> > nodes for Control, Mixer, PCM Playback, and PCM Capture devices.
> > 
> > snd_usb_hw_params() will call Media Controller enable source handler
> > interface to request the media resource. If resource request is granted,
> > it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
> > returned.
> > 
> > Media specific cleanup is done in usb_audio_disconnect().
> > 
> > Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
> 
> Hi Takashi,
> 
> If you are good with this patch, could you please Ack it, so Mauro
> can pull it into media tree with the other two patches in this series,
> when he is ready to do so.

Sorry for the late review.

The biggest concern right now is the way the driver is probed.  As
mentioned in the review reply, we need to consider the cases where the
probe is called multiple times.  Other things are minor.


thanks,

Takashi

Re: [PATCH v6 3/3] sound/usb: Use Media Controller API to share media resources

[Laurent Pinchart]

Hi Shuah,

On Monday 05 Dec 2016 17:38:23 Shuah Khan wrote:
> On 12/05/2016 04:21 PM, Laurent Pinchart wrote:
> > On Monday 05 Dec 2016 15:44:30 Shuah Khan wrote:
> >> On 11/30/2016 03:01 PM, Shuah Khan wrote:
> >>> Change ALSA driver to use Media Controller API to share media resources
> >>> with DVB, and V4L2 drivers on a AU0828 media device.
> >>> 
> >>> Media Controller specific initialization is done after sound card is
> >>> registered. ALSA creates Media interface and entity function graph
> >>> nodes for Control, Mixer, PCM Playback, and PCM Capture devices.
> >>> 
> >>> snd_usb_hw_params() will call Media Controller enable source handler
> >>> interface to request the media resource. If resource request is granted,
> >>> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY
> >>> is returned.
> >>> 
> >>> Media specific cleanup is done in usb_audio_disconnect().
> >>> 
> >>> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
> >> 
> >> Hi Takashi,
> >> 
> >> If you are good with this patch, could you please Ack it, so Mauro
> >> can pull it into media tree with the other two patches in this series,
> >> when he is ready to do so.
> > 
> > I *really* want to address the concerns raised by Sakari before pulling
> > more code that makes fixing the race conditions more difficult. Please,
> > let's all work on fixing the core code to build a stable base on which we
> > can build additional features. V4L2 and MC need teamwork, it's time to
> > give the subsystem the love it deserves.
> 
> Hi Laurent,
> 
> The issue Sakari brought up is specific to using devm for video_device in
> omap3 and vsp1. I tried reproducing the problem on two different drivers
> and couldn't on Linux 4.9-rc7.
> 
> After sharing that with Sakari, I suggested to Sakari to pull up his patch
> that removes the devm usage and see if he still needs all the patches in his
> patch series. He didn't back to me on that. I also requested him to rebase
> on top of media dev allocator because the allocator routines he has don't
> address the shared media device need.
> 
> He also didn't respond to my response regarding the reasons for choosing
> graph_mutex to protect enable_source and disable_source handlers.
> 
> So I am not sure how to move forward at the moment without a concrete plan
> for Sakari's RFC series. Sakari's patch series is still RFC and doesn't
> address shared media_device and requires all drivers to change.

Today is a public holiday in Finland, I don't expect Sakari to be available. 
Let's check this with him tomorrow.

-- 
Regards,

Laurent Pinchart

Re: [PATCH v6 3/3] sound/usb: Use Media Controller API to share media resources

[Shuah Khan]

Hi Takashi,

On 12/05/2016 11:50 PM, Takashi Iwai wrote:
> On Wed, 30 Nov 2016 23:01:16 +0100,
> Shuah Khan wrote:
>>
>> --- a/sound/usb/card.c
>> +++ b/sound/usb/card.c
> (snip)
>> @@ -616,6 +617,11 @@ static int usb_audio_probe(struct usb_interface *intf,
>>  	if (err < 0)
>>  		goto __error;
>>  
>> +	if (quirk && quirk->media_device) {
>> +		/* don't want to fail when media_snd_device_create() fails */
>> +		media_snd_device_create(chip, intf);
> 
> Note that the usb-audio driver is probed for each usb interface, and
> there are multiple interfaces per device.  That is, usb_audio_probe()
> may be called multiple times per device, and at the second or the
> later calls, it appends the stuff onto the previously created card
> object.
> 
> So, you'd have to make this call also conditional (e.g. check
> chip->num_interfaces == 0, indicating the very first one), or allow to
> be called multiple times.
> 
> In the former case, it'd be better to split media_snd_device_create()
> and media_snd_mixer_init().  The *_device_create() needs to be called
> only once, while *_mixer_init() still has to be called for each time
> because the new mixer element may be added for each interface.
> 

Thanks for the catch. I am able to fix this in media_snd_device_create()
I made a change to do media_dev allocate only once. media_snd_mixer_init()
will get called every time media_snd_device_create() is called. This way
new mixers can be handled. media_snd_mixer_init() has logic to deal with
mixers that are already initialized. We are good here with the following
change:

@@ -272,6 +258,14 @@ int media_snd_device_create(struct snd_usb_audio *chip,
        struct usb_device *usbdev = interface_to_usbdev(iface);
        int ret;
 
+       /* usb-audio driver is probed for each usb interface, and
+        * there are multiple interfaces per device. Avoid calling
+        * media_device_usb_allocate() each time usb_audio_probe()
+        * is called. Do it only once.
+        */
+       if (chip->media_dev)
+               goto snd_mixer_init;
+
        mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
        if (!mdev)
                return -ENOMEM;
@@ -291,6 +285,7 @@ int media_snd_device_create(struct snd_usb_audio *chip,
        /* save media device - avoid lookups */
        chip->media_dev = mdev;
 
+snd_mixer_init:
        /* Create media entities for mixer and control dev */
        ret = media_snd_mixer_init(chip);
        if (ret) {

> 
>> +	}
>> +
>>  	usb_chip[chip->index] = chip;
>>  	chip->num_interfaces++;
>>  	usb_set_intfdata(intf, chip);
>> @@ -672,6 +678,14 @@ static void usb_audio_disconnect(struct usb_interface *intf)
>>  		list_for_each(p, &chip->midi_list) {
>>  			snd_usbmidi_disconnect(p);
>>  		}
>> +		/*
>> +		 * Nice to check quirk && quirk->media_device and
>> +		 * then call media_snd_device_delete(). Don't have
>> +		 * access to quirk here. media_snd_device_delete()
>> +		 * acceses mixer_list
>> +		 */
>> +		media_snd_device_delete(chip);
> 
> ... meanwhile this is OK, as it's called only once.
> 
> (BTW, is it OK to call media_* stuff while the device is still in use?
>  The disconnect callback gets called for hot-unplug.)
> 

Yes. All of the media_* functions that get called during run-time check for
chip->media_dev or media_ctl and do nothing when these are null.

media_device itself will not be free'd until all reference are gone. When
usb_audio_disconnect() happens via unbind snd_usb_audio or physical remove,
media_dev sticks around until au0828 (media driver) goes away. There is
handling for any user apps. that have /dev/mediaX open.

Does this sound correct? Did I miss any of your concerns?

> 
>> --- /dev/null
>> +++ b/sound/usb/media.c
> (snip)
>> +void media_snd_stream_delete(struct snd_usb_substream *subs)
>> +{
>> +	struct media_ctl *mctl = subs->media_ctl;
>> +
>> +	if (mctl && mctl->media_dev) {
> 
> mctl->media_dev NULL check here is superfluous, as it's checked
> mctl->below.
> 

Done.

>> +		struct media_device *mdev;
>> +
>> +		mdev = mctl->media_dev;
>> +		if (mdev && media_devnode_is_registered(mdev->devnode)) {
>> +			media_devnode_remove(mctl->intf_devnode);
>> +			media_device_unregister_entity(&mctl->media_entity);
>> +			media_entity_cleanup(&mctl->media_entity);
>> +		}
>> +		kfree(mctl);
>> +		subs->media_ctl = NULL;
>> +	}
>> +}
>> +
>> +int media_snd_start_pipeline(struct snd_usb_substream *subs)
>> +{
>> +	struct media_ctl *mctl = subs->media_ctl;
>> +
>> +	if (mctl)
>> +		return media_snd_enable_source(mctl);
>> +	return 0;
>> +}
> 
> It's merely a wrapper, and media_snd_enable_source() itself checks
> NULL mctl.  So we can replace media_snd_enable_source() with
> media_snd_start_pipeline().

Done.

> 
>> +void media_snd_stop_pipeline(struct snd_usb_substream *subs)
>> +{
>> +	struct media_ctl *mctl = subs->media_ctl;
>> +
>> +	if (mctl)
>> +		media_snd_disable_source(mctl);
>> +}
> 
> Ditto.

Done.

> 
>> diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c
>> index 44d178e..0e4e0640 100644
>> --- a/sound/usb/pcm.c
>> +++ b/sound/usb/pcm.c
> (snip)
>> @@ -717,10 +718,14 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
>>  	struct audioformat *fmt;
>>  	int ret;
>>  
>> +	ret = media_snd_start_pipeline(subs);
>> +	if (ret)
>> +		return ret;
> 
> It's an open question at which timing we should call
> media_snd_start_pipeline().  The hw_params is mostly OK, while the
> real timing where the stream might be started is the prepare
> callback.  I guess we can keep as is for now.

Okay.

> 
> Also, one more thing to be considered is whether
> media_snd_start_pipeline() can be called multiple times.  hw_params
> and prepare callbacks may be called multiple times.  I suppose it's
> OK, but just to be sure.

Yes. media_snd_start_pipeline() can be called multiple times. v4l2 apps.
also call start_pipeline multiple times. So we are good here.

> 
> 
>> @@ -1234,7 +1246,12 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream, int direction)
>>  	subs->dsd_dop.channel = 0;
>>  	subs->dsd_dop.marker = 1;
>>  
>> -	return setup_hw_info(runtime, subs);
>> +	ret = setup_hw_info(runtime, subs);
>> +	if (ret == 0)
>> +		ret = media_snd_stream_init(subs, as->pcm, direction);
>> +	if (ret)
>> +		snd_usb_autosuspend(subs->stream->chip);
>> +	return ret;
> 
> This leads to the PM refcount unbalance.  The call of
> snd_usb_autosuspend() must be in the former if block,
> 
> 	ret = setup_hw_info(runtime, subs);
> 	if (ret == 0) {
> 		ret = media_snd_stream_init(subs, as->pcm, direction);
> 		if (ret)
> 			snd_usb_autosuspend(subs->stream->chip);
> 	}
> 	return ret;

Done.

> 
> 
> thanks,
> 
> Takashi
> 

Mauro,

I will send patch v7 and include the others in the series that don't need
changes to keep them grouped.

thanks,
-- Shuah

Re: [PATCH v6 3/3] sound/usb: Use Media Controller API to share media resources

[Takashi Iwai]

On Tue, 06 Dec 2016 19:41:37 +0100,
Shuah Khan wrote:
> 
> Hi Takashi,
> 
> On 12/05/2016 11:50 PM, Takashi Iwai wrote:
> > On Wed, 30 Nov 2016 23:01:16 +0100,
> > Shuah Khan wrote:
> >>
> >> --- a/sound/usb/card.c
> >> +++ b/sound/usb/card.c
> > (snip)
> >> @@ -616,6 +617,11 @@ static int usb_audio_probe(struct usb_interface *intf,
> >>  	if (err < 0)
> >>  		goto __error;
> >>  
> >> +	if (quirk && quirk->media_device) {
> >> +		/* don't want to fail when media_snd_device_create() fails */
> >> +		media_snd_device_create(chip, intf);
> > 
> > Note that the usb-audio driver is probed for each usb interface, and
> > there are multiple interfaces per device.  That is, usb_audio_probe()
> > may be called multiple times per device, and at the second or the
> > later calls, it appends the stuff onto the previously created card
> > object.
> > 
> > So, you'd have to make this call also conditional (e.g. check
> > chip->num_interfaces == 0, indicating the very first one), or allow to
> > be called multiple times.
> > 
> > In the former case, it'd be better to split media_snd_device_create()
> > and media_snd_mixer_init().  The *_device_create() needs to be called
> > only once, while *_mixer_init() still has to be called for each time
> > because the new mixer element may be added for each interface.
> > 
> 
> Thanks for the catch. I am able to fix this in media_snd_device_create()
> I made a change to do media_dev allocate only once. media_snd_mixer_init()
> will get called every time media_snd_device_create() is called. This way
> new mixers can be handled. media_snd_mixer_init() has logic to deal with
> mixers that are already initialized. We are good here with the following
> change:
> 
> @@ -272,6 +258,14 @@ int media_snd_device_create(struct snd_usb_audio *chip,
>         struct usb_device *usbdev = interface_to_usbdev(iface);
>         int ret;
>  
> +       /* usb-audio driver is probed for each usb interface, and
> +        * there are multiple interfaces per device. Avoid calling
> +        * media_device_usb_allocate() each time usb_audio_probe()
> +        * is called. Do it only once.
> +        */
> +       if (chip->media_dev)
> +               goto snd_mixer_init;
> +
>         mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME);
>         if (!mdev)
>                 return -ENOMEM;
> @@ -291,6 +285,7 @@ int media_snd_device_create(struct snd_usb_audio *chip,
>         /* save media device - avoid lookups */
>         chip->media_dev = mdev;
>  
> +snd_mixer_init:
>         /* Create media entities for mixer and control dev */
>         ret = media_snd_mixer_init(chip);
>         if (ret) {

This looks good enough, yes.

> 
> > 
> >> +	}
> >> +
> >>  	usb_chip[chip->index] = chip;
> >>  	chip->num_interfaces++;
> >>  	usb_set_intfdata(intf, chip);
> >> @@ -672,6 +678,14 @@ static void usb_audio_disconnect(struct usb_interface *intf)
> >>  		list_for_each(p, &chip->midi_list) {
> >>  			snd_usbmidi_disconnect(p);
> >>  		}
> >> +		/*
> >> +		 * Nice to check quirk && quirk->media_device and
> >> +		 * then call media_snd_device_delete(). Don't have
> >> +		 * access to quirk here. media_snd_device_delete()
> >> +		 * acceses mixer_list
> >> +		 */
> >> +		media_snd_device_delete(chip);
> > 
> > ... meanwhile this is OK, as it's called only once.
> > 
> > (BTW, is it OK to call media_* stuff while the device is still in use?
> >  The disconnect callback gets called for hot-unplug.)
> > 
> 
> Yes. All of the media_* functions that get called during run-time check for
> chip->media_dev or media_ctl and do nothing when these are null.
> 
> media_device itself will not be free'd until all reference are gone. When
> usb_audio_disconnect() happens via unbind snd_usb_audio or physical remove,
> media_dev sticks around until au0828 (media driver) goes away. There is
> handling for any user apps. that have /dev/mediaX open.
> 
> Does this sound correct? Did I miss any of your concerns?

That sounds all good, so it's safe to call there.


thanks,

Takashi

Re: [PATCH v6 3/3] sound/usb: Use Media Controller API to share media resources

[Sakari Ailus]

Hi Shuah,

On Mon, Dec 05, 2016 at 05:38:23PM -0700, Shuah Khan wrote:
> On 12/05/2016 04:21 PM, Laurent Pinchart wrote:
> > Hi Shuah,
> > 
> > On Monday 05 Dec 2016 15:44:30 Shuah Khan wrote:
> >> On 11/30/2016 03:01 PM, Shuah Khan wrote:
> >>> Change ALSA driver to use Media Controller API to share media resources
> >>> with DVB, and V4L2 drivers on a AU0828 media device.
> >>>
> >>> Media Controller specific initialization is done after sound card is
> >>> registered. ALSA creates Media interface and entity function graph
> >>> nodes for Control, Mixer, PCM Playback, and PCM Capture devices.
> >>>
> >>> snd_usb_hw_params() will call Media Controller enable source handler
> >>> interface to request the media resource. If resource request is granted,
> >>> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
> >>> returned.
> >>>
> >>> Media specific cleanup is done in usb_audio_disconnect().
> >>>
> >>> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
> >>
> >> Hi Takashi,
> >>
> >> If you are good with this patch, could you please Ack it, so Mauro
> >> can pull it into media tree with the other two patches in this series,
> >> when he is ready to do so.
> > 
> > I *really* want to address the concerns raised by Sakari before pulling more 
> > code that makes fixing the race conditions more difficult. Please, let's all 
> > work on fixing the core code to build a stable base on which we can build 
> > additional features. V4L2 and MC need teamwork, it's time to give the 
> > subsystem the love it deserves.
> > 
> 
> Hi Laurent,
> 
> The issue Sakari brought up is specific to using devm for video_device in
> omap3 and vsp1. I tried reproducing the problem on two different drivers
> and couldn't on Linux 4.9-rc7.
> 
> After sharing that with Sakari, I suggested to Sakari to pull up his patch
> that removes the devm usage and see if he still needs all the patches in his
> patch series. He didn't back to me on that. I also requested him to rebase on

Just to see what remains, I made a small hack to test this with omap3isp by
just replacing the devm_() functions by their plain counterparts. The memory
is thus never released, for there is no really a proper moment to release it
--- something which the patchset resolves. The result is here:

<URL:http://www.retiisi.org.uk/v4l2/tmp/media-ref-dmesg.txt>

What happens there is that as part of the device unbinding, the graph
objects are unregistered (by media_device_unregister()) while streaming is
ongoing. Their parent media device pointers are set to NULL.

Then, when the user process exists, the streaming media pipeline is stopped.
This requires parsing of the media graph, a job which will obviously fail
miserably and immediately: the media device is obtained from the entity
where graph traversal is expected to begin.

Two mutexes are also acquired but they've been already destroyed at that
point (and the memory of which would also be released now without the hack
to test this "without devm"). There's just a single warning on this though.

> top of media dev allocator because the allocator routines he has don't address
> the shared media device need.

I do strongly prefer fixing the existing object lifetime issues in the
framework before extending it and thus making the problem domain more
complex than it is already.

What I mentioned as an example of this are the other callbacks to the media
device: presumably they do suffer from the exactly same problems as the
enable_source() and disable_source() ones.

As drivers do refer to other entities and controls they do expose also to
the user space, we can't really even remove entities safely at the moment.
We should have a solution to this as well, my patchset at the moment does
not address this.

What we do need a sound basis for the framework rather than hastily written
improvements to support a particular device. Also, testing device removal
with a particular device in a particular use case does not guarantee that no
further object lifetime issues related to device removal exist, even with
that same driver, let alone other devices.

Instead, we must consider how the frameworks and drivers manage the memory
allocated for the various objects, what are relations of those objects and
how they're exposed to the user space either directly or indirectly. As long
as objects (such as the media graph objects) with different lifetimes are
referred to from other objects without taking a reference or alternatively
serialising code paths that may access those objects and those that remove
them, we do have a problem.

The media graph with all the subsystem specific device nodes that are
exported to the user space is a rather complex data structure. I don't think
that acquiring the media graph lock in order to fix all the serialisation
problems is the right approach here.

> 
> He also didn't respond to my response regarding the reasons for choosing
> graph_mutex to protect enable_source and disable_source handlers.

I did. The point is that this is a partial fix and that does not properly
address the problem of device removal. It probably does decrease of the
probability of hitting the bug but it does not fix it.

> 
> So I am not sure how to move forward at the moment without a concrete plan
> for Sakari's RFC series. Sakari's patch series is still RFC and doesn't address
> shared media_device and requires all drivers to change.

I'm all for extending the functionality of the Media controller framework,
but existing known problems touching the same parts of the framework should
be fixed first.

Reviewing patches, pointing out problems and proposing improvements would
certainly help to achieve that goal.

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [PATCH v6 3/3] sound/usb: Use Media Controller API to share media resources

[Shuah Khan]

Hi Sakari,

On 12/07/2016 03:52 AM, Sakari Ailus wrote:
> Hi Shuah,
> 
> On Mon, Dec 05, 2016 at 05:38:23PM -0700, Shuah Khan wrote:
>> On 12/05/2016 04:21 PM, Laurent Pinchart wrote:
>>> Hi Shuah,
>>>
>>> On Monday 05 Dec 2016 15:44:30 Shuah Khan wrote:
>>>> On 11/30/2016 03:01 PM, Shuah Khan wrote:
>>>>> Change ALSA driver to use Media Controller API to share media resources
>>>>> with DVB, and V4L2 drivers on a AU0828 media device.
>>>>>
>>>>> Media Controller specific initialization is done after sound card is
>>>>> registered. ALSA creates Media interface and entity function graph
>>>>> nodes for Control, Mixer, PCM Playback, and PCM Capture devices.
>>>>>
>>>>> snd_usb_hw_params() will call Media Controller enable source handler
>>>>> interface to request the media resource. If resource request is granted,
>>>>> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
>>>>> returned.
>>>>>
>>>>> Media specific cleanup is done in usb_audio_disconnect().
>>>>>
>>>>> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
>>>>
>>>> Hi Takashi,
>>>>
>>>> If you are good with this patch, could you please Ack it, so Mauro
>>>> can pull it into media tree with the other two patches in this series,
>>>> when he is ready to do so.
>>>
>>> I *really* want to address the concerns raised by Sakari before pulling more 
>>> code that makes fixing the race conditions more difficult. Please, let's all 
>>> work on fixing the core code to build a stable base on which we can build 
>>> additional features. V4L2 and MC need teamwork, it's time to give the 
>>> subsystem the love it deserves.
>>>
>>
>> Hi Laurent,
>>
>> The issue Sakari brought up is specific to using devm for video_device in
>> omap3 and vsp1. I tried reproducing the problem on two different drivers
>> and couldn't on Linux 4.9-rc7.
>>
>> After sharing that with Sakari, I suggested to Sakari to pull up his patch
>> that removes the devm usage and see if he still needs all the patches in his
>> patch series. He didn't back to me on that. I also requested him to rebase on
> 
> Just to see what remains, I made a small hack to test this with omap3isp by
> just replacing the devm_() functions by their plain counterparts. The memory
> is thus never released, for there is no really a proper moment to release it
> --- something which the patchset resolves. The result is here:
> 
> <URL:http://www.retiisi.org.uk/v4l2/tmp/media-ref-dmesg.txt>

Did you test this on 4.9-rc7 without any of your other patches? If you
haven't could you please run this test with just the removing devm usage
from omap3isp?

It would be good to get a baseline on the current with just the not using
devm first and then see what needs fixing.

Also, could you please send me the complete dmesg.

thanks,
-- Shuah

> 
> What happens there is that as part of the device unbinding, the graph
> objects are unregistered (by media_device_unregister()) while streaming is
> ongoing. Their parent media device pointers are set to NULL.
> 
> Then, when the user process exists, the streaming media pipeline is stopped.
> This requires parsing of the media graph, a job which will obviously fail
> miserably and immediately: the media device is obtained from the entity
> where graph traversal is expected to begin.
> 
> Two mutexes are also acquired but they've been already destroyed at that
> point (and the memory of which would also be released now without the hack
> to test this "without devm"). There's just a single warning on this though.
> 
>> top of media dev allocator because the allocator routines he has don't address
>> the shared media device need.
> 
> I do strongly prefer fixing the existing object lifetime issues in the
> framework before extending it and thus making the problem domain more
> complex than it is already.
> 
> What I mentioned as an example of this are the other callbacks to the media
> device: presumably they do suffer from the exactly same problems as the
> enable_source() and disable_source() ones.
> 
> As drivers do refer to other entities and controls they do expose also to
> the user space, we can't really even remove entities safely at the moment.
> We should have a solution to this as well, my patchset at the moment does
> not address this.
> 
> What we do need a sound basis for the framework rather than hastily written
> improvements to support a particular device. Also, testing device removal
> with a particular device in a particular use case does not guarantee that no
> further object lifetime issues related to device removal exist, even with
> that same driver, let alone other devices.
> 
> Instead, we must consider how the frameworks and drivers manage the memory
> allocated for the various objects, what are relations of those objects and
> how they're exposed to the user space either directly or indirectly. As long
> as objects (such as the media graph objects) with different lifetimes are
> referred to from other objects without taking a reference or alternatively
> serialising code paths that may access those objects and those that remove
> them, we do have a problem.
> 
> The media graph with all the subsystem specific device nodes that are
> exported to the user space is a rather complex data structure. I don't think
> that acquiring the media graph lock in order to fix all the serialisation
> problems is the right approach here.
> 
>>
>> He also didn't respond to my response regarding the reasons for choosing
>> graph_mutex to protect enable_source and disable_source handlers.
> 
> I did. The point is that this is a partial fix and that does not properly
> address the problem of device removal. It probably does decrease of the
> probability of hitting the bug but it does not fix it.
> 
>>
>> So I am not sure how to move forward at the moment without a concrete plan
>> for Sakari's RFC series. Sakari's patch series is still RFC and doesn't address
>> shared media_device and requires all drivers to change.
> 
> I'm all for extending the functionality of the Media controller framework,
> but existing known problems touching the same parts of the framework should
> be fixed first.
> 
> Reviewing patches, pointing out problems and proposing improvements would
> certainly help to achieve that goal.
> 

Re: [PATCH v6 3/3] sound/usb: Use Media Controller API to share media resources

[Sakari Ailus]

Hi Shuah,

On Wed, Dec 07, 2016 at 01:03:59PM -0700, Shuah Khan wrote:
> Hi Sakari,
> 
> On 12/07/2016 03:52 AM, Sakari Ailus wrote:
> > Hi Shuah,
> > 
> > On Mon, Dec 05, 2016 at 05:38:23PM -0700, Shuah Khan wrote:
> >> On 12/05/2016 04:21 PM, Laurent Pinchart wrote:
> >>> Hi Shuah,
> >>>
> >>> On Monday 05 Dec 2016 15:44:30 Shuah Khan wrote:
> >>>> On 11/30/2016 03:01 PM, Shuah Khan wrote:
> >>>>> Change ALSA driver to use Media Controller API to share media resources
> >>>>> with DVB, and V4L2 drivers on a AU0828 media device.
> >>>>>
> >>>>> Media Controller specific initialization is done after sound card is
> >>>>> registered. ALSA creates Media interface and entity function graph
> >>>>> nodes for Control, Mixer, PCM Playback, and PCM Capture devices.
> >>>>>
> >>>>> snd_usb_hw_params() will call Media Controller enable source handler
> >>>>> interface to request the media resource. If resource request is granted,
> >>>>> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
> >>>>> returned.
> >>>>>
> >>>>> Media specific cleanup is done in usb_audio_disconnect().
> >>>>>
> >>>>> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
> >>>>
> >>>> Hi Takashi,
> >>>>
> >>>> If you are good with this patch, could you please Ack it, so Mauro
> >>>> can pull it into media tree with the other two patches in this series,
> >>>> when he is ready to do so.
> >>>
> >>> I *really* want to address the concerns raised by Sakari before pulling more 
> >>> code that makes fixing the race conditions more difficult. Please, let's all 
> >>> work on fixing the core code to build a stable base on which we can build 
> >>> additional features. V4L2 and MC need teamwork, it's time to give the 
> >>> subsystem the love it deserves.
> >>>
> >>
> >> Hi Laurent,
> >>
> >> The issue Sakari brought up is specific to using devm for video_device in
> >> omap3 and vsp1. I tried reproducing the problem on two different drivers
> >> and couldn't on Linux 4.9-rc7.
> >>
> >> After sharing that with Sakari, I suggested to Sakari to pull up his patch
> >> that removes the devm usage and see if he still needs all the patches in his
> >> patch series. He didn't back to me on that. I also requested him to rebase on
> > 
> > Just to see what remains, I made a small hack to test this with omap3isp by
> > just replacing the devm_() functions by their plain counterparts. The memory
> > is thus never released, for there is no really a proper moment to release it
> > --- something which the patchset resolves. The result is here:
> > 
> > <URL:http://www.retiisi.org.uk/v4l2/tmp/media-ref-dmesg.txt>
> 
> Did you test this on 4.9-rc7 without any of your other patches? If you
> haven't could you please run this test with just the removing devm usage
> from omap3isp?
> 
> It would be good to get a baseline on the current with just the not using
> devm first and then see what needs fixing.
> 
> Also, could you please send me the complete dmesg.

Updated from v4.9-rc6 to rc7 and with increased CONFIG_LOG_BUF_SHIFT. The
diff and dmesg are here:

<URL:http://www.retiisi.org.uk/v4l2/tmp/media-ref-diff2.txt>
<URL:http://www.retiisi.org.uk/v4l2/tmp/media-ref-dmesg2.txt>

-- 
Regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [PATCH v6 3/3] sound/usb: Use Media Controller API to share media resources

[Shuah Khan]

Hi Sakari,

On 12/07/2016 03:27 PM, Sakari Ailus wrote:
> Hi Shuah,
> 
> On Wed, Dec 07, 2016 at 01:03:59PM -0700, Shuah Khan wrote:
>> Hi Sakari,
>>
>> On 12/07/2016 03:52 AM, Sakari Ailus wrote:
>>> Hi Shuah,
>>>
>>> On Mon, Dec 05, 2016 at 05:38:23PM -0700, Shuah Khan wrote:
>>>> On 12/05/2016 04:21 PM, Laurent Pinchart wrote:
>>>>> Hi Shuah,
>>>>>
>>>>> On Monday 05 Dec 2016 15:44:30 Shuah Khan wrote:
>>>>>> On 11/30/2016 03:01 PM, Shuah Khan wrote:
>>>>>>> Change ALSA driver to use Media Controller API to share media resources
>>>>>>> with DVB, and V4L2 drivers on a AU0828 media device.
>>>>>>>
>>>>>>> Media Controller specific initialization is done after sound card is
>>>>>>> registered. ALSA creates Media interface and entity function graph
>>>>>>> nodes for Control, Mixer, PCM Playback, and PCM Capture devices.
>>>>>>>
>>>>>>> snd_usb_hw_params() will call Media Controller enable source handler
>>>>>>> interface to request the media resource. If resource request is granted,
>>>>>>> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
>>>>>>> returned.
>>>>>>>
>>>>>>> Media specific cleanup is done in usb_audio_disconnect().
>>>>>>>
>>>>>>> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
>>>>>>
>>>>>> Hi Takashi,
>>>>>>
>>>>>> If you are good with this patch, could you please Ack it, so Mauro
>>>>>> can pull it into media tree with the other two patches in this series,
>>>>>> when he is ready to do so.
>>>>>
>>>>> I *really* want to address the concerns raised by Sakari before pulling more 
>>>>> code that makes fixing the race conditions more difficult. Please, let's all 
>>>>> work on fixing the core code to build a stable base on which we can build 
>>>>> additional features. V4L2 and MC need teamwork, it's time to give the 
>>>>> subsystem the love it deserves.
>>>>>
>>>>
>>>> Hi Laurent,
>>>>
>>>> The issue Sakari brought up is specific to using devm for video_device in
>>>> omap3 and vsp1. I tried reproducing the problem on two different drivers
>>>> and couldn't on Linux 4.9-rc7.
>>>>
>>>> After sharing that with Sakari, I suggested to Sakari to pull up his patch
>>>> that removes the devm usage and see if he still needs all the patches in his
>>>> patch series. He didn't back to me on that. I also requested him to rebase on
>>>
>>> Just to see what remains, I made a small hack to test this with omap3isp by
>>> just replacing the devm_() functions by their plain counterparts. The memory
>>> is thus never released, for there is no really a proper moment to release it
>>> --- something which the patchset resolves. The result is here:
>>>
>>> <URL:http://www.retiisi.org.uk/v4l2/tmp/media-ref-dmesg.txt>
>>
>> Did you test this on 4.9-rc7 without any of your other patches? If you
>> haven't could you please run this test with just the removing devm usage
>> from omap3isp?
>>
>> It would be good to get a baseline on the current with just the not using
>> devm first and then see what needs fixing.
>>
>> Also, could you please send me the complete dmesg.
> 
> Updated from v4.9-rc6 to rc7 and with increased CONFIG_LOG_BUF_SHIFT. The
> diff and dmesg are here:
> 
> <URL:http://www.retiisi.org.uk/v4l2/tmp/media-ref-diff2.txt>
> <URL:http://www.retiisi.org.uk/v4l2/tmp/media-ref-dmesg2.txt>
> 

Does unbind work on this even without streaming? Could you suppress
debug messages and run unbind without streaming. It might fail. Let
me know if what you see with just unbind on this driver.

I unearthed an old Gumstix Overo that was hiding in my hardware stash
and I am setting it up to test.

thanks,
-- Shuah

Re: [PATCH v6 3/3] sound/usb: Use Media Controller API to share media resources

[Sakari Ailus]

Hi Shuah,

On Thu, Dec 08, 2016 at 07:46:03AM -0700, Shuah Khan wrote:
> Hi Sakari,
> 
> On 12/07/2016 03:27 PM, Sakari Ailus wrote:
> > Hi Shuah,
> > 
> > On Wed, Dec 07, 2016 at 01:03:59PM -0700, Shuah Khan wrote:
> >> Hi Sakari,
> >>
> >> On 12/07/2016 03:52 AM, Sakari Ailus wrote:
> >>> Hi Shuah,
> >>>
> >>> On Mon, Dec 05, 2016 at 05:38:23PM -0700, Shuah Khan wrote:
> >>>> On 12/05/2016 04:21 PM, Laurent Pinchart wrote:
> >>>>> Hi Shuah,
> >>>>>
> >>>>> On Monday 05 Dec 2016 15:44:30 Shuah Khan wrote:
> >>>>>> On 11/30/2016 03:01 PM, Shuah Khan wrote:
> >>>>>>> Change ALSA driver to use Media Controller API to share media resources
> >>>>>>> with DVB, and V4L2 drivers on a AU0828 media device.
> >>>>>>>
> >>>>>>> Media Controller specific initialization is done after sound card is
> >>>>>>> registered. ALSA creates Media interface and entity function graph
> >>>>>>> nodes for Control, Mixer, PCM Playback, and PCM Capture devices.
> >>>>>>>
> >>>>>>> snd_usb_hw_params() will call Media Controller enable source handler
> >>>>>>> interface to request the media resource. If resource request is granted,
> >>>>>>> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
> >>>>>>> returned.
> >>>>>>>
> >>>>>>> Media specific cleanup is done in usb_audio_disconnect().
> >>>>>>>
> >>>>>>> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
> >>>>>>
> >>>>>> Hi Takashi,
> >>>>>>
> >>>>>> If you are good with this patch, could you please Ack it, so Mauro
> >>>>>> can pull it into media tree with the other two patches in this series,
> >>>>>> when he is ready to do so.
> >>>>>
> >>>>> I *really* want to address the concerns raised by Sakari before pulling more 
> >>>>> code that makes fixing the race conditions more difficult. Please, let's all 
> >>>>> work on fixing the core code to build a stable base on which we can build 
> >>>>> additional features. V4L2 and MC need teamwork, it's time to give the 
> >>>>> subsystem the love it deserves.
> >>>>>
> >>>>
> >>>> Hi Laurent,
> >>>>
> >>>> The issue Sakari brought up is specific to using devm for video_device in
> >>>> omap3 and vsp1. I tried reproducing the problem on two different drivers
> >>>> and couldn't on Linux 4.9-rc7.
> >>>>
> >>>> After sharing that with Sakari, I suggested to Sakari to pull up his patch
> >>>> that removes the devm usage and see if he still needs all the patches in his
> >>>> patch series. He didn't back to me on that. I also requested him to rebase on
> >>>
> >>> Just to see what remains, I made a small hack to test this with omap3isp by
> >>> just replacing the devm_() functions by their plain counterparts. The memory
> >>> is thus never released, for there is no really a proper moment to release it
> >>> --- something which the patchset resolves. The result is here:
> >>>
> >>> <URL:http://www.retiisi.org.uk/v4l2/tmp/media-ref-dmesg.txt>
> >>
> >> Did you test this on 4.9-rc7 without any of your other patches? If you
> >> haven't could you please run this test with just the removing devm usage
> >> from omap3isp?
> >>
> >> It would be good to get a baseline on the current with just the not using
> >> devm first and then see what needs fixing.
> >>
> >> Also, could you please send me the complete dmesg.
> > 
> > Updated from v4.9-rc6 to rc7 and with increased CONFIG_LOG_BUF_SHIFT. The
> > diff and dmesg are here:
> > 
> > <URL:http://www.retiisi.org.uk/v4l2/tmp/media-ref-diff2.txt>
> > <URL:http://www.retiisi.org.uk/v4l2/tmp/media-ref-dmesg2.txt>
> > 
> 
> Does unbind work on this even without streaming? Could you suppress

It's been working as long as I remember. There are caveats though --- the
clock to the sensor is provided by the ISP, so obviously things will not
work until the sensor is rebound as well. That is unrelated to the media
framework though; just FYI.

> debug messages and run unbind without streaming. It might fail. Let
> me know if what you see with just unbind on this driver.

I don't think disabling debug messages makes a difference but sure I can
provide you a log on that. But that'll be tomorrow.

> 
> I unearthed an old Gumstix Overo that was hiding in my hardware stash
> and I am setting it up to test.

Is it supported in mainline? I have a Nokia N9...

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [PATCH v6 3/3] sound/usb: Use Media Controller API to share media resources

[Sakari Ailus]

Hi Shuah,

On Fri, Dec 09, 2016 at 01:44:07AM +0200, Sakari Ailus wrote:
> Hi Shuah,
> 
> On Thu, Dec 08, 2016 at 07:46:03AM -0700, Shuah Khan wrote:
> > Hi Sakari,
> > 
> > On 12/07/2016 03:27 PM, Sakari Ailus wrote:
> > > Hi Shuah,
> > > 
> > > On Wed, Dec 07, 2016 at 01:03:59PM -0700, Shuah Khan wrote:
> > >> Hi Sakari,
> > >>
> > >> On 12/07/2016 03:52 AM, Sakari Ailus wrote:
> > >>> Hi Shuah,
> > >>>
> > >>> On Mon, Dec 05, 2016 at 05:38:23PM -0700, Shuah Khan wrote:
> > >>>> On 12/05/2016 04:21 PM, Laurent Pinchart wrote:
> > >>>>> Hi Shuah,
> > >>>>>
> > >>>>> On Monday 05 Dec 2016 15:44:30 Shuah Khan wrote:
> > >>>>>> On 11/30/2016 03:01 PM, Shuah Khan wrote:
> > >>>>>>> Change ALSA driver to use Media Controller API to share media resources
> > >>>>>>> with DVB, and V4L2 drivers on a AU0828 media device.
> > >>>>>>>
> > >>>>>>> Media Controller specific initialization is done after sound card is
> > >>>>>>> registered. ALSA creates Media interface and entity function graph
> > >>>>>>> nodes for Control, Mixer, PCM Playback, and PCM Capture devices.
> > >>>>>>>
> > >>>>>>> snd_usb_hw_params() will call Media Controller enable source handler
> > >>>>>>> interface to request the media resource. If resource request is granted,
> > >>>>>>> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
> > >>>>>>> returned.
> > >>>>>>>
> > >>>>>>> Media specific cleanup is done in usb_audio_disconnect().
> > >>>>>>>
> > >>>>>>> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
> > >>>>>>
> > >>>>>> Hi Takashi,
> > >>>>>>
> > >>>>>> If you are good with this patch, could you please Ack it, so Mauro
> > >>>>>> can pull it into media tree with the other two patches in this series,
> > >>>>>> when he is ready to do so.
> > >>>>>
> > >>>>> I *really* want to address the concerns raised by Sakari before pulling more 
> > >>>>> code that makes fixing the race conditions more difficult. Please, let's all 
> > >>>>> work on fixing the core code to build a stable base on which we can build 
> > >>>>> additional features. V4L2 and MC need teamwork, it's time to give the 
> > >>>>> subsystem the love it deserves.
> > >>>>>
> > >>>>
> > >>>> Hi Laurent,
> > >>>>
> > >>>> The issue Sakari brought up is specific to using devm for video_device in
> > >>>> omap3 and vsp1. I tried reproducing the problem on two different drivers
> > >>>> and couldn't on Linux 4.9-rc7.
> > >>>>
> > >>>> After sharing that with Sakari, I suggested to Sakari to pull up his patch
> > >>>> that removes the devm usage and see if he still needs all the patches in his
> > >>>> patch series. He didn't back to me on that. I also requested him to rebase on
> > >>>
> > >>> Just to see what remains, I made a small hack to test this with omap3isp by
> > >>> just replacing the devm_() functions by their plain counterparts. The memory
> > >>> is thus never released, for there is no really a proper moment to release it
> > >>> --- something which the patchset resolves. The result is here:
> > >>>
> > >>> <URL:http://www.retiisi.org.uk/v4l2/tmp/media-ref-dmesg.txt>
> > >>
> > >> Did you test this on 4.9-rc7 without any of your other patches? If you
> > >> haven't could you please run this test with just the removing devm usage
> > >> from omap3isp?
> > >>
> > >> It would be good to get a baseline on the current with just the not using
> > >> devm first and then see what needs fixing.
> > >>
> > >> Also, could you please send me the complete dmesg.
> > > 
> > > Updated from v4.9-rc6 to rc7 and with increased CONFIG_LOG_BUF_SHIFT. The
> > > diff and dmesg are here:
> > > 
> > > <URL:http://www.retiisi.org.uk/v4l2/tmp/media-ref-diff2.txt>
> > > <URL:http://www.retiisi.org.uk/v4l2/tmp/media-ref-dmesg2.txt>
> > > 
> > 
> > Does unbind work on this even without streaming? Could you suppress
> 
> It's been working as long as I remember. There are caveats though --- the
> clock to the sensor is provided by the ISP, so obviously things will not
> work until the sensor is rebound as well. That is unrelated to the media
> framework though; just FYI.
> 
> > debug messages and run unbind without streaming. It might fail. Let
> > me know if what you see with just unbind on this driver.
> 
> I don't think disabling debug messages makes a difference but sure I can
> provide you a log on that. But that'll be tomorrow.

There you go:

<URL:http://www.retiisi.org.uk/v4l2/tmp/dmesg-test.txt>

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [PATCH v6 3/3] sound/usb: Use Media Controller API to share media resources

[Shuah Khan]

On 12/09/2016 06:17 AM, Sakari Ailus wrote:
> Hi Shuah,
> 
> On Fri, Dec 09, 2016 at 01:44:07AM +0200, Sakari Ailus wrote:
>> Hi Shuah,
>>
>> On Thu, Dec 08, 2016 at 07:46:03AM -0700, Shuah Khan wrote:
>>> Hi Sakari,
>>>
>>> On 12/07/2016 03:27 PM, Sakari Ailus wrote:
>>>> Hi Shuah,
>>>>
>>>> On Wed, Dec 07, 2016 at 01:03:59PM -0700, Shuah Khan wrote:
>>>>> Hi Sakari,
>>>>>
>>>>> On 12/07/2016 03:52 AM, Sakari Ailus wrote:
>>>>>> Hi Shuah,
>>>>>>
>>>>>> On Mon, Dec 05, 2016 at 05:38:23PM -0700, Shuah Khan wrote:
>>>>>>> On 12/05/2016 04:21 PM, Laurent Pinchart wrote:
>>>>>>>> Hi Shuah,
>>>>>>>>
>>>>>>>> On Monday 05 Dec 2016 15:44:30 Shuah Khan wrote:
>>>>>>>>> On 11/30/2016 03:01 PM, Shuah Khan wrote:
>>>>>>>>>> Change ALSA driver to use Media Controller API to share media resources
>>>>>>>>>> with DVB, and V4L2 drivers on a AU0828 media device.
>>>>>>>>>>
>>>>>>>>>> Media Controller specific initialization is done after sound card is
>>>>>>>>>> registered. ALSA creates Media interface and entity function graph
>>>>>>>>>> nodes for Control, Mixer, PCM Playback, and PCM Capture devices.
>>>>>>>>>>
>>>>>>>>>> snd_usb_hw_params() will call Media Controller enable source handler
>>>>>>>>>> interface to request the media resource. If resource request is granted,
>>>>>>>>>> it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is
>>>>>>>>>> returned.
>>>>>>>>>>
>>>>>>>>>> Media specific cleanup is done in usb_audio_disconnect().
>>>>>>>>>>
>>>>>>>>>> Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
>>>>>>>>>
>>>>>>>>> Hi Takashi,
>>>>>>>>>
>>>>>>>>> If you are good with this patch, could you please Ack it, so Mauro
>>>>>>>>> can pull it into media tree with the other two patches in this series,
>>>>>>>>> when he is ready to do so.
>>>>>>>>
>>>>>>>> I *really* want to address the concerns raised by Sakari before pulling more 
>>>>>>>> code that makes fixing the race conditions more difficult. Please, let's all 
>>>>>>>> work on fixing the core code to build a stable base on which we can build 
>>>>>>>> additional features. V4L2 and MC need teamwork, it's time to give the 
>>>>>>>> subsystem the love it deserves.
>>>>>>>>
>>>>>>>
>>>>>>> Hi Laurent,
>>>>>>>
>>>>>>> The issue Sakari brought up is specific to using devm for video_device in
>>>>>>> omap3 and vsp1. I tried reproducing the problem on two different drivers
>>>>>>> and couldn't on Linux 4.9-rc7.
>>>>>>>
>>>>>>> After sharing that with Sakari, I suggested to Sakari to pull up his patch
>>>>>>> that removes the devm usage and see if he still needs all the patches in his
>>>>>>> patch series. He didn't back to me on that. I also requested him to rebase on
>>>>>>
>>>>>> Just to see what remains, I made a small hack to test this with omap3isp by
>>>>>> just replacing the devm_() functions by their plain counterparts. The memory
>>>>>> is thus never released, for there is no really a proper moment to release it
>>>>>> --- something which the patchset resolves. The result is here:
>>>>>>
>>>>>> <URL:http://www.retiisi.org.uk/v4l2/tmp/media-ref-dmesg.txt>
>>>>>
>>>>> Did you test this on 4.9-rc7 without any of your other patches? If you
>>>>> haven't could you please run this test with just the removing devm usage
>>>>> from omap3isp?
>>>>>
>>>>> It would be good to get a baseline on the current with just the not using
>>>>> devm first and then see what needs fixing.
>>>>>
>>>>> Also, could you please send me the complete dmesg.
>>>>
>>>> Updated from v4.9-rc6 to rc7 and with increased CONFIG_LOG_BUF_SHIFT. The
>>>> diff and dmesg are here:
>>>>
>>>> <URL:http://www.retiisi.org.uk/v4l2/tmp/media-ref-diff2.txt>
>>>> <URL:http://www.retiisi.org.uk/v4l2/tmp/media-ref-dmesg2.txt>
>>>>
>>>
>>> Does unbind work on this even without streaming? Could you suppress
>>
>> It's been working as long as I remember. There are caveats though --- the
>> clock to the sensor is provided by the ISP, so obviously things will not
>> work until the sensor is rebound as well. That is unrelated to the media
>> framework though; just FYI.
>>
>>> debug messages and run unbind without streaming. It might fail. Let
>>> me know if what you see with just unbind on this driver.
>>
>> I don't think disabling debug messages makes a difference but sure I can
>> provide you a log on that. But that'll be tomorrow.
> 
> There you go:
> 
> <URL:http://www.retiisi.org.uk/v4l2/tmp/dmesg-test.txt>

Sakari,

I am changing the subject line and sending a new email about some oddities in
omap3remove sequence. I would have worked on cleaning these up and sending
patches, but I am still working on getting the latest gumstix booting on my
Gumstix overo board. I will spare audio folks from the email barrage of our
debug logs :)

https://github.com/gumstix/linux/tree/master

thanks,
-- Shuah