[RFC/PATCH v4 00/11] Media controller (core and V4L2)

[RFC/PATCH v4 00/11] Media controller (core and V4L2)

[Laurent Pinchart]

Hi everybody,

Here's the fourth version of the media controller patches. All comments received
so far have hopefully been incorporated.

Compared to the previous version, the patches have been rebased on top of 2.6.35
and a MEDIA_IOC_DEVICE_INFO ioctl has been added.

I won't submit a rebased version of the V4L2 API additions and OMAP3 ISP patches
right now. I will first clean up (and document) the V4L2 API additions patches,
and I will submit them as a proper RFC instead of sample code.

Laurent Pinchart (9):
  media: Media device node support
  media: Media device
  media: Entities, pads and links
  media: Media device information query
  media: Entities, pads and links enumeration
  media: Links setup
  v4l: Add a media_device pointer to the v4l2_device structure
  v4l: Make video_device inherit from media_entity
  v4l: Make v4l2_subdev inherit from media_entity

Sakari Ailus (2):
  media: Entity graph traversal
  media: Reference count and power handling

 Documentation/media-framework.txt            |  574 ++++++++++++++++++++++++
 Documentation/video4linux/v4l2-framework.txt |   72 +++-
 drivers/media/Makefile                       |    8 +-
 drivers/media/media-device.c                 |  377 ++++++++++++++++
 drivers/media/media-devnode.c                |  310 +++++++++++++
 drivers/media/media-entity.c                 |  614 ++++++++++++++++++++++++++
 drivers/media/video/v4l2-dev.c               |   35 ++-
 drivers/media/video/v4l2-device.c            |   45 ++-
 drivers/media/video/v4l2-subdev.c            |   27 ++-
 include/linux/media.h                        |  105 +++++
 include/media/media-device.h                 |   90 ++++
 include/media/media-devnode.h                |   78 ++++
 include/media/media-entity.h                 |  112 +++++
 include/media/v4l2-dev.h                     |    6 +
 include/media/v4l2-device.h                  |    2 +
 include/media/v4l2-subdev.h                  |    7 +
 16 files changed, 2440 insertions(+), 22 deletions(-)
 create mode 100644 Documentation/media-framework.txt
 create mode 100644 drivers/media/media-device.c
 create mode 100644 drivers/media/media-devnode.c
 create mode 100644 drivers/media/media-entity.c
 create mode 100644 include/linux/media.h
 create mode 100644 include/media/media-device.h
 create mode 100644 include/media/media-devnode.h
 create mode 100644 include/media/media-entity.h

-- 
Regards,

Laurent Pinchart

[RFC/PATCH v4 01/11] media: Media device node support

[Laurent Pinchart]

The media_devnode structure provides support for registering and
unregistering character devices using a dynamic major number. Reference
counting is handled internally, making device drivers easier to write
without having to solve the open/disconnect race condition issue over
and over again.

The code is based on video/v4l2-dev.c.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 drivers/media/Makefile        |    8 +-
 drivers/media/media-devnode.c |  310 +++++++++++++++++++++++++++++++++++++++++
 include/media/media-devnode.h |   78 ++++++++++
 3 files changed, 394 insertions(+), 2 deletions(-)
 create mode 100644 drivers/media/media-devnode.c
 create mode 100644 include/media/media-devnode.h

diff --git a/drivers/media/Makefile b/drivers/media/Makefile
index 499b081..c1b5938 100644
--- a/drivers/media/Makefile
+++ b/drivers/media/Makefile
@@ -2,7 +2,11 @@
 # Makefile for the kernel multimedia device drivers.
 #
 
+media-objs	:= media-devnode.o
+
+obj-$(CONFIG_MEDIA_SUPPORT)	+= media.o
+
 obj-y += common/ IR/ video/
 
-obj-$(CONFIG_VIDEO_DEV) += radio/
-obj-$(CONFIG_DVB_CORE)  += dvb/
+obj-$(CONFIG_VIDEO_DEV)		+= radio/
+obj-$(CONFIG_DVB_CORE)		+= dvb/
diff --git a/drivers/media/media-devnode.c b/drivers/media/media-devnode.c
new file mode 100644
index 0000000..3e261f1
--- /dev/null
+++ b/drivers/media/media-devnode.c
@@ -0,0 +1,310 @@
+/*
+ * Media device node
+ *
+ * Generic media device node infrastructure to register and unregister
+ * character devices using a dynamic major number and proper reference
+ * counting.
+ *
+ * Copyright 2010 Laurent Pinchart <laurent.pinchart@ideasonboard.com>
+ *
+ * Based on drivers/media/video/v4l2_dev.c code authored by
+ *
+ *	Mauro Carvalho Chehab <mchehab@infradead.org> (version 2)
+ *	Alan Cox, <alan@lxorguk.ukuu.org.uk> (version 1)
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ *
+ */
+
+#include <linux/errno.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/kmod.h>
+#include <linux/slab.h>
+#include <linux/mm.h>
+#include <linux/smp_lock.h>
+#include <linux/string.h>
+#include <linux/types.h>
+#include <linux/uaccess.h>
+#include <asm/system.h>
+
+#include <media/media-devnode.h>
+
+#define MEDIA_NUM_DEVICES	256
+#define MEDIA_NAME		"media"
+
+static dev_t media_dev_t;
+
+/*
+ *	Active devices
+ */
+static DEFINE_MUTEX(media_devnode_lock);
+static DECLARE_BITMAP(media_devnode_nums, MEDIA_NUM_DEVICES);
+
+/* Called when the last user of the media device exits. */
+static void media_devnode_release(struct device *cd)
+{
+	struct media_devnode *mdev = to_media_devnode(cd);
+
+	mutex_lock(&media_devnode_lock);
+
+	/* Delete the cdev on this minor as well */
+	cdev_del(&mdev->cdev);
+
+	/* Mark device node number as free */
+	clear_bit(mdev->minor, media_devnode_nums);
+
+	mutex_unlock(&media_devnode_lock);
+
+	/* Release media_devnode and perform other cleanups as needed. */
+	if (mdev->release)
+		mdev->release(mdev);
+}
+
+static struct class media_class = {
+	.name = MEDIA_NAME,
+};
+
+static ssize_t media_read(struct file *filp, char __user *buf,
+		size_t sz, loff_t *off)
+{
+	struct media_devnode *mdev = media_devnode_data(filp);
+
+	if (!mdev->fops->read)
+		return -EINVAL;
+	if (!media_devnode_is_registered(mdev))
+		return -EIO;
+	return mdev->fops->read(filp, buf, sz, off);
+}
+
+static ssize_t media_write(struct file *filp, const char __user *buf,
+		size_t sz, loff_t *off)
+{
+	struct media_devnode *mdev = media_devnode_data(filp);
+
+	if (!mdev->fops->write)
+		return -EINVAL;
+	if (!media_devnode_is_registered(mdev))
+		return -EIO;
+	return mdev->fops->write(filp, buf, sz, off);
+}
+
+static unsigned int media_poll(struct file *filp,
+			       struct poll_table_struct *poll)
+{
+	struct media_devnode *mdev = media_devnode_data(filp);
+
+	if (!mdev->fops->poll || !media_devnode_is_registered(mdev))
+		return DEFAULT_POLLMASK;
+	return mdev->fops->poll(filp, poll);
+}
+
+static long media_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
+{
+	struct media_devnode *mdev = media_devnode_data(filp);
+
+	if (!mdev->fops->unlocked_ioctl)
+		return -ENOTTY;
+
+	if (!media_devnode_is_registered(mdev))
+		return -EIO;
+
+	return mdev->fops->unlocked_ioctl(filp, cmd, arg);
+}
+
+/* Override for the open function */
+static int media_open(struct inode *inode, struct file *filp)
+{
+	struct media_devnode *mdev;
+	int ret;
+
+	/* Check if the media device is available. This needs to be done with
+	 * the media_devnode_lock held to prevent an open/unregister race:
+	 * without the lock, the device could be unregistered and freed between
+	 * the media_devnode_is_registered() and get_device() calls, leading to
+	 * a crash.
+	 */
+	mutex_lock(&media_devnode_lock);
+	mdev = container_of(inode->i_cdev, struct media_devnode, cdev);
+	/* return ENODEV if the media device has been removed
+	   already or if it is not registered anymore. */
+	if (!media_devnode_is_registered(mdev)) {
+		mutex_unlock(&media_devnode_lock);
+		return -ENODEV;
+	}
+	/* and increase the device refcount */
+	get_device(&mdev->dev);
+	mutex_unlock(&media_devnode_lock);
+
+	filp->private_data = mdev;
+
+	if (mdev->fops->open) {
+		ret = mdev->fops->open(filp);
+		if (ret) {
+			put_device(&mdev->dev);
+			return ret;
+		}
+	}
+
+	return 0;
+}
+
+/* Override for the release function */
+static int media_release(struct inode *inode, struct file *filp)
+{
+	struct media_devnode *mdev = media_devnode_data(filp);
+	int ret = 0;
+
+	if (mdev->fops->release)
+		mdev->fops->release(filp);
+
+	/* decrease the refcount unconditionally since the release()
+	   return value is ignored. */
+	put_device(&mdev->dev);
+	filp->private_data = NULL;
+	return ret;
+}
+
+static const struct file_operations media_devnode_fops = {
+	.owner = THIS_MODULE,
+	.read = media_read,
+	.write = media_write,
+	.open = media_open,
+	.unlocked_ioctl = media_ioctl,
+	.release = media_release,
+	.poll = media_poll,
+	.llseek = no_llseek,
+};
+
+/**
+ * media_devnode_register - register a media device node
+ * @mdev: media device node structure we want to register
+ *
+ * The registration code assigns minor numbers and registers the new device node
+ * with the kernel. An error is returned if no free minor number can be found,
+ * or if the registration of the device node fails.
+ *
+ * Zero is returned on success.
+ *
+ * Note that if the media_devnode_register call fails, the release() callback of
+ * the media_devnode structure is *not* called, so the caller is responsible for
+ * freeing any data.
+ */
+int __must_check media_devnode_register(struct media_devnode *mdev)
+{
+	int minor;
+	int ret;
+
+	/* Part 1: Find a free minor number */
+	mutex_lock(&media_devnode_lock);
+	minor = find_next_zero_bit(media_devnode_nums, 0, MEDIA_NUM_DEVICES);
+	if (minor == MEDIA_NUM_DEVICES) {
+		mutex_unlock(&media_devnode_lock);
+		printk(KERN_ERR "could not get a free minor\n");
+		return -ENFILE;
+	}
+
+	set_bit(mdev->minor, media_devnode_nums);
+	mutex_unlock(&media_devnode_lock);
+
+	mdev->minor = minor;
+
+	/* Part 2: Initialize and register the character device */
+	cdev_init(&mdev->cdev, &media_devnode_fops);
+	mdev->cdev.owner = mdev->fops->owner;
+
+	ret = cdev_add(&mdev->cdev, MKDEV(MAJOR(media_dev_t), mdev->minor), 1);
+	if (ret < 0) {
+		printk(KERN_ERR "%s: cdev_add failed\n", __func__);
+		goto error;
+	}
+
+	/* Part 3: Register the media device */
+	mdev->dev.class = &media_class;
+	mdev->dev.devt = MKDEV(MAJOR(media_dev_t), mdev->minor);
+	mdev->dev.release = media_devnode_release;
+	if (mdev->parent)
+		mdev->dev.parent = mdev->parent;
+	dev_set_name(&mdev->dev, "media%d", mdev->minor);
+	ret = device_register(&mdev->dev);
+	if (ret < 0) {
+		printk(KERN_ERR "%s: device_register failed\n", __func__);
+		goto error;
+	}
+
+	/* Part 4: Activate this minor. The char device can now be used. */
+	set_bit(MEDIA_FLAG_REGISTERED, &mdev->flags);
+
+	return 0;
+
+error:
+	cdev_del(&mdev->cdev);
+	mutex_lock(&media_devnode_lock);
+	clear_bit(mdev->minor, media_devnode_nums);
+	mutex_unlock(&media_devnode_lock);
+	return ret;
+}
+
+/**
+ * media_devnode_unregister - unregister a media device node
+ * @mdev: the device node to unregister
+ *
+ * This unregisters the passed device. Future open calls will be met with
+ * errors.
+ *
+ * This function can safely be called if the device node has never been
+ * registered or has already been unregistered.
+ */
+void media_devnode_unregister(struct media_devnode *mdev)
+{
+	/* Check if mdev was ever registered at all */
+	if (!media_devnode_is_registered(mdev))
+		return;
+
+	mutex_lock(&media_devnode_lock);
+	clear_bit(MEDIA_FLAG_REGISTERED, &mdev->flags);
+	mutex_unlock(&media_devnode_lock);
+	device_unregister(&mdev->dev);
+}
+
+/*
+ *	Initialise media for linux
+ */
+static int __init media_devnode_init(void)
+{
+	int ret;
+
+	printk(KERN_INFO "Linux media interface: v0.10\n");
+	ret = alloc_chrdev_region(&media_dev_t, 0, MEDIA_NUM_DEVICES,
+				  MEDIA_NAME);
+	if (ret < 0) {
+		printk(KERN_WARNING "media: unable to allcoate major\n");
+		return ret;
+	}
+
+	ret = class_register(&media_class);
+	if (ret < 0) {
+		unregister_chrdev_region(media_dev_t, MEDIA_NUM_DEVICES);
+		printk(KERN_WARNING "media: class_register failed\n");
+		return -EIO;
+	}
+
+	return 0;
+}
+
+static void __exit media_devnode_exit(void)
+{
+	class_unregister(&media_class);
+	unregister_chrdev_region(media_dev_t, MEDIA_NUM_DEVICES);
+}
+
+module_init(media_devnode_init)
+module_exit(media_devnode_exit)
+
+MODULE_AUTHOR("Laurent Pinchart <laurent.pinchart@ideasonboard.com>");
+MODULE_DESCRIPTION("Device node registration for media drivers");
+MODULE_LICENSE("GPL");
diff --git a/include/media/media-devnode.h b/include/media/media-devnode.h
new file mode 100644
index 0000000..25dda37
--- /dev/null
+++ b/include/media/media-devnode.h
@@ -0,0 +1,78 @@
+/*
+ * Media device node handling
+ *
+ * Copyright (C) 2010  Laurent Pinchart <laurent.pinchart@ideasonboard.com>
+ *
+ * Common functions for media-related drivers to register and unregister media
+ * device nodes.
+ */
+#ifndef _MEDIA_DEVNODE_H
+#define _MEDIA_DEVNODE_H
+
+#include <linux/poll.h>
+#include <linux/fs.h>
+#include <linux/device.h>
+#include <linux/cdev.h>
+
+/*
+ * Flag to mark the media_devnode struct as registered. Drivers must not touch
+ * this flag directly, it will be set and cleared by media_devnode_register and
+ * media_devnode_unregister.
+ */
+#define MEDIA_FLAG_REGISTERED	0
+
+struct media_file_operations {
+	struct module *owner;
+	ssize_t (*read) (struct file *, char __user *, size_t, loff_t *);
+	ssize_t (*write) (struct file *, const char __user *, size_t, loff_t *);
+	unsigned int (*poll) (struct file *, struct poll_table_struct *);
+	long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long);
+	int (*open) (struct file *);
+	int (*release) (struct file *);
+};
+
+/**
+ * struct media_devnode - Media device node
+ * @parent:	parent device
+ * @minor:	device node minor number
+ * @flags:	flags, combination of the MEDIA_FLAG_* constants
+ *
+ * This structure represents a media-related device node.
+ *
+ * The @parent is a physical device. It must be set by core or device drivers
+ * before registering the node.
+ */
+struct media_devnode {
+	/* device ops */
+	const struct media_file_operations *fops;
+
+	/* sysfs */
+	struct device dev;		/* media device */
+	struct cdev cdev;		/* character device */
+	struct device *parent;		/* device parent */
+
+	/* device info */
+	int minor;
+	unsigned long flags;		/* Use bitops to access flags */
+
+	/* callbacks */
+	void (*release)(struct media_devnode *mdev);
+};
+
+/* dev to media_devnode */
+#define to_media_devnode(cd) container_of(cd, struct media_devnode, dev)
+
+int __must_check media_devnode_register(struct media_devnode *mdev);
+void media_devnode_unregister(struct media_devnode *mdev);
+
+static inline struct media_devnode *media_devnode_data(struct file *filp)
+{
+	return filp->private_data;
+}
+
+static inline int media_devnode_is_registered(struct media_devnode *mdev)
+{
+	return test_bit(MEDIA_FLAG_REGISTERED, &mdev->flags);
+}
+
+#endif /* _MEDIA_DEVNODE_H */
-- 
1.7.1

[RFC/PATCH v4 02/11] media: Media device

[Laurent Pinchart]

The media_device structure abstracts functions common to all kind of
media devices (v4l2, dvb, alsa, ...). It manages media entities and
offers a userspace API to discover and configure the media device
internal topology.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 Documentation/media-framework.txt |   91 ++++++++++++++++++++++++++++++++++
 drivers/media/Makefile            |    2 +-
 drivers/media/media-device.c      |   98 +++++++++++++++++++++++++++++++++++++
 include/media/media-device.h      |   64 ++++++++++++++++++++++++
 4 files changed, 254 insertions(+), 1 deletions(-)
 create mode 100644 Documentation/media-framework.txt
 create mode 100644 drivers/media/media-device.c
 create mode 100644 include/media/media-device.h

diff --git a/Documentation/media-framework.txt b/Documentation/media-framework.txt
new file mode 100644
index 0000000..89dc7ad
--- /dev/null
+++ b/Documentation/media-framework.txt
@@ -0,0 +1,91 @@
+Linux kernel media framework
+============================
+
+This document describes the Linux kernel media framework, its data structures,
+functions and their usage.
+
+
+Introduction
+------------
+
+Media devices increasingly handle multiple related functions. Many USB cameras
+include microphones, video capture hardware can also output video, or SoC
+camera interfaces also perform memory-to-memory operations similar to video
+codecs.
+
+Independent functions, even when implemented in the same hardware, can be
+modeled by separate devices. A USB camera with a microphone will be presented
+to userspace applications as V4L2 and ALSA capture devices. The devices
+relationships (when using a webcam, end-users shouldn't have to manually
+select the associated USB microphone), while not made available directly to
+applications by the drivers, can usually be retrieved from sysfs.
+
+With more and more advanced SoC devices being introduced, the current approach
+will not scale. Device topologies are getting increasingly complex and can't
+always be represented by a tree structure. Hardware blocks are shared between
+different functions, creating dependencies between seemingly unrelated
+devices.
+
+Kernel abstraction APIs such as V4L2 and ALSA provide means for applications
+to access hardware parameters. As newer hardware expose an increasingly high
+number of those parameters, drivers need to guess what applications really
+require based on limited information, thereby implementing policies that
+belong to userspace.
+
+The media kernel API aims at solving those problems.
+
+
+Media device
+------------
+
+A media device is represented by a struct media_device instance, defined in
+include/media/media-device.h. Allocation of the structure is handled by the
+media device driver, usually by embedding the media_device instance in a
+larger driver-specific structure.
+
+Drivers register media device instances by calling
+
+	media_device_register(struct media_device *mdev);
+
+The caller is responsible for initializing the media_device structure before
+registration. The following fields must be set:
+
+ - dev must point to the parent device (usually a pci_dev, usb_interface or
+   platform_device instance).
+
+ - model must be filled with the device model name as a NUL-terminated UTF-8
+   string. The device/model revision must not be stored in this field.
+
+The following fields are optional:
+
+ - serial is a unique serial number stored as an ASCII string. The string must
+   be NUL-terminated unless exactly 32 characters long. This allows storing
+   GUIDs in a text form. If the hardware doesn't provide a unique serial
+   number this field must be left empty.
+
+ - bus_info represents the location of the device in the system as a
+   NUL-terminated ASCII string. For PCI/PCIe devices bus_info must be set to
+   "PCI:" (or "PCIe:") followed by the value of pci_name(). For USB devices,
+   the usb_make_path() function must be used. This field is used by
+   applications to distinguish between otherwise identical devices that don't
+   provide a serial number.
+
+ - device_version is the hardware device version number in a driver-specific
+   format. When possible the version should be formatted with the
+   KERNEL_VERSION macro.
+
+ - driver_version is formatted with the KERNEL_VERSION macro. The version
+   minor must be incremented when new features are added to the userspace API
+   without breaking binary compatibility. The version major must be
+   incremented when binary compatibility is broken.
+
+Upon successful registration a character device named media[0-9]+ is created.
+The device major and minor numbers are dynamic. The model name is exported as
+a sysfs attribute.
+
+Drivers unregister media device instances by calling
+
+	media_device_unregister(struct media_device *mdev);
+
+Unregistering a media device that hasn't been registered is *NOT* safe.
+
diff --git a/drivers/media/Makefile b/drivers/media/Makefile
index c1b5938..f8d8dcb 100644
--- a/drivers/media/Makefile
+++ b/drivers/media/Makefile
@@ -2,7 +2,7 @@
 # Makefile for the kernel multimedia device drivers.
 #
 
-media-objs	:= media-devnode.o
+media-objs	:= media-device.o media-devnode.o
 
 obj-$(CONFIG_MEDIA_SUPPORT)	+= media.o
 
diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
new file mode 100644
index 0000000..781c641
--- /dev/null
+++ b/drivers/media/media-device.c
@@ -0,0 +1,98 @@
+/*
+ *  Media device support.
+ *
+ *  Copyright (C) 2010  Laurent Pinchart <laurent.pinchart@ideasonboard.com>
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; either version 2 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+ */
+
+#include <linux/types.h>
+#include <linux/ioctl.h>
+
+#include <media/media-device.h>
+#include <media/media-devnode.h>
+
+static const struct media_file_operations media_device_fops = {
+	.owner = THIS_MODULE,
+};
+
+/* -----------------------------------------------------------------------------
+ * sysfs
+ */
+
+static ssize_t show_model(struct device *cd,
+			  struct device_attribute *attr, char *buf)
+{
+	struct media_device *mdev = to_media_device(to_media_devnode(cd));
+
+	return sprintf(buf, "%.*s\n", (int)sizeof(mdev->model), mdev->model);
+}
+
+static DEVICE_ATTR(model, S_IRUGO, show_model, NULL);
+
+/* -----------------------------------------------------------------------------
+ * Registration/unregistration
+ */
+
+static void media_device_release(struct media_devnode *mdev)
+{
+}
+
+/**
+ * media_device_register - register a media device
+ * @mdev:	The media device
+ *
+ * The caller is responsible for initializing the media device before
+ * registration. The following fields must be set:
+ *
+ * - dev must point to the parent device
+ * - model must be filled with the device model name
+ */
+int __must_check media_device_register(struct media_device *mdev)
+{
+	int ret;
+
+	if (WARN_ON(mdev->dev == NULL || mdev->model[0] == 0))
+		return 0;
+
+	/* Register the device node. */
+	mdev->devnode.fops = &media_device_fops;
+	mdev->devnode.parent = mdev->dev;
+	mdev->devnode.release = media_device_release;
+	ret = media_devnode_register(&mdev->devnode);
+	if (ret < 0)
+		return ret;
+
+	ret = device_create_file(&mdev->devnode.dev, &dev_attr_model);
+	if (ret < 0) {
+		media_devnode_unregister(&mdev->devnode);
+		return ret;
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(media_device_register);
+
+/**
+ * media_device_unregister - unregister a media device
+ * @mdev:	The media device
+ *
+ */
+void media_device_unregister(struct media_device *mdev)
+{
+	device_remove_file(&mdev->devnode.dev, &dev_attr_model);
+	media_devnode_unregister(&mdev->devnode);
+}
+EXPORT_SYMBOL_GPL(media_device_unregister);
diff --git a/include/media/media-device.h b/include/media/media-device.h
new file mode 100644
index 0000000..4fe949e
--- /dev/null
+++ b/include/media/media-device.h
@@ -0,0 +1,64 @@
+/*
+ *  Media device support header.
+ *
+ *  Copyright (C) 2010  Laurent Pinchart <laurent.pinchart@ideasonboard.com>
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; either version 2 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+ */
+
+#ifndef _MEDIA_DEVICE_H
+#define _MEDIA_DEVICE_H
+
+#include <linux/device.h>
+#include <linux/list.h>
+
+#include <media/media-devnode.h>
+
+/**
+ * struct media_device - Media device
+ * @dev:	Parent device
+ * @devnode:	Media device node
+ * @model:	Device model name
+ * @serial:	Device serial number (optional)
+ * @bus_info:	Unique and stable device location identifier
+ * @device_version: Hardware device version
+ * @driver_version: Device driver version
+ *
+ * This structure represents an abstract high-level media device. It allows easy
+ * access to entities and provides basic media device-level support. The
+ * structure can be allocated directly or embedded in a larger structure.
+ *
+ * The parent @dev is a physical device. It must be set before registering the
+ * media device.
+ *
+ * @model is a descriptive model name exported through sysfs. It doesn't have to
+ * be unique.
+ */
+struct media_device {
+	/* dev->driver_data points to this struct. */
+	struct device *dev;
+	struct media_devnode devnode;
+
+	u8 model[32];
+	u8 serial[32];
+	u8 bus_info[32];
+	u32 device_version;
+	u32 driver_version;
+};
+
+int __must_check media_device_register(struct media_device *mdev);
+void media_device_unregister(struct media_device *mdev);
+
+#endif
-- 
1.7.1

[RFC/PATCH v4 03/11] media: Entities, pads and links

[Laurent Pinchart]

As video hardware pipelines become increasingly complex and
configurable, the current hardware description through v4l2 subdevices
reaches its limits. In addition to enumerating and configuring
subdevices, video camera drivers need a way to discover and modify at
runtime how those subdevices are connected. This is done through new
elements called entities, pads and links.

An entity is a basic media hardware building block. It can correspond to
a large variety of logical blocks such as physical hardware devices
(CMOS sensor for instance), logical hardware devices (a building block
in a System-on-Chip image processing pipeline), DMA channels or physical
connectors.

A pad is a connection endpoint through which an entity can interact with
other entities. Data (not restricted to video) produced by an entity
flows from the entity's output to one or more entity inputs. Pads should
not be confused with physical pins at chip boundaries.

A link is a point-to-point oriented connection between two pads, either
on the same entity or on different entities. Data flows from a source
pad to a sink pad.

Links are stored in the source entity. To make backwards graph walk
faster, a copy of all links is also stored in the sink entity. The copy
is known as a backlink and is only used to help graph traversal.

The entity API is made of three functions:

- media_entity_init() initializes an entity. The caller must provide an
array of pads as well as an estimated number of links. The links array
is allocated dynamically and will be reallocated if it grows beyond the
initial estimate.

- media_entity_cleanup() frees resources allocated for an entity. It
must be called during the cleanup phase after unregistering the entity
and before freeing it.

- media_entity_create_link() creates a link between two entities. An
entry in the link array of each entity is allocated and stores pointers
to source and sink pads.

When a media device is unregistered, all its entities are unregistered
automatically.

The code is based on Hans Verkuil <hverkuil@xs4all.nl> initial work.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
---
 Documentation/media-framework.txt |  149 +++++++++++++++++++++++++++++++++++++
 drivers/media/Makefile            |    2 +-
 drivers/media/media-device.c      |   53 +++++++++++++
 drivers/media/media-entity.c      |  145 ++++++++++++++++++++++++++++++++++++
 include/media/media-device.h      |   19 +++++
 include/media/media-entity.h      |   96 ++++++++++++++++++++++++
 6 files changed, 463 insertions(+), 1 deletions(-)
 create mode 100644 drivers/media/media-entity.c
 create mode 100644 include/media/media-entity.h

diff --git a/Documentation/media-framework.txt b/Documentation/media-framework.txt
index 89dc7ad..35d74e4 100644
--- a/Documentation/media-framework.txt
+++ b/Documentation/media-framework.txt
@@ -35,6 +35,30 @@ belong to userspace.
 The media kernel API aims at solving those problems.
 
 
+Abstract media device model
+---------------------------
+
+Discovering a device internal topology, and configuring it at runtime, is one
+of the goals of the media framework. To achieve this, hardware devices are
+modeled as an oriented graph of building blocks called entities connected
+through pads.
+
+An entity is a basic media hardware building block. It can correspond to
+a large variety of logical blocks such as physical hardware devices
+(CMOS sensor for instance), logical hardware devices (a building block
+in a System-on-Chip image processing pipeline), DMA channels or physical
+connectors.
+
+A pad is a connection endpoint through which an entity can interact with
+other entities. Data (not restricted to video) produced by an entity
+flows from the entity's output to one or more entity inputs. Pads should
+not be confused with physical pins at chip boundaries.
+
+A link is a point-to-point oriented connection between two pads, either
+on the same entity or on different entities. Data flows from a source
+pad to a sink pad.
+
+
 Media device
 ------------
 
@@ -89,3 +113,128 @@ Drivers unregister media device instances by calling
 
 Unregistering a media device that hasn't been registered is *NOT* safe.
 
+
+Entities, pads and links
+------------------------
+
+- Entities
+
+Entities are represented by a struct media_entity instance, defined in
+include/media/media-entity.h. The structure is usually embedded into a
+higher-level structure, such as a v4l2_subdev or video_device instance,
+although drivers can allocate entities directly.
+
+Drivers initialize entities by calling
+
+	media_entity_init(struct media_entity *entity, u16 num_pads,
+			  struct media_pad *pads, u16 extra_links);
+
+The media_entity name, type, flags, revision and group_id fields can be
+initialized before or after calling media_entity_init. Entities embedded in
+higher-level standard structures can have some of those fields set by the
+higher-level framework.
+
+As the number of pads is known in advance, the pads array is not allocated
+dynamically but is managed by the entity driver. Most drivers will embed the
+pads array in a driver-specific structure, avoiding dynamic allocation.
+
+Drivers must set the direction of every pad in the pads array before calling
+media_entity_init. The function will initialize the other pads fields.
+
+Unlike the number of pads, the total number of links isn't always known in
+advance by the entity driver. As an initial estimate, media_entity_init
+pre-allocates a number of links equal to the number of pads plus an optional
+number of extra links. The links array will be reallocated if it grows beyond
+the initial estimate.
+
+Drivers register entities with a media device by calling
+
+	media_device_register_entity(struct media_device *mdev,
+				     struct media_entity *entity);
+
+When registered the entity is assigned an ID. Entity IDs are positive integers
+and are guaranteed to be unique in the context of the media device. The
+framework doesn't guarantee that IDs will always be continuous.
+
+Drivers unregister entities by calling
+
+	media_device_unregister_entity(struct media_entity *entity);
+
+Unregistering an entity will not change the IDs of the other entities, and the
+ID will never be reused for a newly registered entity.
+
+When a media device is unregistered, all its entities are unregistered
+automatically. No manual entities unregistration is then required.
+
+Drivers free resources associated with an entity by calling
+
+	media_entity_cleanup(struct media_entity *entity);
+
+This function must be called during the cleanup phase after unregistering the
+entity. Note that the media_entity instance itself must be freed explicitly by
+the driver if required.
+
+Entities have flags that describe the entity capabilities and state.
+
+	MEDIA_ENTITY_FLAG_DEFAULT indicates the default entity for a given
+	type. This can be used to report the default audio and video devices
+	or the default camera sensor.
+
+Logical entity groups can be defined by setting the group ID of all member
+entities to the same non-zero value. An entity group serves no purpose in the
+kernel, but is reported to userspace during entities enumeration. The group_id
+field belongs to the media device driver and must not by touched by entity
+drivers.
+
+Media device drivers should define groups if several entities are logically
+bound together. Example usages include reporting
+
+	- ALSA, VBI and video nodes that carry the same media stream
+	- lens and flash controllers associated with a sensor
+
+- Pads
+
+Pads are represented by a struct media_pad instance, defined in
+include/media/media-entity.h. Each entity stores its pads in a pads array
+managed by the entity driver. Drivers usually embed the array in a
+driver-specific structure.
+
+Pads are identified by their entity and their 0-based index in the pads array.
+Both information are stored in the media_pad structure, making the media_pad
+pointer the canonical way to store and pass link references.
+
+Pads have flags that describe the pad capabilities and state.
+
+	MEDIA_PAD_FLAG_INPUT indicates that the pad supports sinking data.
+	MEDIA_PAD_FLAG_OUTPUT indicates that the pad supports sourcing data.
+
+One and only one of MEDIA_PAD_FLAG_INPUT and MEDIA_PAD_FLAG_OUTPUT must be set
+for each pad.
+
+- Links
+
+Links are represented by a struct media_link instance, defined in
+include/media/media-entity.h. Each entity stores all links originating at or
+targetting any of its pads in a links array. A given link is thus stored
+twice, once in the source entity and once in the target entity. The array is
+pre-allocated and grows dynamically as needed.
+
+Drivers create links by calling
+
+	media_entity_create_link(struct media_entity *source, u16 source_pad,
+				 struct media_entity *sink,   u16 sink_pad,
+				 u32 flags);
+
+An entry in the link array of each entity is allocated and stores pointers
+to source and sink pads.
+
+Links have flags that describe the link capabilities and state.
+
+	MEDIA_LINK_FLAG_ACTIVE indicates that the link is active and can be
+	used to transfer media data. When two or more links target a sink pad,
+	only one of them can be active at a time.
+	MEDIA_LINK_FLAG_IMMUTABLE indicates that the link active state can't
+	be modified at runtime. If MEDIA_LINK_FLAG_IMMUTABLE is set, then
+	MEDIA_LINK_FLAG_ACTIVE must also be set since an immutable link is
+	always active.
+
diff --git a/drivers/media/Makefile b/drivers/media/Makefile
index f8d8dcb..a425581 100644
--- a/drivers/media/Makefile
+++ b/drivers/media/Makefile
@@ -2,7 +2,7 @@
 # Makefile for the kernel multimedia device drivers.
 #
 
-media-objs	:= media-device.o media-devnode.o
+media-objs	:= media-device.o media-devnode.o media-entity.o
 
 obj-$(CONFIG_MEDIA_SUPPORT)	+= media.o
 
diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
index 781c641..eeb002e 100644
--- a/drivers/media/media-device.c
+++ b/drivers/media/media-device.c
@@ -23,6 +23,7 @@
 
 #include <media/media-device.h>
 #include <media/media-devnode.h>
+#include <media/media-entity.h>
 
 static const struct media_file_operations media_device_fops = {
 	.owner = THIS_MODULE,
@@ -67,6 +68,10 @@ int __must_check media_device_register(struct media_device *mdev)
 	if (WARN_ON(mdev->dev == NULL || mdev->model[0] == 0))
 		return 0;
 
+	mdev->entity_id = 1;
+	INIT_LIST_HEAD(&mdev->entities);
+	spin_lock_init(&mdev->lock);
+
 	/* Register the device node. */
 	mdev->devnode.fops = &media_device_fops;
 	mdev->devnode.parent = mdev->dev;
@@ -92,7 +97,55 @@ EXPORT_SYMBOL_GPL(media_device_register);
  */
 void media_device_unregister(struct media_device *mdev)
 {
+	struct media_entity *entity;
+	struct media_entity *next;
+
+	list_for_each_entry_safe(entity, next, &mdev->entities, list)
+		media_device_unregister_entity(entity);
+
 	device_remove_file(&mdev->devnode.dev, &dev_attr_model);
 	media_devnode_unregister(&mdev->devnode);
 }
 EXPORT_SYMBOL_GPL(media_device_unregister);
+
+/**
+ * media_device_register_entity - Register an entity with a media device
+ * @mdev:	The media device
+ * @entity:	The entity
+ */
+int __must_check media_device_register_entity(struct media_device *mdev,
+					      struct media_entity *entity)
+{
+	/* Warn if we apparently re-register an entity */
+	WARN_ON(entity->parent != NULL);
+	entity->parent = mdev;
+
+	spin_lock(&mdev->lock);
+	entity->id = mdev->entity_id++;
+	list_add_tail(&entity->list, &mdev->entities);
+	spin_unlock(&mdev->lock);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(media_device_register_entity);
+
+/**
+ * media_device_unregister_entity - Unregister an entity
+ * @entity:	The entity
+ *
+ * If the entity has never been registered this function will return
+ * immediately.
+ */
+void media_device_unregister_entity(struct media_entity *entity)
+{
+	struct media_device *mdev = entity->parent;
+
+	if (mdev == NULL)
+		return;
+
+	spin_lock(&mdev->lock);
+	list_del(&entity->list);
+	spin_unlock(&mdev->lock);
+	entity->parent = NULL;
+}
+EXPORT_SYMBOL_GPL(media_device_unregister_entity);
diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
new file mode 100644
index 0000000..541063b
--- /dev/null
+++ b/drivers/media/media-entity.c
@@ -0,0 +1,145 @@
+/*
+ *  Media Entity support
+ *
+ *  Copyright (C) 2010 Laurent Pinchart <laurent.pinchart@ideasonboard.com>
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; either version 2 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+ */
+
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <media/media-entity.h>
+
+/**
+ * media_entity_init - Initialize a media entity
+ *
+ * @num_pads: Total number of input and output pads.
+ * @extra_links: Initial estimate of the number of extra links.
+ * @pads: Array of 'num_pads' pads.
+ *
+ * The total number of pads is an intrinsic property of entities known by the
+ * entity driver, while the total number of links depends on hardware design
+ * and is an extrinsic property unknown to the entity driver. However, in most
+ * use cases the entity driver can guess the number of links which can safely
+ * be assumed to be equal to or larger than the number of pads.
+ *
+ * For those reasons the links array can be preallocated based on the entity
+ * driver guess and will be reallocated later if extra links need to be
+ * created.
+ *
+ * This function allocates a links array with enough space to hold at least
+ * 'num_pads' + 'extra_links' elements. The media_entity::max_links field will
+ * be set to the number of allocated elements.
+ *
+ * The pads array is managed by the entity driver and passed to
+ * media_entity_init() where its pointer will be stored in the entity structure.
+ */
+int
+media_entity_init(struct media_entity *entity, u16 num_pads,
+		  struct media_pad *pads, u16 extra_links)
+{
+	struct media_link *links;
+	unsigned int max_links = num_pads + extra_links;
+	unsigned int i;
+
+	links = kzalloc(max_links * sizeof(links[0]), GFP_KERNEL);
+	if (links == NULL)
+		return -ENOMEM;
+
+	entity->group_id = 0;
+	entity->max_links = max_links;
+	entity->num_links = 0;
+	entity->num_backlinks = 0;
+	entity->num_pads = num_pads;
+	entity->pads = pads;
+	entity->links = links;
+
+	for (i = 0; i < num_pads; i++) {
+		pads[i].entity = entity;
+		pads[i].index = i;
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL(media_entity_init);
+
+void
+media_entity_cleanup(struct media_entity *entity)
+{
+	kfree(entity->links);
+}
+EXPORT_SYMBOL(media_entity_cleanup);
+
+static struct media_link *media_entity_add_link(struct media_entity *entity)
+{
+	if (entity->num_links >= entity->max_links) {
+		struct media_link *links = entity->links;
+		unsigned int max_links = entity->max_links + 2;
+		unsigned int i;
+
+		links = krealloc(links, max_links * sizeof(*links), GFP_KERNEL);
+		if (links == NULL)
+			return NULL;
+
+		for (i = 0; i < entity->num_links; i++)
+			links[i].reverse->reverse = &links[i];
+
+		entity->max_links = max_links;
+		entity->links = links;
+	}
+
+	return &entity->links[entity->num_links++];
+}
+
+int
+media_entity_create_link(struct media_entity *source, u16 source_pad,
+			 struct media_entity *sink, u16 sink_pad, u32 flags)
+{
+	struct media_link *link;
+	struct media_link *backlink;
+
+	BUG_ON(source == NULL || sink == NULL);
+	BUG_ON(source_pad >= source->num_pads);
+	BUG_ON(sink_pad >= sink->num_pads);
+
+	link = media_entity_add_link(source);
+	if (link == NULL)
+		return -ENOMEM;
+
+	link->source = &source->pads[source_pad];
+	link->sink = &sink->pads[sink_pad];
+	link->flags = flags;
+
+	/* Create the backlink. Backlinks are used to help graph traversal and
+	 * are not reported to userspace.
+	 */
+	backlink = media_entity_add_link(sink);
+	if (backlink == NULL) {
+		source->num_links--;
+		return -ENOMEM;
+	}
+
+	backlink->source = &source->pads[source_pad];
+	backlink->sink = &sink->pads[sink_pad];
+	backlink->flags = flags;
+
+	link->reverse = backlink;
+	backlink->reverse = link;
+
+	sink->num_backlinks++;
+
+	return 0;
+}
+EXPORT_SYMBOL(media_entity_create_link);
diff --git a/include/media/media-device.h b/include/media/media-device.h
index 4fe949e..7e2dac2 100644
--- a/include/media/media-device.h
+++ b/include/media/media-device.h
@@ -23,8 +23,10 @@
 
 #include <linux/device.h>
 #include <linux/list.h>
+#include <linux/spinlock.h>
 
 #include <media/media-devnode.h>
+#include <media/media-entity.h>
 
 /**
  * struct media_device - Media device
@@ -35,6 +37,9 @@
  * @bus_info:	Unique and stable device location identifier
  * @device_version: Hardware device version
  * @driver_version: Device driver version
+ * @entity_id:	ID of the next entity to be registered
+ * @entities:	List of registered entities
+ * @lock:	Entities list lock
  *
  * This structure represents an abstract high-level media device. It allows easy
  * access to entities and provides basic media device-level support. The
@@ -56,9 +61,23 @@ struct media_device {
 	u8 bus_info[32];
 	u32 device_version;
 	u32 driver_version;
+
+	u32 entity_id;
+	struct list_head entities;
+
+	/* Protects the entities list */
+	spinlock_t lock;
 };
 
 int __must_check media_device_register(struct media_device *mdev);
 void media_device_unregister(struct media_device *mdev);
 
+int __must_check media_device_register_entity(struct media_device *mdev,
+					      struct media_entity *entity);
+void media_device_unregister_entity(struct media_entity *entity);
+
+/* Iterate over all entities. */
+#define media_device_for_each_entity(entity, mdev)			\
+	list_for_each_entry(entity, &(mdev)->entities, list)
+
 #endif
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
new file mode 100644
index 0000000..32bb20a
--- /dev/null
+++ b/include/media/media-entity.h
@@ -0,0 +1,96 @@
+#ifndef _MEDIA_ENTITY_H
+#define _MEDIA_ENTITY_H
+
+#include <linux/list.h>
+
+#define MEDIA_ENTITY_TYPE_SHIFT			16
+#define MEDIA_ENTITY_TYPE_MASK			0x00ff0000
+#define MEDIA_ENTITY_SUBTYPE_MASK		0x0000ffff
+
+#define MEDIA_ENTITY_TYPE_NODE			(1 << MEDIA_ENTITY_TYPE_SHIFT)
+#define MEDIA_ENTITY_TYPE_NODE_V4L		(MEDIA_ENTITY_TYPE_NODE + 1)
+#define MEDIA_ENTITY_TYPE_NODE_FB		(MEDIA_ENTITY_TYPE_NODE + 2)
+#define MEDIA_ENTITY_TYPE_NODE_ALSA		(MEDIA_ENTITY_TYPE_NODE + 3)
+#define MEDIA_ENTITY_TYPE_NODE_DVB		(MEDIA_ENTITY_TYPE_NODE + 4)
+
+#define MEDIA_ENTITY_TYPE_SUBDEV		(2 << MEDIA_ENTITY_TYPE_SHIFT)
+#define MEDIA_ENTITY_TYPE_SUBDEV_SENSOR		(MEDIA_ENTITY_TYPE_SUBDEV + 1)
+#define MEDIA_ENTITY_TYPE_SUBDEV_FLASH		(MEDIA_ENTITY_TYPE_SUBDEV + 2)
+#define MEDIA_ENTITY_TYPE_SUBDEV_LENS		(MEDIA_ENTITY_TYPE_SUBDEV + 3)
+
+#define MEDIA_ENTITY_FLAG_DEFAULT		(1 << 0)
+
+#define MEDIA_LINK_FLAG_ACTIVE			(1 << 0)
+#define MEDIA_LINK_FLAG_IMMUTABLE		(1 << 1)
+
+#define MEDIA_PAD_FLAG_INPUT			(1 << 0)
+#define MEDIA_PAD_FLAG_OUTPUT			(1 << 1)
+
+struct media_link {
+	struct media_pad *source;	/* Source pad */
+	struct media_pad *sink;		/* Sink pad  */
+	struct media_link *reverse;	/* Link in the reverse direction */
+	unsigned long flags;		/* Link flags (MEDIA_LINK_FLAG_*) */
+};
+
+struct media_pad {
+	struct media_entity *entity;	/* Entity this pad belongs to */
+	u16 index;			/* Pad index in the entity pads array */
+	unsigned long flags;		/* Pad flags (MEDIA_PAD_FLAG_*) */
+};
+
+struct media_entity {
+	struct list_head list;
+	struct media_device *parent;	/* Media device this entity belongs to*/
+	u32 id;				/* Entity ID, unique in the parent media
+					 * device context */
+	const char *name;		/* Entity name */
+	u32 type;			/* Entity type (MEDIA_ENTITY_TYPE_*) */
+	u32 revision;			/* Entity revision, driver specific */
+	unsigned long flags;		/* Entity flags (MEDIA_ENTITY_FLAG_*) */
+	u32 group_id;			/* Entity group ID */
+
+	u16 num_pads;			/* Number of input and output pads */
+	u16 num_links;			/* Number of existing links, both active
+					 * and inactive */
+	u16 num_backlinks;		/* Number of backlinks */
+	u16 max_links;			/* Maximum number of links */
+
+	struct media_pad *pads;		/* Pads array (num_pads elements) */
+	struct media_link *links;	/* Links array (max_links elements)*/
+
+	union {
+		/* Node specifications */
+		struct {
+			u32 major;
+			u32 minor;
+		} v4l;
+		struct {
+			u32 major;
+			u32 minor;
+		} fb;
+		int alsa;
+		int dvb;
+
+		/* Sub-device specifications */
+		/* Nothing needed yet */
+	};
+};
+
+static inline u32 media_entity_type(struct media_entity *entity)
+{
+	return entity->type & MEDIA_ENTITY_TYPE_MASK;
+}
+
+static inline u32 media_entity_subtype(struct media_entity *entity)
+{
+	return entity->type & MEDIA_ENTITY_SUBTYPE_MASK;
+}
+
+int media_entity_init(struct media_entity *entity, u16 num_pads,
+		struct media_pad *pads, u16 extra_links);
+void media_entity_cleanup(struct media_entity *entity);
+int media_entity_create_link(struct media_entity *source, u16 source_pad,
+		struct media_entity *sink, u16 sink_pad, u32 flags);
+
+#endif
-- 
1.7.1

[RFC/PATCH v4 04/11] media: Entity graph traversal

[Laurent Pinchart]

From: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>

Add media entity graph traversal. The traversal follows active links by
depth first. Traversing graph backwards is prevented by comparing the next
possible entity in the graph with the previous one. Multiply connected
graphs are thus not supported.

Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Vimarsh Zutshi <vimarsh.zutshi@nokia.com>
---
 Documentation/media-framework.txt |   40 +++++++++++++
 drivers/media/media-entity.c      |  115 +++++++++++++++++++++++++++++++++++++
 include/media/media-entity.h      |   15 +++++
 3 files changed, 170 insertions(+), 0 deletions(-)

diff --git a/Documentation/media-framework.txt b/Documentation/media-framework.txt
index 35d74e4..a599824 100644
--- a/Documentation/media-framework.txt
+++ b/Documentation/media-framework.txt
@@ -238,3 +238,43 @@ Links have flags that describe the link capabilities and state.
 	MEDIA_LINK_FLAG_ACTIVE must also be set since an immutable link is
 	always active.
 
+
+Graph traversal
+---------------
+
+The media framework provides APIs to iterate over entities in a graph.
+
+To iterate over all entities belonging to a media device, drivers can use the
+media_device_for_each_entity macro, defined in include/media/media-device.h.
+
+	struct media_entity *entity;
+
+	media_device_for_each_entity(entity, mdev) {
+		/* entity will point to each entity in turn */
+		...
+	}
+
+Drivers might also need to iterate over all entities in a graph that can be
+reached only through active links starting at a given entity. The media
+framework provides a depth-first graph traversal API for that purpose.
+
+Note that graphs with cycles (whether directed or undirected) are *NOT*
+supported by the graph traversal API.
+
+Drivers initiate a graph traversal by calling
+
+	media_entity_graph_walk_start(struct media_entity_graph *graph,
+				      struct media_entity *entity);
+
+The graph structure, provided by the caller, is initialized to start graph
+traversal at the given entity.
+
+Drivers can then retrieve the next entity by calling
+
+	media_entity_graph_walk_next(struct media_entity_graph *graph);
+
+When the graph traversal is complete the function will return NULL.
+
+Graph traversal can be interrupted at any moment. No cleanup function call is
+required and the graph structure can be freed normally.
+
diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index 541063b..c277c18 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -82,6 +82,121 @@ media_entity_cleanup(struct media_entity *entity)
 }
 EXPORT_SYMBOL(media_entity_cleanup);
 
+/* -----------------------------------------------------------------------------
+ * Graph traversal
+ */
+
+static struct media_entity *
+media_entity_other(struct media_entity *entity, struct media_link *link)
+{
+	if (link->source->entity == entity)
+		return link->sink->entity;
+	else
+		return link->source->entity;
+}
+
+/* push an entity to traversal stack */
+static void stack_push(struct media_entity_graph *graph,
+		       struct media_entity *entity)
+{
+	if (graph->top == MEDIA_ENTITY_ENUM_MAX_DEPTH - 1) {
+		WARN_ON(1);
+		return;
+	}
+	graph->top++;
+	graph->stack[graph->top].link = 0;
+	graph->stack[graph->top].entity = entity;
+}
+
+static struct media_entity *stack_pop(struct media_entity_graph *graph)
+{
+	struct media_entity *entity;
+
+	entity = graph->stack[graph->top].entity;
+	graph->top--;
+
+	return entity;
+}
+
+#define stack_peek(en)	((en)->stack[(en)->top - 1].entity)
+#define link_top(en)	((en)->stack[(en)->top].link)
+#define stack_top(en)	((en)->stack[(en)->top].entity)
+
+/**
+ * media_entity_graph_walk_start - Start walking the media graph at a given entity
+ * @graph: Media graph structure that will be used to walk the graph
+ * @entity: Starting entity
+ *
+ * This function initializes the graph traversal structure to walk the entities
+ * graph starting at the given entity. The traversal structure must not be
+ * modified by the caller during graph traversal. When done the structure can
+ * safely be freed.
+ */
+void media_entity_graph_walk_start(struct media_entity_graph *graph,
+				   struct media_entity *entity)
+{
+	graph->top = 0;
+	graph->stack[graph->top].entity = NULL;
+	stack_push(graph, entity);
+}
+EXPORT_SYMBOL_GPL(media_entity_graph_walk_start);
+
+/**
+ * media_entity_graph_walk_next - Get the next entity in the graph
+ * @graph: Media graph structure
+ *
+ * Perform a depth-first traversal of the given media entities graph.
+ *
+ * The graph structure must have been previously initialized with a call to
+ * media_entity_graph_walk_start().
+ *
+ * Return the next entity in the graph or NULL if the whole graph have been
+ * traversed.
+ */
+struct media_entity *
+media_entity_graph_walk_next(struct media_entity_graph *graph)
+{
+	if (stack_top(graph) == NULL)
+		return NULL;
+
+	/*
+	 * Depth first search. Push entity to stack and continue from
+	 * top of the stack until no more entities on the level can be
+	 * found.
+	 */
+	while (link_top(graph) < stack_top(graph)->num_links) {
+		struct media_entity *entity = stack_top(graph);
+		struct media_link *link = &entity->links[link_top(graph)];
+		struct media_entity *next;
+
+		/* The link is not active so we do not follow. */
+		if (!(link->flags & MEDIA_LINK_FLAG_ACTIVE)) {
+			link_top(graph)++;
+			continue;
+		}
+
+		/* Get the entity in the other end of the link . */
+		next = media_entity_other(entity, link);
+
+		/* Was it the entity we came here from? */
+		if (next == stack_peek(graph)) {
+			link_top(graph)++;
+			continue;
+		}
+
+		/* Push the new entity to stack and start over. */
+		link_top(graph)++;
+		stack_push(graph, next);
+	}
+
+	return stack_pop(graph);
+}
+EXPORT_SYMBOL_GPL(media_entity_graph_walk_next);
+
+/* -----------------------------------------------------------------------------
+ * Links management
+ */
+
 static struct media_link *media_entity_add_link(struct media_entity *entity)
 {
 	if (entity->num_links >= entity->max_links) {
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 32bb20a..3a7c74d 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -87,10 +87,25 @@ static inline u32 media_entity_subtype(struct media_entity *entity)
 	return entity->type & MEDIA_ENTITY_SUBTYPE_MASK;
 }
 
+#define MEDIA_ENTITY_ENUM_MAX_DEPTH	16
+
+struct media_entity_graph {
+	struct {
+		struct media_entity *entity;
+		int link;
+	} stack[MEDIA_ENTITY_ENUM_MAX_DEPTH];
+	int top;
+};
+
 int media_entity_init(struct media_entity *entity, u16 num_pads,
 		struct media_pad *pads, u16 extra_links);
 void media_entity_cleanup(struct media_entity *entity);
 int media_entity_create_link(struct media_entity *source, u16 source_pad,
 		struct media_entity *sink, u16 sink_pad, u32 flags);
 
+void media_entity_graph_walk_start(struct media_entity_graph *graph,
+		struct media_entity *entity);
+struct media_entity *
+media_entity_graph_walk_next(struct media_entity_graph *graph);
+
 #endif
-- 
1.7.1

[RFC/PATCH v4 05/11] media: Reference count and power handling

[Laurent Pinchart]

From: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>

Basically these are the interface functions:

media_entity_get() - acquire entity
media_entity_put() - release entity

	If the entity is of node type, the power change is distributed to
	all connected entities. For non-nodes it only affects that very
	node. A mutex is used to serialise access to the entity graph.

In the background there's a depth-first search algorithm that traverses the
active links in the graph. All these functions parse the graph to implement
whatever they're to do.

The module counters are increased/decreased in media_entity_get/put to
prevent module unloading when an entity is referenced.

Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Stanimir Varbanov <svarbanov@mm-sol.com>
---
 Documentation/media-framework.txt |   37 +++++++++
 drivers/media/media-device.c      |    1 +
 drivers/media/media-entity.c      |  146 +++++++++++++++++++++++++++++++++++++
 include/media/media-device.h      |    4 +
 include/media/media-entity.h      |   15 ++++
 5 files changed, 203 insertions(+), 0 deletions(-)

diff --git a/Documentation/media-framework.txt b/Documentation/media-framework.txt
index a599824..59649e9 100644
--- a/Documentation/media-framework.txt
+++ b/Documentation/media-framework.txt
@@ -278,3 +278,40 @@ When the graph traversal is complete the function will return NULL.
 Graph traversal can be interrupted at any moment. No cleanup function call is
 required and the graph structure can be freed normally.
 
+
+Reference counting and power handling
+-------------------------------------
+
+Before accessing type-specific entities operations (such as the V4L2
+sub-device operations), drivers must acquire a reference to the entity. This
+ensures that the entity will be powered on and ready to accept requests.
+Similarly, after being done with an entity, drivers must release the
+reference.
+
+	media_entity_get(struct media_entity *entity)
+
+The function will increase the entity reference count. If the entity is a node
+(MEDIA_ENTITY_TYPE_NODE type), the reference count of all entities it is
+connected to, both directly or indirectly, through active links is increased.
+This ensures that the whole media pipeline will be ready to process
+
+Acquiring a reference to an entity increases the media device module reference
+count to prevent module unloading when an entity is being used.
+
+media_entity_get will return a pointer to the entity if successful, or NULL
+otherwise.
+
+	media_entity_put(struct media_entity *entity)
+
+The function will decrease the entity reference count and, for node entities,
+like media_entity_get, the reference count of all connected entities. Calling
+media_entity_put with a NULL argument is valid and will return immediately.
+
+When the first reference to an entity is acquired, or the last reference
+released, the entity's set_power operation is called. Entity drivers must
+implement the operation if they need to perform any power management task,
+such as turning powers or clocks on or off. If no power management is
+required, drivers don't need to provide a set_power operation. The operation
+is allowed to fail when turning power on, in which case the media_entity_get
+function will return NULL.
+
diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
index eeb002e..c309d3c 100644
--- a/drivers/media/media-device.c
+++ b/drivers/media/media-device.c
@@ -71,6 +71,7 @@ int __must_check media_device_register(struct media_device *mdev)
 	mdev->entity_id = 1;
 	INIT_LIST_HEAD(&mdev->entities);
 	spin_lock_init(&mdev->lock);
+	mutex_init(&mdev->graph_mutex);
 
 	/* Register the device node. */
 	mdev->devnode.fops = &media_device_fops;
diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index c277c18..da4fef6 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -21,6 +21,7 @@
 #include <linux/module.h>
 #include <linux/slab.h>
 #include <media/media-entity.h>
+#include <media/media-device.h>
 
 /**
  * media_entity_init - Initialize a media entity
@@ -194,6 +195,151 @@ media_entity_graph_walk_next(struct media_entity_graph *graph)
 EXPORT_SYMBOL_GPL(media_entity_graph_walk_next);
 
 /* -----------------------------------------------------------------------------
+ * Power state handling
+ */
+
+/* Apply use count to an entity. */
+static void media_entity_use_apply_one(struct media_entity *entity, int change)
+{
+	entity->use_count += change;
+	WARN_ON(entity->use_count < 0);
+}
+
+/*
+ * Apply use count change to an entity and change power state based on
+ * new use count.
+ */
+static int media_entity_power_apply_one(struct media_entity *entity, int change)
+{
+	int ret;
+
+	if (entity->use_count == 0 && change > 0 &&
+	    entity->ops && entity->ops->set_power) {
+		ret = entity->ops->set_power(entity, 1);
+		if (ret)
+			return ret;
+	}
+
+	media_entity_use_apply_one(entity, change);
+
+	if (entity->use_count == 0 && change < 0 &&
+	    entity->ops && entity->ops->set_power)
+		entity->ops->set_power(entity, 0);
+
+	return 0;
+}
+
+/*
+ * Apply power change to all connected entities. This ignores the
+ * nodes.
+ */
+static int media_entity_power_apply(struct media_entity *entity, int change)
+{
+	struct media_entity_graph graph;
+	struct media_entity *first = entity;
+	int ret = 0;
+
+	if (!change)
+		return 0;
+
+	media_entity_graph_walk_start(&graph, entity);
+
+	while (!ret && (entity = media_entity_graph_walk_next(&graph)))
+		if (media_entity_type(entity) != MEDIA_ENTITY_TYPE_NODE)
+			ret = media_entity_power_apply_one(entity, change);
+
+	if (!ret)
+		return 0;
+
+	media_entity_graph_walk_start(&graph, first);
+
+	while ((first = media_entity_graph_walk_next(&graph))
+	       && first != entity)
+		if (media_entity_type(first) != MEDIA_ENTITY_TYPE_NODE)
+			media_entity_power_apply_one(first, -change);
+
+	return ret;
+}
+
+/*
+ * Apply use count change to graph and change power state of entities
+ * accordingly.
+ */
+static int media_entity_node_power_change(struct media_entity *entity,
+					  int change)
+{
+	/* Apply use count to node. */
+	media_entity_use_apply_one(entity, change);
+
+	/* Apply power change to connected non-nodes. */
+	return media_entity_power_apply(entity, change);
+}
+
+/*
+ * Node entity use changes are reflected on power state of all
+ * connected (directly or indirectly) entities whereas non-node entity
+ * use count changes are limited to that very entity.
+ */
+static int media_entity_use_change(struct media_entity *entity, int change)
+{
+	if (media_entity_type(entity) == MEDIA_ENTITY_TYPE_NODE)
+		return media_entity_node_power_change(entity, change);
+	else
+		return media_entity_power_apply_one(entity, change);
+}
+
+static struct media_entity *__media_entity_get(struct media_entity *entity)
+{
+	if (media_entity_use_change(entity, 1))
+		return NULL;
+
+	return entity;
+}
+
+static void __media_entity_put(struct media_entity *entity)
+{
+	media_entity_use_change(entity, -1);
+}
+
+/* user open()s media entity */
+struct media_entity *media_entity_get(struct media_entity *entity)
+{
+	struct media_entity *e;
+
+	if (entity == NULL)
+		return NULL;
+
+	if (entity->parent->dev &&
+	    !try_module_get(entity->parent->dev->driver->owner))
+		return NULL;
+
+	mutex_lock(&entity->parent->graph_mutex);
+	e = __media_entity_get(entity);
+	mutex_unlock(&entity->parent->graph_mutex);
+
+	if (e == NULL && entity->parent->dev)
+		module_put(entity->parent->dev->driver->owner);
+
+	return e;
+}
+EXPORT_SYMBOL_GPL(media_entity_get);
+
+/* user release()s media entity */
+void media_entity_put(struct media_entity *entity)
+{
+	if (entity == NULL)
+		return;
+
+	mutex_lock(&entity->parent->graph_mutex);
+	__media_entity_put(entity);
+	mutex_unlock(&entity->parent->graph_mutex);
+
+	if (entity->parent->dev)
+		module_put(entity->parent->dev->driver->owner);
+}
+EXPORT_SYMBOL_GPL(media_entity_put);
+
+/* -----------------------------------------------------------------------------
  * Links management
  */
 
diff --git a/include/media/media-device.h b/include/media/media-device.h
index 7e2dac2..3c9a5e0 100644
--- a/include/media/media-device.h
+++ b/include/media/media-device.h
@@ -23,6 +23,7 @@
 
 #include <linux/device.h>
 #include <linux/list.h>
+#include <linux/mutex.h>
 #include <linux/spinlock.h>
 
 #include <media/media-devnode.h>
@@ -40,6 +41,7 @@
  * @entity_id:	ID of the next entity to be registered
  * @entities:	List of registered entities
  * @lock:	Entities list lock
+ * @graph_mutex: Entities graph operation lock
  *
  * This structure represents an abstract high-level media device. It allows easy
  * access to entities and provides basic media device-level support. The
@@ -67,6 +69,8 @@ struct media_device {
 
 	/* Protects the entities list */
 	spinlock_t lock;
+	/* Serializes graph operations. */
+	struct mutex graph_mutex;
 };
 
 int __must_check media_device_register(struct media_device *mdev);
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 3a7c74d..edcafeb 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -39,6 +39,10 @@ struct media_pad {
 	unsigned long flags;		/* Pad flags (MEDIA_PAD_FLAG_*) */
 };
 
+struct media_entity_operations {
+	int (*set_power)(struct media_entity *entity, int power);
+};
+
 struct media_entity {
 	struct list_head list;
 	struct media_device *parent;	/* Media device this entity belongs to*/
@@ -59,6 +63,10 @@ struct media_entity {
 	struct media_pad *pads;		/* Pads array (num_pads elements) */
 	struct media_link *links;	/* Links array (max_links elements)*/
 
+	const struct media_entity_operations *ops;	/* Entity operations */
+
+	int use_count;			/* Use count for the entity. */
+
 	union {
 		/* Node specifications */
 		struct {
@@ -103,9 +111,16 @@ void media_entity_cleanup(struct media_entity *entity);
 int media_entity_create_link(struct media_entity *source, u16 source_pad,
 		struct media_entity *sink, u16 sink_pad, u32 flags);
 
+struct media_entity *media_entity_get(struct media_entity *entity);
+void media_entity_put(struct media_entity *entity);
+
 void media_entity_graph_walk_start(struct media_entity_graph *graph,
 		struct media_entity *entity);
 struct media_entity *
 media_entity_graph_walk_next(struct media_entity_graph *graph);
 
+#define media_entity_call(entity, operation, args...)			\
+	(((entity)->ops && (entity)->ops->operation) ?			\
+	 (entity)->ops->operation((entity) , ##args) : -ENOIOCTLCMD)
+
 #endif
-- 
1.7.1

[RFC/PATCH v4 06/11] media: Media device information query

[Laurent Pinchart]

Create the following ioctl and implement it at the media device level to
query device information.

- MEDIA_IOC_DEVICE_INFO: Query media device information

The ioctl and its data structure are defined in the new kernel header
linux/media.h available to userspace applications.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 Documentation/media-framework.txt |   42 +++++++++++++++++++++++++++
 drivers/media/media-device.c      |   57 +++++++++++++++++++++++++++++++++++++
 include/linux/media.h             |   23 +++++++++++++++
 include/media/media-device.h      |    3 ++
 4 files changed, 125 insertions(+), 0 deletions(-)
 create mode 100644 include/linux/media.h

diff --git a/Documentation/media-framework.txt b/Documentation/media-framework.txt
index 59649e9..66f7f6c 100644
--- a/Documentation/media-framework.txt
+++ b/Documentation/media-framework.txt
@@ -315,3 +315,45 @@ required, drivers don't need to provide a set_power operation. The operation
 is allowed to fail when turning power on, in which case the media_entity_get
 function will return NULL.
 
+
+Userspace application API
+-------------------------
+
+Media devices offer an API to userspace application to query device information
+through ioctls.
+
+	MEDIA_IOC_DEVICE_INFO - Get device information
+	----------------------------------------------
+
+	ioctl(int fd, int request, struct media_device_info *argp);
+
+To query device information, call the MEDIA_IOC_ENUM_ENTITIES ioctl with a
+pointer to a media_device_info structure. The driver fills the structure and
+returns the information to the application. The ioctl never fails.
+
+The media_device_info structure is defined as
+
+- struct media_device_info
+
+__u8	driver[16]	Driver name as a NUL-terminated ASCII string. The
+			driver version is stored in the driver_version field.
+__u8	model[32]	Device model name as a NUL-terminated UTF-8 string. The
+			device version is stored in the device_version field and
+			is not be appended to the model name.
+__u8	serial[32]	Serial number as an ASCII string. The string is
+			NUL-terminated unless the serial number is exactly 32
+			characters long.
+__u8	bus_info[32]	Location of the device in the system as a NUL-terminated
+			ASCII string. This includes the bus type name (PCI, USB,
+			...) and a bus-specific identifier.
+__u32	media_version	Media API version, formatted with the KERNEL_VERSION
+			macro.
+__u32	device_version	Media device driver version in a driver-specific format.
+__u32	driver_version	Media device driver version, formatted with the
+			KERNEL_VERSION macro.
+
+The serial and bus_info fields can be used to distinguish between multiple
+instances of otherwise identical hardware. The serial number takes precedence
+when provided and can be assumed to be unique. If the serial number is an
+empty string, the bus_info field can be used instead. The bus_info field is
+guaranteed to be unique, but can vary across reboots or device unplug/replug.
diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
index c309d3c..1415ebd 100644
--- a/drivers/media/media-device.c
+++ b/drivers/media/media-device.c
@@ -20,13 +20,70 @@
 
 #include <linux/types.h>
 #include <linux/ioctl.h>
+#include <linux/media.h>
 
 #include <media/media-device.h>
 #include <media/media-devnode.h>
 #include <media/media-entity.h>
 
+/* -----------------------------------------------------------------------------
+ * Userspace API
+ */
+
+static int media_device_open(struct file *filp)
+{
+	return 0;
+}
+
+static int media_device_close(struct file *filp)
+{
+	return 0;
+}
+
+static int media_device_get_info(struct media_device *dev,
+				 struct media_device_info __user *__info)
+{
+	struct media_device_info info;
+
+	memset(&info, 0, sizeof(info));
+
+	strlcpy(info.driver, dev->dev->driver->name, sizeof(info.driver));
+	strlcpy(info.model, dev->model, sizeof(info.model));
+	strncpy(info.serial, dev->serial, sizeof(info.serial));
+	strlcpy(info.bus_info, dev->bus_info, sizeof(info.bus_info));
+
+	info.media_version = MEDIA_API_VERSION;
+	info.device_version = dev->device_version;
+	info.driver_version = dev->driver_version;
+
+	return copy_to_user(__info, &info, sizeof(*__info));
+}
+
+static long media_device_ioctl(struct file *filp, unsigned int cmd,
+			       unsigned long arg)
+{
+	struct media_devnode *devnode = media_devnode_data(filp);
+	struct media_device *dev = to_media_device(devnode);
+	long ret;
+
+	switch (cmd) {
+	case MEDIA_IOC_DEVICE_INFO:
+		ret = media_device_get_info(dev,
+				(struct media_device_info __user *)arg);
+		break;
+
+	default:
+		ret = -ENOIOCTLCMD;
+	}
+
+	return ret;
+}
+
 static const struct media_file_operations media_device_fops = {
 	.owner = THIS_MODULE,
+	.open = media_device_open,
+	.unlocked_ioctl = media_device_ioctl,
+	.release = media_device_close,
 };
 
 /* -----------------------------------------------------------------------------
diff --git a/include/linux/media.h b/include/linux/media.h
new file mode 100644
index 0000000..bca08a7
--- /dev/null
+++ b/include/linux/media.h
@@ -0,0 +1,23 @@
+#ifndef __LINUX_MEDIA_H
+#define __LINUX_MEDIA_H
+
+#include <linux/ioctl.h>
+#include <linux/types.h>
+#include <linux/version.h>
+
+#define MEDIA_API_VERSION	KERNEL_VERSION(0, 1, 0)
+
+struct media_device_info {
+	__u8 driver[16];
+	__u8 model[32];
+	__u8 serial[32];
+	__u8 bus_info[32];
+	__u32 media_version;
+	__u32 device_version;
+	__u32 driver_version;
+	__u32 reserved[5];
+};
+
+#define MEDIA_IOC_DEVICE_INFO		_IOWR('M', 1, struct media_device_info)
+
+#endif /* __LINUX_MEDIA_H */
diff --git a/include/media/media-device.h b/include/media/media-device.h
index 3c9a5e0..6f57f41 100644
--- a/include/media/media-device.h
+++ b/include/media/media-device.h
@@ -73,6 +73,9 @@ struct media_device {
 	struct mutex graph_mutex;
 };
 
+/* media_devnode to media_device */
+#define to_media_device(node) container_of(node, struct media_device, devnode)
+
 int __must_check media_device_register(struct media_device *mdev);
 void media_device_unregister(struct media_device *mdev);
 
-- 
1.7.1

[RFC/PATCH v4 07/11] media: Entities, pads and links enumeration

[Laurent Pinchart]

Create the following two ioctls and implement them at the media device
level to enumerate entities, pads and links.

- MEDIA_IOC_ENUM_ENTITIES: Enumerate entities and their properties
- MEDIA_IOC_ENUM_LINKS: Enumerate all pads and links for a given entity

Entity IDs can be non-contiguous. Userspace applications should
enumerate entities using the MEDIA_ENTITY_ID_FLAG_NEXT flag. When the
flag is set in the entity ID, the MEDIA_IOC_ENUM_ENTITIES will return
the next entity with an ID bigger than the requested one.

Only forward links that originate at one of the entity's source pads are
returned during the enumeration process.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
---
 Documentation/media-framework.txt |  142 ++++++++++++++++++++++++++++++++++++-
 drivers/media/media-device.c      |  123 ++++++++++++++++++++++++++++++++
 include/linux/media.h             |   81 +++++++++++++++++++++
 include/media/media-entity.h      |   24 +------
 4 files changed, 346 insertions(+), 24 deletions(-)

diff --git a/Documentation/media-framework.txt b/Documentation/media-framework.txt
index 66f7f6c..74a137d 100644
--- a/Documentation/media-framework.txt
+++ b/Documentation/media-framework.txt
@@ -320,7 +320,7 @@ Userspace application API
 -------------------------
 
 Media devices offer an API to userspace application to query device information
-through ioctls.
+and discover the device internal topology through ioctls.
 
 	MEDIA_IOC_DEVICE_INFO - Get device information
 	----------------------------------------------
@@ -357,3 +357,143 @@ instances of otherwise identical hardware. The serial number takes precedence
 when provided and can be assumed to be unique. If the serial number is an
 empty string, the bus_info field can be used instead. The bus_info field is
 guaranteed to be unique, but can vary across reboots or device unplug/replug.
+
+
+	MEDIA_IOC_ENUM_ENTITIES - Enumerate entities and their properties
+	-----------------------------------------------------------------
+
+	ioctl(int fd, int request, struct media_entity_desc *argp);
+
+To query the attributes of an entity, applications set the id field of a
+media_entity_desc structure and call the MEDIA_IOC_ENUM_ENTITIES ioctl with a
+pointer to this structure. The driver fills the rest of the structure or
+returns a EINVAL error code when the id is invalid.
+
+Entities can be enumerated by or'ing the id with the MEDIA_ENTITY_ID_FLAG_NEXT
+flag. The driver will return information about the entity with the smallest id
+strictly larger than the requested one ('next entity'), or EINVAL if there is
+none.
+
+Entity IDs can be non-contiguous. Applications must *not* try to enumerate
+entities by calling MEDIA_IOC_ENUM_ENTITIES with increasing id's until they
+get an error.
+
+Two or more entities that share a common non-zero group_id value are
+considered as logically grouped. Groups are used to report
+
+	- ALSA, VBI and video nodes that carry the same media stream
+	- lens and flash controllers associated with a sensor
+
+The media_entity_desc structure is defined as
+
+- struct media_entity_desc
+
+__u32	id		Entity id, set by the application. When the id is
+			or'ed with MEDIA_ENTITY_ID_FLAG_NEXT, the driver
+			clears the flag and returns the first entity with a
+			larger id.
+char	name[32]	Entity name. UTF-8 NULL-terminated string.
+__u32	type		Entity type.
+__u32	revision	Entity revision in a driver/hardware specific format.
+__u32	flags		Entity flags.
+__u32	group_id	Entity group ID.
+__u16	pads		Number of pads.
+__u16	links		Total number of outbound links. Inbound links are not
+			counted in this field.
+/* union */
+	/* struct v4l, Valid for V4L sub-devices and nodes only */
+__u32	major		V4L device node major number. For V4L sub-devices with
+			no device node, set by the driver to 0.
+__u32	minor		V4L device node minor number. For V4L sub-devices with
+			no device node, set by the driver to 0.
+	/* struct fb, Valid for frame buffer nodes only */
+__u32	major		FB device node major number
+__u32	minor		FB device node minor number
+	/* Valid for ALSA devices only */
+int	alsa		ALSA card number
+	/* Valid for DVB devices only */
+int	dvb		DVB card number
+
+Valid entity types are
+
+	MEDIA_ENTITY_TYPE_NODE - Unknown device node
+	MEDIA_ENTITY_TYPE_NODE_V4L - V4L video, radio or vbi device node
+	MEDIA_ENTITY_TYPE_NODE_FB - Frame buffer device node
+	MEDIA_ENTITY_TYPE_NODE_ALSA - ALSA card
+	MEDIA_ENTITY_TYPE_NODE_DVB - DVB card
+
+	MEDIA_ENTITY_TYPE_SUBDEV - Unknown V4L sub-device
+	MEDIA_ENTITY_TYPE_SUBDEV_SENSOR - Video sensor
+	MEDIA_ENTITY_TYPE_SUBDEV_FLASH - Flash controller
+	MEDIA_ENTITY_TYPE_SUBDEV_LENS - Lens controller
+
+Valid entity flags are
+
+	MEDIA_ENTITY_FLAG_DEFAULT - Default entity for its type. Used to
+		discover the default audio, VBI and video devices, the default
+		camera sensor, ...
+
+
+	MEDIA_IOC_ENUM_LINKS - Enumerate all pads and links for a given entity
+	----------------------------------------------------------------------
+
+	ioctl(int fd, int request, struct media_links_enum *argp);
+
+Only forward links that originate at one of the entity's source pads are
+returned during the enumeration process.
+
+To enumerate pads and/or links for a given entity, applications set the entity
+field of a media_links_enum structure and initialize the media_pad_desc and
+media_link_desc structure arrays pointed by the pads and links fields. They then
+call the MEDIA_IOC_ENUM_LINKS ioctl with a pointer to this structure.
+
+If the pads field is not NULL, the driver fills the pads array with
+information about the entity's pads. The array must have enough room to store
+all the entity's pads. The number of pads can be retrieved with the
+MEDIA_IOC_ENUM_ENTITIES ioctl.
+
+If the links field is not NULL, the driver fills the links array with
+information about the entity's outbound links. The array must have enough room
+to store all the entity's outbound links. The number of outbound links can be
+retrieved with the MEDIA_IOC_ENUM_ENTITIES ioctl.
+
+The media_pad_desc, media_link_desc and media_links_enum structures are defined
+as
+
+- struct media_pad_desc
+
+__u32		entity		ID of the entity this pad belongs to.
+__u16		index		0-based pad index.
+__u32		flags		Pad flags.
+
+Valid pad flags are
+
+	MEDIA_PAD_FLAG_INPUT -	Input pad, relative to the entity. Input pads
+				sink data and are targets of links.
+	MEDIA_PAD_FLAG_OUTPUT -	Output pad, relative to the entity. Output
+				pads source data and are origins of links.
+
+One and only one of MEDIA_PAD_FLAG_INPUT and MEDIA_PAD_FLAG_OUTPUT must be set
+for every pad.
+
+- struct media_link_desc
+
+struct media_pad_desc	source	Pad at the origin of this link.
+struct media_pad_desc	sink	Pad at the target of this link.
+__u32			flags	Link flags.
+
+Valid link flags are
+
+	MEDIA_LINK_FLAG_ACTIVE - The link is active and can be used to
+		transfer media data. When two or more links target a sink pad,
+		only one of them can be active at a time.
+	MEDIA_LINK_FLAG_IMMUTABLE - The link active state can't be modified at
+		runtime. An immutable link is always active.
+
+- struct media_links_enum
+
+__u32			entity	Entity id, set by the application.
+struct media_pad_desc	*pads	Pointer to a pads array allocated by the
+				application. Ignored if NULL.
+struct media_link_desc	*links	Pointer to a links array allocated by the
+				application. Ignored if NULL.
diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
index 1415ebd..7e020f9 100644
--- a/drivers/media/media-device.c
+++ b/drivers/media/media-device.c
@@ -59,6 +59,117 @@ static int media_device_get_info(struct media_device *dev,
 	return copy_to_user(__info, &info, sizeof(*__info));
 }
 
+static struct media_entity *find_entity(struct media_device *mdev, u32 id)
+{
+	struct media_entity *entity;
+	int next = id & MEDIA_ENTITY_ID_FLAG_NEXT;
+
+	id &= ~MEDIA_ENTITY_ID_FLAG_NEXT;
+
+	spin_lock(&mdev->lock);
+
+	media_device_for_each_entity(entity, mdev) {
+		if ((entity->id == id && !next) ||
+		    (entity->id > id && next)) {
+			spin_unlock(&mdev->lock);
+			return entity;
+		}
+	}
+
+	spin_unlock(&mdev->lock);
+
+	return NULL;
+}
+
+static long media_device_enum_entities(struct media_device *mdev,
+				       struct media_entity_desc __user *uent)
+{
+	struct media_entity *ent;
+	struct media_entity_desc u_ent;
+
+	if (copy_from_user(&u_ent.id, &uent->id, sizeof(u_ent.id)))
+		return -EFAULT;
+
+	ent = find_entity(mdev, u_ent.id);
+
+	if (ent == NULL)
+		return -EINVAL;
+
+	u_ent.id = ent->id;
+	u_ent.name[0] = '\0';
+	if (ent->name)
+		strlcpy(u_ent.name, ent->name, sizeof(u_ent.name));
+	u_ent.type = ent->type;
+	u_ent.revision = ent->revision;
+	u_ent.flags = ent->flags;
+	u_ent.group_id = ent->group_id;
+	u_ent.pads = ent->num_pads;
+	u_ent.links = ent->num_links - ent->num_backlinks;
+	u_ent.v4l.major = ent->v4l.major;
+	u_ent.v4l.minor = ent->v4l.minor;
+	if (copy_to_user(uent, &u_ent, sizeof(u_ent)))
+		return -EFAULT;
+	return 0;
+}
+
+static void media_device_kpad_to_upad(const struct media_pad *kpad,
+				      struct media_pad_desc *upad)
+{
+	upad->entity = kpad->entity->id;
+	upad->index = kpad->index;
+	upad->flags = kpad->flags;
+}
+
+static long media_device_enum_links(struct media_device *mdev,
+				    struct media_links_enum __user *ulinks)
+{
+	struct media_entity *entity;
+	struct media_links_enum links;
+
+	if (copy_from_user(&links, ulinks, sizeof(links)))
+		return -EFAULT;
+
+	entity = find_entity(mdev, links.entity);
+	if (entity == NULL)
+		return -EINVAL;
+
+	if (links.pads) {
+		unsigned int p;
+
+		for (p = 0; p < entity->num_pads; p++) {
+			struct media_pad_desc pad;
+			media_device_kpad_to_upad(&entity->pads[p], &pad);
+			if (copy_to_user(&links.pads[p], &pad, sizeof(pad)))
+				return -EFAULT;
+		}
+	}
+
+	if (links.links) {
+		struct media_link_desc __user *ulink;
+		unsigned int l;
+
+		for (l = 0, ulink = links.links; l < entity->num_links; l++) {
+			struct media_link_desc link;
+
+			/* Ignore backlinks. */
+			if (entity->links[l].source->entity != entity)
+				continue;
+
+			media_device_kpad_to_upad(entity->links[l].source,
+						  &link.source);
+			media_device_kpad_to_upad(entity->links[l].sink,
+						  &link.sink);
+			link.flags = entity->links[l].flags;
+			if (copy_to_user(ulink, &link, sizeof(*ulink)))
+				return -EFAULT;
+			ulink++;
+		}
+	}
+	if (copy_to_user(ulinks, &links, sizeof(*ulinks)))
+		return -EFAULT;
+	return 0;
+}
+
 static long media_device_ioctl(struct file *filp, unsigned int cmd,
 			       unsigned long arg)
 {
@@ -72,6 +183,18 @@ static long media_device_ioctl(struct file *filp, unsigned int cmd,
 				(struct media_device_info __user *)arg);
 		break;
 
+	case MEDIA_IOC_ENUM_ENTITIES:
+		ret = media_device_enum_entities(dev,
+				(struct media_entity_desc __user *)arg);
+		break;
+
+	case MEDIA_IOC_ENUM_LINKS:
+		mutex_lock(&dev->graph_mutex);
+		ret = media_device_enum_links(dev,
+				(struct media_links_enum __user *)arg);
+		mutex_unlock(&dev->graph_mutex);
+		break;
+
 	default:
 		ret = -ENOIOCTLCMD;
 	}
diff --git a/include/linux/media.h b/include/linux/media.h
index bca08a7..542509b 100644
--- a/include/linux/media.h
+++ b/include/linux/media.h
@@ -18,6 +18,87 @@ struct media_device_info {
 	__u32 reserved[5];
 };
 
+#define MEDIA_ENTITY_ID_FLAG_NEXT		(1 << 31)
+
+#define MEDIA_ENTITY_TYPE_SHIFT			16
+#define MEDIA_ENTITY_TYPE_MASK			0x00ff0000
+#define MEDIA_ENTITY_SUBTYPE_MASK		0x0000ffff
+
+#define MEDIA_ENTITY_TYPE_NODE			(1 << MEDIA_ENTITY_TYPE_SHIFT)
+#define MEDIA_ENTITY_TYPE_NODE_V4L		(MEDIA_ENTITY_TYPE_NODE + 1)
+#define MEDIA_ENTITY_TYPE_NODE_FB		(MEDIA_ENTITY_TYPE_NODE + 2)
+#define MEDIA_ENTITY_TYPE_NODE_ALSA		(MEDIA_ENTITY_TYPE_NODE + 3)
+#define MEDIA_ENTITY_TYPE_NODE_DVB		(MEDIA_ENTITY_TYPE_NODE + 4)
+
+#define MEDIA_ENTITY_TYPE_SUBDEV		(2 << MEDIA_ENTITY_TYPE_SHIFT)
+#define MEDIA_ENTITY_TYPE_SUBDEV_SENSOR		(MEDIA_ENTITY_TYPE_SUBDEV + 1)
+#define MEDIA_ENTITY_TYPE_SUBDEV_FLASH		(MEDIA_ENTITY_TYPE_SUBDEV + 2)
+#define MEDIA_ENTITY_TYPE_SUBDEV_LENS		(MEDIA_ENTITY_TYPE_SUBDEV + 3)
+
+#define MEDIA_ENTITY_FLAG_DEFAULT		(1 << 0)
+
+struct media_entity_desc {
+	__u32 id;
+	char name[32];
+	__u32 type;
+	__u32 revision;
+	__u32 flags;
+	__u32 group_id;
+	__u16 pads;
+	__u16 links;
+
+	__u32 reserved[4];
+
+	union {
+		/* Node specifications */
+		struct {
+			__u32 major;
+			__u32 minor;
+		} v4l;
+		struct {
+			__u32 major;
+			__u32 minor;
+		} fb;
+		int alsa;
+		int dvb;
+
+		/* Sub-device specifications */
+		/* Nothing needed yet */
+		__u8 raw[64];
+	};
+};
+
+#define MEDIA_PAD_FLAG_INPUT			(1 << 0)
+#define MEDIA_PAD_FLAG_OUTPUT			(1 << 1)
+
+struct media_pad_desc {
+	__u32 entity;		/* entity ID */
+	__u16 index;		/* pad index */
+	__u32 flags;		/* pad flags */
+	__u32 reserved[2];
+};
+
+#define MEDIA_LINK_FLAG_ACTIVE			(1 << 0)
+#define MEDIA_LINK_FLAG_IMMUTABLE		(1 << 1)
+
+struct media_link_desc {
+	struct media_pad_desc source;
+	struct media_pad_desc sink;
+	__u32 flags;
+	__u32 reserved[2];
+};
+
+struct media_links_enum {
+	__u32 entity;
+	/* Should have enough room for pads elements */
+	struct media_pad_desc __user *pads;
+	/* Should have enough room for links elements */
+	struct media_link_desc __user *links;
+	__u32 reserved[4];
+};
+
 #define MEDIA_IOC_DEVICE_INFO		_IOWR('M', 1, struct media_device_info)
+#define MEDIA_IOC_ENUM_ENTITIES		_IOWR('M', 2, struct media_entity_desc)
+#define MEDIA_IOC_ENUM_LINKS		_IOWR('M', 3, struct media_links_enum)
 
 #endif /* __LINUX_MEDIA_H */
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index edcafeb..8c40d5e 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -2,29 +2,7 @@
 #define _MEDIA_ENTITY_H
 
 #include <linux/list.h>
-
-#define MEDIA_ENTITY_TYPE_SHIFT			16
-#define MEDIA_ENTITY_TYPE_MASK			0x00ff0000
-#define MEDIA_ENTITY_SUBTYPE_MASK		0x0000ffff
-
-#define MEDIA_ENTITY_TYPE_NODE			(1 << MEDIA_ENTITY_TYPE_SHIFT)
-#define MEDIA_ENTITY_TYPE_NODE_V4L		(MEDIA_ENTITY_TYPE_NODE + 1)
-#define MEDIA_ENTITY_TYPE_NODE_FB		(MEDIA_ENTITY_TYPE_NODE + 2)
-#define MEDIA_ENTITY_TYPE_NODE_ALSA		(MEDIA_ENTITY_TYPE_NODE + 3)
-#define MEDIA_ENTITY_TYPE_NODE_DVB		(MEDIA_ENTITY_TYPE_NODE + 4)
-
-#define MEDIA_ENTITY_TYPE_SUBDEV		(2 << MEDIA_ENTITY_TYPE_SHIFT)
-#define MEDIA_ENTITY_TYPE_SUBDEV_SENSOR		(MEDIA_ENTITY_TYPE_SUBDEV + 1)
-#define MEDIA_ENTITY_TYPE_SUBDEV_FLASH		(MEDIA_ENTITY_TYPE_SUBDEV + 2)
-#define MEDIA_ENTITY_TYPE_SUBDEV_LENS		(MEDIA_ENTITY_TYPE_SUBDEV + 3)
-
-#define MEDIA_ENTITY_FLAG_DEFAULT		(1 << 0)
-
-#define MEDIA_LINK_FLAG_ACTIVE			(1 << 0)
-#define MEDIA_LINK_FLAG_IMMUTABLE		(1 << 1)
-
-#define MEDIA_PAD_FLAG_INPUT			(1 << 0)
-#define MEDIA_PAD_FLAG_OUTPUT			(1 << 1)
+#include <linux/media.h>
 
 struct media_link {
 	struct media_pad *source;	/* Source pad */
-- 
1.7.1

[RFC/PATCH v4 08/11] media: Links setup

[Laurent Pinchart]

Create the following ioctl and implement it at the media device level to
setup links.

- MEDIA_IOC_SETUP_LINK: Modify the properties of a given link

The only property that can currently be modified is the ACTIVE link flag
to activate/deactivate a link. Links marked with the IMMUTABLE link flag
can not be activated or deactivated.

Activating and deactivating a link has effects on entities' use count.
Those changes are automatically propagated through the graph.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Stanimir Varbanov <svarbanov@mm-sol.com>
Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
---
 Documentation/media-framework.txt |   81 ++++++++++++++-
 drivers/media/media-device.c      |   45 ++++++++
 drivers/media/media-entity.c      |  208 +++++++++++++++++++++++++++++++++++++
 include/linux/media.h             |    1 +
 include/media/media-entity.h      |    8 ++
 5 files changed, 340 insertions(+), 3 deletions(-)

diff --git a/Documentation/media-framework.txt b/Documentation/media-framework.txt
index 74a137d..7894ef3 100644
--- a/Documentation/media-framework.txt
+++ b/Documentation/media-framework.txt
@@ -278,6 +278,16 @@ When the graph traversal is complete the function will return NULL.
 Graph traversal can be interrupted at any moment. No cleanup function call is
 required and the graph structure can be freed normally.
 
+Helper functions can be used to find a link between two given pads, or a pad
+connected to another pad through an active link
+
+	media_entity_find_link(struct media_pad *source,
+			       struct media_pad *sink);
+
+	media_entity_remote_pad(struct media_pad *pad);
+
+Refer to the kerneldoc documentation for more information.
+
 
 Reference counting and power handling
 -------------------------------------
@@ -316,6 +326,46 @@ is allowed to fail when turning power on, in which case the media_entity_get
 function will return NULL.
 
 
+Links setup
+-----------
+
+Link properties can be modified at runtime by calling
+
+	media_entity_setup_link(struct media_link *link, u32 flags);
+
+The flags argument contains the requested new link flags.
+
+The only configurable property is the ACTIVE link flag to activate/deactivate
+a link. Links marked with the IMMUTABLE link flag can not be activated or
+deactivated.
+
+When a link is activated or deactivated, the media framework calls the
+link_setup operation for the two entities at the source and sink of the link,
+in that order. If the second link_setup call fails, another link_setup call is
+made on the first entity to restore the original link flags.
+
+Entity drivers must implement the link_setup operation if any of their links
+is non-immutable. The operation must either configure the hardware or store
+the configuration information to be applied later.
+
+Link activation must not have any side effect on other links. If an active
+link at a sink pad prevents another link at the same pad from being
+deactivated, the link_setup operation must return -EBUSY and can't implicitly
+deactivate the first active link.
+
+Activating and deactivating a link has effects on entities' reference counts.
+When two sub-graphs are connected, the reference count of each of them is
+incremented by the total reference count of all node entities in the other
+sub-graph. When two sub-graphs are disconnected, the reverse operation is
+performed. In both cases the set_power operations are called accordingly,
+ensuring that the link_setup calls are made with power active on the source
+and sink entities.
+
+In other words, activating or deactivating a link propagates reference count
+changes through the graph, and the final state is identical to what it would
+have been if the link had been active or inactive from the start.
+
+
 Userspace application API
 -------------------------
 
@@ -439,9 +489,6 @@ Valid entity flags are
 
 	ioctl(int fd, int request, struct media_links_enum *argp);
 
-Only forward links that originate at one of the entity's source pads are
-returned during the enumeration process.
-
 To enumerate pads and/or links for a given entity, applications set the entity
 field of a media_links_enum structure and initialize the media_pad_desc and
 media_link_desc structure arrays pointed by the pads and links fields. They then
@@ -457,6 +504,9 @@ information about the entity's outbound links. The array must have enough room
 to store all the entity's outbound links. The number of outbound links can be
 retrieved with the MEDIA_IOC_ENUM_ENTITIES ioctl.
 
+Only outbound (forward) links that originate at one of the entity's source
+pads are returned during the enumeration process.
+
 The media_pad_desc, media_link_desc and media_links_enum structures are defined
 as
 
@@ -497,3 +547,28 @@ struct media_pad_desc	*pads	Pointer to a pads array allocated by the
 				application. Ignored if NULL.
 struct media_link_desc	*links	Pointer to a links array allocated by the
 				application. Ignored if NULL.
+
+
+	MEDIA_IOC_SETUP_LINK - Modify the properties of a link
+	------------------------------------------------------
+
+	ioctl(int fd, int request, struct media_link_desc *argp);
+
+To change link properties applications fill a media_link_desc structure with
+link identification information (source and sink pad) and the new requested link
+flags. They then call the MEDIA_IOC_SETUP_LINK ioctl with a pointer to that
+structure.
+
+The only configurable property is the ACTIVE link flag to activate/deactivate
+a link. Links marked with the IMMUTABLE link flag can not be activated or
+deactivated.
+
+Link activation has no side effect on other links. If an active link at the
+sink pad prevents the link from being activated, the driver returns with a
+EBUSY error code.
+
+If the specified link can't be found the driver returns with a EINVAL error
+code.
+
+The media_pad_desc and media_link_desc structures are described in the
+MEDIA_IOC_ENUM_LINKS ioctl documentation.
diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
index 7e020f9..06655d9 100644
--- a/drivers/media/media-device.c
+++ b/drivers/media/media-device.c
@@ -170,6 +170,44 @@ static long media_device_enum_links(struct media_device *mdev,
 	return 0;
 }
 
+static long media_device_setup_link(struct media_device *mdev,
+				    struct media_link_desc __user *_ulink)
+{
+	struct media_link *link = NULL;
+	struct media_link_desc ulink;
+	struct media_entity *source;
+	struct media_entity *sink;
+	int ret;
+
+	if (copy_from_user(&ulink, _ulink, sizeof(ulink)))
+		return -EFAULT;
+
+	/* Find the source and sink entities and link.
+	 */
+	source = find_entity(mdev, ulink.source.entity);
+	sink = find_entity(mdev, ulink.sink.entity);
+
+	if (source == NULL || sink == NULL)
+		return -EINVAL;
+
+	if (ulink.source.index >= source->num_pads ||
+	    ulink.sink.index >= sink->num_pads)
+		return -EINVAL;
+
+	link = media_entity_find_link(&source->pads[ulink.source.index],
+				      &sink->pads[ulink.sink.index]);
+	if (link == NULL)
+		return -EINVAL;
+
+	/* Setup the link on both entities. */
+	ret = __media_entity_setup_link(link, ulink.flags);
+
+	if (copy_to_user(_ulink, &ulink, sizeof(ulink)))
+		return -EFAULT;
+
+	return ret;
+}
+
 static long media_device_ioctl(struct file *filp, unsigned int cmd,
 			       unsigned long arg)
 {
@@ -195,6 +233,13 @@ static long media_device_ioctl(struct file *filp, unsigned int cmd,
 		mutex_unlock(&dev->graph_mutex);
 		break;
 
+	case MEDIA_IOC_SETUP_LINK:
+		mutex_lock(&dev->graph_mutex);
+		ret = media_device_setup_link(dev,
+				(struct media_link_desc __user *)arg);
+		mutex_unlock(&dev->graph_mutex);
+		break;
+
 	default:
 		ret = -ENOIOCTLCMD;
 	}
diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index da4fef6..bc97b78 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -198,6 +198,25 @@ EXPORT_SYMBOL_GPL(media_entity_graph_walk_next);
  * Power state handling
  */
 
+/*
+ * Return power count of nodes directly or indirectly connected to
+ * a given entity.
+ */
+static int media_entity_count_node(struct media_entity *entity)
+{
+	struct media_entity_graph graph;
+	int use = 0;
+
+	media_entity_graph_walk_start(&graph, entity);
+
+	while ((entity = media_entity_graph_walk_next(&graph))) {
+		if (media_entity_type(entity) == MEDIA_ENTITY_TYPE_NODE)
+			use += entity->use_count;
+	}
+
+	return use;
+}
+
 /* Apply use count to an entity. */
 static void media_entity_use_apply_one(struct media_entity *entity, int change)
 {
@@ -261,6 +280,32 @@ static int media_entity_power_apply(struct media_entity *entity, int change)
 	return ret;
 }
 
+/* Apply the power state changes when connecting two entities. */
+static int media_entity_power_connect(struct media_entity *one,
+				      struct media_entity *theother)
+{
+	int power_one = media_entity_count_node(one);
+	int power_theother = media_entity_count_node(theother);
+	int ret = 0;
+
+	ret = media_entity_power_apply(one, power_theother);
+	if (ret < 0)
+		return ret;
+
+	return media_entity_power_apply(theother, power_one);
+}
+
+static void media_entity_power_disconnect(struct media_entity *one,
+					  struct media_entity *theother)
+{
+	int power_one = media_entity_count_node(one);
+	int power_theother = media_entity_count_node(theother);
+
+	/* Powering off entities is assumed to never fail. */
+	media_entity_power_apply(one, -power_theother);
+	media_entity_power_apply(theother, -power_one);
+}
+
 /*
  * Apply use count change to graph and change power state of entities
  * accordingly.
@@ -404,3 +449,166 @@ media_entity_create_link(struct media_entity *source, u16 source_pad,
 	return 0;
 }
 EXPORT_SYMBOL(media_entity_create_link);
+
+static int __media_entity_setup_link_notify(struct media_link *link, u32 flags)
+{
+	const u32 mask = MEDIA_LINK_FLAG_ACTIVE;
+	int ret;
+
+	/* Notify both entities. */
+	ret = media_entity_call(link->source->entity, link_setup,
+				link->source, link->sink, flags);
+	if (ret < 0 && ret != -ENOIOCTLCMD)
+		return ret;
+
+	ret = media_entity_call(link->sink->entity, link_setup,
+				link->sink, link->source, flags);
+	if (ret < 0 && ret != -ENOIOCTLCMD) {
+		media_entity_call(link->source->entity, link_setup,
+				  link->source, link->sink, link->flags);
+		return ret;
+	}
+
+	link->flags = (link->flags & ~mask) | (flags & mask);
+	link->reverse->flags = link->flags;
+
+	return 0;
+}
+
+/**
+ * __media_entity_setup_link - Configure a media link
+ * @link: The link being configured
+ * @flags: Link configuration flags
+ *
+ * The bulk of link setup is handled by the two entities connected through the
+ * link. This function notifies both entities of the link configuration change.
+ *
+ * If the link is immutable or if the current and new configuration are
+ * identical, return immediately.
+ *
+ * The user is expected to hold link->source->parent->mutex. If not,
+ * media_entity_setup_link() should be used instead.
+ */
+int __media_entity_setup_link(struct media_link *link, u32 flags)
+{
+	struct media_entity *source, *sink;
+	int ret = -EBUSY;
+
+	if (link == NULL)
+		return -EINVAL;
+
+	if (link->flags & MEDIA_LINK_FLAG_IMMUTABLE)
+		return link->flags == flags ? 0 : -EINVAL;
+
+	if (link->flags == flags)
+		return 0;
+
+	source = __media_entity_get(link->source->entity);
+	if (!source)
+		return ret;
+
+	sink = __media_entity_get(link->sink->entity);
+	if (!sink)
+		goto err___media_entity_get;
+
+	if (flags & MEDIA_LINK_FLAG_ACTIVE) {
+		ret = media_entity_power_connect(source, sink);
+		if (ret < 0)
+			goto err_media_entity_power_connect;
+	}
+
+	ret = __media_entity_setup_link_notify(link, flags);
+	if (ret < 0)
+		goto err___media_entity_setup_link_notify;
+
+	if (!(flags & MEDIA_LINK_FLAG_ACTIVE))
+		media_entity_power_disconnect(source, sink);
+
+	__media_entity_put(sink);
+	__media_entity_put(source);
+
+	return 0;
+
+err___media_entity_setup_link_notify:
+	if (flags & MEDIA_LINK_FLAG_ACTIVE)
+		media_entity_power_disconnect(source, sink);
+err_media_entity_power_connect:
+	__media_entity_put(sink);
+err___media_entity_get:
+	__media_entity_put(source);
+
+	return ret;
+}
+
+int media_entity_setup_link(struct media_link *link, u32 flags)
+{
+	int ret;
+
+	mutex_lock(&link->source->entity->parent->graph_mutex);
+	ret = __media_entity_setup_link(link, flags);
+	mutex_unlock(&link->source->entity->parent->graph_mutex);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(media_entity_setup_link);
+
+/**
+ * media_entity_find_link - Find a link between two pads
+ * @source: Source pad
+ * @sink: Sink pad
+ *
+ * Return a pointer to the link between the two entities. If no such link
+ * exists, return NULL.
+ */
+struct media_link *
+media_entity_find_link(struct media_pad *source, struct media_pad *sink)
+{
+	struct media_link *link;
+	unsigned int i;
+
+	for (i = 0; i < source->entity->num_links; ++i) {
+		link = &source->entity->links[i];
+
+		if (link->source->entity == source->entity &&
+		    link->source->index == source->index &&
+		    link->sink->entity == sink->entity &&
+		    link->sink->index == sink->index)
+			return link;
+	}
+
+	return NULL;
+}
+EXPORT_SYMBOL_GPL(media_entity_find_link);
+
+/**
+ * media_entity_remote_pad - Locate the pad at the remote end of a link
+ * @entity: Local entity
+ * @pad: Pad at the local end of the link
+ *
+ * Search for a remote pad connected to the given pad by iterating over all
+ * links originating or terminating at that pad until an active link is found.
+ *
+ * Return a pointer to the pad at the remote end of the first found active link,
+ * or NULL if no active link has been found.
+ */
+struct media_pad *media_entity_remote_pad(struct media_pad *pad)
+{
+	unsigned int i;
+
+	for (i = 0; i < pad->entity->num_links; i++) {
+		struct media_link *link = &pad->entity->links[i];
+
+		if (!(link->flags & MEDIA_LINK_FLAG_ACTIVE))
+			continue;
+
+		if (link->source == pad)
+			return link->sink;
+
+		if (link->sink == pad)
+			return link->source;
+	}
+
+	return NULL;
+
+}
+EXPORT_SYMBOL_GPL(media_entity_remote_pad);
diff --git a/include/linux/media.h b/include/linux/media.h
index 542509b..edb53c2 100644
--- a/include/linux/media.h
+++ b/include/linux/media.h
@@ -100,5 +100,6 @@ struct media_links_enum {
 #define MEDIA_IOC_DEVICE_INFO		_IOWR('M', 1, struct media_device_info)
 #define MEDIA_IOC_ENUM_ENTITIES		_IOWR('M', 2, struct media_entity_desc)
 #define MEDIA_IOC_ENUM_LINKS		_IOWR('M', 3, struct media_links_enum)
+#define MEDIA_IOC_SETUP_LINK		_IOWR('M', 4, struct media_link_desc)
 
 #endif /* __LINUX_MEDIA_H */
diff --git a/include/media/media-entity.h b/include/media/media-entity.h
index 8c40d5e..0f0697b 100644
--- a/include/media/media-entity.h
+++ b/include/media/media-entity.h
@@ -18,6 +18,9 @@ struct media_pad {
 };
 
 struct media_entity_operations {
+	int (*link_setup)(struct media_entity *entity,
+			  const struct media_pad *local,
+			  const struct media_pad *remote, u32 flags);
 	int (*set_power)(struct media_entity *entity, int power);
 };
 
@@ -88,6 +91,11 @@ int media_entity_init(struct media_entity *entity, u16 num_pads,
 void media_entity_cleanup(struct media_entity *entity);
 int media_entity_create_link(struct media_entity *source, u16 source_pad,
 		struct media_entity *sink, u16 sink_pad, u32 flags);
+int __media_entity_setup_link(struct media_link *link, u32 flags);
+int media_entity_setup_link(struct media_link *link, u32 flags);
+struct media_link *media_entity_find_link(struct media_pad *source,
+		struct media_pad *sink);
+struct media_pad *media_entity_remote_pad(struct media_pad *pad);
 
 struct media_entity *media_entity_get(struct media_entity *entity);
 void media_entity_put(struct media_entity *entity);
-- 
1.7.1

[RFC/PATCH v4 09/11] v4l: Add a media_device pointer to the v4l2_device structure

[Laurent Pinchart]

The pointer will later be used to register/unregister media entities
when registering/unregistering a v4l2_subdev or a video_device.

With the introduction of media devices, device drivers need to store a
pointer to a driver-specific structure in the device's drvdata.
v4l2_device can't claim ownership of the drvdata anymore.

To maintain compatibility with drivers that rely on v4l2_device storing
a pointer to itself in the device's drvdata, v4l2_device_register() will
keep doing so if the drvdata is NULL.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 Documentation/video4linux/v4l2-framework.txt |   17 ++++++++++++-----
 drivers/media/video/v4l2-device.c            |   13 +++++++------
 include/media/v4l2-device.h                  |    2 ++
 3 files changed, 21 insertions(+), 11 deletions(-)

diff --git a/Documentation/video4linux/v4l2-framework.txt b/Documentation/video4linux/v4l2-framework.txt
index 89bd881..8a3f14e 100644
--- a/Documentation/video4linux/v4l2-framework.txt
+++ b/Documentation/video4linux/v4l2-framework.txt
@@ -83,11 +83,17 @@ You must register the device instance:
 
 	v4l2_device_register(struct device *dev, struct v4l2_device *v4l2_dev);
 
-Registration will initialize the v4l2_device struct and link dev->driver_data
-to v4l2_dev. If v4l2_dev->name is empty then it will be set to a value derived
-from dev (driver name followed by the bus_id, to be precise). If you set it
-up before calling v4l2_device_register then it will be untouched. If dev is
-NULL, then you *must* setup v4l2_dev->name before calling v4l2_device_register.
+Registration will initialize the v4l2_device struct. If the dev->driver_data
+field is NULL, it will be linked to v4l2_dev. Drivers that use the media
+device framework in addition to the V4L2 framework need to set
+dev->driver_data manually to point to the driver-specific device structure
+that embed the struct v4l2_device instance. This is achieved by a
+dev_set_drvdata() call before registering the V4L2 device instance.
+
+If v4l2_dev->name is empty then it will be set to a value derived from dev
+(driver name followed by the bus_id, to be precise). If you set it up before
+calling v4l2_device_register then it will be untouched. If dev is NULL, then
+you *must* setup v4l2_dev->name before calling v4l2_device_register.
 
 You can use v4l2_device_set_name() to set the name based on a driver name and
 a driver-global atomic_t instance. This will generate names like ivtv0, ivtv1,
@@ -108,6 +114,7 @@ You unregister with:
 
 	v4l2_device_unregister(struct v4l2_device *v4l2_dev);
 
+If the dev->driver_data field points to v4l2_dev, it will be reset to NULL.
 Unregistering will also automatically unregister all subdevs from the device.
 
 If you have a hotpluggable device (e.g. a USB device), then when a disconnect
diff --git a/drivers/media/video/v4l2-device.c b/drivers/media/video/v4l2-device.c
index b287aaa..91452cd 100644
--- a/drivers/media/video/v4l2-device.c
+++ b/drivers/media/video/v4l2-device.c
@@ -45,9 +45,8 @@ int v4l2_device_register(struct device *dev, struct v4l2_device *v4l2_dev)
 	if (!v4l2_dev->name[0])
 		snprintf(v4l2_dev->name, sizeof(v4l2_dev->name), "%s %s",
 			dev->driver->name, dev_name(dev));
-	if (dev_get_drvdata(dev))
-		v4l2_warn(v4l2_dev, "Non-NULL drvdata on register\n");
-	dev_set_drvdata(dev, v4l2_dev);
+	if (!dev_get_drvdata(dev))
+		dev_set_drvdata(dev, v4l2_dev);
 	return 0;
 }
 EXPORT_SYMBOL_GPL(v4l2_device_register);
@@ -70,10 +69,12 @@ EXPORT_SYMBOL_GPL(v4l2_device_set_name);
 
 void v4l2_device_disconnect(struct v4l2_device *v4l2_dev)
 {
-	if (v4l2_dev->dev) {
+	if (v4l2_dev->dev == NULL)
+		return;
+
+	if (dev_get_drvdata(v4l2_dev->dev) == v4l2_dev)
 		dev_set_drvdata(v4l2_dev->dev, NULL);
-		v4l2_dev->dev = NULL;
-	}
+	v4l2_dev->dev = NULL;
 }
 EXPORT_SYMBOL_GPL(v4l2_device_disconnect);
 
diff --git a/include/media/v4l2-device.h b/include/media/v4l2-device.h
index 5d5d550..83b5966 100644
--- a/include/media/v4l2-device.h
+++ b/include/media/v4l2-device.h
@@ -21,6 +21,7 @@
 #ifndef _V4L2_DEVICE_H
 #define _V4L2_DEVICE_H
 
+#include <media/media-device.h>
 #include <media/v4l2-subdev.h>
 
 /* Each instance of a V4L2 device should create the v4l2_device struct,
@@ -37,6 +38,7 @@ struct v4l2_device {
 	   Note: dev might be NULL if there is no parent device
 	   as is the case with e.g. ISA devices. */
 	struct device *dev;
+	struct media_device *mdev;
 	/* used to keep track of the registered subdevs */
 	struct list_head subdevs;
 	/* lock this struct; can be used by the driver as well if this
-- 
1.7.1

[RFC/PATCH v4 10/11] v4l: Make video_device inherit from media_entity

[Laurent Pinchart]

V4L2 devices are media entities. As such they need to inherit from
(include) the media_entity structure.

When registering/unregistering the device, the media entity is
automatically registered/unregistered. The entity is acquired on device
open and released on device close.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
---
 Documentation/video4linux/v4l2-framework.txt |   38 +++++++++++++++++++++++--
 drivers/media/video/v4l2-dev.c               |   35 ++++++++++++++++++++++-
 include/media/v4l2-dev.h                     |    6 ++++
 3 files changed, 74 insertions(+), 5 deletions(-)

diff --git a/Documentation/video4linux/v4l2-framework.txt b/Documentation/video4linux/v4l2-framework.txt
index 8a3f14e..7ff4016 100644
--- a/Documentation/video4linux/v4l2-framework.txt
+++ b/Documentation/video4linux/v4l2-framework.txt
@@ -71,6 +71,10 @@ sub-device instances, the video_device struct stores V4L2 device node data
 and in the future a v4l2_fh struct will keep track of filehandle instances
 (this is not yet implemented).
 
+The V4L2 framework also optionally integrates with the media framework. If a
+driver sets the struct v4l2_device mdev field, sub-devices and video nodes
+will automatically appear in the media framework as entities.
+
 
 struct v4l2_device
 ------------------
@@ -84,11 +88,14 @@ You must register the device instance:
 	v4l2_device_register(struct device *dev, struct v4l2_device *v4l2_dev);
 
 Registration will initialize the v4l2_device struct. If the dev->driver_data
-field is NULL, it will be linked to v4l2_dev. Drivers that use the media
-device framework in addition to the V4L2 framework need to set
+field is NULL, it will be linked to v4l2_dev.
+
+Drivers that want integration with the media device framework need to set
 dev->driver_data manually to point to the driver-specific device structure
 that embed the struct v4l2_device instance. This is achieved by a
-dev_set_drvdata() call before registering the V4L2 device instance.
+dev_set_drvdata() call before registering the V4L2 device instance. They must
+also set the struct v4l2_device mdev field to point to a properly initialized
+and registered media_device instance.
 
 If v4l2_dev->name is empty then it will be set to a value derived from dev
 (driver name followed by the bus_id, to be precise). If you set it up before
@@ -523,6 +530,21 @@ If you use v4l2_ioctl_ops, then you should set either .unlocked_ioctl or
 The v4l2_file_operations struct is a subset of file_operations. The main
 difference is that the inode argument is omitted since it is never used.
 
+If integration with the media framework is needed, you must initialize the
+media_entity struct embedded in the video_device struct (entity field) by
+calling media_entity_init():
+
+	struct media_pad *pad = &my_vdev->pad;
+	int err;
+
+	err = media_entity_init(&vdev->entity, 1, pad, 0);
+
+The pads array must have been previously initialized. There is no need to
+manually set the struct media_entity type and name fields.
+
+A reference to the entity will be automatically acquired/released when the
+video device is opened/closed.
+
 
 video_device registration
 -------------------------
@@ -536,6 +558,9 @@ for you.
 		return err;
 	}
 
+If the v4l2_device parent device has a non-NULL mdev field, the video device
+entity will be automatically registered with the media device.
+
 Which device is registered depends on the type argument. The following
 types exist:
 
@@ -613,6 +638,13 @@ those will still be passed on since some buffer ioctls may still be needed.
 When the last user of the video device node exits, then the vdev->release()
 callback is called and you can do the final cleanup there.
 
+Don't forget to cleanup the media entity associated with the video device if
+it has been initialized:
+
+	media_entity_cleanup(&vdev->entity);
+
+This can be done from the release callback.
+
 
 video_device helper functions
 -----------------------------
diff --git a/drivers/media/video/v4l2-dev.c b/drivers/media/video/v4l2-dev.c
index bcd47a0..0cbd3e6 100644
--- a/drivers/media/video/v4l2-dev.c
+++ b/drivers/media/video/v4l2-dev.c
@@ -269,6 +269,7 @@ static int v4l2_mmap(struct file *filp, struct vm_area_struct *vm)
 static int v4l2_open(struct inode *inode, struct file *filp)
 {
 	struct video_device *vdev;
+	struct media_entity *entity = NULL;
 	int ret = 0;
 
 	/* Check if the video device is available */
@@ -283,12 +284,23 @@ static int v4l2_open(struct inode *inode, struct file *filp)
 	/* and increase the device refcount */
 	video_get(vdev);
 	mutex_unlock(&videodev_lock);
+	if (vdev->v4l2_dev && vdev->v4l2_dev->mdev) {
+		entity = media_entity_get(&vdev->entity);
+		if (!entity) {
+			ret = -EBUSY;
+			video_put(vdev);
+			return ret;
+		}
+	}
 	if (vdev->fops->open)
 		ret = vdev->fops->open(filp);
 
 	/* decrease the refcount in case of an error */
-	if (ret)
+	if (ret) {
+		if (vdev->v4l2_dev && vdev->v4l2_dev->mdev)
+			media_entity_put(entity);
 		video_put(vdev);
+	}
 	return ret;
 }
 
@@ -301,6 +313,9 @@ static int v4l2_release(struct inode *inode, struct file *filp)
 	if (vdev->fops->release)
 		vdev->fops->release(filp);
 
+	if (vdev->v4l2_dev && vdev->v4l2_dev->mdev)
+		media_entity_put(&vdev->entity);
+
 	/* decrease the refcount unconditionally since the release()
 	   return value is ignored. */
 	video_put(vdev);
@@ -563,11 +578,24 @@ int __video_register_device(struct video_device *vdev, int type, int nr,
 		printk(KERN_WARNING "%s: requested %s%d, got %s\n", __func__,
 			name_base, nr, video_device_node_name(vdev));
 
-	/* Part 5: Activate this minor. The char device can now be used. */
+	/* Part 5: Register the entity. */
+	if (vdev->v4l2_dev && vdev->v4l2_dev->mdev) {
+		vdev->entity.type = MEDIA_ENTITY_TYPE_NODE_V4L;
+		vdev->entity.name = vdev->name;
+		vdev->entity.v4l.major = VIDEO_MAJOR;
+		vdev->entity.v4l.minor = vdev->minor;
+		ret = media_device_register_entity(vdev->v4l2_dev->mdev,
+			&vdev->entity);
+		if (ret < 0)
+			printk(KERN_ERR "error\n"); /* TODO */
+	}
+
+	/* Part 6: Activate this minor. The char device can now be used. */
 	set_bit(V4L2_FL_REGISTERED, &vdev->flags);
 	mutex_lock(&videodev_lock);
 	video_device[vdev->minor] = vdev;
 	mutex_unlock(&videodev_lock);
+
 	return 0;
 
 cleanup:
@@ -595,6 +623,9 @@ void video_unregister_device(struct video_device *vdev)
 	if (!vdev || !video_is_registered(vdev))
 		return;
 
+	if (vdev->v4l2_dev && vdev->v4l2_dev->mdev)
+		media_device_unregister_entity(&vdev->entity);
+
 	clear_bit(V4L2_FL_REGISTERED, &vdev->flags);
 	device_unregister(&vdev->dev);
 }
diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h
index 195fa56..447b154 100644
--- a/include/media/v4l2-dev.h
+++ b/include/media/v4l2-dev.h
@@ -16,6 +16,8 @@
 #include <linux/mutex.h>
 #include <linux/videodev2.h>
 
+#include <media/media-entity.h>
+
 #define VIDEO_MAJOR	81
 
 #define VFL_TYPE_GRABBER	0
@@ -57,6 +59,8 @@ struct v4l2_file_operations {
 
 struct video_device
 {
+	struct media_entity entity;
+
 	/* device ops */
 	const struct v4l2_file_operations *fops;
 
@@ -96,6 +100,8 @@ struct video_device
 	const struct v4l2_ioctl_ops *ioctl_ops;
 };
 
+#define media_entity_to_video_device(entity) \
+	container_of(entity, struct video_device, entity)
 /* dev to video-device */
 #define to_video_device(cd) container_of(cd, struct video_device, dev)
 
-- 
1.7.1

[RFC/PATCH v4 11/11] v4l: Make v4l2_subdev inherit from media_entity

[Laurent Pinchart]

V4L2 subdevices are media entities. As such they need to inherit from
(include) the media_entity structure.

When registering/unregistering the subdevice, the media entity is
automatically registered/unregistered. The entity is acquired on device
open and released on device close.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
---
 Documentation/video4linux/v4l2-framework.txt |   23 ++++++++++++++++++
 drivers/media/video/v4l2-device.c            |   32 +++++++++++++++++++++-----
 drivers/media/video/v4l2-subdev.c            |   27 +++++++++++++++++++++-
 include/media/v4l2-subdev.h                  |    7 +++++
 4 files changed, 82 insertions(+), 7 deletions(-)

diff --git a/Documentation/video4linux/v4l2-framework.txt b/Documentation/video4linux/v4l2-framework.txt
index 7ff4016..3416d93 100644
--- a/Documentation/video4linux/v4l2-framework.txt
+++ b/Documentation/video4linux/v4l2-framework.txt
@@ -263,6 +263,26 @@ A sub-device driver initializes the v4l2_subdev struct using:
 Afterwards you need to initialize subdev->name with a unique name and set the
 module owner. This is done for you if you use the i2c helper functions.
 
+If integration with the media framework is needed, you must initialize the
+media_entity struct embedded in the v4l2_subdev struct (entity field) by
+calling media_entity_init():
+
+	struct media_pad *pads = &my_sd->pads;
+	int err;
+
+	err = media_entity_init(&sd->entity, npads, pads, 0);
+
+The pads array must have been previously initialized. There is no need to
+manually set the struct media_entity type and name fields, but the revision
+field must be initialized if needed.
+
+A reference to the entity will be automatically acquired/released when the
+subdev device node (if any) is opened/closed.
+
+Don't forget to cleanup the media entity before the sub-device is destroyed:
+
+	media_entity_cleanup(&sd->entity);
+
 A device (bridge) driver needs to register the v4l2_subdev with the
 v4l2_device:
 
@@ -272,6 +292,9 @@ This can fail if the subdev module disappeared before it could be registered.
 After this function was called successfully the subdev->dev field points to
 the v4l2_device.
 
+If the v4l2_device parent device has a non-NULL mdev field, the sub-device
+entity will be automatically registered with the media device.
+
 You can unregister a sub-device using:
 
 	v4l2_device_unregister_subdev(sd);
diff --git a/drivers/media/video/v4l2-device.c b/drivers/media/video/v4l2-device.c
index 91452cd..4f74d01 100644
--- a/drivers/media/video/v4l2-device.c
+++ b/drivers/media/video/v4l2-device.c
@@ -114,10 +114,11 @@ void v4l2_device_unregister(struct v4l2_device *v4l2_dev)
 EXPORT_SYMBOL_GPL(v4l2_device_unregister);
 
 int v4l2_device_register_subdev(struct v4l2_device *v4l2_dev,
-						struct v4l2_subdev *sd)
+				struct v4l2_subdev *sd)
 {
+	struct media_entity *entity = &sd->entity;
 	struct video_device *vdev;
-	int ret = 0;
+	int ret;
 
 	/* Check for valid input */
 	if (v4l2_dev == NULL || sd == NULL || !sd->name[0])
@@ -129,6 +130,15 @@ int v4l2_device_register_subdev(struct v4l2_device *v4l2_dev,
 	if (!try_module_get(sd->owner))
 		return -ENODEV;
 
+	/* Register the entity. */
+	if (v4l2_dev->mdev) {
+		ret = media_device_register_entity(v4l2_dev->mdev, entity);
+		if (ret < 0) {
+			module_put(sd->owner);
+			return ret;
+		}
+	}
+
 	sd->v4l2_dev = v4l2_dev;
 	spin_lock(&v4l2_dev->lock);
 	list_add_tail(&sd->list, &v4l2_dev->subdevs);
@@ -143,26 +153,36 @@ int v4l2_device_register_subdev(struct v4l2_device *v4l2_dev,
 	if (sd->flags & V4L2_SUBDEV_FL_HAS_DEVNODE) {
 		ret = __video_register_device(vdev, VFL_TYPE_SUBDEV, -1, 1,
 					      sd->owner);
-		if (ret < 0)
+		if (ret < 0) {
 			v4l2_device_unregister_subdev(sd);
+			return ret;
+		}
 	}
 
-	return ret;
+	entity->v4l.major = VIDEO_MAJOR;
+	entity->v4l.minor = vdev->minor;
+	return 0;
 }
 EXPORT_SYMBOL_GPL(v4l2_device_register_subdev);
 
 void v4l2_device_unregister_subdev(struct v4l2_subdev *sd)
 {
+	struct v4l2_device *v4l2_dev;
+
 	/* return if it isn't registered */
 	if (sd == NULL || sd->v4l2_dev == NULL)
 		return;
 
-	spin_lock(&sd->v4l2_dev->lock);
+	v4l2_dev = sd->v4l2_dev;
+
+	spin_lock(&v4l2_dev->lock);
 	list_del(&sd->list);
-	spin_unlock(&sd->v4l2_dev->lock);
+	spin_unlock(&v4l2_dev->lock);
 	sd->v4l2_dev = NULL;
 
 	module_put(sd->owner);
+	if (v4l2_dev->mdev)
+		media_device_unregister_entity(&sd->entity);
 	video_unregister_device(&sd->devnode);
 }
 EXPORT_SYMBOL_GPL(v4l2_device_unregister_subdev);
diff --git a/drivers/media/video/v4l2-subdev.c b/drivers/media/video/v4l2-subdev.c
index b063195..1efa267 100644
--- a/drivers/media/video/v4l2-subdev.c
+++ b/drivers/media/video/v4l2-subdev.c
@@ -32,7 +32,8 @@ static int subdev_open(struct file *file)
 {
 	struct video_device *vdev = video_devdata(file);
 	struct v4l2_subdev *sd = vdev_to_v4l2_subdev(vdev);
-	struct v4l2_fh *vfh;
+	struct media_entity *entity;
+	struct v4l2_fh *vfh = NULL;
 	int ret;
 
 	if (!sd->initialized)
@@ -59,10 +60,17 @@ static int subdev_open(struct file *file)
 		file->private_data = vfh;
 	}
 
+	entity = media_entity_get(&sd->entity);
+	if (!entity) {
+		ret = -EBUSY;
+		goto err;
+	}
+
 	return 0;
 
 err:
 	if (vfh != NULL) {
+		v4l2_fh_del(vfh);
 		v4l2_fh_exit(vfh);
 		kfree(vfh);
 	}
@@ -72,8 +80,12 @@ err:
 
 static int subdev_close(struct file *file)
 {
+	struct video_device *vdev = video_devdata(file);
+	struct v4l2_subdev *sd = vdev_to_v4l2_subdev(vdev);
 	struct v4l2_fh *vfh = file->private_data;
 
+	media_entity_put(&sd->entity);
+
 	if (vfh != NULL) {
 		v4l2_fh_del(vfh);
 		v4l2_fh_exit(vfh);
@@ -172,5 +184,18 @@ void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
 	sd->grp_id = 0;
 	sd->priv = NULL;
 	sd->initialized = 1;
+	sd->entity.name = sd->name;
+	sd->entity.type = MEDIA_ENTITY_TYPE_SUBDEV;
 }
 EXPORT_SYMBOL(v4l2_subdev_init);
+
+int v4l2_subdev_set_power(struct media_entity *entity, int power)
+{
+	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+
+	dev_dbg(entity->parent->dev,
+		"%s power%s\n", entity->name, power ? "on" : "off");
+
+	return v4l2_subdev_call(sd, core, s_power, power);
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_set_power);
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 55a8c93..f9e1897 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -21,6 +21,7 @@
 #ifndef _V4L2_SUBDEV_H
 #define _V4L2_SUBDEV_H
 
+#include <media/media-entity.h>
 #include <media/v4l2-common.h>
 #include <media/v4l2-dev.h>
 #include <media/v4l2-mediabus.h>
@@ -421,6 +422,8 @@ struct v4l2_subdev_ops {
    stand-alone or embedded in a larger struct.
  */
 struct v4l2_subdev {
+	struct media_entity entity;
+
 	struct list_head list;
 	struct module *owner;
 	u32 flags;
@@ -439,6 +442,8 @@ struct v4l2_subdev {
 	unsigned int nevents;
 };
 
+#define media_entity_to_v4l2_subdev(ent) \
+	container_of(ent, struct v4l2_subdev, entity)
 #define vdev_to_v4l2_subdev(vdev) \
 	container_of(vdev, struct v4l2_subdev, devnode)
 
@@ -457,6 +462,8 @@ static inline void *v4l2_get_subdevdata(const struct v4l2_subdev *sd)
 void v4l2_subdev_init(struct v4l2_subdev *sd,
 		      const struct v4l2_subdev_ops *ops);
 
+int v4l2_subdev_set_power(struct media_entity *entity, int power);
+
 /* Call an ops of a v4l2_subdev, doing the right checks against
    NULL pointers.
 
-- 
1.7.1

Re: [RFC/PATCH v4 02/11] media: Media device

[Hans Verkuil]

On Friday, August 20, 2010 17:29:04 Laurent Pinchart wrote:
> The media_device structure abstracts functions common to all kind of
> media devices (v4l2, dvb, alsa, ...). It manages media entities and
> offers a userspace API to discover and configure the media device
> internal topology.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
>  Documentation/media-framework.txt |   91 ++++++++++++++++++++++++++++++++++
>  drivers/media/Makefile            |    2 +-
>  drivers/media/media-device.c      |   98 +++++++++++++++++++++++++++++++++++++
>  include/media/media-device.h      |   64 ++++++++++++++++++++++++
>  4 files changed, 254 insertions(+), 1 deletions(-)
>  create mode 100644 Documentation/media-framework.txt
>  create mode 100644 drivers/media/media-device.c
>  create mode 100644 include/media/media-device.h
> 
> diff --git a/Documentation/media-framework.txt b/Documentation/media-framework.txt
> new file mode 100644
> index 0000000..89dc7ad
> --- /dev/null
> +++ b/Documentation/media-framework.txt
> @@ -0,0 +1,91 @@
> +Linux kernel media framework
> +============================
> +
> +This document describes the Linux kernel media framework, its data structures,
> +functions and their usage.
> +
> +
> +Introduction
> +------------
> +
> +Media devices increasingly handle multiple related functions. Many USB cameras
> +include microphones, video capture hardware can also output video, or SoC
> +camera interfaces also perform memory-to-memory operations similar to video
> +codecs.
> +
> +Independent functions, even when implemented in the same hardware, can be
> +modeled by separate devices. A USB camera with a microphone will be presented
> +to userspace applications as V4L2 and ALSA capture devices. The devices
> +relationships (when using a webcam, end-users shouldn't have to manually
> +select the associated USB microphone), while not made available directly to
> +applications by the drivers, can usually be retrieved from sysfs.
> +
> +With more and more advanced SoC devices being introduced, the current approach
> +will not scale. Device topologies are getting increasingly complex and can't
> +always be represented by a tree structure. Hardware blocks are shared between
> +different functions, creating dependencies between seemingly unrelated
> +devices.
> +
> +Kernel abstraction APIs such as V4L2 and ALSA provide means for applications
> +to access hardware parameters. As newer hardware expose an increasingly high
> +number of those parameters, drivers need to guess what applications really
> +require based on limited information, thereby implementing policies that
> +belong to userspace.
> +
> +The media kernel API aims at solving those problems.
> +
> +
> +Media device
> +------------
> +
> +A media device is represented by a struct media_device instance, defined in
> +include/media/media-device.h. Allocation of the structure is handled by the
> +media device driver, usually by embedding the media_device instance in a
> +larger driver-specific structure.
> +
> +Drivers register media device instances by calling
> +
> +	media_device_register(struct media_device *mdev);
> +
> +The caller is responsible for initializing the media_device structure before
> +registration. The following fields must be set:
> +
> + - dev must point to the parent device (usually a pci_dev, usb_interface or
> +   platform_device instance).
> +
> + - model must be filled with the device model name as a NUL-terminated UTF-8
> +   string. The device/model revision must not be stored in this field.
> +
> +The following fields are optional:
> +
> + - serial is a unique serial number stored as an ASCII string. The string must
> +   be NUL-terminated unless exactly 32 characters long. This allows storing
> +   GUIDs in a text form. If the hardware doesn't provide a unique serial
> +   number this field must be left empty.
> +
> + - bus_info represents the location of the device in the system as a
> +   NUL-terminated ASCII string. For PCI/PCIe devices bus_info must be set to
> +   "PCI:" (or "PCIe:") followed by the value of pci_name(). For USB devices,
> +   the usb_make_path() function must be used. This field is used by
> +   applications to distinguish between otherwise identical devices that don't
> +   provide a serial number.
> +
> + - device_version is the hardware device version number in a driver-specific
> +   format. When possible the version should be formatted with the
> +   KERNEL_VERSION macro.
> +
> + - driver_version is formatted with the KERNEL_VERSION macro. The version
> +   minor must be incremented when new features are added to the userspace API
> +   without breaking binary compatibility. The version major must be
> +   incremented when binary compatibility is broken.
> +
> +Upon successful registration a character device named media[0-9]+ is created.
> +The device major and minor numbers are dynamic. The model name is exported as
> +a sysfs attribute.
> +
> +Drivers unregister media device instances by calling
> +
> +	media_device_unregister(struct media_device *mdev);
> +
> +Unregistering a media device that hasn't been registered is *NOT* safe.
> +
> diff --git a/drivers/media/Makefile b/drivers/media/Makefile
> index c1b5938..f8d8dcb 100644
> --- a/drivers/media/Makefile
> +++ b/drivers/media/Makefile
> @@ -2,7 +2,7 @@
>  # Makefile for the kernel multimedia device drivers.
>  #
>  
> -media-objs	:= media-devnode.o
> +media-objs	:= media-device.o media-devnode.o
>  
>  obj-$(CONFIG_MEDIA_SUPPORT)	+= media.o
>  
> diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> new file mode 100644
> index 0000000..781c641
> --- /dev/null
> +++ b/drivers/media/media-device.c
> @@ -0,0 +1,98 @@
> +/*
> + *  Media device support.
> + *
> + *  Copyright (C) 2010  Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> + *
> + *  This program is free software; you can redistribute it and/or modify
> + *  it under the terms of the GNU General Public License as published by
> + *  the Free Software Foundation; either version 2 of the License, or
> + *  (at your option) any later version.
> + *
> + *  This program is distributed in the hope that it will be useful,
> + *  but WITHOUT ANY WARRANTY; without even the implied warranty of
> + *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + *  GNU General Public License for more details.
> + *
> + *  You should have received a copy of the GNU General Public License
> + *  along with this program; if not, write to the Free Software
> + *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
> + */
> +
> +#include <linux/types.h>
> +#include <linux/ioctl.h>
> +
> +#include <media/media-device.h>
> +#include <media/media-devnode.h>
> +
> +static const struct media_file_operations media_device_fops = {
> +	.owner = THIS_MODULE,
> +};
> +
> +/* -----------------------------------------------------------------------------
> + * sysfs
> + */
> +
> +static ssize_t show_model(struct device *cd,
> +			  struct device_attribute *attr, char *buf)
> +{
> +	struct media_device *mdev = to_media_device(to_media_devnode(cd));
> +
> +	return sprintf(buf, "%.*s\n", (int)sizeof(mdev->model), mdev->model);
> +}
> +
> +static DEVICE_ATTR(model, S_IRUGO, show_model, NULL);
> +
> +/* -----------------------------------------------------------------------------
> + * Registration/unregistration
> + */
> +
> +static void media_device_release(struct media_devnode *mdev)
> +{
> +}
> +
> +/**
> + * media_device_register - register a media device
> + * @mdev:	The media device
> + *
> + * The caller is responsible for initializing the media device before
> + * registration. The following fields must be set:
> + *
> + * - dev must point to the parent device
> + * - model must be filled with the device model name
> + */
> +int __must_check media_device_register(struct media_device *mdev)
> +{
> +	int ret;
> +
> +	if (WARN_ON(mdev->dev == NULL || mdev->model[0] == 0))
> +		return 0;
> +
> +	/* Register the device node. */
> +	mdev->devnode.fops = &media_device_fops;
> +	mdev->devnode.parent = mdev->dev;
> +	mdev->devnode.release = media_device_release;
> +	ret = media_devnode_register(&mdev->devnode);
> +	if (ret < 0)
> +		return ret;
> +
> +	ret = device_create_file(&mdev->devnode.dev, &dev_attr_model);
> +	if (ret < 0) {
> +		media_devnode_unregister(&mdev->devnode);
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(media_device_register);
> +
> +/**
> + * media_device_unregister - unregister a media device
> + * @mdev:	The media device
> + *
> + */
> +void media_device_unregister(struct media_device *mdev)
> +{
> +	device_remove_file(&mdev->devnode.dev, &dev_attr_model);
> +	media_devnode_unregister(&mdev->devnode);
> +}
> +EXPORT_SYMBOL_GPL(media_device_unregister);
> diff --git a/include/media/media-device.h b/include/media/media-device.h
> new file mode 100644
> index 0000000..4fe949e
> --- /dev/null
> +++ b/include/media/media-device.h
> @@ -0,0 +1,64 @@
> +/*
> + *  Media device support header.
> + *
> + *  Copyright (C) 2010  Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> + *
> + *  This program is free software; you can redistribute it and/or modify
> + *  it under the terms of the GNU General Public License as published by
> + *  the Free Software Foundation; either version 2 of the License, or
> + *  (at your option) any later version.
> + *
> + *  This program is distributed in the hope that it will be useful,
> + *  but WITHOUT ANY WARRANTY; without even the implied warranty of
> + *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + *  GNU General Public License for more details.
> + *
> + *  You should have received a copy of the GNU General Public License
> + *  along with this program; if not, write to the Free Software
> + *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
> + */
> +
> +#ifndef _MEDIA_DEVICE_H
> +#define _MEDIA_DEVICE_H
> +
> +#include <linux/device.h>
> +#include <linux/list.h>
> +
> +#include <media/media-devnode.h>
> +
> +/**
> + * struct media_device - Media device
> + * @dev:	Parent device
> + * @devnode:	Media device node
> + * @model:	Device model name
> + * @serial:	Device serial number (optional)
> + * @bus_info:	Unique and stable device location identifier
> + * @device_version: Hardware device version
> + * @driver_version: Device driver version
> + *
> + * This structure represents an abstract high-level media device. It allows easy
> + * access to entities and provides basic media device-level support. The
> + * structure can be allocated directly or embedded in a larger structure.
> + *
> + * The parent @dev is a physical device. It must be set before registering the
> + * media device.
> + *
> + * @model is a descriptive model name exported through sysfs. It doesn't have to
> + * be unique.
> + */
> +struct media_device {
> +	/* dev->driver_data points to this struct. */
> +	struct device *dev;
> +	struct media_devnode devnode;
> +
> +	u8 model[32];
> +	u8 serial[32];
> +	u8 bus_info[32];
> +	u32 device_version;

I prefer hw_revision or possibly hw_device_revision. 'device' is too ambiguous.
And 'revision' is more applicable to hardware than 'version' IMHO.


Regards,

	Hans

> +	u32 driver_version;
> +};
> +
> +int __must_check media_device_register(struct media_device *mdev);
> +void media_device_unregister(struct media_device *mdev);
> +
> +#endif
> 

-- 
Hans Verkuil - video4linux developer - sponsored by TANDBERG, part of Cisco

Re: [RFC/PATCH v4 03/11] media: Entities, pads and links

[Hans Verkuil]

> +#define MEDIA_ENTITY_TYPE_NODE			(1 << MEDIA_ENTITY_TYPE_SHIFT)
> +#define MEDIA_ENTITY_TYPE_NODE_V4L		(MEDIA_ENTITY_TYPE_NODE + 1)
> +#define MEDIA_ENTITY_TYPE_NODE_FB		(MEDIA_ENTITY_TYPE_NODE + 2)
> +#define MEDIA_ENTITY_TYPE_NODE_ALSA		(MEDIA_ENTITY_TYPE_NODE + 3)
> +#define MEDIA_ENTITY_TYPE_NODE_DVB		(MEDIA_ENTITY_TYPE_NODE + 4)

During discussions at work I realized that another type that might be needed
in the future (not needed in the first version, I think) is NODE_MTB for flash
memory. There are devices that have flash memory on board (basically a kind of
BIOS) and it would be handy for a flash utility to find the corresponding mtd
device.

It shouldn't be hard to add this when needed.

Regards,

	Hans

-- 
Hans Verkuil - video4linux developer - sponsored by TANDBERG, part of Cisco

Re: [RFC/PATCH v4 06/11] media: Media device information query

[Hans Verkuil]

On Friday, August 20, 2010 17:29:08 Laurent Pinchart wrote:
> Create the following ioctl and implement it at the media device level to
> query device information.
> 
> - MEDIA_IOC_DEVICE_INFO: Query media device information
> 
> The ioctl and its data structure are defined in the new kernel header
> linux/media.h available to userspace applications.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
>  Documentation/media-framework.txt |   42 +++++++++++++++++++++++++++
>  drivers/media/media-device.c      |   57 +++++++++++++++++++++++++++++++++++++
>  include/linux/media.h             |   23 +++++++++++++++
>  include/media/media-device.h      |    3 ++
>  4 files changed, 125 insertions(+), 0 deletions(-)
>  create mode 100644 include/linux/media.h
> 
> diff --git a/Documentation/media-framework.txt b/Documentation/media-framework.txt
> index 59649e9..66f7f6c 100644
> --- a/Documentation/media-framework.txt
> +++ b/Documentation/media-framework.txt
> @@ -315,3 +315,45 @@ required, drivers don't need to provide a set_power operation. The operation
>  is allowed to fail when turning power on, in which case the media_entity_get
>  function will return NULL.
>  
> +
> +Userspace application API
> +-------------------------
> +
> +Media devices offer an API to userspace application to query device information
> +through ioctls.
> +
> +	MEDIA_IOC_DEVICE_INFO - Get device information
> +	----------------------------------------------
> +
> +	ioctl(int fd, int request, struct media_device_info *argp);
> +
> +To query device information, call the MEDIA_IOC_ENUM_ENTITIES ioctl with a
> +pointer to a media_device_info structure. The driver fills the structure and
> +returns the information to the application. The ioctl never fails.
> +
> +The media_device_info structure is defined as
> +
> +- struct media_device_info
> +
> +__u8	driver[16]	Driver name as a NUL-terminated ASCII string. The
> +			driver version is stored in the driver_version field.

Proposed improvement: "Name of the driver implementing the media API
as a NUL-terminated ASCII string."

The media API overarches multiple drivers, so it's probably useful to say
which driver name should be filled in here.

> +__u8	model[32]	Device model name as a NUL-terminated UTF-8 string. The
> +			device version is stored in the device_version field and
> +			is not be appended to the model name.

Why UTF-8 instead of ASCII?

> +__u8	serial[32]	Serial number as an ASCII string. The string is
> +			NUL-terminated unless the serial number is exactly 32
> +			characters long.
> +__u8	bus_info[32]	Location of the device in the system as a NUL-terminated
> +			ASCII string. This includes the bus type name (PCI, USB,
> +			...) and a bus-specific identifier.
> +__u32	media_version	Media API version, formatted with the KERNEL_VERSION
> +			macro.
> +__u32	device_version	Media device driver version in a driver-specific format.
> +__u32	driver_version	Media device driver version, formatted with the
> +			KERNEL_VERSION macro.

These last two are very confusing. Does device_version actually refer to the
hardware revision? In that case the description next to it is really wrong.
And I also think it should be renamed to hw_revision.

> +
> +The serial and bus_info fields can be used to distinguish between multiple
> +instances of otherwise identical hardware. The serial number takes precedence
> +when provided and can be assumed to be unique. If the serial number is an
> +empty string, the bus_info field can be used instead. The bus_info field is
> +guaranteed to be unique, but can vary across reboots or device unplug/replug.
> diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> index c309d3c..1415ebd 100644
> --- a/drivers/media/media-device.c
> +++ b/drivers/media/media-device.c
> @@ -20,13 +20,70 @@
>  
>  #include <linux/types.h>
>  #include <linux/ioctl.h>
> +#include <linux/media.h>
>  
>  #include <media/media-device.h>
>  #include <media/media-devnode.h>
>  #include <media/media-entity.h>
>  
> +/* -----------------------------------------------------------------------------
> + * Userspace API
> + */
> +
> +static int media_device_open(struct file *filp)
> +{
> +	return 0;
> +}
> +
> +static int media_device_close(struct file *filp)
> +{
> +	return 0;
> +}
> +
> +static int media_device_get_info(struct media_device *dev,
> +				 struct media_device_info __user *__info)
> +{
> +	struct media_device_info info;
> +
> +	memset(&info, 0, sizeof(info));

No need to zero the whole struct. info.reserved would be enough.

> +
> +	strlcpy(info.driver, dev->dev->driver->name, sizeof(info.driver));
> +	strlcpy(info.model, dev->model, sizeof(info.model));
> +	strncpy(info.serial, dev->serial, sizeof(info.serial));

Why not strlcpy?

> +	strlcpy(info.bus_info, dev->bus_info, sizeof(info.bus_info));
> +
> +	info.media_version = MEDIA_API_VERSION;
> +	info.device_version = dev->device_version;
> +	info.driver_version = dev->driver_version;
> +
> +	return copy_to_user(__info, &info, sizeof(*__info));
> +}
> +
> +static long media_device_ioctl(struct file *filp, unsigned int cmd,
> +			       unsigned long arg)
> +{
> +	struct media_devnode *devnode = media_devnode_data(filp);
> +	struct media_device *dev = to_media_device(devnode);
> +	long ret;
> +
> +	switch (cmd) {
> +	case MEDIA_IOC_DEVICE_INFO:
> +		ret = media_device_get_info(dev,
> +				(struct media_device_info __user *)arg);
> +		break;
> +
> +	default:
> +		ret = -ENOIOCTLCMD;
> +	}
> +
> +	return ret;
> +}
> +
>  static const struct media_file_operations media_device_fops = {
>  	.owner = THIS_MODULE,
> +	.open = media_device_open,
> +	.unlocked_ioctl = media_device_ioctl,
> +	.release = media_device_close,
>  };
>  
>  /* -----------------------------------------------------------------------------
> diff --git a/include/linux/media.h b/include/linux/media.h
> new file mode 100644
> index 0000000..bca08a7
> --- /dev/null
> +++ b/include/linux/media.h
> @@ -0,0 +1,23 @@
> +#ifndef __LINUX_MEDIA_H
> +#define __LINUX_MEDIA_H
> +
> +#include <linux/ioctl.h>
> +#include <linux/types.h>
> +#include <linux/version.h>
> +
> +#define MEDIA_API_VERSION	KERNEL_VERSION(0, 1, 0)
> +
> +struct media_device_info {
> +	__u8 driver[16];
> +	__u8 model[32];
> +	__u8 serial[32];
> +	__u8 bus_info[32];
> +	__u32 media_version;
> +	__u32 device_version;
> +	__u32 driver_version;
> +	__u32 reserved[5];

I'd increase this to reserved[33] as [5] seems very low to me.
Total struct size is then 256 bytes.

> +};
> +
> +#define MEDIA_IOC_DEVICE_INFO		_IOWR('M', 1, struct media_device_info)
> +
> +#endif /* __LINUX_MEDIA_H */
> diff --git a/include/media/media-device.h b/include/media/media-device.h
> index 3c9a5e0..6f57f41 100644
> --- a/include/media/media-device.h
> +++ b/include/media/media-device.h
> @@ -73,6 +73,9 @@ struct media_device {
>  	struct mutex graph_mutex;
>  };
>  
> +/* media_devnode to media_device */
> +#define to_media_device(node) container_of(node, struct media_device, devnode)
> +
>  int __must_check media_device_register(struct media_device *mdev);
>  void media_device_unregister(struct media_device *mdev);
>  
> 

Regards,

	Hans

-- 
Hans Verkuil - video4linux developer - sponsored by TANDBERG, part of Cisco

Re: [RFC/PATCH v4 07/11] media: Entities, pads and links enumeration

[Hans Verkuil]

On Friday, August 20, 2010 17:29:09 Laurent Pinchart wrote:
> Create the following two ioctls and implement them at the media device
> level to enumerate entities, pads and links.
> 
> - MEDIA_IOC_ENUM_ENTITIES: Enumerate entities and their properties
> - MEDIA_IOC_ENUM_LINKS: Enumerate all pads and links for a given entity
> 
> Entity IDs can be non-contiguous. Userspace applications should
> enumerate entities using the MEDIA_ENTITY_ID_FLAG_NEXT flag. When the
> flag is set in the entity ID, the MEDIA_IOC_ENUM_ENTITIES will return
> the next entity with an ID bigger than the requested one.
> 
> Only forward links that originate at one of the entity's source pads are
> returned during the enumeration process.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
> ---
>  Documentation/media-framework.txt |  142 ++++++++++++++++++++++++++++++++++++-
>  drivers/media/media-device.c      |  123 ++++++++++++++++++++++++++++++++
>  include/linux/media.h             |   81 +++++++++++++++++++++
>  include/media/media-entity.h      |   24 +------
>  4 files changed, 346 insertions(+), 24 deletions(-)
> 
> diff --git a/Documentation/media-framework.txt b/Documentation/media-framework.txt
> index 66f7f6c..74a137d 100644
> --- a/Documentation/media-framework.txt
> +++ b/Documentation/media-framework.txt
> @@ -320,7 +320,7 @@ Userspace application API
>  -------------------------
>  
>  Media devices offer an API to userspace application to query device information
> -through ioctls.
> +and discover the device internal topology through ioctls.
>  
>  	MEDIA_IOC_DEVICE_INFO - Get device information
>  	----------------------------------------------
> @@ -357,3 +357,143 @@ instances of otherwise identical hardware. The serial number takes precedence
>  when provided and can be assumed to be unique. If the serial number is an
>  empty string, the bus_info field can be used instead. The bus_info field is
>  guaranteed to be unique, but can vary across reboots or device unplug/replug.
> +
> +
> +	MEDIA_IOC_ENUM_ENTITIES - Enumerate entities and their properties
> +	-----------------------------------------------------------------
> +
> +	ioctl(int fd, int request, struct media_entity_desc *argp);
> +
> +To query the attributes of an entity, applications set the id field of a
> +media_entity_desc structure and call the MEDIA_IOC_ENUM_ENTITIES ioctl with a
> +pointer to this structure. The driver fills the rest of the structure or
> +returns a EINVAL error code when the id is invalid.
> +
> +Entities can be enumerated by or'ing the id with the MEDIA_ENTITY_ID_FLAG_NEXT
> +flag. The driver will return information about the entity with the smallest id
> +strictly larger than the requested one ('next entity'), or EINVAL if there is
> +none.
> +
> +Entity IDs can be non-contiguous. Applications must *not* try to enumerate
> +entities by calling MEDIA_IOC_ENUM_ENTITIES with increasing id's until they
> +get an error.
> +
> +Two or more entities that share a common non-zero group_id value are
> +considered as logically grouped. Groups are used to report
> +
> +	- ALSA, VBI and video nodes that carry the same media stream
> +	- lens and flash controllers associated with a sensor
> +
> +The media_entity_desc structure is defined as
> +
> +- struct media_entity_desc
> +
> +__u32	id		Entity id, set by the application. When the id is
> +			or'ed with MEDIA_ENTITY_ID_FLAG_NEXT, the driver
> +			clears the flag and returns the first entity with a
> +			larger id.
> +char	name[32]	Entity name. UTF-8 NULL-terminated string.

Why UTF-8 instead of ASCII?

> +__u32	type		Entity type.
> +__u32	revision	Entity revision in a driver/hardware specific format.
> +__u32	flags		Entity flags.
> +__u32	group_id	Entity group ID.
> +__u16	pads		Number of pads.
> +__u16	links		Total number of outbound links. Inbound links are not
> +			counted in this field.
> +/* union */
> +	/* struct v4l, Valid for V4L sub-devices and nodes only */
> +__u32	major		V4L device node major number. For V4L sub-devices with
> +			no device node, set by the driver to 0.
> +__u32	minor		V4L device node minor number. For V4L sub-devices with
> +			no device node, set by the driver to 0.
> +	/* struct fb, Valid for frame buffer nodes only */
> +__u32	major		FB device node major number
> +__u32	minor		FB device node minor number
> +	/* Valid for ALSA devices only */
> +int	alsa		ALSA card number
> +	/* Valid for DVB devices only */
> +int	dvb		DVB card number
> +
> +Valid entity types are
> +
> +	MEDIA_ENTITY_TYPE_NODE - Unknown device node
> +	MEDIA_ENTITY_TYPE_NODE_V4L - V4L video, radio or vbi device node
> +	MEDIA_ENTITY_TYPE_NODE_FB - Frame buffer device node
> +	MEDIA_ENTITY_TYPE_NODE_ALSA - ALSA card
> +	MEDIA_ENTITY_TYPE_NODE_DVB - DVB card
> +
> +	MEDIA_ENTITY_TYPE_SUBDEV - Unknown V4L sub-device
> +	MEDIA_ENTITY_TYPE_SUBDEV_SENSOR - Video sensor
> +	MEDIA_ENTITY_TYPE_SUBDEV_FLASH - Flash controller
> +	MEDIA_ENTITY_TYPE_SUBDEV_LENS - Lens controller
> +
> +Valid entity flags are
> +
> +	MEDIA_ENTITY_FLAG_DEFAULT - Default entity for its type. Used to
> +		discover the default audio, VBI and video devices, the default
> +		camera sensor, ...
> +
> +
> +	MEDIA_IOC_ENUM_LINKS - Enumerate all pads and links for a given entity
> +	----------------------------------------------------------------------
> +
> +	ioctl(int fd, int request, struct media_links_enum *argp);
> +
> +Only forward links that originate at one of the entity's source pads are
> +returned during the enumeration process.
> +
> +To enumerate pads and/or links for a given entity, applications set the entity
> +field of a media_links_enum structure and initialize the media_pad_desc and
> +media_link_desc structure arrays pointed by the pads and links fields. They then
> +call the MEDIA_IOC_ENUM_LINKS ioctl with a pointer to this structure.
> +
> +If the pads field is not NULL, the driver fills the pads array with
> +information about the entity's pads. The array must have enough room to store
> +all the entity's pads. The number of pads can be retrieved with the
> +MEDIA_IOC_ENUM_ENTITIES ioctl.
> +
> +If the links field is not NULL, the driver fills the links array with
> +information about the entity's outbound links. The array must have enough room
> +to store all the entity's outbound links. The number of outbound links can be
> +retrieved with the MEDIA_IOC_ENUM_ENTITIES ioctl.
> +
> +The media_pad_desc, media_link_desc and media_links_enum structures are defined
> +as
> +
> +- struct media_pad_desc
> +
> +__u32		entity		ID of the entity this pad belongs to.
> +__u16		index		0-based pad index.
> +__u32		flags		Pad flags.
> +
> +Valid pad flags are
> +
> +	MEDIA_PAD_FLAG_INPUT -	Input pad, relative to the entity. Input pads
> +				sink data and are targets of links.
> +	MEDIA_PAD_FLAG_OUTPUT -	Output pad, relative to the entity. Output
> +				pads source data and are origins of links.
> +
> +One and only one of MEDIA_PAD_FLAG_INPUT and MEDIA_PAD_FLAG_OUTPUT must be set
> +for every pad.
> +
> +- struct media_link_desc
> +
> +struct media_pad_desc	source	Pad at the origin of this link.
> +struct media_pad_desc	sink	Pad at the target of this link.
> +__u32			flags	Link flags.
> +
> +Valid link flags are
> +
> +	MEDIA_LINK_FLAG_ACTIVE - The link is active and can be used to
> +		transfer media data. When two or more links target a sink pad,
> +		only one of them can be active at a time.
> +	MEDIA_LINK_FLAG_IMMUTABLE - The link active state can't be modified at
> +		runtime. An immutable link is always active.
> +
> +- struct media_links_enum
> +
> +__u32			entity	Entity id, set by the application.
> +struct media_pad_desc	*pads	Pointer to a pads array allocated by the
> +				application. Ignored if NULL.
> +struct media_link_desc	*links	Pointer to a links array allocated by the
> +				application. Ignored if NULL.
> diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> index 1415ebd..7e020f9 100644
> --- a/drivers/media/media-device.c
> +++ b/drivers/media/media-device.c
> @@ -59,6 +59,117 @@ static int media_device_get_info(struct media_device *dev,
>  	return copy_to_user(__info, &info, sizeof(*__info));
>  }
>  
> +static struct media_entity *find_entity(struct media_device *mdev, u32 id)
> +{
> +	struct media_entity *entity;
> +	int next = id & MEDIA_ENTITY_ID_FLAG_NEXT;
> +
> +	id &= ~MEDIA_ENTITY_ID_FLAG_NEXT;
> +
> +	spin_lock(&mdev->lock);
> +
> +	media_device_for_each_entity(entity, mdev) {
> +		if ((entity->id == id && !next) ||
> +		    (entity->id > id && next)) {
> +			spin_unlock(&mdev->lock);
> +			return entity;
> +		}
> +	}
> +
> +	spin_unlock(&mdev->lock);
> +
> +	return NULL;
> +}
> +
> +static long media_device_enum_entities(struct media_device *mdev,
> +				       struct media_entity_desc __user *uent)
> +{
> +	struct media_entity *ent;
> +	struct media_entity_desc u_ent;
> +
> +	if (copy_from_user(&u_ent.id, &uent->id, sizeof(u_ent.id)))
> +		return -EFAULT;
> +
> +	ent = find_entity(mdev, u_ent.id);
> +
> +	if (ent == NULL)
> +		return -EINVAL;
> +
> +	u_ent.id = ent->id;
> +	u_ent.name[0] = '\0';
> +	if (ent->name)
> +		strlcpy(u_ent.name, ent->name, sizeof(u_ent.name));
> +	u_ent.type = ent->type;
> +	u_ent.revision = ent->revision;
> +	u_ent.flags = ent->flags;
> +	u_ent.group_id = ent->group_id;
> +	u_ent.pads = ent->num_pads;
> +	u_ent.links = ent->num_links - ent->num_backlinks;
> +	u_ent.v4l.major = ent->v4l.major;
> +	u_ent.v4l.minor = ent->v4l.minor;
> +	if (copy_to_user(uent, &u_ent, sizeof(u_ent)))
> +		return -EFAULT;
> +	return 0;
> +}
> +
> +static void media_device_kpad_to_upad(const struct media_pad *kpad,
> +				      struct media_pad_desc *upad)
> +{
> +	upad->entity = kpad->entity->id;
> +	upad->index = kpad->index;
> +	upad->flags = kpad->flags;
> +}
> +
> +static long media_device_enum_links(struct media_device *mdev,
> +				    struct media_links_enum __user *ulinks)
> +{
> +	struct media_entity *entity;
> +	struct media_links_enum links;
> +
> +	if (copy_from_user(&links, ulinks, sizeof(links)))
> +		return -EFAULT;
> +
> +	entity = find_entity(mdev, links.entity);
> +	if (entity == NULL)
> +		return -EINVAL;
> +
> +	if (links.pads) {
> +		unsigned int p;
> +
> +		for (p = 0; p < entity->num_pads; p++) {
> +			struct media_pad_desc pad;
> +			media_device_kpad_to_upad(&entity->pads[p], &pad);
> +			if (copy_to_user(&links.pads[p], &pad, sizeof(pad)))
> +				return -EFAULT;
> +		}
> +	}
> +
> +	if (links.links) {
> +		struct media_link_desc __user *ulink;
> +		unsigned int l;
> +
> +		for (l = 0, ulink = links.links; l < entity->num_links; l++) {
> +			struct media_link_desc link;
> +
> +			/* Ignore backlinks. */
> +			if (entity->links[l].source->entity != entity)
> +				continue;
> +
> +			media_device_kpad_to_upad(entity->links[l].source,
> +						  &link.source);
> +			media_device_kpad_to_upad(entity->links[l].sink,
> +						  &link.sink);
> +			link.flags = entity->links[l].flags;
> +			if (copy_to_user(ulink, &link, sizeof(*ulink)))
> +				return -EFAULT;
> +			ulink++;
> +		}
> +	}
> +	if (copy_to_user(ulinks, &links, sizeof(*ulinks)))
> +		return -EFAULT;
> +	return 0;
> +}
> +
>  static long media_device_ioctl(struct file *filp, unsigned int cmd,
>  			       unsigned long arg)
>  {
> @@ -72,6 +183,18 @@ static long media_device_ioctl(struct file *filp, unsigned int cmd,
>  				(struct media_device_info __user *)arg);
>  		break;
>  
> +	case MEDIA_IOC_ENUM_ENTITIES:
> +		ret = media_device_enum_entities(dev,
> +				(struct media_entity_desc __user *)arg);
> +		break;
> +
> +	case MEDIA_IOC_ENUM_LINKS:
> +		mutex_lock(&dev->graph_mutex);
> +		ret = media_device_enum_links(dev,
> +				(struct media_links_enum __user *)arg);
> +		mutex_unlock(&dev->graph_mutex);
> +		break;
> +
>  	default:
>  		ret = -ENOIOCTLCMD;
>  	}
> diff --git a/include/linux/media.h b/include/linux/media.h
> index bca08a7..542509b 100644
> --- a/include/linux/media.h
> +++ b/include/linux/media.h
> @@ -18,6 +18,87 @@ struct media_device_info {
>  	__u32 reserved[5];
>  };
>  
> +#define MEDIA_ENTITY_ID_FLAG_NEXT		(1 << 31)
> +
> +#define MEDIA_ENTITY_TYPE_SHIFT			16
> +#define MEDIA_ENTITY_TYPE_MASK			0x00ff0000
> +#define MEDIA_ENTITY_SUBTYPE_MASK		0x0000ffff
> +
> +#define MEDIA_ENTITY_TYPE_NODE			(1 << MEDIA_ENTITY_TYPE_SHIFT)
> +#define MEDIA_ENTITY_TYPE_NODE_V4L		(MEDIA_ENTITY_TYPE_NODE + 1)
> +#define MEDIA_ENTITY_TYPE_NODE_FB		(MEDIA_ENTITY_TYPE_NODE + 2)
> +#define MEDIA_ENTITY_TYPE_NODE_ALSA		(MEDIA_ENTITY_TYPE_NODE + 3)
> +#define MEDIA_ENTITY_TYPE_NODE_DVB		(MEDIA_ENTITY_TYPE_NODE + 4)
> +
> +#define MEDIA_ENTITY_TYPE_SUBDEV		(2 << MEDIA_ENTITY_TYPE_SHIFT)
> +#define MEDIA_ENTITY_TYPE_SUBDEV_SENSOR		(MEDIA_ENTITY_TYPE_SUBDEV + 1)
> +#define MEDIA_ENTITY_TYPE_SUBDEV_FLASH		(MEDIA_ENTITY_TYPE_SUBDEV + 2)
> +#define MEDIA_ENTITY_TYPE_SUBDEV_LENS		(MEDIA_ENTITY_TYPE_SUBDEV + 3)
> +
> +#define MEDIA_ENTITY_FLAG_DEFAULT		(1 << 0)
> +
> +struct media_entity_desc {
> +	__u32 id;
> +	char name[32];
> +	__u32 type;
> +	__u32 revision;
> +	__u32 flags;
> +	__u32 group_id;
> +	__u16 pads;
> +	__u16 links;
> +
> +	__u32 reserved[4];
> +
> +	union {
> +		/* Node specifications */
> +		struct {
> +			__u32 major;
> +			__u32 minor;
> +		} v4l;
> +		struct {
> +			__u32 major;
> +			__u32 minor;
> +		} fb;
> +		int alsa;
> +		int dvb;
> +
> +		/* Sub-device specifications */
> +		/* Nothing needed yet */
> +		__u8 raw[64];
> +	};
> +};

Should this be a packed struct?

> +
> +#define MEDIA_PAD_FLAG_INPUT			(1 << 0)
> +#define MEDIA_PAD_FLAG_OUTPUT			(1 << 1)
> +
> +struct media_pad_desc {
> +	__u32 entity;		/* entity ID */
> +	__u16 index;		/* pad index */
> +	__u32 flags;		/* pad flags */
> +	__u32 reserved[2];
> +};
> +
> +#define MEDIA_LINK_FLAG_ACTIVE			(1 << 0)
> +#define MEDIA_LINK_FLAG_IMMUTABLE		(1 << 1)
> +
> +struct media_link_desc {
> +	struct media_pad_desc source;
> +	struct media_pad_desc sink;
> +	__u32 flags;
> +	__u32 reserved[2];
> +};
> +
> +struct media_links_enum {
> +	__u32 entity;
> +	/* Should have enough room for pads elements */
> +	struct media_pad_desc __user *pads;
> +	/* Should have enough room for links elements */
> +	struct media_link_desc __user *links;
> +	__u32 reserved[4];
> +};

Ditto for these other structs.

Regards,

	Hans

> +
>  #define MEDIA_IOC_DEVICE_INFO		_IOWR('M', 1, struct media_device_info)
> +#define MEDIA_IOC_ENUM_ENTITIES		_IOWR('M', 2, struct media_entity_desc)
> +#define MEDIA_IOC_ENUM_LINKS		_IOWR('M', 3, struct media_links_enum)
>  
>  #endif /* __LINUX_MEDIA_H */
> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> index edcafeb..8c40d5e 100644
> --- a/include/media/media-entity.h
> +++ b/include/media/media-entity.h
> @@ -2,29 +2,7 @@
>  #define _MEDIA_ENTITY_H
>  
>  #include <linux/list.h>
> -
> -#define MEDIA_ENTITY_TYPE_SHIFT			16
> -#define MEDIA_ENTITY_TYPE_MASK			0x00ff0000
> -#define MEDIA_ENTITY_SUBTYPE_MASK		0x0000ffff
> -
> -#define MEDIA_ENTITY_TYPE_NODE			(1 << MEDIA_ENTITY_TYPE_SHIFT)
> -#define MEDIA_ENTITY_TYPE_NODE_V4L		(MEDIA_ENTITY_TYPE_NODE + 1)
> -#define MEDIA_ENTITY_TYPE_NODE_FB		(MEDIA_ENTITY_TYPE_NODE + 2)
> -#define MEDIA_ENTITY_TYPE_NODE_ALSA		(MEDIA_ENTITY_TYPE_NODE + 3)
> -#define MEDIA_ENTITY_TYPE_NODE_DVB		(MEDIA_ENTITY_TYPE_NODE + 4)
> -
> -#define MEDIA_ENTITY_TYPE_SUBDEV		(2 << MEDIA_ENTITY_TYPE_SHIFT)
> -#define MEDIA_ENTITY_TYPE_SUBDEV_SENSOR		(MEDIA_ENTITY_TYPE_SUBDEV + 1)
> -#define MEDIA_ENTITY_TYPE_SUBDEV_FLASH		(MEDIA_ENTITY_TYPE_SUBDEV + 2)
> -#define MEDIA_ENTITY_TYPE_SUBDEV_LENS		(MEDIA_ENTITY_TYPE_SUBDEV + 3)
> -
> -#define MEDIA_ENTITY_FLAG_DEFAULT		(1 << 0)
> -
> -#define MEDIA_LINK_FLAG_ACTIVE			(1 << 0)
> -#define MEDIA_LINK_FLAG_IMMUTABLE		(1 << 1)
> -
> -#define MEDIA_PAD_FLAG_INPUT			(1 << 0)
> -#define MEDIA_PAD_FLAG_OUTPUT			(1 << 1)
> +#include <linux/media.h>
>  
>  struct media_link {
>  	struct media_pad *source;	/* Source pad */
> 

-- 
Hans Verkuil - video4linux developer - sponsored by TANDBERG, part of Cisco

Re: [RFC/PATCH v4 08/11] media: Links setup

[Hans Verkuil]

On Friday, August 20, 2010 17:29:10 Laurent Pinchart wrote:
> Create the following ioctl and implement it at the media device level to
> setup links.
> 
> - MEDIA_IOC_SETUP_LINK: Modify the properties of a given link
> 
> The only property that can currently be modified is the ACTIVE link flag
> to activate/deactivate a link. Links marked with the IMMUTABLE link flag
> can not be activated or deactivated.
> 
> Activating and deactivating a link has effects on entities' use count.
> Those changes are automatically propagated through the graph.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Stanimir Varbanov <svarbanov@mm-sol.com>
> Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
> ---
>  Documentation/media-framework.txt |   81 ++++++++++++++-
>  drivers/media/media-device.c      |   45 ++++++++
>  drivers/media/media-entity.c      |  208 +++++++++++++++++++++++++++++++++++++
>  include/linux/media.h             |    1 +
>  include/media/media-entity.h      |    8 ++
>  5 files changed, 340 insertions(+), 3 deletions(-)
> 
> diff --git a/Documentation/media-framework.txt b/Documentation/media-framework.txt
> index 74a137d..7894ef3 100644
> --- a/Documentation/media-framework.txt
> +++ b/Documentation/media-framework.txt
> @@ -278,6 +278,16 @@ When the graph traversal is complete the function will return NULL.
>  Graph traversal can be interrupted at any moment. No cleanup function call is
>  required and the graph structure can be freed normally.
>  
> +Helper functions can be used to find a link between two given pads, or a pad
> +connected to another pad through an active link
> +
> +	media_entity_find_link(struct media_pad *source,
> +			       struct media_pad *sink);
> +
> +	media_entity_remote_pad(struct media_pad *pad);
> +
> +Refer to the kerneldoc documentation for more information.
> +
>  
>  Reference counting and power handling
>  -------------------------------------
> @@ -316,6 +326,46 @@ is allowed to fail when turning power on, in which case the media_entity_get
>  function will return NULL.
>  
>  
> +Links setup
> +-----------
> +
> +Link properties can be modified at runtime by calling
> +
> +	media_entity_setup_link(struct media_link *link, u32 flags);
> +
> +The flags argument contains the requested new link flags.
> +
> +The only configurable property is the ACTIVE link flag to activate/deactivate
> +a link. Links marked with the IMMUTABLE link flag can not be activated or
> +deactivated.
> +
> +When a link is activated or deactivated, the media framework calls the
> +link_setup operation for the two entities at the source and sink of the link,
> +in that order. If the second link_setup call fails, another link_setup call is
> +made on the first entity to restore the original link flags.
> +
> +Entity drivers must implement the link_setup operation if any of their links
> +is non-immutable. The operation must either configure the hardware or store
> +the configuration information to be applied later.
> +
> +Link activation must not have any side effect on other links. If an active
> +link at a sink pad prevents another link at the same pad from being
> +deactivated, the link_setup operation must return -EBUSY and can't implicitly
> +deactivate the first active link.
> +
> +Activating and deactivating a link has effects on entities' reference counts.
> +When two sub-graphs are connected, the reference count of each of them is
> +incremented by the total reference count of all node entities in the other
> +sub-graph. When two sub-graphs are disconnected, the reverse operation is
> +performed. In both cases the set_power operations are called accordingly,
> +ensuring that the link_setup calls are made with power active on the source
> +and sink entities.
> +
> +In other words, activating or deactivating a link propagates reference count
> +changes through the graph, and the final state is identical to what it would
> +have been if the link had been active or inactive from the start.
> +
> +
>  Userspace application API
>  -------------------------
>  
> @@ -439,9 +489,6 @@ Valid entity flags are
>  
>  	ioctl(int fd, int request, struct media_links_enum *argp);
>  
> -Only forward links that originate at one of the entity's source pads are
> -returned during the enumeration process.
> -
>  To enumerate pads and/or links for a given entity, applications set the entity
>  field of a media_links_enum structure and initialize the media_pad_desc and
>  media_link_desc structure arrays pointed by the pads and links fields. They then
> @@ -457,6 +504,9 @@ information about the entity's outbound links. The array must have enough room
>  to store all the entity's outbound links. The number of outbound links can be
>  retrieved with the MEDIA_IOC_ENUM_ENTITIES ioctl.
>  
> +Only outbound (forward) links that originate at one of the entity's source
> +pads are returned during the enumeration process.
> +
>  The media_pad_desc, media_link_desc and media_links_enum structures are defined
>  as
>  
> @@ -497,3 +547,28 @@ struct media_pad_desc	*pads	Pointer to a pads array allocated by the
>  				application. Ignored if NULL.
>  struct media_link_desc	*links	Pointer to a links array allocated by the
>  				application. Ignored if NULL.
> +
> +
> +	MEDIA_IOC_SETUP_LINK - Modify the properties of a link
> +	------------------------------------------------------
> +
> +	ioctl(int fd, int request, struct media_link_desc *argp);
> +
> +To change link properties applications fill a media_link_desc structure with
> +link identification information (source and sink pad) and the new requested link
> +flags. They then call the MEDIA_IOC_SETUP_LINK ioctl with a pointer to that
> +structure.
> +
> +The only configurable property is the ACTIVE link flag to activate/deactivate
> +a link. Links marked with the IMMUTABLE link flag can not be activated or
> +deactivated.
> +
> +Link activation has no side effect on other links. If an active link at the
> +sink pad prevents the link from being activated, the driver returns with a
> +EBUSY error code.
> +
> +If the specified link can't be found the driver returns with a EINVAL error
> +code.
> +
> +The media_pad_desc and media_link_desc structures are described in the
> +MEDIA_IOC_ENUM_LINKS ioctl documentation.
> diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> index 7e020f9..06655d9 100644
> --- a/drivers/media/media-device.c
> +++ b/drivers/media/media-device.c
> @@ -170,6 +170,44 @@ static long media_device_enum_links(struct media_device *mdev,
>  	return 0;
>  }
>  
> +static long media_device_setup_link(struct media_device *mdev,
> +				    struct media_link_desc __user *_ulink)
> +{
> +	struct media_link *link = NULL;
> +	struct media_link_desc ulink;
> +	struct media_entity *source;
> +	struct media_entity *sink;
> +	int ret;
> +
> +	if (copy_from_user(&ulink, _ulink, sizeof(ulink)))
> +		return -EFAULT;
> +
> +	/* Find the source and sink entities and link.
> +	 */
> +	source = find_entity(mdev, ulink.source.entity);
> +	sink = find_entity(mdev, ulink.sink.entity);
> +
> +	if (source == NULL || sink == NULL)
> +		return -EINVAL;
> +
> +	if (ulink.source.index >= source->num_pads ||
> +	    ulink.sink.index >= sink->num_pads)
> +		return -EINVAL;
> +
> +	link = media_entity_find_link(&source->pads[ulink.source.index],
> +				      &sink->pads[ulink.sink.index]);
> +	if (link == NULL)
> +		return -EINVAL;
> +
> +	/* Setup the link on both entities. */
> +	ret = __media_entity_setup_link(link, ulink.flags);
> +
> +	if (copy_to_user(_ulink, &ulink, sizeof(ulink)))
> +		return -EFAULT;
> +
> +	return ret;
> +}
> +
>  static long media_device_ioctl(struct file *filp, unsigned int cmd,
>  			       unsigned long arg)
>  {
> @@ -195,6 +233,13 @@ static long media_device_ioctl(struct file *filp, unsigned int cmd,
>  		mutex_unlock(&dev->graph_mutex);
>  		break;
>  
> +	case MEDIA_IOC_SETUP_LINK:
> +		mutex_lock(&dev->graph_mutex);
> +		ret = media_device_setup_link(dev,
> +				(struct media_link_desc __user *)arg);
> +		mutex_unlock(&dev->graph_mutex);
> +		break;
> +
>  	default:
>  		ret = -ENOIOCTLCMD;
>  	}
> diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
> index da4fef6..bc97b78 100644
> --- a/drivers/media/media-entity.c
> +++ b/drivers/media/media-entity.c
> @@ -198,6 +198,25 @@ EXPORT_SYMBOL_GPL(media_entity_graph_walk_next);
>   * Power state handling
>   */
>  
> +/*
> + * Return power count of nodes directly or indirectly connected to
> + * a given entity.
> + */
> +static int media_entity_count_node(struct media_entity *entity)
> +{
> +	struct media_entity_graph graph;
> +	int use = 0;
> +
> +	media_entity_graph_walk_start(&graph, entity);
> +
> +	while ((entity = media_entity_graph_walk_next(&graph))) {
> +		if (media_entity_type(entity) == MEDIA_ENTITY_TYPE_NODE)
> +			use += entity->use_count;
> +	}
> +
> +	return use;
> +}
> +
>  /* Apply use count to an entity. */
>  static void media_entity_use_apply_one(struct media_entity *entity, int change)
>  {
> @@ -261,6 +280,32 @@ static int media_entity_power_apply(struct media_entity *entity, int change)
>  	return ret;
>  }
>  
> +/* Apply the power state changes when connecting two entities. */
> +static int media_entity_power_connect(struct media_entity *one,
> +				      struct media_entity *theother)
> +{
> +	int power_one = media_entity_count_node(one);
> +	int power_theother = media_entity_count_node(theother);
> +	int ret = 0;
> +
> +	ret = media_entity_power_apply(one, power_theother);
> +	if (ret < 0)
> +		return ret;
> +
> +	return media_entity_power_apply(theother, power_one);
> +}
> +
> +static void media_entity_power_disconnect(struct media_entity *one,
> +					  struct media_entity *theother)
> +{
> +	int power_one = media_entity_count_node(one);
> +	int power_theother = media_entity_count_node(theother);
> +
> +	/* Powering off entities is assumed to never fail. */
> +	media_entity_power_apply(one, -power_theother);
> +	media_entity_power_apply(theother, -power_one);
> +}
> +
>  /*
>   * Apply use count change to graph and change power state of entities
>   * accordingly.
> @@ -404,3 +449,166 @@ media_entity_create_link(struct media_entity *source, u16 source_pad,
>  	return 0;
>  }
>  EXPORT_SYMBOL(media_entity_create_link);
> +
> +static int __media_entity_setup_link_notify(struct media_link *link, u32 flags)
> +{
> +	const u32 mask = MEDIA_LINK_FLAG_ACTIVE;
> +	int ret;
> +
> +	/* Notify both entities. */
> +	ret = media_entity_call(link->source->entity, link_setup,
> +				link->source, link->sink, flags);
> +	if (ret < 0 && ret != -ENOIOCTLCMD)
> +		return ret;
> +
> +	ret = media_entity_call(link->sink->entity, link_setup,
> +				link->sink, link->source, flags);
> +	if (ret < 0 && ret != -ENOIOCTLCMD) {
> +		media_entity_call(link->source->entity, link_setup,
> +				  link->source, link->sink, link->flags);
> +		return ret;
> +	}
> +
> +	link->flags = (link->flags & ~mask) | (flags & mask);
> +	link->reverse->flags = link->flags;
> +
> +	return 0;
> +}
> +
> +/**
> + * __media_entity_setup_link - Configure a media link
> + * @link: The link being configured
> + * @flags: Link configuration flags
> + *
> + * The bulk of link setup is handled by the two entities connected through the
> + * link. This function notifies both entities of the link configuration change.
> + *
> + * If the link is immutable or if the current and new configuration are
> + * identical, return immediately.
> + *
> + * The user is expected to hold link->source->parent->mutex. If not,
> + * media_entity_setup_link() should be used instead.
> + */
> +int __media_entity_setup_link(struct media_link *link, u32 flags)
> +{
> +	struct media_entity *source, *sink;
> +	int ret = -EBUSY;
> +
> +	if (link == NULL)
> +		return -EINVAL;
> +
> +	if (link->flags & MEDIA_LINK_FLAG_IMMUTABLE)
> +		return link->flags == flags ? 0 : -EINVAL;
> +
> +	if (link->flags == flags)
> +		return 0;
> +
> +	source = __media_entity_get(link->source->entity);
> +	if (!source)
> +		return ret;
> +
> +	sink = __media_entity_get(link->sink->entity);
> +	if (!sink)
> +		goto err___media_entity_get;
> +
> +	if (flags & MEDIA_LINK_FLAG_ACTIVE) {
> +		ret = media_entity_power_connect(source, sink);
> +		if (ret < 0)
> +			goto err_media_entity_power_connect;
> +	}
> +
> +	ret = __media_entity_setup_link_notify(link, flags);
> +	if (ret < 0)
> +		goto err___media_entity_setup_link_notify;
> +
> +	if (!(flags & MEDIA_LINK_FLAG_ACTIVE))
> +		media_entity_power_disconnect(source, sink);
> +
> +	__media_entity_put(sink);
> +	__media_entity_put(source);
> +
> +	return 0;
> +
> +err___media_entity_setup_link_notify:
> +	if (flags & MEDIA_LINK_FLAG_ACTIVE)
> +		media_entity_power_disconnect(source, sink);
> +err_media_entity_power_connect:
> +	__media_entity_put(sink);
> +err___media_entity_get:
> +	__media_entity_put(source);
> +
> +	return ret;
> +}
> +
> +int media_entity_setup_link(struct media_link *link, u32 flags)
> +{
> +	int ret;
> +
> +	mutex_lock(&link->source->entity->parent->graph_mutex);
> +	ret = __media_entity_setup_link(link, flags);
> +	mutex_unlock(&link->source->entity->parent->graph_mutex);
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(media_entity_setup_link);
> +
> +/**
> + * media_entity_find_link - Find a link between two pads
> + * @source: Source pad
> + * @sink: Sink pad
> + *
> + * Return a pointer to the link between the two entities. If no such link
> + * exists, return NULL.
> + */
> +struct media_link *
> +media_entity_find_link(struct media_pad *source, struct media_pad *sink)
> +{
> +	struct media_link *link;
> +	unsigned int i;
> +
> +	for (i = 0; i < source->entity->num_links; ++i) {
> +		link = &source->entity->links[i];
> +
> +		if (link->source->entity == source->entity &&
> +		    link->source->index == source->index &&
> +		    link->sink->entity == sink->entity &&
> +		    link->sink->index == sink->index)
> +			return link;
> +	}
> +
> +	return NULL;
> +}
> +EXPORT_SYMBOL_GPL(media_entity_find_link);
> +
> +/**
> + * media_entity_remote_pad - Locate the pad at the remote end of a link
> + * @entity: Local entity
> + * @pad: Pad at the local end of the link
> + *
> + * Search for a remote pad connected to the given pad by iterating over all
> + * links originating or terminating at that pad until an active link is found.
> + *
> + * Return a pointer to the pad at the remote end of the first found active link,
> + * or NULL if no active link has been found.
> + */
> +struct media_pad *media_entity_remote_pad(struct media_pad *pad)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < pad->entity->num_links; i++) {
> +		struct media_link *link = &pad->entity->links[i];
> +
> +		if (!(link->flags & MEDIA_LINK_FLAG_ACTIVE))
> +			continue;
> +
> +		if (link->source == pad)
> +			return link->sink;
> +
> +		if (link->sink == pad)
> +			return link->source;
> +	}
> +
> +	return NULL;
> +
> +}

Why is this needed? Esp. since there can be multiple active remote pads if
you have multiple active outgoing links. Something this function doesn't deal
with.

> +EXPORT_SYMBOL_GPL(media_entity_remote_pad);
> diff --git a/include/linux/media.h b/include/linux/media.h
> index 542509b..edb53c2 100644
> --- a/include/linux/media.h
> +++ b/include/linux/media.h
> @@ -100,5 +100,6 @@ struct media_links_enum {
>  #define MEDIA_IOC_DEVICE_INFO		_IOWR('M', 1, struct media_device_info)
>  #define MEDIA_IOC_ENUM_ENTITIES		_IOWR('M', 2, struct media_entity_desc)
>  #define MEDIA_IOC_ENUM_LINKS		_IOWR('M', 3, struct media_links_enum)
> +#define MEDIA_IOC_SETUP_LINK		_IOWR('M', 4, struct media_link_desc)
>  
>  #endif /* __LINUX_MEDIA_H */
> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> index 8c40d5e..0f0697b 100644
> --- a/include/media/media-entity.h
> +++ b/include/media/media-entity.h
> @@ -18,6 +18,9 @@ struct media_pad {
>  };
>  
>  struct media_entity_operations {
> +	int (*link_setup)(struct media_entity *entity,
> +			  const struct media_pad *local,
> +			  const struct media_pad *remote, u32 flags);
>  	int (*set_power)(struct media_entity *entity, int power);
>  };
>  
> @@ -88,6 +91,11 @@ int media_entity_init(struct media_entity *entity, u16 num_pads,
>  void media_entity_cleanup(struct media_entity *entity);
>  int media_entity_create_link(struct media_entity *source, u16 source_pad,
>  		struct media_entity *sink, u16 sink_pad, u32 flags);
> +int __media_entity_setup_link(struct media_link *link, u32 flags);
> +int media_entity_setup_link(struct media_link *link, u32 flags);
> +struct media_link *media_entity_find_link(struct media_pad *source,
> +		struct media_pad *sink);
> +struct media_pad *media_entity_remote_pad(struct media_pad *pad);
>  
>  struct media_entity *media_entity_get(struct media_entity *entity);
>  void media_entity_put(struct media_entity *entity);
> 

This patch made me wonder about something else: how is power management handled 
for immutable links? They are by definition active, so they should be powered on
automatically as well. I'm not sure whether that happens right now.

Regards,

	Hans

-- 
Hans Verkuil - video4linux developer - sponsored by TANDBERG, part of Cisco

Re: [RFC/PATCH v4 02/11] media: Media device

[Laurent Pinchart]

Hi Hans,

On Saturday 28 August 2010 12:26:15 Hans Verkuil wrote:
> On Friday, August 20, 2010 17:29:04 Laurent Pinchart wrote:

[snip]

> > +struct media_device {
> > +	/* dev->driver_data points to this struct. */
> > +	struct device *dev;
> > +	struct media_devnode devnode;
> > +
> > +	u8 model[32];
> > +	u8 serial[32];
> > +	u8 bus_info[32];
> > +	u32 device_version;
> 
> I prefer hw_revision or possibly hw_device_revision. 'device' is too
> ambiguous. And 'revision' is more applicable to hardware than 'version'
> IMHO.

Agreed.

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v4 03/11] media: Entities, pads and links

[Laurent Pinchart]

Hi Hans,

On Saturday 28 August 2010 12:31:29 Hans Verkuil wrote:
> > +#define MEDIA_ENTITY_TYPE_NODE			(1 << MEDIA_ENTITY_TYPE_SHIFT)
> > +#define MEDIA_ENTITY_TYPE_NODE_V4L		(MEDIA_ENTITY_TYPE_NODE + 1)
> > +#define MEDIA_ENTITY_TYPE_NODE_FB		(MEDIA_ENTITY_TYPE_NODE + 2)
> > +#define MEDIA_ENTITY_TYPE_NODE_ALSA		(MEDIA_ENTITY_TYPE_NODE + 3)
> > +#define MEDIA_ENTITY_TYPE_NODE_DVB		(MEDIA_ENTITY_TYPE_NODE + 4)
> 
> During discussions at work I realized that another type that might be
> needed in the future (not needed in the first version, I think) is
> NODE_MTB for flash memory. There are devices that have flash memory on
> board (basically a kind of BIOS) and it would be handy for a flash utility
> to find the corresponding mtd device.
> 
> It shouldn't be hard to add this when needed.

Agreed.

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v4 06/11] media: Media device information query

[Laurent Pinchart]

Hi Hans,

On Saturday 28 August 2010 12:44:15 Hans Verkuil wrote:
> On Friday, August 20, 2010 17:29:08 Laurent Pinchart wrote:

[snip]

> > diff --git a/Documentation/media-framework.txt
> > b/Documentation/media-framework.txt index 59649e9..66f7f6c 100644
> > --- a/Documentation/media-framework.txt
> > +++ b/Documentation/media-framework.txt
> > @@ -315,3 +315,45 @@ required, drivers don't need to provide a set_power
> > operation. The operation
> > 
> >  is allowed to fail when turning power on, in which case the
> >  media_entity_get function will return NULL.
> > 
> > +
> > +Userspace application API
> > +-------------------------
> > +
> > +Media devices offer an API to userspace application to query device
> > information +through ioctls.
> > +
> > +	MEDIA_IOC_DEVICE_INFO - Get device information
> > +	----------------------------------------------
> > +
> > +	ioctl(int fd, int request, struct media_device_info *argp);
> > +
> > +To query device information, call the MEDIA_IOC_ENUM_ENTITIES ioctl with
> > a +pointer to a media_device_info structure. The driver fills the
> > structure and +returns the information to the application. The ioctl
> > never fails. +
> > +The media_device_info structure is defined as
> > +
> > +- struct media_device_info
> > +
> > +__u8	driver[16]	Driver name as a NUL-terminated ASCII string. The
> > +			driver version is stored in the driver_version field.
> 
> Proposed improvement: "Name of the driver implementing the media API
> as a NUL-terminated ASCII string."
> 
> The media API overarches multiple drivers, so it's probably useful to say
> which driver name should be filled in here.

OK I'll change this.

> > +__u8	model[32]	Device model name as a NUL-terminated UTF-8 string. The
> > +			device version is stored in the device_version field and
> > +			is not be appended to the model name.
> 
> Why UTF-8 instead of ASCII?

Because the model name could contain non-ASCII characters.

> > +__u8	serial[32]	Serial number as an ASCII string. The string is
> > +			NUL-terminated unless the serial number is exactly 32
> > +			characters long.
> > +__u8	bus_info[32]	Location of the device in the system as a
> > NUL-terminated
> > +			ASCII string. This includes the bus type name (PCI, USB,
> > +			...) and a bus-specific identifier.
> > +__u32	media_version	Media API version, formatted with the
> > KERNEL_VERSION
> > +			macro.
> > +__u32	device_version	Media device driver version in a driver-specific
> > format.
> > +__u32	driver_version	Media device driver version, formatted with the
> > +			KERNEL_VERSION macro.
> 
> These last two are very confusing. Does device_version actually refer to
> the hardware revision? In that case the description next to it is really
> wrong. And I also think it should be renamed to hw_revision.

My bad. I'll rename device_version to hw_revision.

[snip]

> > diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> > index c309d3c..1415ebd 100644
> > --- a/drivers/media/media-device.c
> > +++ b/drivers/media/media-device.c
> > @@ -20,13 +20,70 @@

[snip]

> > +static int media_device_get_info(struct media_device *dev,
> > +				 struct media_device_info __user *__info)
> > +{
> > +	struct media_device_info info;
> > +
> > +	memset(&info, 0, sizeof(info));
> 
> No need to zero the whole struct. info.reserved would be enough.

The strings should be zeroed, otherwise we could leak information to userspace 
by copying uninitialized stack data.

> > +
> > +	strlcpy(info.driver, dev->dev->driver->name, sizeof(info.driver));
> > +	strlcpy(info.model, dev->model, sizeof(info.model));
> > +	strncpy(info.serial, dev->serial, sizeof(info.serial));
> 
> Why not strlcpy?

Because the serial number can be 32 bytes long, and thus not NULL-terminated.

> > +	strlcpy(info.bus_info, dev->bus_info, sizeof(info.bus_info));
> > +
> > +	info.media_version = MEDIA_API_VERSION;
> > +	info.device_version = dev->device_version;
> > +	info.driver_version = dev->driver_version;
> > +
> > +	return copy_to_user(__info, &info, sizeof(*__info));
> > +}
> > +
> > +static long media_device_ioctl(struct file *filp, unsigned int cmd,
> > +			       unsigned long arg)
> > +{
> > +	struct media_devnode *devnode = media_devnode_data(filp);
> > +	struct media_device *dev = to_media_device(devnode);
> > +	long ret;
> > +
> > +	switch (cmd) {
> > +	case MEDIA_IOC_DEVICE_INFO:
> > +		ret = media_device_get_info(dev,
> > +				(struct media_device_info __user *)arg);
> > +		break;
> > +
> > +	default:
> > +		ret = -ENOIOCTLCMD;
> > +	}
> > +
> > +	return ret;
> > +}
> > +
> > 
> >  static const struct media_file_operations media_device_fops = {
> >  
> >  	.owner = THIS_MODULE,
> > 
> > +	.open = media_device_open,
> > +	.unlocked_ioctl = media_device_ioctl,
> > +	.release = media_device_close,
> > 
> >  };
> >  
> >  /*
> >  -----------------------------------------------------------------------
> >  ------
> > 
> > diff --git a/include/linux/media.h b/include/linux/media.h
> > new file mode 100644
> > index 0000000..bca08a7
> > --- /dev/null
> > +++ b/include/linux/media.h
> > @@ -0,0 +1,23 @@
> > +#ifndef __LINUX_MEDIA_H
> > +#define __LINUX_MEDIA_H
> > +
> > +#include <linux/ioctl.h>
> > +#include <linux/types.h>
> > +#include <linux/version.h>
> > +
> > +#define MEDIA_API_VERSION	KERNEL_VERSION(0, 1, 0)
> > +
> > +struct media_device_info {
> > +	__u8 driver[16];
> > +	__u8 model[32];
> > +	__u8 serial[32];
> > +	__u8 bus_info[32];
> > +	__u32 media_version;
> > +	__u32 device_version;
> > +	__u32 driver_version;
> > +	__u32 reserved[5];
> 
> I'd increase this to reserved[33] as [5] seems very low to me.
> Total struct size is then 256 bytes.

OK.

[snip]

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v4 07/11] media: Entities, pads and links enumeration

[Laurent Pinchart]

Hi Hans,

On Saturday 28 August 2010 13:02:22 Hans Verkuil wrote:
> On Friday, August 20, 2010 17:29:09 Laurent Pinchart wrote:

[snip]

> > diff --git a/Documentation/media-framework.txt
> > b/Documentation/media-framework.txt index 66f7f6c..74a137d 100644
> > --- a/Documentation/media-framework.txt
> > +++ b/Documentation/media-framework.txt

[snip]

> > +The media_entity_desc structure is defined as
> > +
> > +- struct media_entity_desc
> > +
> > +__u32	id		Entity id, set by the application. When the id is
> > +			or'ed with MEDIA_ENTITY_ID_FLAG_NEXT, the driver
> > +			clears the flag and returns the first entity with a
> > +			larger id.
> > +char	name[32]	Entity name. UTF-8 NULL-terminated string.
> 
> Why UTF-8 instead of ASCII?

Because vendor-specific names could include non-ASCII characters (same reason 
for the model name in the media_device structure, if we decice to make the 
model name ASCII I'll make the entity name ASCII as well).

[snip]

> > +struct media_entity_desc {
> > +	__u32 id;
> > +	char name[32];
> > +	__u32 type;
> > +	__u32 revision;
> > +	__u32 flags;
> > +	__u32 group_id;
> > +	__u16 pads;
> > +	__u16 links;
> > +
> > +	__u32 reserved[4];
> > +
> > +	union {
> > +		/* Node specifications */
> > +		struct {
> > +			__u32 major;
> > +			__u32 minor;
> > +		} v4l;
> > +		struct {
> > +			__u32 major;
> > +			__u32 minor;
> > +		} fb;
> > +		int alsa;
> > +		int dvb;
> > +
> > +		/* Sub-device specifications */
> > +		/* Nothing needed yet */
> > +		__u8 raw[64];
> > +	};
> > +};
> 
> Should this be a packed struct?

Why ? :-) Packed struct are most useful when they need to match hardware 
structures or network protocols. Packing a structure can generate unaligned 
fields, which are bad performance-wise.

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v4 08/11] media: Links setup

[Laurent Pinchart]

Hi Hans,

On Saturday 28 August 2010 13:14:18 Hans Verkuil wrote:
> On Friday, August 20, 2010 17:29:10 Laurent Pinchart wrote:

[snip]

> > +/**
> > + * media_entity_remote_pad - Locate the pad at the remote end of a link
> > + * @entity: Local entity
> > + * @pad: Pad at the local end of the link
> > + *
> > + * Search for a remote pad connected to the given pad by iterating over
> > all
> > + * links originating or terminating at that pad until an active link is
> > found.
> > + *
> > + * Return a pointer to the pad at the remote end of the first found
> > active link,
> > + * or NULL if no active link has been found.
> > + */
> > +struct media_pad *media_entity_remote_pad(struct media_pad *pad)
> > +{
> > +	unsigned int i;
> > +
> > +	for (i = 0; i < pad->entity->num_links; i++) {
> > +		struct media_link *link = &pad->entity->links[i];
> > +
> > +		if (!(link->flags & MEDIA_LINK_FLAG_ACTIVE))
> > +			continue;
> > +
> > +		if (link->source == pad)
> > +			return link->sink;
> > +
> > +		if (link->sink == pad)
> > +			return link->source;
> > +	}
> > +
> > +	return NULL;
> > +
> > +}
> 
> Why is this needed? Esp. since there can be multiple active remote pads if
> you have multiple active outgoing links. Something this function doesn't
> deal with.

The function is meant to be used when only one of the links can be active. 
It's most useful to find the entity connected to a given input pad, as input 
pads can't be connected by multiple simultaneously active links.

[snip]

> This patch made me wonder about something else: how is power management
> handled for immutable links? They are by definition active, so they should
> be powered on automatically as well. I'm not sure whether that happens
> right now.

Links are not powered, entities are. Whether a link is immutable or not 
doesn't make much of a difference, it will just always be considered as 
active.

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v4 07/11] media: Entities, pads and links enumeration

[Hans Verkuil]

On Wednesday, September 01, 2010 16:05:10 Laurent Pinchart wrote:
> Hi Hans,
> 
> On Saturday 28 August 2010 13:02:22 Hans Verkuil wrote:
> > On Friday, August 20, 2010 17:29:09 Laurent Pinchart wrote:
> 
> [snip]
> 
> > > diff --git a/Documentation/media-framework.txt
> > > b/Documentation/media-framework.txt index 66f7f6c..74a137d 100644
> > > --- a/Documentation/media-framework.txt
> > > +++ b/Documentation/media-framework.txt
> 
> [snip]
> 
> > > +The media_entity_desc structure is defined as
> > > +
> > > +- struct media_entity_desc
> > > +
> > > +__u32	id		Entity id, set by the application. When the id is
> > > +			or'ed with MEDIA_ENTITY_ID_FLAG_NEXT, the driver
> > > +			clears the flag and returns the first entity with a
> > > +			larger id.
> > > +char	name[32]	Entity name. UTF-8 NULL-terminated string.
> > 
> > Why UTF-8 instead of ASCII?
> 
> Because vendor-specific names could include non-ASCII characters (same reason 
> for the model name in the media_device structure, if we decice to make the 
> model name ASCII I'll make the entity name ASCII as well).
> 
> [snip]
> 
> > > +struct media_entity_desc {
> > > +	__u32 id;
> > > +	char name[32];
> > > +	__u32 type;
> > > +	__u32 revision;
> > > +	__u32 flags;
> > > +	__u32 group_id;
> > > +	__u16 pads;
> > > +	__u16 links;
> > > +
> > > +	__u32 reserved[4];
> > > +
> > > +	union {
> > > +		/* Node specifications */
> > > +		struct {
> > > +			__u32 major;
> > > +			__u32 minor;
> > > +		} v4l;
> > > +		struct {
> > > +			__u32 major;
> > > +			__u32 minor;
> > > +		} fb;
> > > +		int alsa;
> > > +		int dvb;
> > > +
> > > +		/* Sub-device specifications */
> > > +		/* Nothing needed yet */
> > > +		__u8 raw[64];
> > > +	};
> > > +};
> > 
> > Should this be a packed struct?
> 
> Why ? :-) Packed struct are most useful when they need to match hardware 
> structures or network protocols. Packing a structure can generate unaligned 
> fields, which are bad performance-wise.

I'm thinking about preventing a compat32 mess as we have for v4l.

It is my understanding that the only way to prevent different struct sizes
between 32 and 64 bit is to use packed.

Regards,

	Hans

-- 
Hans Verkuil - video4linux developer - sponsored by TANDBERG, part of Cisco

Re: [RFC/PATCH v4 08/11] media: Links setup

[Hans Verkuil]

On Wednesday, September 01, 2010 16:08:29 Laurent Pinchart wrote:
> Hi Hans,
> 
> On Saturday 28 August 2010 13:14:18 Hans Verkuil wrote:
> > On Friday, August 20, 2010 17:29:10 Laurent Pinchart wrote:
> 
> [snip]
> 
> > > +/**
> > > + * media_entity_remote_pad - Locate the pad at the remote end of a link
> > > + * @entity: Local entity
> > > + * @pad: Pad at the local end of the link
> > > + *
> > > + * Search for a remote pad connected to the given pad by iterating over
> > > all
> > > + * links originating or terminating at that pad until an active link is
> > > found.
> > > + *
> > > + * Return a pointer to the pad at the remote end of the first found
> > > active link,
> > > + * or NULL if no active link has been found.
> > > + */
> > > +struct media_pad *media_entity_remote_pad(struct media_pad *pad)
> > > +{
> > > +	unsigned int i;
> > > +
> > > +	for (i = 0; i < pad->entity->num_links; i++) {
> > > +		struct media_link *link = &pad->entity->links[i];
> > > +
> > > +		if (!(link->flags & MEDIA_LINK_FLAG_ACTIVE))
> > > +			continue;
> > > +
> > > +		if (link->source == pad)
> > > +			return link->sink;
> > > +
> > > +		if (link->sink == pad)
> > > +			return link->source;
> > > +	}
> > > +
> > > +	return NULL;
> > > +
> > > +}
> > 
> > Why is this needed? Esp. since there can be multiple active remote pads if
> > you have multiple active outgoing links. Something this function doesn't
> > deal with.
> 
> The function is meant to be used when only one of the links can be active. 
> It's most useful to find the entity connected to a given input pad, as input 
> pads can't be connected by multiple simultaneously active links.

In that case I would rename it media_entity_remote_source to match the use
case (and only look for remote sources).

Alternatively you could add an index for the nth active link, then it would
be truly generic.

Right now the function either does too much or too little :-)

Regards,

	Hans


-- 
Hans Verkuil - video4linux developer - sponsored by TANDBERG, part of Cisco

Re: [RFC/PATCH v4 03/11] media: Entities, pads and links

[Mauro Carvalho Chehab]

Em 20-08-2010 12:29, Laurent Pinchart escreveu:
> As video hardware pipelines become increasingly complex and
> configurable, the current hardware description through v4l2 subdevices
> reaches its limits. In addition to enumerating and configuring
> subdevices, video camera drivers need a way to discover and modify at
> runtime how those subdevices are connected. This is done through new
> elements called entities, pads and links.
> 
> An entity is a basic media hardware building block. It can correspond to
> a large variety of logical blocks such as physical hardware devices
> (CMOS sensor for instance), logical hardware devices (a building block
> in a System-on-Chip image processing pipeline), DMA channels or physical
> connectors.
> 
> A pad is a connection endpoint through which an entity can interact with
> other entities. Data (not restricted to video) produced by an entity
> flows from the entity's output to one or more entity inputs. Pads should
> not be confused with physical pins at chip boundaries.
> 
> A link is a point-to-point oriented connection between two pads, either
> on the same entity or on different entities. Data flows from a source
> pad to a sink pad.
> 
> Links are stored in the source entity. To make backwards graph walk
> faster, a copy of all links is also stored in the sink entity. The copy
> is known as a backlink and is only used to help graph traversal.
> 
> The entity API is made of three functions:
> 
> - media_entity_init() initializes an entity. The caller must provide an
> array of pads as well as an estimated number of links. The links array
> is allocated dynamically and will be reallocated if it grows beyond the
> initial estimate.
> 
> - media_entity_cleanup() frees resources allocated for an entity. It
> must be called during the cleanup phase after unregistering the entity
> and before freeing it.
> 
> - media_entity_create_link() creates a link between two entities. An
> entry in the link array of each entity is allocated and stores pointers
> to source and sink pads.
> 
> When a media device is unregistered, all its entities are unregistered
> automatically.
> 
> The code is based on Hans Verkuil <hverkuil@xs4all.nl> initial work.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
> ---
>  Documentation/media-framework.txt |  149 +++++++++++++++++++++++++++++++++++++
>  drivers/media/Makefile            |    2 +-
>  drivers/media/media-device.c      |   53 +++++++++++++
>  drivers/media/media-entity.c      |  145 ++++++++++++++++++++++++++++++++++++
>  include/media/media-device.h      |   19 +++++
>  include/media/media-entity.h      |   96 ++++++++++++++++++++++++
>  6 files changed, 463 insertions(+), 1 deletions(-)
>  create mode 100644 drivers/media/media-entity.c
>  create mode 100644 include/media/media-entity.h
> 
> diff --git a/Documentation/media-framework.txt b/Documentation/media-framework.txt
> index 89dc7ad..35d74e4 100644
> --- a/Documentation/media-framework.txt
> +++ b/Documentation/media-framework.txt
> @@ -35,6 +35,30 @@ belong to userspace.
>  The media kernel API aims at solving those problems.
>  
>  
> +Abstract media device model
> +---------------------------
> +
> +Discovering a device internal topology, and configuring it at runtime, is one
> +of the goals of the media framework. To achieve this, hardware devices are
> +modeled as an oriented graph of building blocks called entities connected
> +through pads.
> +
> +An entity is a basic media hardware building block. It can correspond to
> +a large variety of logical blocks such as physical hardware devices
> +(CMOS sensor for instance), logical hardware devices (a building block
> +in a System-on-Chip image processing pipeline), DMA channels or physical
> +connectors.
> +
> +A pad is a connection endpoint through which an entity can interact with
> +other entities. Data (not restricted to video) produced by an entity
> +flows from the entity's output to one or more entity inputs. Pads should
> +not be confused with physical pins at chip boundaries.
> +
> +A link is a point-to-point oriented connection between two pads, either
> +on the same entity or on different entities. Data flows from a source
> +pad to a sink pad.
> +
> +
>  Media device
>  ------------
>  
> @@ -89,3 +113,128 @@ Drivers unregister media device instances by calling
>  
>  Unregistering a media device that hasn't been registered is *NOT* safe.
>  
> +
> +Entities, pads and links
> +------------------------
> +
> +- Entities
> +
> +Entities are represented by a struct media_entity instance, defined in
> +include/media/media-entity.h. The structure is usually embedded into a
> +higher-level structure, such as a v4l2_subdev or video_device instance,
> +although drivers can allocate entities directly.
> +
> +Drivers initialize entities by calling
> +
> +	media_entity_init(struct media_entity *entity, u16 num_pads,
> +			  struct media_pad *pads, u16 extra_links);
> +
> +The media_entity name, type, flags, revision and group_id fields can be
> +initialized before or after calling media_entity_init. Entities embedded in
> +higher-level standard structures can have some of those fields set by the
> +higher-level framework.
> +
> +As the number of pads is known in advance, the pads array is not allocated
> +dynamically but is managed by the entity driver. Most drivers will embed the
> +pads array in a driver-specific structure, avoiding dynamic allocation.
> +
> +Drivers must set the direction of every pad in the pads array before calling
> +media_entity_init. The function will initialize the other pads fields.
> +
> +Unlike the number of pads, the total number of links isn't always known in
> +advance by the entity driver. As an initial estimate, media_entity_init
> +pre-allocates a number of links equal to the number of pads plus an optional
> +number of extra links. The links array will be reallocated if it grows beyond
> +the initial estimate.
> +
> +Drivers register entities with a media device by calling
> +
> +	media_device_register_entity(struct media_device *mdev,
> +				     struct media_entity *entity);
> +
> +When registered the entity is assigned an ID. Entity IDs are positive integers
> +and are guaranteed to be unique in the context of the media device. The
> +framework doesn't guarantee that IDs will always be continuous.
> +
> +Drivers unregister entities by calling
> +
> +	media_device_unregister_entity(struct media_entity *entity);
> +
> +Unregistering an entity will not change the IDs of the other entities, and the
> +ID will never be reused for a newly registered entity.
> +
> +When a media device is unregistered, all its entities are unregistered
> +automatically. No manual entities unregistration is then required.
> +
> +Drivers free resources associated with an entity by calling
> +
> +	media_entity_cleanup(struct media_entity *entity);
> +
> +This function must be called during the cleanup phase after unregistering the
> +entity. Note that the media_entity instance itself must be freed explicitly by
> +the driver if required.
> +
> +Entities have flags that describe the entity capabilities and state.
> +
> +	MEDIA_ENTITY_FLAG_DEFAULT indicates the default entity for a given
> +	type. This can be used to report the default audio and video devices
> +	or the default camera sensor.
> +
> +Logical entity groups can be defined by setting the group ID of all member
> +entities to the same non-zero value. An entity group serves no purpose in the
> +kernel, but is reported to userspace during entities enumeration. The group_id
> +field belongs to the media device driver and must not by touched by entity
> +drivers.
> +
> +Media device drivers should define groups if several entities are logically
> +bound together. Example usages include reporting
> +
> +	- ALSA, VBI and video nodes that carry the same media stream
> +	- lens and flash controllers associated with a sensor
> +
> +- Pads
> +
> +Pads are represented by a struct media_pad instance, defined in
> +include/media/media-entity.h. Each entity stores its pads in a pads array
> +managed by the entity driver. Drivers usually embed the array in a
> +driver-specific structure.
> +
> +Pads are identified by their entity and their 0-based index in the pads array.
> +Both information are stored in the media_pad structure, making the media_pad
> +pointer the canonical way to store and pass link references.
> +
> +Pads have flags that describe the pad capabilities and state.
> +
> +	MEDIA_PAD_FLAG_INPUT indicates that the pad supports sinking data.
> +	MEDIA_PAD_FLAG_OUTPUT indicates that the pad supports sourcing data.
> +
> +One and only one of MEDIA_PAD_FLAG_INPUT and MEDIA_PAD_FLAG_OUTPUT must be set
> +for each pad.
> +
> +- Links
> +
> +Links are represented by a struct media_link instance, defined in
> +include/media/media-entity.h. Each entity stores all links originating at or
> +targetting any of its pads in a links array. A given link is thus stored
> +twice, once in the source entity and once in the target entity. The array is
> +pre-allocated and grows dynamically as needed.
> +
> +Drivers create links by calling
> +
> +	media_entity_create_link(struct media_entity *source, u16 source_pad,
> +				 struct media_entity *sink,   u16 sink_pad,
> +				 u32 flags);
> +
> +An entry in the link array of each entity is allocated and stores pointers
> +to source and sink pads.
> +
> +Links have flags that describe the link capabilities and state.
> +
> +	MEDIA_LINK_FLAG_ACTIVE indicates that the link is active and can be
> +	used to transfer media data. When two or more links target a sink pad,
> +	only one of them can be active at a time.
> +	MEDIA_LINK_FLAG_IMMUTABLE indicates that the link active state can't
> +	be modified at runtime. If MEDIA_LINK_FLAG_IMMUTABLE is set, then
> +	MEDIA_LINK_FLAG_ACTIVE must also be set since an immutable link is
> +	always active.
> +
> diff --git a/drivers/media/Makefile b/drivers/media/Makefile
> index f8d8dcb..a425581 100644
> --- a/drivers/media/Makefile
> +++ b/drivers/media/Makefile
> @@ -2,7 +2,7 @@
>  # Makefile for the kernel multimedia device drivers.
>  #
>  
> -media-objs	:= media-device.o media-devnode.o
> +media-objs	:= media-device.o media-devnode.o media-entity.o
>  
>  obj-$(CONFIG_MEDIA_SUPPORT)	+= media.o
>  
> diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> index 781c641..eeb002e 100644
> --- a/drivers/media/media-device.c
> +++ b/drivers/media/media-device.c
> @@ -23,6 +23,7 @@
>  
>  #include <media/media-device.h>
>  #include <media/media-devnode.h>
> +#include <media/media-entity.h>
>  
>  static const struct media_file_operations media_device_fops = {
>  	.owner = THIS_MODULE,
> @@ -67,6 +68,10 @@ int __must_check media_device_register(struct media_device *mdev)
>  	if (WARN_ON(mdev->dev == NULL || mdev->model[0] == 0))
>  		return 0;
>  
> +	mdev->entity_id = 1;
> +	INIT_LIST_HEAD(&mdev->entities);
> +	spin_lock_init(&mdev->lock);
> +
>  	/* Register the device node. */
>  	mdev->devnode.fops = &media_device_fops;
>  	mdev->devnode.parent = mdev->dev;
> @@ -92,7 +97,55 @@ EXPORT_SYMBOL_GPL(media_device_register);
>   */
>  void media_device_unregister(struct media_device *mdev)
>  {
> +	struct media_entity *entity;
> +	struct media_entity *next;
> +
> +	list_for_each_entry_safe(entity, next, &mdev->entities, list)
> +		media_device_unregister_entity(entity);
> +
>  	device_remove_file(&mdev->devnode.dev, &dev_attr_model);
>  	media_devnode_unregister(&mdev->devnode);
>  }
>  EXPORT_SYMBOL_GPL(media_device_unregister);
> +
> +/**
> + * media_device_register_entity - Register an entity with a media device
> + * @mdev:	The media device
> + * @entity:	The entity
> + */
> +int __must_check media_device_register_entity(struct media_device *mdev,
> +					      struct media_entity *entity)
> +{
> +	/* Warn if we apparently re-register an entity */
> +	WARN_ON(entity->parent != NULL);

Instead, it should just return -EINVAL and use __must_check. 

What's the sense of allowing registering it twice?

> +	entity->parent = mdev;
> +
> +	spin_lock(&mdev->lock);
> +	entity->id = mdev->entity_id++;
> +	list_add_tail(&entity->list, &mdev->entities);
> +	spin_unlock(&mdev->lock);
> +
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(media_device_register_entity);
> +
> +/**
> + * media_device_unregister_entity - Unregister an entity
> + * @entity:	The entity
> + *
> + * If the entity has never been registered this function will return
> + * immediately.
> + */
> +void media_device_unregister_entity(struct media_entity *entity)
> +{
> +	struct media_device *mdev = entity->parent;
> +
> +	if (mdev == NULL)
> +		return;
> +
> +	spin_lock(&mdev->lock);
> +	list_del(&entity->list);
> +	spin_unlock(&mdev->lock);
> +	entity->parent = NULL;
> +}
> +EXPORT_SYMBOL_GPL(media_device_unregister_entity);
> diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
> new file mode 100644
> index 0000000..541063b
> --- /dev/null
> +++ b/drivers/media/media-entity.c
> @@ -0,0 +1,145 @@
> +/*
> + *  Media Entity support
> + *
> + *  Copyright (C) 2010 Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> + *
> + *  This program is free software; you can redistribute it and/or modify
> + *  it under the terms of the GNU General Public License as published by
> + *  the Free Software Foundation; either version 2 of the License, or
> + *  (at your option) any later version.
> + *
> + *  This program is distributed in the hope that it will be useful,
> + *  but WITHOUT ANY WARRANTY; without even the implied warranty of
> + *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + *  GNU General Public License for more details.
> + *
> + *  You should have received a copy of the GNU General Public License
> + *  along with this program; if not, write to the Free Software
> + *  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
> + */
> +
> +#include <linux/module.h>
> +#include <linux/slab.h>
> +#include <media/media-entity.h>
> +
> +/**
> + * media_entity_init - Initialize a media entity
> + *
> + * @num_pads: Total number of input and output pads.
> + * @extra_links: Initial estimate of the number of extra links.
> + * @pads: Array of 'num_pads' pads.
> + *
> + * The total number of pads is an intrinsic property of entities known by the
> + * entity driver, while the total number of links depends on hardware design
> + * and is an extrinsic property unknown to the entity driver. However, in most
> + * use cases the entity driver can guess the number of links which can safely
> + * be assumed to be equal to or larger than the number of pads.
> + *
> + * For those reasons the links array can be preallocated based on the entity
> + * driver guess and will be reallocated later if extra links need to be
> + * created.
> + *
> + * This function allocates a links array with enough space to hold at least
> + * 'num_pads' + 'extra_links' elements. The media_entity::max_links field will
> + * be set to the number of allocated elements.
> + *
> + * The pads array is managed by the entity driver and passed to
> + * media_entity_init() where its pointer will be stored in the entity structure.
> + */
> +int
> +media_entity_init(struct media_entity *entity, u16 num_pads,
> +		  struct media_pad *pads, u16 extra_links)
> +{
> +	struct media_link *links;
> +	unsigned int max_links = num_pads + extra_links;
> +	unsigned int i;
> +
> +	links = kzalloc(max_links * sizeof(links[0]), GFP_KERNEL);
> +	if (links == NULL)
> +		return -ENOMEM;
> +
> +	entity->group_id = 0;
> +	entity->max_links = max_links;
> +	entity->num_links = 0;
> +	entity->num_backlinks = 0;
> +	entity->num_pads = num_pads;
> +	entity->pads = pads;
> +	entity->links = links;
> +
> +	for (i = 0; i < num_pads; i++) {
> +		pads[i].entity = entity;
> +		pads[i].index = i;
> +	}
> +
> +	return 0;
> +}
> +EXPORT_SYMBOL(media_entity_init);

Just use EXPORT_SYMBOL_GPL() for all symbols that needs to be exported.

> +
> +void
> +media_entity_cleanup(struct media_entity *entity)
> +{
> +	kfree(entity->links);
> +}
> +EXPORT_SYMBOL(media_entity_cleanup);
> +
> +static struct media_link *media_entity_add_link(struct media_entity *entity)
> +{
> +	if (entity->num_links >= entity->max_links) {
> +		struct media_link *links = entity->links;
> +		unsigned int max_links = entity->max_links + 2;
> +		unsigned int i;
> +
> +		links = krealloc(links, max_links * sizeof(*links), GFP_KERNEL);
> +		if (links == NULL)
> +			return NULL;
> +
> +		for (i = 0; i < entity->num_links; i++)
> +			links[i].reverse->reverse = &links[i];
> +
> +		entity->max_links = max_links;
> +		entity->links = links;
> +	}
> +
> +	return &entity->links[entity->num_links++];
> +}
> +
> +int
> +media_entity_create_link(struct media_entity *source, u16 source_pad,
> +			 struct media_entity *sink, u16 sink_pad, u32 flags)
> +{
> +	struct media_link *link;
> +	struct media_link *backlink;
> +
> +	BUG_ON(source == NULL || sink == NULL);
> +	BUG_ON(source_pad >= source->num_pads);
> +	BUG_ON(sink_pad >= sink->num_pads);
> +
> +	link = media_entity_add_link(source);
> +	if (link == NULL)
> +		return -ENOMEM;
> +
> +	link->source = &source->pads[source_pad];
> +	link->sink = &sink->pads[sink_pad];
> +	link->flags = flags;
> +
> +	/* Create the backlink. Backlinks are used to help graph traversal and
> +	 * are not reported to userspace.
> +	 */
> +	backlink = media_entity_add_link(sink);
> +	if (backlink == NULL) {
> +		source->num_links--;
> +		return -ENOMEM;
> +	}
> +
> +	backlink->source = &source->pads[source_pad];
> +	backlink->sink = &sink->pads[sink_pad];
> +	backlink->flags = flags;
> +
> +	link->reverse = backlink;
> +	backlink->reverse = link;
> +
> +	sink->num_backlinks++;
> +
> +	return 0;
> +}
> +EXPORT_SYMBOL(media_entity_create_link);
> diff --git a/include/media/media-device.h b/include/media/media-device.h
> index 4fe949e..7e2dac2 100644
> --- a/include/media/media-device.h
> +++ b/include/media/media-device.h
> @@ -23,8 +23,10 @@
>  
>  #include <linux/device.h>
>  #include <linux/list.h>
> +#include <linux/spinlock.h>
>  
>  #include <media/media-devnode.h>
> +#include <media/media-entity.h>
>  
>  /**
>   * struct media_device - Media device
> @@ -35,6 +37,9 @@
>   * @bus_info:	Unique and stable device location identifier
>   * @device_version: Hardware device version
>   * @driver_version: Device driver version
> + * @entity_id:	ID of the next entity to be registered
> + * @entities:	List of registered entities
> + * @lock:	Entities list lock
>   *
>   * This structure represents an abstract high-level media device. It allows easy
>   * access to entities and provides basic media device-level support. The
> @@ -56,9 +61,23 @@ struct media_device {
>  	u8 bus_info[32];
>  	u32 device_version;
>  	u32 driver_version;
> +
> +	u32 entity_id;
> +	struct list_head entities;
> +
> +	/* Protects the entities list */
> +	spinlock_t lock;
>  };
>  
>  int __must_check media_device_register(struct media_device *mdev);
>  void media_device_unregister(struct media_device *mdev);
>  
> +int __must_check media_device_register_entity(struct media_device *mdev,
> +					      struct media_entity *entity);
> +void media_device_unregister_entity(struct media_entity *entity);
> +
> +/* Iterate over all entities. */
> +#define media_device_for_each_entity(entity, mdev)			\
> +	list_for_each_entry(entity, &(mdev)->entities, list)
> +
>  #endif
> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> new file mode 100644
> index 0000000..32bb20a
> --- /dev/null
> +++ b/include/media/media-entity.h
> @@ -0,0 +1,96 @@
> +#ifndef _MEDIA_ENTITY_H
> +#define _MEDIA_ENTITY_H
> +
> +#include <linux/list.h>
> +
> +#define MEDIA_ENTITY_TYPE_SHIFT			16
> +#define MEDIA_ENTITY_TYPE_MASK			0x00ff0000
> +#define MEDIA_ENTITY_SUBTYPE_MASK		0x0000ffff
> +
> +#define MEDIA_ENTITY_TYPE_NODE			(1 << MEDIA_ENTITY_TYPE_SHIFT)
> +#define MEDIA_ENTITY_TYPE_NODE_V4L		(MEDIA_ENTITY_TYPE_NODE + 1)
> +#define MEDIA_ENTITY_TYPE_NODE_FB		(MEDIA_ENTITY_TYPE_NODE + 2)
> +#define MEDIA_ENTITY_TYPE_NODE_ALSA		(MEDIA_ENTITY_TYPE_NODE + 3)
> +#define MEDIA_ENTITY_TYPE_NODE_DVB		(MEDIA_ENTITY_TYPE_NODE + 4)
> +
> +#define MEDIA_ENTITY_TYPE_SUBDEV		(2 << MEDIA_ENTITY_TYPE_SHIFT)
> +#define MEDIA_ENTITY_TYPE_SUBDEV_SENSOR		(MEDIA_ENTITY_TYPE_SUBDEV + 1)
> +#define MEDIA_ENTITY_TYPE_SUBDEV_FLASH		(MEDIA_ENTITY_TYPE_SUBDEV + 2)
> +#define MEDIA_ENTITY_TYPE_SUBDEV_LENS		(MEDIA_ENTITY_TYPE_SUBDEV + 3)
> +
> +#define MEDIA_ENTITY_FLAG_DEFAULT		(1 << 0)
> +
> +#define MEDIA_LINK_FLAG_ACTIVE			(1 << 0)
> +#define MEDIA_LINK_FLAG_IMMUTABLE		(1 << 1)
> +
> +#define MEDIA_PAD_FLAG_INPUT			(1 << 0)
> +#define MEDIA_PAD_FLAG_OUTPUT			(1 << 1)
> +
> +struct media_link {
> +	struct media_pad *source;	/* Source pad */
> +	struct media_pad *sink;		/* Sink pad  */
> +	struct media_link *reverse;	/* Link in the reverse direction */
> +	unsigned long flags;		/* Link flags (MEDIA_LINK_FLAG_*) */
> +};
> +
> +struct media_pad {
> +	struct media_entity *entity;	/* Entity this pad belongs to */
> +	u16 index;			/* Pad index in the entity pads array */
> +	unsigned long flags;		/* Pad flags (MEDIA_PAD_FLAG_*) */
> +};
> +
> +struct media_entity {
> +	struct list_head list;
> +	struct media_device *parent;	/* Media device this entity belongs to*/
> +	u32 id;				/* Entity ID, unique in the parent media
> +					 * device context */
> +	const char *name;		/* Entity name */
> +	u32 type;			/* Entity type (MEDIA_ENTITY_TYPE_*) */
> +	u32 revision;			/* Entity revision, driver specific */
> +	unsigned long flags;		/* Entity flags (MEDIA_ENTITY_FLAG_*) */
> +	u32 group_id;			/* Entity group ID */
> +
> +	u16 num_pads;			/* Number of input and output pads */
> +	u16 num_links;			/* Number of existing links, both active
> +					 * and inactive */
> +	u16 num_backlinks;		/* Number of backlinks */
> +	u16 max_links;			/* Maximum number of links */
> +
> +	struct media_pad *pads;		/* Pads array (num_pads elements) */
> +	struct media_link *links;	/* Links array (max_links elements)*/
> +
> +	union {
> +		/* Node specifications */
> +		struct {
> +			u32 major;
> +			u32 minor;
> +		} v4l;
> +		struct {
> +			u32 major;
> +			u32 minor;
> +		} fb;
> +		int alsa;
> +		int dvb;
> +
> +		/* Sub-device specifications */
> +		/* Nothing needed yet */
> +	};
> +};
> +
> +static inline u32 media_entity_type(struct media_entity *entity)
> +{
> +	return entity->type & MEDIA_ENTITY_TYPE_MASK;
> +}
> +
> +static inline u32 media_entity_subtype(struct media_entity *entity)
> +{
> +	return entity->type & MEDIA_ENTITY_SUBTYPE_MASK;
> +}
> +
> +int media_entity_init(struct media_entity *entity, u16 num_pads,
> +		struct media_pad *pads, u16 extra_links);
> +void media_entity_cleanup(struct media_entity *entity);
> +int media_entity_create_link(struct media_entity *source, u16 source_pad,
> +		struct media_entity *sink, u16 sink_pad, u32 flags);
> +
> +#endif

Re: [RFC/PATCH v4 04/11] media: Entity graph traversal

[Mauro Carvalho Chehab]

Em 20-08-2010 12:29, Laurent Pinchart escreveu:
> From: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
> 
> Add media entity graph traversal. The traversal follows active links by
> depth first. Traversing graph backwards is prevented by comparing the next
> possible entity in the graph with the previous one. Multiply connected
> graphs are thus not supported.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Vimarsh Zutshi <vimarsh.zutshi@nokia.com>
> ---
>  Documentation/media-framework.txt |   40 +++++++++++++
>  drivers/media/media-entity.c      |  115 +++++++++++++++++++++++++++++++++++++
>  include/media/media-entity.h      |   15 +++++
>  3 files changed, 170 insertions(+), 0 deletions(-)
> 
> diff --git a/Documentation/media-framework.txt b/Documentation/media-framework.txt
> index 35d74e4..a599824 100644
> --- a/Documentation/media-framework.txt
> +++ b/Documentation/media-framework.txt
> @@ -238,3 +238,43 @@ Links have flags that describe the link capabilities and state.
>  	MEDIA_LINK_FLAG_ACTIVE must also be set since an immutable link is
>  	always active.
>  
> +
> +Graph traversal
> +---------------
> +
> +The media framework provides APIs to iterate over entities in a graph.
> +
> +To iterate over all entities belonging to a media device, drivers can use the
> +media_device_for_each_entity macro, defined in include/media/media-device.h.
> +
> +	struct media_entity *entity;
> +
> +	media_device_for_each_entity(entity, mdev) {
> +		/* entity will point to each entity in turn */
> +		...
> +	}
> +
> +Drivers might also need to iterate over all entities in a graph that can be
> +reached only through active links starting at a given entity. The media
> +framework provides a depth-first graph traversal API for that purpose.
> +
> +Note that graphs with cycles (whether directed or undirected) are *NOT*
> +supported by the graph traversal API.

Please document that a maximum depth exists to prevent loops, currently
defined as 16 (MEDIA_ENTITY_ENUM_MAX_DEPTH).

> +
> +Drivers initiate a graph traversal by calling
> +
> +	media_entity_graph_walk_start(struct media_entity_graph *graph,
> +				      struct media_entity *entity);
> +
> +The graph structure, provided by the caller, is initialized to start graph
> +traversal at the given entity.
> +
> +Drivers can then retrieve the next entity by calling
> +
> +	media_entity_graph_walk_next(struct media_entity_graph *graph);
> +
> +When the graph traversal is complete the function will return NULL.
> +
> +Graph traversal can be interrupted at any moment. No cleanup function call is
> +required and the graph structure can be freed normally.
> +
> diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
> index 541063b..c277c18 100644
> --- a/drivers/media/media-entity.c
> +++ b/drivers/media/media-entity.c
> @@ -82,6 +82,121 @@ media_entity_cleanup(struct media_entity *entity)
>  }
>  EXPORT_SYMBOL(media_entity_cleanup);
>  
> +/* -----------------------------------------------------------------------------
> + * Graph traversal
> + */
> +
> +static struct media_entity *
> +media_entity_other(struct media_entity *entity, struct media_link *link)
> +{
> +	if (link->source->entity == entity)
> +		return link->sink->entity;
> +	else
> +		return link->source->entity;
> +}
> +
> +/* push an entity to traversal stack */
> +static void stack_push(struct media_entity_graph *graph,
> +		       struct media_entity *entity)
> +{
> +	if (graph->top == MEDIA_ENTITY_ENUM_MAX_DEPTH - 1) {
> +		WARN_ON(1);
> +		return;
> +	}
> +	graph->top++;
> +	graph->stack[graph->top].link = 0;
> +	graph->stack[graph->top].entity = entity;
> +}
> +
> +static struct media_entity *stack_pop(struct media_entity_graph *graph)
> +{
> +	struct media_entity *entity;
> +
> +	entity = graph->stack[graph->top].entity;
> +	graph->top--;
> +
> +	return entity;
> +}
> +
> +#define stack_peek(en)	((en)->stack[(en)->top - 1].entity)
> +#define link_top(en)	((en)->stack[(en)->top].link)
> +#define stack_top(en)	((en)->stack[(en)->top].entity)
> +
> +/**
> + * media_entity_graph_walk_start - Start walking the media graph at a given entity
> + * @graph: Media graph structure that will be used to walk the graph
> + * @entity: Starting entity
> + *
> + * This function initializes the graph traversal structure to walk the entities
> + * graph starting at the given entity. The traversal structure must not be
> + * modified by the caller during graph traversal. When done the structure can
> + * safely be freed.
> + */
> +void media_entity_graph_walk_start(struct media_entity_graph *graph,
> +				   struct media_entity *entity)
> +{
> +	graph->top = 0;
> +	graph->stack[graph->top].entity = NULL;
> +	stack_push(graph, entity);
> +}
> +EXPORT_SYMBOL_GPL(media_entity_graph_walk_start);
> +
> +/**
> + * media_entity_graph_walk_next - Get the next entity in the graph
> + * @graph: Media graph structure
> + *
> + * Perform a depth-first traversal of the given media entities graph.
> + *
> + * The graph structure must have been previously initialized with a call to
> + * media_entity_graph_walk_start().
> + *
> + * Return the next entity in the graph or NULL if the whole graph have been
> + * traversed.
> + */
> +struct media_entity *
> +media_entity_graph_walk_next(struct media_entity_graph *graph)
> +{
> +	if (stack_top(graph) == NULL)
> +		return NULL;
> +
> +	/*
> +	 * Depth first search. Push entity to stack and continue from
> +	 * top of the stack until no more entities on the level can be
> +	 * found.
> +	 */
> +	while (link_top(graph) < stack_top(graph)->num_links) {
> +		struct media_entity *entity = stack_top(graph);
> +		struct media_link *link = &entity->links[link_top(graph)];
> +		struct media_entity *next;
> +
> +		/* The link is not active so we do not follow. */
> +		if (!(link->flags & MEDIA_LINK_FLAG_ACTIVE)) {
> +			link_top(graph)++;
> +			continue;
> +		}
> +
> +		/* Get the entity in the other end of the link . */
> +		next = media_entity_other(entity, link);
> +
> +		/* Was it the entity we came here from? */
> +		if (next == stack_peek(graph)) {
> +			link_top(graph)++;
> +			continue;
> +		}
> +
> +		/* Push the new entity to stack and start over. */
> +		link_top(graph)++;
> +		stack_push(graph, next);
> +	}
> +
> +	return stack_pop(graph);
> +}
> +EXPORT_SYMBOL_GPL(media_entity_graph_walk_next);
> +
> +/* -----------------------------------------------------------------------------
> + * Links management
> + */
> +
>  static struct media_link *media_entity_add_link(struct media_entity *entity)
>  {
>  	if (entity->num_links >= entity->max_links) {
> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> index 32bb20a..3a7c74d 100644
> --- a/include/media/media-entity.h
> +++ b/include/media/media-entity.h
> @@ -87,10 +87,25 @@ static inline u32 media_entity_subtype(struct media_entity *entity)
>  	return entity->type & MEDIA_ENTITY_SUBTYPE_MASK;
>  }
>  
> +#define MEDIA_ENTITY_ENUM_MAX_DEPTH	16
> +
> +struct media_entity_graph {
> +	struct {
> +		struct media_entity *entity;
> +		int link;
> +	} stack[MEDIA_ENTITY_ENUM_MAX_DEPTH];
> +	int top;
> +};
> +
>  int media_entity_init(struct media_entity *entity, u16 num_pads,
>  		struct media_pad *pads, u16 extra_links);
>  void media_entity_cleanup(struct media_entity *entity);
>  int media_entity_create_link(struct media_entity *source, u16 source_pad,
>  		struct media_entity *sink, u16 sink_pad, u32 flags);
>  
> +void media_entity_graph_walk_start(struct media_entity_graph *graph,
> +		struct media_entity *entity);
> +struct media_entity *
> +media_entity_graph_walk_next(struct media_entity_graph *graph);
> +
>  #endif

Re: [RFC/PATCH v4 05/11] media: Reference count and power handling

[Mauro Carvalho Chehab]

Em 20-08-2010 12:29, Laurent Pinchart escreveu:
> From: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
> 
> Basically these are the interface functions:
> 
> media_entity_get() - acquire entity
> media_entity_put() - release entity
> 
> 	If the entity is of node type, the power change is distributed to
> 	all connected entities. For non-nodes it only affects that very
> 	node. A mutex is used to serialise access to the entity graph.
> 
> In the background there's a depth-first search algorithm that traverses the
> active links in the graph. All these functions parse the graph to implement
> whatever they're to do.
> 
> The module counters are increased/decreased in media_entity_get/put to
> prevent module unloading when an entity is referenced.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Stanimir Varbanov <svarbanov@mm-sol.com>
> ---
>  Documentation/media-framework.txt |   37 +++++++++
>  drivers/media/media-device.c      |    1 +
>  drivers/media/media-entity.c      |  146 +++++++++++++++++++++++++++++++++++++
>  include/media/media-device.h      |    4 +
>  include/media/media-entity.h      |   15 ++++
>  5 files changed, 203 insertions(+), 0 deletions(-)
> 
> diff --git a/Documentation/media-framework.txt b/Documentation/media-framework.txt
> index a599824..59649e9 100644
> --- a/Documentation/media-framework.txt
> +++ b/Documentation/media-framework.txt
> @@ -278,3 +278,40 @@ When the graph traversal is complete the function will return NULL.
>  Graph traversal can be interrupted at any moment. No cleanup function call is
>  required and the graph structure can be freed normally.
>  
> +
> +Reference counting and power handling
> +-------------------------------------
> +
> +Before accessing type-specific entities operations (such as the V4L2
> +sub-device operations), drivers must acquire a reference to the entity. This
> +ensures that the entity will be powered on and ready to accept requests.
> +Similarly, after being done with an entity, drivers must release the
> +reference.
> +
> +	media_entity_get(struct media_entity *entity)
> +
> +The function will increase the entity reference count. If the entity is a node
> +(MEDIA_ENTITY_TYPE_NODE type), the reference count of all entities it is
> +connected to, both directly or indirectly, through active links is increased.
> +This ensures that the whole media pipeline will be ready to process
> +
> +Acquiring a reference to an entity increases the media device module reference
> +count to prevent module unloading when an entity is being used.
> +
> +media_entity_get will return a pointer to the entity if successful, or NULL
> +otherwise.
> +
> +	media_entity_put(struct media_entity *entity)
> +
> +The function will decrease the entity reference count and, for node entities,
> +like media_entity_get, the reference count of all connected entities. Calling
> +media_entity_put with a NULL argument is valid and will return immediately.
> +
> +When the first reference to an entity is acquired, or the last reference
> +released, the entity's set_power operation is called. Entity drivers must
> +implement the operation if they need to perform any power management task,
> +such as turning powers or clocks on or off. If no power management is
> +required, drivers don't need to provide a set_power operation. The operation
> +is allowed to fail when turning power on, in which case the media_entity_get
> +function will return NULL.

The idea of doing power management via media entity get/put doesn't seem right.
The mediabus interface and its usage should be optional, and only specialized
applications will likely implement it. If a refcount 0 means power off, it ends
that a device implementing the media bus will not work with V4L2 applications.

> +
> diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> index eeb002e..c309d3c 100644
> --- a/drivers/media/media-device.c
> +++ b/drivers/media/media-device.c
> @@ -71,6 +71,7 @@ int __must_check media_device_register(struct media_device *mdev)
>  	mdev->entity_id = 1;
>  	INIT_LIST_HEAD(&mdev->entities);
>  	spin_lock_init(&mdev->lock);
> +	mutex_init(&mdev->graph_mutex);
>  
>  	/* Register the device node. */
>  	mdev->devnode.fops = &media_device_fops;
> diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
> index c277c18..da4fef6 100644
> --- a/drivers/media/media-entity.c
> +++ b/drivers/media/media-entity.c
> @@ -21,6 +21,7 @@
>  #include <linux/module.h>
>  #include <linux/slab.h>
>  #include <media/media-entity.h>
> +#include <media/media-device.h>
>  
>  /**
>   * media_entity_init - Initialize a media entity
> @@ -194,6 +195,151 @@ media_entity_graph_walk_next(struct media_entity_graph *graph)
>  EXPORT_SYMBOL_GPL(media_entity_graph_walk_next);
>  
>  /* -----------------------------------------------------------------------------
> + * Power state handling
> + */
> +
> +/* Apply use count to an entity. */
> +static void media_entity_use_apply_one(struct media_entity *entity, int change)
> +{
> +	entity->use_count += change;
> +	WARN_ON(entity->use_count < 0);

Instead of producing a warning, just deny it to have usage bellow zero. As this will
be called from userspace, the entire interface should be reliable enough to avoid
dumb applications to miss-use it.

Also: what happens if an userspace application dies or suffer any troubles? You
need to reset all use_count's at release() callback.

> +}
> +
> +/*
> + * Apply use count change to an entity and change power state based on
> + * new use count.
> + */
> +static int media_entity_power_apply_one(struct media_entity *entity, int change)
> +{
> +	int ret;
> +
> +	if (entity->use_count == 0 && change > 0 &&
> +	    entity->ops && entity->ops->set_power) {
> +		ret = entity->ops->set_power(entity, 1);
> +		if (ret)
> +			return ret;
> +	}
> +
> +	media_entity_use_apply_one(entity, change);
> +
> +	if (entity->use_count == 0 && change < 0 &&
> +	    entity->ops && entity->ops->set_power)
> +		entity->ops->set_power(entity, 0);
> +
> +	return 0;
> +}
> +
> +/*
> + * Apply power change to all connected entities. This ignores the
> + * nodes.
> + */
> +static int media_entity_power_apply(struct media_entity *entity, int change)
> +{
> +	struct media_entity_graph graph;
> +	struct media_entity *first = entity;
> +	int ret = 0;
> +
> +	if (!change)
> +		return 0;
> +
> +	media_entity_graph_walk_start(&graph, entity);
> +
> +	while (!ret && (entity = media_entity_graph_walk_next(&graph)))
> +		if (media_entity_type(entity) != MEDIA_ENTITY_TYPE_NODE)
> +			ret = media_entity_power_apply_one(entity, change);
> +
> +	if (!ret)
> +		return 0;
> +
> +	media_entity_graph_walk_start(&graph, first);
> +
> +	while ((first = media_entity_graph_walk_next(&graph))
> +	       && first != entity)
> +		if (media_entity_type(first) != MEDIA_ENTITY_TYPE_NODE)
> +			media_entity_power_apply_one(first, -change);
> +
> +	return ret;
> +}
> +
> +/*
> + * Apply use count change to graph and change power state of entities
> + * accordingly.
> + */
> +static int media_entity_node_power_change(struct media_entity *entity,
> +					  int change)
> +{
> +	/* Apply use count to node. */
> +	media_entity_use_apply_one(entity, change);
> +
> +	/* Apply power change to connected non-nodes. */
> +	return media_entity_power_apply(entity, change);
> +}
> +
> +/*
> + * Node entity use changes are reflected on power state of all
> + * connected (directly or indirectly) entities whereas non-node entity
> + * use count changes are limited to that very entity.
> + */
> +static int media_entity_use_change(struct media_entity *entity, int change)
> +{
> +	if (media_entity_type(entity) == MEDIA_ENTITY_TYPE_NODE)
> +		return media_entity_node_power_change(entity, change);
> +	else
> +		return media_entity_power_apply_one(entity, change);
> +}
> +
> +static struct media_entity *__media_entity_get(struct media_entity *entity)
> +{
> +	if (media_entity_use_change(entity, 1))
> +		return NULL;
> +
> +	return entity;
> +}
> +
> +static void __media_entity_put(struct media_entity *entity)
> +{
> +	media_entity_use_change(entity, -1);
> +}
> +
> +/* user open()s media entity */
> +struct media_entity *media_entity_get(struct media_entity *entity)
> +{
> +	struct media_entity *e;
> +
> +	if (entity == NULL)
> +		return NULL;
> +
> +	if (entity->parent->dev &&
> +	    !try_module_get(entity->parent->dev->driver->owner))
> +		return NULL;
> +
> +	mutex_lock(&entity->parent->graph_mutex);
> +	e = __media_entity_get(entity);
> +	mutex_unlock(&entity->parent->graph_mutex);
> +
> +	if (e == NULL && entity->parent->dev)
> +		module_put(entity->parent->dev->driver->owner);
> +
> +	return e;
> +}
> +EXPORT_SYMBOL_GPL(media_entity_get);
> +
> +/* user release()s media entity */
> +void media_entity_put(struct media_entity *entity)
> +{
> +	if (entity == NULL)
> +		return;
> +
> +	mutex_lock(&entity->parent->graph_mutex);
> +	__media_entity_put(entity);
> +	mutex_unlock(&entity->parent->graph_mutex);
> +
> +	if (entity->parent->dev)
> +		module_put(entity->parent->dev->driver->owner);
> +}
> +EXPORT_SYMBOL_GPL(media_entity_put);
> +
> +/* -----------------------------------------------------------------------------
>   * Links management
>   */
>  
> diff --git a/include/media/media-device.h b/include/media/media-device.h
> index 7e2dac2..3c9a5e0 100644
> --- a/include/media/media-device.h
> +++ b/include/media/media-device.h
> @@ -23,6 +23,7 @@
>  
>  #include <linux/device.h>
>  #include <linux/list.h>
> +#include <linux/mutex.h>
>  #include <linux/spinlock.h>
>  
>  #include <media/media-devnode.h>
> @@ -40,6 +41,7 @@
>   * @entity_id:	ID of the next entity to be registered
>   * @entities:	List of registered entities
>   * @lock:	Entities list lock
> + * @graph_mutex: Entities graph operation lock
>   *
>   * This structure represents an abstract high-level media device. It allows easy
>   * access to entities and provides basic media device-level support. The
> @@ -67,6 +69,8 @@ struct media_device {
>  
>  	/* Protects the entities list */
>  	spinlock_t lock;
> +	/* Serializes graph operations. */
> +	struct mutex graph_mutex;
>  };
>  
>  int __must_check media_device_register(struct media_device *mdev);
> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> index 3a7c74d..edcafeb 100644
> --- a/include/media/media-entity.h
> +++ b/include/media/media-entity.h
> @@ -39,6 +39,10 @@ struct media_pad {
>  	unsigned long flags;		/* Pad flags (MEDIA_PAD_FLAG_*) */
>  };
>  
> +struct media_entity_operations {
> +	int (*set_power)(struct media_entity *entity, int power);
> +};
> +
>  struct media_entity {
>  	struct list_head list;
>  	struct media_device *parent;	/* Media device this entity belongs to*/
> @@ -59,6 +63,10 @@ struct media_entity {
>  	struct media_pad *pads;		/* Pads array (num_pads elements) */
>  	struct media_link *links;	/* Links array (max_links elements)*/
>  
> +	const struct media_entity_operations *ops;	/* Entity operations */
> +
> +	int use_count;			/* Use count for the entity. */
> +
>  	union {
>  		/* Node specifications */
>  		struct {
> @@ -103,9 +111,16 @@ void media_entity_cleanup(struct media_entity *entity);
>  int media_entity_create_link(struct media_entity *source, u16 source_pad,
>  		struct media_entity *sink, u16 sink_pad, u32 flags);
>  
> +struct media_entity *media_entity_get(struct media_entity *entity);
> +void media_entity_put(struct media_entity *entity);
> +
>  void media_entity_graph_walk_start(struct media_entity_graph *graph,
>  		struct media_entity *entity);
>  struct media_entity *
>  media_entity_graph_walk_next(struct media_entity_graph *graph);
>  
> +#define media_entity_call(entity, operation, args...)			\
> +	(((entity)->ops && (entity)->ops->operation) ?			\
> +	 (entity)->ops->operation((entity) , ##args) : -ENOIOCTLCMD)
> +
>  #endif

Re: [RFC/PATCH v4 08/11] media: Links setup

[Mauro Carvalho Chehab]

Em 20-08-2010 12:29, Laurent Pinchart escreveu:
> Create the following ioctl and implement it at the media device level to
> setup links.
> 
> - MEDIA_IOC_SETUP_LINK: Modify the properties of a given link
> 
> The only property that can currently be modified is the ACTIVE link flag
> to activate/deactivate a link. Links marked with the IMMUTABLE link flag
> can not be activated or deactivated.
> 
> Activating and deactivating a link has effects on entities' use count.
> Those changes are automatically propagated through the graph.

You need to address here the release() call: if the userspace application
dies or just exits, the device should be set into a sane state, e. g. devices
powered on should be turned off, and links activated by the application
should be de-activated.

> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Stanimir Varbanov <svarbanov@mm-sol.com>
> Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
> ---
>  Documentation/media-framework.txt |   81 ++++++++++++++-
>  drivers/media/media-device.c      |   45 ++++++++
>  drivers/media/media-entity.c      |  208 +++++++++++++++++++++++++++++++++++++
>  include/linux/media.h             |    1 +
>  include/media/media-entity.h      |    8 ++
>  5 files changed, 340 insertions(+), 3 deletions(-)
> 
> diff --git a/Documentation/media-framework.txt b/Documentation/media-framework.txt
> index 74a137d..7894ef3 100644
> --- a/Documentation/media-framework.txt
> +++ b/Documentation/media-framework.txt
> @@ -278,6 +278,16 @@ When the graph traversal is complete the function will return NULL.
>  Graph traversal can be interrupted at any moment. No cleanup function call is
>  required and the graph structure can be freed normally.
>  
> +Helper functions can be used to find a link between two given pads, or a pad
> +connected to another pad through an active link
> +
> +	media_entity_find_link(struct media_pad *source,
> +			       struct media_pad *sink);
> +
> +	media_entity_remote_pad(struct media_pad *pad);
> +
> +Refer to the kerneldoc documentation for more information.
> +
>  
>  Reference counting and power handling
>  -------------------------------------
> @@ -316,6 +326,46 @@ is allowed to fail when turning power on, in which case the media_entity_get
>  function will return NULL.
>  
>  
> +Links setup
> +-----------
> +
> +Link properties can be modified at runtime by calling
> +
> +	media_entity_setup_link(struct media_link *link, u32 flags);
> +
> +The flags argument contains the requested new link flags.
> +
> +The only configurable property is the ACTIVE link flag to activate/deactivate
> +a link. Links marked with the IMMUTABLE link flag can not be activated or
> +deactivated.
> +
> +When a link is activated or deactivated, the media framework calls the
> +link_setup operation for the two entities at the source and sink of the link,
> +in that order. If the second link_setup call fails, another link_setup call is
> +made on the first entity to restore the original link flags.
> +
> +Entity drivers must implement the link_setup operation if any of their links
> +is non-immutable. The operation must either configure the hardware or store
> +the configuration information to be applied later.
> +
> +Link activation must not have any side effect on other links. If an active
> +link at a sink pad prevents another link at the same pad from being
> +deactivated, the link_setup operation must return -EBUSY and can't implicitly
> +deactivate the first active link.
> +
> +Activating and deactivating a link has effects on entities' reference counts.
> +When two sub-graphs are connected, the reference count of each of them is
> +incremented by the total reference count of all node entities in the other
> +sub-graph. When two sub-graphs are disconnected, the reverse operation is
> +performed. In both cases the set_power operations are called accordingly,
> +ensuring that the link_setup calls are made with power active on the source
> +and sink entities.
> +
> +In other words, activating or deactivating a link propagates reference count
> +changes through the graph, and the final state is identical to what it would
> +have been if the link had been active or inactive from the start.
> +
> +
>  Userspace application API
>  -------------------------
>  
> @@ -439,9 +489,6 @@ Valid entity flags are
>  
>  	ioctl(int fd, int request, struct media_links_enum *argp);
>  
> -Only forward links that originate at one of the entity's source pads are
> -returned during the enumeration process.
> -
>  To enumerate pads and/or links for a given entity, applications set the entity
>  field of a media_links_enum structure and initialize the media_pad_desc and
>  media_link_desc structure arrays pointed by the pads and links fields. They then
> @@ -457,6 +504,9 @@ information about the entity's outbound links. The array must have enough room
>  to store all the entity's outbound links. The number of outbound links can be
>  retrieved with the MEDIA_IOC_ENUM_ENTITIES ioctl.
>  
> +Only outbound (forward) links that originate at one of the entity's source
> +pads are returned during the enumeration process.
> +
>  The media_pad_desc, media_link_desc and media_links_enum structures are defined
>  as
>  
> @@ -497,3 +547,28 @@ struct media_pad_desc	*pads	Pointer to a pads array allocated by the
>  				application. Ignored if NULL.
>  struct media_link_desc	*links	Pointer to a links array allocated by the
>  				application. Ignored if NULL.
> +
> +
> +	MEDIA_IOC_SETUP_LINK - Modify the properties of a link
> +	------------------------------------------------------
> +
> +	ioctl(int fd, int request, struct media_link_desc *argp);
> +
> +To change link properties applications fill a media_link_desc structure with
> +link identification information (source and sink pad) and the new requested link
> +flags. They then call the MEDIA_IOC_SETUP_LINK ioctl with a pointer to that
> +structure.
> +
> +The only configurable property is the ACTIVE link flag to activate/deactivate
> +a link. Links marked with the IMMUTABLE link flag can not be activated or
> +deactivated.
> +
> +Link activation has no side effect on other links. If an active link at the
> +sink pad prevents the link from being activated, the driver returns with a
> +EBUSY error code.
> +
> +If the specified link can't be found the driver returns with a EINVAL error
> +code.
> +
> +The media_pad_desc and media_link_desc structures are described in the
> +MEDIA_IOC_ENUM_LINKS ioctl documentation.
> diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> index 7e020f9..06655d9 100644
> --- a/drivers/media/media-device.c
> +++ b/drivers/media/media-device.c
> @@ -170,6 +170,44 @@ static long media_device_enum_links(struct media_device *mdev,
>  	return 0;
>  }
>  
> +static long media_device_setup_link(struct media_device *mdev,
> +				    struct media_link_desc __user *_ulink)
> +{
> +	struct media_link *link = NULL;
> +	struct media_link_desc ulink;
> +	struct media_entity *source;
> +	struct media_entity *sink;
> +	int ret;
> +
> +	if (copy_from_user(&ulink, _ulink, sizeof(ulink)))
> +		return -EFAULT;
> +
> +	/* Find the source and sink entities and link.
> +	 */
> +	source = find_entity(mdev, ulink.source.entity);
> +	sink = find_entity(mdev, ulink.sink.entity);
> +
> +	if (source == NULL || sink == NULL)
> +		return -EINVAL;
> +
> +	if (ulink.source.index >= source->num_pads ||
> +	    ulink.sink.index >= sink->num_pads)
> +		return -EINVAL;
> +
> +	link = media_entity_find_link(&source->pads[ulink.source.index],
> +				      &sink->pads[ulink.sink.index]);
> +	if (link == NULL)
> +		return -EINVAL;
> +
> +	/* Setup the link on both entities. */
> +	ret = __media_entity_setup_link(link, ulink.flags);
> +
> +	if (copy_to_user(_ulink, &ulink, sizeof(ulink)))
> +		return -EFAULT;
> +
> +	return ret;
> +}
> +
>  static long media_device_ioctl(struct file *filp, unsigned int cmd,
>  			       unsigned long arg)
>  {
> @@ -195,6 +233,13 @@ static long media_device_ioctl(struct file *filp, unsigned int cmd,
>  		mutex_unlock(&dev->graph_mutex);
>  		break;
>  
> +	case MEDIA_IOC_SETUP_LINK:
> +		mutex_lock(&dev->graph_mutex);
> +		ret = media_device_setup_link(dev,
> +				(struct media_link_desc __user *)arg);
> +		mutex_unlock(&dev->graph_mutex);
> +		break;
> +
>  	default:
>  		ret = -ENOIOCTLCMD;
>  	}
> diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
> index da4fef6..bc97b78 100644
> --- a/drivers/media/media-entity.c
> +++ b/drivers/media/media-entity.c
> @@ -198,6 +198,25 @@ EXPORT_SYMBOL_GPL(media_entity_graph_walk_next);
>   * Power state handling
>   */
>  
> +/*
> + * Return power count of nodes directly or indirectly connected to
> + * a given entity.
> + */
> +static int media_entity_count_node(struct media_entity *entity)
> +{
> +	struct media_entity_graph graph;
> +	int use = 0;
> +
> +	media_entity_graph_walk_start(&graph, entity);
> +
> +	while ((entity = media_entity_graph_walk_next(&graph))) {
> +		if (media_entity_type(entity) == MEDIA_ENTITY_TYPE_NODE)
> +			use += entity->use_count;
> +	}
> +
> +	return use;
> +}
> +
>  /* Apply use count to an entity. */
>  static void media_entity_use_apply_one(struct media_entity *entity, int change)
>  {
> @@ -261,6 +280,32 @@ static int media_entity_power_apply(struct media_entity *entity, int change)
>  	return ret;
>  }
>  
> +/* Apply the power state changes when connecting two entities. */
> +static int media_entity_power_connect(struct media_entity *one,
> +				      struct media_entity *theother)
> +{
> +	int power_one = media_entity_count_node(one);
> +	int power_theother = media_entity_count_node(theother);
> +	int ret = 0;
> +
> +	ret = media_entity_power_apply(one, power_theother);
> +	if (ret < 0)
> +		return ret;
> +
> +	return media_entity_power_apply(theother, power_one);
> +}
> +
> +static void media_entity_power_disconnect(struct media_entity *one,
> +					  struct media_entity *theother)
> +{
> +	int power_one = media_entity_count_node(one);
> +	int power_theother = media_entity_count_node(theother);
> +
> +	/* Powering off entities is assumed to never fail. */
> +	media_entity_power_apply(one, -power_theother);
> +	media_entity_power_apply(theother, -power_one);
> +}
> +
>  /*
>   * Apply use count change to graph and change power state of entities
>   * accordingly.
> @@ -404,3 +449,166 @@ media_entity_create_link(struct media_entity *source, u16 source_pad,
>  	return 0;
>  }
>  EXPORT_SYMBOL(media_entity_create_link);
> +
> +static int __media_entity_setup_link_notify(struct media_link *link, u32 flags)
> +{
> +	const u32 mask = MEDIA_LINK_FLAG_ACTIVE;
> +	int ret;
> +
> +	/* Notify both entities. */
> +	ret = media_entity_call(link->source->entity, link_setup,
> +				link->source, link->sink, flags);
> +	if (ret < 0 && ret != -ENOIOCTLCMD)
> +		return ret;
> +
> +	ret = media_entity_call(link->sink->entity, link_setup,
> +				link->sink, link->source, flags);
> +	if (ret < 0 && ret != -ENOIOCTLCMD) {
> +		media_entity_call(link->source->entity, link_setup,
> +				  link->source, link->sink, link->flags);
> +		return ret;
> +	}
> +
> +	link->flags = (link->flags & ~mask) | (flags & mask);
> +	link->reverse->flags = link->flags;
> +
> +	return 0;
> +}
> +
> +/**
> + * __media_entity_setup_link - Configure a media link
> + * @link: The link being configured
> + * @flags: Link configuration flags
> + *
> + * The bulk of link setup is handled by the two entities connected through the
> + * link. This function notifies both entities of the link configuration change.
> + *
> + * If the link is immutable or if the current and new configuration are
> + * identical, return immediately.
> + *
> + * The user is expected to hold link->source->parent->mutex. If not,
> + * media_entity_setup_link() should be used instead.
> + */
> +int __media_entity_setup_link(struct media_link *link, u32 flags)
> +{
> +	struct media_entity *source, *sink;
> +	int ret = -EBUSY;
> +
> +	if (link == NULL)
> +		return -EINVAL;
> +
> +	if (link->flags & MEDIA_LINK_FLAG_IMMUTABLE)
> +		return link->flags == flags ? 0 : -EINVAL;
> +
> +	if (link->flags == flags)
> +		return 0;
> +
> +	source = __media_entity_get(link->source->entity);
> +	if (!source)
> +		return ret;
> +
> +	sink = __media_entity_get(link->sink->entity);
> +	if (!sink)
> +		goto err___media_entity_get;
> +
> +	if (flags & MEDIA_LINK_FLAG_ACTIVE) {
> +		ret = media_entity_power_connect(source, sink);
> +		if (ret < 0)
> +			goto err_media_entity_power_connect;
> +	}
> +
> +	ret = __media_entity_setup_link_notify(link, flags);
> +	if (ret < 0)
> +		goto err___media_entity_setup_link_notify;
> +
> +	if (!(flags & MEDIA_LINK_FLAG_ACTIVE))
> +		media_entity_power_disconnect(source, sink);
> +
> +	__media_entity_put(sink);
> +	__media_entity_put(source);
> +
> +	return 0;
> +
> +err___media_entity_setup_link_notify:
> +	if (flags & MEDIA_LINK_FLAG_ACTIVE)
> +		media_entity_power_disconnect(source, sink);
> +err_media_entity_power_connect:
> +	__media_entity_put(sink);
> +err___media_entity_get:
> +	__media_entity_put(source);
> +
> +	return ret;
> +}
> +
> +int media_entity_setup_link(struct media_link *link, u32 flags)
> +{
> +	int ret;
> +
> +	mutex_lock(&link->source->entity->parent->graph_mutex);
> +	ret = __media_entity_setup_link(link, flags);
> +	mutex_unlock(&link->source->entity->parent->graph_mutex);
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(media_entity_setup_link);
> +
> +/**
> + * media_entity_find_link - Find a link between two pads
> + * @source: Source pad
> + * @sink: Sink pad
> + *
> + * Return a pointer to the link between the two entities. If no such link
> + * exists, return NULL.
> + */
> +struct media_link *
> +media_entity_find_link(struct media_pad *source, struct media_pad *sink)
> +{
> +	struct media_link *link;
> +	unsigned int i;
> +
> +	for (i = 0; i < source->entity->num_links; ++i) {
> +		link = &source->entity->links[i];
> +
> +		if (link->source->entity == source->entity &&
> +		    link->source->index == source->index &&
> +		    link->sink->entity == sink->entity &&
> +		    link->sink->index == sink->index)
> +			return link;
> +	}
> +
> +	return NULL;
> +}
> +EXPORT_SYMBOL_GPL(media_entity_find_link);
> +
> +/**
> + * media_entity_remote_pad - Locate the pad at the remote end of a link
> + * @entity: Local entity
> + * @pad: Pad at the local end of the link
> + *
> + * Search for a remote pad connected to the given pad by iterating over all
> + * links originating or terminating at that pad until an active link is found.
> + *
> + * Return a pointer to the pad at the remote end of the first found active link,
> + * or NULL if no active link has been found.
> + */
> +struct media_pad *media_entity_remote_pad(struct media_pad *pad)
> +{
> +	unsigned int i;
> +
> +	for (i = 0; i < pad->entity->num_links; i++) {
> +		struct media_link *link = &pad->entity->links[i];
> +
> +		if (!(link->flags & MEDIA_LINK_FLAG_ACTIVE))
> +			continue;
> +
> +		if (link->source == pad)
> +			return link->sink;
> +
> +		if (link->sink == pad)
> +			return link->source;
> +	}
> +
> +	return NULL;
> +
> +}
> +EXPORT_SYMBOL_GPL(media_entity_remote_pad);
> diff --git a/include/linux/media.h b/include/linux/media.h
> index 542509b..edb53c2 100644
> --- a/include/linux/media.h
> +++ b/include/linux/media.h
> @@ -100,5 +100,6 @@ struct media_links_enum {
>  #define MEDIA_IOC_DEVICE_INFO		_IOWR('M', 1, struct media_device_info)
>  #define MEDIA_IOC_ENUM_ENTITIES		_IOWR('M', 2, struct media_entity_desc)
>  #define MEDIA_IOC_ENUM_LINKS		_IOWR('M', 3, struct media_links_enum)
> +#define MEDIA_IOC_SETUP_LINK		_IOWR('M', 4, struct media_link_desc)
>  
>  #endif /* __LINUX_MEDIA_H */
> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> index 8c40d5e..0f0697b 100644
> --- a/include/media/media-entity.h
> +++ b/include/media/media-entity.h
> @@ -18,6 +18,9 @@ struct media_pad {
>  };
>  
>  struct media_entity_operations {
> +	int (*link_setup)(struct media_entity *entity,
> +			  const struct media_pad *local,
> +			  const struct media_pad *remote, u32 flags);
>  	int (*set_power)(struct media_entity *entity, int power);
>  };
>  
> @@ -88,6 +91,11 @@ int media_entity_init(struct media_entity *entity, u16 num_pads,
>  void media_entity_cleanup(struct media_entity *entity);
>  int media_entity_create_link(struct media_entity *source, u16 source_pad,
>  		struct media_entity *sink, u16 sink_pad, u32 flags);
> +int __media_entity_setup_link(struct media_link *link, u32 flags);
> +int media_entity_setup_link(struct media_link *link, u32 flags);
> +struct media_link *media_entity_find_link(struct media_pad *source,
> +		struct media_pad *sink);
> +struct media_pad *media_entity_remote_pad(struct media_pad *pad);
>  
>  struct media_entity *media_entity_get(struct media_entity *entity);
>  void media_entity_put(struct media_entity *entity);

Re: [RFC/PATCH v4 11/11] v4l: Make v4l2_subdev inherit from media_entity

[Mauro Carvalho Chehab]

Em 20-08-2010 12:29, Laurent Pinchart escreveu:
> V4L2 subdevices are media entities. As such they need to inherit from
> (include) the media_entity structure.
> 
> When registering/unregistering the subdevice, the media entity is
> automatically registered/unregistered. The entity is acquired on device
> open and released on device close.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
> ---
>  Documentation/video4linux/v4l2-framework.txt |   23 ++++++++++++++++++
>  drivers/media/video/v4l2-device.c            |   32 +++++++++++++++++++++-----
>  drivers/media/video/v4l2-subdev.c            |   27 +++++++++++++++++++++-
>  include/media/v4l2-subdev.h                  |    7 +++++
>  4 files changed, 82 insertions(+), 7 deletions(-)
> 
> diff --git a/Documentation/video4linux/v4l2-framework.txt b/Documentation/video4linux/v4l2-framework.txt
> index 7ff4016..3416d93 100644
> --- a/Documentation/video4linux/v4l2-framework.txt
> +++ b/Documentation/video4linux/v4l2-framework.txt
> @@ -263,6 +263,26 @@ A sub-device driver initializes the v4l2_subdev struct using:
>  Afterwards you need to initialize subdev->name with a unique name and set the
>  module owner. This is done for you if you use the i2c helper functions.
>  
> +If integration with the media framework is needed, you must initialize the
> +media_entity struct embedded in the v4l2_subdev struct (entity field) by
> +calling media_entity_init():
> +
> +	struct media_pad *pads = &my_sd->pads;
> +	int err;
> +
> +	err = media_entity_init(&sd->entity, npads, pads, 0);
> +
> +The pads array must have been previously initialized. There is no need to
> +manually set the struct media_entity type and name fields, but the revision
> +field must be initialized if needed.
> +
> +A reference to the entity will be automatically acquired/released when the
> +subdev device node (if any) is opened/closed.
> +
> +Don't forget to cleanup the media entity before the sub-device is destroyed:
> +
> +	media_entity_cleanup(&sd->entity);
> +
>  A device (bridge) driver needs to register the v4l2_subdev with the
>  v4l2_device:
>  
> @@ -272,6 +292,9 @@ This can fail if the subdev module disappeared before it could be registered.
>  After this function was called successfully the subdev->dev field points to
>  the v4l2_device.
>  
> +If the v4l2_device parent device has a non-NULL mdev field, the sub-device
> +entity will be automatically registered with the media device.
> +
>  You can unregister a sub-device using:
>  
>  	v4l2_device_unregister_subdev(sd);
> diff --git a/drivers/media/video/v4l2-device.c b/drivers/media/video/v4l2-device.c
> index 91452cd..4f74d01 100644
> --- a/drivers/media/video/v4l2-device.c
> +++ b/drivers/media/video/v4l2-device.c
> @@ -114,10 +114,11 @@ void v4l2_device_unregister(struct v4l2_device *v4l2_dev)
>  EXPORT_SYMBOL_GPL(v4l2_device_unregister);
>  
>  int v4l2_device_register_subdev(struct v4l2_device *v4l2_dev,
> -						struct v4l2_subdev *sd)
> +				struct v4l2_subdev *sd)
>  {
> +	struct media_entity *entity = &sd->entity;
>  	struct video_device *vdev;
> -	int ret = 0;
> +	int ret;
>  
>  	/* Check for valid input */
>  	if (v4l2_dev == NULL || sd == NULL || !sd->name[0])
> @@ -129,6 +130,15 @@ int v4l2_device_register_subdev(struct v4l2_device *v4l2_dev,
>  	if (!try_module_get(sd->owner))
>  		return -ENODEV;
>  
> +	/* Register the entity. */
> +	if (v4l2_dev->mdev) {
> +		ret = media_device_register_entity(v4l2_dev->mdev, entity);
> +		if (ret < 0) {
> +			module_put(sd->owner);
> +			return ret;
> +		}
> +	}
> +
>  	sd->v4l2_dev = v4l2_dev;
>  	spin_lock(&v4l2_dev->lock);
>  	list_add_tail(&sd->list, &v4l2_dev->subdevs);
> @@ -143,26 +153,36 @@ int v4l2_device_register_subdev(struct v4l2_device *v4l2_dev,
>  	if (sd->flags & V4L2_SUBDEV_FL_HAS_DEVNODE) {
>  		ret = __video_register_device(vdev, VFL_TYPE_SUBDEV, -1, 1,
>  					      sd->owner);
> -		if (ret < 0)
> +		if (ret < 0) {
>  			v4l2_device_unregister_subdev(sd);
> +			return ret;
> +		}
>  	}
>  
> -	return ret;
> +	entity->v4l.major = VIDEO_MAJOR;
> +	entity->v4l.minor = vdev->minor;

Hmm... it needs to check if entity is not null.

> +	return 0;
>  }
>  EXPORT_SYMBOL_GPL(v4l2_device_register_subdev);
>  
>  void v4l2_device_unregister_subdev(struct v4l2_subdev *sd)
>  {
> +	struct v4l2_device *v4l2_dev;
> +
>  	/* return if it isn't registered */
>  	if (sd == NULL || sd->v4l2_dev == NULL)
>  		return;
>  
> -	spin_lock(&sd->v4l2_dev->lock);
> +	v4l2_dev = sd->v4l2_dev;
> +
> +	spin_lock(&v4l2_dev->lock);
>  	list_del(&sd->list);
> -	spin_unlock(&sd->v4l2_dev->lock);
> +	spin_unlock(&v4l2_dev->lock);
>  	sd->v4l2_dev = NULL;
>  
>  	module_put(sd->owner);
> +	if (v4l2_dev->mdev)
> +		media_device_unregister_entity(&sd->entity);
>  	video_unregister_device(&sd->devnode);
>  }
>  EXPORT_SYMBOL_GPL(v4l2_device_unregister_subdev);
> diff --git a/drivers/media/video/v4l2-subdev.c b/drivers/media/video/v4l2-subdev.c
> index b063195..1efa267 100644
> --- a/drivers/media/video/v4l2-subdev.c
> +++ b/drivers/media/video/v4l2-subdev.c
> @@ -32,7 +32,8 @@ static int subdev_open(struct file *file)
>  {
>  	struct video_device *vdev = video_devdata(file);
>  	struct v4l2_subdev *sd = vdev_to_v4l2_subdev(vdev);
> -	struct v4l2_fh *vfh;
> +	struct media_entity *entity;
> +	struct v4l2_fh *vfh = NULL;
>  	int ret;
>  
>  	if (!sd->initialized)
> @@ -59,10 +60,17 @@ static int subdev_open(struct file *file)
>  		file->private_data = vfh;
>  	}
>  
> +	entity = media_entity_get(&sd->entity);
> +	if (!entity) {
> +		ret = -EBUSY;
> +		goto err;
> +	}
> +

It needs to check if v4l2_dev->mdev is not null, as it makes no sense to get
an entity if the driver is not using the media entity.

>  	return 0;
>  
>  err:
>  	if (vfh != NULL) {
> +		v4l2_fh_del(vfh);
>  		v4l2_fh_exit(vfh);
>  		kfree(vfh);
>  	}
> @@ -72,8 +80,12 @@ err:
>  
>  static int subdev_close(struct file *file)
>  {
> +	struct video_device *vdev = video_devdata(file);
> +	struct v4l2_subdev *sd = vdev_to_v4l2_subdev(vdev);
>  	struct v4l2_fh *vfh = file->private_data;
>  
> +	media_entity_put(&sd->entity);

Same here: don't put it, if v4l2_dev->mdev = NULL (I think that you're already
checking for it at media_entity_put).

> +
>  	if (vfh != NULL) {
>  		v4l2_fh_del(vfh);
>  		v4l2_fh_exit(vfh);
> @@ -172,5 +184,18 @@ void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
>  	sd->grp_id = 0;
>  	sd->priv = NULL;
>  	sd->initialized = 1;
> +	sd->entity.name = sd->name;
> +	sd->entity.type = MEDIA_ENTITY_TYPE_SUBDEV;
>  }
>  EXPORT_SYMBOL(v4l2_subdev_init);
> +
> +int v4l2_subdev_set_power(struct media_entity *entity, int power)
> +{
> +	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
> +
> +	dev_dbg(entity->parent->dev,
> +		"%s power%s\n", entity->name, power ? "on" : "off");
> +
> +	return v4l2_subdev_call(sd, core, s_power, power);
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_set_power);

Also, should do nothing if media entity is null. 

PS.: Not sure where this function is called, as the caller is not on this patch series.
The better would be to add this together with the patch calling v4l2_subdev_set_power().

> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 55a8c93..f9e1897 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -21,6 +21,7 @@
>  #ifndef _V4L2_SUBDEV_H
>  #define _V4L2_SUBDEV_H
>  
> +#include <media/media-entity.h>
>  #include <media/v4l2-common.h>
>  #include <media/v4l2-dev.h>
>  #include <media/v4l2-mediabus.h>
> @@ -421,6 +422,8 @@ struct v4l2_subdev_ops {
>     stand-alone or embedded in a larger struct.
>   */
>  struct v4l2_subdev {
> +	struct media_entity entity;
> +
>  	struct list_head list;
>  	struct module *owner;
>  	u32 flags;
> @@ -439,6 +442,8 @@ struct v4l2_subdev {
>  	unsigned int nevents;
>  };
>  
> +#define media_entity_to_v4l2_subdev(ent) \
> +	container_of(ent, struct v4l2_subdev, entity)
>  #define vdev_to_v4l2_subdev(vdev) \
>  	container_of(vdev, struct v4l2_subdev, devnode)
>  
> @@ -457,6 +462,8 @@ static inline void *v4l2_get_subdevdata(const struct v4l2_subdev *sd)
>  void v4l2_subdev_init(struct v4l2_subdev *sd,
>  		      const struct v4l2_subdev_ops *ops);
>  
> +int v4l2_subdev_set_power(struct media_entity *entity, int power);
> +
>  /* Call an ops of a v4l2_subdev, doing the right checks against
>     NULL pointers.
>  

Re: [RFC/PATCH v4 00/11] Media controller (core and V4L2)

[Mauro Carvalho Chehab]

Em 20-08-2010 12:29, Laurent Pinchart escreveu:
> Hi everybody,
> 
> Here's the fourth version of the media controller patches. All comments received
> so far have hopefully been incorporated.
> 
> Compared to the previous version, the patches have been rebased on top of 2.6.35
> and a MEDIA_IOC_DEVICE_INFO ioctl has been added.
> 
> I won't submit a rebased version of the V4L2 API additions and OMAP3 ISP patches
> right now. I will first clean up (and document) the V4L2 API additions patches,
> and I will submit them as a proper RFC instead of sample code.

Hi Laurent,

Sorry for a late review on the media controller API. I got flooded by patches and
other work since the merge window. Anyway, just finished my review, and sent a
per-patch comment for most patches.

One general comment about it: the userspace API should be documented via DocBook, 
to be consistent with V4L2 and DVB API specs.

It should also be clear at the API specs there that not all media drivers will 
implement the media controller API, as its main focus is to allow better control
of SoC devices, where there are needs to control some intrinsic characteristics of 
parts of the devices, complementing the V4L2 spec. 

This means that it is needed to add some comments at the kernelspace API doc, saying that
the drivers implementing the media controller API are required to work properly even 
when userspace is not using the media controller API;

This also means that it is needed to add some comments at the userspace API doc, saying
that userspace applications should not assume that media drivers will implement the 
media controller API. So, userspace applications implementing the media controller 
and V4L2 API's are required to work properly if the device doesn't present a media 
controler API interface. It should also say that no driver should just implement the
media controller API.

> 
> Laurent Pinchart (9):
>   media: Media device node support
>   media: Media device
>   media: Entities, pads and links
>   media: Media device information query
>   media: Entities, pads and links enumeration
>   media: Links setup
>   v4l: Add a media_device pointer to the v4l2_device structure
>   v4l: Make video_device inherit from media_entity
>   v4l: Make v4l2_subdev inherit from media_entity
> 
> Sakari Ailus (2):
>   media: Entity graph traversal
>   media: Reference count and power handling
> 
>  Documentation/media-framework.txt            |  574 ++++++++++++++++++++++++
>  Documentation/video4linux/v4l2-framework.txt |   72 +++-
>  drivers/media/Makefile                       |    8 +-
>  drivers/media/media-device.c                 |  377 ++++++++++++++++
>  drivers/media/media-devnode.c                |  310 +++++++++++++
>  drivers/media/media-entity.c                 |  614 ++++++++++++++++++++++++++
>  drivers/media/video/v4l2-dev.c               |   35 ++-
>  drivers/media/video/v4l2-device.c            |   45 ++-
>  drivers/media/video/v4l2-subdev.c            |   27 ++-
>  include/linux/media.h                        |  105 +++++
>  include/media/media-device.h                 |   90 ++++
>  include/media/media-devnode.h                |   78 ++++
>  include/media/media-entity.h                 |  112 +++++
>  include/media/v4l2-dev.h                     |    6 +
>  include/media/v4l2-device.h                  |    2 +
>  include/media/v4l2-subdev.h                  |    7 +
>  16 files changed, 2440 insertions(+), 22 deletions(-)
>  create mode 100644 Documentation/media-framework.txt
>  create mode 100644 drivers/media/media-device.c
>  create mode 100644 drivers/media/media-devnode.c
>  create mode 100644 drivers/media/media-entity.c
>  create mode 100644 include/linux/media.h
>  create mode 100644 include/media/media-device.h
>  create mode 100644 include/media/media-devnode.h
>  create mode 100644 include/media/media-entity.h
> 

Re: [RFC/PATCH v4 05/11] media: Reference count and power handling

[Sakari Ailus]

Hi Mauro,

Thanks for the comments!

Mauro Carvalho Chehab wrote:
> Em 20-08-2010 12:29, Laurent Pinchart escreveu:
>> From: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
>>
>> Basically these are the interface functions:
>>
>> media_entity_get() - acquire entity
>> media_entity_put() - release entity
>>
>> 	If the entity is of node type, the power change is distributed to
>> 	all connected entities. For non-nodes it only affects that very
>> 	node. A mutex is used to serialise access to the entity graph.
>>
>> In the background there's a depth-first search algorithm that traverses the
>> active links in the graph. All these functions parse the graph to implement
>> whatever they're to do.
>>
>> The module counters are increased/decreased in media_entity_get/put to
>> prevent module unloading when an entity is referenced.
>>
>> Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
>> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>> Signed-off-by: Stanimir Varbanov <svarbanov@mm-sol.com>
>> ---
>>  Documentation/media-framework.txt |   37 +++++++++
>>  drivers/media/media-device.c      |    1 +
>>  drivers/media/media-entity.c      |  146 +++++++++++++++++++++++++++++++++++++
>>  include/media/media-device.h      |    4 +
>>  include/media/media-entity.h      |   15 ++++
>>  5 files changed, 203 insertions(+), 0 deletions(-)
>>
>> diff --git a/Documentation/media-framework.txt b/Documentation/media-framework.txt
>> index a599824..59649e9 100644
>> --- a/Documentation/media-framework.txt
>> +++ b/Documentation/media-framework.txt
>> @@ -278,3 +278,40 @@ When the graph traversal is complete the function will return NULL.
>>  Graph traversal can be interrupted at any moment. No cleanup function call is
>>  required and the graph structure can be freed normally.
>>  
>> +
>> +Reference counting and power handling
>> +-------------------------------------
>> +
>> +Before accessing type-specific entities operations (such as the V4L2
>> +sub-device operations), drivers must acquire a reference to the entity. This
>> +ensures that the entity will be powered on and ready to accept requests.
>> +Similarly, after being done with an entity, drivers must release the
>> +reference.
>> +
>> +	media_entity_get(struct media_entity *entity)
>> +
>> +The function will increase the entity reference count. If the entity is a node
>> +(MEDIA_ENTITY_TYPE_NODE type), the reference count of all entities it is
>> +connected to, both directly or indirectly, through active links is increased.
>> +This ensures that the whole media pipeline will be ready to process
>> +
>> +Acquiring a reference to an entity increases the media device module reference
>> +count to prevent module unloading when an entity is being used.
>> +
>> +media_entity_get will return a pointer to the entity if successful, or NULL
>> +otherwise.
>> +
>> +	media_entity_put(struct media_entity *entity)
>> +
>> +The function will decrease the entity reference count and, for node entities,
>> +like media_entity_get, the reference count of all connected entities. Calling
>> +media_entity_put with a NULL argument is valid and will return immediately.
>> +
>> +When the first reference to an entity is acquired, or the last reference
>> +released, the entity's set_power operation is called. Entity drivers must
>> +implement the operation if they need to perform any power management task,
>> +such as turning powers or clocks on or off. If no power management is
>> +required, drivers don't need to provide a set_power operation. The operation
>> +is allowed to fail when turning power on, in which case the media_entity_get
>> +function will return NULL.
> 
> The idea of doing power management via media entity get/put doesn't seem right.
> The mediabus interface and its usage should be optional, and only specialized
> applications will likely implement it. If a refcount 0 means power off, it ends
> that a device implementing the media bus will not work with V4L2 applications.

The Media controller does handle the power through reference count but
this does not limit to subdev entities. The reference count is also
applied recursively to all entities which are connected through active
links.

There are two cases:

1. The user application opens a subdev node. The subdev entity use count
will be incremented and the subdev will be powered up.

2. The user application opens a video node. The reference count for all
entities connected to the video node entity through active links will be
incremented. Subdevs will be powered up as well (if they are not already
because of (1) above). The same works if the entities connected through
a video node are connected to another entity and the link to that entity
is activated. In this case the use_counts of the entity sets are applied
across the both sets.

The user application does not need to use the Media controller interface
to get this functionality.

Another thing is that the user likely wants to use the device through
libv4l most likely, at least in the case of OMAP 3 ISP case. The link
configuration can be made by libv4l so that the regular V4L2
applications will work as expected.

>> +
>> diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
>> index eeb002e..c309d3c 100644
>> --- a/drivers/media/media-device.c
>> +++ b/drivers/media/media-device.c
>> @@ -71,6 +71,7 @@ int __must_check media_device_register(struct media_device *mdev)
>>  	mdev->entity_id = 1;
>>  	INIT_LIST_HEAD(&mdev->entities);
>>  	spin_lock_init(&mdev->lock);
>> +	mutex_init(&mdev->graph_mutex);
>>  
>>  	/* Register the device node. */
>>  	mdev->devnode.fops = &media_device_fops;
>> diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
>> index c277c18..da4fef6 100644
>> --- a/drivers/media/media-entity.c
>> +++ b/drivers/media/media-entity.c
>> @@ -21,6 +21,7 @@
>>  #include <linux/module.h>
>>  #include <linux/slab.h>
>>  #include <media/media-entity.h>
>> +#include <media/media-device.h>
>>  
>>  /**
>>   * media_entity_init - Initialize a media entity
>> @@ -194,6 +195,151 @@ media_entity_graph_walk_next(struct media_entity_graph *graph)
>>  EXPORT_SYMBOL_GPL(media_entity_graph_walk_next);
>>  
>>  /* -----------------------------------------------------------------------------
>> + * Power state handling
>> + */
>> +
>> +/* Apply use count to an entity. */
>> +static void media_entity_use_apply_one(struct media_entity *entity, int change)
>> +{
>> +	entity->use_count += change;
>> +	WARN_ON(entity->use_count < 0);
> 
> Instead of producing a warning, just deny it to have usage bellow zero. As this will
> be called from userspace, the entire interface should be reliable enough to avoid
> dumb applications to miss-use it.

This WARN_ON() always indicates a driver (or MC) bug. The entity
use_count should never be under 0, thus the warning.

The calls to this function should be always related to an open file
handle in a way or another. There is no direct user influence over this.

> Also: what happens if an userspace application dies or suffer any troubles? You
> need to reset all use_count's at release() callback.

Yes, this is true. media_entity_{get,put} should always be called when
file handles are open()ed or release()d.

I guess Laurent will correct me if required. :-)

Cheers,

-- 
Sakari Ailus
sakari.ailus@maxwell.research.nokia.com

Re: [RFC/PATCH v4 00/11] Media controller (core and V4L2)

[Laurent Pinchart]

Hi Mauro,

On Thursday 09 September 2010 03:44:15 Mauro Carvalho Chehab wrote:
> Em 20-08-2010 12:29, Laurent Pinchart escreveu:
> > Hi everybody,
> > 
> > Here's the fourth version of the media controller patches. All comments
> > received so far have hopefully been incorporated.
> > 
> > Compared to the previous version, the patches have been rebased on top of
> > 2.6.35 and a MEDIA_IOC_DEVICE_INFO ioctl has been added.
> > 
> > I won't submit a rebased version of the V4L2 API additions and OMAP3 ISP
> > patches right now. I will first clean up (and document) the V4L2 API
> > additions patches, and I will submit them as a proper RFC instead of
> > sample code.
> 
> Hi Laurent,
> 
> Sorry for a late review on the media controller API. I got flooded by
> patches and other work since the merge window.

No worries. I was on holidays last week anyway.

> Anyway, just finished my review, and sent a per-patch comment for most
> patches.

Thanks.

> One general comment about it: the userspace API should be documented via
> DocBook, to be consistent with V4L2 and DVB API specs.

I feared so :-) I'll work on it.

> It should also be clear at the API specs there that not all media drivers
> will implement the media controller API,

I agree.

> as its main focus is to allow better control of SoC devices, where there are
> needs to control some intrinsic characteristics of parts of the devices,
> complementing the V4L2 spec.

Some consumer devices (ivtv for instance) will also benefit from the media 
controller, the API is not specific to SoC devices only.

> This means that it is needed to add some comments at the kernelspace API
> doc, saying that the drivers implementing the media controller API are
> required to work properly even when userspace is not using the media
> controller API;

That's another issue. Drivers should make a best effort to allow pure V4L2 
applications to work with a subset of the video device nodes, but they will 
only offer a subset of the hardware capabilities. For SoC devices it's even 
worse, it might be way too difficult to implement support for pure V4L2 
applications in the kernel driver(s). In that case a device-specific libv4l 
plugin will configure the driver using the media controller API for pure V4L2 
applications.

> This also means that it is needed to add some comments at the userspace API
> doc, saying that userspace applications should not assume that media
> drivers will implement the media controller API.

Agreed. Many V4L2 drivers will not implement the media controller API.

> So, userspace applications implementing the media controller and V4L2 API's
> are required to work properly if the device doesn't present a media
> controler API interface.

Applications can require support for the media controller API, but they should 
only do so for specific cases (for instance applications tied to specific SoC 
hardware, or graphical user interfaces on top of the media controller API 
similar to qv4l2).

> It should also say that no driver should just implement the media controller
> API.

I haven't thought about that, as it would be pretty useless :-)

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v4 00/11] Media controller (core and V4L2)

[Hans Verkuil]

> Hi Mauro,
>
> On Thursday 09 September 2010 03:44:15 Mauro Carvalho Chehab wrote:
>> Em 20-08-2010 12:29, Laurent Pinchart escreveu:
>> > Hi everybody,
>> >
>> > Here's the fourth version of the media controller patches. All
>> comments
>> > received so far have hopefully been incorporated.
>> >
>> > Compared to the previous version, the patches have been rebased on top
>> of
>> > 2.6.35 and a MEDIA_IOC_DEVICE_INFO ioctl has been added.
>> >
>> > I won't submit a rebased version of the V4L2 API additions and OMAP3
>> ISP
>> > patches right now. I will first clean up (and document) the V4L2 API
>> > additions patches, and I will submit them as a proper RFC instead of
>> > sample code.
>>
>> Hi Laurent,
>>
>> Sorry for a late review on the media controller API. I got flooded by
>> patches and other work since the merge window.
>
> No worries. I was on holidays last week anyway.
>
>> Anyway, just finished my review, and sent a per-patch comment for most
>> patches.
>
> Thanks.
>
>> One general comment about it: the userspace API should be documented via
>> DocBook, to be consistent with V4L2 and DVB API specs.
>
> I feared so :-) I'll work on it.
>
>> It should also be clear at the API specs there that not all media
>> drivers
>> will implement the media controller API,
>
> I agree.
>
>> as its main focus is to allow better control of SoC devices, where there
>> are
>> needs to control some intrinsic characteristics of parts of the devices,
>> complementing the V4L2 spec.
>
> Some consumer devices (ivtv for instance) will also benefit from the media
> controller, the API is not specific to SoC devices only.
>
>> This means that it is needed to add some comments at the kernelspace API
>> doc, saying that the drivers implementing the media controller API are
>> required to work properly even when userspace is not using the media
>> controller API;
>
> That's another issue. Drivers should make a best effort to allow pure V4L2
> applications to work with a subset of the video device nodes, but they
> will
> only offer a subset of the hardware capabilities. For SoC devices it's
> even
> worse, it might be way too difficult to implement support for pure V4L2
> applications in the kernel driver(s). In that case a device-specific
> libv4l
> plugin will configure the driver using the media controller API for pure
> V4L2
> applications.
>
>> This also means that it is needed to add some comments at the userspace
>> API
>> doc, saying that userspace applications should not assume that media
>> drivers will implement the media controller API.
>
> Agreed. Many V4L2 drivers will not implement the media controller API.
>
>> So, userspace applications implementing the media controller and V4L2
>> API's
>> are required to work properly if the device doesn't present a media
>> controler API interface.
>
> Applications can require support for the media controller API, but they
> should
> only do so for specific cases (for instance applications tied to specific
> SoC
> hardware, or graphical user interfaces on top of the media controller API
> similar to qv4l2).
>
>> It should also say that no driver should just implement the media
>> controller
>> API.
>
> I haven't thought about that, as it would be pretty useless :-)

I actually think that it should be possible without too much effort to
make the media API available automatically for those drivers that do not
implement it themselves. For the standard drivers it basically just has to
enumerate what is already known.

It would help a lot with apps like MythTV that want to find related
devices (e.g. audio/video/vbi).

Regards,

           Hans

-- 
Hans Verkuil - video4linux developer - sponsored by TANDBERG, part of Cisco

Re: [RFC/PATCH v4 00/11] Media controller (core and V4L2)

[Mauro Carvalho Chehab]

Em 14-09-2010 09:25, Laurent Pinchart escreveu:
> Hi Mauro,
> 
> On Thursday 09 September 2010 03:44:15 Mauro Carvalho Chehab wrote:
>> Em 20-08-2010 12:29, Laurent Pinchart escreveu:
>>> Hi everybody,
>>>
>>> Here's the fourth version of the media controller patches. All comments
>>> received so far have hopefully been incorporated.
>>>
>>> Compared to the previous version, the patches have been rebased on top of
>>> 2.6.35 and a MEDIA_IOC_DEVICE_INFO ioctl has been added.
>>>
>>> I won't submit a rebased version of the V4L2 API additions and OMAP3 ISP
>>> patches right now. I will first clean up (and document) the V4L2 API
>>> additions patches, and I will submit them as a proper RFC instead of
>>> sample code.
>>
>> Hi Laurent,
>>
>> Sorry for a late review on the media controller API. I got flooded by
>> patches and other work since the merge window.
> 
> No worries. I was on holidays last week anyway.
> 
>> Anyway, just finished my review, and sent a per-patch comment for most
>> patches.
> 
> Thanks.
> 
>> One general comment about it: the userspace API should be documented via
>> DocBook, to be consistent with V4L2 and DVB API specs.
> 
> I feared so :-) I'll work on it.
> 
>> It should also be clear at the API specs there that not all media drivers
>> will implement the media controller API,
> 
> I agree.
> 
>> as its main focus is to allow better control of SoC devices, where there are
>> needs to control some intrinsic characteristics of parts of the devices,
>> complementing the V4L2 spec.
> 
> Some consumer devices (ivtv for instance) will also benefit from the media 
> controller, the API is not specific to SoC devices only.

I'm still in doubt about adding support for it at not so complex hardware devices.
Yes, there are some consumer devices that can benefit on it, like cx25821, where
it offers to 16 input/output video streams that can be grouped and changed from
input into output (as far as I understood). So, for such devices, yes, it makes
sense to use. But a simpler devices that has just one or two stream inputs, alsa
and IR, I don't see much sense on using it.

>> This means that it is needed to add some comments at the kernelspace API
>> doc, saying that the drivers implementing the media controller API are
>> required to work properly even when userspace is not using the media
>> controller API;
> 
> That's another issue. Drivers should make a best effort to allow pure V4L2 
> applications to work with a subset of the video device nodes, but they will 
> only offer a subset of the hardware capabilities.

Ok, but this subset is enough for 99.9% of the cases on non-embedded hardware.

> For SoC devices it's even 
> worse, it might be way too difficult to implement support for pure V4L2 
> applications in the kernel driver(s). In that case a device-specific libv4l 
> plugin will configure the driver using the media controller API for pure V4L2 
> applications.

SoC devices used on embedded hardware will have different requirements. I'm ok on
having an ancillary open source application to help to set the device, as we've
discussed at Helsinki's meeting, but this should be an exception, handled case by 
case, and not the rule.

>> This also means that it is needed to add some comments at the userspace API
>> doc, saying that userspace applications should not assume that media
>> drivers will implement the media controller API.
> 
> Agreed. Many V4L2 drivers will not implement the media controller API.
> 
>> So, userspace applications implementing the media controller and V4L2 API's
>> are required to work properly if the device doesn't present a media
>> controler API interface.
> 
> Applications can require support for the media controller API, but they should 
> only do so for specific cases (for instance applications tied to specific SoC 
> hardware, or graphical user interfaces on top of the media controller API 
> similar to qv4l2).

Yes.

>> It should also say that no driver should just implement the media controller
>> API.
> 
> I haven't thought about that, as it would be pretty useless :-)

Well, it doesn't hurt to write it at the spec ;) I won't doubt that people might
try to merge bad things, abusing on the api, and later saying that they have an
"open source driver" that it is not open at all. We've seen cases like that in
the past, where one "open" webcam driver used to work only with an specific
closed-source userspace application [1]. We need to take extra care with any 
API that might be used to control devices in raw mode, as we'll never allow
any driver to abuse on it to deny open source applications to use it.

Cheers,
Mauro

[1] http://marc.info/?l=linux-kernel&m=109356393610149&w=2

Re: [RFC/PATCH v4 00/11] Media controller (core and V4L2)

[Laurent Pinchart]

Hi Mauro,

On Tuesday 14 September 2010 15:34:26 Mauro Carvalho Chehab wrote:
> Em 14-09-2010 09:25, Laurent Pinchart escreveu:
> > On Thursday 09 September 2010 03:44:15 Mauro Carvalho Chehab wrote:
> >> Em 20-08-2010 12:29, Laurent Pinchart escreveu:
> >>> Hi everybody,
> >>> 
> >>> Here's the fourth version of the media controller patches. All comments
> >>> received so far have hopefully been incorporated.
> >>> 
> >>> Compared to the previous version, the patches have been rebased on top
> >>> of 2.6.35 and a MEDIA_IOC_DEVICE_INFO ioctl has been added.
> >>> 
> >>> I won't submit a rebased version of the V4L2 API additions and OMAP3
> >>> ISP patches right now. I will first clean up (and document) the V4L2
> >>> API additions patches, and I will submit them as a proper RFC instead
> >>> of sample code.
> >> 
> >> Hi Laurent,
> >> 
> >> Sorry for a late review on the media controller API. I got flooded by
> >> patches and other work since the merge window.
> > 
> > No worries. I was on holidays last week anyway.
> > 
> >> Anyway, just finished my review, and sent a per-patch comment for most
> >> patches.
> > 
> > Thanks.
> > 
> >> One general comment about it: the userspace API should be documented via
> >> DocBook, to be consistent with V4L2 and DVB API specs.
> > 
> > I feared so :-) I'll work on it.
> > 
> >> It should also be clear at the API specs there that not all media
> >> drivers will implement the media controller API,
> > 
> > I agree.
> > 
> >> as its main focus is to allow better control of SoC devices, where there
> >> are needs to control some intrinsic characteristics of parts of the
> >> devices, complementing the V4L2 spec.
> > 
> > Some consumer devices (ivtv for instance) will also benefit from the
> > media controller, the API is not specific to SoC devices only.
> 
> I'm still in doubt about adding support for it at not so complex hardware
> devices. Yes, there are some consumer devices that can benefit on it, like
> cx25821, where it offers to 16 input/output video streams that can be
> grouped and changed from input into output (as far as I understood). So,
> for such devices, yes, it makes sense to use. But a simpler devices that
> has just one or two stream inputs, alsa and IR, I don't see much sense on
> using it.

The media controller API is also used to find out which audio, dvb, fb, ir and 
v4l devices belong together. Applications such as mythtv are interested in 
that.

> >> This means that it is needed to add some comments at the kernelspace API
> >> doc, saying that the drivers implementing the media controller API are
> >> required to work properly even when userspace is not using the media
> >> controller API;
> > 
> > That's another issue. Drivers should make a best effort to allow pure
> > V4L2 applications to work with a subset of the video device nodes, but
> > they will only offer a subset of the hardware capabilities.
> 
> Ok, but this subset is enough for 99.9% of the cases on non-embedded
> hardware.

Yes, I was thinking about embedded hardware.
 
> > For SoC devices it's even worse, it might be way too difficult to
> > implement support for pure V4L2 applications in the kernel driver(s). In
> > that case a device-specific libv4l plugin will configure the driver using
> > the media controller API for pure V4L2 applications.
> 
> SoC devices used on embedded hardware will have different requirements. I'm
> ok on having an ancillary open source application to help to set the
> device, as we've discussed at Helsinki's meeting, but this should be an
> exception, handled case by case, and not the rule.
> 
> >> This also means that it is needed to add some comments at the userspace
> >> API doc, saying that userspace applications should not assume that
> >> media drivers will implement the media controller API.
> > 
> > Agreed. Many V4L2 drivers will not implement the media controller API.
> > 
> >> So, userspace applications implementing the media controller and V4L2
> >> API's are required to work properly if the device doesn't present a
> >> media controler API interface.
> > 
> > Applications can require support for the media controller API, but they
> > should only do so for specific cases (for instance applications tied to
> > specific SoC hardware, or graphical user interfaces on top of the media
> > controller API similar to qv4l2).
> 
> Yes.
> 
> >> It should also say that no driver should just implement the media
> >> controller API.
> > 
> > I haven't thought about that, as it would be pretty useless :-)
> 
> Well, it doesn't hurt to write it at the spec ;) I won't doubt that people
> might try to merge bad things, abusing on the api, and later saying that
> they have an "open source driver" that it is not open at all. We've seen
> cases like that in the past, where one "open" webcam driver used to work
> only with an specific closed-source userspace application [1]. We need to
> take extra care with any API that might be used to control devices in raw
> mode, as we'll never allow any driver to abuse on it to deny open source
> applications to use it.
> 
> [1] http://marc.info/?l=linux-kernel&m=109356393610149&w=2

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v4 00/11] Media controller (core and V4L2)

[Laurent Pinchart]

Hi Hans,

On Tuesday 14 September 2010 15:24:41 Hans Verkuil wrote:
> > On Thursday 09 September 2010 03:44:15 Mauro Carvalho Chehab wrote:
> >
> >> It should also say that no driver should just implement the media
> >> controller API.
> > 
> > I haven't thought about that, as it would be pretty useless :-)
> 
> I actually think that it should be possible without too much effort to
> make the media API available automatically for those drivers that do not
> implement it themselves. For the standard drivers it basically just has to
> enumerate what is already known.
> 
> It would help a lot with apps like MythTV that want to find related
> devices (e.g. audio/video/vbi).

I think Mauro meant that no driver should implement the media controller API 
without implementing any other standard API (ALSA, DVB, FB, IR, V4L, ...).

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v4 03/11] media: Entities, pads and links

[Laurent Pinchart]

Hi Mauro,

On Thursday 09 September 2010 02:41:54 Mauro Carvalho Chehab wrote:
> Em 20-08-2010 12:29, Laurent Pinchart escreveu:

[snip]

> > diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> > index 781c641..eeb002e 100644
> > --- a/drivers/media/media-device.c
> > +++ b/drivers/media/media-device.c

[snip]

> > +/**
> > + * media_device_register_entity - Register an entity with a media device
> > + * @mdev:	The media device
> > + * @entity:	The entity
> > + */
> > +int __must_check media_device_register_entity(struct media_device *mdev,
> > +					      struct media_entity *entity)
> > +{
> > +	/* Warn if we apparently re-register an entity */
> > +	WARN_ON(entity->parent != NULL);
> 
> Instead, it should just return -EINVAL and use __must_check.
> 
> What's the sense of allowing registering it twice?

It's a driver bug. Entities must not be registered twice. The WARN_ON is meant 
as a way to catch driver bugs. I can replace it with a BUG_ON.

[snip]

> > +EXPORT_SYMBOL(media_entity_init);
> 
> Just use EXPORT_SYMBOL_GPL() for all symbols that needs to be exported.

OK.

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v4 04/11] media: Entity graph traversal

[Laurent Pinchart]

Hi Mauro,

On Thursday 09 September 2010 02:46:45 Mauro Carvalho Chehab wrote:
> Em 20-08-2010 12:29, Laurent Pinchart escreveu:
> > From: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
> > 
> > Add media entity graph traversal. The traversal follows active links by
> > depth first. Traversing graph backwards is prevented by comparing the
> > next possible entity in the graph with the previous one. Multiply
> > connected graphs are thus not supported.
> > 
> > Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > Signed-off-by: Vimarsh Zutshi <vimarsh.zutshi@nokia.com>
> > ---
> > 
> >  Documentation/media-framework.txt |   40 +++++++++++++
> >  drivers/media/media-entity.c      |  115
> >  +++++++++++++++++++++++++++++++++++++ include/media/media-entity.h     
> >  |   15 +++++
> >  3 files changed, 170 insertions(+), 0 deletions(-)
> > 
> > diff --git a/Documentation/media-framework.txt
> > b/Documentation/media-framework.txt index 35d74e4..a599824 100644
> > --- a/Documentation/media-framework.txt
> > +++ b/Documentation/media-framework.txt
> > @@ -238,3 +238,43 @@ Links have flags that describe the link capabilities
> > and state.
> > 
> >  	MEDIA_LINK_FLAG_ACTIVE must also be set since an immutable link is
> >  	always active.
> > 
> > +
> > +Graph traversal
> > +---------------
> > +
> > +The media framework provides APIs to iterate over entities in a graph.
> > +
> > +To iterate over all entities belonging to a media device, drivers can
> > use the +media_device_for_each_entity macro, defined in
> > include/media/media-device.h. +
> > +	struct media_entity *entity;
> > +
> > +	media_device_for_each_entity(entity, mdev) {
> > +		/* entity will point to each entity in turn */
> > +		...
> > +	}
> > +
> > +Drivers might also need to iterate over all entities in a graph that can
> > be +reached only through active links starting at a given entity. The
> > media +framework provides a depth-first graph traversal API for that
> > purpose. +
> > +Note that graphs with cycles (whether directed or undirected) are *NOT*
> > +supported by the graph traversal API.
> 
> Please document that a maximum depth exists to prevent loops, currently
> defined as 16 (MEDIA_ENTITY_ENUM_MAX_DEPTH).

Good idea, will do.

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v4 05/11] media: Reference count and power handling

[Laurent Pinchart]

Hi,

On Saturday 11 September 2010 22:38:33 Sakari Ailus wrote:
> Mauro Carvalho Chehab wrote:
> > Em 20-08-2010 12:29, Laurent Pinchart escreveu:
> >> From: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
> >> 
> >> Basically these are the interface functions:
> >> 
> >> media_entity_get() - acquire entity
> >> media_entity_put() - release entity
> >> 
> >> 	If the entity is of node type, the power change is distributed to
> >> 	all connected entities. For non-nodes it only affects that very
> >> 	node. A mutex is used to serialise access to the entity graph.
> >> 
> >> In the background there's a depth-first search algorithm that traverses
> >> the active links in the graph. All these functions parse the graph to
> >> implement whatever they're to do.
> >> 
> >> The module counters are increased/decreased in media_entity_get/put to
> >> prevent module unloading when an entity is referenced.
> >> 
> >> Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
> >> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> >> Signed-off-by: Stanimir Varbanov <svarbanov@mm-sol.com>
> >> ---
> >> 
> >>  Documentation/media-framework.txt |   37 +++++++++
> >>  drivers/media/media-device.c      |    1 +
> >>  drivers/media/media-entity.c      |  146
> >>  +++++++++++++++++++++++++++++++++++++ include/media/media-device.h    
> >>   |    4 +
> >>  include/media/media-entity.h      |   15 ++++
> >>  5 files changed, 203 insertions(+), 0 deletions(-)
> >> 
> >> diff --git a/Documentation/media-framework.txt
> >> b/Documentation/media-framework.txt index a599824..59649e9 100644
> >> --- a/Documentation/media-framework.txt
> >> +++ b/Documentation/media-framework.txt
> >> @@ -278,3 +278,40 @@ When the graph traversal is complete the function
> >> will return NULL.
> >> 
> >>  Graph traversal can be interrupted at any moment. No cleanup function
> >>  call is required and the graph structure can be freed normally.
> >> 
> >> +
> >> +Reference counting and power handling
> >> +-------------------------------------
> >> +
> >> +Before accessing type-specific entities operations (such as the V4L2
> >> +sub-device operations), drivers must acquire a reference to the entity.
> >> This +ensures that the entity will be powered on and ready to accept
> >> requests. +Similarly, after being done with an entity, drivers must
> >> release the +reference.
> >> +
> >> +	media_entity_get(struct media_entity *entity)
> >> +
> >> +The function will increase the entity reference count. If the entity is
> >> a node +(MEDIA_ENTITY_TYPE_NODE type), the reference count of all
> >> entities it is +connected to, both directly or indirectly, through
> >> active links is increased. +This ensures that the whole media pipeline
> >> will be ready to process +
> >> +Acquiring a reference to an entity increases the media device module
> >> reference +count to prevent module unloading when an entity is being
> >> used. +
> >> +media_entity_get will return a pointer to the entity if successful, or
> >> NULL +otherwise.
> >> +
> >> +	media_entity_put(struct media_entity *entity)
> >> +
> >> +The function will decrease the entity reference count and, for node
> >> entities, +like media_entity_get, the reference count of all connected
> >> entities. Calling +media_entity_put with a NULL argument is valid and
> >> will return immediately. +
> >> +When the first reference to an entity is acquired, or the last
> >> reference +released, the entity's set_power operation is called. Entity
> >> drivers must +implement the operation if they need to perform any power
> >> management task, +such as turning powers or clocks on or off. If no
> >> power management is +required, drivers don't need to provide a
> >> set_power operation. The operation +is allowed to fail when turning
> >> power on, in which case the media_entity_get +function will return
> >> NULL.
> > 
> > The idea of doing power management via media entity get/put doesn't seem
> > right. The mediabus interface and its usage should be optional, and only
> > specialized applications will likely implement it. If a refcount 0 means
> > power off, it ends that a device implementing the media bus will not
> > work with V4L2 applications.
> 
> The Media controller does handle the power through reference count but
> this does not limit to subdev entities. The reference count is also
> applied recursively to all entities which are connected through active
> links.
> 
> There are two cases:
> 
> 1. The user application opens a subdev node. The subdev entity use count
> will be incremented and the subdev will be powered up.
> 
> 2. The user application opens a video node. The reference count for all
> entities connected to the video node entity through active links will be
> incremented. Subdevs will be powered up as well (if they are not already
> because of (1) above). The same works if the entities connected through
> a video node are connected to another entity and the link to that entity
> is activated. In this case the use_counts of the entity sets are applied
> across the both sets.
> 
> The user application does not need to use the Media controller interface
> to get this functionality.

That's correct. The subdev s_power operation is still there and can be called 
directly by non-MC bridge drivers as required.

> Another thing is that the user likely wants to use the device through
> libv4l most likely, at least in the case of OMAP 3 ISP case. The link
> configuration can be made by libv4l so that the regular V4L2
> applications will work as expected.
> 
> >> +
> >> diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
> >> index eeb002e..c309d3c 100644
> >> --- a/drivers/media/media-device.c
> >> +++ b/drivers/media/media-device.c
> >> @@ -71,6 +71,7 @@ int __must_check media_device_register(struct
> >> media_device *mdev)
> >> 
> >>  	mdev->entity_id = 1;
> >>  	INIT_LIST_HEAD(&mdev->entities);
> >>  	spin_lock_init(&mdev->lock);
> >> 
> >> +	mutex_init(&mdev->graph_mutex);
> >> 
> >>  	/* Register the device node. */
> >>  	mdev->devnode.fops = &media_device_fops;
> >> 
> >> diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
> >> index c277c18..da4fef6 100644
> >> --- a/drivers/media/media-entity.c
> >> +++ b/drivers/media/media-entity.c
> >> @@ -21,6 +21,7 @@
> >> 
> >>  #include <linux/module.h>
> >>  #include <linux/slab.h>
> >>  #include <media/media-entity.h>
> >> 
> >> +#include <media/media-device.h>
> >> 
> >>  /**
> >>  
> >>   * media_entity_init - Initialize a media entity
> >> 
> >> @@ -194,6 +195,151 @@ media_entity_graph_walk_next(struct
> >> media_entity_graph *graph)
> >> 
> >>  EXPORT_SYMBOL_GPL(media_entity_graph_walk_next);
> >>  
> >>  /*
> >>  ----------------------------------------------------------------------
> >>  -------
> >> 
> >> + * Power state handling
> >> + */
> >> +
> >> +/* Apply use count to an entity. */
> >> +static void media_entity_use_apply_one(struct media_entity *entity, int
> >> change) +{
> >> +	entity->use_count += change;
> >> +	WARN_ON(entity->use_count < 0);
> > 
> > Instead of producing a warning, just deny it to have usage bellow zero.
> > As this will be called from userspace, the entire interface should be
> > reliable enough to avoid dumb applications to miss-use it.
> 
> This WARN_ON() always indicates a driver (or MC) bug. The entity
> use_count should never be under 0, thus the warning.
> 
> The calls to this function should be always related to an open file
> handle in a way or another. There is no direct user influence over this.
> 
> > Also: what happens if an userspace application dies or suffer any
> > troubles? You need to reset all use_count's at release() callback.
> 
> Yes, this is true. media_entity_{get,put} should always be called when
> file handles are open()ed or release()d.

media_entity_{get,put} are already called on open() and release(). There's not 
explicit call to media_entity_{get,put} from userspace.

> I guess Laurent will correct me if required. :-)

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v4 11/11] v4l: Make v4l2_subdev inherit from media_entity

[Laurent Pinchart]

Hi Mauro,

On Thursday 09 September 2010 03:25:38 Mauro Carvalho Chehab wrote:
> Em 20-08-2010 12:29, Laurent Pinchart escreveu:
> > V4L2 subdevices are media entities. As such they need to inherit from
> > (include) the media_entity structure.
> > 
> > When registering/unregistering the subdevice, the media entity is
> > automatically registered/unregistered. The entity is acquired on device
> > open and released on device close.
> > 
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
> > ---
> > 
> >  Documentation/video4linux/v4l2-framework.txt |   23 ++++++++++++++++++
> >  drivers/media/video/v4l2-device.c            |   32
> >  +++++++++++++++++++++----- drivers/media/video/v4l2-subdev.c           
> >  |   27 +++++++++++++++++++++- include/media/v4l2-subdev.h              
> >     |    7 +++++
> >  4 files changed, 82 insertions(+), 7 deletions(-)
> > 
> > diff --git a/Documentation/video4linux/v4l2-framework.txt
> > b/Documentation/video4linux/v4l2-framework.txt index 7ff4016..3416d93
> > 100644
> > --- a/Documentation/video4linux/v4l2-framework.txt
> > +++ b/Documentation/video4linux/v4l2-framework.txt
> > 
> > @@ -263,6 +263,26 @@ A sub-device driver initializes the v4l2_subdev 
struct using:
> >  Afterwards you need to initialize subdev->name with a unique name and
> >  set the module owner. This is done for you if you use the i2c helper
> >  functions.
> > 
> > +If integration with the media framework is needed, you must initialize
> > the +media_entity struct embedded in the v4l2_subdev struct (entity
> > field) by +calling media_entity_init():
> > +
> > +	struct media_pad *pads = &my_sd->pads;
> > +	int err;
> > +
> > +	err = media_entity_init(&sd->entity, npads, pads, 0);
> > +
> > +The pads array must have been previously initialized. There is no need
> > to +manually set the struct media_entity type and name fields, but the
> > revision +field must be initialized if needed.
> > +
> > +A reference to the entity will be automatically acquired/released when
> > the +subdev device node (if any) is opened/closed.
> > +
> > +Don't forget to cleanup the media entity before the sub-device is
> > destroyed: +
> > +	media_entity_cleanup(&sd->entity);
> > +
> > 
> >  A device (bridge) driver needs to register the v4l2_subdev with the
> > 
> >  v4l2_device:
> > @@ -272,6 +292,9 @@ This can fail if the subdev module disappeared before
> > it could be registered.
> > 
> >  After this function was called successfully the subdev->dev field points
> >  to the v4l2_device.
> > 
> > +If the v4l2_device parent device has a non-NULL mdev field, the
> > sub-device +entity will be automatically registered with the media
> > device.
> > +
> > 
> >  You can unregister a sub-device using:
> >  	v4l2_device_unregister_subdev(sd);
> > 
> > diff --git a/drivers/media/video/v4l2-device.c
> > b/drivers/media/video/v4l2-device.c index 91452cd..4f74d01 100644
> > --- a/drivers/media/video/v4l2-device.c
> > +++ b/drivers/media/video/v4l2-device.c
> > @@ -114,10 +114,11 @@ void v4l2_device_unregister(struct v4l2_device
> > *v4l2_dev)
> > 
> >  EXPORT_SYMBOL_GPL(v4l2_device_unregister);
> >  
> >  int v4l2_device_register_subdev(struct v4l2_device *v4l2_dev,
> > 
> > -						struct v4l2_subdev *sd)
> > +				struct v4l2_subdev *sd)
> > 
> >  {
> > 
> > +	struct media_entity *entity = &sd->entity;
> > 
> >  	struct video_device *vdev;
> > 
> > -	int ret = 0;
> > +	int ret;
> > 
> >  	/* Check for valid input */
> >  	if (v4l2_dev == NULL || sd == NULL || !sd->name[0])
> > 
> > @@ -129,6 +130,15 @@ int v4l2_device_register_subdev(struct v4l2_device
> > *v4l2_dev,
> > 
> >  	if (!try_module_get(sd->owner))
> >  	
> >  		return -ENODEV;
> > 
> > +	/* Register the entity. */
> > +	if (v4l2_dev->mdev) {
> > +		ret = media_device_register_entity(v4l2_dev->mdev, entity);
> > +		if (ret < 0) {
> > +			module_put(sd->owner);
> > +			return ret;
> > +		}
> > +	}
> > +
> > 
> >  	sd->v4l2_dev = v4l2_dev;
> >  	spin_lock(&v4l2_dev->lock);
> >  	list_add_tail(&sd->list, &v4l2_dev->subdevs);
> > 
> > @@ -143,26 +153,36 @@ int v4l2_device_register_subdev(struct v4l2_device
> > *v4l2_dev,
> > 
> >  	if (sd->flags & V4L2_SUBDEV_FL_HAS_DEVNODE) {
> >  	
> >  		ret = __video_register_device(vdev, VFL_TYPE_SUBDEV, -1, 1,
> >  		
> >  					      sd->owner);
> > 
> > -		if (ret < 0)
> > +		if (ret < 0) {
> > 
> >  			v4l2_device_unregister_subdev(sd);
> > 
> > +			return ret;
> > +		}
> > 
> >  	}
> > 
> > -	return ret;
> > +	entity->v4l.major = VIDEO_MAJOR;
> > +	entity->v4l.minor = vdev->minor;
> 
> Hmm... it needs to check if entity is not null.

Entity is set to

struct media_entity *entity = &sd->entity;

It can't be NULL.

> > +	return 0;
> > 
> >  }
> >  EXPORT_SYMBOL_GPL(v4l2_device_register_subdev);
> >  
> >  void v4l2_device_unregister_subdev(struct v4l2_subdev *sd)
> >  {
> > 
> > +	struct v4l2_device *v4l2_dev;
> > +
> > 
> >  	/* return if it isn't registered */
> >  	if (sd == NULL || sd->v4l2_dev == NULL)
> >  	
> >  		return;
> > 
> > -	spin_lock(&sd->v4l2_dev->lock);
> > +	v4l2_dev = sd->v4l2_dev;
> > +
> > +	spin_lock(&v4l2_dev->lock);
> > 
> >  	list_del(&sd->list);
> > 
> > -	spin_unlock(&sd->v4l2_dev->lock);
> > +	spin_unlock(&v4l2_dev->lock);
> > 
> >  	sd->v4l2_dev = NULL;
> >  	
> >  	module_put(sd->owner);
> > 
> > +	if (v4l2_dev->mdev)
> > +		media_device_unregister_entity(&sd->entity);
> > 
> >  	video_unregister_device(&sd->devnode);
> >  
> >  }
> >  EXPORT_SYMBOL_GPL(v4l2_device_unregister_subdev);
> > 
> > diff --git a/drivers/media/video/v4l2-subdev.c
> > b/drivers/media/video/v4l2-subdev.c index b063195..1efa267 100644
> > --- a/drivers/media/video/v4l2-subdev.c
> > +++ b/drivers/media/video/v4l2-subdev.c
> > @@ -32,7 +32,8 @@ static int subdev_open(struct file *file)
> > 
> >  {
> >  
> >  	struct video_device *vdev = video_devdata(file);
> >  	struct v4l2_subdev *sd = vdev_to_v4l2_subdev(vdev);
> > 
> > -	struct v4l2_fh *vfh;
> > +	struct media_entity *entity;
> > +	struct v4l2_fh *vfh = NULL;
> > 
> >  	int ret;
> >  	
> >  	if (!sd->initialized)
> > 
> > @@ -59,10 +60,17 @@ static int subdev_open(struct file *file)
> > 
> >  		file->private_data = vfh;
> >  	
> >  	}
> > 
> > +	entity = media_entity_get(&sd->entity);
> > +	if (!entity) {
> > +		ret = -EBUSY;
> > +		goto err;
> > +	}
> > +
> 
> It needs to check if v4l2_dev->mdev is not null, as it makes no sense to
> get an entity if the driver is not using the media entity.

OK.

> >  	return 0;
> >  
> >  err:
> >  	if (vfh != NULL) {
> > 
> > +		v4l2_fh_del(vfh);
> > 
> >  		v4l2_fh_exit(vfh);
> >  		kfree(vfh);
> >  	
> >  	}
> > 
> > @@ -72,8 +80,12 @@ err:
> >  static int subdev_close(struct file *file)
> >  {
> > 
> > +	struct video_device *vdev = video_devdata(file);
> > +	struct v4l2_subdev *sd = vdev_to_v4l2_subdev(vdev);
> > 
> >  	struct v4l2_fh *vfh = file->private_data;
> > 
> > +	media_entity_put(&sd->entity);
> 
> Same here: don't put it, if v4l2_dev->mdev = NULL (I think that you're
> already checking for it at media_entity_put).
> 
> > +
> > 
> >  	if (vfh != NULL) {
> >  	
> >  		v4l2_fh_del(vfh);
> >  		v4l2_fh_exit(vfh);
> > 
> > @@ -172,5 +184,18 @@ void v4l2_subdev_init(struct v4l2_subdev *sd, const
> > struct v4l2_subdev_ops *ops)
> > 
> >  	sd->grp_id = 0;
> >  	sd->priv = NULL;
> >  	sd->initialized = 1;
> > 
> > +	sd->entity.name = sd->name;
> > +	sd->entity.type = MEDIA_ENTITY_TYPE_SUBDEV;
> > 
> >  }
> >  EXPORT_SYMBOL(v4l2_subdev_init);
> > 
> > +
> > +int v4l2_subdev_set_power(struct media_entity *entity, int power)
> > +{
> > +	struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
> > +
> > +	dev_dbg(entity->parent->dev,
> > +		"%s power%s\n", entity->name, power ? "on" : "off");
> > +
> > +	return v4l2_subdev_call(sd, core, s_power, power);
> > +}
> > +EXPORT_SYMBOL_GPL(v4l2_subdev_set_power);
> 
> Also, should do nothing if media entity is null.
> 
> PS.: Not sure where this function is called, as the caller is not on this
> patch series. The better would be to add this together with the patch
> calling v4l2_subdev_set_power().

v4l2_subdev_set_power() is a helper meant to be assigned to struct 
media_entity_operations::set_power in subdev drivers.

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v4 08/11] media: Links setup

[Laurent Pinchart]

Hi Hans,

On Monday 06 September 2010 19:09:20 Hans Verkuil wrote:
> On Wednesday, September 01, 2010 16:08:29 Laurent Pinchart wrote:
> > On Saturday 28 August 2010 13:14:18 Hans Verkuil wrote:
> > > On Friday, August 20, 2010 17:29:10 Laurent Pinchart wrote:
> > [snip]
> > 
> > > > +/**
> > > > + * media_entity_remote_pad - Locate the pad at the remote end of a
> > > > link + * @entity: Local entity
> > > > + * @pad: Pad at the local end of the link
> > > > + *
> > > > + * Search for a remote pad connected to the given pad by iterating
> > > > over all
> > > > + * links originating or terminating at that pad until an active link
> > > > is found.
> > > > + *
> > > > + * Return a pointer to the pad at the remote end of the first found
> > > > active link,
> > > > + * or NULL if no active link has been found.
> > > > + */
> > > > +struct media_pad *media_entity_remote_pad(struct media_pad *pad)
> > > > +{
> > > > +	unsigned int i;
> > > > +
> > > > +	for (i = 0; i < pad->entity->num_links; i++) {
> > > > +		struct media_link *link = &pad->entity->links[i];
> > > > +
> > > > +		if (!(link->flags & MEDIA_LINK_FLAG_ACTIVE))
> > > > +			continue;
> > > > +
> > > > +		if (link->source == pad)
> > > > +			return link->sink;
> > > > +
> > > > +		if (link->sink == pad)
> > > > +			return link->source;
> > > > +	}
> > > > +
> > > > +	return NULL;
> > > > +
> > > > +}
> > > 
> > > Why is this needed? Esp. since there can be multiple active remote pads
> > > if you have multiple active outgoing links. Something this function
> > > doesn't deal with.
> > 
> > The function is meant to be used when only one of the links can be
> > active. It's most useful to find the entity connected to a given input
> > pad, as input pads can't be connected by multiple simultaneously active
> > links.
> 
> In that case I would rename it media_entity_remote_source to match the use
> case (and only look for remote sources).

OK, I'll rename the function.

> Alternatively you could add an index for the nth active link, then it would
> be truly generic.
> 
> Right now the function either does too much or too little :-)

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v4 08/11] media: Links setup

[Laurent Pinchart]

Hi Mauro,

On Thursday 09 September 2010 03:14:44 Mauro Carvalho Chehab wrote:
> Em 20-08-2010 12:29, Laurent Pinchart escreveu:
> > Create the following ioctl and implement it at the media device level to
> > setup links.
> > 
> > - MEDIA_IOC_SETUP_LINK: Modify the properties of a given link
> > 
> > The only property that can currently be modified is the ACTIVE link flag
> > to activate/deactivate a link. Links marked with the IMMUTABLE link flag
> > can not be activated or deactivated.
> > 
> > Activating and deactivating a link has effects on entities' use count.
> > Those changes are automatically propagated through the graph.
> 
> You need to address here the release() call: if the userspace application
> dies or just exits, the device should be set into a sane state, e. g.
> devices powered on should be turned off,

That's already handled, as media_entity_put() is called in the vdev and subdev 
release() functions.

> and links activated by the application should be de-activated.

I don't think that's required. When an application exits with a video device 
node open, we don't reset all controls and formats. Power needs to be turned 
off and resources need to be released on exit, but configuration doesn't need 
to be reset.

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v4 07/11] media: Entities, pads and links enumeration

[Laurent Pinchart]

Hi Hans,

On Monday 06 September 2010 18:51:59 Hans Verkuil wrote:
> On Wednesday, September 01, 2010 16:05:10 Laurent Pinchart wrote:
> > On Saturday 28 August 2010 13:02:22 Hans Verkuil wrote:
> > > On Friday, August 20, 2010 17:29:09 Laurent Pinchart wrote:
> > [snip]
> > 
> > > > diff --git a/Documentation/media-framework.txt
> > > > b/Documentation/media-framework.txt index 66f7f6c..74a137d 100644
> > > > --- a/Documentation/media-framework.txt
> > > > +++ b/Documentation/media-framework.txt
> > 
> > [snip]
> > 
> > > > +The media_entity_desc structure is defined as
> > > > +
> > > > +- struct media_entity_desc
> > > > +
> > > > +__u32	id		Entity id, set by the application. When the id is
> > > > +			or'ed with MEDIA_ENTITY_ID_FLAG_NEXT, the driver
> > > > +			clears the flag and returns the first entity with a
> > > > +			larger id.
> > > > +char	name[32]	Entity name. UTF-8 NULL-terminated string.
> > > 
> > > Why UTF-8 instead of ASCII?
> > 
> > Because vendor-specific names could include non-ASCII characters (same
> > reason for the model name in the media_device structure, if we decice to
> > make the model name ASCII I'll make the entity name ASCII as well).
> > 
> > [snip]
> > 
> > > > +struct media_entity_desc {
> > > > +	__u32 id;
> > > > +	char name[32];
> > > > +	__u32 type;
> > > > +	__u32 revision;
> > > > +	__u32 flags;
> > > > +	__u32 group_id;
> > > > +	__u16 pads;
> > > > +	__u16 links;
> > > > +
> > > > +	__u32 reserved[4];
> > > > +
> > > > +	union {
> > > > +		/* Node specifications */
> > > > +		struct {
> > > > +			__u32 major;
> > > > +			__u32 minor;
> > > > +		} v4l;
> > > > +		struct {
> > > > +			__u32 major;
> > > > +			__u32 minor;
> > > > +		} fb;
> > > > +		int alsa;
> > > > +		int dvb;
> > > > +
> > > > +		/* Sub-device specifications */
> > > > +		/* Nothing needed yet */
> > > > +		__u8 raw[64];
> > > > +	};
> > > > +};
> > > 
> > > Should this be a packed struct?
> > 
> > Why ? :-) Packed struct are most useful when they need to match hardware
> > structures or network protocols. Packing a structure can generate
> > unaligned fields, which are bad performance-wise.
> 
> I'm thinking about preventing a compat32 mess as we have for v4l.
> 
> It is my understanding that the only way to prevent different struct sizes
> between 32 and 64 bit is to use packed.

I don't think that's correct. Different struct sizes between 32bit and 64bit 
are caused by variable-size fields, such as 'long' (32bit on 32bit 
architectures, 64bit on 64bit architectures). I might be wrong though.

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v4 05/11] media: Reference count and power handling

[Mauro Carvalho Chehab]

Em 16-09-2010 05:46, Laurent Pinchart escreveu:
> Hi,
> 
> On Saturday 11 September 2010 22:38:33 Sakari Ailus wrote:
>> Mauro Carvalho Chehab wrote:
>>> Em 20-08-2010 12:29, Laurent Pinchart escreveu:
>>>> From: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
>>>>
>>>> Basically these are the interface functions:
>>>>
>>>> media_entity_get() - acquire entity
>>>> media_entity_put() - release entity
>>>>
>>>> 	If the entity is of node type, the power change is distributed to
>>>> 	all connected entities. For non-nodes it only affects that very
>>>> 	node. A mutex is used to serialise access to the entity graph.
>>>>
>>>> In the background there's a depth-first search algorithm that traverses
>>>> the active links in the graph. All these functions parse the graph to
>>>> implement whatever they're to do.
>>>>
>>>> The module counters are increased/decreased in media_entity_get/put to
>>>> prevent module unloading when an entity is referenced.
>>>>
>>>> Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
>>>> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>>>> Signed-off-by: Stanimir Varbanov <svarbanov@mm-sol.com>
>>>> ---
>>>>
>>>>  Documentation/media-framework.txt |   37 +++++++++
>>>>  drivers/media/media-device.c      |    1 +
>>>>  drivers/media/media-entity.c      |  146
>>>>  +++++++++++++++++++++++++++++++++++++ include/media/media-device.h    
>>>>   |    4 +
>>>>  include/media/media-entity.h      |   15 ++++
>>>>  5 files changed, 203 insertions(+), 0 deletions(-)
>>>>
>>>> diff --git a/Documentation/media-framework.txt
>>>> b/Documentation/media-framework.txt index a599824..59649e9 100644
>>>> --- a/Documentation/media-framework.txt
>>>> +++ b/Documentation/media-framework.txt
>>>> @@ -278,3 +278,40 @@ When the graph traversal is complete the function
>>>> will return NULL.
>>>>
>>>>  Graph traversal can be interrupted at any moment. No cleanup function
>>>>  call is required and the graph structure can be freed normally.
>>>>
>>>> +
>>>> +Reference counting and power handling
>>>> +-------------------------------------
>>>> +
>>>> +Before accessing type-specific entities operations (such as the V4L2
>>>> +sub-device operations), drivers must acquire a reference to the entity.
>>>> This +ensures that the entity will be powered on and ready to accept
>>>> requests. +Similarly, after being done with an entity, drivers must
>>>> release the +reference.
>>>> +
>>>> +	media_entity_get(struct media_entity *entity)
>>>> +
>>>> +The function will increase the entity reference count. If the entity is
>>>> a node +(MEDIA_ENTITY_TYPE_NODE type), the reference count of all
>>>> entities it is +connected to, both directly or indirectly, through
>>>> active links is increased. +This ensures that the whole media pipeline
>>>> will be ready to process +
>>>> +Acquiring a reference to an entity increases the media device module
>>>> reference +count to prevent module unloading when an entity is being
>>>> used. +
>>>> +media_entity_get will return a pointer to the entity if successful, or
>>>> NULL +otherwise.
>>>> +
>>>> +	media_entity_put(struct media_entity *entity)
>>>> +
>>>> +The function will decrease the entity reference count and, for node
>>>> entities, +like media_entity_get, the reference count of all connected
>>>> entities. Calling +media_entity_put with a NULL argument is valid and
>>>> will return immediately. +
>>>> +When the first reference to an entity is acquired, or the last
>>>> reference +released, the entity's set_power operation is called. Entity
>>>> drivers must +implement the operation if they need to perform any power
>>>> management task, +such as turning powers or clocks on or off. If no
>>>> power management is +required, drivers don't need to provide a
>>>> set_power operation. The operation +is allowed to fail when turning
>>>> power on, in which case the media_entity_get +function will return
>>>> NULL.
>>>
>>> The idea of doing power management via media entity get/put doesn't seem
>>> right. The mediabus interface and its usage should be optional, and only
>>> specialized applications will likely implement it. If a refcount 0 means
>>> power off, it ends that a device implementing the media bus will not
>>> work with V4L2 applications.
>>
>> The Media controller does handle the power through reference count but
>> this does not limit to subdev entities. The reference count is also
>> applied recursively to all entities which are connected through active
>> links.
>>
>> There are two cases:
>>
>> 1. The user application opens a subdev node. The subdev entity use count
>> will be incremented and the subdev will be powered up.
>>
>> 2. The user application opens a video node. The reference count for all
>> entities connected to the video node entity through active links will be
>> incremented. Subdevs will be powered up as well (if they are not already
>> because of (1) above). The same works if the entities connected through
>> a video node are connected to another entity and the link to that entity
>> is activated. In this case the use_counts of the entity sets are applied
>> across the both sets.
>>
>> The user application does not need to use the Media controller interface
>> to get this functionality.
> 
> That's correct. The subdev s_power operation is still there and can be called 
> directly by non-MC bridge drivers as required.

It is clear that non-MC bridge devices will not be affected by MC.

My concern is about MC bridge drivers. Userspace may not be implementing MC.
Yet, the device needs to fully work. If you're relying at MC to power up parts
of the device, this means that a pure V4L2 application won't work anymore.

I didn't see any patch on this series addressing this case.

>> Another thing is that the user likely wants to use the device through
>> libv4l most likely, at least in the case of OMAP 3 ISP case. The link
>> configuration can be made by libv4l so that the regular V4L2
>> applications will work as expected.
>>
>>>> +
>>>> diff --git a/drivers/media/media-device.c b/drivers/media/media-device.c
>>>> index eeb002e..c309d3c 100644
>>>> --- a/drivers/media/media-device.c
>>>> +++ b/drivers/media/media-device.c
>>>> @@ -71,6 +71,7 @@ int __must_check media_device_register(struct
>>>> media_device *mdev)
>>>>
>>>>  	mdev->entity_id = 1;
>>>>  	INIT_LIST_HEAD(&mdev->entities);
>>>>  	spin_lock_init(&mdev->lock);
>>>>
>>>> +	mutex_init(&mdev->graph_mutex);
>>>>
>>>>  	/* Register the device node. */
>>>>  	mdev->devnode.fops = &media_device_fops;
>>>>
>>>> diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
>>>> index c277c18..da4fef6 100644
>>>> --- a/drivers/media/media-entity.c
>>>> +++ b/drivers/media/media-entity.c
>>>> @@ -21,6 +21,7 @@
>>>>
>>>>  #include <linux/module.h>
>>>>  #include <linux/slab.h>
>>>>  #include <media/media-entity.h>
>>>>
>>>> +#include <media/media-device.h>
>>>>
>>>>  /**
>>>>  
>>>>   * media_entity_init - Initialize a media entity
>>>>
>>>> @@ -194,6 +195,151 @@ media_entity_graph_walk_next(struct
>>>> media_entity_graph *graph)
>>>>
>>>>  EXPORT_SYMBOL_GPL(media_entity_graph_walk_next);
>>>>  
>>>>  /*
>>>>  ----------------------------------------------------------------------
>>>>  -------
>>>>
>>>> + * Power state handling
>>>> + */
>>>> +
>>>> +/* Apply use count to an entity. */
>>>> +static void media_entity_use_apply_one(struct media_entity *entity, int
>>>> change) +{
>>>> +	entity->use_count += change;
>>>> +	WARN_ON(entity->use_count < 0);
>>>
>>> Instead of producing a warning, just deny it to have usage bellow zero.
>>> As this will be called from userspace, the entire interface should be
>>> reliable enough to avoid dumb applications to miss-use it.
>>
>> This WARN_ON() always indicates a driver (or MC) bug. The entity
>> use_count should never be under 0, thus the warning.
>>
>> The calls to this function should be always related to an open file
>> handle in a way or another. There is no direct user influence over this.
>>
>>> Also: what happens if an userspace application dies or suffer any
>>> troubles? You need to reset all use_count's at release() callback.
>>
>> Yes, this is true. media_entity_{get,put} should always be called when
>> file handles are open()ed or release()d.
> 
> media_entity_{get,put} are already called on open() and release(). There's not 
> explicit call to media_entity_{get,put} from userspace.

Ok.

Cheers,
Mauro

Re: [RFC/PATCH v4 05/11] media: Reference count and power handling

[Laurent Pinchart]

Hi Mauro,

On Thursday 16 September 2010 12:35:45 Mauro Carvalho Chehab wrote:
> Em 16-09-2010 05:46, Laurent Pinchart escreveu:
> > On Saturday 11 September 2010 22:38:33 Sakari Ailus wrote:
> >> Mauro Carvalho Chehab wrote:
> >>> Em 20-08-2010 12:29, Laurent Pinchart escreveu:
> >>>> From: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
> >>>> 
> >>>> Basically these are the interface functions:
> >>>> 
> >>>> media_entity_get() - acquire entity
> >>>> media_entity_put() - release entity
> >>>> 
> >>>> 	If the entity is of node type, the power change is distributed to
> >>>> 	all connected entities. For non-nodes it only affects that very
> >>>> 	node. A mutex is used to serialise access to the entity graph.
> >>>> 
> >>>> In the background there's a depth-first search algorithm that
> >>>> traverses the active links in the graph. All these functions parse
> >>>> the graph to implement whatever they're to do.
> >>>> 
> >>>> The module counters are increased/decreased in media_entity_get/put to
> >>>> prevent module unloading when an entity is referenced.
> >>>> 
> >>>> Signed-off-by: Sakari Ailus <sakari.ailus@maxwell.research.nokia.com>
> >>>> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> >>>> Signed-off-by: Stanimir Varbanov <svarbanov@mm-sol.com>
> >>>> ---
> >>>> 
> >>>>  Documentation/media-framework.txt |   37 +++++++++
> >>>>  drivers/media/media-device.c      |    1 +
> >>>>  drivers/media/media-entity.c      |  146
> >>>>  +++++++++++++++++++++++++++++++++++++ include/media/media-device.h
> >>>>  
> >>>>   |    4 +
> >>>>  
> >>>>  include/media/media-entity.h      |   15 ++++
> >>>>  5 files changed, 203 insertions(+), 0 deletions(-)
> >>>> 
> >>>> diff --git a/Documentation/media-framework.txt
> >>>> b/Documentation/media-framework.txt index a599824..59649e9 100644
> >>>> --- a/Documentation/media-framework.txt
> >>>> +++ b/Documentation/media-framework.txt
> >>>> @@ -278,3 +278,40 @@ When the graph traversal is complete the function
> >>>> will return NULL.
> >>>> 
> >>>>  Graph traversal can be interrupted at any moment. No cleanup function
> >>>>  call is required and the graph structure can be freed normally.
> >>>> 
> >>>> +
> >>>> +Reference counting and power handling
> >>>> +-------------------------------------
> >>>> +
> >>>> +Before accessing type-specific entities operations (such as the V4L2
> >>>> +sub-device operations), drivers must acquire a reference to the
> >>>> entity. This +ensures that the entity will be powered on and ready to
> >>>> accept requests. +Similarly, after being done with an entity, drivers
> >>>> must release the +reference.
> >>>> +
> >>>> +	media_entity_get(struct media_entity *entity)
> >>>> +
> >>>> +The function will increase the entity reference count. If the entity
> >>>> is a node +(MEDIA_ENTITY_TYPE_NODE type), the reference count of all
> >>>> entities it is +connected to, both directly or indirectly, through
> >>>> active links is increased. +This ensures that the whole media
> >>>> pipeline will be ready to process +
> >>>> +Acquiring a reference to an entity increases the media device module
> >>>> reference +count to prevent module unloading when an entity is being
> >>>> used. +
> >>>> +media_entity_get will return a pointer to the entity if successful,
> >>>> or NULL +otherwise.
> >>>> +
> >>>> +	media_entity_put(struct media_entity *entity)
> >>>> +
> >>>> +The function will decrease the entity reference count and, for node
> >>>> entities, +like media_entity_get, the reference count of all connected
> >>>> entities. Calling +media_entity_put with a NULL argument is valid and
> >>>> will return immediately. +
> >>>> +When the first reference to an entity is acquired, or the last
> >>>> reference +released, the entity's set_power operation is called.
> >>>> Entity drivers must +implement the operation if they need to perform
> >>>> any power management task, +such as turning powers or clocks on or
> >>>> off. If no power management is +required, drivers don't need to
> >>>> provide a set_power operation. The operation +is allowed to fail when
> >>>> turning power on, in which case the media_entity_get +function will
> >>>> return NULL.
> >>> 
> >>> The idea of doing power management via media entity get/put doesn't
> >>> seem right. The mediabus interface and its usage should be optional,
> >>> and only specialized applications will likely implement it. If a
> >>> refcount 0 means power off, it ends that a device implementing the
> >>> media bus will not work with V4L2 applications.
> >> 
> >> The Media controller does handle the power through reference count but
> >> this does not limit to subdev entities. The reference count is also
> >> applied recursively to all entities which are connected through active
> >> links.
> >> 
> >> There are two cases:
> >> 
> >> 1. The user application opens a subdev node. The subdev entity use count
> >> will be incremented and the subdev will be powered up.
> >> 
> >> 2. The user application opens a video node. The reference count for all
> >> entities connected to the video node entity through active links will be
> >> incremented. Subdevs will be powered up as well (if they are not already
> >> because of (1) above). The same works if the entities connected through
> >> a video node are connected to another entity and the link to that entity
> >> is activated. In this case the use_counts of the entity sets are applied
> >> across the both sets.
> >> 
> >> The user application does not need to use the Media controller interface
> >> to get this functionality.
> > 
> > That's correct. The subdev s_power operation is still there and can be
> > called directly by non-MC bridge drivers as required.
> 
> It is clear that non-MC bridge devices will not be affected by MC.
> 
> My concern is about MC bridge drivers. Userspace may not be implementing
> MC. Yet, the device needs to fully work. If you're relying at MC to power
> up parts of the device, this means that a pure V4L2 application won't work
> anymore.
> 
> I didn't see any patch on this series addressing this case.

When the non-MC userspace application will open a video device node, the 
media_entity_get() call will power up all entities (subdevs in this case) in 
the pipeline, so it will be completely transparent for V4L2-only applications.

-- 
Regards,

Laurent Pinchart

Re: [RFC/PATCH v4 07/11] media: Entities, pads and links enumeration

[Sakari Ailus]

Hi Laurent and Hans,

Laurent Pinchart wrote:
> Hi Hans,
> 
> On Monday 06 September 2010 18:51:59 Hans Verkuil wrote:
>> On Wednesday, September 01, 2010 16:05:10 Laurent Pinchart wrote:
>>> On Saturday 28 August 2010 13:02:22 Hans Verkuil wrote:
>>>> On Friday, August 20, 2010 17:29:09 Laurent Pinchart wrote:

...

>>>>> +};
>>>>
>>>> Should this be a packed struct?
>>>
>>> Why ? :-) Packed struct are most useful when they need to match hardware
>>> structures or network protocols. Packing a structure can generate
>>> unaligned fields, which are bad performance-wise.
>>
>> I'm thinking about preventing a compat32 mess as we have for v4l.
>>
>> It is my understanding that the only way to prevent different struct sizes
>> between 32 and 64 bit is to use packed.
> 
> I don't think that's correct. Different struct sizes between 32bit and 64bit 
> are caused by variable-size fields, such as 'long' (32bit on 32bit 
> architectures, 64bit on 64bit architectures). I might be wrong though.

As far as I understand that's another reason for the structures not
being exactly the same. Alignment of different data types in structures
depends on ABI. I don't know the exact rules for all the architectures
Linux supports if there are cases where the alignment would be different
for 32-bit and 64-bit and smaller than the data type. On ARM there have
been different alignments depending on ABI (EABI vs. GNU ABI) which is
now practically history though.

I couldn't find a better reference than this:

<URL:http://developers.sun.com/solaris/articles/about_amd64_abi.html>

64-bit integers are aligned differently on 32-bit and 64-bit x86 but the
alignment is still smaller or equal to the size of the data type.

I'd also pack them to be sure. The structures should be constructed so
that the alignment is sane even if they are packed. In that case there's
no harm done by packing. It just prevents a failure (32-bit vs. 64-bit)
if there's a problem with the definition.

Just my 2 cents worth. :)

Regards,

-- 
Sakari Ailus
sakari.ailus@maxwell.research.nokia.com

Re: [RFC/PATCH v4 07/11] media: Entities, pads and links enumeration

[Laurent Pinchart]

Hi Sakari,

On Thursday 16 September 2010 17:36:29 Sakari Ailus wrote:
> Laurent Pinchart wrote:
> > On Monday 06 September 2010 18:51:59 Hans Verkuil wrote:
> >> On Wednesday, September 01, 2010 16:05:10 Laurent Pinchart wrote:
> >>> On Saturday 28 August 2010 13:02:22 Hans Verkuil wrote:
> >>>> On Friday, August 20, 2010 17:29:09 Laurent Pinchart wrote:
> ...
> 
> >>>>> +};
> >>>> 
> >>>> Should this be a packed struct?
> >>> 
> >>> Why ? :-) Packed struct are most useful when they need to match
> >>> hardware structures or network protocols. Packing a structure can
> >>> generate unaligned fields, which are bad performance-wise.
> >> 
> >> I'm thinking about preventing a compat32 mess as we have for v4l.
> >> 
> >> It is my understanding that the only way to prevent different struct
> >> sizes between 32 and 64 bit is to use packed.
> > 
> > I don't think that's correct. Different struct sizes between 32bit and
> > 64bit are caused by variable-size fields, such as 'long' (32bit on 32bit
> > architectures, 64bit on 64bit architectures). I might be wrong though.
> 
> As far as I understand that's another reason for the structures not
> being exactly the same. Alignment of different data types in structures
> depends on ABI. I don't know the exact rules for all the architectures
> Linux supports if there are cases where the alignment would be different
> for 32-bit and 64-bit and smaller than the data type. On ARM there have
> been different alignments depending on ABI (EABI vs. GNU ABI) which is
> now practically history though.
> 
> I couldn't find a better reference than this:
> 
> <URL:http://developers.sun.com/solaris/articles/about_amd64_abi.html>
> 
> 64-bit integers are aligned differently on 32-bit and 64-bit x86 but the
> alignment is still smaller or equal to the size of the data type.

Thanks for the link.

> I'd also pack them to be sure. The structures should be constructed so
> that the alignment is sane even if they are packed. In that case there's
> no harm done by packing. It just prevents a failure (32-bit vs. 64-bit)
> if there's a problem with the definition.

Even if the structures were packed we would have a problem with 
media_user_links. The pads and links pointers will be 4-bytes long on 32-bit 
and 8-bytes long on 64-bit. There's no way around that. I could pad the 
structure to a fixed size with

struct media_links_enum {
        __u32 entity;
        /* Should have enough room for pads elements */
        struct media_pad_desc __user *pads;
        __u8 __pad1[8 - sizeof(void *)];
        /* Should have enough room for links elements */
        struct media_link_desc __user *links;
        __u8 __pad2[8 - sizeof(void *)];
        __u32 reserved[4];
};

or with

struct media_links_enum {
        __u32 entity;
        union {
                struct {
                        /* Should have enough room for pads elements */
                        struct media_pad_desc __user *pads;
                        /* Should have enough room for links elements */
                        struct media_link_desc __user *links;
                };
                __u32 __pad[4];
        };
        __u32 reserved[4];
};

Is there any other way ?

-- 
Regards,

Laurent Pinchart