* [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support
@ 2023-11-09 21:02 Umang Jain
2023-11-09 21:02 ` [PATCH v2 01/15] staging: vc04_services: Add new vc-sm-cma driver Umang Jain
` (16 more replies)
0 siblings, 17 replies; 45+ messages in thread
From: Umang Jain @ 2023-11-09 21:02 UTC (permalink / raw)
To: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Stefan Wahren, Umang Jain
This series aims to upport bcm2835-isp from the RPi kernel.
It is developed on top of staging-next which comprises many
VC04 changes for it's de-staging. Hence, the merge of this
driver is targeted when VC04 is de-staged completely (which I
have been pushing), but it can be helped getting reviewed meanwhile.
Hence, the reason for posting the series.
Patch (01-02)/15 adds a new driver named vc-sm-cma to handle memory sharing
with the VC4 VPU.
Patch 03/15 adds a small extension to videobuf2 to allow exporting as a
dma_buf instead of a file-descriptor.
Patch (04-05)/15 adds a couple of improvements/support for
bcm2835-isp(event callback and zero-copy) to vchiq-mmal.
Patch (06-10)/15 adds the core bcm2835-isp driver along with headers
and format defintions. 09/15 is a standalone patch which can be merged
independently I believe.
Patch (11-12)/15 deals with the colorspace support.
Patch 13/15 allows multiple instances of the ISP.
Patch 14/15 adds a admin-guide document on bcm2835-isp.
Patch 15/15 deals with driver registeration.
Dave Stevenson (6):
staging: vc04_services: Add new vc-sm-cma driver
media: videobuf2: Allow exporting of a struct dmabuf
staging: mmal-vchiq: Add support for event callbacks
staging: mmal-vchiq: Use vc-sm-cma to support zero copy
staging: mmal_vchiq: Add image formats to be used by bcm2835-isp
uapi: bcm2835-isp: Add bcm2835-isp uapi header file
David Plowman (2):
vc04_services: bcm2835-isp: Allow formats with different colour spaces
vc04_services: bcm2835-isp: Permit all sRGB colour spaces on ISP
outputs
Naushir Patuck (4):
media: uapi: v4l2-core: Add ISP statistics output V4L2 fourcc type
staging: vc04_services: bcm2835-isp: Add a more complex ISP processing
component
staging: vc04_services: bcm2835_isp: Allow multiple users
docs: admin-guide: media: bcm2835-isp: Add documentation for
bcm2835-isp
Umang Jain (3):
staging: vc04_services: vchiq_arm: Register vcsm-cma driver
staging: vc04_services: Add helpers for vchiq driver data
staging: vc04_services: vchiq: Register bcm2835-isp
.../admin-guide/media/bcm2835-isp.rst | 127 ++
.../userspace-api/media/drivers/index.rst | 1 +
.../userspace-api/media/v4l/meta-formats.rst | 1 +
.../v4l/pixfmt-meta-bcm2835-isp-stats.rst | 32 +
MAINTAINERS | 9 +
.../media/common/videobuf2/videobuf2-core.c | 36 +-
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
drivers/staging/vc04_services/Kconfig | 4 +
drivers/staging/vc04_services/Makefile | 3 +-
.../staging/vc04_services/bcm2835-isp/Kconfig | 14 +
.../vc04_services/bcm2835-isp/Makefile | 4 +
.../bcm2835-isp/bcm2835-isp-ctrls.h | 72 +
.../bcm2835-isp/bcm2835-isp-fmts.h | 559 +++++
.../bcm2835-isp/bcm2835-v4l2-isp.c | 1822 +++++++++++++++++
.../interface/vchiq_arm/vchiq_arm.c | 6 +
.../interface/vchiq_arm/vchiq_bus.h | 10 +
.../staging/vc04_services/vc-sm-cma/Kconfig | 10 +
.../staging/vc04_services/vc-sm-cma/Makefile | 4 +
.../staging/vc04_services/vc-sm-cma/vc_sm.c | 817 ++++++++
.../staging/vc04_services/vc-sm-cma/vc_sm.h | 54 +
.../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c | 507 +++++
.../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h | 63 +
.../vc04_services/vc-sm-cma/vc_sm_defs.h | 187 ++
.../vc04_services/vc-sm-cma/vc_sm_knl.h | 28 +
.../staging/vc04_services/vchiq-mmal/Kconfig | 1 +
.../vc04_services/vchiq-mmal/mmal-common.h | 5 +
.../vc04_services/vchiq-mmal/mmal-encodings.h | 66 +
.../vc04_services/vchiq-mmal/mmal-msg.h | 35 +
.../vchiq-mmal/mmal-parameters.h | 165 +-
.../vc04_services/vchiq-mmal/mmal-vchiq.c | 253 ++-
.../vc04_services/vchiq-mmal/mmal-vchiq.h | 5 +
include/media/videobuf2-core.h | 15 +
include/uapi/linux/bcm2835-isp.h | 347 ++++
include/uapi/linux/v4l2-controls.h | 5 +
include/uapi/linux/videodev2.h | 1 +
35 files changed, 5235 insertions(+), 34 deletions(-)
create mode 100644 Documentation/admin-guide/media/bcm2835-isp.rst
create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-ctrls.h
create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
create mode 100644 include/uapi/linux/bcm2835-isp.h
--
2.41.0
^ permalink raw reply [flat|nested] 45+ messages in thread
* [PATCH v2 01/15] staging: vc04_services: Add new vc-sm-cma driver
2023-11-09 21:02 [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Umang Jain
@ 2023-11-09 21:02 ` Umang Jain
2023-11-13 17:56 ` Stefan Wahren
2023-11-20 9:52 ` Dan Carpenter
2023-11-09 21:02 ` [PATCH v2 02/15] staging: vc04_services: vchiq_arm: Register vcsm-cma driver Umang Jain
` (15 subsequent siblings)
16 siblings, 2 replies; 45+ messages in thread
From: Umang Jain @ 2023-11-09 21:02 UTC (permalink / raw)
To: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Stefan Wahren, Umang Jain
From: Dave Stevenson <dave.stevenson@raspberrypi.com>
Add Broadcom VideoCore Shared Memory support.
This new driver allows contiguous memory blocks to be imported
into the VideoCore VPU memory map, and manages the lifetime of
those objects, only releasing the source dmabuf once the VPU has
confirmed it has finished with it.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
---
drivers/staging/vc04_services/Kconfig | 2 +
drivers/staging/vc04_services/Makefile | 2 +-
.../staging/vc04_services/vc-sm-cma/Kconfig | 10 +
.../staging/vc04_services/vc-sm-cma/Makefile | 4 +
.../staging/vc04_services/vc-sm-cma/vc_sm.c | 817 ++++++++++++++++++
.../staging/vc04_services/vc-sm-cma/vc_sm.h | 54 ++
.../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c | 507 +++++++++++
.../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h | 63 ++
.../vc04_services/vc-sm-cma/vc_sm_defs.h | 187 ++++
.../vc04_services/vc-sm-cma/vc_sm_knl.h | 28 +
10 files changed, 1673 insertions(+), 1 deletion(-)
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
diff --git a/drivers/staging/vc04_services/Kconfig b/drivers/staging/vc04_services/Kconfig
index 31e58c9d1a11..6c0e77d64376 100644
--- a/drivers/staging/vc04_services/Kconfig
+++ b/drivers/staging/vc04_services/Kconfig
@@ -46,5 +46,7 @@ source "drivers/staging/vc04_services/bcm2835-camera/Kconfig"
source "drivers/staging/vc04_services/vchiq-mmal/Kconfig"
+source "drivers/staging/vc04_services/vc-sm-cma/Kconfig"
+
endif
diff --git a/drivers/staging/vc04_services/Makefile b/drivers/staging/vc04_services/Makefile
index e8b897a7b9a6..1f61a263f22d 100644
--- a/drivers/staging/vc04_services/Makefile
+++ b/drivers/staging/vc04_services/Makefile
@@ -15,4 +15,4 @@ endif
obj-$(CONFIG_SND_BCM2835) += bcm2835-audio/
obj-$(CONFIG_VIDEO_BCM2835) += bcm2835-camera/
obj-$(CONFIG_BCM2835_VCHIQ_MMAL) += vchiq-mmal/
-
+obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma/
diff --git a/drivers/staging/vc04_services/vc-sm-cma/Kconfig b/drivers/staging/vc04_services/vc-sm-cma/Kconfig
new file mode 100644
index 000000000000..bbd296f5826b
--- /dev/null
+++ b/drivers/staging/vc04_services/vc-sm-cma/Kconfig
@@ -0,0 +1,10 @@
+config BCM_VC_SM_CMA
+ tristate "VideoCore Shared Memory (CMA) driver"
+ depends on BCM2835_VCHIQ
+ select RBTREE
+ select DMA_SHARED_BUFFER
+ help
+ Say Y here to enable the shared memory interface that
+ supports sharing dmabufs with VideoCore.
+ This operates over the VCHIQ interface to a service
+ running on VideoCore.
diff --git a/drivers/staging/vc04_services/vc-sm-cma/Makefile b/drivers/staging/vc04_services/vc-sm-cma/Makefile
new file mode 100644
index 000000000000..98c8ed6c9a50
--- /dev/null
+++ b/drivers/staging/vc04_services/vc-sm-cma/Makefile
@@ -0,0 +1,4 @@
+vc-sm-cma-$(CONFIG_BCM_VC_SM_CMA) := \
+ vc_sm.o vc_sm_cma_vchi.o
+
+obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma.o
diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
new file mode 100644
index 000000000000..5bbdb3a8468b
--- /dev/null
+++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
@@ -0,0 +1,817 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * VideoCore Shared Memory driver using CMA.
+ *
+ * Copyright: 2018, Raspberry Pi (Trading) Ltd
+ * Dave Stevenson <dave.stevenson@raspberrypi.com>
+ *
+ * Based on vmcs_sm driver from Broadcom Corporation for some API,
+ * and taking some code for buffer allocation and dmabuf handling from
+ * videobuf2.
+ *
+ * This driver handles taking a dmabuf, mapping it into the VPU address space,
+ * and returning a handle that the VPU can use.
+ */
+
+/* ---- Include Files ----------------------------------------------------- */
+#include <linux/debugfs.h>
+#include <linux/dma-buf.h>
+#include <linux/dma-mapping.h>
+#include <linux/errno.h>
+#include <linux/miscdevice.h>
+#include <linux/syscalls.h>
+
+#include "../interface/vchiq_arm/vchiq_bus.h"
+#include "../interface/vchiq_arm/vchiq_connected.h"
+#include "vc_sm_cma_vchi.h"
+
+#include "vc_sm.h"
+#include "vc_sm_knl.h"
+
+MODULE_IMPORT_NS(DMA_BUF);
+
+/* ---- Private Constants and Types --------------------------------------- */
+
+#define VC_SM_RESOURCE_NAME_DEFAULT "sm-host-resource"
+
+#define VC_SM_DIR_ROOT_NAME "vcsm-cma"
+#define VC_SM_STATE "state"
+
+/* Private file data associated with each opened device. */
+struct vc_sm_privdata_t {
+};
+
+typedef int (*VC_SM_SHOW) (struct seq_file *s, void *v);
+struct sm_pde_t {
+ VC_SM_SHOW show; /* Debug fs function hookup. */
+ struct dentry *dir_entry; /* Debug fs directory entry. */
+ void *priv_data; /* Private data */
+};
+
+/* Global state information. */
+struct sm_state_t {
+ struct vchiq_device *device;
+
+ struct miscdevice misc_dev;
+
+ struct sm_instance *sm_handle; /* Handle for videocore service. */
+
+ spinlock_t kernelid_map_lock; /* Spinlock protecting kernelid_map */
+ struct idr kernelid_map;
+
+ struct mutex map_lock; /* Global map lock. */
+ struct list_head buffer_list; /* List of buffer. */
+
+ struct dentry *dir_root; /* Debug fs entries root. */
+ struct sm_pde_t dir_state; /* Debug fs entries state sub-tree. */
+
+ bool require_released_callback; /* VPU will send a released msg when it
+ * has finished with a resource.
+ */
+ /* State for transactions */
+ int restart_sys; /* Tracks restart on interrupt. */
+ enum vc_sm_msg_type int_action; /* Interrupted action. */
+
+ u32 int_trans_id; /* Interrupted transaction. */
+ struct vchiq_instance *vchiq_instance;
+};
+
+struct vc_sm_dma_buf_attachment {
+ struct device *dev;
+ struct sg_table sg_table;
+ struct list_head list;
+ enum dma_data_direction dma_dir;
+};
+
+/* ---- Private Variables ----------------------------------------------- */
+
+static struct sm_state_t *sm_state;
+static int sm_inited;
+
+/* ---- Private Function Prototypes -------------------------------------- */
+
+/* ---- Private Functions ------------------------------------------------ */
+
+static int get_kernel_id(struct vc_sm_buffer *buffer)
+{
+ int handle;
+
+ spin_lock(&sm_state->kernelid_map_lock);
+ handle = idr_alloc(&sm_state->kernelid_map, buffer, 0, 0, GFP_KERNEL);
+ spin_unlock(&sm_state->kernelid_map_lock);
+
+ return handle;
+}
+
+static struct vc_sm_buffer *lookup_kernel_id(int handle)
+{
+ return idr_find(&sm_state->kernelid_map, handle);
+}
+
+static void free_kernel_id(int handle)
+{
+ spin_lock(&sm_state->kernelid_map_lock);
+ idr_remove(&sm_state->kernelid_map, handle);
+ spin_unlock(&sm_state->kernelid_map_lock);
+}
+
+static int vc_sm_cma_seq_file_show(struct seq_file *s, void *v)
+{
+ struct sm_pde_t *sm_pde;
+
+ sm_pde = (struct sm_pde_t *)(s->private);
+
+ if (sm_pde && sm_pde->show)
+ sm_pde->show(s, v);
+
+ return 0;
+}
+
+static int vc_sm_cma_single_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, vc_sm_cma_seq_file_show, inode->i_private);
+}
+
+static const struct file_operations vc_sm_cma_debug_fs_fops = {
+ .open = vc_sm_cma_single_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+static int vc_sm_cma_global_state_show(struct seq_file *s, void *v)
+{
+ struct vc_sm_buffer *resource = NULL;
+ int resource_count = 0;
+
+ if (!sm_state)
+ return 0;
+
+ seq_printf(s, "\nVC-ServiceHandle %p\n", sm_state->sm_handle);
+
+ /* Log all applicable mapping(s). */
+
+ mutex_lock(&sm_state->map_lock);
+ seq_puts(s, "\nResources\n");
+ if (!list_empty(&sm_state->buffer_list)) {
+ list_for_each_entry(resource, &sm_state->buffer_list,
+ global_buffer_list) {
+ resource_count++;
+
+ seq_printf(s, "\nResource %p\n",
+ resource);
+ seq_printf(s, " NAME %s\n",
+ resource->name);
+ seq_printf(s, " SIZE %zu\n",
+ resource->size);
+ seq_printf(s, " DMABUF %p\n",
+ resource->dma_buf);
+ seq_printf(s, " IMPORTED_DMABUF %p\n",
+ resource->imported_dma_buf);
+ seq_printf(s, " ATTACH %p\n",
+ resource->attach);
+ seq_printf(s, " SGT %p\n",
+ resource->sgt);
+ seq_printf(s, " DMA_ADDR %pad\n",
+ &resource->dma_addr);
+ seq_printf(s, " VC_HANDLE %08x\n",
+ resource->vc_handle);
+ seq_printf(s, " VC_MAPPING %d\n",
+ resource->vpu_state);
+ }
+ }
+ seq_printf(s, "\n\nTotal resource count: %d\n\n", resource_count);
+
+ mutex_unlock(&sm_state->map_lock);
+
+ return 0;
+}
+
+/*
+ * Adds a buffer to the private data list which tracks all the allocated
+ * data.
+ */
+static void vc_sm_add_resource(struct vc_sm_buffer *buffer)
+{
+ mutex_lock(&sm_state->map_lock);
+ list_add(&buffer->global_buffer_list, &sm_state->buffer_list);
+ mutex_unlock(&sm_state->map_lock);
+
+ dev_dbg(&sm_state->device->dev,"[%s]: added buffer %p (name %s, size %zu)\n",
+ __func__, buffer, buffer->name, buffer->size);
+}
+
+/*
+ * Cleans up imported dmabuf.
+ * Should be called with mutex held.
+ */
+static void vc_sm_clean_up_dmabuf(struct vc_sm_buffer *buffer)
+{
+ /* Handle cleaning up imported dmabufs */
+ if (buffer->sgt) {
+ dma_buf_unmap_attachment_unlocked(buffer->attach, buffer->sgt,
+ DMA_BIDIRECTIONAL);
+ buffer->sgt = NULL;
+ }
+ if (buffer->attach) {
+ dma_buf_detach(buffer->imported_dma_buf, buffer->attach);
+ buffer->attach = NULL;
+ }
+}
+
+/*
+ * Instructs VPU to decrement the refcount on a buffer.
+ */
+static void vc_sm_vpu_free(struct vc_sm_buffer *buffer)
+{
+ if (buffer->vc_handle && buffer->vpu_state == VPU_MAPPED) {
+ struct vc_sm_free_t free = { buffer->vc_handle, 0 };
+ int status = vc_sm_cma_vchi_free(sm_state->sm_handle, &free,
+ &sm_state->int_trans_id);
+ if (status != 0 && status != -EINTR) {
+ dev_err(&sm_state->device->dev,
+ "[%s]: failed to free memory on videocore (status: %u, trans_id: %u)\n",
+ __func__, status, sm_state->int_trans_id);
+ }
+
+ if (sm_state->require_released_callback) {
+ /* Need to wait for the VPU to confirm the free. */
+
+ /* Retain a reference on this until the VPU has
+ * released it
+ */
+ buffer->vpu_state = VPU_UNMAPPING;
+ } else {
+ buffer->vpu_state = VPU_NOT_MAPPED;
+ buffer->vc_handle = 0;
+ }
+ }
+}
+
+/*
+ * Release an allocation.
+ * All refcounting is done via the dma buf object.
+ *
+ * Must be called with the mutex held. The function will either release the
+ * mutex (if defering the release) or destroy it. The caller must therefore not
+ * reuse the buffer on return.
+ */
+static void vc_sm_release_resource(struct vc_sm_buffer *buffer)
+{
+ dev_dbg(&sm_state->device->dev, "[%s]: buffer %p (name %s, size %zu)\n",
+ __func__, buffer, buffer->name, buffer->size);
+
+ if (buffer->vc_handle) {
+ /* We've sent the unmap request but not had the response. */
+ dev_dbg(&sm_state->device->dev, "[%s]: Waiting for VPU unmap response on %p\n",
+ __func__, buffer);
+ goto defer;
+ }
+ if (buffer->in_use) {
+ /* dmabuf still in use - we await the release */
+ dev_dbg(&sm_state->device->dev, "[%s]: buffer %p is still in use\n",
+ __func__, buffer);
+ goto defer;
+ }
+
+ /* Release the allocation */
+ if (buffer->imported_dma_buf)
+ dma_buf_put(buffer->imported_dma_buf);
+ else
+ dev_err(&sm_state->device->dev, "%s: Imported dmabuf already been put for buf %p\n",
+ __func__, buffer);
+ buffer->imported_dma_buf = NULL;
+
+ /* Free our buffer. Start by removing it from the list */
+ mutex_lock(&sm_state->map_lock);
+ list_del(&buffer->global_buffer_list);
+ mutex_unlock(&sm_state->map_lock);
+
+ dev_dbg(&sm_state->device->dev, "%s: Release our allocation - done\n", __func__);
+ mutex_unlock(&buffer->lock);
+
+ mutex_destroy(&buffer->lock);
+
+ kfree(buffer);
+ return;
+
+defer:
+ mutex_unlock(&buffer->lock);
+}
+
+/* Dma_buf operations for chaining through to an imported dma_buf */
+
+static void vc_sm_dma_buf_release(struct dma_buf *dmabuf)
+{
+ struct vc_sm_buffer *buffer;
+
+ if (!dmabuf)
+ return;
+
+ buffer = (struct vc_sm_buffer *)dmabuf->priv;
+
+ mutex_lock(&buffer->lock);
+
+ dev_dbg(&sm_state->device->dev, "%s dmabuf %p, buffer %p\n", __func__, dmabuf, buffer);
+
+ buffer->in_use = 0;
+
+ /* Unmap on the VPU */
+ vc_sm_vpu_free(buffer);
+ dev_dbg(&sm_state->device->dev, "%s vpu_free done\n", __func__);
+
+ /* Unmap our dma_buf object (the vc_sm_buffer remains until released
+ * on the VPU).
+ */
+ vc_sm_clean_up_dmabuf(buffer);
+ dev_dbg(&sm_state->device->dev, "%s clean_up dmabuf done\n", __func__);
+
+ /* buffer->lock will be destroyed by vc_sm_release_resource if finished
+ * with, otherwise unlocked. Do NOT unlock here.
+ */
+ vc_sm_release_resource(buffer);
+ dev_dbg(&sm_state->device->dev, "%s done\n", __func__);
+}
+
+static
+int vc_sm_import_dma_buf_attach(struct dma_buf *dmabuf,
+ struct dma_buf_attachment *attachment)
+{
+ struct vc_sm_buffer *buf = dmabuf->priv;
+
+ return buf->imported_dma_buf->ops->attach(buf->imported_dma_buf,
+ attachment);
+}
+
+static
+void vc_sm_import_dma_buf_detatch(struct dma_buf *dmabuf,
+ struct dma_buf_attachment *attachment)
+{
+ struct vc_sm_buffer *buf = dmabuf->priv;
+
+ buf->imported_dma_buf->ops->detach(buf->imported_dma_buf, attachment);
+}
+
+static
+struct sg_table *vc_sm_import_map_dma_buf(struct dma_buf_attachment *attachment,
+ enum dma_data_direction direction)
+{
+ struct vc_sm_buffer *buf = attachment->dmabuf->priv;
+
+ return buf->imported_dma_buf->ops->map_dma_buf(attachment,
+ direction);
+}
+
+static
+void vc_sm_import_unmap_dma_buf(struct dma_buf_attachment *attachment,
+ struct sg_table *table,
+ enum dma_data_direction direction)
+{
+ struct vc_sm_buffer *buf = attachment->dmabuf->priv;
+
+ buf->imported_dma_buf->ops->unmap_dma_buf(attachment, table, direction);
+}
+
+static
+int vc_sm_import_dmabuf_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma)
+{
+ struct vc_sm_buffer *buf = dmabuf->priv;
+
+ dev_dbg(&sm_state->device->dev, "%s: mmap dma_buf %p, buf %p, imported db %p\n", __func__,
+ dmabuf, buf, buf->imported_dma_buf);
+
+ return buf->imported_dma_buf->ops->mmap(buf->imported_dma_buf, vma);
+}
+
+static
+int vc_sm_import_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
+ enum dma_data_direction direction)
+{
+ struct vc_sm_buffer *buf = dmabuf->priv;
+
+ return buf->imported_dma_buf->ops->begin_cpu_access(buf->imported_dma_buf,
+ direction);
+}
+
+static
+int vc_sm_import_dma_buf_end_cpu_access(struct dma_buf *dmabuf,
+ enum dma_data_direction direction)
+{
+ struct vc_sm_buffer *buf = dmabuf->priv;
+
+ return buf->imported_dma_buf->ops->end_cpu_access(buf->imported_dma_buf,
+ direction);
+}
+
+static const struct dma_buf_ops dma_buf_import_ops = {
+ .map_dma_buf = vc_sm_import_map_dma_buf,
+ .unmap_dma_buf = vc_sm_import_unmap_dma_buf,
+ .mmap = vc_sm_import_dmabuf_mmap,
+ .release = vc_sm_dma_buf_release,
+ .attach = vc_sm_import_dma_buf_attach,
+ .detach = vc_sm_import_dma_buf_detatch,
+ .begin_cpu_access = vc_sm_import_dma_buf_begin_cpu_access,
+ .end_cpu_access = vc_sm_import_dma_buf_end_cpu_access,
+};
+
+/* Import a dma_buf to be shared with VC. */
+int
+vc_sm_cma_import_dmabuf_internal(struct sm_state_t *state,
+ struct dma_buf *dma_buf,
+ int fd,
+ struct dma_buf **imported_buf)
+{
+ DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
+ struct vc_sm_buffer *buffer = NULL;
+ struct vc_sm_import import = { };
+ struct vc_sm_import_result result = { };
+ struct dma_buf_attachment *attach = NULL;
+ struct sg_table *sgt = NULL;
+ dma_addr_t dma_addr;
+ int ret = 0;
+ int status;
+
+ /* Setup our allocation parameters */
+ dev_dbg(&sm_state->device->dev, "%s: importing dma_buf %p/fd %d\n", __func__, dma_buf, fd);
+
+ if (fd < 0)
+ get_dma_buf(dma_buf);
+ else
+ dma_buf = dma_buf_get(fd);
+
+ if (!dma_buf)
+ return -EINVAL;
+
+ attach = dma_buf_attach(dma_buf, &sm_state->device->dev);
+ if (IS_ERR(attach)) {
+ ret = PTR_ERR(attach);
+ goto error;
+ }
+
+ sgt = dma_buf_map_attachment_unlocked(attach, DMA_BIDIRECTIONAL);
+ if (IS_ERR(sgt)) {
+ ret = PTR_ERR(sgt);
+ goto error;
+ }
+
+ /* Verify that the address block is contiguous */
+ if (sgt->nents != 1) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ /* Allocate local buffer to track this allocation. */
+ buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
+ if (!buffer) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ import.type = VC_SM_ALLOC_NON_CACHED;
+ dma_addr = sg_dma_address(sgt->sgl);
+ import.addr = (u32)dma_addr;
+ if ((import.addr & 0xC0000000) != 0xC0000000) {
+ dev_err(&sm_state->device->dev, "%s: Expecting an uncached alias for dma_addr %pad\n",
+ __func__, &dma_addr);
+ import.addr |= 0xC0000000;
+ }
+ import.size = sg_dma_len(sgt->sgl);
+ import.allocator = current->tgid;
+ import.kernel_id = get_kernel_id(buffer);
+
+ memcpy(import.name, VC_SM_RESOURCE_NAME_DEFAULT,
+ sizeof(VC_SM_RESOURCE_NAME_DEFAULT));
+
+ dev_dbg(&sm_state->device->dev,
+ "[%s]: attempt to import \"%s\" data - type %u, addr %pad, size %u.\n",
+ __func__, import.name, import.type, &dma_addr, import.size);
+
+ /* Allocate the videocore buffer. */
+ status = vc_sm_cma_vchi_import(sm_state->sm_handle, &import, &result,
+ &sm_state->int_trans_id);
+ if (status == -EINTR) {
+ dev_dbg(&sm_state->device->dev,
+ "[%s]: requesting import memory action restart (trans_id: %u)\n",
+ __func__, sm_state->int_trans_id);
+ ret = -ERESTARTSYS;
+ sm_state->restart_sys = -EINTR;
+ sm_state->int_action = VC_SM_MSG_TYPE_IMPORT;
+ goto error;
+ } else if (status || !result.res_handle) {
+ dev_dbg(&sm_state->device->dev,
+ "[%s]: failed to import memory on videocore (status: %u, trans_id: %u)\n",
+ __func__, status, sm_state->int_trans_id);
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ mutex_init(&buffer->lock);
+ INIT_LIST_HEAD(&buffer->attachments);
+ memcpy(buffer->name, import.name,
+ min(sizeof(buffer->name), sizeof(import.name) - 1));
+
+ /* Keep track of the buffer we created. */
+ buffer->vc_handle = result.res_handle;
+ buffer->size = import.size;
+ buffer->vpu_state = VPU_MAPPED;
+
+ buffer->imported_dma_buf = dma_buf;
+
+ buffer->attach = attach;
+ buffer->sgt = sgt;
+ buffer->dma_addr = dma_addr;
+ buffer->in_use = 1;
+ buffer->kernel_id = import.kernel_id;
+
+ /*
+ * We're done - we need to export a new dmabuf chaining through most
+ * functions, but enabling us to release our own internal references
+ * here.
+ */
+ exp_info.ops = &dma_buf_import_ops;
+ exp_info.size = import.size;
+ exp_info.flags = O_RDWR;
+ exp_info.priv = buffer;
+
+ buffer->dma_buf = dma_buf_export(&exp_info);
+ if (IS_ERR(buffer->dma_buf)) {
+ ret = PTR_ERR(buffer->dma_buf);
+ goto error;
+ }
+
+ vc_sm_add_resource(buffer);
+
+ *imported_buf = buffer->dma_buf;
+
+ return 0;
+
+error:
+ if (result.res_handle) {
+ struct vc_sm_free_t free = { result.res_handle, 0 };
+
+ vc_sm_cma_vchi_free(sm_state->sm_handle, &free,
+ &sm_state->int_trans_id);
+ }
+ free_kernel_id(import.kernel_id);
+ kfree(buffer);
+ if (sgt)
+ dma_buf_unmap_attachment_unlocked(attach, sgt, DMA_BIDIRECTIONAL);
+ if (attach)
+ dma_buf_detach(dma_buf, attach);
+ dma_buf_put(dma_buf);
+ return ret;
+}
+
+static void
+vc_sm_vpu_event(struct sm_instance *instance, struct vc_sm_result_t *reply,
+ int reply_len)
+{
+ switch (reply->trans_id & ~0x80000000) {
+ case VC_SM_MSG_TYPE_CLIENT_VERSION:
+ {
+ /* Acknowledge that the firmware supports the version command */
+ dev_dbg(&sm_state->device->dev,
+ "%s: firmware acked version msg. Require release cb\n", __func__);
+ sm_state->require_released_callback = true;
+ }
+ break;
+ case VC_SM_MSG_TYPE_RELEASED:
+ {
+ struct vc_sm_released *release = (struct vc_sm_released *)reply;
+ struct vc_sm_buffer *buffer =
+ lookup_kernel_id(release->kernel_id);
+ if (!buffer) {
+ dev_err(&sm_state->device->dev,
+ "%s: VC released a buffer that is already released, kernel_id %d\n",
+ __func__, release->kernel_id);
+ break;
+ }
+ mutex_lock(&buffer->lock);
+
+ dev_dbg(&sm_state->device->dev,
+ "%s: Released addr %08x, size %u, id %08x, mem_handle %08x\n",
+ __func__, release->addr, release->size,
+ release->kernel_id, release->vc_handle);
+
+ buffer->vc_handle = 0;
+ buffer->vpu_state = VPU_NOT_MAPPED;
+ free_kernel_id(release->kernel_id);
+
+ vc_sm_release_resource(buffer);
+ }
+ break;
+ default:
+ dev_err(&sm_state->device->dev, "%s: Unknown vpu cmd %x\n",
+ __func__, reply->trans_id);
+ break;
+ }
+}
+
+/* Driver load/unload functions */
+/* Videocore connected. */
+static void vc_sm_connected_init(void)
+{
+ int ret;
+ struct vc_sm_version version;
+ struct vc_sm_result_t version_result;
+
+ pr_info("[%s]: start\n", __func__);
+
+ /*
+ * Initialize and create a VCHI connection for the shared memory service
+ * running on videocore.
+ */
+ ret = vchiq_initialise(&sm_state->vchiq_instance);
+ if (ret) {
+ dev_err(&sm_state->device->dev, "[%s]: failed to initialise VCHI instance (ret=%d)\n",
+ __func__, ret);
+
+ return;
+ }
+
+ ret = vchiq_connect(sm_state->vchiq_instance);
+ if (ret) {
+ dev_err(&sm_state->device->dev, "[%s]: failed to connect VCHI instance (ret=%d)\n",
+ __func__, ret);
+
+ return;
+ }
+
+ /* Initialize an instance of the shared memory service. */
+ sm_state->sm_handle = vc_sm_cma_vchi_init(sm_state->vchiq_instance, 1,
+ vc_sm_vpu_event);
+ if (!sm_state->sm_handle) {
+ dev_err(&sm_state->device->dev,
+ "[%s]: failed to initialize shared memory service\n", __func__);
+
+ return;
+ }
+
+ /* Create a debug fs directory entry (root). */
+ sm_state->dir_root = debugfs_create_dir(VC_SM_DIR_ROOT_NAME, NULL);
+
+ sm_state->dir_state.show = &vc_sm_cma_global_state_show;
+ sm_state->dir_state.dir_entry =
+ debugfs_create_file(VC_SM_STATE, 0444, sm_state->dir_root,
+ &sm_state->dir_state,
+ &vc_sm_cma_debug_fs_fops);
+
+ INIT_LIST_HEAD(&sm_state->buffer_list);
+
+ version.version = 2;
+ ret = vc_sm_cma_vchi_client_version(sm_state->sm_handle, &version,
+ &version_result,
+ &sm_state->int_trans_id);
+ if (ret) {
+ dev_err(&sm_state->device->dev,
+ "[%s]: Failed to send version request %d\n", __func__, ret);
+ }
+
+ /* Done! */
+ sm_inited = 1;
+ pr_info("[%s]: installed successfully\n", __func__);
+}
+
+/* Driver loading. */
+static int bcm2835_vc_sm_cma_probe(struct vchiq_device *device)
+{
+ struct device *dev = &device->dev;
+ int err;
+
+ err = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
+ if (err) {
+ dev_err(dev, "dma_set_mask_and_coherent failed: %d\n", err);
+ return err;
+ }
+
+ sm_state = devm_kzalloc(dev, sizeof(*sm_state), GFP_KERNEL);
+ if (!sm_state)
+ return -ENOMEM;
+ sm_state->device = device;
+ mutex_init(&sm_state->map_lock);
+
+ spin_lock_init(&sm_state->kernelid_map_lock);
+ idr_init_base(&sm_state->kernelid_map, 1);
+
+ dev->dma_parms = devm_kzalloc(dev, sizeof(dev->dma_parms), GFP_KERNEL);
+
+ /* dma_set_max_seg_size checks if dma_parms is NULL. */
+ dma_set_max_seg_size(dev, 0x3FFFFFFF);
+
+ vchiq_add_connected_callback(vc_sm_connected_init);
+ return 0;
+}
+
+/* Driver unloading. */
+static void bcm2835_vc_sm_cma_remove(struct vchiq_device *device)
+{
+ if (sm_inited) {
+ misc_deregister(&sm_state->misc_dev);
+
+ /* Remove all proc entries. */
+ debugfs_remove_recursive(sm_state->dir_root);
+
+ /* Stop the videocore shared memory service. */
+ vc_sm_cma_vchi_stop(sm_state->vchiq_instance, &sm_state->sm_handle);
+ }
+
+ if (sm_state) {
+ idr_destroy(&sm_state->kernelid_map);
+
+ /* Free the memory for the state structure. */
+ mutex_destroy(&sm_state->map_lock);
+ }
+}
+
+/* Kernel API calls */
+/* Get an internal resource handle mapped from the external one. */
+int vc_sm_cma_int_handle(void *handle)
+{
+ struct dma_buf *dma_buf = (struct dma_buf *)handle;
+ struct vc_sm_buffer *buf;
+
+ /* Validate we can work with this device. */
+ if (!sm_state || !handle) {
+ dev_err(&sm_state->device->dev, "[%s]: invalid input\n", __func__);
+ return 0;
+ }
+
+ buf = (struct vc_sm_buffer *)dma_buf->priv;
+ return buf->vc_handle;
+}
+EXPORT_SYMBOL_GPL(vc_sm_cma_int_handle);
+
+/* Free a previously allocated shared memory handle and block. */
+int vc_sm_cma_free(void *handle)
+{
+ struct dma_buf *dma_buf = (struct dma_buf *)handle;
+
+ /* Validate we can work with this device. */
+ if (!sm_state || !handle) {
+ dev_err(&sm_state->device->dev, "[%s]: invalid input\n", __func__);
+ return -EPERM;
+ }
+
+ dev_dbg(&sm_state->device->dev, "%s: handle %p/dmabuf %p\n", __func__, handle, dma_buf);
+
+ dma_buf_put(dma_buf);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(vc_sm_cma_free);
+
+/* Import a dmabuf to be shared with VC. */
+int vc_sm_cma_import_dmabuf(struct dma_buf *src_dmabuf, void **handle)
+{
+ struct dma_buf *new_dma_buf;
+ int ret;
+
+ /* Validate we can work with this device. */
+ if (!sm_state || !src_dmabuf || !handle) {
+ dev_err(&sm_state->device->dev, "[%s]: invalid input\n", __func__);
+ return -EPERM;
+ }
+
+ ret = vc_sm_cma_import_dmabuf_internal(sm_state, src_dmabuf,
+ -1, &new_dma_buf);
+
+ if (!ret) {
+ dev_dbg(&sm_state->device->dev, "%s: imported to ptr %p\n",
+ __func__, new_dma_buf);
+
+ /* Assign valid handle at this time.*/
+ *handle = new_dma_buf;
+ } else {
+ /*
+ * succeeded in importing the dma_buf, but then
+ * failed to look it up again. How?
+ * Release the fd again.
+ */
+ dev_err(&sm_state->device->dev, "%s: imported vc_sm_cma_get_buffer failed %d\n",
+ __func__, ret);
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(vc_sm_cma_import_dmabuf);
+
+static struct vchiq_device_id device_id_table[] = {
+ { .name = "vcsm-cma" },
+ {}
+};
+MODULE_DEVICE_TABLE(vchiq, device_id_table);
+
+static struct vchiq_driver bcm2835_vcsm_cma_driver = {
+ .probe = bcm2835_vc_sm_cma_probe,
+ .remove = bcm2835_vc_sm_cma_remove,
+ .id_table = device_id_table,
+ .driver = {
+ .name = "vcsm-cma",
+ },
+};
+
+module_vchiq_driver(bcm2835_vcsm_cma_driver);
+
+MODULE_AUTHOR("Dave Stevenson");
+MODULE_DESCRIPTION("VideoCore CMA Shared Memory Driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
new file mode 100644
index 000000000000..61e110ec414d
--- /dev/null
+++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
@@ -0,0 +1,54 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * VideoCore Shared Memory driver using CMA.
+ *
+ * Copyright: 2018, Raspberry Pi (Trading) Ltd
+ *
+ */
+
+#ifndef VC_SM_H
+#define VC_SM_H
+
+#define VC_SM_MAX_NAME_LEN 32
+
+enum vc_sm_vpu_mapping_state {
+ VPU_NOT_MAPPED,
+ VPU_MAPPED,
+ VPU_UNMAPPING
+};
+
+struct vc_sm_buffer {
+ struct list_head global_buffer_list; /* Global list of buffers. */
+
+ /* Index in the kernel_id idr so that we can find the
+ * mmal_msg_context again when servicing the VCHI reply.
+ */
+ int kernel_id;
+
+ size_t size;
+
+ /* Lock over all the following state for this buffer */
+ struct mutex lock;
+ struct list_head attachments;
+
+ char name[VC_SM_MAX_NAME_LEN];
+
+ int in_use:1; /* Kernel is still using this resource */
+
+ enum vc_sm_vpu_mapping_state vpu_state;
+ u32 vc_handle; /* VideoCore handle for this buffer */
+
+ /* DMABUF related fields */
+ struct dma_buf *dma_buf;
+ dma_addr_t dma_addr;
+ void *cookie;
+
+ struct vc_sm_privdata_t *private;
+
+ struct dma_buf *imported_dma_buf;
+ struct dma_buf_attachment *attach;
+ struct sg_table *sgt;
+};
+
+#endif
diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
new file mode 100644
index 000000000000..08f70e955acf
--- /dev/null
+++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
@@ -0,0 +1,507 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * VideoCore Shared Memory CMA allocator
+ *
+ * Copyright: 2018, Raspberry Pi (Trading) Ltd
+ * Copyright 2011-2012 Broadcom Corporation. All rights reserved.
+ *
+ * Based on vmcs_sm driver from Broadcom Corporation.
+ *
+ */
+
+/* ---- Include Files ----------------------------------------------------- */
+#include <linux/kthread.h>
+#include <linux/list.h>
+#include <linux/semaphore.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+
+#include "vc_sm_cma_vchi.h"
+
+#define VC_SM_VER 1
+#define VC_SM_MIN_VER 0
+
+/* ---- Private Constants and Types -------------------------------------- */
+
+/* Command blocks come from a pool */
+#define SM_MAX_NUM_CMD_RSP_BLKS 32
+
+/* The number of supported connections */
+#define SM_MAX_NUM_CONNECTIONS 3
+
+struct sm_cmd_rsp_blk {
+ struct list_head head; /* To create lists */
+ /* To be signaled when the response is there */
+ struct completion cmplt;
+
+ u32 id;
+ u16 length;
+
+ u8 msg[VC_SM_MAX_MSG_LEN];
+
+ uint32_t wait:1;
+ uint32_t sent:1;
+ uint32_t alloc:1;
+
+};
+
+struct sm_instance {
+ u32 num_connections;
+ unsigned int service_handle[SM_MAX_NUM_CONNECTIONS];
+ struct task_struct *io_thread;
+ struct completion io_cmplt;
+
+ vpu_event_cb vpu_event;
+
+ /* Mutex over the following lists */
+ struct mutex lock;
+ u32 trans_id;
+ struct list_head cmd_list;
+ struct list_head rsp_list;
+ struct list_head dead_list;
+
+ struct sm_cmd_rsp_blk free_blk[SM_MAX_NUM_CMD_RSP_BLKS];
+
+ /* Mutex over the free_list */
+ struct mutex free_lock;
+ struct list_head free_list;
+
+ struct semaphore free_sema;
+ struct vchiq_instance *vchiq_instance;
+};
+
+/* ---- Private Variables ------------------------------------------------ */
+
+/* ---- Private Function Prototypes -------------------------------------- */
+
+/* ---- Private Functions ------------------------------------------------ */
+static int
+bcm2835_vchi_msg_queue(struct vchiq_instance *vchiq_instance, unsigned int handle,
+ void *data,
+ unsigned int size)
+{
+ return vchiq_queue_kernel_message(vchiq_instance, handle, data, size);
+}
+
+static struct
+sm_cmd_rsp_blk *vc_vchi_cmd_create(struct sm_instance *instance,
+ enum vc_sm_msg_type id, void *msg,
+ u32 size, int wait)
+{
+ struct sm_cmd_rsp_blk *blk;
+ struct vc_sm_msg_hdr_t *hdr;
+
+ if (down_interruptible(&instance->free_sema)) {
+ blk = kmalloc(sizeof(*blk), GFP_KERNEL);
+ if (!blk)
+ return NULL;
+
+ blk->alloc = 1;
+ init_completion(&blk->cmplt);
+ } else {
+ mutex_lock(&instance->free_lock);
+ blk =
+ list_first_entry(&instance->free_list,
+ struct sm_cmd_rsp_blk, head);
+ list_del(&blk->head);
+ mutex_unlock(&instance->free_lock);
+ }
+
+ blk->sent = 0;
+ blk->wait = wait;
+ blk->length = sizeof(*hdr) + size;
+
+ hdr = (struct vc_sm_msg_hdr_t *)blk->msg;
+ hdr->type = id;
+ mutex_lock(&instance->lock);
+ instance->trans_id++;
+ /*
+ * Retain the top bit for identifying asynchronous events, or VPU cmds.
+ */
+ instance->trans_id &= ~0x80000000;
+ hdr->trans_id = instance->trans_id;
+ blk->id = instance->trans_id;
+ mutex_unlock(&instance->lock);
+
+ if (size)
+ memcpy(hdr->body, msg, size);
+
+ return blk;
+}
+
+static void
+vc_vchi_cmd_delete(struct sm_instance *instance, struct sm_cmd_rsp_blk *blk)
+{
+ if (blk->alloc) {
+ kfree(blk);
+ return;
+ }
+
+ mutex_lock(&instance->free_lock);
+ list_add(&blk->head, &instance->free_list);
+ mutex_unlock(&instance->free_lock);
+ up(&instance->free_sema);
+}
+
+static void vc_sm_cma_vchi_rx_ack(struct sm_instance *instance,
+ struct sm_cmd_rsp_blk *cmd,
+ struct vc_sm_result_t *reply,
+ u32 reply_len)
+{
+ mutex_lock(&instance->lock);
+ list_for_each_entry(cmd,
+ &instance->rsp_list,
+ head) {
+ if (cmd->id == reply->trans_id)
+ break;
+ }
+ mutex_unlock(&instance->lock);
+
+ if (&cmd->head == &instance->rsp_list) {
+ //pr_debug("%s: received response %u, throw away...",
+ pr_err("%s: received response %u, throw away...",
+ __func__,
+ reply->trans_id);
+ } else if (reply_len > sizeof(cmd->msg)) {
+ pr_err("%s: reply too big (%u) %u, throw away...",
+ __func__, reply_len,
+ reply->trans_id);
+ } else {
+ memcpy(cmd->msg, reply,
+ reply_len);
+ complete(&cmd->cmplt);
+ }
+}
+
+static int vc_sm_cma_vchi_videocore_io(void *arg)
+{
+ struct sm_instance *instance = arg;
+ struct sm_cmd_rsp_blk *cmd = NULL, *cmd_tmp;
+ struct vc_sm_result_t *reply;
+ struct vchiq_header *header;
+ s32 status;
+ int svc_use = 1;
+
+ while (1) {
+ if (svc_use)
+ vchiq_release_service(instance->vchiq_instance,
+ instance->service_handle[0]);
+ svc_use = 0;
+
+ if (wait_for_completion_interruptible(&instance->io_cmplt))
+ continue;
+ vchiq_use_service(instance->vchiq_instance, instance->service_handle[0]);
+ svc_use = 1;
+
+ do {
+ /*
+ * Get new command and move it to response list
+ */
+ mutex_lock(&instance->lock);
+ if (list_empty(&instance->cmd_list)) {
+ /* no more commands to process */
+ mutex_unlock(&instance->lock);
+ break;
+ }
+ cmd = list_first_entry(&instance->cmd_list,
+ struct sm_cmd_rsp_blk, head);
+ list_move(&cmd->head, &instance->rsp_list);
+ cmd->sent = 1;
+ mutex_unlock(&instance->lock);
+ /* Send the command */
+ status =
+ bcm2835_vchi_msg_queue(instance->vchiq_instance,
+ instance->service_handle[0],
+ cmd->msg, cmd->length);
+ if (status) {
+ pr_err("%s: failed to queue message (%d)",
+ __func__, status);
+ }
+
+ /* If no reply is needed then we're done */
+ if (!cmd->wait) {
+ mutex_lock(&instance->lock);
+ list_del(&cmd->head);
+ mutex_unlock(&instance->lock);
+ vc_vchi_cmd_delete(instance, cmd);
+ continue;
+ }
+
+ if (status) {
+ complete(&cmd->cmplt);
+ continue;
+ }
+
+ } while (1);
+
+ while ((header = vchiq_msg_hold(instance->vchiq_instance,
+ instance->service_handle[0]))) {
+ reply = (struct vc_sm_result_t *)header->data;
+ if (reply->trans_id & 0x80000000) {
+ /* Async event or cmd from the VPU */
+ if (instance->vpu_event)
+ instance->vpu_event(instance, reply,
+ header->size);
+ } else {
+ vc_sm_cma_vchi_rx_ack(instance, cmd, reply,
+ header->size);
+ }
+
+ vchiq_release_message(instance->vchiq_instance,
+ instance->service_handle[0],
+ header);
+ }
+
+ /* Go through the dead list and free them */
+ mutex_lock(&instance->lock);
+ list_for_each_entry_safe(cmd, cmd_tmp, &instance->dead_list,
+ head) {
+ list_del(&cmd->head);
+ vc_vchi_cmd_delete(instance, cmd);
+ }
+ mutex_unlock(&instance->lock);
+ }
+
+ return 0;
+}
+
+static int vc_sm_cma_vchi_callback(struct vchiq_instance *vchiq_instance,
+ enum vchiq_reason reason,
+ struct vchiq_header *header,
+ unsigned int handle, void *userdata)
+{
+ struct sm_instance *instance = vchiq_get_service_userdata(vchiq_instance, handle);
+
+ switch (reason) {
+ case VCHIQ_MESSAGE_AVAILABLE:
+ vchiq_msg_queue_push(vchiq_instance, handle, header);
+ complete(&instance->io_cmplt);
+ break;
+
+ case VCHIQ_SERVICE_CLOSED:
+ pr_info("%s: service CLOSED!!", __func__);
+ default:
+ break;
+ }
+
+ return 0;
+}
+
+struct sm_instance *vc_sm_cma_vchi_init(struct vchiq_instance *vchiq_instance,
+ unsigned int num_connections,
+ vpu_event_cb vpu_event)
+{
+ u32 i;
+ struct sm_instance *instance;
+ int status;
+
+ pr_debug("%s: start", __func__);
+
+ if (num_connections > SM_MAX_NUM_CONNECTIONS) {
+ pr_err("%s: unsupported number of connections %u (max=%u)",
+ __func__, num_connections, SM_MAX_NUM_CONNECTIONS);
+
+ goto err_null;
+ }
+ /* Allocate memory for this instance */
+ instance = kzalloc(sizeof(*instance), GFP_KERNEL);
+
+ /* Misc initialisations */
+ mutex_init(&instance->lock);
+ init_completion(&instance->io_cmplt);
+ INIT_LIST_HEAD(&instance->cmd_list);
+ INIT_LIST_HEAD(&instance->rsp_list);
+ INIT_LIST_HEAD(&instance->dead_list);
+ INIT_LIST_HEAD(&instance->free_list);
+ sema_init(&instance->free_sema, SM_MAX_NUM_CMD_RSP_BLKS);
+ mutex_init(&instance->free_lock);
+ for (i = 0; i < SM_MAX_NUM_CMD_RSP_BLKS; i++) {
+ init_completion(&instance->free_blk[i].cmplt);
+ list_add(&instance->free_blk[i].head, &instance->free_list);
+ }
+
+ instance->vchiq_instance = vchiq_instance;
+
+ /* Open the VCHI service connections */
+ instance->num_connections = num_connections;
+ for (i = 0; i < num_connections; i++) {
+ struct vchiq_service_params_kernel params = {
+ .version = VC_SM_VER,
+ .version_min = VC_SM_MIN_VER,
+ .fourcc = VCHIQ_MAKE_FOURCC('S', 'M', 'E', 'M'),
+ .callback = vc_sm_cma_vchi_callback,
+ .userdata = instance,
+ };
+
+ status = vchiq_open_service(vchiq_instance, ¶ms,
+ &instance->service_handle[i]);
+ if (status) {
+ pr_err("%s: failed to open VCHI service (%d)",
+ __func__, status);
+
+ goto err_close_services;
+ }
+ }
+ /* Create the thread which takes care of all io to/from videoocore. */
+ instance->io_thread = kthread_create(&vc_sm_cma_vchi_videocore_io,
+ (void *)instance, "SMIO");
+ if (!instance->io_thread) {
+ pr_err("%s: failed to create SMIO thread", __func__);
+
+ goto err_close_services;
+ }
+ instance->vpu_event = vpu_event;
+ set_user_nice(instance->io_thread, -10);
+ wake_up_process(instance->io_thread);
+
+ pr_debug("%s: success - instance %p", __func__, instance);
+ return instance;
+
+err_close_services:
+ for (i = 0; i < instance->num_connections; i++) {
+ if (instance->service_handle[i])
+ vchiq_close_service(vchiq_instance, instance->service_handle[i]);
+ }
+ kfree(instance);
+err_null:
+ pr_debug("%s: FAILED", __func__);
+ return NULL;
+}
+
+int vc_sm_cma_vchi_stop(struct vchiq_instance *vchiq_instance, struct sm_instance **handle)
+{
+ struct sm_instance *instance;
+ u32 i;
+
+ if (!handle) {
+ pr_err("%s: invalid pointer to handle %p", __func__, handle);
+ goto lock;
+ }
+
+ if (!*handle) {
+ pr_err("%s: invalid handle %p", __func__, *handle);
+ goto lock;
+ }
+
+ instance = *handle;
+
+ /* Close all VCHI service connections */
+ for (i = 0; i < instance->num_connections; i++) {
+ vchiq_use_service(vchiq_instance, instance->service_handle[i]);
+ vchiq_close_service(vchiq_instance, instance->service_handle[i]);
+ }
+
+ kfree(instance);
+
+ *handle = NULL;
+ return 0;
+
+lock:
+ return -EINVAL;
+}
+
+static int vc_sm_cma_vchi_send_msg(struct sm_instance *handle,
+ enum vc_sm_msg_type msg_id, void *msg,
+ u32 msg_size, void *result, u32 result_size,
+ u32 *cur_trans_id, u8 wait_reply)
+{
+ int status = 0;
+ struct sm_instance *instance = handle;
+ struct sm_cmd_rsp_blk *cmd_blk;
+
+ if (!handle) {
+ pr_err("%s: invalid handle", __func__);
+ return -EINVAL;
+ }
+ if (!msg) {
+ pr_err("%s: invalid msg pointer", __func__);
+ return -EINVAL;
+ }
+
+ cmd_blk =
+ vc_vchi_cmd_create(instance, msg_id, msg, msg_size, wait_reply);
+ if (!cmd_blk) {
+ pr_err("[%s]: failed to allocate global tracking resource",
+ __func__);
+ return -ENOMEM;
+ }
+
+ if (cur_trans_id)
+ *cur_trans_id = cmd_blk->id;
+
+ mutex_lock(&instance->lock);
+ list_add_tail(&cmd_blk->head, &instance->cmd_list);
+ mutex_unlock(&instance->lock);
+ complete(&instance->io_cmplt);
+
+ if (!wait_reply)
+ /* We're done */
+ return 0;
+
+ /* Wait for the response */
+ if (wait_for_completion_interruptible(&cmd_blk->cmplt)) {
+ mutex_lock(&instance->lock);
+ if (!cmd_blk->sent) {
+ list_del(&cmd_blk->head);
+ mutex_unlock(&instance->lock);
+ vc_vchi_cmd_delete(instance, cmd_blk);
+ return -ENXIO;
+ }
+
+ list_move(&cmd_blk->head, &instance->dead_list);
+ mutex_unlock(&instance->lock);
+ complete(&instance->io_cmplt);
+ return -EINTR; /* We're done */
+ }
+
+ if (result && result_size) {
+ memcpy(result, cmd_blk->msg, result_size);
+ } else {
+ struct vc_sm_result_t *res =
+ (struct vc_sm_result_t *)cmd_blk->msg;
+ status = (res->success == 0) ? 0 : -ENXIO;
+ }
+
+ mutex_lock(&instance->lock);
+ list_del(&cmd_blk->head);
+ mutex_unlock(&instance->lock);
+ vc_vchi_cmd_delete(instance, cmd_blk);
+ return status;
+}
+
+int vc_sm_cma_vchi_free(struct sm_instance *handle, struct vc_sm_free_t *msg,
+ u32 *cur_trans_id)
+{
+ return vc_sm_cma_vchi_send_msg(handle, VC_SM_MSG_TYPE_FREE,
+ msg, sizeof(*msg), 0, 0, cur_trans_id, 0);
+}
+
+int vc_sm_cma_vchi_import(struct sm_instance *handle, struct vc_sm_import *msg,
+ struct vc_sm_import_result *result, u32 *cur_trans_id)
+{
+ return vc_sm_cma_vchi_send_msg(handle, VC_SM_MSG_TYPE_IMPORT,
+ msg, sizeof(*msg), result, sizeof(*result),
+ cur_trans_id, 1);
+}
+
+int vc_sm_cma_vchi_client_version(struct sm_instance *handle,
+ struct vc_sm_version *msg,
+ struct vc_sm_result_t *result,
+ u32 *cur_trans_id)
+{
+ return vc_sm_cma_vchi_send_msg(handle, VC_SM_MSG_TYPE_CLIENT_VERSION,
+ //msg, sizeof(*msg), result, sizeof(*result),
+ //cur_trans_id, 1);
+ msg, sizeof(*msg), NULL, 0,
+ cur_trans_id, 0);
+}
+
+int vc_sm_vchi_client_vc_mem_req_reply(struct sm_instance *handle,
+ struct vc_sm_vc_mem_request_result *msg,
+ uint32_t *cur_trans_id)
+{
+ return vc_sm_cma_vchi_send_msg(handle,
+ VC_SM_MSG_TYPE_VC_MEM_REQUEST_REPLY,
+ msg, sizeof(*msg), 0, 0, cur_trans_id,
+ 0);
+}
diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
new file mode 100644
index 000000000000..5dd23ad408fc
--- /dev/null
+++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * VideoCore Shared Memory CMA allocator
+ *
+ * Copyright: 2018, Raspberry Pi (Trading) Ltd
+ * Copyright 2011-2012 Broadcom Corporation. All rights reserved.
+ *
+ * Based on vmcs_sm driver from Broadcom Corporation.
+ *
+ */
+
+#ifndef __VC_SM_CMA_VCHI_H__INCLUDED__
+#define __VC_SM_CMA_VCHI_H__INCLUDED__
+
+#include "../include/linux/raspberrypi/vchiq.h"
+
+#include "vc_sm_defs.h"
+
+/*
+ * Forward declare.
+ */
+struct sm_instance;
+
+typedef void (*vpu_event_cb)(struct sm_instance *instance,
+ struct vc_sm_result_t *reply, int reply_len);
+
+/*
+ * Initialize the shared memory service, opens up vchi connection to talk to it.
+ */
+struct sm_instance *vc_sm_cma_vchi_init(struct vchiq_instance *vchi_instance,
+ unsigned int num_connections,
+ vpu_event_cb vpu_event);
+
+/*
+ * Terminates the shared memory service.
+ */
+int vc_sm_cma_vchi_stop(struct vchiq_instance *vchi_instance, struct sm_instance **handle);
+
+/*
+ * Ask the shared memory service to free up some memory that was previously
+ * allocated by the vc_sm_cma_vchi_alloc function call.
+ */
+int vc_sm_cma_vchi_free(struct sm_instance *handle, struct vc_sm_free_t *msg,
+ u32 *cur_trans_id);
+
+/*
+ * Import a contiguous block of memory and wrap it in a GPU MEM_HANDLE_T.
+ */
+int vc_sm_cma_vchi_import(struct sm_instance *handle, struct vc_sm_import *msg,
+ struct vc_sm_import_result *result,
+ u32 *cur_trans_id);
+
+int vc_sm_cma_vchi_client_version(struct sm_instance *handle,
+ struct vc_sm_version *msg,
+ struct vc_sm_result_t *result,
+ u32 *cur_trans_id);
+
+int vc_sm_vchi_client_vc_mem_req_reply(struct sm_instance *handle,
+ struct vc_sm_vc_mem_request_result *msg,
+ uint32_t *cur_trans_id);
+
+#endif /* __VC_SM_CMA_VCHI_H__INCLUDED__ */
diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
new file mode 100644
index 000000000000..ad4a3ec194d3
--- /dev/null
+++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
@@ -0,0 +1,187 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * VideoCore Shared Memory CMA allocator
+ *
+ * Copyright: 2018, Raspberry Pi (Trading) Ltd
+ *
+ * Based on vc_sm_defs.h from the vmcs_sm driver Copyright Broadcom Corporation.
+ * All IPC messages are copied across to this file, even if the vc-sm-cma
+ * driver is not currently using them.
+ *
+ ****************************************************************************
+ */
+
+#ifndef __VC_SM_DEFS_H__INCLUDED__
+#define __VC_SM_DEFS_H__INCLUDED__
+
+/* Maximum message length */
+#define VC_SM_MAX_MSG_LEN (sizeof(union vc_sm_msg_union_t) + \
+ sizeof(struct vc_sm_msg_hdr_t))
+#define VC_SM_MAX_RSP_LEN (sizeof(union vc_sm_msg_union_t))
+
+/* Resource name maximum size */
+#define VC_SM_RESOURCE_NAME 32
+
+/*
+ * Version to be reported to the VPU
+ * VPU assumes 0 (aka 1) which does not require the released callback, nor
+ * expect the client to handle VC_MEM_REQUESTS.
+ * Version 2 requires the released callback, and must support VC_MEM_REQUESTS.
+ */
+#define VC_SM_PROTOCOL_VERSION 2
+
+enum vc_sm_msg_type {
+ /* Message types supported for HOST->VC direction */
+
+ /* Allocate shared memory block */
+ VC_SM_MSG_TYPE_ALLOC,
+ /* Lock allocated shared memory block */
+ VC_SM_MSG_TYPE_LOCK,
+ /* Unlock allocated shared memory block */
+ VC_SM_MSG_TYPE_UNLOCK,
+ /* Unlock allocated shared memory block, do not answer command */
+ VC_SM_MSG_TYPE_UNLOCK_NOANS,
+ /* Free shared memory block */
+ VC_SM_MSG_TYPE_FREE,
+ /* Resize a shared memory block */
+ VC_SM_MSG_TYPE_RESIZE,
+ /* Walk the allocated shared memory block(s) */
+ VC_SM_MSG_TYPE_WALK_ALLOC,
+
+ /* A previously applied action will need to be reverted */
+ VC_SM_MSG_TYPE_ACTION_CLEAN,
+
+ /*
+ * Import a physical address and wrap into a MEM_HANDLE_T.
+ * Release with VC_SM_MSG_TYPE_FREE.
+ */
+ VC_SM_MSG_TYPE_IMPORT,
+ /*
+ *Tells VC the protocol version supported by this client.
+ * 2 supports the async/cmd messages from the VPU for final release
+ * of memory, and for VC allocations.
+ */
+ VC_SM_MSG_TYPE_CLIENT_VERSION,
+ /* Response to VC request for memory */
+ VC_SM_MSG_TYPE_VC_MEM_REQUEST_REPLY,
+
+ /*
+ * Asynchronous/cmd messages supported for VC->HOST direction.
+ * Signalled by setting the top bit in vc_sm_result_t trans_id.
+ */
+
+ /*
+ * VC has finished with an imported memory allocation.
+ * Release any Linux reference counts on the underlying block.
+ */
+ VC_SM_MSG_TYPE_RELEASED,
+ /* VC request for memory */
+ VC_SM_MSG_TYPE_VC_MEM_REQUEST,
+
+ VC_SM_MSG_TYPE_MAX
+};
+
+/* Type of memory to be allocated */
+enum vc_sm_alloc_type_t {
+ VC_SM_ALLOC_CACHED,
+ VC_SM_ALLOC_NON_CACHED,
+};
+
+/* Message header for all messages in HOST->VC direction */
+struct vc_sm_msg_hdr_t {
+ u32 type;
+ u32 trans_id;
+ u8 body[0];
+
+};
+
+/* Request to free a previously allocated memory (HOST->VC) */
+struct vc_sm_free_t {
+ /* Resource handle (returned from alloc) */
+ u32 res_handle;
+ /* Resource buffer (returned from alloc) */
+ u32 res_mem;
+
+};
+
+/* Generic result for a request (VC->HOST) */
+struct vc_sm_result_t {
+ /* Transaction identifier */
+ u32 trans_id;
+
+ s32 success;
+
+};
+
+/* Request to import memory (HOST->VC) */
+struct vc_sm_import {
+ /* type of memory to allocate */
+ enum vc_sm_alloc_type_t type;
+ /* pointer to the VC (ie physical) address of the allocated memory */
+ u32 addr;
+ /* size of buffer */
+ u32 size;
+ /* opaque handle returned in RELEASED messages */
+ u32 kernel_id;
+ /* Allocator identifier */
+ u32 allocator;
+ /* resource name (for easier tracking on vc side) */
+ char name[VC_SM_RESOURCE_NAME];
+};
+
+/* Result of a requested memory import (VC->HOST) */
+struct vc_sm_import_result {
+ /* Transaction identifier */
+ u32 trans_id;
+
+ /* Resource handle */
+ u32 res_handle;
+};
+
+/* Notification that VC has finished with an allocation (VC->HOST) */
+struct vc_sm_released {
+ /* cmd type / trans_id */
+ u32 cmd;
+
+ /* pointer to the VC (ie physical) address of the allocated memory */
+ u32 addr;
+ /* size of buffer */
+ u32 size;
+ /* opaque handle returned in RELEASED messages */
+ u32 kernel_id;
+ u32 vc_handle;
+};
+
+/*
+ * Client informing VC as to the protocol version it supports.
+ * >=2 requires the released callback, and supports VC asking for memory.
+ * Failure means that the firmware doesn't support this call, and therefore the
+ * client should either fail, or NOT rely on getting the released callback.
+ */
+struct vc_sm_version {
+ u32 version;
+};
+
+/* Response from the kernel to provide the VPU with some memory */
+struct vc_sm_vc_mem_request_result {
+ /* Transaction identifier for the VPU */
+ u32 trans_id;
+ /* pointer to the physical address of the allocated memory */
+ u32 addr;
+ /* opaque handle returned in RELEASED messages */
+ u32 kernel_id;
+};
+
+/* Union of ALL messages */
+union vc_sm_msg_union_t {
+ struct vc_sm_free_t free;
+ struct vc_sm_result_t result;
+ struct vc_sm_import import;
+ struct vc_sm_import_result import_result;
+ struct vc_sm_version version;
+ struct vc_sm_released released;
+ struct vc_sm_vc_mem_request_result vc_request_result;
+};
+
+#endif /* __VC_SM_DEFS_H__INCLUDED__ */
diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
new file mode 100644
index 000000000000..988fdd967922
--- /dev/null
+++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * VideoCore Shared Memory CMA allocator
+ *
+ * Copyright: 2018, Raspberry Pi (Trading) Ltd
+ *
+ * Based on vc_sm_defs.h from the vmcs_sm driver Copyright Broadcom Corporation.
+ *
+ */
+
+#ifndef __VC_SM_KNL_H__INCLUDED__
+#define __VC_SM_KNL_H__INCLUDED__
+
+#if !defined(__KERNEL__)
+#error "This interface is for kernel use only..."
+#endif
+
+/* Free a previously allocated or imported shared memory handle and block. */
+int vc_sm_cma_free(void *handle);
+
+/* Get an internal resource handle mapped from the external one. */
+int vc_sm_cma_int_handle(void *handle);
+
+/* Import a block of memory into the GPU space. */
+int vc_sm_cma_import_dmabuf(struct dma_buf *dmabuf, void **handle);
+
+#endif /* __VC_SM_KNL_H__INCLUDED__ */
--
2.41.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 02/15] staging: vc04_services: vchiq_arm: Register vcsm-cma driver
2023-11-09 21:02 [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Umang Jain
2023-11-09 21:02 ` [PATCH v2 01/15] staging: vc04_services: Add new vc-sm-cma driver Umang Jain
@ 2023-11-09 21:02 ` Umang Jain
2023-11-14 19:32 ` Stefan Wahren
2023-11-09 21:02 ` [PATCH v2 03/15] media: videobuf2: Allow exporting of a struct dmabuf Umang Jain
` (14 subsequent siblings)
16 siblings, 1 reply; 45+ messages in thread
From: Umang Jain @ 2023-11-09 21:02 UTC (permalink / raw)
To: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Stefan Wahren, Umang Jain
Register the vcsm-cma driver with the vchiq_bus_type.
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
---
drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
index 9fb8f657cc78..5a9073480dd3 100644
--- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
+++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
@@ -70,6 +70,7 @@ struct vchiq_state g_state;
*/
static struct vchiq_device *bcm2835_audio;
static struct vchiq_device *bcm2835_camera;
+static struct vchiq_device *vcsm_cma;
struct vchiq_drvdata {
const unsigned int cache_line_size;
@@ -1840,6 +1841,7 @@ static int vchiq_probe(struct platform_device *pdev)
goto error_exit;
}
+ vcsm_cma = vchiq_device_register(&pdev->dev, "vcsm-cma");
bcm2835_audio = vchiq_device_register(&pdev->dev, "bcm2835-audio");
bcm2835_camera = vchiq_device_register(&pdev->dev, "bcm2835-camera");
@@ -1855,6 +1857,7 @@ static void vchiq_remove(struct platform_device *pdev)
{
vchiq_device_unregister(bcm2835_audio);
vchiq_device_unregister(bcm2835_camera);
+ vchiq_device_unregister(vcsm_cma);
vchiq_debugfs_deinit();
vchiq_deregister_chrdev();
}
--
2.41.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 03/15] media: videobuf2: Allow exporting of a struct dmabuf
2023-11-09 21:02 [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Umang Jain
2023-11-09 21:02 ` [PATCH v2 01/15] staging: vc04_services: Add new vc-sm-cma driver Umang Jain
2023-11-09 21:02 ` [PATCH v2 02/15] staging: vc04_services: vchiq_arm: Register vcsm-cma driver Umang Jain
@ 2023-11-09 21:02 ` Umang Jain
2023-11-14 19:41 ` Stefan Wahren
2023-11-09 21:02 ` [PATCH v2 04/15] staging: mmal-vchiq: Add support for event callbacks Umang Jain
` (13 subsequent siblings)
16 siblings, 1 reply; 45+ messages in thread
From: Umang Jain @ 2023-11-09 21:02 UTC (permalink / raw)
To: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Stefan Wahren,
Dave Stevenson, Umang Jain
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
videobuf2 only allowed exporting a dmabuf as a file descriptor,
but there are instances where having the struct dma_buf is
useful within the kernel.
Split the current implementation into two, one step which
exports a struct dma_buf, and the second which converts that
into an fd.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
---
.../media/common/videobuf2/videobuf2-core.c | 36 +++++++++++++------
include/media/videobuf2-core.h | 15 ++++++++
2 files changed, 40 insertions(+), 11 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 27aee92f3eea..8ba85c2db375 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -2229,49 +2229,49 @@ static int __find_plane_by_offset(struct vb2_queue *q, unsigned long off,
return -EINVAL;
}
-int vb2_core_expbuf(struct vb2_queue *q, int *fd, unsigned int type,
- unsigned int index, unsigned int plane, unsigned int flags)
+struct dma_buf *vb2_core_expbuf_dmabuf(struct vb2_queue *q, unsigned int type,
+ unsigned int index, unsigned int plane,
+ unsigned int flags)
{
struct vb2_buffer *vb = NULL;
struct vb2_plane *vb_plane;
- int ret;
struct dma_buf *dbuf;
if (q->memory != VB2_MEMORY_MMAP) {
dprintk(q, 1, "queue is not currently set up for mmap\n");
- return -EINVAL;
+ return ERR_PTR(-EINVAL);
}
if (!q->mem_ops->get_dmabuf) {
dprintk(q, 1, "queue does not support DMA buffer exporting\n");
- return -EINVAL;
+ return ERR_PTR(-EINVAL);
}
if (flags & ~(O_CLOEXEC | O_ACCMODE)) {
dprintk(q, 1, "queue does support only O_CLOEXEC and access mode flags\n");
- return -EINVAL;
+ return ERR_PTR(-EINVAL);
}
if (type != q->type) {
dprintk(q, 1, "invalid buffer type\n");
- return -EINVAL;
+ return ERR_PTR(-EINVAL);
}
if (index >= q->num_buffers) {
dprintk(q, 1, "buffer index out of range\n");
- return -EINVAL;
+ return ERR_PTR(-EINVAL);
}
vb = q->bufs[index];
if (plane >= vb->num_planes) {
dprintk(q, 1, "buffer plane out of range\n");
- return -EINVAL;
+ return ERR_PTR(-EINVAL);
}
if (vb2_fileio_is_active(q)) {
dprintk(q, 1, "expbuf: file io in progress\n");
- return -EBUSY;
+ return ERR_PTR(-EBUSY);
}
vb_plane = &vb->planes[plane];
@@ -2283,9 +2283,23 @@ int vb2_core_expbuf(struct vb2_queue *q, int *fd, unsigned int type,
if (IS_ERR_OR_NULL(dbuf)) {
dprintk(q, 1, "failed to export buffer %d, plane %d\n",
index, plane);
- return -EINVAL;
+ return ERR_PTR(-EINVAL);
}
+ return dbuf;
+}
+EXPORT_SYMBOL_GPL(vb2_core_expbuf_dmabuf);
+
+int vb2_core_expbuf(struct vb2_queue *q, int *fd, unsigned int type,
+ unsigned int index, unsigned int plane, unsigned int flags)
+{
+ struct dma_buf *dbuf;
+ int ret;
+
+ dbuf = vb2_core_expbuf_dmabuf(q, type, index, plane, flags);
+ if (IS_ERR(dbuf))
+ return PTR_ERR(dbuf);
+
ret = dma_buf_fd(dbuf, flags & ~O_ACCMODE);
if (ret < 0) {
dprintk(q, 3, "buffer %d, plane %d failed to export (%d)\n",
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 4b6a9d2ea372..cba4e495f6a2 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -925,6 +925,21 @@ int vb2_core_streamon(struct vb2_queue *q, unsigned int type);
*/
int vb2_core_streamoff(struct vb2_queue *q, unsigned int type);
+/**
+ * vb2_core_expbuf_dmabuf() - Export a buffer as a dma_buf structure
+ * @q: videobuf2 queue
+ * @type: buffer type
+ * @index: id number of the buffer
+ * @plane: index of the plane to be exported, 0 for single plane queues
+ * @flags: flags for newly created file, currently only O_CLOEXEC is
+ * supported, refer to manual of open syscall for more details
+ *
+ * Return: Returns the dmabuf pointer
+ */
+struct dma_buf *vb2_core_expbuf_dmabuf(struct vb2_queue *q, unsigned int type,
+ unsigned int index, unsigned int plane,
+ unsigned int flags);
+
/**
* vb2_core_expbuf() - Export a buffer as a file descriptor.
* @q: pointer to &struct vb2_queue with videobuf2 queue.
--
2.41.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 04/15] staging: mmal-vchiq: Add support for event callbacks
2023-11-09 21:02 [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Umang Jain
` (2 preceding siblings ...)
2023-11-09 21:02 ` [PATCH v2 03/15] media: videobuf2: Allow exporting of a struct dmabuf Umang Jain
@ 2023-11-09 21:02 ` Umang Jain
2023-11-20 10:00 ` Dan Carpenter
2023-11-09 21:02 ` [PATCH v2 05/15] staging: mmal-vchiq: Use vc-sm-cma to support zero copy Umang Jain
` (12 subsequent siblings)
16 siblings, 1 reply; 45+ messages in thread
From: Umang Jain @ 2023-11-09 21:02 UTC (permalink / raw)
To: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Stefan Wahren,
Dave Stevenson, Umang Jain
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
The bcm2835-isp will use the event mechanism to report things such as
stream start event. It is signalled by the cmd field of the buffer
being non-zero.
Add support for passing this information out to the client.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
---
.../vc04_services/vchiq-mmal/mmal-common.h | 1 +
.../vc04_services/vchiq-mmal/mmal-msg.h | 35 ++++
.../vc04_services/vchiq-mmal/mmal-vchiq.c | 169 ++++++++++++++++--
.../vc04_services/vchiq-mmal/mmal-vchiq.h | 4 +
4 files changed, 195 insertions(+), 14 deletions(-)
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h
index b33129403a30..0443be8198ea 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h
@@ -50,6 +50,7 @@ struct mmal_buffer {
struct mmal_msg_context *msg_context;
+ u32 cmd; /* MMAL command. 0=data. */
unsigned long length;
u32 mmal_flags;
s64 dts;
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-msg.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-msg.h
index 471413248a14..ef84b4e35608 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-msg.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-msg.h
@@ -346,6 +346,41 @@ struct mmal_msg_port_parameter_get_reply {
/* event messages */
#define MMAL_WORKER_EVENT_SPACE 256
+/* Four CC's for events */
+#define MMAL_FOURCC(a, b, c, d) ((a) | (b << 8) | (c << 16) | (d << 24))
+
+#define MMAL_EVENT_ERROR MMAL_FOURCC('E', 'R', 'R', 'O')
+#define MMAL_EVENT_EOS MMAL_FOURCC('E', 'E', 'O', 'S')
+#define MMAL_EVENT_FORMAT_CHANGED MMAL_FOURCC('E', 'F', 'C', 'H')
+#define MMAL_EVENT_PARAMETER_CHANGED MMAL_FOURCC('E', 'P', 'C', 'H')
+
+/* Structs for each of the event message payloads */
+struct mmal_msg_event_eos {
+ u32 port_type; /**< Type of port that received the end of stream */
+ u32 port_index; /**< Index of port that received the end of stream */
+};
+
+/** Format changed event data. */
+struct mmal_msg_event_format_changed {
+ /* Minimum size of buffers the port requires */
+ u32 buffer_size_min;
+ /* Minimum number of buffers the port requires */
+ u32 buffer_num_min;
+ /* Size of buffers the port recommends for optimal performance.
+ * A value of zero means no special recommendation.
+ */
+ u32 buffer_size_recommended;
+ /* Number of buffers the port recommends for optimal
+ * performance. A value of zero means no special recommendation.
+ */
+ u32 buffer_num_recommended;
+
+ u32 es_ptr;
+ struct mmal_es_format format;
+ union mmal_es_specific_format es;
+ u8 extradata[MMAL_FORMAT_EXTRADATA_MAX_SIZE];
+};
+
struct mmal_msg_event_to_host {
u32 client_component; /* component context */
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
index 258aa0e37f55..6a270f308cf7 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
@@ -143,6 +143,8 @@ struct mmal_msg_context {
/* Presentation and Decode timestamps */
s64 pts;
s64 dts;
+ /* MMAL buffer command flag */
+ u32 cmd;
int status; /* context status */
@@ -230,18 +232,6 @@ release_msg_context(struct mmal_msg_context *msg_context)
kfree(msg_context);
}
-/* deals with receipt of event to host message */
-static void event_to_host_cb(struct vchiq_mmal_instance *instance,
- struct mmal_msg *msg, u32 msg_len)
-{
- pr_debug("unhandled event\n");
- pr_debug("component:%u port type:%d num:%d cmd:0x%x length:%d\n",
- msg->u.event_to_host.client_component,
- msg->u.event_to_host.port_type,
- msg->u.event_to_host.port_num,
- msg->u.event_to_host.cmd, msg->u.event_to_host.length);
-}
-
/* workqueue scheduled callback
*
* we do this because it is important we do not call any other vchiq
@@ -263,13 +253,18 @@ static void buffer_work_cb(struct work_struct *work)
buffer->mmal_flags = msg_context->u.bulk.mmal_flags;
buffer->dts = msg_context->u.bulk.dts;
buffer->pts = msg_context->u.bulk.pts;
+ buffer->cmd = msg_context->u.bulk.cmd;
- atomic_dec(&msg_context->u.bulk.port->buffers_with_vpu);
+ if (!buffer->cmd)
+ atomic_dec(&msg_context->u.bulk.port->buffers_with_vpu);
msg_context->u.bulk.port->buffer_cb(msg_context->u.bulk.instance,
msg_context->u.bulk.port,
msg_context->u.bulk.status,
msg_context->u.bulk.buffer);
+
+ if (buffer->cmd)
+ mutex_unlock(&msg_context->u.bulk.port->event_context_mutex);
}
/* workqueue scheduled callback to handle receiving buffers
@@ -347,6 +342,7 @@ static int bulk_receive(struct vchiq_mmal_instance *instance,
msg_context->u.bulk.buffer_used = rd_len;
msg_context->u.bulk.dts = msg->u.buffer_from_host.buffer_header.dts;
msg_context->u.bulk.pts = msg->u.buffer_from_host.buffer_header.pts;
+ msg_context->u.bulk.cmd = msg->u.buffer_from_host.buffer_header.cmd;
queue_work(msg_context->instance->bulk_wq,
&msg_context->u.bulk.buffer_to_host_work);
@@ -449,6 +445,102 @@ buffer_from_host(struct vchiq_mmal_instance *instance,
return ret;
}
+/* deals with receipt of event to host message */
+static void event_to_host_cb(struct vchiq_mmal_instance *instance,
+ struct mmal_msg *msg, u32 msg_len)
+{
+ int comp_idx = msg->u.event_to_host.client_component;
+ struct vchiq_mmal_component *component =
+ &instance->component[comp_idx];
+ struct vchiq_mmal_port *port = NULL;
+ struct mmal_msg_context *msg_context;
+ u32 port_num = msg->u.event_to_host.port_num;
+
+ if (msg->u.buffer_from_host.drvbuf.magic == MMAL_MAGIC) {
+ pr_err("%s: MMAL_MSG_TYPE_BUFFER_TO_HOST with bad magic\n",
+ __func__);
+ return;
+ }
+
+ switch (msg->u.event_to_host.port_type) {
+ case MMAL_PORT_TYPE_CONTROL:
+ if (port_num) {
+ pr_err("%s: port_num of %u >= number of ports 1",
+ __func__, port_num);
+ return;
+ }
+ port = &component->control;
+ break;
+ case MMAL_PORT_TYPE_INPUT:
+ if (port_num >= component->inputs) {
+ pr_err("%s: port_num of %u >= number of ports %u",
+ __func__, port_num,
+ port_num >= component->inputs);
+ return;
+ }
+ port = &component->input[port_num];
+ break;
+ case MMAL_PORT_TYPE_OUTPUT:
+ if (port_num >= component->outputs) {
+ pr_err("%s: port_num of %u >= number of ports %u",
+ __func__, port_num,
+ port_num >= component->outputs);
+ return;
+ }
+ port = &component->output[port_num];
+ break;
+ case MMAL_PORT_TYPE_CLOCK:
+ if (port_num >= component->clocks) {
+ pr_err("%s: port_num of %u >= number of ports %u",
+ __func__, port_num,
+ port_num >= component->clocks);
+ return;
+ }
+ port = &component->clock[port_num];
+ break;
+ default:
+ break;
+ }
+
+ if (!mutex_trylock(&port->event_context_mutex)) {
+ pr_err("dropping event 0x%x\n", msg->u.event_to_host.cmd);
+ return;
+ }
+ msg_context = port->event_context;
+
+ if (msg->h.status != MMAL_MSG_STATUS_SUCCESS) {
+ /* message reception had an error */
+ pr_err("%s: error %d in reply\n", __func__, msg->h.status);
+
+ msg_context->u.bulk.status = msg->h.status;
+ } else if (msg->u.event_to_host.length > MMAL_WORKER_EVENT_SPACE) {
+ /* data is not in message, queue a bulk receive */
+ pr_err("%s: payload not in message - bulk receive??! NOT SUPPORTED\n",
+ __func__);
+ msg_context->u.bulk.status = -1;
+ } else {
+ memcpy(msg_context->u.bulk.buffer->buffer,
+ msg->u.event_to_host.data,
+ msg->u.event_to_host.length);
+
+ msg_context->u.bulk.buffer_used =
+ msg->u.event_to_host.length;
+
+ msg_context->u.bulk.mmal_flags = 0;
+ msg_context->u.bulk.dts = MMAL_TIME_UNKNOWN;
+ msg_context->u.bulk.pts = MMAL_TIME_UNKNOWN;
+ msg_context->u.bulk.cmd = msg->u.event_to_host.cmd;
+
+ pr_debug("event component:%u port type:%d num:%d cmd:0x%x length:%d\n",
+ msg->u.event_to_host.client_component,
+ msg->u.event_to_host.port_type,
+ msg->u.event_to_host.port_num,
+ msg->u.event_to_host.cmd, msg->u.event_to_host.length);
+ }
+
+ schedule_work(&msg_context->u.bulk.work);
+}
+
/* deals with receipt of buffer to host message */
static void buffer_to_host_cb(struct vchiq_mmal_instance *instance,
struct mmal_msg *msg, u32 msg_len)
@@ -1329,6 +1421,7 @@ static int port_disable(struct vchiq_mmal_instance *instance,
mmalbuf->mmal_flags = 0;
mmalbuf->dts = MMAL_TIME_UNKNOWN;
mmalbuf->pts = MMAL_TIME_UNKNOWN;
+ mmalbuf->cmd = 0;
port->buffer_cb(instance,
port, 0, mmalbuf);
}
@@ -1630,6 +1723,43 @@ int mmal_vchi_buffer_cleanup(struct mmal_buffer *buf)
}
EXPORT_SYMBOL_GPL(mmal_vchi_buffer_cleanup);
+static void init_event_context(struct vchiq_mmal_instance *instance,
+ struct vchiq_mmal_port *port)
+{
+ struct mmal_msg_context *ctx = get_msg_context(instance);
+
+ mutex_init(&port->event_context_mutex);
+
+ port->event_context = ctx;
+ ctx->u.bulk.instance = instance;
+ ctx->u.bulk.port = port;
+ ctx->u.bulk.buffer =
+ kzalloc(sizeof(*ctx->u.bulk.buffer), GFP_KERNEL);
+ if (!ctx->u.bulk.buffer)
+ goto release_msg_context;
+ ctx->u.bulk.buffer->buffer = kzalloc(MMAL_WORKER_EVENT_SPACE,
+ GFP_KERNEL);
+ if (!ctx->u.bulk.buffer->buffer)
+ goto release_buffer;
+
+ INIT_WORK(&ctx->u.bulk.work, buffer_work_cb);
+ return;
+
+release_buffer:
+ kfree(ctx->u.bulk.buffer);
+release_msg_context:
+ release_msg_context(ctx);
+}
+
+static void free_event_context(struct vchiq_mmal_port *port)
+{
+ struct mmal_msg_context *ctx = port->event_context;
+
+ kfree(ctx->u.bulk.buffer->buffer);
+ kfree(ctx->u.bulk.buffer);
+ release_msg_context(ctx);
+}
+
/* Initialise a mmal component and its ports
*
*/
@@ -1679,6 +1809,7 @@ int vchiq_mmal_component_init(struct vchiq_mmal_instance *instance,
ret = port_info_get(instance, &component->control);
if (ret < 0)
goto release_component;
+ init_event_context(instance, &component->control);
for (idx = 0; idx < component->inputs; idx++) {
component->input[idx].type = MMAL_PORT_TYPE_INPUT;
@@ -1689,6 +1820,7 @@ int vchiq_mmal_component_init(struct vchiq_mmal_instance *instance,
ret = port_info_get(instance, &component->input[idx]);
if (ret < 0)
goto release_component;
+ init_event_context(instance, &component->input[idx]);
}
for (idx = 0; idx < component->outputs; idx++) {
@@ -1700,6 +1832,7 @@ int vchiq_mmal_component_init(struct vchiq_mmal_instance *instance,
ret = port_info_get(instance, &component->output[idx]);
if (ret < 0)
goto release_component;
+ init_event_context(instance, &component->output[idx]);
}
for (idx = 0; idx < component->clocks; idx++) {
@@ -1711,6 +1844,7 @@ int vchiq_mmal_component_init(struct vchiq_mmal_instance *instance,
ret = port_info_get(instance, &component->clock[idx]);
if (ret < 0)
goto release_component;
+ init_event_context(instance, &component->clock[idx]);
}
*component_out = component;
@@ -1736,7 +1870,7 @@ EXPORT_SYMBOL_GPL(vchiq_mmal_component_init);
int vchiq_mmal_component_finalise(struct vchiq_mmal_instance *instance,
struct vchiq_mmal_component *component)
{
- int ret;
+ int ret, idx;
if (mutex_lock_interruptible(&instance->vchiq_mutex))
return -EINTR;
@@ -1748,6 +1882,13 @@ int vchiq_mmal_component_finalise(struct vchiq_mmal_instance *instance,
component->in_use = false;
+ for (idx = 0; idx < component->inputs; idx++)
+ free_event_context(&component->input[idx]);
+ for (idx = 0; idx < component->outputs; idx++)
+ free_event_context(&component->output[idx]);
+ for (idx = 0; idx < component->clocks; idx++)
+ free_event_context(&component->clock[idx]);
+
mutex_unlock(&instance->vchiq_mutex);
return ret;
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
index 09f030919d4e..7be26cb39165 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
@@ -79,6 +79,10 @@ struct vchiq_mmal_port {
vchiq_mmal_buffer_cb buffer_cb;
/* callback context */
void *cb_ctx;
+
+ /* ensure serialised use of the one event context structure */
+ struct mutex event_context_mutex;
+ struct mmal_msg_context *event_context;
};
struct vchiq_mmal_component {
--
2.41.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 05/15] staging: mmal-vchiq: Use vc-sm-cma to support zero copy
2023-11-09 21:02 [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Umang Jain
` (3 preceding siblings ...)
2023-11-09 21:02 ` [PATCH v2 04/15] staging: mmal-vchiq: Add support for event callbacks Umang Jain
@ 2023-11-09 21:02 ` Umang Jain
2023-11-15 19:03 ` Stefan Wahren
2023-11-09 21:02 ` [PATCH v2 06/15] staging: mmal_vchiq: Add image formats to be used by bcm2835-isp Umang Jain
` (11 subsequent siblings)
16 siblings, 1 reply; 45+ messages in thread
From: Umang Jain @ 2023-11-09 21:02 UTC (permalink / raw)
To: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Stefan Wahren, Umang Jain
From: Dave Stevenson <dave.stevenson@raspberrypi.com>
With the vc-sm-cma driver we can support zero copy of buffers between
the kernel and VPU. Add this support to mmal-vchiq.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
---
.../staging/vc04_services/vchiq-mmal/Kconfig | 1 +
.../vc04_services/vchiq-mmal/mmal-common.h | 4 +
.../vc04_services/vchiq-mmal/mmal-vchiq.c | 84 +++++++++++++++++--
.../vc04_services/vchiq-mmal/mmal-vchiq.h | 1 +
4 files changed, 83 insertions(+), 7 deletions(-)
diff --git a/drivers/staging/vc04_services/vchiq-mmal/Kconfig b/drivers/staging/vc04_services/vchiq-mmal/Kconfig
index c99525a0bb45..a7c1a7bf516e 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/Kconfig
+++ b/drivers/staging/vc04_services/vchiq-mmal/Kconfig
@@ -1,6 +1,7 @@
config BCM2835_VCHIQ_MMAL
tristate "BCM2835 MMAL VCHIQ service"
depends on BCM2835_VCHIQ
+ select BCM_VC_SM_CMA
help
Enables the MMAL API over VCHIQ interface as used for the
majority of the multimedia services on VideoCore.
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h
index 0443be8198ea..809634924289 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h
@@ -50,6 +50,10 @@ struct mmal_buffer {
struct mmal_msg_context *msg_context;
+ struct dma_buf *dma_buf;/* Exported dmabuf fd from videobuf2 */
+ void *vcsm_handle; /* VCSM handle having imported the dmabuf */
+ u32 vc_handle; /* VC handle to that dmabuf */
+
u32 cmd; /* MMAL command. 0=data. */
unsigned long length;
u32 mmal_flags;
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
index 6a270f308cf7..890669f874d9 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
@@ -27,9 +27,12 @@
#include "../include/linux/raspberrypi/vchiq.h"
#include "mmal-common.h"
+#include "mmal-parameters.h"
#include "mmal-vchiq.h"
#include "mmal-msg.h"
+#include "../vc-sm-cma/vc_sm_knl.h"
+
/*
* maximum number of components supported.
* This matches the maximum permitted by default on the VPU
@@ -416,14 +419,27 @@ buffer_from_host(struct vchiq_mmal_instance *instance,
/* buffer header */
m.u.buffer_from_host.buffer_header.cmd = 0;
- m.u.buffer_from_host.buffer_header.data =
- (u32)(unsigned long)buf->buffer;
+ if (port->zero_copy) {
+ m.u.buffer_from_host.buffer_header.data = buf->vc_handle;
+ } else {
+ m.u.buffer_from_host.buffer_header.data =
+ (u32)(unsigned long)buf->buffer;
+ }
+
m.u.buffer_from_host.buffer_header.alloc_size = buf->buffer_size;
- m.u.buffer_from_host.buffer_header.length = 0; /* nothing used yet */
- m.u.buffer_from_host.buffer_header.offset = 0; /* no offset */
- m.u.buffer_from_host.buffer_header.flags = 0; /* no flags */
- m.u.buffer_from_host.buffer_header.pts = MMAL_TIME_UNKNOWN;
- m.u.buffer_from_host.buffer_header.dts = MMAL_TIME_UNKNOWN;
+ if (port->type == MMAL_PORT_TYPE_OUTPUT) {
+ m.u.buffer_from_host.buffer_header.length = 0;
+ m.u.buffer_from_host.buffer_header.offset = 0;
+ m.u.buffer_from_host.buffer_header.flags = 0;
+ m.u.buffer_from_host.buffer_header.pts = MMAL_TIME_UNKNOWN;
+ m.u.buffer_from_host.buffer_header.dts = MMAL_TIME_UNKNOWN;
+ } else {
+ m.u.buffer_from_host.buffer_header.length = buf->length;
+ m.u.buffer_from_host.buffer_header.offset = 0;
+ m.u.buffer_from_host.buffer_header.flags = buf->mmal_flags;
+ m.u.buffer_from_host.buffer_header.pts = buf->pts;
+ m.u.buffer_from_host.buffer_header.dts = buf->dts;
+ }
/* clear buffer type specific data */
memset(&m.u.buffer_from_host.buffer_header_type_specific, 0,
@@ -574,6 +590,21 @@ static void buffer_to_host_cb(struct vchiq_mmal_instance *instance,
msg_context->u.bulk.status = msg->h.status;
+ } else if (msg->u.buffer_from_host.is_zero_copy) {
+ /*
+ * Zero copy buffer, so nothing to do.
+ * Copy buffer info and make callback.
+ */
+ msg_context->u.bulk.buffer_used =
+ msg->u.buffer_from_host.buffer_header.length;
+ msg_context->u.bulk.mmal_flags =
+ msg->u.buffer_from_host.buffer_header.flags;
+ msg_context->u.bulk.dts =
+ msg->u.buffer_from_host.buffer_header.dts;
+ msg_context->u.bulk.pts =
+ msg->u.buffer_from_host.buffer_header.pts;
+ msg_context->u.bulk.cmd =
+ msg->u.buffer_from_host.buffer_header.cmd;
} else if (msg->u.buffer_from_host.buffer_header.length == 0) {
/* empty buffer */
if (msg->u.buffer_from_host.buffer_header.flags &
@@ -1518,6 +1549,9 @@ int vchiq_mmal_port_parameter_set(struct vchiq_mmal_instance *instance,
mutex_unlock(&instance->vchiq_mutex);
+ if (parameter == MMAL_PARAMETER_ZERO_COPY && !ret)
+ port->zero_copy = !!(*(bool *)value);
+
return ret;
}
EXPORT_SYMBOL_GPL(vchiq_mmal_port_parameter_set);
@@ -1686,6 +1720,31 @@ int vchiq_mmal_submit_buffer(struct vchiq_mmal_instance *instance,
unsigned long flags = 0;
int ret;
+ /*
+ * We really want to do this in mmal_vchi_buffer_init but can't as
+ * videobuf2 won't let us have the dmabuf there.
+ */
+ if (port->zero_copy && buffer->dma_buf && !buffer->vcsm_handle) {
+ pr_debug("%s: import dmabuf %p\n", __func__, buffer->dma_buf);
+ ret = vc_sm_cma_import_dmabuf(buffer->dma_buf,
+ &buffer->vcsm_handle);
+ if (ret) {
+ pr_err("%s: vc_sm_import_dmabuf_fd failed, ret %d\n",
+ __func__, ret);
+ return ret;
+ }
+
+ buffer->vc_handle = vc_sm_cma_int_handle(buffer->vcsm_handle);
+ if (!buffer->vc_handle) {
+ pr_err("%s: vc_sm_int_handle failed %d\n",
+ __func__, ret);
+ vc_sm_cma_free(buffer->vcsm_handle);
+ return ret;
+ }
+ pr_debug("%s: import dmabuf %p - got vc handle %08X\n",
+ __func__, buffer->dma_buf, buffer->vc_handle);
+ }
+
ret = buffer_from_host(instance, port, buffer);
if (ret == -EINVAL) {
/* Port is disabled. Queue for when it is enabled. */
@@ -1719,6 +1778,17 @@ int mmal_vchi_buffer_cleanup(struct mmal_buffer *buf)
release_msg_context(msg_context);
buf->msg_context = NULL;
+ if (buf->vcsm_handle) {
+ int ret;
+
+ pr_debug("%s: vc_sm_cma_free on handle %p\n", __func__,
+ buf->vcsm_handle);
+ ret = vc_sm_cma_free(buf->vcsm_handle);
+ if (ret)
+ pr_err("%s: vcsm_free failed, ret %d\n", __func__, ret);
+ buf->vcsm_handle = 0;
+ }
+
return 0;
}
EXPORT_SYMBOL_GPL(mmal_vchi_buffer_cleanup);
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
index 7be26cb39165..7aeb38c0c94f 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
@@ -49,6 +49,7 @@ typedef void (*vchiq_mmal_buffer_cb)(
struct vchiq_mmal_port {
bool enabled;
+ u32 zero_copy;
u32 handle;
u32 type; /* port type, cached to use on port info set */
u32 index; /* port index, cached to use on port info set */
--
2.41.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 06/15] staging: mmal_vchiq: Add image formats to be used by bcm2835-isp
2023-11-09 21:02 [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Umang Jain
` (4 preceding siblings ...)
2023-11-09 21:02 ` [PATCH v2 05/15] staging: mmal-vchiq: Use vc-sm-cma to support zero copy Umang Jain
@ 2023-11-09 21:02 ` Umang Jain
2023-11-09 21:02 ` [PATCH v2 07/15] media: uapi: v4l2-core: Add ISP statistics output V4L2 fourcc type Umang Jain
` (10 subsequent siblings)
16 siblings, 0 replies; 45+ messages in thread
From: Umang Jain @ 2023-11-09 21:02 UTC (permalink / raw)
To: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Stefan Wahren, Umang Jain
From: Dave Stevenson <dave.stevenson@raspberrypi.com>
The bcm2835-isp supports Bayer, so add in the encodings for them.
Also, Add support for monochrome image formats in the various MIPI
packings.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
---
.../vc04_services/vchiq-mmal/mmal-encodings.h | 62 +++++++++++++++++++
1 file changed, 62 insertions(+)
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
index e15ae7b24f73..4711877a9711 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
@@ -69,6 +69,68 @@
*/
#define MMAL_ENCODING_OPAQUE MMAL_FOURCC('O', 'P', 'Q', 'V')
+/* Bayer formats
+ * FourCC values copied from V4L2 where defined.
+ */
+/* 8 bit per pixel Bayer formats. */
+#define MMAL_ENCODING_BAYER_SBGGR8 MMAL_FOURCC('B', 'A', '8', '1')
+#define MMAL_ENCODING_BAYER_SGBRG8 MMAL_FOURCC('G', 'B', 'R', 'G')
+#define MMAL_ENCODING_BAYER_SGRBG8 MMAL_FOURCC('G', 'R', 'B', 'G')
+#define MMAL_ENCODING_BAYER_SRGGB8 MMAL_FOURCC('R', 'G', 'G', 'B')
+
+/* 10 bit per pixel packed Bayer formats. */
+#define MMAL_ENCODING_BAYER_SBGGR10P MMAL_FOURCC('p', 'B', 'A', 'A')
+#define MMAL_ENCODING_BAYER_SGRBG10P MMAL_FOURCC('p', 'g', 'A', 'A')
+#define MMAL_ENCODING_BAYER_SGBRG10P MMAL_FOURCC('p', 'G', 'A', 'A')
+#define MMAL_ENCODING_BAYER_SRGGB10P MMAL_FOURCC('p', 'R', 'A', 'A')
+
+/* 12 bit per pixel packed Bayer formats. */
+#define MMAL_ENCODING_BAYER_SBGGR12P MMAL_FOURCC('p', 'B', '1', '2')
+#define MMAL_ENCODING_BAYER_SGRBG12P MMAL_FOURCC('p', 'g', '1', '2')
+#define MMAL_ENCODING_BAYER_SGBRG12P MMAL_FOURCC('p', 'G', '1', '2')
+#define MMAL_ENCODING_BAYER_SRGGB12P MMAL_FOURCC('p', 'R', '1', '2')
+
+/* 14 bit per pixel Bayer formats. */
+#define MMAL_ENCODING_BAYER_SBGGR14P MMAL_FOURCC('p', 'B', 'E', 'E')
+#define MMAL_ENCODING_BAYER_SGBRG14P MMAL_FOURCC('p', 'G', 'E', 'E')
+#define MMAL_ENCODING_BAYER_SGRBG14P MMAL_FOURCC('p', 'g', 'E', 'E')
+#define MMAL_ENCODING_BAYER_SRGGB14P MMAL_FOURCC('p', 'R', 'E', 'E')
+
+/* 16 bit per pixel Bayer formats. */
+#define MMAL_ENCODING_BAYER_SBGGR16 MMAL_FOURCC('B', 'G', '1', '6')
+#define MMAL_ENCODING_BAYER_SGBRG16 MMAL_FOURCC('G', 'B', '1', '6')
+#define MMAL_ENCODING_BAYER_SGRBG16 MMAL_FOURCC('G', 'R', '1', '6')
+#define MMAL_ENCODING_BAYER_SRGGB16 MMAL_FOURCC('R', 'G', '1', '6')
+
+/* 10 bit per pixel unpacked (16bit) Bayer formats. */
+#define MMAL_ENCODING_BAYER_SBGGR10 MMAL_FOURCC('B', 'G', '1', '0')
+#define MMAL_ENCODING_BAYER_SGRBG10 MMAL_FOURCC('B', 'A', '1', '0')
+#define MMAL_ENCODING_BAYER_SGBRG10 MMAL_FOURCC('G', 'B', '1', '0')
+#define MMAL_ENCODING_BAYER_SRGGB10 MMAL_FOURCC('R', 'G', '1', '0')
+
+/* 12 bit per pixel unpacked (16bit) Bayer formats */
+#define MMAL_ENCODING_BAYER_SBGGR12 MMAL_FOURCC('B', 'G', '1', '2')
+#define MMAL_ENCODING_BAYER_SGRBG12 MMAL_FOURCC('B', 'A', '1', '2')
+#define MMAL_ENCODING_BAYER_SGBRG12 MMAL_FOURCC('G', 'B', '1', '2')
+#define MMAL_ENCODING_BAYER_SRGGB12 MMAL_FOURCC('R', 'G', '1', '2')
+
+/* 14 bit per pixel unpacked (16bit) Bayer formats */
+#define MMAL_ENCODING_BAYER_SBGGR14 MMAL_FOURCC('B', 'G', '1', '4')
+#define MMAL_ENCODING_BAYER_SGBRG14 MMAL_FOURCC('G', 'B', '1', '4')
+#define MMAL_ENCODING_BAYER_SGRBG14 MMAL_FOURCC('G', 'R', '1', '4')
+#define MMAL_ENCODING_BAYER_SRGGB14 MMAL_FOURCC('R', 'G', '1', '4')
+
+/* MIPI packed monochrome images */
+#define MMAL_ENCODING_GREY MMAL_FOURCC('G', 'R', 'E', 'Y')
+#define MMAL_ENCODING_Y10P MMAL_FOURCC('Y', '1', '0', 'P')
+#define MMAL_ENCODING_Y12P MMAL_FOURCC('Y', '1', '2', 'P')
+#define MMAL_ENCODING_Y14P MMAL_FOURCC('Y', '1', '4', 'P')
+#define MMAL_ENCODING_Y16 MMAL_FOURCC('Y', '1', '6', ' ')
+/* Unpacked monochrome formats (16bit per sample, but only N LSBs used) */
+#define MMAL_ENCODING_Y10 MMAL_FOURCC('Y', '1', '0', ' ')
+#define MMAL_ENCODING_Y12 MMAL_FOURCC('Y', '1', '2', ' ')
+#define MMAL_ENCODING_Y14 MMAL_FOURCC('Y', '1', '4', ' ')
+
/** An EGL image handle
*/
#define MMAL_ENCODING_EGL_IMAGE MMAL_FOURCC('E', 'G', 'L', 'I')
--
2.41.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 07/15] media: uapi: v4l2-core: Add ISP statistics output V4L2 fourcc type
2023-11-09 21:02 [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Umang Jain
` (5 preceding siblings ...)
2023-11-09 21:02 ` [PATCH v2 06/15] staging: mmal_vchiq: Add image formats to be used by bcm2835-isp Umang Jain
@ 2023-11-09 21:02 ` Umang Jain
2023-11-18 13:11 ` Stefan Wahren
2023-11-09 21:03 ` [PATCH v2 08/15] uapi: bcm2835-isp: Add bcm2835-isp uapi header file Umang Jain
` (9 subsequent siblings)
16 siblings, 1 reply; 45+ messages in thread
From: Umang Jain @ 2023-11-09 21:02 UTC (permalink / raw)
To: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Stefan Wahren,
Naushir Patuck, Umang Jain
From: Naushir Patuck <naush@raspberrypi.com>
Add V4L2_META_FMT_BCM2835_ISP_STATS V4L2 format type.
This new format will be used by the bcm2835-isp device to return
out ISP statistics for 3A.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
---
.../userspace-api/media/v4l/meta-formats.rst | 1 +
.../v4l/pixfmt-meta-bcm2835-isp-stats.rst | 32 +++++++++++++++++++
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/uapi/linux/videodev2.h | 1 +
4 files changed, 35 insertions(+)
create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst
index 919f595576b9..534542f235bc 100644
--- a/Documentation/userspace-api/media/v4l/meta-formats.rst
+++ b/Documentation/userspace-api/media/v4l/meta-formats.rst
@@ -12,6 +12,7 @@ These formats are used for the :ref:`metadata` interface only.
.. toctree::
:maxdepth: 1
+ pixfmt-meta-bcm2835-isp-stats
metafmt-d4xx
metafmt-intel-ipu3
metafmt-rkisp1
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst b/Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
new file mode 100644
index 000000000000..2219235e0ecb
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
@@ -0,0 +1,32 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+.. _v4l2-meta-fmt-bcm2835-isp-stats:
+
++*****************************************
+V4L2_META_FMT_BCM2835_ISP_STATS ('BSTA')
+*****************************************
+
+BCM2835 ISP Statistics
+
+Description
+===========
+
+The BCM2835 ISP hardware calculate image statistics for an input Bayer frame.
+These statistics are obtained from the "bcm2835-isp0-capture3" device node
+using the :c:type:`v4l2_meta_format` interface. They are formatted as described
+by the :c:type:`bcm2835_isp_stats` structure below.
+
+.. code-block:: c
+
+ #define DEFAULT_AWB_REGIONS_X 16
+ #define DEFAULT_AWB_REGIONS_Y 12
+
+ #define NUM_HISTOGRAMS 2
+ #define NUM_HISTOGRAM_BINS 128
+ #define AWB_REGIONS (DEFAULT_AWB_REGIONS_X * DEFAULT_AWB_REGIONS_Y)
+ #define FLOATING_REGIONS 16
+ #define AGC_REGIONS 16
+ #define FOCUS_REGIONS 12
+
+.. kernel-doc:: include/uapi/linux/bcm2835-isp.h
+ :functions: bcm2835_isp_stats_hist bcm2835_isp_stats_region
+ bcm2835_isp_stats_focus bcm2835_isp_stats
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 7f29886568b8..6f0f0f6526c9 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1463,6 +1463,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_META_FMT_GENERIC_CSI2_20: descr = "8b Generic Meta, 20b CSI-2"; break;
case V4L2_META_FMT_GENERIC_CSI2_24: descr = "8b Generic Meta, 24b CSI-2"; break;
case V4L2_META_FMT_GENERIC_CSI2_2_24: descr = "2x8b Generic Meta, 24b CSI-2"; break;
+ case V4L2_META_FMT_BCM2835_ISP_STATS: descr = "BCM2835 ISP Image Statistics"; break;
default:
/* Compressed formats */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 85b60199c618..eddaea8598db 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -836,6 +836,7 @@ struct v4l2_pix_format {
#define V4L2_META_FMT_UVC v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
#define V4L2_META_FMT_D4XX v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
#define V4L2_META_FMT_VIVID v4l2_fourcc('V', 'I', 'V', 'D') /* Vivid Metadata */
+#define V4L2_META_FMT_BCM2835_ISP_STATS v4l2_fourcc('B', 'S', 'T', 'A') /* BCM2835 ISP image statistics output */
/* Vendor specific - used for RK_ISP1 camera sub-system */
#define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
--
2.41.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 08/15] uapi: bcm2835-isp: Add bcm2835-isp uapi header file
2023-11-09 21:02 [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Umang Jain
` (6 preceding siblings ...)
2023-11-09 21:02 ` [PATCH v2 07/15] media: uapi: v4l2-core: Add ISP statistics output V4L2 fourcc type Umang Jain
@ 2023-11-09 21:03 ` Umang Jain
2023-11-09 21:03 ` [PATCH v2 09/15] staging: vc04_services: Add helpers for vchiq driver data Umang Jain
` (8 subsequent siblings)
16 siblings, 0 replies; 45+ messages in thread
From: Umang Jain @ 2023-11-09 21:03 UTC (permalink / raw)
To: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Stefan Wahren,
Naushir Patuck, Umang Jain
From: Dave Stevenson <dave.stevenson@raspberrypi.com>
This file defines the userland interface to the bcm2835-isp driver
that will follow in a separate commit.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
---
include/uapi/linux/bcm2835-isp.h | 320 +++++++++++++++++++++++++++++
include/uapi/linux/v4l2-controls.h | 5 +
2 files changed, 325 insertions(+)
create mode 100644 include/uapi/linux/bcm2835-isp.h
diff --git a/include/uapi/linux/bcm2835-isp.h b/include/uapi/linux/bcm2835-isp.h
new file mode 100644
index 000000000000..cf8c0437f159
--- /dev/null
+++ b/include/uapi/linux/bcm2835-isp.h
@@ -0,0 +1,320 @@
+/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) */
+/*
+ * bcm2835-isp.h
+ *
+ * BCM2835 ISP driver - user space header file.
+ *
+ * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
+ *
+ * Author: Naushir Patuck (naush@raspberrypi.com)
+ *
+ */
+
+#ifndef __BCM2835_ISP_H_
+#define __BCM2835_ISP_H_
+
+#include <linux/v4l2-controls.h>
+
+#define V4L2_CID_USER_BCM2835_ISP_CC_MATRIX \
+ (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0001)
+#define V4L2_CID_USER_BCM2835_ISP_LENS_SHADING \
+ (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0002)
+#define V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL \
+ (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0003)
+#define V4L2_CID_USER_BCM2835_ISP_GEQ \
+ (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0004)
+#define V4L2_CID_USER_BCM2835_ISP_GAMMA \
+ (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0005)
+#define V4L2_CID_USER_BCM2835_ISP_DENOISE \
+ (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0006)
+#define V4L2_CID_USER_BCM2835_ISP_SHARPEN \
+ (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0007)
+#define V4L2_CID_USER_BCM2835_ISP_DPC \
+ (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0008)
+
+/*
+ * All structs below are directly mapped onto the equivalent structs in
+ * drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
+ * for convenience.
+ */
+
+/**
+ * struct bcm2835_isp_rational - Rational value type.
+ *
+ * @num: Numerator.
+ * @den: Denominator.
+ */
+struct bcm2835_isp_rational {
+ __s32 num;
+ __u32 den;
+};
+
+/**
+ * struct bcm2835_isp_ccm - Colour correction matrix.
+ *
+ * @ccm: 3x3 correction matrix coefficients.
+ * @offsets: 1x3 correction offsets.
+ */
+struct bcm2835_isp_ccm {
+ struct bcm2835_isp_rational ccm[3][3];
+ __s32 offsets[3];
+};
+
+/**
+ * struct bcm2835_isp_custom_ccm - Custom CCM applied with the
+ * V4L2_CID_USER_BCM2835_ISP_CC_MATRIX ctrl.
+ *
+ * @enabled: Enable custom CCM.
+ * @ccm: Custom CCM coefficients and offsets.
+ */
+struct bcm2835_isp_custom_ccm {
+ __u32 enabled;
+ struct bcm2835_isp_ccm ccm;
+};
+
+/**
+ * enum bcm2835_isp_gain_format - format of the gains in the lens shading
+ * tables used with the
+ * V4L2_CID_USER_BCM2835_ISP_LENS_SHADING ctrl.
+ *
+ * @GAIN_FORMAT_U0P8_1: Gains are u0.8 format, starting at 1.0
+ * @GAIN_FORMAT_U1P7_0: Gains are u1.7 format, starting at 0.0
+ * @GAIN_FORMAT_U1P7_1: Gains are u1.7 format, starting at 1.0
+ * @GAIN_FORMAT_U2P6_0: Gains are u2.6 format, starting at 0.0
+ * @GAIN_FORMAT_U2P6_1: Gains are u2.6 format, starting at 1.0
+ * @GAIN_FORMAT_U3P5_0: Gains are u3.5 format, starting at 0.0
+ * @GAIN_FORMAT_U3P5_1: Gains are u3.5 format, starting at 1.0
+ * @GAIN_FORMAT_U4P10: Gains are u4.10 format, starting at 0.0
+ */
+enum bcm2835_isp_gain_format {
+ GAIN_FORMAT_U0P8_1 = 0,
+ GAIN_FORMAT_U1P7_0 = 1,
+ GAIN_FORMAT_U1P7_1 = 2,
+ GAIN_FORMAT_U2P6_0 = 3,
+ GAIN_FORMAT_U2P6_1 = 4,
+ GAIN_FORMAT_U3P5_0 = 5,
+ GAIN_FORMAT_U3P5_1 = 6,
+ GAIN_FORMAT_U4P10 = 7,
+};
+
+/**
+ * struct bcm2835_isp_lens_shading - Lens shading tables supplied with the
+ * V4L2_CID_USER_BCM2835_ISP_LENS_SHADING
+ * ctrl.
+ *
+ * @enabled: Enable lens shading.
+ * @grid_cell_size: Size of grid cells in samples (16, 32, 64, 128 or 256).
+ * @grid_width: Width of lens shading tables in grid cells.
+ * @grid_stride: Row to row distance (in grid cells) between grid cells
+ * in the same horizontal location.
+ * @grid_height: Height of lens shading tables in grid cells.
+ * @dmabuf: dmabuf file handle containing the table.
+ * @ref_transform: Reference transform - unsupported, please pass zero.
+ * @corner_sampled: Whether the gains are sampled at the corner points
+ * of the grid cells or in the cell centres.
+ * @gain_format: Format of the gains (see enum &bcm2835_isp_gain_format).
+ */
+struct bcm2835_isp_lens_shading {
+ __u32 enabled;
+ __u32 grid_cell_size;
+ __u32 grid_width;
+ __u32 grid_stride;
+ __u32 grid_height;
+ __s32 dmabuf;
+ __u32 ref_transform;
+ __u32 corner_sampled;
+ __u32 gain_format;
+};
+
+/**
+ * struct bcm2835_isp_black_level - Sensor black level set with the
+ * V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL ctrl.
+ *
+ * @enabled: Enable black level.
+ * @black_level_r: Black level for red channel.
+ * @black_level_g: Black level for green channels.
+ * @black_level_b: Black level for blue channel.
+ */
+struct bcm2835_isp_black_level {
+ __u32 enabled;
+ __u16 black_level_r;
+ __u16 black_level_g;
+ __u16 black_level_b;
+ __u8 padding[2]; /* Unused */
+};
+
+/**
+ * struct bcm2835_isp_geq - Green equalisation parameters set with the
+ * V4L2_CID_USER_BCM2835_ISP_GEQ ctrl.
+ *
+ * @enabled: Enable green equalisation.
+ * @offset: Fixed offset of the green equalisation threshold.
+ * @slope: Slope of the green equalisation threshold.
+ */
+struct bcm2835_isp_geq {
+ __u32 enabled;
+ __u32 offset;
+ struct bcm2835_isp_rational slope;
+};
+
+#define BCM2835_NUM_GAMMA_PTS 33
+
+/**
+ * struct bcm2835_isp_gamma - Gamma parameters set with the
+ * V4L2_CID_USER_BCM2835_ISP_GAMMA ctrl.
+ *
+ * @enabled: Enable gamma adjustment.
+ * @X: X values of the points defining the gamma curve.
+ * Values should be scaled to 16 bits.
+ * @Y: Y values of the points defining the gamma curve.
+ * Values should be scaled to 16 bits.
+ */
+struct bcm2835_isp_gamma {
+ __u32 enabled;
+ __u16 x[BCM2835_NUM_GAMMA_PTS];
+ __u16 y[BCM2835_NUM_GAMMA_PTS];
+};
+
+/**
+ * struct bcm2835_isp_denoise - Denoise parameters set with the
+ * V4L2_CID_USER_BCM2835_ISP_DENOISE ctrl.
+ *
+ * @enabled: Enable denoise.
+ * @constant: Fixed offset of the noise threshold.
+ * @slope: Slope of the noise threshold.
+ * @strength: Denoise strength between 0.0 (off) and 1.0 (maximum).
+ */
+struct bcm2835_isp_denoise {
+ __u32 enabled;
+ __u32 constant;
+ struct bcm2835_isp_rational slope;
+ struct bcm2835_isp_rational strength;
+};
+
+/**
+ * struct bcm2835_isp_sharpen - Sharpen parameters set with the
+ * V4L2_CID_USER_BCM2835_ISP_SHARPEN ctrl.
+ *
+ * @enabled: Enable sharpening.
+ * @threshold: Threshold at which to start sharpening pixels.
+ * @strength: Strength with which pixel sharpening increases.
+ * @limit: Limit to the amount of sharpening applied.
+ */
+struct bcm2835_isp_sharpen {
+ __u32 enabled;
+ struct bcm2835_isp_rational threshold;
+ struct bcm2835_isp_rational strength;
+ struct bcm2835_isp_rational limit;
+};
+
+/**
+ * enum bcm2835_isp_dpc_mode - defective pixel correction (DPC) strength.
+ *
+ * @DPC_MODE_OFF: No DPC.
+ * @DPC_MODE_NORMAL: Normal DPC.
+ * @DPC_MODE_STRONG: Strong DPC.
+ */
+enum bcm2835_isp_dpc_mode {
+ DPC_MODE_OFF = 0,
+ DPC_MODE_NORMAL = 1,
+ DPC_MODE_STRONG = 2,
+};
+
+/**
+ * struct bcm2835_isp_dpc - Defective pixel correction (DPC) parameters set
+ * with the V4L2_CID_USER_BCM2835_ISP_DPC ctrl.
+ *
+ * @enabled: Enable DPC.
+ * @strength: DPC strength (see enum &bcm2835_isp_dpc_mode).
+ */
+struct bcm2835_isp_dpc {
+ __u32 enabled;
+ __u32 strength;
+};
+
+/*
+ * ISP statistics structures.
+ *
+ * The bcm2835_isp_stats structure is generated at the output of the
+ * statistics node. Note that this does not directly map onto the statistics
+ * output of the ISP HW. Instead, the MMAL firmware code maps the HW statistics
+ * to the bcm2835_isp_stats structure.
+ */
+#define DEFAULT_AWB_REGIONS_X 16
+#define DEFAULT_AWB_REGIONS_Y 12
+
+#define NUM_HISTOGRAMS 2
+#define NUM_HISTOGRAM_BINS 128
+#define AWB_REGIONS (DEFAULT_AWB_REGIONS_X * DEFAULT_AWB_REGIONS_Y)
+#define FLOATING_REGIONS 16
+#define AGC_REGIONS 16
+#define FOCUS_REGIONS 12
+
+/**
+ * struct bcm2835_isp_stats_hist - Histogram statistics
+ *
+ * @r_hist: Red channel histogram.
+ * @g_hist: Combined green channel histogram.
+ * @b_hist: Blue channel histogram.
+ */
+struct bcm2835_isp_stats_hist {
+ __u32 r_hist[NUM_HISTOGRAM_BINS];
+ __u32 g_hist[NUM_HISTOGRAM_BINS];
+ __u32 b_hist[NUM_HISTOGRAM_BINS];
+};
+
+/**
+ * struct bcm2835_isp_stats_region - Region sums.
+ *
+ * @counted: The number of 2x2 bayer tiles accumulated.
+ * @notcounted: The number of 2x2 bayer tiles not accumulated.
+ * @r_sum: Total sum of counted pixels in the red channel for a region.
+ * @g_sum: Total sum of counted pixels in the green channel for a region.
+ * @b_sum: Total sum of counted pixels in the blue channel for a region.
+ */
+struct bcm2835_isp_stats_region {
+ __u32 counted;
+ __u32 notcounted;
+ __u64 r_sum;
+ __u64 g_sum;
+ __u64 b_sum;
+};
+
+/**
+ * struct bcm2835_isp_stats_focus - Focus statistics.
+ *
+ * @contrast_val: Focus measure - accumulated output of the focus filter.
+ * In the first dimension, index [0] counts pixels below a
+ * preset threshold, and index [1] counts pixels above the
+ * threshold. In the second dimension, index [0] uses the
+ * first predefined filter, and index [1] uses the second
+ * predefined filter.
+ * @contrast_val_num: The number of counted pixels in the above accumulation.
+ */
+struct bcm2835_isp_stats_focus {
+ __u64 contrast_val[2][2];
+ __u32 contrast_val_num[2][2];
+};
+
+/**
+ * struct bcm2835_isp_stats - ISP statistics.
+ *
+ * @version: Version of the bcm2835_isp_stats structure.
+ * @size: Size of the bcm2835_isp_stats structure.
+ * @hist: Histogram statistics for the entire image.
+ * @awb_stats: Statistics for the regions defined for AWB calculations.
+ * @floating_stats: Statistics for arbitrarily placed (floating) regions.
+ * @agc_stats: Statistics for the regions defined for AGC calculations.
+ * @focus_stats: Focus filter statistics for the focus regions.
+ */
+struct bcm2835_isp_stats {
+ __u32 version;
+ __u32 size;
+ struct bcm2835_isp_stats_hist hist[NUM_HISTOGRAMS];
+ struct bcm2835_isp_stats_region awb_stats[AWB_REGIONS];
+ struct bcm2835_isp_stats_region floating_stats[FLOATING_REGIONS];
+ struct bcm2835_isp_stats_region agc_stats[AGC_REGIONS];
+ struct bcm2835_isp_stats_focus focus_stats[FOCUS_REGIONS];
+};
+
+#endif /* __BCM2835_ISP_H_ */
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 68db66d4aae8..31d7b6fd7a2b 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -209,6 +209,11 @@ enum v4l2_colorfx {
*/
#define V4L2_CID_USER_NPCM_BASE (V4L2_CID_USER_BASE + 0x11b0)
+/* The base for the bcm2835-isp driver controls.
+ * We reserve 16 controls for this driver.
+ */
+#define V4L2_CID_USER_BCM2835_ISP_BASE (V4L2_CID_USER_BASE + 0x11c0)
+
/* MPEG-class control IDs */
/* The MPEG controls are applicable to all codec controls
* and the 'MPEG' part of the define is historical */
--
2.41.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 09/15] staging: vc04_services: Add helpers for vchiq driver data
2023-11-09 21:02 [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Umang Jain
` (7 preceding siblings ...)
2023-11-09 21:03 ` [PATCH v2 08/15] uapi: bcm2835-isp: Add bcm2835-isp uapi header file Umang Jain
@ 2023-11-09 21:03 ` Umang Jain
2023-11-18 14:43 ` Stefan Wahren
2023-11-09 21:03 ` [PATCH v2 10/15] staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component Umang Jain
` (7 subsequent siblings)
16 siblings, 1 reply; 45+ messages in thread
From: Umang Jain @ 2023-11-09 21:03 UTC (permalink / raw)
To: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Stefan Wahren, Umang Jain
Add helpers to set and get vchiq driver data.
vchiq_set_drvdata() and vchiq_get_drvdata() wraps
dev_set_drvdata() and dev_get_drvdata() respectively.
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
---
.../vc04_services/interface/vchiq_arm/vchiq_bus.h | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_bus.h b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_bus.h
index caa6fdf25bb1..800f53bb48c2 100644
--- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_bus.h
+++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_bus.h
@@ -34,6 +34,16 @@ static inline struct vchiq_driver *to_vchiq_driver(struct device_driver *d)
return container_of(d, struct vchiq_driver, driver);
}
+static inline void *vchiq_get_drvdata(const struct vchiq_device *device)
+{
+ return dev_get_drvdata(&device->dev);
+}
+
+static inline void vchiq_set_drvdata(struct vchiq_device *device, void *data)
+{
+ dev_set_drvdata(&device->dev, data);
+}
+
extern struct bus_type vchiq_bus_type;
struct vchiq_device *
--
2.41.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 10/15] staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component
2023-11-09 21:02 [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Umang Jain
` (8 preceding siblings ...)
2023-11-09 21:03 ` [PATCH v2 09/15] staging: vc04_services: Add helpers for vchiq driver data Umang Jain
@ 2023-11-09 21:03 ` Umang Jain
2023-11-13 14:07 ` [PATCH] fixup! " Umang Jain
` (2 more replies)
2023-11-09 21:03 ` [PATCH v2 11/15] vc04_services: bcm2835-isp: Allow formats with different colour spaces Umang Jain
` (6 subsequent siblings)
16 siblings, 3 replies; 45+ messages in thread
From: Umang Jain @ 2023-11-09 21:03 UTC (permalink / raw)
To: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Stefan Wahren,
Naushir Patuck, Umang Jain
From: Naushir Patuck <naush@raspberrypi.com>
Driver for the BCM2835 ISP hardware block. This driver uses the MMAL
component to program the ISP hardware through the VC firmware.
The ISP component can produce two video stream outputs, and Bayer
image statistics. This can't be encompassed in a simple V4L2
M2M device, so create a new device that registers 4 video nodes.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
---
MAINTAINERS | 9 +
drivers/staging/vc04_services/Kconfig | 2 +
drivers/staging/vc04_services/Makefile | 1 +
.../staging/vc04_services/bcm2835-isp/Kconfig | 14 +
.../vc04_services/bcm2835-isp/Makefile | 4 +
.../bcm2835-isp/bcm2835-isp-ctrls.h | 72 +
.../bcm2835-isp/bcm2835-isp-fmts.h | 482 +++++
.../bcm2835-isp/bcm2835-v4l2-isp.c | 1712 +++++++++++++++++
.../vc04_services/vchiq-mmal/mmal-encodings.h | 4 +
.../vchiq-mmal/mmal-parameters.h | 165 +-
include/uapi/linux/bcm2835-isp.h | 27 +
11 files changed, 2491 insertions(+), 1 deletion(-)
create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-ctrls.h
create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 0748cc15588c..8670a8d95400 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3976,6 +3976,15 @@ F: Documentation/devicetree/bindings/mips/brcm/
F: arch/mips/bcm47xx/*
F: arch/mips/include/asm/mach-bcm47xx/*
+BROADCOM BCM2835 ISP DRIVER
+M: Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
+L: linux-media@vger.kernel.org
+S: Maintained
+F: Documentation/media/uapi/v4l/pixfmt-meta-bcm2835-isp-stats.rst
+F: Documentation/media/v4l-drivers/bcm2835-isp.rst
+F: drivers/staging/vc04_services/bcm2835-isp
+F: include/uapi/linux/bcm2835-isp.h
+
BROADCOM BCM4908 ETHERNET DRIVER
M: Rafał Miłecki <rafal@milecki.pl>
R: Broadcom internal kernel review list <bcm-kernel-feedback-list@broadcom.com>
diff --git a/drivers/staging/vc04_services/Kconfig b/drivers/staging/vc04_services/Kconfig
index 6c0e77d64376..e71e5a8b820a 100644
--- a/drivers/staging/vc04_services/Kconfig
+++ b/drivers/staging/vc04_services/Kconfig
@@ -44,6 +44,8 @@ source "drivers/staging/vc04_services/bcm2835-audio/Kconfig"
source "drivers/staging/vc04_services/bcm2835-camera/Kconfig"
+source "drivers/staging/vc04_services/bcm2835-isp/Kconfig"
+
source "drivers/staging/vc04_services/vchiq-mmal/Kconfig"
source "drivers/staging/vc04_services/vc-sm-cma/Kconfig"
diff --git a/drivers/staging/vc04_services/Makefile b/drivers/staging/vc04_services/Makefile
index 1f61a263f22d..2cebf92525a0 100644
--- a/drivers/staging/vc04_services/Makefile
+++ b/drivers/staging/vc04_services/Makefile
@@ -16,3 +16,4 @@ obj-$(CONFIG_SND_BCM2835) += bcm2835-audio/
obj-$(CONFIG_VIDEO_BCM2835) += bcm2835-camera/
obj-$(CONFIG_BCM2835_VCHIQ_MMAL) += vchiq-mmal/
obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma/
+obj-$(CONFIG_VIDEO_ISP_BCM2835) += bcm2835-isp/
diff --git a/drivers/staging/vc04_services/bcm2835-isp/Kconfig b/drivers/staging/vc04_services/bcm2835-isp/Kconfig
new file mode 100644
index 000000000000..6222799ebe16
--- /dev/null
+++ b/drivers/staging/vc04_services/bcm2835-isp/Kconfig
@@ -0,0 +1,14 @@
+config VIDEO_ISP_BCM2835
+ tristate "BCM2835 ISP support"
+ depends on MEDIA_SUPPORT
+ depends on VIDEO_DEV && (ARCH_BCM2835 || COMPILE_TEST)
+ depends on MEDIA_CONTROLLER
+ select BCM2835_VCHIQ_MMAL
+ select VIDEOBUF2_DMA_CONTIG
+ help
+ This is the V4L2 driver for the Broadcom BCM2835 ISP hardware.
+ This operates over the VCHIQ interface to a service running on
+ VideoCore.
+
+ To compile this driver as a module, choose M here: the module
+ will be called bcm2835-isp.
diff --git a/drivers/staging/vc04_services/bcm2835-isp/Makefile b/drivers/staging/vc04_services/bcm2835-isp/Makefile
new file mode 100644
index 000000000000..2c78163264a9
--- /dev/null
+++ b/drivers/staging/vc04_services/bcm2835-isp/Makefile
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0
+bcm2835-isp-objs := bcm2835-v4l2-isp.o
+
+obj-$(CONFIG_VIDEO_ISP_BCM2835) += bcm2835-isp.o
diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-ctrls.h b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-ctrls.h
new file mode 100644
index 000000000000..172605718cdf
--- /dev/null
+++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-ctrls.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Broadcom BCM2835 ISP driver
+ *
+ * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
+ *
+ * Author: Naushir Patuck (naush@raspberrypi.com)
+ *
+ */
+
+#ifndef BCM2835_ISP_CTRLS
+#define BCM2835_ISP_CTRLS
+
+#include <linux/bcm2835-isp.h>
+
+struct bcm2835_isp_custom_ctrl {
+ const char *name;
+ u32 id;
+ u32 size;
+ u32 flags;
+};
+
+static const struct bcm2835_isp_custom_ctrl custom_ctrls[] = {
+ {
+ .name = "Colour Correction Matrix",
+ .id = V4L2_CID_USER_BCM2835_ISP_CC_MATRIX,
+ .size = sizeof(struct bcm2835_isp_custom_ccm),
+ .flags = 0
+ }, {
+ .name = "Lens Shading",
+ .id = V4L2_CID_USER_BCM2835_ISP_LENS_SHADING,
+ .size = sizeof(struct bcm2835_isp_lens_shading),
+ .flags = V4L2_CTRL_FLAG_EXECUTE_ON_WRITE
+ }, {
+ .name = "Black Level",
+ .id = V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL,
+ .size = sizeof(struct bcm2835_isp_black_level),
+ .flags = 0
+ }, {
+ .name = "Green Equalisation",
+ .id = V4L2_CID_USER_BCM2835_ISP_GEQ,
+ .size = sizeof(struct bcm2835_isp_geq),
+ .flags = 0
+ }, {
+ .name = "Gamma",
+ .id = V4L2_CID_USER_BCM2835_ISP_GAMMA,
+ .size = sizeof(struct bcm2835_isp_gamma),
+ .flags = 0
+ }, {
+ .name = "Sharpen",
+ .id = V4L2_CID_USER_BCM2835_ISP_SHARPEN,
+ .size = sizeof(struct bcm2835_isp_sharpen),
+ .flags = 0
+ }, {
+ .name = "Denoise",
+ .id = V4L2_CID_USER_BCM2835_ISP_DENOISE,
+ .size = sizeof(struct bcm2835_isp_denoise),
+ .flags = 0
+ }, {
+ .name = "Colour Denoise",
+ .id = V4L2_CID_USER_BCM2835_ISP_CDN,
+ .size = sizeof(struct bcm2835_isp_cdn),
+ .flags = 0
+ }, {
+ .name = "Defective Pixel Correction",
+ .id = V4L2_CID_USER_BCM2835_ISP_DPC,
+ .size = sizeof(struct bcm2835_isp_dpc),
+ .flags = 0
+ }
+};
+
+#endif
diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
new file mode 100644
index 000000000000..6dcd196fc6a5
--- /dev/null
+++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
@@ -0,0 +1,482 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Broadcom BCM2835 ISP driver
+ *
+ * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
+ *
+ * Author: Naushir Patuck (naush@raspberrypi.com)
+ *
+ */
+
+#ifndef BCM2835_ISP_FMTS
+#define BCM2835_ISP_FMTS
+
+#include <linux/videodev2.h>
+
+#include "../vchiq-mmal/mmal-encodings.h"
+
+struct bcm2835_isp_fmt {
+ u32 fourcc;
+ int depth;
+ int bytesperline_align;
+ u32 mmal_fmt;
+ int size_multiplier_x2;
+ enum v4l2_colorspace colorspace;
+ unsigned int step_size;
+};
+
+static const struct bcm2835_isp_fmt supported_formats[] = {
+ {
+ /* YUV formats */
+ .fourcc = V4L2_PIX_FMT_YUV420,
+ .depth = 8,
+ .bytesperline_align = 64,
+ .mmal_fmt = MMAL_ENCODING_I420,
+ .size_multiplier_x2 = 3,
+ .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_YVU420,
+ .depth = 8,
+ .bytesperline_align = 64,
+ .mmal_fmt = MMAL_ENCODING_YV12,
+ .size_multiplier_x2 = 3,
+ .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_NV12,
+ .depth = 8,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_NV12,
+ .size_multiplier_x2 = 3,
+ .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_NV21,
+ .depth = 8,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_NV21,
+ .size_multiplier_x2 = 3,
+ .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_YUYV,
+ .depth = 16,
+ .bytesperline_align = 64,
+ .mmal_fmt = MMAL_ENCODING_YUYV,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_UYVY,
+ .depth = 16,
+ .bytesperline_align = 64,
+ .mmal_fmt = MMAL_ENCODING_UYVY,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_YVYU,
+ .depth = 16,
+ .bytesperline_align = 64,
+ .mmal_fmt = MMAL_ENCODING_YVYU,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_VYUY,
+ .depth = 16,
+ .bytesperline_align = 64,
+ .mmal_fmt = MMAL_ENCODING_VYUY,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .step_size = 2,
+ }, {
+ /* RGB formats */
+ .fourcc = V4L2_PIX_FMT_RGB24,
+ .depth = 24,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_RGB24,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+ .step_size = 1,
+ }, {
+ .fourcc = V4L2_PIX_FMT_RGB565,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_RGB16,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+ .step_size = 1,
+ }, {
+ .fourcc = V4L2_PIX_FMT_BGR24,
+ .depth = 24,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BGR24,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+ .step_size = 1,
+ }, {
+ .fourcc = V4L2_PIX_FMT_XBGR32,
+ .depth = 32,
+ .bytesperline_align = 64,
+ .mmal_fmt = MMAL_ENCODING_BGRA,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+ .step_size = 1,
+ }, {
+ .fourcc = V4L2_PIX_FMT_RGBX32,
+ .depth = 32,
+ .bytesperline_align = 64,
+ .mmal_fmt = MMAL_ENCODING_RGBA,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+ .step_size = 1,
+ }, {
+ /* Bayer formats */
+ /* 8 bit */
+ .fourcc = V4L2_PIX_FMT_SRGGB8,
+ .depth = 8,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB8,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR8,
+ .depth = 8,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR8,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG8,
+ .depth = 8,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG8,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG8,
+ .depth = 8,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG8,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* 10 bit */
+ .fourcc = V4L2_PIX_FMT_SRGGB10P,
+ .depth = 10,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB10P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR10P,
+ .depth = 10,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR10P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG10P,
+ .depth = 10,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG10P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG10P,
+ .depth = 10,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG10P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* 12 bit */
+ .fourcc = V4L2_PIX_FMT_SRGGB12P,
+ .depth = 12,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB12P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR12P,
+ .depth = 12,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR12P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG12P,
+ .depth = 12,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG12P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG12P,
+ .depth = 12,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG12P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* 14 bit */
+ .fourcc = V4L2_PIX_FMT_SRGGB14P,
+ .depth = 14,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB14P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR14P,
+ .depth = 14,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR14P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG14P,
+ .depth = 14,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG14P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG14P,
+ .depth = 14,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG14P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* 16 bit */
+ .fourcc = V4L2_PIX_FMT_SRGGB16,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB16,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR16,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR16,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG16,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG16,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG16,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG16,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* Bayer formats unpacked to 16bpp */
+ /* 10 bit */
+ .fourcc = V4L2_PIX_FMT_SRGGB10,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB10,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR10,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR10,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG10,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG10,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG10,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG10,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* 12 bit */
+ .fourcc = V4L2_PIX_FMT_SRGGB12,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB12,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR12,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR12,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG12,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG12,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG12,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG12,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* 14 bit */
+ .fourcc = V4L2_PIX_FMT_SRGGB14,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB14,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR14,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR14,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG14,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG14,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG14,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG14,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* Monochrome MIPI formats */
+ /* 8 bit */
+ .fourcc = V4L2_PIX_FMT_GREY,
+ .depth = 8,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_GREY,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* 10 bit */
+ .fourcc = V4L2_PIX_FMT_Y10P,
+ .depth = 10,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_Y10P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* 12 bit */
+ .fourcc = V4L2_PIX_FMT_Y12P,
+ .depth = 12,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_Y12P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* 14 bit */
+ .fourcc = V4L2_PIX_FMT_Y14P,
+ .depth = 14,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_Y14P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* 16 bit */
+ .fourcc = V4L2_PIX_FMT_Y16,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_Y16,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* 10 bit as 16bpp */
+ .fourcc = V4L2_PIX_FMT_Y10,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_Y10,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* 12 bit as 16bpp */
+ .fourcc = V4L2_PIX_FMT_Y12,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_Y12,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* 14 bit as 16bpp */
+ .fourcc = V4L2_PIX_FMT_Y14,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .mmal_fmt = MMAL_ENCODING_Y14,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_META_FMT_BCM2835_ISP_STATS,
+ .depth = 8,
+ .mmal_fmt = MMAL_ENCODING_BRCM_STATS,
+ /* The rest are not valid fields for stats. */
+ }
+};
+
+#endif
diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
new file mode 100644
index 000000000000..f1ea56b4849e
--- /dev/null
+++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
@@ -0,0 +1,1712 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Broadcom BCM2835 ISP driver
+ *
+ * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
+ *
+ * Author: Naushir Patuck (naush@raspberrypi.com)
+ *
+ */
+
+#include <linux/module.h>
+
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "../vchiq-mmal/mmal-msg.h"
+#include "../vchiq-mmal/mmal-parameters.h"
+#include "../vchiq-mmal/mmal-vchiq.h"
+
+#include "../vc-sm-cma/vc_sm_knl.h"
+
+#include "../interface/vchiq_arm/vchiq_bus.h"
+#include "bcm2835-isp-ctrls.h"
+#include "bcm2835-isp-fmts.h"
+
+MODULE_IMPORT_NS(DMA_BUF);
+
+static unsigned int debug;
+module_param(debug, uint, 0644);
+MODULE_PARM_DESC(debug, "activates debug info");
+
+static unsigned int video_nr = 13;
+module_param(video_nr, uint, 0644);
+MODULE_PARM_DESC(video_nr, "base video device number");
+
+#define BCM2835_ISP_NAME "bcm2835-isp"
+#define BCM2835_ISP_ENTITY_NAME_LEN 32
+
+#define BCM2835_ISP_NUM_OUTPUTS 1
+#define BCM2835_ISP_NUM_CAPTURES 2
+#define BCM2835_ISP_NUM_METADATA 1
+
+#define BCM2835_ISP_NUM_NODES \
+ (BCM2835_ISP_NUM_OUTPUTS + BCM2835_ISP_NUM_CAPTURES + \
+ BCM2835_ISP_NUM_METADATA)
+
+/* Default frame dimension of 1280 pixels. */
+#define DEFAULT_DIM 1280U
+/*
+ * Maximum frame dimension of 16384 pixels. Even though the ISP runs in tiles,
+ * have a sensible limit so that we do not create an excessive number of tiles
+ * to process.
+ */
+#define MAX_DIM 16384U
+/*
+ * Minimum frame dimension of 64 pixels. Anything lower, and the tiling
+ * algorithm may not be able to cope when applying filter context.
+ */
+#define MIN_DIM 64U
+
+/* Timeout for stop_streaming to allow all buffers to return */
+#define COMPLETE_TIMEOUT (2 * HZ)
+
+/* Per-queue, driver-specific private data */
+struct bcm2835_isp_q_data {
+ /*
+ * These parameters should be treated as gospel, with everything else
+ * being determined from them.
+ */
+ unsigned int bytesperline;
+ unsigned int width;
+ unsigned int height;
+ unsigned int sizeimage;
+ const struct bcm2835_isp_fmt *fmt;
+};
+
+/*
+ * Structure to describe a single node /dev/video<N> which represents a single
+ * input or output queue to the ISP device.
+ */
+struct bcm2835_isp_node {
+ int vfl_dir;
+ unsigned int id;
+ const char *name;
+ struct vchiq_mmal_port *port;
+ struct video_device vfd;
+ struct media_pad pad;
+ struct media_intf_devnode *intf_devnode;
+ struct media_link *intf_link;
+ struct mutex lock; /* top level device node lock */
+ struct mutex queue_lock;
+
+ struct vb2_queue queue;
+ unsigned int sequence;
+
+ /* The list of formats supported on the node. */
+ struct bcm2835_isp_fmt const **supported_fmts;
+ unsigned int num_supported_fmts;
+
+ struct bcm2835_isp_q_data q_data;
+
+ /* Parent device structure */
+ struct bcm2835_isp_dev *dev;
+
+ bool registered;
+ bool media_node_registered;
+};
+
+/*
+ * Structure representing the entire ISP device, comprising several input and
+ * output nodes /dev/video<N>.
+ */
+struct bcm2835_isp_dev {
+ struct v4l2_device v4l2_dev;
+ struct device *dev;
+ struct v4l2_ctrl_handler ctrl_handler;
+ struct media_device mdev;
+ struct media_entity entity;
+ bool media_device_registered;
+ bool media_entity_registered;
+ struct vchiq_mmal_instance *mmal_instance;
+ struct vchiq_mmal_component *component;
+ struct completion frame_cmplt;
+
+ struct bcm2835_isp_node node[BCM2835_ISP_NUM_NODES];
+ struct media_pad pad[BCM2835_ISP_NUM_NODES];
+ atomic_t num_streaming;
+
+ /* Image pipeline controls. */
+ int r_gain;
+ int b_gain;
+};
+
+struct bcm2835_isp_buffer {
+ struct vb2_v4l2_buffer vb;
+ struct mmal_buffer mmal;
+};
+
+static
+inline struct bcm2835_isp_dev *node_get_dev(struct bcm2835_isp_node *node)
+{
+ return node->dev;
+}
+
+static inline bool node_is_output(struct bcm2835_isp_node *node)
+{
+ return node->queue.type == V4L2_BUF_TYPE_VIDEO_OUTPUT;
+}
+
+static inline bool node_is_capture(struct bcm2835_isp_node *node)
+{
+ return node->queue.type == V4L2_BUF_TYPE_VIDEO_CAPTURE;
+}
+
+static inline bool node_is_stats(struct bcm2835_isp_node *node)
+{
+ return node->queue.type == V4L2_BUF_TYPE_META_CAPTURE;
+}
+
+static inline enum v4l2_buf_type index_to_queue_type(int index)
+{
+ if (index < BCM2835_ISP_NUM_OUTPUTS)
+ return V4L2_BUF_TYPE_VIDEO_OUTPUT;
+ else if (index < BCM2835_ISP_NUM_OUTPUTS + BCM2835_ISP_NUM_CAPTURES)
+ return V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ else
+ return V4L2_BUF_TYPE_META_CAPTURE;
+}
+
+static int set_isp_param(struct bcm2835_isp_node *node, u32 parameter,
+ void *value, u32 value_size)
+{
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+
+ return vchiq_mmal_port_parameter_set(dev->mmal_instance, node->port,
+ parameter, value, value_size);
+}
+
+static int set_wb_gains(struct bcm2835_isp_node *node)
+{
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ struct mmal_parameter_awbgains gains = {
+ .r_gain = { dev->r_gain, 1000 },
+ .b_gain = { dev->b_gain, 1000 }
+ };
+
+ return set_isp_param(node, MMAL_PARAMETER_CUSTOM_AWB_GAINS,
+ &gains, sizeof(gains));
+}
+
+static int set_digital_gain(struct bcm2835_isp_node *node, uint32_t gain)
+{
+ struct s32_fract digital_gain = {
+ .numerator = gain,
+ .denominator = 1000
+ };
+
+ return set_isp_param(node, MMAL_PARAMETER_DIGITAL_GAIN,
+ &digital_gain, sizeof(digital_gain));
+}
+
+static const struct bcm2835_isp_fmt *get_fmt(u32 mmal_fmt)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(supported_formats); i++) {
+ if (supported_formats[i].mmal_fmt == mmal_fmt)
+ return &supported_formats[i];
+ }
+ return NULL;
+}
+
+static const
+struct bcm2835_isp_fmt *find_format_by_fourcc(unsigned int fourcc,
+ struct bcm2835_isp_node *node)
+{
+ const struct bcm2835_isp_fmt *fmt;
+ unsigned int i;
+
+ for (i = 0; i < node->num_supported_fmts; i++) {
+ fmt = node->supported_fmts[i];
+ if (fmt->fourcc == fourcc)
+ return fmt;
+ }
+
+ return NULL;
+}
+
+static const
+struct bcm2835_isp_fmt *find_format(struct v4l2_format *f,
+ struct bcm2835_isp_node *node)
+{
+ return find_format_by_fourcc(node_is_stats(node) ?
+ f->fmt.meta.dataformat :
+ f->fmt.pix.pixelformat,
+ node);
+}
+
+/* vb2_to_mmal_buffer() - converts vb2 buffer header to MMAL
+ *
+ * Copies all the required fields from a VB2 buffer to the MMAL buffer header,
+ * ready for sending to the VPU.
+ */
+static void vb2_to_mmal_buffer(struct mmal_buffer *buf,
+ struct vb2_v4l2_buffer *vb2)
+{
+ u64 pts;
+
+ buf->mmal_flags = 0;
+ if (vb2->flags & V4L2_BUF_FLAG_KEYFRAME)
+ buf->mmal_flags |= MMAL_BUFFER_HEADER_FLAG_KEYFRAME;
+
+ /* Data must be framed correctly as one frame per buffer. */
+ buf->mmal_flags |= MMAL_BUFFER_HEADER_FLAG_FRAME_END;
+
+ buf->length = vb2->vb2_buf.planes[0].bytesused;
+ /*
+ * Minor ambiguity in the V4L2 spec as to whether passing in a 0 length
+ * buffer, or one with V4L2_BUF_FLAG_LAST set denotes end of stream.
+ * Handle either.
+ */
+ if (!buf->length || vb2->flags & V4L2_BUF_FLAG_LAST)
+ buf->mmal_flags |= MMAL_BUFFER_HEADER_FLAG_EOS;
+
+ /* vb2 timestamps in nsecs, mmal in usecs */
+ pts = vb2->vb2_buf.timestamp;
+ do_div(pts, 1000);
+ buf->pts = pts;
+ buf->dts = MMAL_TIME_UNKNOWN;
+}
+
+static void mmal_buffer_cb(struct vchiq_mmal_instance *instance,
+ struct vchiq_mmal_port *port, int status,
+ struct mmal_buffer *mmal_buf)
+{
+ struct bcm2835_isp_buffer *q_buf;
+ struct bcm2835_isp_node *node = port->cb_ctx;
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ struct vb2_v4l2_buffer *vb2;
+
+ q_buf = container_of(mmal_buf, struct bcm2835_isp_buffer, mmal);
+ vb2 = &q_buf->vb;
+ v4l2_dbg(2, debug, &dev->v4l2_dev,
+ "%s: port:%s[%d], status:%d, buf:%p, dmabuf:%p, length:%lu, flags %u, pts %lld\n",
+ __func__, node_is_output(node) ? "input" : "output", node->id,
+ status, mmal_buf, mmal_buf->dma_buf, mmal_buf->length,
+ mmal_buf->mmal_flags, mmal_buf->pts);
+
+ if (mmal_buf->cmd)
+ v4l2_err(&dev->v4l2_dev,
+ "%s: Unexpected event on output callback - %08x\n",
+ __func__, mmal_buf->cmd);
+
+ if (status) {
+ /* error in transfer */
+ if (vb2) {
+ /* there was a buffer with the error so return it */
+ vb2_buffer_done(&vb2->vb2_buf, VB2_BUF_STATE_ERROR);
+ }
+ return;
+ }
+
+ /* vb2 timestamps in nsecs, mmal in usecs */
+ vb2->vb2_buf.timestamp = mmal_buf->pts * 1000;
+ vb2->sequence = node->sequence++;
+ vb2_set_plane_payload(&vb2->vb2_buf, 0, mmal_buf->length);
+ vb2_buffer_done(&vb2->vb2_buf, VB2_BUF_STATE_DONE);
+
+ if (!port->enabled)
+ complete(&dev->frame_cmplt);
+}
+
+static void setup_mmal_port_format(struct bcm2835_isp_node *node,
+ struct vchiq_mmal_port *port)
+{
+ struct bcm2835_isp_q_data *q_data = &node->q_data;
+
+ port->format.encoding = q_data->fmt->mmal_fmt;
+ /* Raw image format - set width/height */
+ port->es.video.width = (q_data->bytesperline << 3) / q_data->fmt->depth;
+ port->es.video.height = q_data->height;
+ port->es.video.crop.width = q_data->width;
+ port->es.video.crop.height = q_data->height;
+ port->es.video.crop.x = 0;
+ port->es.video.crop.y = 0;
+};
+
+static int setup_mmal_port(struct bcm2835_isp_node *node)
+{
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ unsigned int enable = 1;
+ int ret;
+
+ v4l2_dbg(2, debug, &dev->v4l2_dev, "%s: setup %s[%d]\n", __func__,
+ node->name, node->id);
+
+ vchiq_mmal_port_parameter_set(dev->mmal_instance, node->port,
+ MMAL_PARAMETER_ZERO_COPY, &enable,
+ sizeof(enable));
+ setup_mmal_port_format(node, node->port);
+ ret = vchiq_mmal_port_set_format(dev->mmal_instance, node->port);
+ if (ret < 0) {
+ v4l2_dbg(1, debug, &dev->v4l2_dev,
+ "%s: vchiq_mmal_port_set_format failed\n",
+ __func__);
+ return ret;
+ }
+
+ if (node->q_data.sizeimage < node->port->minimum_buffer.size) {
+ v4l2_err(&dev->v4l2_dev,
+ "buffer size mismatch sizeimage %u < min size %u\n",
+ node->q_data.sizeimage,
+ node->port->minimum_buffer.size);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int bcm2835_isp_mmal_buf_cleanup(struct mmal_buffer *mmal_buf)
+{
+ mmal_vchi_buffer_cleanup(mmal_buf);
+
+ if (mmal_buf->dma_buf) {
+ dma_buf_put(mmal_buf->dma_buf);
+ mmal_buf->dma_buf = NULL;
+ }
+
+ return 0;
+}
+
+static int bcm2835_isp_node_queue_setup(struct vb2_queue *q,
+ unsigned int *nbuffers,
+ unsigned int *nplanes,
+ unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct bcm2835_isp_node *node = vb2_get_drv_priv(q);
+ unsigned int size;
+
+ if (setup_mmal_port(node))
+ return -EINVAL;
+
+ size = node->q_data.sizeimage;
+ if (size == 0) {
+ v4l2_info(&node_get_dev(node)->v4l2_dev,
+ "%s: Image size unset in queue_setup for node %s[%d]\n",
+ __func__, node->name, node->id);
+ return -EINVAL;
+ }
+
+ if (*nplanes)
+ return sizes[0] < size ? -EINVAL : 0;
+
+ *nplanes = 1;
+ sizes[0] = size;
+
+ node->port->current_buffer.size = size;
+
+ if (*nbuffers < node->port->minimum_buffer.num)
+ *nbuffers = node->port->minimum_buffer.num;
+
+ node->port->current_buffer.num = *nbuffers;
+
+ v4l2_dbg(2, debug, &node_get_dev(node)->v4l2_dev,
+ "%s: Image size %u, nbuffers %u for node %s[%d]\n",
+ __func__, sizes[0], *nbuffers, node->name, node->id);
+ return 0;
+}
+
+static int bcm2835_isp_buf_init(struct vb2_buffer *vb)
+{
+ struct bcm2835_isp_node *node = vb2_get_drv_priv(vb->vb2_queue);
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(vb);
+ struct bcm2835_isp_buffer *buf =
+ container_of(vb2, struct bcm2835_isp_buffer, vb);
+
+ v4l2_dbg(3, debug, &dev->v4l2_dev, "%s: vb %p\n", __func__, vb);
+
+ buf->mmal.buffer = vb2_plane_vaddr(&buf->vb.vb2_buf, 0);
+ buf->mmal.buffer_size = vb2_plane_size(&buf->vb.vb2_buf, 0);
+ mmal_vchi_buffer_init(dev->mmal_instance, &buf->mmal);
+ return 0;
+}
+
+static int bcm2835_isp_buf_prepare(struct vb2_buffer *vb)
+{
+ struct bcm2835_isp_node *node = vb2_get_drv_priv(vb->vb2_queue);
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(vb);
+ struct bcm2835_isp_buffer *buf =
+ container_of(vb2, struct bcm2835_isp_buffer, vb);
+ struct dma_buf *dma_buf;
+ int ret;
+
+ v4l2_dbg(3, debug, &dev->v4l2_dev, "%s: type: %d ptr %p\n",
+ __func__, vb->vb2_queue->type, vb);
+
+ if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
+ if (vb2->field == V4L2_FIELD_ANY)
+ vb2->field = V4L2_FIELD_NONE;
+ if (vb2->field != V4L2_FIELD_NONE) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s field isn't supported\n", __func__);
+ return -EINVAL;
+ }
+ }
+
+ if (vb2_plane_size(vb, 0) < node->q_data.sizeimage) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s data will not fit into plane (%lu < %lu)\n",
+ __func__, vb2_plane_size(vb, 0),
+ (long)node->q_data.sizeimage);
+ return -EINVAL;
+ }
+
+ if (!V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
+ vb2_set_plane_payload(vb, 0, node->q_data.sizeimage);
+
+ switch (vb->memory) {
+ case VB2_MEMORY_DMABUF:
+ dma_buf = dma_buf_get(vb->planes[0].m.fd);
+
+ if (dma_buf != buf->mmal.dma_buf) {
+ /*
+ * dmabuf either hasn't already been mapped, or it has
+ * changed.
+ */
+ if (buf->mmal.dma_buf) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s Buffer changed - why did the core not call cleanup?\n",
+ __func__);
+ bcm2835_isp_mmal_buf_cleanup(&buf->mmal);
+ }
+
+ buf->mmal.dma_buf = dma_buf;
+ } else {
+ /*
+ * Already have a reference to the buffer, so release it
+ * here.
+ */
+ dma_buf_put(dma_buf);
+ }
+ ret = 0;
+ break;
+ case VB2_MEMORY_MMAP:
+ /*
+ * We want to do this at init, but vb2_core_expbuf checks that
+ * the index < q->num_buffers, and q->num_buffers only gets
+ * updated once all the buffers are allocated.
+ */
+ if (!buf->mmal.dma_buf) {
+ buf->mmal.dma_buf = vb2_core_expbuf_dmabuf(vb->vb2_queue,
+ vb->vb2_queue->type,
+ vb->index, 0, O_CLOEXEC);
+ v4l2_dbg(3, debug, &dev->v4l2_dev,
+ "%s: exporting ptr %p to dmabuf %p\n",
+ __func__, vb, buf->mmal.dma_buf);
+ if (IS_ERR(buf->mmal.dma_buf)) {
+ ret = PTR_ERR(buf->mmal.dma_buf);
+ v4l2_err(&dev->v4l2_dev,
+ "%s: Failed to expbuf idx %d, ret %d\n",
+ __func__, vb->index, ret);
+ }
+ } else {
+ ret = 0;
+ }
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+
+static void bcm2835_isp_node_buffer_queue(struct vb2_buffer *buf)
+{
+ struct bcm2835_isp_node *node = vb2_get_drv_priv(buf->vb2_queue);
+ struct vb2_v4l2_buffer *vbuf =
+ container_of(buf, struct vb2_v4l2_buffer, vb2_buf);
+ struct bcm2835_isp_buffer *buffer =
+ container_of(vbuf, struct bcm2835_isp_buffer, vb);
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+
+ v4l2_dbg(3, debug, &dev->v4l2_dev, "%s: node %s[%d], buffer %p\n",
+ __func__, node->name, node->id, buffer);
+
+ vb2_to_mmal_buffer(&buffer->mmal, &buffer->vb);
+ v4l2_dbg(3, debug, &dev->v4l2_dev,
+ "%s: node %s[%d] - submitting mmal dmabuf %p\n", __func__,
+ node->name, node->id, buffer->mmal.dma_buf);
+ vchiq_mmal_submit_buffer(dev->mmal_instance, node->port, &buffer->mmal);
+}
+
+static void bcm2835_isp_buffer_cleanup(struct vb2_buffer *vb)
+{
+ struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(vb);
+ struct bcm2835_isp_buffer *buffer =
+ container_of(vb2, struct bcm2835_isp_buffer, vb);
+
+ bcm2835_isp_mmal_buf_cleanup(&buffer->mmal);
+}
+
+static int bcm2835_isp_node_start_streaming(struct vb2_queue *q,
+ unsigned int count)
+{
+ struct bcm2835_isp_node *node = vb2_get_drv_priv(q);
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ int ret;
+
+ v4l2_dbg(1, debug, &dev->v4l2_dev, "%s: node %s[%d] (count %u)\n",
+ __func__, node->name, node->id, count);
+
+ ret = vchiq_mmal_component_enable(dev->mmal_instance, dev->component);
+ if (ret) {
+ v4l2_err(&dev->v4l2_dev, "%s: Failed enabling component, ret %d\n",
+ __func__, ret);
+ return -EIO;
+ }
+
+ node->sequence = 0;
+ node->port->cb_ctx = node;
+ ret = vchiq_mmal_port_enable(dev->mmal_instance, node->port,
+ mmal_buffer_cb);
+ if (!ret)
+ atomic_inc(&dev->num_streaming);
+ else
+ v4l2_err(&dev->v4l2_dev,
+ "%s: Failed enabling port, ret %d\n", __func__, ret);
+
+ return ret;
+}
+
+static void bcm2835_isp_node_stop_streaming(struct vb2_queue *q)
+{
+ struct bcm2835_isp_node *node = vb2_get_drv_priv(q);
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ int ret;
+
+ v4l2_dbg(1, debug, &dev->v4l2_dev, "%s: node %s[%d], mmal port %p\n",
+ __func__, node->name, node->id, node->port);
+
+ init_completion(&dev->frame_cmplt);
+
+ /* Disable MMAL port - this will flush buffers back */
+ ret = vchiq_mmal_port_disable(dev->mmal_instance, node->port);
+ if (ret)
+ v4l2_err(&dev->v4l2_dev,
+ "%s: Failed disabling %s port, ret %d\n", __func__,
+ node_is_output(node) ? "i/p" : "o/p",
+ ret);
+
+ while (atomic_read(&node->port->buffers_with_vpu)) {
+ v4l2_dbg(1, debug, &dev->v4l2_dev,
+ "%s: Waiting for buffers to be returned - %d outstanding\n",
+ __func__, atomic_read(&node->port->buffers_with_vpu));
+ ret = wait_for_completion_timeout(&dev->frame_cmplt,
+ COMPLETE_TIMEOUT);
+ if (ret <= 0) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s: Timeout waiting for buffers to be returned - %d outstanding\n",
+ __func__,
+ atomic_read(&node->port->buffers_with_vpu));
+ break;
+ }
+ }
+
+ atomic_dec(&dev->num_streaming);
+ /* If all ports disabled, then disable the component */
+ if (atomic_read(&dev->num_streaming) == 0) {
+ struct bcm2835_isp_lens_shading ls;
+ /*
+ * The ISP component on the firmware has a reference to the
+ * dmabuf handle for the lens shading table. Pass a null handle
+ * to remove that reference now.
+ */
+ memset(&ls, 0, sizeof(ls));
+ /* Must set a valid grid size for the FW */
+ ls.grid_cell_size = 16;
+ set_isp_param(&dev->node[0],
+ MMAL_PARAMETER_LENS_SHADING_OVERRIDE,
+ &ls, sizeof(ls));
+
+ ret = vchiq_mmal_component_disable(dev->mmal_instance,
+ dev->component);
+ if (ret) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s: Failed disabling component, ret %d\n",
+ __func__, ret);
+ }
+ }
+
+ /*
+ * Simply wait for any vb2 buffers to finish. We could take steps to
+ * make them complete more quickly if we care, or even return them
+ * ourselves.
+ */
+ vb2_wait_for_all_buffers(&node->queue);
+}
+
+static const struct vb2_ops bcm2835_isp_node_queue_ops = {
+ .queue_setup = bcm2835_isp_node_queue_setup,
+ .buf_init = bcm2835_isp_buf_init,
+ .buf_prepare = bcm2835_isp_buf_prepare,
+ .buf_queue = bcm2835_isp_node_buffer_queue,
+ .buf_cleanup = bcm2835_isp_buffer_cleanup,
+ .start_streaming = bcm2835_isp_node_start_streaming,
+ .stop_streaming = bcm2835_isp_node_stop_streaming,
+};
+
+static const
+struct bcm2835_isp_fmt *get_default_format(struct bcm2835_isp_node *node)
+{
+ return node->supported_fmts[0];
+}
+
+static inline unsigned int get_bytesperline(int width,
+ const struct bcm2835_isp_fmt *fmt)
+{
+ /* GPU aligns 24bpp images to a multiple of 32 pixels (not bytes). */
+ if (fmt->depth == 24)
+ return ALIGN(width, 32) * 3;
+ else
+ return ALIGN((width * fmt->depth) >> 3, fmt->bytesperline_align);
+}
+
+static inline unsigned int get_sizeimage(int bpl, int width, int height,
+ const struct bcm2835_isp_fmt *fmt)
+{
+ return (bpl * height * fmt->size_multiplier_x2) >> 1;
+}
+
+static int bcm2835_isp_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct bcm2835_isp_dev *dev =
+ container_of(ctrl->handler, struct bcm2835_isp_dev, ctrl_handler);
+ struct bcm2835_isp_node *node = &dev->node[0];
+ int ret = 0;
+
+ /*
+ * The ISP firmware driver will ensure these settings are applied on
+ * a frame boundary, so we are safe to write them as they come in.
+ *
+ * Note that the bcm2835_isp_* param structures are identical to the
+ * mmal-parameters.h definitions. This avoids the need for unnecessary
+ * field-by-field copying between structures.
+ */
+ switch (ctrl->id) {
+ case V4L2_CID_RED_BALANCE:
+ dev->r_gain = ctrl->val;
+ ret = set_wb_gains(node);
+ break;
+ case V4L2_CID_BLUE_BALANCE:
+ dev->b_gain = ctrl->val;
+ ret = set_wb_gains(node);
+ break;
+ case V4L2_CID_DIGITAL_GAIN:
+ ret = set_digital_gain(node, ctrl->val);
+ break;
+ case V4L2_CID_USER_BCM2835_ISP_CC_MATRIX:
+ ret = set_isp_param(node, MMAL_PARAMETER_CUSTOM_CCM,
+ ctrl->p_new.p_u8,
+ sizeof(struct bcm2835_isp_custom_ccm));
+ break;
+ case V4L2_CID_USER_BCM2835_ISP_LENS_SHADING:
+ {
+ struct bcm2835_isp_lens_shading *v4l2_ls;
+ struct mmal_parameter_lens_shading_v2 ls;
+ struct dma_buf *dmabuf;
+ void *vcsm_handle;
+
+ v4l2_ls = (struct bcm2835_isp_lens_shading *)ctrl->p_new.p_u8;
+ /*
+ * struct bcm2835_isp_lens_shading and struct
+ * mmal_parameter_lens_shading_v2 match so that we can do a
+ * simple memcpy here.
+ * Only the dmabuf to the actual table needs any manipulation.
+ */
+ memcpy(&ls, v4l2_ls, sizeof(ls));
+
+ dmabuf = dma_buf_get(v4l2_ls->dmabuf);
+ if (IS_ERR_OR_NULL(dmabuf))
+ return -EINVAL;
+
+ ret = vc_sm_cma_import_dmabuf(dmabuf, &vcsm_handle);
+ if (ret) {
+ dma_buf_put(dmabuf);
+ return -EINVAL;
+ }
+
+ ls.mem_handle_table = vc_sm_cma_int_handle(vcsm_handle);
+ if (ls.mem_handle_table)
+ /* The VPU will take a reference on the vcsm handle,
+ * which in turn will retain a reference on the dmabuf.
+ * This code can therefore safely release all
+ * references to the buffer.
+ */
+ ret = set_isp_param(node,
+ MMAL_PARAMETER_LENS_SHADING_OVERRIDE,
+ &ls,
+ sizeof(ls));
+ else
+ ret = -EINVAL;
+
+ vc_sm_cma_free(vcsm_handle);
+ dma_buf_put(dmabuf);
+ break;
+ }
+ case V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL:
+ ret = set_isp_param(node, MMAL_PARAMETER_BLACK_LEVEL,
+ ctrl->p_new.p_u8,
+ sizeof(struct bcm2835_isp_black_level));
+ break;
+ case V4L2_CID_USER_BCM2835_ISP_GEQ:
+ ret = set_isp_param(node, MMAL_PARAMETER_GEQ,
+ ctrl->p_new.p_u8,
+ sizeof(struct bcm2835_isp_geq));
+ break;
+ case V4L2_CID_USER_BCM2835_ISP_GAMMA:
+ ret = set_isp_param(node, MMAL_PARAMETER_GAMMA,
+ ctrl->p_new.p_u8,
+ sizeof(struct bcm2835_isp_gamma));
+ break;
+ case V4L2_CID_USER_BCM2835_ISP_DENOISE:
+ ret = set_isp_param(node, MMAL_PARAMETER_DENOISE,
+ ctrl->p_new.p_u8,
+ sizeof(struct bcm2835_isp_denoise));
+ break;
+ case V4L2_CID_USER_BCM2835_ISP_CDN:
+ ret = set_isp_param(node, MMAL_PARAMETER_CDN,
+ ctrl->p_new.p_u8,
+ sizeof(struct bcm2835_isp_cdn));
+ break;
+ case V4L2_CID_USER_BCM2835_ISP_SHARPEN:
+ ret = set_isp_param(node, MMAL_PARAMETER_SHARPEN,
+ ctrl->p_new.p_u8,
+ sizeof(struct bcm2835_isp_sharpen));
+ break;
+ case V4L2_CID_USER_BCM2835_ISP_DPC:
+ ret = set_isp_param(node, MMAL_PARAMETER_DPC,
+ ctrl->p_new.p_u8,
+ sizeof(struct bcm2835_isp_dpc));
+ break;
+ default:
+ v4l2_info(&dev->v4l2_dev, "Unrecognised control\n");
+ ret = -EINVAL;
+ }
+
+ if (ret) {
+ v4l2_err(&dev->v4l2_dev, "%s: Failed setting ctrl \"%s\" (%08x), err %d\n",
+ __func__, ctrl->name, ctrl->id, ret);
+ ret = -EIO;
+ }
+
+ return ret;
+}
+
+static const struct v4l2_ctrl_ops bcm2835_isp_ctrl_ops = {
+ .s_ctrl = bcm2835_isp_s_ctrl,
+};
+
+static const struct v4l2_file_operations bcm2835_isp_fops = {
+ .owner = THIS_MODULE,
+ .open = v4l2_fh_open,
+ .release = vb2_fop_release,
+ .poll = vb2_fop_poll,
+ .unlocked_ioctl = video_ioctl2,
+ .mmap = vb2_fop_mmap
+};
+
+static int populate_qdata_fmt(struct v4l2_format *f,
+ struct bcm2835_isp_node *node)
+{
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ struct bcm2835_isp_q_data *q_data = &node->q_data;
+ int ret;
+
+ if (!node_is_stats(node)) {
+ v4l2_dbg(1, debug, &dev->v4l2_dev,
+ "%s: Setting pix format for type %d, wxh: %ux%u, fmt: %08x, size %u\n",
+ __func__, f->type, f->fmt.pix.width, f->fmt.pix.height,
+ f->fmt.pix.pixelformat, f->fmt.pix.sizeimage);
+
+ q_data->fmt = find_format(f, node);
+ q_data->width = f->fmt.pix.width;
+ q_data->height = f->fmt.pix.height;
+ q_data->height = f->fmt.pix.height;
+
+ /* All parameters should have been set correctly by try_fmt */
+ q_data->bytesperline = f->fmt.pix.bytesperline;
+ q_data->sizeimage = f->fmt.pix.sizeimage;
+ } else {
+ v4l2_dbg(1, debug, &dev->v4l2_dev,
+ "%s: Setting meta format for fmt: %08x, size %u\n",
+ __func__, f->fmt.meta.dataformat,
+ f->fmt.meta.buffersize);
+
+ q_data->fmt = find_format(f, node);
+ q_data->width = 0;
+ q_data->height = 0;
+ q_data->bytesperline = 0;
+ q_data->sizeimage = f->fmt.meta.buffersize;
+ }
+
+ v4l2_dbg(1, debug, &dev->v4l2_dev,
+ "%s: Calculated bpl as %u, size %u\n", __func__,
+ q_data->bytesperline, q_data->sizeimage);
+
+ setup_mmal_port_format(node, node->port);
+ ret = vchiq_mmal_port_set_format(dev->mmal_instance, node->port);
+ if (ret) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s: Failed vchiq_mmal_port_set_format on port, ret %d\n",
+ __func__, ret);
+ ret = -EINVAL;
+ }
+
+ if (q_data->sizeimage < node->port->minimum_buffer.size) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s: Current buffer size of %u < min buf size %u - driver mismatch to MMAL\n",
+ __func__,
+ q_data->sizeimage,
+ node->port->minimum_buffer.size);
+ }
+
+ v4l2_dbg(1, debug, &dev->v4l2_dev,
+ "%s: Set format for type %d, wxh: %dx%d, fmt: %08x, size %u\n",
+ __func__, f->type, q_data->width, q_data->height,
+ q_data->fmt->fourcc, q_data->sizeimage);
+
+ return ret;
+}
+
+static int bcm2835_isp_node_querycap(struct file *file, void *priv,
+ struct v4l2_capability *cap)
+{
+ strscpy(cap->driver, BCM2835_ISP_NAME, sizeof(cap->driver));
+ strscpy(cap->card, BCM2835_ISP_NAME, sizeof(cap->card));
+ snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+ BCM2835_ISP_NAME);
+
+ return 0;
+}
+
+static int bcm2835_isp_node_g_fmt(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct bcm2835_isp_node *node = video_drvdata(file);
+
+ if (f->type != node->queue.type)
+ return -EINVAL;
+
+ if (node_is_stats(node)) {
+ f->fmt.meta.dataformat = V4L2_META_FMT_BCM2835_ISP_STATS;
+ f->fmt.meta.buffersize =
+ node->port->minimum_buffer.size;
+ } else {
+ struct bcm2835_isp_q_data *q_data = &node->q_data;
+
+ f->fmt.pix.width = q_data->width;
+ f->fmt.pix.height = q_data->height;
+ f->fmt.pix.field = V4L2_FIELD_NONE;
+ f->fmt.pix.pixelformat = q_data->fmt->fourcc;
+ f->fmt.pix.bytesperline = q_data->bytesperline;
+ f->fmt.pix.sizeimage = q_data->sizeimage;
+ f->fmt.pix.colorspace = q_data->fmt->colorspace;
+ }
+
+ return 0;
+}
+
+static int bcm2835_isp_node_enum_fmt(struct file *file, void *priv,
+ struct v4l2_fmtdesc *f)
+{
+ struct bcm2835_isp_node *node = video_drvdata(file);
+
+ if (f->type != node->queue.type)
+ return -EINVAL;
+
+ if (f->index < node->num_supported_fmts) {
+ /* Format found */
+ f->pixelformat = node->supported_fmts[f->index]->fourcc;
+ f->flags = 0;
+ return 0;
+ }
+
+ return -EINVAL;
+}
+
+static int bcm2835_isp_enum_framesizes(struct file *file, void *priv,
+ struct v4l2_frmsizeenum *fsize)
+{
+ struct bcm2835_isp_node *node = video_drvdata(file);
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ const struct bcm2835_isp_fmt *fmt;
+
+ if (node_is_stats(node) || fsize->index)
+ return -EINVAL;
+
+ fmt = find_format_by_fourcc(fsize->pixel_format, node);
+ if (!fmt) {
+ v4l2_err(&dev->v4l2_dev, "Invalid pixel code: %x\n",
+ fsize->pixel_format);
+ return -EINVAL;
+ }
+
+ fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+ fsize->stepwise.min_width = MIN_DIM;
+ fsize->stepwise.max_width = MAX_DIM;
+ fsize->stepwise.step_width = fmt->step_size;
+
+ fsize->stepwise.min_height = MIN_DIM;
+ fsize->stepwise.max_height = MAX_DIM;
+ fsize->stepwise.step_height = fmt->step_size;
+
+ return 0;
+}
+
+static int bcm2835_isp_node_try_fmt(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct bcm2835_isp_node *node = video_drvdata(file);
+ const struct bcm2835_isp_fmt *fmt;
+
+ if (f->type != node->queue.type)
+ return -EINVAL;
+
+ fmt = find_format(f, node);
+ if (!fmt)
+ fmt = get_default_format(node);
+
+ if (!node_is_stats(node)) {
+ f->fmt.pix.width = max(min(f->fmt.pix.width, MAX_DIM),
+ MIN_DIM);
+ f->fmt.pix.height = max(min(f->fmt.pix.height, MAX_DIM),
+ MIN_DIM);
+
+ f->fmt.pix.pixelformat = fmt->fourcc;
+ f->fmt.pix.colorspace = fmt->colorspace;
+ f->fmt.pix.bytesperline = get_bytesperline(f->fmt.pix.width,
+ fmt);
+ f->fmt.pix.field = V4L2_FIELD_NONE;
+ f->fmt.pix.sizeimage =
+ get_sizeimage(f->fmt.pix.bytesperline, f->fmt.pix.width,
+ f->fmt.pix.height, fmt);
+ } else {
+ f->fmt.meta.dataformat = fmt->fourcc;
+ f->fmt.meta.buffersize = node->port->minimum_buffer.size;
+ }
+
+ return 0;
+}
+
+static int bcm2835_isp_node_s_fmt(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct bcm2835_isp_node *node = video_drvdata(file);
+ int ret;
+
+ if (f->type != node->queue.type)
+ return -EINVAL;
+
+ ret = bcm2835_isp_node_try_fmt(file, priv, f);
+ if (ret)
+ return ret;
+
+ v4l2_dbg(1, debug, &node_get_dev(node)->v4l2_dev,
+ "%s: Set format for node %s[%d]\n",
+ __func__, node->name, node->id);
+
+ return populate_qdata_fmt(f, node);
+}
+
+static int bcm2835_isp_node_s_selection(struct file *file, void *fh,
+ struct v4l2_selection *s)
+{
+ struct mmal_parameter_crop crop;
+ struct bcm2835_isp_node *node = video_drvdata(file);
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+
+ /* This return value is required for V4L2 compliance. */
+ if (node_is_stats(node))
+ return -ENOTTY;
+
+ if (!s->r.width || !s->r.height)
+ return -EINVAL;
+
+ /* We can only set crop on the input. */
+ switch (s->target) {
+ case V4L2_SEL_TGT_CROP:
+ /*
+ * Adjust the crop window if it goes outside of the frame
+ * dimensions.
+ */
+ s->r.left = min((unsigned int)max(s->r.left, 0),
+ node->q_data.width - MIN_DIM);
+ s->r.top = min((unsigned int)max(s->r.top, 0),
+ node->q_data.height - MIN_DIM);
+ s->r.width = max(min(s->r.width,
+ node->q_data.width - s->r.left), MIN_DIM);
+ s->r.height = max(min(s->r.height,
+ node->q_data.height - s->r.top), MIN_DIM);
+ break;
+ case V4L2_SEL_TGT_CROP_DEFAULT:
+ /* Default (i.e. no) crop window. */
+ s->r.left = 0;
+ s->r.top = 0;
+ s->r.width = node->q_data.width;
+ s->r.height = node->q_data.height;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ crop.rect.x = s->r.left;
+ crop.rect.y = s->r.top;
+ crop.rect.width = s->r.width;
+ crop.rect.height = s->r.height;
+
+ return vchiq_mmal_port_parameter_set(dev->mmal_instance, node->port,
+ MMAL_PARAMETER_CROP,
+ &crop, sizeof(crop));
+}
+
+static int bcm2835_isp_node_g_selection(struct file *file, void *fh,
+ struct v4l2_selection *s)
+{
+ struct mmal_parameter_crop crop;
+ struct bcm2835_isp_node *node = video_drvdata(file);
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ u32 crop_size = sizeof(crop);
+ int ret;
+
+ /* We can only return out an input crop. */
+ switch (s->target) {
+ case V4L2_SEL_TGT_CROP:
+ ret = vchiq_mmal_port_parameter_get(dev->mmal_instance,
+ node->port,
+ MMAL_PARAMETER_CROP,
+ &crop, &crop_size);
+ if (!ret) {
+ s->r.left = crop.rect.x;
+ s->r.top = crop.rect.y;
+ s->r.width = crop.rect.width;
+ s->r.height = crop.rect.height;
+ }
+ break;
+ case V4L2_SEL_TGT_CROP_DEFAULT:
+ case V4L2_SEL_TGT_CROP_BOUNDS:
+ /* Default (i.e. no) crop window. */
+ s->r.left = 0;
+ s->r.top = 0;
+ s->r.width = node->q_data.width;
+ s->r.height = node->q_data.height;
+ ret = 0;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
+static int bcm3285_isp_subscribe_event(struct v4l2_fh *fh,
+ const struct v4l2_event_subscription *s)
+{
+ switch (s->type) {
+ /* Cannot change source parameters dynamically at runtime. */
+ case V4L2_EVENT_SOURCE_CHANGE:
+ return -EINVAL;
+ case V4L2_EVENT_CTRL:
+ return v4l2_ctrl_subscribe_event(fh, s);
+ default:
+ return v4l2_event_subscribe(fh, s, 4, NULL);
+ }
+}
+
+static const struct v4l2_ioctl_ops bcm2835_isp_node_ioctl_ops = {
+ .vidioc_querycap = bcm2835_isp_node_querycap,
+ .vidioc_g_fmt_vid_cap = bcm2835_isp_node_g_fmt,
+ .vidioc_g_fmt_vid_out = bcm2835_isp_node_g_fmt,
+ .vidioc_g_fmt_meta_cap = bcm2835_isp_node_g_fmt,
+ .vidioc_s_fmt_vid_cap = bcm2835_isp_node_s_fmt,
+ .vidioc_s_fmt_vid_out = bcm2835_isp_node_s_fmt,
+ .vidioc_s_fmt_meta_cap = bcm2835_isp_node_s_fmt,
+ .vidioc_try_fmt_vid_cap = bcm2835_isp_node_try_fmt,
+ .vidioc_try_fmt_vid_out = bcm2835_isp_node_try_fmt,
+ .vidioc_try_fmt_meta_cap = bcm2835_isp_node_try_fmt,
+ .vidioc_s_selection = bcm2835_isp_node_s_selection,
+ .vidioc_g_selection = bcm2835_isp_node_g_selection,
+
+ .vidioc_enum_fmt_vid_cap = bcm2835_isp_node_enum_fmt,
+ .vidioc_enum_fmt_vid_out = bcm2835_isp_node_enum_fmt,
+ .vidioc_enum_fmt_meta_cap = bcm2835_isp_node_enum_fmt,
+ .vidioc_enum_framesizes = bcm2835_isp_enum_framesizes,
+
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_qbuf = vb2_ioctl_qbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_expbuf = vb2_ioctl_expbuf,
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+
+ .vidioc_subscribe_event = bcm3285_isp_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+/*
+ * Size of the array to provide to the VPU when asking for the list of supported
+ * formats.
+ *
+ * The ISP component currently advertises 62 input formats, so add a small
+ * overhead on that. Should the component advertise more formats then the excess
+ * will be dropped and a warning logged.
+ */
+#define MAX_SUPPORTED_ENCODINGS 70
+
+/* Populate node->supported_fmts with the formats supported by those ports. */
+static int bcm2835_isp_get_supported_fmts(struct bcm2835_isp_node *node)
+{
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ struct bcm2835_isp_fmt const **list;
+ unsigned int i, j, num_encodings;
+ u32 fourccs[MAX_SUPPORTED_ENCODINGS];
+ u32 param_size = sizeof(fourccs);
+ int ret;
+
+ ret = vchiq_mmal_port_parameter_get(dev->mmal_instance, node->port,
+ MMAL_PARAMETER_SUPPORTED_ENCODINGS,
+ &fourccs, ¶m_size);
+
+ if (ret) {
+ if (ret == MMAL_MSG_STATUS_ENOSPC) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s: port has more encodings than we provided space for. Some are dropped (%zu vs %u).\n",
+ __func__, param_size / sizeof(u32),
+ MAX_SUPPORTED_ENCODINGS);
+ num_encodings = MAX_SUPPORTED_ENCODINGS;
+ } else {
+ v4l2_err(&dev->v4l2_dev, "%s: get_param ret %u.\n",
+ __func__, ret);
+ return -EINVAL;
+ }
+ } else {
+ num_encodings = param_size / sizeof(u32);
+ }
+
+ /*
+ * Assume at this stage that all encodings will be supported in V4L2.
+ * Any that aren't supported will waste a very small amount of memory.
+ */
+ list = devm_kzalloc(dev->dev,
+ sizeof(struct bcm2835_isp_fmt *) * num_encodings,
+ GFP_KERNEL);
+ if (!list)
+ return -ENOMEM;
+ node->supported_fmts = list;
+
+ for (i = 0, j = 0; i < num_encodings; i++) {
+ const struct bcm2835_isp_fmt *fmt = get_fmt(fourccs[i]);
+
+ if (fmt) {
+ list[j] = fmt;
+ j++;
+ }
+ }
+ node->num_supported_fmts = j;
+
+ return 0;
+}
+
+/*
+ * Register a device node /dev/video<N> to go along with one of the ISP's input
+ * or output nodes.
+ */
+static int register_node(struct bcm2835_isp_dev *dev,
+ struct bcm2835_isp_node *node,
+ int index)
+{
+ struct video_device *vfd;
+ struct vb2_queue *queue;
+ int ret;
+
+ mutex_init(&node->lock);
+ mutex_init(&node->queue_lock);
+
+ node->dev = dev;
+ vfd = &node->vfd;
+ queue = &node->queue;
+ queue->type = index_to_queue_type(index);
+ /*
+ * Setup the node type-specific params.
+ *
+ * Only the OUTPUT node can set controls and crop windows. However,
+ * we must allow the s/g_selection ioctl on the stats node as v4l2
+ * compliance expects it to return a -ENOTTY, and the framework
+ * does not handle it if the ioctl is disabled.
+ */
+ switch (queue->type) {
+ case V4L2_BUF_TYPE_VIDEO_OUTPUT:
+ vfd->device_caps = V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_STREAMING;
+ node->id = index;
+ node->vfl_dir = VFL_DIR_TX;
+ node->name = "output";
+ node->port = &dev->component->input[node->id];
+ break;
+ case V4L2_BUF_TYPE_VIDEO_CAPTURE:
+ vfd->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
+ /* First Capture node starts at id 0, etc. */
+ node->id = index - BCM2835_ISP_NUM_OUTPUTS;
+ node->vfl_dir = VFL_DIR_RX;
+ node->name = "capture";
+ node->port = &dev->component->output[node->id];
+ v4l2_disable_ioctl(&node->vfd, VIDIOC_S_CTRL);
+ v4l2_disable_ioctl(&node->vfd, VIDIOC_S_SELECTION);
+ v4l2_disable_ioctl(&node->vfd, VIDIOC_G_SELECTION);
+ break;
+ case V4L2_BUF_TYPE_META_CAPTURE:
+ vfd->device_caps = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING;
+ node->id = index - BCM2835_ISP_NUM_OUTPUTS;
+ node->vfl_dir = VFL_DIR_RX;
+ node->name = "stats";
+ node->port = &dev->component->output[node->id];
+ v4l2_disable_ioctl(&node->vfd, VIDIOC_S_CTRL);
+ v4l2_disable_ioctl(&node->vfd, VIDIOC_S_SELECTION);
+ v4l2_disable_ioctl(&node->vfd, VIDIOC_G_SELECTION);
+ break;
+ }
+
+ /* We use the selection API instead of the old crop API. */
+ v4l2_disable_ioctl(vfd, VIDIOC_CROPCAP);
+ v4l2_disable_ioctl(vfd, VIDIOC_G_CROP);
+ v4l2_disable_ioctl(vfd, VIDIOC_S_CROP);
+
+ ret = bcm2835_isp_get_supported_fmts(node);
+ if (ret)
+ return ret;
+
+ /* Initialise the video node. */
+ vfd->vfl_type = VFL_TYPE_VIDEO;
+ vfd->fops = &bcm2835_isp_fops,
+ vfd->ioctl_ops = &bcm2835_isp_node_ioctl_ops,
+ vfd->minor = -1,
+ vfd->release = video_device_release_empty,
+ vfd->queue = &node->queue;
+ vfd->lock = &node->lock;
+ vfd->v4l2_dev = &dev->v4l2_dev;
+ vfd->vfl_dir = node->vfl_dir;
+
+ node->q_data.fmt = get_default_format(node);
+ node->q_data.width = DEFAULT_DIM;
+ node->q_data.height = DEFAULT_DIM;
+ node->q_data.bytesperline =
+ get_bytesperline(DEFAULT_DIM, node->q_data.fmt);
+ node->q_data.sizeimage = node_is_stats(node) ?
+ node->port->recommended_buffer.size :
+ get_sizeimage(node->q_data.bytesperline,
+ node->q_data.width,
+ node->q_data.height,
+ node->q_data.fmt);
+
+ queue->io_modes = VB2_MMAP | VB2_DMABUF;
+ queue->drv_priv = node;
+ queue->ops = &bcm2835_isp_node_queue_ops;
+ queue->mem_ops = &vb2_dma_contig_memops;
+ queue->buf_struct_size = sizeof(struct bcm2835_isp_buffer);
+ queue->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ queue->dev = dev->dev;
+ queue->lock = &node->queue_lock;
+
+ ret = vb2_queue_init(queue);
+ if (ret < 0) {
+ v4l2_info(&dev->v4l2_dev, "vb2_queue_init failed\n");
+ return ret;
+ }
+
+ /* Set some controls and defaults, but only on the VIDEO_OUTPUT node. */
+ if (node_is_output(node)) {
+ unsigned int i;
+
+ /* Use this ctrl template to assign custom ISP ctrls. */
+ struct v4l2_ctrl_config ctrl_template = {
+ .ops = &bcm2835_isp_ctrl_ops,
+ .type = V4L2_CTRL_TYPE_U8,
+ .def = 0,
+ .min = 0x00,
+ .max = 0xff,
+ .step = 1,
+ };
+
+ /* 3 standard controls, and an array of custom controls */
+ ret = v4l2_ctrl_handler_init(&dev->ctrl_handler,
+ 3 + ARRAY_SIZE(custom_ctrls));
+ if (ret) {
+ v4l2_err(&dev->v4l2_dev, "ctrl_handler init failed (%d)\n",
+ ret);
+ goto queue_cleanup;
+ }
+
+ dev->r_gain = 1000;
+ dev->b_gain = 1000;
+
+ v4l2_ctrl_new_std(&dev->ctrl_handler, &bcm2835_isp_ctrl_ops,
+ V4L2_CID_RED_BALANCE, 1, 0xffff, 1,
+ dev->r_gain);
+
+ v4l2_ctrl_new_std(&dev->ctrl_handler, &bcm2835_isp_ctrl_ops,
+ V4L2_CID_BLUE_BALANCE, 1, 0xffff, 1,
+ dev->b_gain);
+
+ v4l2_ctrl_new_std(&dev->ctrl_handler, &bcm2835_isp_ctrl_ops,
+ V4L2_CID_DIGITAL_GAIN, 1, 0xffff, 1, 1000);
+
+ for (i = 0; i < ARRAY_SIZE(custom_ctrls); i++) {
+ ctrl_template.name = custom_ctrls[i].name;
+ ctrl_template.id = custom_ctrls[i].id;
+ ctrl_template.dims[0] = custom_ctrls[i].size;
+ ctrl_template.flags = custom_ctrls[i].flags;
+ v4l2_ctrl_new_custom(&dev->ctrl_handler,
+ &ctrl_template, NULL);
+ }
+
+ node->vfd.ctrl_handler = &dev->ctrl_handler;
+ if (dev->ctrl_handler.error) {
+ ret = dev->ctrl_handler.error;
+ v4l2_err(&dev->v4l2_dev, "controls init failed (%d)\n",
+ ret);
+ v4l2_ctrl_handler_free(&dev->ctrl_handler);
+ goto ctrl_cleanup;
+ }
+ }
+
+ /* Define the device names */
+ snprintf(vfd->name, sizeof(node->vfd.name), "%s-%s%d", BCM2835_ISP_NAME,
+ node->name, node->id);
+
+ ret = video_register_device(vfd, VFL_TYPE_VIDEO, video_nr + index);
+ if (ret) {
+ v4l2_err(&dev->v4l2_dev,
+ "Failed to register video %s[%d] device node\n",
+ node->name, node->id);
+ goto ctrl_cleanup;
+ }
+
+ node->registered = true;
+ video_set_drvdata(vfd, node);
+
+ v4l2_info(&dev->v4l2_dev,
+ "Device node %s[%d] registered as /dev/video%d\n",
+ node->name, node->id, vfd->num);
+
+ return 0;
+
+ctrl_cleanup:
+ if (node_is_output(node))
+ v4l2_ctrl_handler_free(&dev->ctrl_handler);
+queue_cleanup:
+ vb2_queue_release(&node->queue);
+ return ret;
+}
+
+/* Unregister one of the /dev/video<N> nodes associated with the ISP. */
+static void bcm2835_isp_unregister_node(struct bcm2835_isp_node *node)
+{
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+
+ v4l2_info(&dev->v4l2_dev,
+ "Unregistering node %s[%d] device node /dev/video%d\n",
+ node->name, node->id, node->vfd.num);
+
+ if (node->registered) {
+ video_unregister_device(&node->vfd);
+ if (node_is_output(node))
+ v4l2_ctrl_handler_free(&dev->ctrl_handler);
+ vb2_queue_release(&node->queue);
+ }
+
+ /*
+ * node->supported_fmts.list is free'd automatically
+ * as a managed resource.
+ */
+ node->supported_fmts = NULL;
+ node->num_supported_fmts = 0;
+ node->vfd.ctrl_handler = NULL;
+ node->registered = false;
+}
+
+static void media_controller_unregister(struct bcm2835_isp_dev *dev)
+{
+ unsigned int i;
+
+ v4l2_info(&dev->v4l2_dev, "Unregister from media controller\n");
+
+ if (dev->media_device_registered) {
+ media_device_unregister(&dev->mdev);
+ media_device_cleanup(&dev->mdev);
+ dev->media_device_registered = false;
+ }
+
+ kfree(dev->entity.name);
+ dev->entity.name = NULL;
+
+ if (dev->media_entity_registered) {
+ media_device_unregister_entity(&dev->entity);
+ dev->media_entity_registered = false;
+ }
+
+ for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
+ struct bcm2835_isp_node *node = &dev->node[i];
+
+ if (node->media_node_registered) {
+ media_remove_intf_links(node->intf_link->intf);
+ media_entity_remove_links(&dev->node[i].vfd.entity);
+ media_devnode_remove(node->intf_devnode);
+ media_device_unregister_entity(&node->vfd.entity);
+ kfree(node->vfd.entity.name);
+ }
+ node->media_node_registered = false;
+ }
+
+ dev->v4l2_dev.mdev = NULL;
+}
+
+static int media_controller_register_node(struct bcm2835_isp_dev *dev, int num)
+{
+ struct bcm2835_isp_node *node = &dev->node[num];
+ struct media_entity *entity = &node->vfd.entity;
+ int output = node_is_output(node);
+ char *name;
+ int ret;
+
+ v4l2_info(&dev->v4l2_dev,
+ "Register %s node %d with media controller\n",
+ output ? "output" : "capture", num);
+ entity->obj_type = MEDIA_ENTITY_TYPE_VIDEO_DEVICE;
+ entity->function = MEDIA_ENT_F_IO_V4L;
+ entity->info.dev.major = VIDEO_MAJOR;
+ entity->info.dev.minor = node->vfd.minor;
+ name = kmalloc(BCM2835_ISP_ENTITY_NAME_LEN, GFP_KERNEL);
+ if (!name) {
+ ret = -ENOMEM;
+ goto error_no_mem;
+ }
+ snprintf(name, BCM2835_ISP_ENTITY_NAME_LEN, "%s0-%s%d",
+ BCM2835_ISP_NAME, output ? "output" : "capture", num);
+ entity->name = name;
+ node->pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
+ ret = media_entity_pads_init(entity, 1, &node->pad);
+ if (ret)
+ goto error_pads_init;
+ ret = media_device_register_entity(&dev->mdev, entity);
+ if (ret)
+ goto error_register_entity;
+
+ node->intf_devnode = media_devnode_create(&dev->mdev,
+ MEDIA_INTF_T_V4L_VIDEO, 0,
+ VIDEO_MAJOR, node->vfd.minor);
+ if (!node->intf_devnode) {
+ ret = -ENOMEM;
+ goto error_devnode_create;
+ }
+
+ node->intf_link = media_create_intf_link(entity,
+ &node->intf_devnode->intf,
+ MEDIA_LNK_FL_IMMUTABLE |
+ MEDIA_LNK_FL_ENABLED);
+ if (!node->intf_link) {
+ ret = -ENOMEM;
+ goto error_create_intf_link;
+ }
+
+ if (output)
+ ret = media_create_pad_link(entity, 0, &dev->entity, num,
+ MEDIA_LNK_FL_IMMUTABLE |
+ MEDIA_LNK_FL_ENABLED);
+ else
+ ret = media_create_pad_link(&dev->entity, num, entity, 0,
+ MEDIA_LNK_FL_IMMUTABLE |
+ MEDIA_LNK_FL_ENABLED);
+ if (ret)
+ goto error_create_pad_link;
+
+ dev->node[num].media_node_registered = true;
+ return 0;
+
+error_create_pad_link:
+ media_remove_intf_links(&node->intf_devnode->intf);
+error_create_intf_link:
+ media_devnode_remove(node->intf_devnode);
+error_devnode_create:
+ media_device_unregister_entity(&node->vfd.entity);
+error_register_entity:
+error_pads_init:
+ kfree(entity->name);
+ entity->name = NULL;
+error_no_mem:
+ if (ret)
+ v4l2_info(&dev->v4l2_dev, "Error registering node\n");
+
+ return ret;
+}
+
+static int media_controller_register(struct bcm2835_isp_dev *dev)
+{
+ char *name;
+ unsigned int i;
+ int ret;
+
+ v4l2_dbg(2, debug, &dev->v4l2_dev, "Registering with media controller\n");
+ dev->mdev.dev = dev->dev;
+ strscpy(dev->mdev.model, "bcm2835-isp",
+ sizeof(dev->mdev.model));
+ strscpy(dev->mdev.bus_info, "platform:bcm2835-isp",
+ sizeof(dev->mdev.bus_info));
+ media_device_init(&dev->mdev);
+ dev->v4l2_dev.mdev = &dev->mdev;
+
+ v4l2_dbg(2, debug, &dev->v4l2_dev, "Register entity for nodes\n");
+
+ name = kmalloc(BCM2835_ISP_ENTITY_NAME_LEN, GFP_KERNEL);
+ if (!name) {
+ ret = -ENOMEM;
+ goto done;
+ }
+ snprintf(name, BCM2835_ISP_ENTITY_NAME_LEN, "bcm2835_isp0");
+ dev->entity.name = name;
+ dev->entity.obj_type = MEDIA_ENTITY_TYPE_BASE;
+ dev->entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
+
+ for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
+ dev->pad[i].flags = node_is_output(&dev->node[i]) ?
+ MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+ }
+
+ ret = media_entity_pads_init(&dev->entity, BCM2835_ISP_NUM_NODES,
+ dev->pad);
+ if (ret)
+ goto done;
+
+ ret = media_device_register_entity(&dev->mdev, &dev->entity);
+ if (ret)
+ goto done;
+
+ dev->media_entity_registered = true;
+ for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
+ ret = media_controller_register_node(dev, i);
+ if (ret)
+ goto done;
+ }
+
+ ret = media_device_register(&dev->mdev);
+ if (!ret)
+ dev->media_device_registered = true;
+done:
+ return ret;
+}
+
+static void bcm2835_isp_remove(struct vchiq_device *device)
+{
+ struct bcm2835_isp_dev *dev = vchiq_get_drvdata(device);
+ unsigned int i;
+
+ media_controller_unregister(dev);
+
+ for (i = 0; i < BCM2835_ISP_NUM_NODES; i++)
+ bcm2835_isp_unregister_node(&dev->node[i]);
+
+ v4l2_device_unregister(&dev->v4l2_dev);
+
+ if (dev->component)
+ vchiq_mmal_component_finalise(dev->mmal_instance,
+ dev->component);
+
+ vchiq_mmal_finalise(dev->mmal_instance);
+}
+
+static int bcm2835_isp_probe(struct vchiq_device *device)
+{
+ struct bcm2835_isp_dev *dev;
+ unsigned int i;
+ int ret;
+
+ dev = devm_kzalloc(&device->dev, sizeof(*dev), GFP_KERNEL);
+ if (!dev)
+ return -ENOMEM;
+
+ dev->dev = &device->dev;
+
+ ret = v4l2_device_register(&device->dev, &dev->v4l2_dev);
+ if (ret)
+ return ret;
+
+ ret = vchiq_mmal_init(&dev->mmal_instance);
+ if (ret) {
+ v4l2_device_unregister(&dev->v4l2_dev);
+ return ret;
+ }
+
+ ret = vchiq_mmal_component_init(dev->mmal_instance, "ril.isp",
+ &dev->component);
+ if (ret) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s: failed to create ril.isp component\n", __func__);
+ goto error;
+ }
+
+ if (dev->component->inputs < BCM2835_ISP_NUM_OUTPUTS ||
+ dev->component->outputs < BCM2835_ISP_NUM_CAPTURES +
+ BCM2835_ISP_NUM_METADATA) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s: ril.isp returned %d i/p (%d expected), %d o/p (%d expected) ports\n",
+ __func__, dev->component->inputs,
+ BCM2835_ISP_NUM_OUTPUTS,
+ dev->component->outputs,
+ BCM2835_ISP_NUM_CAPTURES + BCM2835_ISP_NUM_METADATA);
+ goto error;
+ }
+
+ atomic_set(&dev->num_streaming, 0);
+
+ for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
+ struct bcm2835_isp_node *node = &dev->node[i];
+
+ ret = register_node(dev, node, i);
+ if (ret)
+ goto error;
+ }
+
+ ret = media_controller_register(dev);
+ if (ret)
+ goto error;
+
+ vchiq_set_drvdata(device, dev);
+ v4l2_info(&dev->v4l2_dev, "Loaded V4L2 %s\n", BCM2835_ISP_NAME);
+ return 0;
+
+error:
+ bcm2835_isp_remove(device);
+
+ return ret;
+}
+
+static struct vchiq_device_id device_id_table[] = {
+ { .name = "bcm2835-isp" },
+ {}
+};
+MODULE_DEVICE_TABLE(vchiq, device_id_table);
+
+static struct vchiq_driver bcm2835_isp = {
+ .probe = bcm2835_isp_probe,
+ .remove = bcm2835_isp_remove,
+ .id_table = device_id_table,
+ .driver = {
+ .name = BCM2835_ISP_NAME,
+ },
+};
+
+module_vchiq_driver(bcm2835_isp);
+
+MODULE_DESCRIPTION("BCM2835 ISP driver");
+MODULE_AUTHOR("Naushir Patuck <naush@raspberrypi.com>");
+MODULE_LICENSE("GPL");
+MODULE_VERSION("1.0");
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
index 4711877a9711..538c347db433 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
@@ -135,6 +135,10 @@
*/
#define MMAL_ENCODING_EGL_IMAGE MMAL_FOURCC('E', 'G', 'L', 'I')
+/** ISP image statistics format
+ */
+#define MMAL_ENCODING_BRCM_STATS MMAL_FOURCC('S', 'T', 'A', 'T')
+
/* }@ */
/** \name Pre-defined audio encodings */
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
index a0cdd28101f2..5e6276aca143 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
@@ -223,6 +223,63 @@ enum mmal_parameter_camera_type {
MMAL_PARAMETER_SHUTTER_SPEED,
/**< Takes a @ref MMAL_PARAMETER_AWB_GAINS_T */
MMAL_PARAMETER_CUSTOM_AWB_GAINS,
+ /**< Takes a @ref MMAL_PARAMETER_CAMERA_SETTINGS_T */
+ MMAL_PARAMETER_CAMERA_SETTINGS,
+ /**< Takes a @ref MMAL_PARAMETER_PRIVACY_INDICATOR_T */
+ MMAL_PARAMETER_PRIVACY_INDICATOR,
+ /**< Takes a @ref MMAL_PARAMETER_BOOLEAN_T */
+ MMAL_PARAMETER_VIDEO_DENOISE,
+ /**< Takes a @ref MMAL_PARAMETER_BOOLEAN_T */
+ MMAL_PARAMETER_STILLS_DENOISE,
+ /**< Takes a @ref MMAL_PARAMETER_CAMERA_ANNOTATE_T */
+ MMAL_PARAMETER_ANNOTATE,
+ /**< Takes a @ref MMAL_PARAMETER_STEREOSCOPIC_MODE_T */
+ MMAL_PARAMETER_STEREOSCOPIC_MODE,
+ /**< Takes a @ref MMAL_PARAMETER_CAMERA_INTERFACE_T */
+ MMAL_PARAMETER_CAMERA_INTERFACE,
+ /**< Takes a @ref MMAL_PARAMETER_CAMERA_CLOCKING_MODE_T */
+ MMAL_PARAMETER_CAMERA_CLOCKING_MODE,
+ /**< Takes a @ref MMAL_PARAMETER_CAMERA_RX_CONFIG_T */
+ MMAL_PARAMETER_CAMERA_RX_CONFIG,
+ /**< Takes a @ref MMAL_PARAMETER_CAMERA_RX_TIMING_T */
+ MMAL_PARAMETER_CAMERA_RX_TIMING,
+ /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
+ MMAL_PARAMETER_DPF_CONFIG,
+ /* 0x50 */
+ /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
+ MMAL_PARAMETER_JPEG_RESTART_INTERVAL,
+ /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
+ MMAL_PARAMETER_CAMERA_ISP_BLOCK_OVERRIDE,
+ /**< Takes a @ref MMAL_PARAMETER_LENS_SHADING_T */
+ MMAL_PARAMETER_LENS_SHADING_OVERRIDE,
+ /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
+ MMAL_PARAMETER_BLACK_LEVEL,
+ /**< Takes a @ref MMAL_PARAMETER_RESIZE_T */
+ MMAL_PARAMETER_RESIZE_PARAMS,
+ /**< Takes a @ref MMAL_PARAMETER_CROP_T */
+ MMAL_PARAMETER_CROP,
+ /**< Takes a @ref MMAL_PARAMETER_INT32_T */
+ MMAL_PARAMETER_OUTPUT_SHIFT,
+ /**< Takes a @ref MMAL_PARAMETER_INT32_T */
+ MMAL_PARAMETER_CCM_SHIFT,
+ /**< Takes a @ref MMAL_PARAMETER_CUSTOM_CCM_T */
+ MMAL_PARAMETER_CUSTOM_CCM,
+ /**< Takes a @ref MMAL_PARAMETER_RATIONAL_T */
+ MMAL_PARAMETER_ANALOG_GAIN,
+ /**< Takes a @ref MMAL_PARAMETER_RATIONAL_T */
+ MMAL_PARAMETER_DIGITAL_GAIN,
+ /**< Takes a @ref MMAL_PARAMETER_DENOISE_T */
+ MMAL_PARAMETER_DENOISE,
+ /**< Takes a @ref MMAL_PARAMETER_SHARPEN_T */
+ MMAL_PARAMETER_SHARPEN,
+ /**< Takes a @ref MMAL_PARAMETER_GEQ_T */
+ MMAL_PARAMETER_GEQ,
+ /**< Tales a @ref MMAP_PARAMETER_DPC_T */
+ MMAL_PARAMETER_DPC,
+ /**< Tales a @ref MMAP_PARAMETER_GAMMA_T */
+ MMAL_PARAMETER_GAMMA,
+ /**< Takes a @ref MMAL_PARAMETER_CDN_T */
+ MMAL_PARAMETER_CDN,
};
enum mmal_parameter_camera_config_timestamp_mode {
@@ -746,7 +803,113 @@ struct mmal_parameter_camera_info {
struct mmal_parameter_camera_info_camera
cameras[MMAL_PARAMETER_CAMERA_INFO_MAX_CAMERAS];
struct mmal_parameter_camera_info_flash
- flashes[MMAL_PARAMETER_CAMERA_INFO_MAX_FLASHES];
+ flashes[MMAL_PARAMETER_CAMERA_INFO_MAX_FLASHES];
+};
+
+struct mmal_parameter_ccm {
+ struct s32_fract cm[3][3];
+ s32 offsets[3];
+};
+
+struct mmal_parameter_custom_ccm {
+ u32 enabled; /**< Enable the custom CCM. */
+ struct mmal_parameter_ccm ccm; /**< CCM to be used. */
+};
+
+struct mmal_parameter_lens_shading {
+ u32 enabled;
+ u32 grid_cell_size;
+ u32 grid_width;
+ u32 grid_stride;
+ u32 grid_height;
+ u32 mem_handle_table;
+ u32 ref_transform;
+};
+
+enum mmal_parameter_ls_gain_format_type {
+ MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U0P8_1 = 0,
+ MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U1P7_0 = 1,
+ MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U1P7_1 = 2,
+ MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U2P6_0 = 3,
+ MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U2P6_1 = 4,
+ MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U3P5_0 = 5,
+ MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U3P5_1 = 6,
+ MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U4P10 = 7,
+ MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_DUMMY = 0x7FFFFFFF
+};
+
+struct mmal_parameter_lens_shading_v2 {
+ u32 enabled;
+ u32 grid_cell_size;
+ u32 grid_width;
+ u32 grid_stride;
+ u32 grid_height;
+ u32 mem_handle_table;
+ u32 ref_transform;
+ u32 corner_sampled;
+ enum mmal_parameter_ls_gain_format_type gain_format;
+};
+
+struct mmal_parameter_black_level {
+ u32 enabled;
+ u16 black_level_r;
+ u16 black_level_g;
+ u16 black_level_b;
+ u8 pad_[2]; /* Unused */
+};
+
+struct mmal_parameter_geq {
+ u32 enabled;
+ u32 offset;
+ struct s32_fract slope;
+};
+
+#define MMAL_NUM_GAMMA_PTS 33
+struct mmal_parameter_gamma {
+ u32 enabled;
+ u16 x[MMAL_NUM_GAMMA_PTS];
+ u16 y[MMAL_NUM_GAMMA_PTS];
+};
+
+enum mmal_parameter_cdn_mode {
+ MMAL_PARAM_CDN_FAST = 0,
+ MMAL_PARAM_CDN_HIGH_QUALITY = 1,
+ MMAL_PARAM_CDN_DUMMY = 0x7FFFFFFF
+};
+
+struct mmal_parameter_colour_denoise {
+ u32 enabled;
+ enum mmal_parameter_cdn_mode mode;
+};
+
+struct mmal_parameter_denoise {
+ u32 enabled;
+ u32 constant;
+ struct s32_fract slope;
+ struct s32_fract strength;
+};
+
+struct mmal_parameter_sharpen {
+ u32 enabled;
+ struct s32_fract threshold;
+ struct s32_fract strength;
+ struct s32_fract limit;
+};
+
+enum mmal_dpc_mode {
+ MMAL_DPC_MODE_OFF = 0,
+ MMAL_DPC_MODE_NORMAL = 1,
+ MMAL_DPC_MODE_STRONG = 2,
+ MMAL_DPC_MODE_MAX = 0x7FFFFFFF,
+};
+
+struct mmal_parameter_dpc {
+ u32 enabled;
+ u32 strength;
+};
+
+struct mmal_parameter_crop {
+ struct vchiq_mmal_rect rect;
};
#endif
diff --git a/include/uapi/linux/bcm2835-isp.h b/include/uapi/linux/bcm2835-isp.h
index cf8c0437f159..c50e3ca81565 100644
--- a/include/uapi/linux/bcm2835-isp.h
+++ b/include/uapi/linux/bcm2835-isp.h
@@ -31,6 +31,8 @@
(V4L2_CID_USER_BCM2835_ISP_BASE + 0x0007)
#define V4L2_CID_USER_BCM2835_ISP_DPC \
(V4L2_CID_USER_BCM2835_ISP_BASE + 0x0008)
+#define V4L2_CID_USER_BCM2835_ISP_CDN \
+ (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0009)
/*
* All structs below are directly mapped onto the equivalent structs in
@@ -175,6 +177,31 @@ struct bcm2835_isp_gamma {
__u16 y[BCM2835_NUM_GAMMA_PTS];
};
+/**
+ * enum bcm2835_isp_cdn_mode - Mode of operation for colour denoise.
+ *
+ * @CDN_MODE_FAST: Fast (but lower quality) colour denoise
+ * algorithm, typically used for video recording.
+ * @CDN_HIGH_QUALITY: High quality (but slower) colour denoise
+ * algorithm, typically used for stills capture.
+ */
+enum bcm2835_isp_cdn_mode {
+ CDN_MODE_FAST = 0,
+ CDN_MODE_HIGH_QUALITY = 1,
+};
+
+/**
+ * struct bcm2835_isp_cdn - Colour denoise parameters set with the
+ * V4L2_CID_USER_BCM2835_ISP_CDN ctrl.
+ *
+ * @enabled: Enable colour denoise.
+ * @mode: Colour denoise operating mode (see enum &bcm2835_isp_cdn_mode)
+ */
+struct bcm2835_isp_cdn {
+ __u32 enabled;
+ __u32 mode;
+};
+
/**
* struct bcm2835_isp_denoise - Denoise parameters set with the
* V4L2_CID_USER_BCM2835_ISP_DENOISE ctrl.
--
2.41.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 11/15] vc04_services: bcm2835-isp: Allow formats with different colour spaces
2023-11-09 21:02 [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Umang Jain
` (9 preceding siblings ...)
2023-11-09 21:03 ` [PATCH v2 10/15] staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component Umang Jain
@ 2023-11-09 21:03 ` Umang Jain
2023-11-09 21:03 ` [PATCH v2 12/15] vc04_services: bcm2835-isp: Permit all sRGB colour spaces on ISP outputs Umang Jain
` (5 subsequent siblings)
16 siblings, 0 replies; 45+ messages in thread
From: Umang Jain @ 2023-11-09 21:03 UTC (permalink / raw)
To: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Stefan Wahren,
David Plowman, Umang Jain
From: David Plowman <david.plowman@raspberrypi.com>
Each supported format now includes a mask showing the allowed colour
spaces, as well as a default colour space for when one was not
specified.
Additionally we translate the colour space to mmal format and pass it
over to the VideoCore.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
---
.../bcm2835-isp/bcm2835-isp-fmts.h | 180 ++++++++++++------
.../bcm2835-isp/bcm2835-v4l2-isp.c | 69 ++++++-
2 files changed, 193 insertions(+), 56 deletions(-)
diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
index 6dcd196fc6a5..5c874cdd2fe7 100644
--- a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
+++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
@@ -21,10 +21,29 @@ struct bcm2835_isp_fmt {
int bytesperline_align;
u32 mmal_fmt;
int size_multiplier_x2;
- enum v4l2_colorspace colorspace;
+ u32 colorspace_mask;
+ enum v4l2_colorspace colorspace_default;
unsigned int step_size;
};
+#define V4L2_COLORSPACE_MASK(colorspace) BIT(colorspace)
+
+#define V4L2_COLORSPACE_MASK_JPEG V4L2_COLORSPACE_MASK(V4L2_COLORSPACE_JPEG)
+#define V4L2_COLORSPACE_MASK_SMPTE170M V4L2_COLORSPACE_MASK(V4L2_COLORSPACE_SMPTE170M)
+#define V4L2_COLORSPACE_MASK_REC709 V4L2_COLORSPACE_MASK(V4L2_COLORSPACE_REC709)
+#define V4L2_COLORSPACE_MASK_SRGB V4L2_COLORSPACE_MASK(V4L2_COLORSPACE_SRGB)
+#define V4L2_COLORSPACE_MASK_RAW V4L2_COLORSPACE_MASK(V4L2_COLORSPACE_RAW)
+
+/*
+ * The colour spaces we support for YUV outputs. SRGB features here because,
+ * once you assign the default transfer func and so on, it and JPEG effectively
+ * mean the same.
+ */
+#define V4L2_COLORSPACE_MASK_YUV (V4L2_COLORSPACE_MASK_JPEG | \
+ V4L2_COLORSPACE_MASK_SRGB | \
+ V4L2_COLORSPACE_MASK_SMPTE170M | \
+ V4L2_COLORSPACE_MASK_REC709)
+
static const struct bcm2835_isp_fmt supported_formats[] = {
{
/* YUV formats */
@@ -33,7 +52,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 64,
.mmal_fmt = MMAL_ENCODING_I420,
.size_multiplier_x2 = 3,
- .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_YUV,
+ .colorspace_default = V4L2_COLORSPACE_JPEG,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_YVU420,
@@ -41,7 +61,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 64,
.mmal_fmt = MMAL_ENCODING_YV12,
.size_multiplier_x2 = 3,
- .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_YUV,
+ .colorspace_default = V4L2_COLORSPACE_SMPTE170M,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_NV12,
@@ -49,7 +70,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_NV12,
.size_multiplier_x2 = 3,
- .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_YUV,
+ .colorspace_default = V4L2_COLORSPACE_SMPTE170M,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_NV21,
@@ -57,7 +79,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_NV21,
.size_multiplier_x2 = 3,
- .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_YUV,
+ .colorspace_default = V4L2_COLORSPACE_SMPTE170M,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_YUYV,
@@ -65,7 +88,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 64,
.mmal_fmt = MMAL_ENCODING_YUYV,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_YUV,
+ .colorspace_default = V4L2_COLORSPACE_SMPTE170M,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_UYVY,
@@ -73,7 +97,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 64,
.mmal_fmt = MMAL_ENCODING_UYVY,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_YUV,
+ .colorspace_default = V4L2_COLORSPACE_SMPTE170M,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_YVYU,
@@ -81,7 +106,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 64,
.mmal_fmt = MMAL_ENCODING_YVYU,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_YUV,
+ .colorspace_default = V4L2_COLORSPACE_SMPTE170M,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_VYUY,
@@ -89,7 +115,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 64,
.mmal_fmt = MMAL_ENCODING_VYUY,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_YUV,
+ .colorspace_default = V4L2_COLORSPACE_SMPTE170M,
.step_size = 2,
}, {
/* RGB formats */
@@ -98,7 +125,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_RGB24,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_SRGB,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_SRGB,
+ .colorspace_default = V4L2_COLORSPACE_SRGB,
.step_size = 1,
}, {
.fourcc = V4L2_PIX_FMT_RGB565,
@@ -106,7 +134,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_RGB16,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_SRGB,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_SRGB,
+ .colorspace_default = V4L2_COLORSPACE_SRGB,
.step_size = 1,
}, {
.fourcc = V4L2_PIX_FMT_BGR24,
@@ -114,7 +143,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BGR24,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_SRGB,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_SRGB,
+ .colorspace_default = V4L2_COLORSPACE_SRGB,
.step_size = 1,
}, {
.fourcc = V4L2_PIX_FMT_XBGR32,
@@ -122,7 +152,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 64,
.mmal_fmt = MMAL_ENCODING_BGRA,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_SRGB,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_SRGB,
+ .colorspace_default = V4L2_COLORSPACE_SRGB,
.step_size = 1,
}, {
.fourcc = V4L2_PIX_FMT_RGBX32,
@@ -130,7 +161,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 64,
.mmal_fmt = MMAL_ENCODING_RGBA,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_SRGB,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_SRGB,
+ .colorspace_default = V4L2_COLORSPACE_SRGB,
.step_size = 1,
}, {
/* Bayer formats */
@@ -140,7 +172,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SRGGB8,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SBGGR8,
@@ -148,7 +181,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SBGGR8,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SGRBG8,
@@ -156,7 +190,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SGRBG8,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SGBRG8,
@@ -164,7 +199,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SGBRG8,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
/* 10 bit */
@@ -173,7 +209,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SRGGB10P,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SBGGR10P,
@@ -181,7 +218,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SBGGR10P,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SGRBG10P,
@@ -189,7 +227,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SGRBG10P,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SGBRG10P,
@@ -197,7 +236,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SGBRG10P,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
/* 12 bit */
@@ -206,7 +246,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SRGGB12P,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SBGGR12P,
@@ -214,7 +255,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SBGGR12P,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SGRBG12P,
@@ -222,7 +264,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SGRBG12P,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SGBRG12P,
@@ -230,7 +273,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SGBRG12P,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
/* 14 bit */
@@ -239,7 +283,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SRGGB14P,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SBGGR14P,
@@ -247,7 +292,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SBGGR14P,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SGRBG14P,
@@ -255,7 +301,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SGRBG14P,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SGBRG14P,
@@ -263,7 +310,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SGBRG14P,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
/* 16 bit */
@@ -272,7 +320,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SRGGB16,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SBGGR16,
@@ -280,7 +329,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SBGGR16,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SGRBG16,
@@ -288,7 +338,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SGRBG16,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SGBRG16,
@@ -296,7 +347,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SGBRG16,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
/* Bayer formats unpacked to 16bpp */
@@ -306,7 +358,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SRGGB10,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SBGGR10,
@@ -314,7 +367,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SBGGR10,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SGRBG10,
@@ -322,7 +376,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SGRBG10,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SGBRG10,
@@ -330,7 +385,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SGBRG10,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
/* 12 bit */
@@ -339,7 +395,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SRGGB12,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SBGGR12,
@@ -347,7 +404,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SBGGR12,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SGRBG12,
@@ -355,7 +413,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SGRBG12,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SGBRG12,
@@ -363,7 +422,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SGBRG12,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
/* 14 bit */
@@ -372,7 +432,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SRGGB14,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SBGGR14,
@@ -380,7 +441,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SBGGR14,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SGRBG14,
@@ -388,7 +450,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SGRBG14,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_PIX_FMT_SGBRG14,
@@ -396,7 +459,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BAYER_SGBRG14,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
/* Monochrome MIPI formats */
@@ -406,7 +470,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_GREY,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
/* 10 bit */
@@ -415,7 +480,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_Y10P,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
/* 12 bit */
@@ -424,7 +490,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_Y12P,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
/* 14 bit */
@@ -433,7 +500,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_Y14P,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
/* 16 bit */
@@ -442,7 +510,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_Y16,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
/* 10 bit as 16bpp */
@@ -451,7 +520,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_Y10,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
/* 12 bit as 16bpp */
@@ -460,7 +530,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_Y12,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
/* 14 bit as 16bpp */
@@ -469,7 +540,8 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_Y14,
.size_multiplier_x2 = 2,
- .colorspace = V4L2_COLORSPACE_RAW,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_RAW,
+ .colorspace_default = V4L2_COLORSPACE_RAW,
.step_size = 2,
}, {
.fourcc = V4L2_META_FMT_BCM2835_ISP_STATS,
diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
index f1ea56b4849e..b6ac1ee8f9f3 100644
--- a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
+++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
@@ -74,6 +74,7 @@ struct bcm2835_isp_q_data {
unsigned int width;
unsigned int height;
unsigned int sizeimage;
+ enum v4l2_colorspace colorspace;
const struct bcm2835_isp_fmt *fmt;
};
@@ -313,6 +314,43 @@ static void mmal_buffer_cb(struct vchiq_mmal_instance *instance,
complete(&dev->frame_cmplt);
}
+struct colorspace_translation {
+ enum v4l2_colorspace v4l2_value;
+ u32 mmal_value;
+};
+
+static u32 translate_color_space(enum v4l2_colorspace color_space)
+{
+ static const struct colorspace_translation translations[] = {
+ { V4L2_COLORSPACE_DEFAULT, MMAL_COLOR_SPACE_UNKNOWN },
+ { V4L2_COLORSPACE_SMPTE170M, MMAL_COLOR_SPACE_ITUR_BT601 },
+ { V4L2_COLORSPACE_SMPTE240M, MMAL_COLOR_SPACE_SMPTE240M },
+ { V4L2_COLORSPACE_REC709, MMAL_COLOR_SPACE_ITUR_BT709 },
+ /* V4L2_COLORSPACE_BT878 unavailable */
+ { V4L2_COLORSPACE_470_SYSTEM_M, MMAL_COLOR_SPACE_BT470_2_M },
+ { V4L2_COLORSPACE_470_SYSTEM_BG, MMAL_COLOR_SPACE_BT470_2_BG },
+ { V4L2_COLORSPACE_JPEG, MMAL_COLOR_SPACE_JPEG_JFIF },
+ /*
+ * We don't have an encoding for SRGB as such, but VideoCore
+ * will do the right thing if it gets "unknown".
+ */
+ { V4L2_COLORSPACE_SRGB, MMAL_COLOR_SPACE_UNKNOWN },
+ /* V4L2_COLORSPACE_OPRGB unavailable */
+ /* V4L2_COLORSPACE_BT2020 unavailable */
+ /* V4L2_COLORSPACE_RAW unavailable */
+ /* V4L2_COLORSPACE_DCI_P3 unavailable */
+ };
+
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(translations); i++) {
+ if (color_space == translations[i].v4l2_value)
+ return translations[i].mmal_value;
+ }
+
+ return MMAL_COLOR_SPACE_UNKNOWN;
+}
+
static void setup_mmal_port_format(struct bcm2835_isp_node *node,
struct vchiq_mmal_port *port)
{
@@ -326,6 +364,7 @@ static void setup_mmal_port_format(struct bcm2835_isp_node *node,
port->es.video.crop.height = q_data->height;
port->es.video.crop.x = 0;
port->es.video.crop.y = 0;
+ port->es.video.color_space = translate_color_space(q_data->colorspace);
};
static int setup_mmal_port(struct bcm2835_isp_node *node)
@@ -834,6 +873,9 @@ static int populate_qdata_fmt(struct v4l2_format *f,
/* All parameters should have been set correctly by try_fmt */
q_data->bytesperline = f->fmt.pix.bytesperline;
q_data->sizeimage = f->fmt.pix.sizeimage;
+
+ /* We must indicate which of the allowed colour spaces we have. */
+ q_data->colorspace = f->fmt.pix.colorspace;
} else {
v4l2_dbg(1, debug, &dev->v4l2_dev,
"%s: Setting meta format for fmt: %08x, size %u\n",
@@ -845,6 +887,9 @@ static int populate_qdata_fmt(struct v4l2_format *f,
q_data->height = 0;
q_data->bytesperline = 0;
q_data->sizeimage = f->fmt.meta.buffersize;
+
+ /* This won't mean anything for metadata, but may as well fill it in. */
+ q_data->colorspace = V4L2_COLORSPACE_DEFAULT;
}
v4l2_dbg(1, debug, &dev->v4l2_dev,
@@ -908,7 +953,7 @@ static int bcm2835_isp_node_g_fmt(struct file *file, void *priv,
f->fmt.pix.pixelformat = q_data->fmt->fourcc;
f->fmt.pix.bytesperline = q_data->bytesperline;
f->fmt.pix.sizeimage = q_data->sizeimage;
- f->fmt.pix.colorspace = q_data->fmt->colorspace;
+ f->fmt.pix.colorspace = q_data->colorspace;
}
return 0;
@@ -975,13 +1020,32 @@ static int bcm2835_isp_node_try_fmt(struct file *file, void *priv,
fmt = get_default_format(node);
if (!node_is_stats(node)) {
+ int is_rgb;
+
f->fmt.pix.width = max(min(f->fmt.pix.width, MAX_DIM),
MIN_DIM);
f->fmt.pix.height = max(min(f->fmt.pix.height, MAX_DIM),
MIN_DIM);
f->fmt.pix.pixelformat = fmt->fourcc;
- f->fmt.pix.colorspace = fmt->colorspace;
+
+ /*
+ * Fill in the actual colour space when the requested one was
+ * not supported. This also catches the case when the "default"
+ * colour space was requested (as that's never in the mask).
+ */
+ if (!(V4L2_COLORSPACE_MASK(f->fmt.pix.colorspace) & fmt->colorspace_mask))
+ f->fmt.pix.colorspace = fmt->colorspace_default;
+ /* In all cases, we only support the defaults for these: */
+ f->fmt.pix.ycbcr_enc = V4L2_MAP_YCBCR_ENC_DEFAULT(f->fmt.pix.colorspace);
+ f->fmt.pix.xfer_func = V4L2_MAP_XFER_FUNC_DEFAULT(f->fmt.pix.colorspace);
+ /* RAW counts as sRGB here so that we get full range. */
+ is_rgb = f->fmt.pix.colorspace == V4L2_COLORSPACE_SRGB ||
+ f->fmt.pix.colorspace == V4L2_COLORSPACE_RAW;
+ f->fmt.pix.quantization =
+ V4L2_MAP_QUANTIZATION_DEFAULT(is_rgb, f->fmt.pix.colorspace,
+ f->fmt.pix.ycbcr_enc);
+
f->fmt.pix.bytesperline = get_bytesperline(f->fmt.pix.width,
fmt);
f->fmt.pix.field = V4L2_FIELD_NONE;
@@ -1307,6 +1371,7 @@ static int register_node(struct bcm2835_isp_dev *dev,
node->q_data.width,
node->q_data.height,
node->q_data.fmt);
+ node->q_data.colorspace = node->q_data.fmt->colorspace_default;
queue->io_modes = VB2_MMAP | VB2_DMABUF;
queue->drv_priv = node;
--
2.41.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 12/15] vc04_services: bcm2835-isp: Permit all sRGB colour spaces on ISP outputs
2023-11-09 21:02 [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Umang Jain
` (10 preceding siblings ...)
2023-11-09 21:03 ` [PATCH v2 11/15] vc04_services: bcm2835-isp: Allow formats with different colour spaces Umang Jain
@ 2023-11-09 21:03 ` Umang Jain
2023-11-09 21:03 ` [PATCH v2 13/15] staging: vc04_services: bcm2835_isp: Allow multiple users Umang Jain
` (4 subsequent siblings)
16 siblings, 0 replies; 45+ messages in thread
From: Umang Jain @ 2023-11-09 21:03 UTC (permalink / raw)
To: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Stefan Wahren,
David Plowman, Umang Jain
From: David Plowman <david.plowman@raspberrypi.com>
bcm2835-isp outputs actually support all colour spaces that are
fundamentally sRGB underneath, regardless of whether an RGB or YUV output
format is actually requested.
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
---
.../bcm2835-isp/bcm2835-isp-fmts.h | 45 ++++++++++---------
1 file changed, 25 insertions(+), 20 deletions(-)
diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
index 5c874cdd2fe7..30c14a44f399 100644
--- a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
+++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
@@ -35,14 +35,19 @@ struct bcm2835_isp_fmt {
#define V4L2_COLORSPACE_MASK_RAW V4L2_COLORSPACE_MASK(V4L2_COLORSPACE_RAW)
/*
- * The colour spaces we support for YUV outputs. SRGB features here because,
- * once you assign the default transfer func and so on, it and JPEG effectively
- * mean the same.
+ * All three colour spaces JPEG, SMPTE170M and REC709 are fundamentally sRGB
+ * underneath (as near as makes no difference to us), just with different YCbCr
+ * encodings. Therefore the ISP can generate sRGB on its main output and any of
+ * the others on its low resolution output. Applications should, when using both
+ * outputs, program the colour spaces on them to be the same, matching whatever
+ * is requested for the low resolution output, even if the main output is
+ * producing an RGB format. In turn this requires us to allow all these colour
+ * spaces for every YUV/RGB output format.
*/
-#define V4L2_COLORSPACE_MASK_YUV (V4L2_COLORSPACE_MASK_JPEG | \
- V4L2_COLORSPACE_MASK_SRGB | \
- V4L2_COLORSPACE_MASK_SMPTE170M | \
- V4L2_COLORSPACE_MASK_REC709)
+#define V4L2_COLORSPACE_MASK_ALL_SRGB (V4L2_COLORSPACE_MASK_JPEG | \
+ V4L2_COLORSPACE_MASK_SRGB | \
+ V4L2_COLORSPACE_MASK_SMPTE170M | \
+ V4L2_COLORSPACE_MASK_REC709)
static const struct bcm2835_isp_fmt supported_formats[] = {
{
@@ -52,7 +57,7 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 64,
.mmal_fmt = MMAL_ENCODING_I420,
.size_multiplier_x2 = 3,
- .colorspace_mask = V4L2_COLORSPACE_MASK_YUV,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_ALL_SRGB,
.colorspace_default = V4L2_COLORSPACE_JPEG,
.step_size = 2,
}, {
@@ -61,7 +66,7 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 64,
.mmal_fmt = MMAL_ENCODING_YV12,
.size_multiplier_x2 = 3,
- .colorspace_mask = V4L2_COLORSPACE_MASK_YUV,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_ALL_SRGB,
.colorspace_default = V4L2_COLORSPACE_SMPTE170M,
.step_size = 2,
}, {
@@ -70,7 +75,7 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_NV12,
.size_multiplier_x2 = 3,
- .colorspace_mask = V4L2_COLORSPACE_MASK_YUV,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_ALL_SRGB,
.colorspace_default = V4L2_COLORSPACE_SMPTE170M,
.step_size = 2,
}, {
@@ -79,7 +84,7 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_NV21,
.size_multiplier_x2 = 3,
- .colorspace_mask = V4L2_COLORSPACE_MASK_YUV,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_ALL_SRGB,
.colorspace_default = V4L2_COLORSPACE_SMPTE170M,
.step_size = 2,
}, {
@@ -88,7 +93,7 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 64,
.mmal_fmt = MMAL_ENCODING_YUYV,
.size_multiplier_x2 = 2,
- .colorspace_mask = V4L2_COLORSPACE_MASK_YUV,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_ALL_SRGB,
.colorspace_default = V4L2_COLORSPACE_SMPTE170M,
.step_size = 2,
}, {
@@ -97,7 +102,7 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 64,
.mmal_fmt = MMAL_ENCODING_UYVY,
.size_multiplier_x2 = 2,
- .colorspace_mask = V4L2_COLORSPACE_MASK_YUV,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_ALL_SRGB,
.colorspace_default = V4L2_COLORSPACE_SMPTE170M,
.step_size = 2,
}, {
@@ -106,7 +111,7 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 64,
.mmal_fmt = MMAL_ENCODING_YVYU,
.size_multiplier_x2 = 2,
- .colorspace_mask = V4L2_COLORSPACE_MASK_YUV,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_ALL_SRGB,
.colorspace_default = V4L2_COLORSPACE_SMPTE170M,
.step_size = 2,
}, {
@@ -115,7 +120,7 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 64,
.mmal_fmt = MMAL_ENCODING_VYUY,
.size_multiplier_x2 = 2,
- .colorspace_mask = V4L2_COLORSPACE_MASK_YUV,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_ALL_SRGB,
.colorspace_default = V4L2_COLORSPACE_SMPTE170M,
.step_size = 2,
}, {
@@ -125,7 +130,7 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_RGB24,
.size_multiplier_x2 = 2,
- .colorspace_mask = V4L2_COLORSPACE_MASK_SRGB,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_ALL_SRGB,
.colorspace_default = V4L2_COLORSPACE_SRGB,
.step_size = 1,
}, {
@@ -134,7 +139,7 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_RGB16,
.size_multiplier_x2 = 2,
- .colorspace_mask = V4L2_COLORSPACE_MASK_SRGB,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_ALL_SRGB,
.colorspace_default = V4L2_COLORSPACE_SRGB,
.step_size = 1,
}, {
@@ -143,7 +148,7 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 32,
.mmal_fmt = MMAL_ENCODING_BGR24,
.size_multiplier_x2 = 2,
- .colorspace_mask = V4L2_COLORSPACE_MASK_SRGB,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_ALL_SRGB,
.colorspace_default = V4L2_COLORSPACE_SRGB,
.step_size = 1,
}, {
@@ -152,7 +157,7 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 64,
.mmal_fmt = MMAL_ENCODING_BGRA,
.size_multiplier_x2 = 2,
- .colorspace_mask = V4L2_COLORSPACE_MASK_SRGB,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_ALL_SRGB,
.colorspace_default = V4L2_COLORSPACE_SRGB,
.step_size = 1,
}, {
@@ -161,7 +166,7 @@ static const struct bcm2835_isp_fmt supported_formats[] = {
.bytesperline_align = 64,
.mmal_fmt = MMAL_ENCODING_RGBA,
.size_multiplier_x2 = 2,
- .colorspace_mask = V4L2_COLORSPACE_MASK_SRGB,
+ .colorspace_mask = V4L2_COLORSPACE_MASK_ALL_SRGB,
.colorspace_default = V4L2_COLORSPACE_SRGB,
.step_size = 1,
}, {
--
2.41.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 13/15] staging: vc04_services: bcm2835_isp: Allow multiple users
2023-11-09 21:02 [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Umang Jain
` (11 preceding siblings ...)
2023-11-09 21:03 ` [PATCH v2 12/15] vc04_services: bcm2835-isp: Permit all sRGB colour spaces on ISP outputs Umang Jain
@ 2023-11-09 21:03 ` Umang Jain
2023-11-09 21:03 ` [PATCH v2 14/15] docs: admin-guide: media: bcm2835-isp: Add documentation for bcm2835-isp Umang Jain
` (3 subsequent siblings)
16 siblings, 0 replies; 45+ messages in thread
From: Umang Jain @ 2023-11-09 21:03 UTC (permalink / raw)
To: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Stefan Wahren,
Naushir Patuck, Umang Jain
From: Naushir Patuck <naush@raspberrypi.com>
Add a second (identical) set of device nodes to allow concurrent use of
the bcm2835-isp hardware by another user. This change effectively
creates a second state structure (struct bcm2835_isp_dev) to maintain
independent state for the second user. Node and media entity names are
appened with the instance index appropriately.
Further users can be added by changing the BCM2835_ISP_NUM_INSTANCES
define.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
---
.../bcm2835-isp/bcm2835-v4l2-isp.c | 73 +++++++++++++++----
1 file changed, 59 insertions(+), 14 deletions(-)
diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
index b6ac1ee8f9f3..316d35d5f19d 100644
--- a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
+++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
@@ -28,13 +28,19 @@
MODULE_IMPORT_NS(DMA_BUF);
+/*
+ * We want to instantiate 2 independent instances allowing 2 simultaneous users
+ * of the ISP hardware.
+ */
+#define BCM2835_ISP_NUM_INSTANCES 2
+
static unsigned int debug;
module_param(debug, uint, 0644);
MODULE_PARM_DESC(debug, "activates debug info");
-static unsigned int video_nr = 13;
-module_param(video_nr, uint, 0644);
-MODULE_PARM_DESC(video_nr, "base video device number");
+static unsigned int video_nr[BCM2835_ISP_NUM_INSTANCES] = { 13, 20 };
+module_param_array(video_nr, uint, NULL, 0644);
+MODULE_PARM_DESC(video_nr, "base video device numbers");
#define BCM2835_ISP_NAME "bcm2835-isp"
#define BCM2835_ISP_ENTITY_NAME_LEN 32
@@ -1287,6 +1293,7 @@ static int bcm2835_isp_get_supported_fmts(struct bcm2835_isp_node *node)
* or output nodes.
*/
static int register_node(struct bcm2835_isp_dev *dev,
+ unsigned int instance,
struct bcm2835_isp_node *node,
int index)
{
@@ -1448,7 +1455,7 @@ static int register_node(struct bcm2835_isp_dev *dev,
snprintf(vfd->name, sizeof(node->vfd.name), "%s-%s%d", BCM2835_ISP_NAME,
node->name, node->id);
- ret = video_register_device(vfd, VFL_TYPE_VIDEO, video_nr + index);
+ ret = video_register_device(vfd, VFL_TYPE_VIDEO, video_nr[instance]);
if (ret) {
v4l2_err(&dev->v4l2_dev,
"Failed to register video %s[%d] device node\n",
@@ -1669,9 +1676,8 @@ static int media_controller_register(struct bcm2835_isp_dev *dev)
return ret;
}
-static void bcm2835_isp_remove(struct vchiq_device *device)
+static void bcm2835_isp_remove_instance(struct bcm2835_isp_dev *dev)
{
- struct bcm2835_isp_dev *dev = vchiq_get_drvdata(device);
unsigned int i;
media_controller_unregister(dev);
@@ -1688,7 +1694,9 @@ static void bcm2835_isp_remove(struct vchiq_device *device)
vchiq_mmal_finalise(dev->mmal_instance);
}
-static int bcm2835_isp_probe(struct vchiq_device *device)
+static int bcm2835_isp_probe_instance(struct vchiq_device *device,
+ struct bcm2835_isp_dev **dev_int,
+ unsigned int instance)
{
struct bcm2835_isp_dev *dev;
unsigned int i;
@@ -1698,6 +1706,7 @@ static int bcm2835_isp_probe(struct vchiq_device *device)
if (!dev)
return -ENOMEM;
+ *dev_int = dev;
dev->dev = &device->dev;
ret = v4l2_device_register(&device->dev, &dev->v4l2_dev);
@@ -1715,7 +1724,7 @@ static int bcm2835_isp_probe(struct vchiq_device *device)
if (ret) {
v4l2_err(&dev->v4l2_dev,
"%s: failed to create ril.isp component\n", __func__);
- goto error;
+ return ret;
}
if (dev->component->inputs < BCM2835_ISP_NUM_OUTPUTS ||
@@ -1727,7 +1736,7 @@ static int bcm2835_isp_probe(struct vchiq_device *device)
BCM2835_ISP_NUM_OUTPUTS,
dev->component->outputs,
BCM2835_ISP_NUM_CAPTURES + BCM2835_ISP_NUM_METADATA);
- goto error;
+ return -EINVAL;
}
atomic_set(&dev->num_streaming, 0);
@@ -1735,17 +1744,53 @@ static int bcm2835_isp_probe(struct vchiq_device *device)
for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
struct bcm2835_isp_node *node = &dev->node[i];
- ret = register_node(dev, node, i);
+ ret = register_node(dev, instance, node, i);
if (ret)
- goto error;
+ return ret;
}
ret = media_controller_register(dev);
if (ret)
- goto error;
+ return ret;
+
+ return 0;
+}
+
+static void bcm2835_isp_remove(struct vchiq_device *device)
+{
+ struct bcm2835_isp_dev **bcm2835_isp_instances;
+ unsigned int i;
+
+ bcm2835_isp_instances = vchiq_get_drvdata(device);
+ for (i = 0; i < BCM2835_ISP_NUM_INSTANCES; i++) {
+ if (bcm2835_isp_instances[i])
+ bcm2835_isp_remove_instance(bcm2835_isp_instances[i]);
+ }
+}
+
+static int bcm2835_isp_probe(struct vchiq_device *device)
+{
+ struct bcm2835_isp_dev **bcm2835_isp_instances;
+ unsigned int i;
+ int ret;
+
+ bcm2835_isp_instances = devm_kzalloc(&device->dev,
+ sizeof(bcm2835_isp_instances) *
+ BCM2835_ISP_NUM_INSTANCES,
+ GFP_KERNEL);
+ if (!bcm2835_isp_instances)
+ return -ENOMEM;
+
+ vchiq_set_drvdata(device, bcm2835_isp_instances);
+
+ for (i = 0; i < BCM2835_ISP_NUM_INSTANCES; i++) {
+ ret = bcm2835_isp_probe_instance(device,
+ &bcm2835_isp_instances[i], i);
+ if (ret)
+ goto error;
+ }
- vchiq_set_drvdata(device, dev);
- v4l2_info(&dev->v4l2_dev, "Loaded V4L2 %s\n", BCM2835_ISP_NAME);
+ dev_info(&device->dev, "Loaded V4L2 %s\n", BCM2835_ISP_NAME);
return 0;
error:
--
2.41.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 14/15] docs: admin-guide: media: bcm2835-isp: Add documentation for bcm2835-isp
2023-11-09 21:02 [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Umang Jain
` (12 preceding siblings ...)
2023-11-09 21:03 ` [PATCH v2 13/15] staging: vc04_services: bcm2835_isp: Allow multiple users Umang Jain
@ 2023-11-09 21:03 ` Umang Jain
2023-11-09 21:03 ` [PATCH v2 15/15] staging: vc04_services: vchiq: Register bcm2835-isp Umang Jain
` (2 subsequent siblings)
16 siblings, 0 replies; 45+ messages in thread
From: Umang Jain @ 2023-11-09 21:03 UTC (permalink / raw)
To: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Stefan Wahren,
Naushir Patuck, Umang Jain
From: Naushir Patuck <naush@raspberrypi.com>
Document device nodes, frame buffer formats and initial set of
configuraiton that can be set from userspace to configure the pipeline.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
---
.../admin-guide/media/bcm2835-isp.rst | 127 ++++++++++++++++++
.../userspace-api/media/drivers/index.rst | 1 +
2 files changed, 128 insertions(+)
create mode 100644 Documentation/admin-guide/media/bcm2835-isp.rst
diff --git a/Documentation/admin-guide/media/bcm2835-isp.rst b/Documentation/admin-guide/media/bcm2835-isp.rst
new file mode 100644
index 000000000000..e1c19f78435e
--- /dev/null
+++ b/Documentation/admin-guide/media/bcm2835-isp.rst
@@ -0,0 +1,127 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+BCM2835 ISP Driver
+==================
+
+Introduction
+------------
+
+The BCM2835 Image Sensor Pipeline (ISP) is a fixed function hardware pipeline
+for performing image processing operations. Images are fed to the input
+of the ISP through memory frame buffers. These images may be in various YUV,
+RGB, or Bayer formats. A typical use case would have Bayer images obtained from
+an image sensor by the BCM2835 Unicam peripheral, written to a memory
+frame buffer, and finally fed into the input of the ISP. Two concurrent output
+images may be generated in YUV or RGB format at different resolutions.
+Statistics output is also generated for Bayer input images.
+
+The bcm2835-isp driver exposes the following media pads as V4L2 device nodes:
+
+.. tabularcolumns:: |l|l|l|l|
+
+.. cssclass: longtable
+
+.. flat-table::
+
+ * - *Pad*
+ - *Direction*
+ - *Purpose*
+ - *Formats*
+
+ * - "bcm2835-isp0-output0"
+ - sink
+ - Accepts Bayer, RGB or YUV format frame buffers as input to the ISP HW
+ pipeline.
+ - :ref:`RAW8 <V4L2-PIX-FMT-SRGGB8>`,
+ :ref:`RAW10P <V4L2-PIX-FMT-SRGGB10P>`,
+ :ref:`RAW12P <V4L2-PIX-FMT-SRGGB12P>`,
+ :ref:`RAW14P <V4L2-PIX-FMT-SRGGB14P>`,
+ :ref:`RAW16 <V4L2-PIX-FMT-SRGGB16>`,
+ :ref:`RGB24/BGR24 <V4L2-PIX-FMT-RGB24>`,
+ :ref:`YUYV <V4L2-PIX-FMT-YUYV>`,
+ :ref:`YVYU <V4L2-PIX-FMT-YVYU>`,
+ :ref:`UYVY <V4L2-PIX-FMT-UYVY>`,
+ :ref:`VYUY <V4L2-PIX-FMT-VYUY>`,
+ :ref:`YUV420/YVU420 <V4L2-PIX-FMT-YUV420>`
+
+ * - "bcm2835-isp0-capture1"
+ - source
+ - High resolution YUV or RGB processed output from the ISP.
+ - :ref:`RGB565 <V4L2-PIX-FMT-RGB565>`,
+ :ref:`RGB24/BGR24 <V4L2-PIX-FMT-RGB24>`,
+ :ref:`ABGR32 <V4L2-PIX-FMT-ABGR32>`,
+ :ref:`YUYV <V4L2-PIX-FMT-YUYV>`,
+ :ref:`YVYU <V4L2-PIX-FMT-YVYU>`,
+ :ref:`UYVY <V4L2-PIX-FMT-UYVY>`,
+ :ref:`VYUY <V4L2-PIX-FMT-VYUY>`.
+ :ref:`YUV420/YVU420 <V4L2-PIX-FMT-YUV420>`,
+ :ref:`NV12/NV21 <V4L2-PIX-FMT-NV12>`,
+
+ * - "bcm2835-isp0-capture2"
+ - source
+ - Low resolution YUV processed output from the ISP. The output of
+ this pad cannot have a resolution larger than the "bcm2835-isp0-capture1" pad in any dimension.
+ - :ref:`YUYV <V4L2-PIX-FMT-YUYV>`,
+ :ref:`YVYU <V4L2-PIX-FMT-YVYU>`,
+ :ref:`UYVY <V4L2-PIX-FMT-UYVY>`,
+ :ref:`VYUY <V4L2-PIX-FMT-VYUY>`.
+ :ref:`YUV420/YVU420 <V4L2-PIX-FMT-YUV420>`,
+ :ref:`NV12/NV21 <V4L2-PIX-FMT-NV12>`,
+
+ * - "bcm2835-isp0-capture1"
+ - source
+ - Image statistics calculated from the input image provided on the
+ "bcm2835-isp0-output0" pad. Statistics are only available for Bayer
+ format input images.
+ - :ref:`v4l2-meta-fmt-bcm2835-isp-stats`.
+
+Pipeline Configuration
+----------------------
+
+The ISP pipeline can be configure through user-space by calling
+:ref:`VIDIOC_S_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` on the “bcm2835-isp0-output0”
+node with the appropriate parameters as shown in the table below.
+
+.. tabularcolumns:: |p{2cm}|p{5.0cm}|
+
+.. cssclass: longtable
+
+.. flat-table::
+
+ * - *id*
+ - *Parameter*
+
+ * - ``V4L2_CID_USER_BCM2835_ISP_CC_MATRIX``
+ - struct :c:type:`bcm2835_isp_custom_ccm`
+
+ * - ``V4L2_CID_USER_BCM2835_ISP_LENS_SHADING``
+ - struct :c:type:`bcm2835_isp_lens_shading`
+
+ * - ``V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL``
+ - struct :c:type:`bcm2835_isp_black_level`
+
+ * - ``V4L2_CID_USER_BCM2835_ISP_GEQ``
+ - struct :c:type:`bcm2835_isp_geq`
+
+ * - ``V4L2_CID_USER_BCM2835_ISP_GAMMA``
+ - struct :c:type:`bcm2835_isp_gamma`
+
+ * - ``V4L2_CID_USER_BCM2835_ISP_DENOISE``
+ - struct :c:type:`bcm2835_isp_denoise`
+
+ * - ``V4L2_CID_USER_BCM2835_ISP_SHARPEN``
+ - struct :c:type:`bcm2835_isp_sharpen`
+
+ * - ``V4L2_CID_USER_BCM2835_ISP_DPC``
+ - struct :c:type:`bcm2835_isp_dpc`
+
+++++++++++++++++++++++++
+Configuration Parameters
+++++++++++++++++++++++++
+
+.. kernel-doc:: include/uapi/linux/bcm2835-isp.h
+ :functions: bcm2835_isp_rational bcm2835_isp_ccm bcm2835_isp_custom_ccm
+ bcm2835_isp_gain_format bcm2835_isp_lens_shading
+ bcm2835_isp_black_level bcm2835_isp_geq bcm2835_isp_gamma
+ bcm2835_isp_denoise bcm2835_isp_sharpen
+ bcm2835_isp_dpc_mode bcm2835_isp_dpc
diff --git a/Documentation/userspace-api/media/drivers/index.rst b/Documentation/userspace-api/media/drivers/index.rst
index 1726f8ec86fa..05e20a4e1d53 100644
--- a/Documentation/userspace-api/media/drivers/index.rst
+++ b/Documentation/userspace-api/media/drivers/index.rst
@@ -33,6 +33,7 @@ For more details see the file COPYING in the source distribution of Linux.
aspeed-video
camera-sensor
+ bcm2835-isp
ccs
cx2341x-uapi
dw100
--
2.41.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* [PATCH v2 15/15] staging: vc04_services: vchiq: Register bcm2835-isp
2023-11-09 21:02 [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Umang Jain
` (13 preceding siblings ...)
2023-11-09 21:03 ` [PATCH v2 14/15] docs: admin-guide: media: bcm2835-isp: Add documentation for bcm2835-isp Umang Jain
@ 2023-11-09 21:03 ` Umang Jain
2023-11-10 10:32 ` [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Stefan Wahren
2023-11-15 19:59 ` Laurent Pinchart
16 siblings, 0 replies; 45+ messages in thread
From: Umang Jain @ 2023-11-09 21:03 UTC (permalink / raw)
To: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Stefan Wahren, Umang Jain
Add vchiq device registeration and unregisteration for
the bcm2835-isp devices.
Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
---
drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
index 5a9073480dd3..85dd259e6efc 100644
--- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
+++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
@@ -70,6 +70,7 @@ struct vchiq_state g_state;
*/
static struct vchiq_device *bcm2835_audio;
static struct vchiq_device *bcm2835_camera;
+static struct vchiq_device *bcm2835_isp;
static struct vchiq_device *vcsm_cma;
struct vchiq_drvdata {
@@ -1844,6 +1845,7 @@ static int vchiq_probe(struct platform_device *pdev)
vcsm_cma = vchiq_device_register(&pdev->dev, "vcsm-cma");
bcm2835_audio = vchiq_device_register(&pdev->dev, "bcm2835-audio");
bcm2835_camera = vchiq_device_register(&pdev->dev, "bcm2835-camera");
+ bcm2835_isp = vchiq_device_register(&pdev->dev, "bcm2835-isp");
return 0;
@@ -1857,6 +1859,7 @@ static void vchiq_remove(struct platform_device *pdev)
{
vchiq_device_unregister(bcm2835_audio);
vchiq_device_unregister(bcm2835_camera);
+ vchiq_device_unregister(bcm2835_isp);
vchiq_device_unregister(vcsm_cma);
vchiq_debugfs_deinit();
vchiq_deregister_chrdev();
--
2.41.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* Re: [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support
2023-11-09 21:02 [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Umang Jain
` (14 preceding siblings ...)
2023-11-09 21:03 ` [PATCH v2 15/15] staging: vc04_services: vchiq: Register bcm2835-isp Umang Jain
@ 2023-11-10 10:32 ` Stefan Wahren
2023-11-13 13:38 ` Umang Jain
2023-11-15 19:59 ` Laurent Pinchart
16 siblings, 1 reply; 45+ messages in thread
From: Stefan Wahren @ 2023-11-10 10:32 UTC (permalink / raw)
To: Umang Jain, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter
Hi Umang,
Am 09.11.23 um 22:02 schrieb Umang Jain:
> This series aims to upport bcm2835-isp from the RPi kernel.
> It is developed on top of staging-next which comprises many
> VC04 changes for it's de-staging. Hence, the merge of this
> driver is targeted when VC04 is de-staged completely (which I
> have been pushing), but it can be helped getting reviewed meanwhile.
> Hence, the reason for posting the series.
>
> Patch (01-02)/15 adds a new driver named vc-sm-cma to handle memory sharing
> with the VC4 VPU.
>
> Patch 03/15 adds a small extension to videobuf2 to allow exporting as a
> dma_buf instead of a file-descriptor.
>
> Patch (04-05)/15 adds a couple of improvements/support for
> bcm2835-isp(event callback and zero-copy) to vchiq-mmal.
>
> Patch (06-10)/15 adds the core bcm2835-isp driver along with headers
> and format defintions. 09/15 is a standalone patch which can be merged
> independently I believe.
>
> Patch (11-12)/15 deals with the colorspace support.
>
> Patch 13/15 allows multiple instances of the ISP.
>
> Patch 14/15 adds a admin-guide document on bcm2835-isp.
>
> Patch 15/15 deals with driver registeration.
since this is V2, i couldn't find a changelog? The first one was one
year ago, so it's very hard to remember. In such a case a link to the
last version is also helpful.
Recently i saw you have a Github account. For such a complex series it's
also very helpful to provide a link to a working test branch with this
series applied. It's not a must, but appreciated.
Best regards
>
> Dave Stevenson (6):
> staging: vc04_services: Add new vc-sm-cma driver
> media: videobuf2: Allow exporting of a struct dmabuf
> staging: mmal-vchiq: Add support for event callbacks
> staging: mmal-vchiq: Use vc-sm-cma to support zero copy
> staging: mmal_vchiq: Add image formats to be used by bcm2835-isp
> uapi: bcm2835-isp: Add bcm2835-isp uapi header file
>
> David Plowman (2):
> vc04_services: bcm2835-isp: Allow formats with different colour spaces
> vc04_services: bcm2835-isp: Permit all sRGB colour spaces on ISP
> outputs
>
> Naushir Patuck (4):
> media: uapi: v4l2-core: Add ISP statistics output V4L2 fourcc type
> staging: vc04_services: bcm2835-isp: Add a more complex ISP processing
> component
> staging: vc04_services: bcm2835_isp: Allow multiple users
> docs: admin-guide: media: bcm2835-isp: Add documentation for
> bcm2835-isp
>
> Umang Jain (3):
> staging: vc04_services: vchiq_arm: Register vcsm-cma driver
> staging: vc04_services: Add helpers for vchiq driver data
> staging: vc04_services: vchiq: Register bcm2835-isp
>
> .../admin-guide/media/bcm2835-isp.rst | 127 ++
> .../userspace-api/media/drivers/index.rst | 1 +
> .../userspace-api/media/v4l/meta-formats.rst | 1 +
> .../v4l/pixfmt-meta-bcm2835-isp-stats.rst | 32 +
> MAINTAINERS | 9 +
> .../media/common/videobuf2/videobuf2-core.c | 36 +-
> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> drivers/staging/vc04_services/Kconfig | 4 +
> drivers/staging/vc04_services/Makefile | 3 +-
> .../staging/vc04_services/bcm2835-isp/Kconfig | 14 +
> .../vc04_services/bcm2835-isp/Makefile | 4 +
> .../bcm2835-isp/bcm2835-isp-ctrls.h | 72 +
> .../bcm2835-isp/bcm2835-isp-fmts.h | 559 +++++
> .../bcm2835-isp/bcm2835-v4l2-isp.c | 1822 +++++++++++++++++
> .../interface/vchiq_arm/vchiq_arm.c | 6 +
> .../interface/vchiq_arm/vchiq_bus.h | 10 +
> .../staging/vc04_services/vc-sm-cma/Kconfig | 10 +
> .../staging/vc04_services/vc-sm-cma/Makefile | 4 +
> .../staging/vc04_services/vc-sm-cma/vc_sm.c | 817 ++++++++
> .../staging/vc04_services/vc-sm-cma/vc_sm.h | 54 +
> .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c | 507 +++++
> .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h | 63 +
> .../vc04_services/vc-sm-cma/vc_sm_defs.h | 187 ++
> .../vc04_services/vc-sm-cma/vc_sm_knl.h | 28 +
> .../staging/vc04_services/vchiq-mmal/Kconfig | 1 +
> .../vc04_services/vchiq-mmal/mmal-common.h | 5 +
> .../vc04_services/vchiq-mmal/mmal-encodings.h | 66 +
> .../vc04_services/vchiq-mmal/mmal-msg.h | 35 +
> .../vchiq-mmal/mmal-parameters.h | 165 +-
> .../vc04_services/vchiq-mmal/mmal-vchiq.c | 253 ++-
> .../vc04_services/vchiq-mmal/mmal-vchiq.h | 5 +
> include/media/videobuf2-core.h | 15 +
> include/uapi/linux/bcm2835-isp.h | 347 ++++
> include/uapi/linux/v4l2-controls.h | 5 +
> include/uapi/linux/videodev2.h | 1 +
> 35 files changed, 5235 insertions(+), 34 deletions(-)
> create mode 100644 Documentation/admin-guide/media/bcm2835-isp.rst
> create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-ctrls.h
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
> create mode 100644 include/uapi/linux/bcm2835-isp.h
>
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support
2023-11-10 10:32 ` [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Stefan Wahren
@ 2023-11-13 13:38 ` Umang Jain
2023-11-13 15:09 ` Stefan Wahren
0 siblings, 1 reply; 45+ messages in thread
From: Umang Jain @ 2023-11-13 13:38 UTC (permalink / raw)
To: Stefan Wahren, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter
Hi Stefan,
On 11/10/23 4:02 PM, Stefan Wahren wrote:
> Hi Umang,
>
> Am 09.11.23 um 22:02 schrieb Umang Jain:
>> This series aims to upport bcm2835-isp from the RPi kernel.
>> It is developed on top of staging-next which comprises many
>> VC04 changes for it's de-staging. Hence, the merge of this
>> driver is targeted when VC04 is de-staged completely (which I
>> have been pushing), but it can be helped getting reviewed meanwhile.
>> Hence, the reason for posting the series.
>>
>> Patch (01-02)/15 adds a new driver named vc-sm-cma to handle memory
>> sharing
>> with the VC4 VPU.
>>
>> Patch 03/15 adds a small extension to videobuf2 to allow exporting as a
>> dma_buf instead of a file-descriptor.
>>
>> Patch (04-05)/15 adds a couple of improvements/support for
>> bcm2835-isp(event callback and zero-copy) to vchiq-mmal.
>>
>> Patch (06-10)/15 adds the core bcm2835-isp driver along with headers
>> and format defintions. 09/15 is a standalone patch which can be merged
>> independently I believe.
>>
>> Patch (11-12)/15 deals with the colorspace support.
>>
>> Patch 13/15 allows multiple instances of the ISP.
>>
>> Patch 14/15 adds a admin-guide document on bcm2835-isp.
>>
>> Patch 15/15 deals with driver registeration.
>
> since this is V2, i couldn't find a changelog? The first one was one
> year ago, so it's very hard to remember. In such a case a link to the
> last version is also helpful.
I am in your same situation. The changelog won't help here since the I
have too have lost count of the iterations.
In brief, I would say, the patches have further cleaned up and aligned,
rebased over the latest staging-next branch.
That majorly includes work I did for VC04 in general for de-staging.
>
> Recently i saw you have a Github account. For such a complex series it's
> also very helpful to provide a link to a working test branch with this
> series applied. It's not a must, but appreciated.
I actually was going to send test branch but at the last minute I
screwed up my .git repo because of a failed branch checkout.
I had to recreate the branches over the weekend and here they are:
I have tested these branches:
On RPi-3-b 32-bit:
Git branch:
https://git.uk.ideasonboard.com/uajain/linux/commits/branch/uajain/rpi3/staging-next/isp
On RPi-4-b 64-bit:
Git branch:
https://git.uk.ideasonboard.com/uajain/linux/commits/branch/uajain/rpi4/staging-next/isp
Both these branches have been tested with IMX219 sensor (there is a DT
overlay included) and libcamera. These branches have bcm2835-unicam with
streams support (which is handled by Laurent Pinchart as I believe) and
on top sits bcm2835-isp (this series).
You'll find DT and kernel configs I have tested with, in the respective
branches.
As far as libcamera branch goes (it requires a special branch due to
streams support):
https://git.uk.ideasonboard.com/uajain/libcamera/commits/branch/pinchartl/rpi/streams
Let me know how you get on if you are testing. I am happy to answer/help
if you decide to test this.
>
> Best regards
>
>>
>> Dave Stevenson (6):
>> staging: vc04_services: Add new vc-sm-cma driver
>> media: videobuf2: Allow exporting of a struct dmabuf
>> staging: mmal-vchiq: Add support for event callbacks
>> staging: mmal-vchiq: Use vc-sm-cma to support zero copy
>> staging: mmal_vchiq: Add image formats to be used by bcm2835-isp
>> uapi: bcm2835-isp: Add bcm2835-isp uapi header file
>>
>> David Plowman (2):
>> vc04_services: bcm2835-isp: Allow formats with different colour
>> spaces
>> vc04_services: bcm2835-isp: Permit all sRGB colour spaces on ISP
>> outputs
>>
>> Naushir Patuck (4):
>> media: uapi: v4l2-core: Add ISP statistics output V4L2 fourcc type
>> staging: vc04_services: bcm2835-isp: Add a more complex ISP
>> processing
>> component
>> staging: vc04_services: bcm2835_isp: Allow multiple users
>> docs: admin-guide: media: bcm2835-isp: Add documentation for
>> bcm2835-isp
>>
>> Umang Jain (3):
>> staging: vc04_services: vchiq_arm: Register vcsm-cma driver
>> staging: vc04_services: Add helpers for vchiq driver data
>> staging: vc04_services: vchiq: Register bcm2835-isp
>>
>> .../admin-guide/media/bcm2835-isp.rst | 127 ++
>> .../userspace-api/media/drivers/index.rst | 1 +
>> .../userspace-api/media/v4l/meta-formats.rst | 1 +
>> .../v4l/pixfmt-meta-bcm2835-isp-stats.rst | 32 +
>> MAINTAINERS | 9 +
>> .../media/common/videobuf2/videobuf2-core.c | 36 +-
>> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
>> drivers/staging/vc04_services/Kconfig | 4 +
>> drivers/staging/vc04_services/Makefile | 3 +-
>> .../staging/vc04_services/bcm2835-isp/Kconfig | 14 +
>> .../vc04_services/bcm2835-isp/Makefile | 4 +
>> .../bcm2835-isp/bcm2835-isp-ctrls.h | 72 +
>> .../bcm2835-isp/bcm2835-isp-fmts.h | 559 +++++
>> .../bcm2835-isp/bcm2835-v4l2-isp.c | 1822 +++++++++++++++++
>> .../interface/vchiq_arm/vchiq_arm.c | 6 +
>> .../interface/vchiq_arm/vchiq_bus.h | 10 +
>> .../staging/vc04_services/vc-sm-cma/Kconfig | 10 +
>> .../staging/vc04_services/vc-sm-cma/Makefile | 4 +
>> .../staging/vc04_services/vc-sm-cma/vc_sm.c | 817 ++++++++
>> .../staging/vc04_services/vc-sm-cma/vc_sm.h | 54 +
>> .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c | 507 +++++
>> .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h | 63 +
>> .../vc04_services/vc-sm-cma/vc_sm_defs.h | 187 ++
>> .../vc04_services/vc-sm-cma/vc_sm_knl.h | 28 +
>> .../staging/vc04_services/vchiq-mmal/Kconfig | 1 +
>> .../vc04_services/vchiq-mmal/mmal-common.h | 5 +
>> .../vc04_services/vchiq-mmal/mmal-encodings.h | 66 +
>> .../vc04_services/vchiq-mmal/mmal-msg.h | 35 +
>> .../vchiq-mmal/mmal-parameters.h | 165 +-
>> .../vc04_services/vchiq-mmal/mmal-vchiq.c | 253 ++-
>> .../vc04_services/vchiq-mmal/mmal-vchiq.h | 5 +
>> include/media/videobuf2-core.h | 15 +
>> include/uapi/linux/bcm2835-isp.h | 347 ++++
>> include/uapi/linux/v4l2-controls.h | 5 +
>> include/uapi/linux/videodev2.h | 1 +
>> 35 files changed, 5235 insertions(+), 34 deletions(-)
>> create mode 100644 Documentation/admin-guide/media/bcm2835-isp.rst
>> create mode 100644
>> Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
>> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
>> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
>> create mode 100644
>> drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-ctrls.h
>> create mode 100644
>> drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
>> create mode 100644
>> drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
>> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
>> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
>> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
>> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
>> create mode 100644
>> drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
>> create mode 100644
>> drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
>> create mode 100644
>> drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
>> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
>> create mode 100644 include/uapi/linux/bcm2835-isp.h
>>
^ permalink raw reply [flat|nested] 45+ messages in thread
* [PATCH] fixup! staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component
2023-11-09 21:03 ` [PATCH v2 10/15] staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component Umang Jain
@ 2023-11-13 14:07 ` Umang Jain
2023-11-13 15:25 ` Stefan Wahren
2023-11-23 12:56 ` Greg KH
2023-11-18 14:53 ` [PATCH v2 10/15] " Stefan Wahren
2023-11-20 10:33 ` Dan Carpenter
2 siblings, 2 replies; 45+ messages in thread
From: Umang Jain @ 2023-11-13 14:07 UTC (permalink / raw)
To: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Laurent Pinchart, Ricardo B . Marliere,
Dan Carpenter, Stefan Wahren, Umang Jain
Setup DMA Mask
---
I realised while (re)creating the test branches (sent in reply to the cover
letter), I missed to squash this hunk in
"staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component"
before sending out the series.
Sending it as a fixup! for now.
---
.../staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
index 316d35d5f19d..4a23f9feb6e4 100644
--- a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
+++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
@@ -1774,6 +1774,12 @@ static int bcm2835_isp_probe(struct vchiq_device *device)
unsigned int i;
int ret;
+ ret = dma_set_mask_and_coherent(&device->dev, DMA_BIT_MASK(32));
+ if (ret) {
+ dev_err(&device->dev, "dma_set_mask_and_coherent failed: %d\n", ret);
+ return ret;
+ }
+
bcm2835_isp_instances = devm_kzalloc(&device->dev,
sizeof(bcm2835_isp_instances) *
BCM2835_ISP_NUM_INSTANCES,
--
2.41.0
^ permalink raw reply related [flat|nested] 45+ messages in thread
* Re: [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support
2023-11-13 13:38 ` Umang Jain
@ 2023-11-13 15:09 ` Stefan Wahren
0 siblings, 0 replies; 45+ messages in thread
From: Stefan Wahren @ 2023-11-13 15:09 UTC (permalink / raw)
To: Umang Jain, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter
Hi Umang,
Am 13.11.23 um 14:38 schrieb Umang Jain:
> Hi Stefan,
>
> On 11/10/23 4:02 PM, Stefan Wahren wrote:
>> Hi Umang,
>>
>> Am 09.11.23 um 22:02 schrieb Umang Jain:
>>> This series aims to upport bcm2835-isp from the RPi kernel.
>>> It is developed on top of staging-next which comprises many
>>> VC04 changes for it's de-staging. Hence, the merge of this
>>> driver is targeted when VC04 is de-staged completely (which I
>>> have been pushing), but it can be helped getting reviewed meanwhile.
>>> Hence, the reason for posting the series.
>>>
>>> Patch (01-02)/15 adds a new driver named vc-sm-cma to handle memory
>>> sharing
>>> with the VC4 VPU.
>>>
>>> Patch 03/15 adds a small extension to videobuf2 to allow exporting as a
>>> dma_buf instead of a file-descriptor.
>>>
>>> Patch (04-05)/15 adds a couple of improvements/support for
>>> bcm2835-isp(event callback and zero-copy) to vchiq-mmal.
>>>
>>> Patch (06-10)/15 adds the core bcm2835-isp driver along with headers
>>> and format defintions. 09/15 is a standalone patch which can be merged
>>> independently I believe.
>>>
>>> Patch (11-12)/15 deals with the colorspace support.
>>>
>>> Patch 13/15 allows multiple instances of the ISP.
>>>
>>> Patch 14/15 adds a admin-guide document on bcm2835-isp.
>>>
>>> Patch 15/15 deals with driver registeration.
>>
>> since this is V2, i couldn't find a changelog? The first one was one
>> year ago, so it's very hard to remember. In such a case a link to the
>> last version is also helpful.
>
> I am in your same situation. The changelog won't help here since the I
> have too have lost count of the iterations.
>
> In brief, I would say, the patches have further cleaned up and
> aligned, rebased over the latest staging-next branch.
> That majorly includes work I did for VC04 in general for de-staging.
since this version
https://lore.kernel.org/lkml/910dbd8f-85f2-4979-49ee-6d760e89af84@ideasonboard.com/T/#mca165d5dcb1ba9f3ff3bf91ca2bc6ec2d3ae1c29
there were some feedback from other people like Laurent. I think it's
necessary to know if all the comments has been addressed or not. I think
there is no need to list every single change.
>
>>
>> Recently i saw you have a Github account. For such a complex series it's
>> also very helpful to provide a link to a working test branch with this
>> series applied. It's not a must, but appreciated.
>
> I actually was going to send test branch but at the last minute I
> screwed up my .git repo because of a failed branch checkout.
>
> I had to recreate the branches over the weekend and here they are:
>
> I have tested these branches:
>
> On RPi-3-b 32-bit:
> Git branch:
> https://git.uk.ideasonboard.com/uajain/linux/commits/branch/uajain/rpi3/staging-next/isp
>
> On RPi-4-b 64-bit:
> Git branch:
> https://git.uk.ideasonboard.com/uajain/linux/commits/branch/uajain/rpi4/staging-next/isp
>
> Both these branches have been tested with IMX219 sensor (there is a DT
> overlay included) and libcamera. These branches have bcm2835-unicam
> with streams support (which is handled by Laurent Pinchart as I
> believe) and on top sits bcm2835-isp (this series).
> You'll find DT and kernel configs I have tested with, in the
> respective branches.
>
> As far as libcamera branch goes (it requires a special branch due to
> streams support):
> https://git.uk.ideasonboard.com/uajain/libcamera/commits/branch/pinchartl/rpi/streams
>
>
> Let me know how you get on if you are testing. I am happy to
> answer/help if you decide to test this.
This is great, i don't know when i've the time to test, but it also give
others the chance to do tests.
Regards
>>
>> Best regards
>>
>>>
>>> Dave Stevenson (6):
>>> staging: vc04_services: Add new vc-sm-cma driver
>>> media: videobuf2: Allow exporting of a struct dmabuf
>>> staging: mmal-vchiq: Add support for event callbacks
>>> staging: mmal-vchiq: Use vc-sm-cma to support zero copy
>>> staging: mmal_vchiq: Add image formats to be used by bcm2835-isp
>>> uapi: bcm2835-isp: Add bcm2835-isp uapi header file
>>>
>>> David Plowman (2):
>>> vc04_services: bcm2835-isp: Allow formats with different colour
>>> spaces
>>> vc04_services: bcm2835-isp: Permit all sRGB colour spaces on ISP
>>> outputs
>>>
>>> Naushir Patuck (4):
>>> media: uapi: v4l2-core: Add ISP statistics output V4L2 fourcc type
>>> staging: vc04_services: bcm2835-isp: Add a more complex ISP
>>> processing
>>> component
>>> staging: vc04_services: bcm2835_isp: Allow multiple users
>>> docs: admin-guide: media: bcm2835-isp: Add documentation for
>>> bcm2835-isp
>>>
>>> Umang Jain (3):
>>> staging: vc04_services: vchiq_arm: Register vcsm-cma driver
>>> staging: vc04_services: Add helpers for vchiq driver data
>>> staging: vc04_services: vchiq: Register bcm2835-isp
>>>
>>> .../admin-guide/media/bcm2835-isp.rst | 127 ++
>>> .../userspace-api/media/drivers/index.rst | 1 +
>>> .../userspace-api/media/v4l/meta-formats.rst | 1 +
>>> .../v4l/pixfmt-meta-bcm2835-isp-stats.rst | 32 +
>>> MAINTAINERS | 9 +
>>> .../media/common/videobuf2/videobuf2-core.c | 36 +-
>>> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
>>> drivers/staging/vc04_services/Kconfig | 4 +
>>> drivers/staging/vc04_services/Makefile | 3 +-
>>> .../staging/vc04_services/bcm2835-isp/Kconfig | 14 +
>>> .../vc04_services/bcm2835-isp/Makefile | 4 +
>>> .../bcm2835-isp/bcm2835-isp-ctrls.h | 72 +
>>> .../bcm2835-isp/bcm2835-isp-fmts.h | 559 +++++
>>> .../bcm2835-isp/bcm2835-v4l2-isp.c | 1822
>>> +++++++++++++++++
>>> .../interface/vchiq_arm/vchiq_arm.c | 6 +
>>> .../interface/vchiq_arm/vchiq_bus.h | 10 +
>>> .../staging/vc04_services/vc-sm-cma/Kconfig | 10 +
>>> .../staging/vc04_services/vc-sm-cma/Makefile | 4 +
>>> .../staging/vc04_services/vc-sm-cma/vc_sm.c | 817 ++++++++
>>> .../staging/vc04_services/vc-sm-cma/vc_sm.h | 54 +
>>> .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c | 507 +++++
>>> .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h | 63 +
>>> .../vc04_services/vc-sm-cma/vc_sm_defs.h | 187 ++
>>> .../vc04_services/vc-sm-cma/vc_sm_knl.h | 28 +
>>> .../staging/vc04_services/vchiq-mmal/Kconfig | 1 +
>>> .../vc04_services/vchiq-mmal/mmal-common.h | 5 +
>>> .../vc04_services/vchiq-mmal/mmal-encodings.h | 66 +
>>> .../vc04_services/vchiq-mmal/mmal-msg.h | 35 +
>>> .../vchiq-mmal/mmal-parameters.h | 165 +-
>>> .../vc04_services/vchiq-mmal/mmal-vchiq.c | 253 ++-
>>> .../vc04_services/vchiq-mmal/mmal-vchiq.h | 5 +
>>> include/media/videobuf2-core.h | 15 +
>>> include/uapi/linux/bcm2835-isp.h | 347 ++++
>>> include/uapi/linux/v4l2-controls.h | 5 +
>>> include/uapi/linux/videodev2.h | 1 +
>>> 35 files changed, 5235 insertions(+), 34 deletions(-)
>>> create mode 100644 Documentation/admin-guide/media/bcm2835-isp.rst
>>> create mode 100644
>>> Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
>>> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
>>> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
>>> create mode 100644
>>> drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-ctrls.h
>>> create mode 100644
>>> drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
>>> create mode 100644
>>> drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
>>> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
>>> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
>>> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
>>> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
>>> create mode 100644
>>> drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
>>> create mode 100644
>>> drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
>>> create mode 100644
>>> drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
>>> create mode 100644
>>> drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
>>> create mode 100644 include/uapi/linux/bcm2835-isp.h
>>>
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH] fixup! staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component
2023-11-13 14:07 ` [PATCH] fixup! " Umang Jain
@ 2023-11-13 15:25 ` Stefan Wahren
2023-11-23 12:56 ` Greg KH
1 sibling, 0 replies; 45+ messages in thread
From: Stefan Wahren @ 2023-11-13 15:25 UTC (permalink / raw)
To: Umang Jain, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Laurent Pinchart, Ricardo B . Marliere,
Dan Carpenter, Stefan Wahren
Hi Umang,
Am 13.11.23 um 15:07 schrieb Umang Jain:
> Setup DMA Mask
> ---
> I realised while (re)creating the test branches (sent in reply to the cover
> letter), I missed to squash this hunk in
> "staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component"
> before sending out the series.
>
> Sending it as a fixup! for now.
sorry, but i lost track of all your vchiq patch series. Usually a
commiter shouldn't send multiple series to a list at the same time. This
is very hard to review / test.
An expected review time for a patch series should be 2 weeks, especially
such big series like bcm2835-isp.
For this specific case, it would be better to reply to your own patch
and mention this mistake and fix. It's clear that we need a V3 of the
bcm2835-isp, so in this series the fix should be finally included.
I didn't had the time for the big series, but sending more patches
doesn't make me faster ...
Regards
Stefan
> ---
> .../staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
> index 316d35d5f19d..4a23f9feb6e4 100644
> --- a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
> +++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
> @@ -1774,6 +1774,12 @@ static int bcm2835_isp_probe(struct vchiq_device *device)
> unsigned int i;
> int ret;
>
> + ret = dma_set_mask_and_coherent(&device->dev, DMA_BIT_MASK(32));
> + if (ret) {
> + dev_err(&device->dev, "dma_set_mask_and_coherent failed: %d\n", ret);
> + return ret;
> + }
> +
> bcm2835_isp_instances = devm_kzalloc(&device->dev,
> sizeof(bcm2835_isp_instances) *
> BCM2835_ISP_NUM_INSTANCES,
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 01/15] staging: vc04_services: Add new vc-sm-cma driver
2023-11-09 21:02 ` [PATCH v2 01/15] staging: vc04_services: Add new vc-sm-cma driver Umang Jain
@ 2023-11-13 17:56 ` Stefan Wahren
2023-11-15 19:56 ` Laurent Pinchart
2023-11-20 9:52 ` Dan Carpenter
1 sibling, 1 reply; 45+ messages in thread
From: Stefan Wahren @ 2023-11-13 17:56 UTC (permalink / raw)
To: Umang Jain, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter
Hi Umang,
Am 09.11.23 um 22:02 schrieb Umang Jain:
> From: Dave Stevenson <dave.stevenson@raspberrypi.com>
>
> Add Broadcom VideoCore Shared Memory support.
>
> This new driver allows contiguous memory blocks to be imported
> into the VideoCore VPU memory map, and manages the lifetime of
> those objects, only releasing the source dmabuf once the VPU has
> confirmed it has finished with it.
>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
> ---
> drivers/staging/vc04_services/Kconfig | 2 +
> drivers/staging/vc04_services/Makefile | 2 +-
> .../staging/vc04_services/vc-sm-cma/Kconfig | 10 +
> .../staging/vc04_services/vc-sm-cma/Makefile | 4 +
> .../staging/vc04_services/vc-sm-cma/vc_sm.c | 817 ++++++++++++++++++
> .../staging/vc04_services/vc-sm-cma/vc_sm.h | 54 ++
> .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c | 507 +++++++++++
> .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h | 63 ++
> .../vc04_services/vc-sm-cma/vc_sm_defs.h | 187 ++++
> .../vc04_services/vc-sm-cma/vc_sm_knl.h | 28 +
> 10 files changed, 1673 insertions(+), 1 deletion(-)
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
>
> diff --git a/drivers/staging/vc04_services/Kconfig b/drivers/staging/vc04_services/Kconfig
> index 31e58c9d1a11..6c0e77d64376 100644
> --- a/drivers/staging/vc04_services/Kconfig
> +++ b/drivers/staging/vc04_services/Kconfig
> @@ -46,5 +46,7 @@ source "drivers/staging/vc04_services/bcm2835-camera/Kconfig"
>
> source "drivers/staging/vc04_services/vchiq-mmal/Kconfig"
>
> +source "drivers/staging/vc04_services/vc-sm-cma/Kconfig"
> +
> endif
>
> diff --git a/drivers/staging/vc04_services/Makefile b/drivers/staging/vc04_services/Makefile
> index e8b897a7b9a6..1f61a263f22d 100644
> --- a/drivers/staging/vc04_services/Makefile
> +++ b/drivers/staging/vc04_services/Makefile
> @@ -15,4 +15,4 @@ endif
> obj-$(CONFIG_SND_BCM2835) += bcm2835-audio/
> obj-$(CONFIG_VIDEO_BCM2835) += bcm2835-camera/
> obj-$(CONFIG_BCM2835_VCHIQ_MMAL) += vchiq-mmal/
> -
> +obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma/
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/Kconfig b/drivers/staging/vc04_services/vc-sm-cma/Kconfig
> new file mode 100644
> index 000000000000..bbd296f5826b
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/Kconfig
> @@ -0,0 +1,10 @@
> +config BCM_VC_SM_CMA
> + tristate "VideoCore Shared Memory (CMA) driver"
> + depends on BCM2835_VCHIQ
> + select RBTREE
> + select DMA_SHARED_BUFFER
> + help
> + Say Y here to enable the shared memory interface that
> + supports sharing dmabufs with VideoCore.
> + This operates over the VCHIQ interface to a service
> + running on VideoCore.
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/Makefile b/drivers/staging/vc04_services/vc-sm-cma/Makefile
> new file mode 100644
> index 000000000000..98c8ed6c9a50
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/Makefile
> @@ -0,0 +1,4 @@
> +vc-sm-cma-$(CONFIG_BCM_VC_SM_CMA) := \
> + vc_sm.o vc_sm_cma_vchi.o
> +
> +obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma.o
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> new file mode 100644
> index 000000000000..5bbdb3a8468b
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> @@ -0,0 +1,817 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * VideoCore Shared Memory driver using CMA.
> + *
> + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> + *
> + * Based on vmcs_sm driver from Broadcom Corporation for some API,
> + * and taking some code for buffer allocation and dmabuf handling from
> + * videobuf2.
> + *
> + * This driver handles taking a dmabuf, mapping it into the VPU address space,
> + * and returning a handle that the VPU can use.
> + */
> +
> +/* ---- Include Files ----------------------------------------------------- */
please drop such obvious comments
> +#include <linux/debugfs.h>
> +#include <linux/dma-buf.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/errno.h>
> +#include <linux/miscdevice.h>
> +#include <linux/syscalls.h>
> +
> +#include "../interface/vchiq_arm/vchiq_bus.h"
> +#include "../interface/vchiq_arm/vchiq_connected.h"
> +#include "vc_sm_cma_vchi.h"
> +
> +#include "vc_sm.h"
> +#include "vc_sm_knl.h"
> +
> +MODULE_IMPORT_NS(DMA_BUF);
> +
> +/* ---- Private Constants and Types --------------------------------------- */
> +
> +#define VC_SM_RESOURCE_NAME_DEFAULT "sm-host-resource"
> +
> +#define VC_SM_DIR_ROOT_NAME "vcsm-cma"
> +#define VC_SM_STATE "state"
> +
> +/* Private file data associated with each opened device. */
> +struct vc_sm_privdata_t {
> +};
> +
> +typedef int (*VC_SM_SHOW) (struct seq_file *s, void *v);
please no typedef
> +struct sm_pde_t {
> + VC_SM_SHOW show; /* Debug fs function hookup. */
> + struct dentry *dir_entry; /* Debug fs directory entry. */
> + void *priv_data; /* Private data */
> +};
> +
> +/* Global state information. */
> +struct sm_state_t {
> + struct vchiq_device *device;
> +
> + struct miscdevice misc_dev;
> +
> + struct sm_instance *sm_handle; /* Handle for videocore service. */
> +
> + spinlock_t kernelid_map_lock; /* Spinlock protecting kernelid_map */
> + struct idr kernelid_map;
> +
> + struct mutex map_lock; /* Global map lock. */
i think the variable name is good, but the comment doesn't provide a
real benefit.
> + struct list_head buffer_list; /* List of buffer. */
dito. What buffers?
> +
> + struct dentry *dir_root; /* Debug fs entries root. */
> + struct sm_pde_t dir_state; /* Debug fs entries state sub-tree. */
> +
> + bool require_released_callback; /* VPU will send a released msg when it
> + * has finished with a resource.
> + */
> + /* State for transactions */
> + int restart_sys; /* Tracks restart on interrupt. */
> + enum vc_sm_msg_type int_action; /* Interrupted action. */
> +
> + u32 int_trans_id; /* Interrupted transaction. */
> + struct vchiq_instance *vchiq_instance;
> +};
> +
> +struct vc_sm_dma_buf_attachment {
> + struct device *dev;
> + struct sg_table sg_table;
> + struct list_head list;
> + enum dma_data_direction dma_dir;
> +};
> +
> +/* ---- Private Variables ----------------------------------------------- */
> +
> +static struct sm_state_t *sm_state;
> +static int sm_inited;
In case the driver cannot be loaded again, a comment about this should
be here.
> +
> +/* ---- Private Function Prototypes -------------------------------------- */
> +
> +/* ---- Private Functions ------------------------------------------------ */
> +
> +static int get_kernel_id(struct vc_sm_buffer *buffer)
> +{
> + int handle;
> +
> + spin_lock(&sm_state->kernelid_map_lock);
> + handle = idr_alloc(&sm_state->kernelid_map, buffer, 0, 0, GFP_KERNEL);
> + spin_unlock(&sm_state->kernelid_map_lock);
> +
> + return handle;
> +}
> +
> +static struct vc_sm_buffer *lookup_kernel_id(int handle)
> +{
> + return idr_find(&sm_state->kernelid_map, handle);
> +}
> +
> +static void free_kernel_id(int handle)
> +{
> + spin_lock(&sm_state->kernelid_map_lock);
> + idr_remove(&sm_state->kernelid_map, handle);
> + spin_unlock(&sm_state->kernelid_map_lock);
> +}
> +
> +static int vc_sm_cma_seq_file_show(struct seq_file *s, void *v)
> +{
> + struct sm_pde_t *sm_pde;
> +
> + sm_pde = (struct sm_pde_t *)(s->private);
Doesn't this fit into one line?
> +
> + if (sm_pde && sm_pde->show)
> + sm_pde->show(s, v);
> +
> + return 0;
> +}
> +
> +static int vc_sm_cma_single_open(struct inode *inode, struct file *file)
> +{
> + return single_open(file, vc_sm_cma_seq_file_show, inode->i_private);
> +}
> +
> +static const struct file_operations vc_sm_cma_debug_fs_fops = {
> + .open = vc_sm_cma_single_open,
> + .read = seq_read,
> + .llseek = seq_lseek,
> + .release = single_release,
> +};
> +
> +static int vc_sm_cma_global_state_show(struct seq_file *s, void *v)
> +{
> + struct vc_sm_buffer *resource = NULL;
> + int resource_count = 0;
> +
> + if (!sm_state)
> + return 0;
> +
> + seq_printf(s, "\nVC-ServiceHandle %p\n", sm_state->sm_handle);
> +
> + /* Log all applicable mapping(s). */
> +
> + mutex_lock(&sm_state->map_lock);
> + seq_puts(s, "\nResources\n");
> + if (!list_empty(&sm_state->buffer_list)) {
> + list_for_each_entry(resource, &sm_state->buffer_list,
> + global_buffer_list) {
> + resource_count++;
> +
> + seq_printf(s, "\nResource %p\n",
> + resource);
> + seq_printf(s, " NAME %s\n",
> + resource->name);
> + seq_printf(s, " SIZE %zu\n",
> + resource->size);
> + seq_printf(s, " DMABUF %p\n",
> + resource->dma_buf);
> + seq_printf(s, " IMPORTED_DMABUF %p\n",
> + resource->imported_dma_buf);
> + seq_printf(s, " ATTACH %p\n",
> + resource->attach);
> + seq_printf(s, " SGT %p\n",
> + resource->sgt);
> + seq_printf(s, " DMA_ADDR %pad\n",
> + &resource->dma_addr);
> + seq_printf(s, " VC_HANDLE %08x\n",
> + resource->vc_handle);
> + seq_printf(s, " VC_MAPPING %d\n",
> + resource->vpu_state);
> + }
> + }
> + seq_printf(s, "\n\nTotal resource count: %d\n\n", resource_count);
> +
> + mutex_unlock(&sm_state->map_lock);
> +
> + return 0;
> +}
> +
> +/*
> + * Adds a buffer to the private data list which tracks all the allocated
> + * data.
> + */
> +static void vc_sm_add_resource(struct vc_sm_buffer *buffer)
> +{
> + mutex_lock(&sm_state->map_lock);
> + list_add(&buffer->global_buffer_list, &sm_state->buffer_list);
> + mutex_unlock(&sm_state->map_lock);
> +
> + dev_dbg(&sm_state->device->dev,"[%s]: added buffer %p (name %s, size %zu)\n",
> + __func__, buffer, buffer->name, buffer->size);
> +}
> +
> +/*
> + * Cleans up imported dmabuf.
> + * Should be called with mutex held.
> + */
> +static void vc_sm_clean_up_dmabuf(struct vc_sm_buffer *buffer)
> +{
> + /* Handle cleaning up imported dmabufs */
> + if (buffer->sgt) {
> + dma_buf_unmap_attachment_unlocked(buffer->attach, buffer->sgt,
> + DMA_BIDIRECTIONAL);
> + buffer->sgt = NULL;
> + }
> + if (buffer->attach) {
> + dma_buf_detach(buffer->imported_dma_buf, buffer->attach);
> + buffer->attach = NULL;
> + }
> +}
> +
> +/*
> + * Instructs VPU to decrement the refcount on a buffer.
> + */
> +static void vc_sm_vpu_free(struct vc_sm_buffer *buffer)
> +{
> + if (buffer->vc_handle && buffer->vpu_state == VPU_MAPPED) {
> + struct vc_sm_free_t free = { buffer->vc_handle, 0 };
> + int status = vc_sm_cma_vchi_free(sm_state->sm_handle, &free,
> + &sm_state->int_trans_id);
> + if (status != 0 && status != -EINTR) {
> + dev_err(&sm_state->device->dev,
> + "[%s]: failed to free memory on videocore (status: %u, trans_id: %u)\n",
> + __func__, status, sm_state->int_trans_id);
> + }
> +
> + if (sm_state->require_released_callback) {
> + /* Need to wait for the VPU to confirm the free. */
> +
> + /* Retain a reference on this until the VPU has
> + * released it
> + */
> + buffer->vpu_state = VPU_UNMAPPING;
> + } else {
> + buffer->vpu_state = VPU_NOT_MAPPED;
> + buffer->vc_handle = 0;
> + }
> + }
> +}
> +
> +/*
> + * Release an allocation.
> + * All refcounting is done via the dma buf object.
> + *
> + * Must be called with the mutex held. The function will either release the
> + * mutex (if defering the release) or destroy it. The caller must therefore not
> + * reuse the buffer on return.
> + */
> +static void vc_sm_release_resource(struct vc_sm_buffer *buffer)
> +{
> + dev_dbg(&sm_state->device->dev, "[%s]: buffer %p (name %s, size %zu)\n",
> + __func__, buffer, buffer->name, buffer->size);
> +
> + if (buffer->vc_handle) {
> + /* We've sent the unmap request but not had the response. */
> + dev_dbg(&sm_state->device->dev, "[%s]: Waiting for VPU unmap response on %p\n",
> + __func__, buffer);
> + goto defer;
> + }
> + if (buffer->in_use) {
> + /* dmabuf still in use - we await the release */
> + dev_dbg(&sm_state->device->dev, "[%s]: buffer %p is still in use\n",
> + __func__, buffer);
> + goto defer;
> + }
> +
> + /* Release the allocation */
> + if (buffer->imported_dma_buf)
> + dma_buf_put(buffer->imported_dma_buf);
> + else
> + dev_err(&sm_state->device->dev, "%s: Imported dmabuf already been put for buf %p\n",
> + __func__, buffer);
> + buffer->imported_dma_buf = NULL;
> +
> + /* Free our buffer. Start by removing it from the list */
> + mutex_lock(&sm_state->map_lock);
> + list_del(&buffer->global_buffer_list);
> + mutex_unlock(&sm_state->map_lock);
> +
> + dev_dbg(&sm_state->device->dev, "%s: Release our allocation - done\n", __func__);
> + mutex_unlock(&buffer->lock);
> +
> + mutex_destroy(&buffer->lock);
> +
> + kfree(buffer);
> + return;
> +
> +defer:
> + mutex_unlock(&buffer->lock);
> +}
> +
> +/* Dma_buf operations for chaining through to an imported dma_buf */
> +
> +static void vc_sm_dma_buf_release(struct dma_buf *dmabuf)
> +{
> + struct vc_sm_buffer *buffer;
> +
> + if (!dmabuf)
> + return;
> +
> + buffer = (struct vc_sm_buffer *)dmabuf->priv;
> +
> + mutex_lock(&buffer->lock);
> +
> + dev_dbg(&sm_state->device->dev, "%s dmabuf %p, buffer %p\n", __func__, dmabuf, buffer);
> +
> + buffer->in_use = 0;
> +
> + /* Unmap on the VPU */
> + vc_sm_vpu_free(buffer);
> + dev_dbg(&sm_state->device->dev, "%s vpu_free done\n", __func__);
> +
> + /* Unmap our dma_buf object (the vc_sm_buffer remains until released
> + * on the VPU).
> + */
> + vc_sm_clean_up_dmabuf(buffer);
> + dev_dbg(&sm_state->device->dev, "%s clean_up dmabuf done\n", __func__);
> +
> + /* buffer->lock will be destroyed by vc_sm_release_resource if finished
> + * with, otherwise unlocked. Do NOT unlock here.
> + */
> + vc_sm_release_resource(buffer);
> + dev_dbg(&sm_state->device->dev, "%s done\n", __func__);
> +}
> +
> +static
> +int vc_sm_import_dma_buf_attach(struct dma_buf *dmabuf,
> + struct dma_buf_attachment *attachment)
> +{
> + struct vc_sm_buffer *buf = dmabuf->priv;
> +
> + return buf->imported_dma_buf->ops->attach(buf->imported_dma_buf,
> + attachment);
> +}
> +
> +static
> +void vc_sm_import_dma_buf_detatch(struct dma_buf *dmabuf,
> + struct dma_buf_attachment *attachment)
> +{
> + struct vc_sm_buffer *buf = dmabuf->priv;
> +
> + buf->imported_dma_buf->ops->detach(buf->imported_dma_buf, attachment);
> +}
> +
> +static
> +struct sg_table *vc_sm_import_map_dma_buf(struct dma_buf_attachment *attachment,
> + enum dma_data_direction direction)
> +{
> + struct vc_sm_buffer *buf = attachment->dmabuf->priv;
> +
> + return buf->imported_dma_buf->ops->map_dma_buf(attachment,
> + direction);
> +}
> +
> +static
> +void vc_sm_import_unmap_dma_buf(struct dma_buf_attachment *attachment,
> + struct sg_table *table,
> + enum dma_data_direction direction)
> +{
> + struct vc_sm_buffer *buf = attachment->dmabuf->priv;
> +
> + buf->imported_dma_buf->ops->unmap_dma_buf(attachment, table, direction);
> +}
> +
> +static
> +int vc_sm_import_dmabuf_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma)
> +{
> + struct vc_sm_buffer *buf = dmabuf->priv;
> +
> + dev_dbg(&sm_state->device->dev, "%s: mmap dma_buf %p, buf %p, imported db %p\n", __func__,
> + dmabuf, buf, buf->imported_dma_buf);
> +
> + return buf->imported_dma_buf->ops->mmap(buf->imported_dma_buf, vma);
> +}
> +
> +static
> +int vc_sm_import_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
> + enum dma_data_direction direction)
> +{
> + struct vc_sm_buffer *buf = dmabuf->priv;
> +
> + return buf->imported_dma_buf->ops->begin_cpu_access(buf->imported_dma_buf,
> + direction);
> +}
> +
> +static
> +int vc_sm_import_dma_buf_end_cpu_access(struct dma_buf *dmabuf,
> + enum dma_data_direction direction)
> +{
> + struct vc_sm_buffer *buf = dmabuf->priv;
> +
> + return buf->imported_dma_buf->ops->end_cpu_access(buf->imported_dma_buf,
> + direction);
> +}
> +
> +static const struct dma_buf_ops dma_buf_import_ops = {
> + .map_dma_buf = vc_sm_import_map_dma_buf,
> + .unmap_dma_buf = vc_sm_import_unmap_dma_buf,
> + .mmap = vc_sm_import_dmabuf_mmap,
> + .release = vc_sm_dma_buf_release,
> + .attach = vc_sm_import_dma_buf_attach,
> + .detach = vc_sm_import_dma_buf_detatch,
> + .begin_cpu_access = vc_sm_import_dma_buf_begin_cpu_access,
> + .end_cpu_access = vc_sm_import_dma_buf_end_cpu_access,
> +};
> +
> +/* Import a dma_buf to be shared with VC. */
> +int
> +vc_sm_cma_import_dmabuf_internal(struct sm_state_t *state,
> + struct dma_buf *dma_buf,
> + int fd,
> + struct dma_buf **imported_buf)
> +{
> + DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
> + struct vc_sm_buffer *buffer = NULL;
> + struct vc_sm_import import = { };
> + struct vc_sm_import_result result = { };
> + struct dma_buf_attachment *attach = NULL;
> + struct sg_table *sgt = NULL;
> + dma_addr_t dma_addr;
> + int ret = 0;
> + int status;
> +
> + /* Setup our allocation parameters */
> + dev_dbg(&sm_state->device->dev, "%s: importing dma_buf %p/fd %d\n", __func__, dma_buf, fd);
> +
> + if (fd < 0)
> + get_dma_buf(dma_buf);
> + else
> + dma_buf = dma_buf_get(fd);
> +
> + if (!dma_buf)
> + return -EINVAL;
> +
> + attach = dma_buf_attach(dma_buf, &sm_state->device->dev);
> + if (IS_ERR(attach)) {
> + ret = PTR_ERR(attach);
> + goto error;
> + }
> +
> + sgt = dma_buf_map_attachment_unlocked(attach, DMA_BIDIRECTIONAL);
> + if (IS_ERR(sgt)) {
> + ret = PTR_ERR(sgt);
> + goto error;
> + }
> +
> + /* Verify that the address block is contiguous */
> + if (sgt->nents != 1) {
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + /* Allocate local buffer to track this allocation. */
> + buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
> + if (!buffer) {
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + import.type = VC_SM_ALLOC_NON_CACHED;
> + dma_addr = sg_dma_address(sgt->sgl);
> + import.addr = (u32)dma_addr;
> + if ((import.addr & 0xC0000000) != 0xC0000000) {
> + dev_err(&sm_state->device->dev, "%s: Expecting an uncached alias for dma_addr %pad\n",
> + __func__, &dma_addr);
> + import.addr |= 0xC0000000;
> + }
Sorry, my DMA knowledge is limited but this looks supicious to me.
Aren't there any DMA API helper for this? Even the magic 0xC0000000
needs an explanation or define.
> + import.size = sg_dma_len(sgt->sgl);
> + import.allocator = current->tgid;
> + import.kernel_id = get_kernel_id(buffer);
> +
> + memcpy(import.name, VC_SM_RESOURCE_NAME_DEFAULT,
> + sizeof(VC_SM_RESOURCE_NAME_DEFAULT));
> +
> + dev_dbg(&sm_state->device->dev,
> + "[%s]: attempt to import \"%s\" data - type %u, addr %pad, size %u.\n",
> + __func__, import.name, import.type, &dma_addr, import.size);
> +
> + /* Allocate the videocore buffer. */
> + status = vc_sm_cma_vchi_import(sm_state->sm_handle, &import, &result,
> + &sm_state->int_trans_id);
> + if (status == -EINTR) {
> + dev_dbg(&sm_state->device->dev,
> + "[%s]: requesting import memory action restart (trans_id: %u)\n",
> + __func__, sm_state->int_trans_id);
> + ret = -ERESTARTSYS;
> + sm_state->restart_sys = -EINTR;
> + sm_state->int_action = VC_SM_MSG_TYPE_IMPORT;
> + goto error;
> + } else if (status || !result.res_handle) {
> + dev_dbg(&sm_state->device->dev,
> + "[%s]: failed to import memory on videocore (status: %u, trans_id: %u)\n",
> + __func__, status, sm_state->int_trans_id);
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + mutex_init(&buffer->lock);
> + INIT_LIST_HEAD(&buffer->attachments);
> + memcpy(buffer->name, import.name,
> + min(sizeof(buffer->name), sizeof(import.name) - 1));
> +
> + /* Keep track of the buffer we created. */
> + buffer->vc_handle = result.res_handle;
> + buffer->size = import.size;
> + buffer->vpu_state = VPU_MAPPED;
> +
> + buffer->imported_dma_buf = dma_buf;
> +
> + buffer->attach = attach;
> + buffer->sgt = sgt;
> + buffer->dma_addr = dma_addr;
> + buffer->in_use = 1;
> + buffer->kernel_id = import.kernel_id;
> +
> + /*
> + * We're done - we need to export a new dmabuf chaining through most
> + * functions, but enabling us to release our own internal references
> + * here.
> + */
> + exp_info.ops = &dma_buf_import_ops;
> + exp_info.size = import.size;
> + exp_info.flags = O_RDWR;
> + exp_info.priv = buffer;
> +
> + buffer->dma_buf = dma_buf_export(&exp_info);
> + if (IS_ERR(buffer->dma_buf)) {
> + ret = PTR_ERR(buffer->dma_buf);
> + goto error;
> + }
> +
> + vc_sm_add_resource(buffer);
> +
> + *imported_buf = buffer->dma_buf;
> +
> + return 0;
> +
> +error:
> + if (result.res_handle) {
> + struct vc_sm_free_t free = { result.res_handle, 0 };
> +
> + vc_sm_cma_vchi_free(sm_state->sm_handle, &free,
> + &sm_state->int_trans_id);
> + }
> + free_kernel_id(import.kernel_id);
> + kfree(buffer);
> + if (sgt)
> + dma_buf_unmap_attachment_unlocked(attach, sgt, DMA_BIDIRECTIONAL);
> + if (attach)
> + dma_buf_detach(dma_buf, attach);
> + dma_buf_put(dma_buf);
> + return ret;
> +}
> +
> +static void
> +vc_sm_vpu_event(struct sm_instance *instance, struct vc_sm_result_t *reply,
> + int reply_len)
> +{
> + switch (reply->trans_id & ~0x80000000) {
This magic number needs a explanation or a define
> + case VC_SM_MSG_TYPE_CLIENT_VERSION:
> + {
> + /* Acknowledge that the firmware supports the version command */
> + dev_dbg(&sm_state->device->dev,
> + "%s: firmware acked version msg. Require release cb\n", __func__);
> + sm_state->require_released_callback = true;
> + }
> + break;
> + case VC_SM_MSG_TYPE_RELEASED:
> + {
> + struct vc_sm_released *release = (struct vc_sm_released *)reply;
> + struct vc_sm_buffer *buffer =
> + lookup_kernel_id(release->kernel_id);
> + if (!buffer) {
> + dev_err(&sm_state->device->dev,
> + "%s: VC released a buffer that is already released, kernel_id %d\n",
> + __func__, release->kernel_id);
> + break;
> + }
> + mutex_lock(&buffer->lock);
> +
> + dev_dbg(&sm_state->device->dev,
> + "%s: Released addr %08x, size %u, id %08x, mem_handle %08x\n",
> + __func__, release->addr, release->size,
> + release->kernel_id, release->vc_handle);
> +
> + buffer->vc_handle = 0;
> + buffer->vpu_state = VPU_NOT_MAPPED;
> + free_kernel_id(release->kernel_id);
> +
> + vc_sm_release_resource(buffer);
> + }
> + break;
> + default:
> + dev_err(&sm_state->device->dev, "%s: Unknown vpu cmd %x\n",
> + __func__, reply->trans_id);
> + break;
> + }
> +}
> +
> +/* Driver load/unload functions */
> +/* Videocore connected. */
> +static void vc_sm_connected_init(void)
> +{
> + int ret;
> + struct vc_sm_version version;
> + struct vc_sm_result_t version_result;
> +
> + pr_info("[%s]: start\n", __func__);
> +
> + /*
> + * Initialize and create a VCHI connection for the shared memory service
> + * running on videocore.
> + */
> + ret = vchiq_initialise(&sm_state->vchiq_instance);
> + if (ret) {
> + dev_err(&sm_state->device->dev, "[%s]: failed to initialise VCHI instance (ret=%d)\n",
> + __func__, ret);
> +
> + return;
> + }
> +
> + ret = vchiq_connect(sm_state->vchiq_instance);
> + if (ret) {
> + dev_err(&sm_state->device->dev, "[%s]: failed to connect VCHI instance (ret=%d)\n",
> + __func__, ret);
> +
> + return;
> + }
> +
> + /* Initialize an instance of the shared memory service. */
> + sm_state->sm_handle = vc_sm_cma_vchi_init(sm_state->vchiq_instance, 1,
> + vc_sm_vpu_event);
> + if (!sm_state->sm_handle) {
> + dev_err(&sm_state->device->dev,
> + "[%s]: failed to initialize shared memory service\n", __func__);
> +
> + return;
> + }
> +
> + /* Create a debug fs directory entry (root). */
> + sm_state->dir_root = debugfs_create_dir(VC_SM_DIR_ROOT_NAME, NULL);
> +
> + sm_state->dir_state.show = &vc_sm_cma_global_state_show;
> + sm_state->dir_state.dir_entry =
> + debugfs_create_file(VC_SM_STATE, 0444, sm_state->dir_root,
> + &sm_state->dir_state,
> + &vc_sm_cma_debug_fs_fops);
> +
> + INIT_LIST_HEAD(&sm_state->buffer_list);
> +
> + version.version = 2;
VC_SM_PROTOCOL_VERSION or it different?
> + ret = vc_sm_cma_vchi_client_version(sm_state->sm_handle, &version,
> + &version_result,
> + &sm_state->int_trans_id);
> + if (ret) {
> + dev_err(&sm_state->device->dev,
> + "[%s]: Failed to send version request %d\n", __func__, ret);
> + }
> +
> + /* Done! */
> + sm_inited = 1;
> + pr_info("[%s]: installed successfully\n", __func__);
> +}
> +
> +/* Driver loading. */
> +static int bcm2835_vc_sm_cma_probe(struct vchiq_device *device)
> +{
> + struct device *dev = &device->dev;
> + int err;
> +
> + err = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
> + if (err) {
> + dev_err(dev, "dma_set_mask_and_coherent failed: %d\n", err);
> + return err;
> + }
> +
> + sm_state = devm_kzalloc(dev, sizeof(*sm_state), GFP_KERNEL);
> + if (!sm_state)
> + return -ENOMEM;
> + sm_state->device = device;
> + mutex_init(&sm_state->map_lock);
> +
> + spin_lock_init(&sm_state->kernelid_map_lock);
> + idr_init_base(&sm_state->kernelid_map, 1);
> +
> + dev->dma_parms = devm_kzalloc(dev, sizeof(dev->dma_parms), GFP_KERNEL);
> +
> + /* dma_set_max_seg_size checks if dma_parms is NULL. */
> + dma_set_max_seg_size(dev, 0x3FFFFFFF);
> +
> + vchiq_add_connected_callback(vc_sm_connected_init);
> + return 0;
> +}
> +
> +/* Driver unloading. */
> +static void bcm2835_vc_sm_cma_remove(struct vchiq_device *device)
> +{
> + if (sm_inited) {
> + misc_deregister(&sm_state->misc_dev);
> +
> + /* Remove all proc entries. */
> + debugfs_remove_recursive(sm_state->dir_root);
> +
> + /* Stop the videocore shared memory service. */
> + vc_sm_cma_vchi_stop(sm_state->vchiq_instance, &sm_state->sm_handle);
> + }
> +
> + if (sm_state) {
> + idr_destroy(&sm_state->kernelid_map);
> +
> + /* Free the memory for the state structure. */
> + mutex_destroy(&sm_state->map_lock);
> + }
> +}
> +
> +/* Kernel API calls */
> +/* Get an internal resource handle mapped from the external one. */
> +int vc_sm_cma_int_handle(void *handle)
> +{
> + struct dma_buf *dma_buf = (struct dma_buf *)handle;
> + struct vc_sm_buffer *buf;
> +
> + /* Validate we can work with this device. */
> + if (!sm_state || !handle) {
> + dev_err(&sm_state->device->dev, "[%s]: invalid input\n", __func__);
> + return 0;
> + }
> +
> + buf = (struct vc_sm_buffer *)dma_buf->priv;
> + return buf->vc_handle;
> +}
> +EXPORT_SYMBOL_GPL(vc_sm_cma_int_handle);
> +
> +/* Free a previously allocated shared memory handle and block. */
> +int vc_sm_cma_free(void *handle)
> +{
> + struct dma_buf *dma_buf = (struct dma_buf *)handle;
> +
> + /* Validate we can work with this device. */
> + if (!sm_state || !handle) {
> + dev_err(&sm_state->device->dev, "[%s]: invalid input\n", __func__);
> + return -EPERM;
> + }
> +
> + dev_dbg(&sm_state->device->dev, "%s: handle %p/dmabuf %p\n", __func__, handle, dma_buf);
> +
> + dma_buf_put(dma_buf);
> +
> + return 0;
> +}
> +EXPORT_SYMBOL_GPL(vc_sm_cma_free);
> +
> +/* Import a dmabuf to be shared with VC. */
> +int vc_sm_cma_import_dmabuf(struct dma_buf *src_dmabuf, void **handle)
> +{
> + struct dma_buf *new_dma_buf;
> + int ret;
> +
> + /* Validate we can work with this device. */
> + if (!sm_state || !src_dmabuf || !handle) {
> + dev_err(&sm_state->device->dev, "[%s]: invalid input\n", __func__);
> + return -EPERM;
> + }
> +
> + ret = vc_sm_cma_import_dmabuf_internal(sm_state, src_dmabuf,
> + -1, &new_dma_buf);
> +
> + if (!ret) {
> + dev_dbg(&sm_state->device->dev, "%s: imported to ptr %p\n",
> + __func__, new_dma_buf);
> +
> + /* Assign valid handle at this time.*/
> + *handle = new_dma_buf;
> + } else {
> + /*
> + * succeeded in importing the dma_buf, but then
> + * failed to look it up again. How?
> + * Release the fd again.
> + */
> + dev_err(&sm_state->device->dev, "%s: imported vc_sm_cma_get_buffer failed %d\n",
> + __func__, ret);
> + }
> +
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(vc_sm_cma_import_dmabuf);
> +
> +static struct vchiq_device_id device_id_table[] = {
> + { .name = "vcsm-cma" },
> + {}
> +};
> +MODULE_DEVICE_TABLE(vchiq, device_id_table);
> +
> +static struct vchiq_driver bcm2835_vcsm_cma_driver = {
> + .probe = bcm2835_vc_sm_cma_probe,
> + .remove = bcm2835_vc_sm_cma_remove,
> + .id_table = device_id_table,
> + .driver = {
> + .name = "vcsm-cma",
> + },
> +};
> +
> +module_vchiq_driver(bcm2835_vcsm_cma_driver);
> +
> +MODULE_AUTHOR("Dave Stevenson");
> +MODULE_DESCRIPTION("VideoCore CMA Shared Memory Driver");
> +MODULE_LICENSE("GPL");
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> new file mode 100644
> index 000000000000..61e110ec414d
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> @@ -0,0 +1,54 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +/*
> + * VideoCore Shared Memory driver using CMA.
> + *
> + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> + *
> + */
> +
> +#ifndef VC_SM_H
> +#define VC_SM_H
> +
> +#define VC_SM_MAX_NAME_LEN 32
> +
> +enum vc_sm_vpu_mapping_state {
> + VPU_NOT_MAPPED,
> + VPU_MAPPED,
> + VPU_UNMAPPING
> +};
> +
> +struct vc_sm_buffer {
> + struct list_head global_buffer_list; /* Global list of buffers. */
> +
> + /* Index in the kernel_id idr so that we can find the
> + * mmal_msg_context again when servicing the VCHI reply.
> + */
> + int kernel_id;
> +
> + size_t size;
> +
> + /* Lock over all the following state for this buffer */
> + struct mutex lock;
> + struct list_head attachments;
> +
> + char name[VC_SM_MAX_NAME_LEN];
> +
> + int in_use:1; /* Kernel is still using this resource */
> +
> + enum vc_sm_vpu_mapping_state vpu_state;
> + u32 vc_handle; /* VideoCore handle for this buffer */
> +
> + /* DMABUF related fields */
> + struct dma_buf *dma_buf;
> + dma_addr_t dma_addr;
> + void *cookie;
> +
> + struct vc_sm_privdata_t *private;
> +
> + struct dma_buf *imported_dma_buf;
> + struct dma_buf_attachment *attach;
> + struct sg_table *sgt;
> +};
> +
> +#endif
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> new file mode 100644
> index 000000000000..08f70e955acf
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> @@ -0,0 +1,507 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * VideoCore Shared Memory CMA allocator
> + *
> + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> + * Copyright 2011-2012 Broadcom Corporation. All rights reserved.
> + *
> + * Based on vmcs_sm driver from Broadcom Corporation.
> + *
> + */
> +
> +/* ---- Include Files ----------------------------------------------------- */
> +#include <linux/kthread.h>
> +#include <linux/list.h>
> +#include <linux/semaphore.h>
> +#include <linux/slab.h>
> +#include <linux/types.h>
> +
> +#include "vc_sm_cma_vchi.h"
> +
> +#define VC_SM_VER 1
> +#define VC_SM_MIN_VER 0
> +
> +/* ---- Private Constants and Types -------------------------------------- */
> +
> +/* Command blocks come from a pool */
> +#define SM_MAX_NUM_CMD_RSP_BLKS 32
> +
> +/* The number of supported connections */
What kind of connections?
> +#define SM_MAX_NUM_CONNECTIONS 3
> +
> +struct sm_cmd_rsp_blk {
> + struct list_head head; /* To create lists */
> + /* To be signaled when the response is there */
> + struct completion cmplt;
I think the completion deserves a better name.
> +
> + u32 id;
> + u16 length;
> +
> + u8 msg[VC_SM_MAX_MSG_LEN];
> +
> + uint32_t wait:1;
> + uint32_t sent:1;
> + uint32_t alloc:1;
> +
> +};
> +
> +struct sm_instance {
> + u32 num_connections;
> + unsigned int service_handle[SM_MAX_NUM_CONNECTIONS];
> + struct task_struct *io_thread;
> + struct completion io_cmplt;
> +
> + vpu_event_cb vpu_event;
> +
> + /* Mutex over the following lists */
> + struct mutex lock;
> + u32 trans_id;
> + struct list_head cmd_list;
> + struct list_head rsp_list;
> + struct list_head dead_list;
> +
> + struct sm_cmd_rsp_blk free_blk[SM_MAX_NUM_CMD_RSP_BLKS];
> +
> + /* Mutex over the free_list */
> + struct mutex free_lock;
> + struct list_head free_list;
> +
> + struct semaphore free_sema;
> + struct vchiq_instance *vchiq_instance;
> +};
> +
> +/* ---- Private Variables ------------------------------------------------ */
> +
> +/* ---- Private Function Prototypes -------------------------------------- */
> +
> +/* ---- Private Functions ------------------------------------------------ */
> +static int
> +bcm2835_vchi_msg_queue(struct vchiq_instance *vchiq_instance, unsigned int handle,
> + void *data,
> + unsigned int size)
> +{
> + return vchiq_queue_kernel_message(vchiq_instance, handle, data, size);
> +}
> +
> +static struct
> +sm_cmd_rsp_blk *vc_vchi_cmd_create(struct sm_instance *instance,
> + enum vc_sm_msg_type id, void *msg,
> + u32 size, int wait)
> +{
> + struct sm_cmd_rsp_blk *blk;
> + struct vc_sm_msg_hdr_t *hdr;
> +
> + if (down_interruptible(&instance->free_sema)) {
> + blk = kmalloc(sizeof(*blk), GFP_KERNEL);
> + if (!blk)
> + return NULL;
> +
> + blk->alloc = 1;
> + init_completion(&blk->cmplt);
> + } else {
> + mutex_lock(&instance->free_lock);
> + blk =
> + list_first_entry(&instance->free_list,
> + struct sm_cmd_rsp_blk, head);
> + list_del(&blk->head);
> + mutex_unlock(&instance->free_lock);
> + }
> +
> + blk->sent = 0;
> + blk->wait = wait;
> + blk->length = sizeof(*hdr) + size;
> +
> + hdr = (struct vc_sm_msg_hdr_t *)blk->msg;
> + hdr->type = id;
> + mutex_lock(&instance->lock);
> + instance->trans_id++;
> + /*
> + * Retain the top bit for identifying asynchronous events, or VPU cmds.
> + */
> + instance->trans_id &= ~0x80000000;
magic bit again
> + hdr->trans_id = instance->trans_id;
> + blk->id = instance->trans_id;
> + mutex_unlock(&instance->lock);
> +
> + if (size)
> + memcpy(hdr->body, msg, size);
> +
> + return blk;
> +}
> +
> +static void
> +vc_vchi_cmd_delete(struct sm_instance *instance, struct sm_cmd_rsp_blk *blk)
> +{
> + if (blk->alloc) {
> + kfree(blk);
> + return;
> + }
> +
> + mutex_lock(&instance->free_lock);
> + list_add(&blk->head, &instance->free_list);
> + mutex_unlock(&instance->free_lock);
> + up(&instance->free_sema);
> +}
> +
> +static void vc_sm_cma_vchi_rx_ack(struct sm_instance *instance,
> + struct sm_cmd_rsp_blk *cmd,
> + struct vc_sm_result_t *reply,
> + u32 reply_len)
> +{
> + mutex_lock(&instance->lock);
> + list_for_each_entry(cmd,
> + &instance->rsp_list,
> + head) {
> + if (cmd->id == reply->trans_id)
> + break;
> + }
> + mutex_unlock(&instance->lock);
> +
> + if (&cmd->head == &instance->rsp_list) {
> + //pr_debug("%s: received response %u, throw away...",
please no uncommented code
> + pr_err("%s: received response %u, throw away...",
> + __func__,
> + reply->trans_id);
> + } else if (reply_len > sizeof(cmd->msg)) {
> + pr_err("%s: reply too big (%u) %u, throw away...",
> + __func__, reply_len,
> + reply->trans_id);
> + } else {
> + memcpy(cmd->msg, reply,
> + reply_len);
> + complete(&cmd->cmplt);
> + }
> +}
> +
> +static int vc_sm_cma_vchi_videocore_io(void *arg)
> +{
> + struct sm_instance *instance = arg;
> + struct sm_cmd_rsp_blk *cmd = NULL, *cmd_tmp;
> + struct vc_sm_result_t *reply;
> + struct vchiq_header *header;
> + s32 status;
> + int svc_use = 1;
> +
> + while (1) {
> + if (svc_use)
> + vchiq_release_service(instance->vchiq_instance,
> + instance->service_handle[0]);
> + svc_use = 0;
> +
> + if (wait_for_completion_interruptible(&instance->io_cmplt))
> + continue;
> + vchiq_use_service(instance->vchiq_instance, instance->service_handle[0]);
> + svc_use = 1;
> +
> + do {
> + /*
> + * Get new command and move it to response list
> + */
> + mutex_lock(&instance->lock);
> + if (list_empty(&instance->cmd_list)) {
> + /* no more commands to process */
> + mutex_unlock(&instance->lock);
> + break;
> + }
> + cmd = list_first_entry(&instance->cmd_list,
> + struct sm_cmd_rsp_blk, head);
> + list_move(&cmd->head, &instance->rsp_list);
> + cmd->sent = 1;
> + mutex_unlock(&instance->lock);
> + /* Send the command */
> + status =
> + bcm2835_vchi_msg_queue(instance->vchiq_instance,
> + instance->service_handle[0],
> + cmd->msg, cmd->length);
> + if (status) {
> + pr_err("%s: failed to queue message (%d)",
> + __func__, status);
> + }
> +
> + /* If no reply is needed then we're done */
> + if (!cmd->wait) {
> + mutex_lock(&instance->lock);
> + list_del(&cmd->head);
> + mutex_unlock(&instance->lock);
> + vc_vchi_cmd_delete(instance, cmd);
> + continue;
> + }
> +
> + if (status) {
> + complete(&cmd->cmplt);
> + continue;
> + }
> +
> + } while (1);
> +
> + while ((header = vchiq_msg_hold(instance->vchiq_instance,
> + instance->service_handle[0]))) {
> + reply = (struct vc_sm_result_t *)header->data;
> + if (reply->trans_id & 0x80000000) {
> + /* Async event or cmd from the VPU */
> + if (instance->vpu_event)
> + instance->vpu_event(instance, reply,
> + header->size);
> + } else {
> + vc_sm_cma_vchi_rx_ack(instance, cmd, reply,
> + header->size);
> + }
> +
> + vchiq_release_message(instance->vchiq_instance,
> + instance->service_handle[0],
> + header);
> + }
> +
> + /* Go through the dead list and free them */
> + mutex_lock(&instance->lock);
> + list_for_each_entry_safe(cmd, cmd_tmp, &instance->dead_list,
> + head) {
> + list_del(&cmd->head);
> + vc_vchi_cmd_delete(instance, cmd);
> + }
> + mutex_unlock(&instance->lock);
> + }
> +
> + return 0;
> +}
> +
> +static int vc_sm_cma_vchi_callback(struct vchiq_instance *vchiq_instance,
> + enum vchiq_reason reason,
> + struct vchiq_header *header,
> + unsigned int handle, void *userdata)
> +{
> + struct sm_instance *instance = vchiq_get_service_userdata(vchiq_instance, handle);
> +
> + switch (reason) {
> + case VCHIQ_MESSAGE_AVAILABLE:
> + vchiq_msg_queue_push(vchiq_instance, handle, header);
> + complete(&instance->io_cmplt);
> + break;
> +
> + case VCHIQ_SERVICE_CLOSED:
> + pr_info("%s: service CLOSED!!", __func__);
> + default:
> + break;
> + }
> +
> + return 0;
> +}
> +
> +struct sm_instance *vc_sm_cma_vchi_init(struct vchiq_instance *vchiq_instance,
> + unsigned int num_connections,
> + vpu_event_cb vpu_event)
> +{
> + u32 i;
> + struct sm_instance *instance;
> + int status;
> +
> + pr_debug("%s: start", __func__);
> +
> + if (num_connections > SM_MAX_NUM_CONNECTIONS) {
> + pr_err("%s: unsupported number of connections %u (max=%u)",
> + __func__, num_connections, SM_MAX_NUM_CONNECTIONS);
> +
> + goto err_null;
> + }
> + /* Allocate memory for this instance */
> + instance = kzalloc(sizeof(*instance), GFP_KERNEL);
> +
> + /* Misc initialisations */
> + mutex_init(&instance->lock);
> + init_completion(&instance->io_cmplt);
> + INIT_LIST_HEAD(&instance->cmd_list);
> + INIT_LIST_HEAD(&instance->rsp_list);
> + INIT_LIST_HEAD(&instance->dead_list);
> + INIT_LIST_HEAD(&instance->free_list);
> + sema_init(&instance->free_sema, SM_MAX_NUM_CMD_RSP_BLKS);
> + mutex_init(&instance->free_lock);
> + for (i = 0; i < SM_MAX_NUM_CMD_RSP_BLKS; i++) {
> + init_completion(&instance->free_blk[i].cmplt);
> + list_add(&instance->free_blk[i].head, &instance->free_list);
> + }
> +
> + instance->vchiq_instance = vchiq_instance;
> +
> + /* Open the VCHI service connections */
> + instance->num_connections = num_connections;
> + for (i = 0; i < num_connections; i++) {
> + struct vchiq_service_params_kernel params = {
> + .version = VC_SM_VER,
> + .version_min = VC_SM_MIN_VER,
> + .fourcc = VCHIQ_MAKE_FOURCC('S', 'M', 'E', 'M'),
> + .callback = vc_sm_cma_vchi_callback,
> + .userdata = instance,
> + };
> +
> + status = vchiq_open_service(vchiq_instance, ¶ms,
> + &instance->service_handle[i]);
> + if (status) {
> + pr_err("%s: failed to open VCHI service (%d)",
> + __func__, status);
> +
> + goto err_close_services;
> + }
> + }
> + /* Create the thread which takes care of all io to/from videoocore. */
> + instance->io_thread = kthread_create(&vc_sm_cma_vchi_videocore_io,
> + (void *)instance, "SMIO");
I think the thread name is too general. It should better point to this
driver.
> + if (!instance->io_thread) {
> + pr_err("%s: failed to create SMIO thread", __func__);
> +
> + goto err_close_services;
> + }
> + instance->vpu_event = vpu_event;
> + set_user_nice(instance->io_thread, -10);
> + wake_up_process(instance->io_thread);
> +
> + pr_debug("%s: success - instance %p", __func__, instance);
> + return instance;
> +
> +err_close_services:
> + for (i = 0; i < instance->num_connections; i++) {
> + if (instance->service_handle[i])
> + vchiq_close_service(vchiq_instance, instance->service_handle[i]);
> + }
> + kfree(instance);
> +err_null:
> + pr_debug("%s: FAILED", __func__);
> + return NULL;
> +}
> +
> +int vc_sm_cma_vchi_stop(struct vchiq_instance *vchiq_instance, struct sm_instance **handle)
> +{
> + struct sm_instance *instance;
> + u32 i;
> +
> + if (!handle) {
> + pr_err("%s: invalid pointer to handle %p", __func__, handle);
> + goto lock;
> + }
> +
> + if (!*handle) {
> + pr_err("%s: invalid handle %p", __func__, *handle);
> + goto lock;
> + }
> +
> + instance = *handle;
> +
> + /* Close all VCHI service connections */
> + for (i = 0; i < instance->num_connections; i++) {
> + vchiq_use_service(vchiq_instance, instance->service_handle[i]);
> + vchiq_close_service(vchiq_instance, instance->service_handle[i]);
> + }
> +
> + kfree(instance);
> +
> + *handle = NULL;
> + return 0;
> +
> +lock:
> + return -EINVAL;
> +}
> +
> +static int vc_sm_cma_vchi_send_msg(struct sm_instance *handle,
> + enum vc_sm_msg_type msg_id, void *msg,
> + u32 msg_size, void *result, u32 result_size,
> + u32 *cur_trans_id, u8 wait_reply)
> +{
> + int status = 0;
> + struct sm_instance *instance = handle;
> + struct sm_cmd_rsp_blk *cmd_blk;
> +
> + if (!handle) {
> + pr_err("%s: invalid handle", __func__);
> + return -EINVAL;
> + }
> + if (!msg) {
> + pr_err("%s: invalid msg pointer", __func__);
> + return -EINVAL;
> + }
> +
> + cmd_blk =
> + vc_vchi_cmd_create(instance, msg_id, msg, msg_size, wait_reply);
> + if (!cmd_blk) {
> + pr_err("[%s]: failed to allocate global tracking resource",
> + __func__);
> + return -ENOMEM;
> + }
> +
> + if (cur_trans_id)
> + *cur_trans_id = cmd_blk->id;
> +
> + mutex_lock(&instance->lock);
> + list_add_tail(&cmd_blk->head, &instance->cmd_list);
> + mutex_unlock(&instance->lock);
> + complete(&instance->io_cmplt);
> +
> + if (!wait_reply)
> + /* We're done */
> + return 0;
> +
> + /* Wait for the response */
> + if (wait_for_completion_interruptible(&cmd_blk->cmplt)) {
> + mutex_lock(&instance->lock);
> + if (!cmd_blk->sent) {
> + list_del(&cmd_blk->head);
> + mutex_unlock(&instance->lock);
> + vc_vchi_cmd_delete(instance, cmd_blk);
> + return -ENXIO;
> + }
> +
> + list_move(&cmd_blk->head, &instance->dead_list);
> + mutex_unlock(&instance->lock);
> + complete(&instance->io_cmplt);
> + return -EINTR; /* We're done */
> + }
> +
> + if (result && result_size) {
> + memcpy(result, cmd_blk->msg, result_size);
> + } else {
> + struct vc_sm_result_t *res =
> + (struct vc_sm_result_t *)cmd_blk->msg;
> + status = (res->success == 0) ? 0 : -ENXIO;
> + }
> +
> + mutex_lock(&instance->lock);
> + list_del(&cmd_blk->head);
> + mutex_unlock(&instance->lock);
> + vc_vchi_cmd_delete(instance, cmd_blk);
> + return status;
> +}
> +
> +int vc_sm_cma_vchi_free(struct sm_instance *handle, struct vc_sm_free_t *msg,
> + u32 *cur_trans_id)
> +{
> + return vc_sm_cma_vchi_send_msg(handle, VC_SM_MSG_TYPE_FREE,
> + msg, sizeof(*msg), 0, 0, cur_trans_id, 0);
> +}
> +
> +int vc_sm_cma_vchi_import(struct sm_instance *handle, struct vc_sm_import *msg,
> + struct vc_sm_import_result *result, u32 *cur_trans_id)
> +{
> + return vc_sm_cma_vchi_send_msg(handle, VC_SM_MSG_TYPE_IMPORT,
> + msg, sizeof(*msg), result, sizeof(*result),
> + cur_trans_id, 1);
> +}
> +
> +int vc_sm_cma_vchi_client_version(struct sm_instance *handle,
> + struct vc_sm_version *msg,
> + struct vc_sm_result_t *result,
> + u32 *cur_trans_id)
> +{
> + return vc_sm_cma_vchi_send_msg(handle, VC_SM_MSG_TYPE_CLIENT_VERSION,
> + //msg, sizeof(*msg), result, sizeof(*result),
> + //cur_trans_id, 1);
> + msg, sizeof(*msg), NULL, 0,
> + cur_trans_id, 0);
> +}
> +
> +int vc_sm_vchi_client_vc_mem_req_reply(struct sm_instance *handle,
> + struct vc_sm_vc_mem_request_result *msg,
> + uint32_t *cur_trans_id)
> +{
> + return vc_sm_cma_vchi_send_msg(handle,
> + VC_SM_MSG_TYPE_VC_MEM_REQUEST_REPLY,
> + msg, sizeof(*msg), 0, 0, cur_trans_id,
> + 0);
> +}
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
> new file mode 100644
> index 000000000000..5dd23ad408fc
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
> @@ -0,0 +1,63 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +/*
> + * VideoCore Shared Memory CMA allocator
> + *
> + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> + * Copyright 2011-2012 Broadcom Corporation. All rights reserved.
> + *
> + * Based on vmcs_sm driver from Broadcom Corporation.
> + *
> + */
> +
> +#ifndef __VC_SM_CMA_VCHI_H__INCLUDED__
> +#define __VC_SM_CMA_VCHI_H__INCLUDED__
I don't think this _INCLUDED_ part necessary.
> +
> +#include "../include/linux/raspberrypi/vchiq.h"
> +
> +#include "vc_sm_defs.h"
> +
> +/*
> + * Forward declare.
> + */
> +struct sm_instance;
> +
> +typedef void (*vpu_event_cb)(struct sm_instance *instance,
> + struct vc_sm_result_t *reply, int reply_len);
> +
> +/*
> + * Initialize the shared memory service, opens up vchi connection to talk to it.
> + */
> +struct sm_instance *vc_sm_cma_vchi_init(struct vchiq_instance *vchi_instance,
> + unsigned int num_connections,
> + vpu_event_cb vpu_event);
> +
> +/*
> + * Terminates the shared memory service.
> + */
> +int vc_sm_cma_vchi_stop(struct vchiq_instance *vchi_instance, struct sm_instance **handle);
> +
> +/*
> + * Ask the shared memory service to free up some memory that was previously
> + * allocated by the vc_sm_cma_vchi_alloc function call.
> + */
> +int vc_sm_cma_vchi_free(struct sm_instance *handle, struct vc_sm_free_t *msg,
> + u32 *cur_trans_id);
> +
> +/*
> + * Import a contiguous block of memory and wrap it in a GPU MEM_HANDLE_T.
> + */
> +int vc_sm_cma_vchi_import(struct sm_instance *handle, struct vc_sm_import *msg,
> + struct vc_sm_import_result *result,
> + u32 *cur_trans_id);
> +
> +int vc_sm_cma_vchi_client_version(struct sm_instance *handle,
> + struct vc_sm_version *msg,
> + struct vc_sm_result_t *result,
> + u32 *cur_trans_id);
> +
> +int vc_sm_vchi_client_vc_mem_req_reply(struct sm_instance *handle,
> + struct vc_sm_vc_mem_request_result *msg,
> + uint32_t *cur_trans_id);
> +
> +#endif /* __VC_SM_CMA_VCHI_H__INCLUDED__ */
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
> new file mode 100644
> index 000000000000..ad4a3ec194d3
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
> @@ -0,0 +1,187 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
empty line
> +/*
> + * VideoCore Shared Memory CMA allocator
> + *
> + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> + *
> + * Based on vc_sm_defs.h from the vmcs_sm driver Copyright Broadcom Corporation.
> + * All IPC messages are copied across to this file, even if the vc-sm-cma
> + * driver is not currently using them.
> + *
> + ****************************************************************************
> + */
> +
> +#ifndef __VC_SM_DEFS_H__INCLUDED__
> +#define __VC_SM_DEFS_H__INCLUDED__
> +
> +/* Maximum message length */
> +#define VC_SM_MAX_MSG_LEN (sizeof(union vc_sm_msg_union_t) + \
> + sizeof(struct vc_sm_msg_hdr_t))
> +#define VC_SM_MAX_RSP_LEN (sizeof(union vc_sm_msg_union_t))
> +
> +/* Resource name maximum size */
> +#define VC_SM_RESOURCE_NAME 32
> +
> +/*
> + * Version to be reported to the VPU
> + * VPU assumes 0 (aka 1) which does not require the released callback, nor
> + * expect the client to handle VC_MEM_REQUESTS.
> + * Version 2 requires the released callback, and must support VC_MEM_REQUESTS.
> + */
> +#define VC_SM_PROTOCOL_VERSION 2
> +
> +enum vc_sm_msg_type {
> + /* Message types supported for HOST->VC direction */
> +
> + /* Allocate shared memory block */
> + VC_SM_MSG_TYPE_ALLOC,
> + /* Lock allocated shared memory block */
> + VC_SM_MSG_TYPE_LOCK,
> + /* Unlock allocated shared memory block */
> + VC_SM_MSG_TYPE_UNLOCK,
> + /* Unlock allocated shared memory block, do not answer command */
> + VC_SM_MSG_TYPE_UNLOCK_NOANS,
> + /* Free shared memory block */
> + VC_SM_MSG_TYPE_FREE,
> + /* Resize a shared memory block */
> + VC_SM_MSG_TYPE_RESIZE,
> + /* Walk the allocated shared memory block(s) */
> + VC_SM_MSG_TYPE_WALK_ALLOC,
> +
> + /* A previously applied action will need to be reverted */
> + VC_SM_MSG_TYPE_ACTION_CLEAN,
> +
> + /*
> + * Import a physical address and wrap into a MEM_HANDLE_T.
> + * Release with VC_SM_MSG_TYPE_FREE.
> + */
> + VC_SM_MSG_TYPE_IMPORT,
> + /*
> + *Tells VC the protocol version supported by this client.
> + * 2 supports the async/cmd messages from the VPU for final release
> + * of memory, and for VC allocations.
> + */
> + VC_SM_MSG_TYPE_CLIENT_VERSION,
> + /* Response to VC request for memory */
> + VC_SM_MSG_TYPE_VC_MEM_REQUEST_REPLY,
> +
> + /*
> + * Asynchronous/cmd messages supported for VC->HOST direction.
> + * Signalled by setting the top bit in vc_sm_result_t trans_id.
> + */
> +
> + /*
> + * VC has finished with an imported memory allocation.
> + * Release any Linux reference counts on the underlying block.
> + */
> + VC_SM_MSG_TYPE_RELEASED,
> + /* VC request for memory */
> + VC_SM_MSG_TYPE_VC_MEM_REQUEST,
> +
> + VC_SM_MSG_TYPE_MAX
> +};
> +
> +/* Type of memory to be allocated */
> +enum vc_sm_alloc_type_t {
> + VC_SM_ALLOC_CACHED,
> + VC_SM_ALLOC_NON_CACHED,
> +};
> +
> +/* Message header for all messages in HOST->VC direction */
> +struct vc_sm_msg_hdr_t {
> + u32 type;
> + u32 trans_id;
> + u8 body[0];
> +
> +};
> +
> +/* Request to free a previously allocated memory (HOST->VC) */
> +struct vc_sm_free_t {
> + /* Resource handle (returned from alloc) */
> + u32 res_handle;
> + /* Resource buffer (returned from alloc) */
> + u32 res_mem;
> +
> +};
> +
> +/* Generic result for a request (VC->HOST) */
> +struct vc_sm_result_t {
> + /* Transaction identifier */
> + u32 trans_id;
> +
> + s32 success;
> +
> +};
> +
> +/* Request to import memory (HOST->VC) */
> +struct vc_sm_import {
> + /* type of memory to allocate */
> + enum vc_sm_alloc_type_t type;
> + /* pointer to the VC (ie physical) address of the allocated memory */
> + u32 addr;
> + /* size of buffer */
> + u32 size;
> + /* opaque handle returned in RELEASED messages */
> + u32 kernel_id;
> + /* Allocator identifier */
> + u32 allocator;
> + /* resource name (for easier tracking on vc side) */
> + char name[VC_SM_RESOURCE_NAME];
> +};
> +
> +/* Result of a requested memory import (VC->HOST) */
> +struct vc_sm_import_result {
> + /* Transaction identifier */
> + u32 trans_id;
> +
> + /* Resource handle */
> + u32 res_handle;
> +};
> +
> +/* Notification that VC has finished with an allocation (VC->HOST) */
> +struct vc_sm_released {
> + /* cmd type / trans_id */
> + u32 cmd;
> +
> + /* pointer to the VC (ie physical) address of the allocated memory */
> + u32 addr;
> + /* size of buffer */
> + u32 size;
> + /* opaque handle returned in RELEASED messages */
> + u32 kernel_id;
> + u32 vc_handle;
> +};
> +
> +/*
> + * Client informing VC as to the protocol version it supports.
> + * >=2 requires the released callback, and supports VC asking for memory.
> + * Failure means that the firmware doesn't support this call, and therefore the
> + * client should either fail, or NOT rely on getting the released callback.
> + */
> +struct vc_sm_version {
> + u32 version;
> +};
> +
> +/* Response from the kernel to provide the VPU with some memory */
> +struct vc_sm_vc_mem_request_result {
> + /* Transaction identifier for the VPU */
> + u32 trans_id;
> + /* pointer to the physical address of the allocated memory */
> + u32 addr;
> + /* opaque handle returned in RELEASED messages */
> + u32 kernel_id;
> +};
> +
> +/* Union of ALL messages */
> +union vc_sm_msg_union_t {
> + struct vc_sm_free_t free;
> + struct vc_sm_result_t result;
> + struct vc_sm_import import;
> + struct vc_sm_import_result import_result;
> + struct vc_sm_version version;
> + struct vc_sm_released released;
> + struct vc_sm_vc_mem_request_result vc_request_result;
> +};
> +
> +#endif /* __VC_SM_DEFS_H__INCLUDED__ */
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
> new file mode 100644
> index 000000000000..988fdd967922
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
Do we actually need this or can we merge it into the other files?
I will review the rest of this series later ...
> @@ -0,0 +1,28 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +/*
> + * VideoCore Shared Memory CMA allocator
> + *
> + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> + *
> + * Based on vc_sm_defs.h from the vmcs_sm driver Copyright Broadcom Corporation.
> + *
> + */
> +
> +#ifndef __VC_SM_KNL_H__INCLUDED__
> +#define __VC_SM_KNL_H__INCLUDED__
> +
> +#if !defined(__KERNEL__)
> +#error "This interface is for kernel use only..."
> +#endif
> +
> +/* Free a previously allocated or imported shared memory handle and block. */
> +int vc_sm_cma_free(void *handle);
> +
> +/* Get an internal resource handle mapped from the external one. */
> +int vc_sm_cma_int_handle(void *handle);
> +
> +/* Import a block of memory into the GPU space. */
> +int vc_sm_cma_import_dmabuf(struct dma_buf *dmabuf, void **handle);
> +
> +#endif /* __VC_SM_KNL_H__INCLUDED__ */
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 02/15] staging: vc04_services: vchiq_arm: Register vcsm-cma driver
2023-11-09 21:02 ` [PATCH v2 02/15] staging: vc04_services: vchiq_arm: Register vcsm-cma driver Umang Jain
@ 2023-11-14 19:32 ` Stefan Wahren
0 siblings, 0 replies; 45+ messages in thread
From: Stefan Wahren @ 2023-11-14 19:32 UTC (permalink / raw)
To: Umang Jain, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter
Hi Umang,
Am 09.11.23 um 22:02 schrieb Umang Jain:
> Register the vcsm-cma driver with the vchiq_bus_type.
please squash into patch #1
>
> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
> ---
> drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> index 9fb8f657cc78..5a9073480dd3 100644
> --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> @@ -70,6 +70,7 @@ struct vchiq_state g_state;
> */
> static struct vchiq_device *bcm2835_audio;
> static struct vchiq_device *bcm2835_camera;
> +static struct vchiq_device *vcsm_cma;
>
> struct vchiq_drvdata {
> const unsigned int cache_line_size;
> @@ -1840,6 +1841,7 @@ static int vchiq_probe(struct platform_device *pdev)
> goto error_exit;
> }
>
> + vcsm_cma = vchiq_device_register(&pdev->dev, "vcsm-cma");
> bcm2835_audio = vchiq_device_register(&pdev->dev, "bcm2835-audio");
> bcm2835_camera = vchiq_device_register(&pdev->dev, "bcm2835-camera");
>
> @@ -1855,6 +1857,7 @@ static void vchiq_remove(struct platform_device *pdev)
> {
> vchiq_device_unregister(bcm2835_audio);
> vchiq_device_unregister(bcm2835_camera);
> + vchiq_device_unregister(vcsm_cma);
> vchiq_debugfs_deinit();
> vchiq_deregister_chrdev();
> }
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 03/15] media: videobuf2: Allow exporting of a struct dmabuf
2023-11-09 21:02 ` [PATCH v2 03/15] media: videobuf2: Allow exporting of a struct dmabuf Umang Jain
@ 2023-11-14 19:41 ` Stefan Wahren
2023-11-15 19:31 ` Laurent Pinchart
0 siblings, 1 reply; 45+ messages in thread
From: Stefan Wahren @ 2023-11-14 19:41 UTC (permalink / raw)
To: Umang Jain, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Dave Stevenson
Hi Umang,
Am 09.11.23 um 22:02 schrieb Umang Jain:
> From: Dave Stevenson <dave.stevenson@raspberrypi.org>
>
> videobuf2 only allowed exporting a dmabuf as a file descriptor,
> but there are instances where having the struct dma_buf is
> useful within the kernel.
>
> Split the current implementation into two, one step which
> exports a struct dma_buf, and the second which converts that
> into an fd.
>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
> ---
...
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index 4b6a9d2ea372..cba4e495f6a2 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -925,6 +925,21 @@ int vb2_core_streamon(struct vb2_queue *q, unsigned int type);
> */
> int vb2_core_streamoff(struct vb2_queue *q, unsigned int type);
>
> +/**
> + * vb2_core_expbuf_dmabuf() - Export a buffer as a dma_buf structure
> + * @q: videobuf2 queue
> + * @type: buffer type
> + * @index: id number of the buffer
> + * @plane: index of the plane to be exported, 0 for single plane queues
> + * @flags: flags for newly created file, currently only O_CLOEXEC is
> + * supported, refer to manual of open syscall for more details
i think "newly created file" could be confusing here
> + *
> + * Return: Returns the dmabuf pointer
> + */
> +struct dma_buf *vb2_core_expbuf_dmabuf(struct vb2_queue *q, unsigned int type,
> + unsigned int index, unsigned int plane,
> + unsigned int flags);
> +
> /**
> * vb2_core_expbuf() - Export a buffer as a file descriptor.
> * @q: pointer to &struct vb2_queue with videobuf2 queue.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 05/15] staging: mmal-vchiq: Use vc-sm-cma to support zero copy
2023-11-09 21:02 ` [PATCH v2 05/15] staging: mmal-vchiq: Use vc-sm-cma to support zero copy Umang Jain
@ 2023-11-15 19:03 ` Stefan Wahren
2023-11-15 19:27 ` Laurent Pinchart
0 siblings, 1 reply; 45+ messages in thread
From: Stefan Wahren @ 2023-11-15 19:03 UTC (permalink / raw)
To: Umang Jain, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter
Hi Umang,
Am 09.11.23 um 22:02 schrieb Umang Jain:
> From: Dave Stevenson <dave.stevenson@raspberrypi.com>
>
> With the vc-sm-cma driver we can support zero copy of buffers between
> the kernel and VPU. Add this support to mmal-vchiq.
>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
> ---
> .../staging/vc04_services/vchiq-mmal/Kconfig | 1 +
> .../vc04_services/vchiq-mmal/mmal-common.h | 4 +
> .../vc04_services/vchiq-mmal/mmal-vchiq.c | 84 +++++++++++++++++--
> .../vc04_services/vchiq-mmal/mmal-vchiq.h | 1 +
> 4 files changed, 83 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/staging/vc04_services/vchiq-mmal/Kconfig b/drivers/staging/vc04_services/vchiq-mmal/Kconfig
> index c99525a0bb45..a7c1a7bf516e 100644
> --- a/drivers/staging/vc04_services/vchiq-mmal/Kconfig
> +++ b/drivers/staging/vc04_services/vchiq-mmal/Kconfig
> @@ -1,6 +1,7 @@
> config BCM2835_VCHIQ_MMAL
> tristate "BCM2835 MMAL VCHIQ service"
> depends on BCM2835_VCHIQ
> + select BCM_VC_SM_CMA
i think we need more explanation in the commit message of the relation
between these both modules.
On the one side BCM_VC_SM_CMA should be a driver, but it's not a driver
for a specific hardware. It looks like more an extension of VCHIQ MMAL
or does other (maybe not yet imported) vc04 driver make also use of this.
My question is: is BCM_VC_SM_CMA a real member on the VCHIQ bus and why?
Best regards
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 05/15] staging: mmal-vchiq: Use vc-sm-cma to support zero copy
2023-11-15 19:03 ` Stefan Wahren
@ 2023-11-15 19:27 ` Laurent Pinchart
0 siblings, 0 replies; 45+ messages in thread
From: Laurent Pinchart @ 2023-11-15 19:27 UTC (permalink / raw)
To: Stefan Wahren
Cc: Umang Jain, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging,
Dave Stevenson, Kieran Bingham, Ricardo B . Marliere,
Dan Carpenter
On Wed, Nov 15, 2023 at 08:03:47PM +0100, Stefan Wahren wrote:
> Hi Umang,
>
> Am 09.11.23 um 22:02 schrieb Umang Jain:
> > From: Dave Stevenson <dave.stevenson@raspberrypi.com>
> >
> > With the vc-sm-cma driver we can support zero copy of buffers between
> > the kernel and VPU. Add this support to mmal-vchiq.
> >
> > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
> > ---
> > .../staging/vc04_services/vchiq-mmal/Kconfig | 1 +
> > .../vc04_services/vchiq-mmal/mmal-common.h | 4 +
> > .../vc04_services/vchiq-mmal/mmal-vchiq.c | 84 +++++++++++++++++--
> > .../vc04_services/vchiq-mmal/mmal-vchiq.h | 1 +
> > 4 files changed, 83 insertions(+), 7 deletions(-)
> >
> > diff --git a/drivers/staging/vc04_services/vchiq-mmal/Kconfig b/drivers/staging/vc04_services/vchiq-mmal/Kconfig
> > index c99525a0bb45..a7c1a7bf516e 100644
> > --- a/drivers/staging/vc04_services/vchiq-mmal/Kconfig
> > +++ b/drivers/staging/vc04_services/vchiq-mmal/Kconfig
> > @@ -1,6 +1,7 @@
> > config BCM2835_VCHIQ_MMAL
> > tristate "BCM2835 MMAL VCHIQ service"
> > depends on BCM2835_VCHIQ
> > + select BCM_VC_SM_CMA
>
> i think we need more explanation in the commit message of the relation
> between these both modules.
>
> On the one side BCM_VC_SM_CMA should be a driver, but it's not a driver
> for a specific hardware. It looks like more an extension of VCHIQ MMAL
> or does other (maybe not yet imported) vc04 driver make also use of this.
>
> My question is: is BCM_VC_SM_CMA a real member on the VCHIQ bus and why?
The vchiq_device pointer passed to the vc_sm_cma probe function is used
in the driver for the sole purpose of accessing the underlying struct
device, which, beside being used for dev_*() log calls, is just passed
to dma_buf_attach(). It does indeed appear to be more of a service than
a device.
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 03/15] media: videobuf2: Allow exporting of a struct dmabuf
2023-11-14 19:41 ` Stefan Wahren
@ 2023-11-15 19:31 ` Laurent Pinchart
2023-11-15 21:00 ` Stefan Wahren
0 siblings, 1 reply; 45+ messages in thread
From: Laurent Pinchart @ 2023-11-15 19:31 UTC (permalink / raw)
To: Stefan Wahren
Cc: Umang Jain, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging,
Dave Stevenson, Kieran Bingham, Ricardo B . Marliere,
Dan Carpenter, Dave Stevenson
On Tue, Nov 14, 2023 at 08:41:11PM +0100, Stefan Wahren wrote:
> Am 09.11.23 um 22:02 schrieb Umang Jain:
> > From: Dave Stevenson <dave.stevenson@raspberrypi.org>
> >
> > videobuf2 only allowed exporting a dmabuf as a file descriptor,
> > but there are instances where having the struct dma_buf is
> > useful within the kernel.
> >
> > Split the current implementation into two, one step which
> > exports a struct dma_buf, and the second which converts that
> > into an fd.
> >
> > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
> > Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
> > ---
>
> ...
>
> > diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> > index 4b6a9d2ea372..cba4e495f6a2 100644
> > --- a/include/media/videobuf2-core.h
> > +++ b/include/media/videobuf2-core.h
> > @@ -925,6 +925,21 @@ int vb2_core_streamon(struct vb2_queue *q, unsigned int type);
> > */
> > int vb2_core_streamoff(struct vb2_queue *q, unsigned int type);
> >
> > +/**
> > + * vb2_core_expbuf_dmabuf() - Export a buffer as a dma_buf structure
> > + * @q: videobuf2 queue
> > + * @type: buffer type
> > + * @index: id number of the buffer
> > + * @plane: index of the plane to be exported, 0 for single plane queues
> > + * @flags: flags for newly created file, currently only O_CLOEXEC is
> > + * supported, refer to manual of open syscall for more details
>
> i think "newly created file" could be confusing here
Would "newly created dmabuf file handle" be clearer and work for
everybody ?
> > + *
> > + * Return: Returns the dmabuf pointer
> > + */
> > +struct dma_buf *vb2_core_expbuf_dmabuf(struct vb2_queue *q, unsigned int type,
> > + unsigned int index, unsigned int plane,
> > + unsigned int flags);
> > +
> > /**
> > * vb2_core_expbuf() - Export a buffer as a file descriptor.
> > * @q: pointer to &struct vb2_queue with videobuf2 queue.
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 01/15] staging: vc04_services: Add new vc-sm-cma driver
2023-11-13 17:56 ` Stefan Wahren
@ 2023-11-15 19:56 ` Laurent Pinchart
2023-11-15 20:48 ` Stefan Wahren
0 siblings, 1 reply; 45+ messages in thread
From: Laurent Pinchart @ 2023-11-15 19:56 UTC (permalink / raw)
To: Stefan Wahren
Cc: Umang Jain, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging,
Dave Stevenson, Kieran Bingham, Ricardo B . Marliere,
Dan Carpenter
On Mon, Nov 13, 2023 at 06:56:06PM +0100, Stefan Wahren wrote:
> Hi Umang,
>
> Am 09.11.23 um 22:02 schrieb Umang Jain:
> > From: Dave Stevenson <dave.stevenson@raspberrypi.com>
> >
> > Add Broadcom VideoCore Shared Memory support.
> >
> > This new driver allows contiguous memory blocks to be imported
> > into the VideoCore VPU memory map, and manages the lifetime of
> > those objects, only releasing the source dmabuf once the VPU has
> > confirmed it has finished with it.
> >
> > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
> > ---
> > drivers/staging/vc04_services/Kconfig | 2 +
> > drivers/staging/vc04_services/Makefile | 2 +-
> > .../staging/vc04_services/vc-sm-cma/Kconfig | 10 +
> > .../staging/vc04_services/vc-sm-cma/Makefile | 4 +
> > .../staging/vc04_services/vc-sm-cma/vc_sm.c | 817 ++++++++++++++++++
> > .../staging/vc04_services/vc-sm-cma/vc_sm.h | 54 ++
> > .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c | 507 +++++++++++
> > .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h | 63 ++
> > .../vc04_services/vc-sm-cma/vc_sm_defs.h | 187 ++++
> > .../vc04_services/vc-sm-cma/vc_sm_knl.h | 28 +
> > 10 files changed, 1673 insertions(+), 1 deletion(-)
> > create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
> > create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
> > create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> > create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> > create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> > create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
> > create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
> > create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
[snip]
> > diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> > new file mode 100644
> > index 000000000000..5bbdb3a8468b
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> > @@ -0,0 +1,817 @@
[snip]
> > +/* ---- Private Variables ----------------------------------------------- */
> > +
> > +static struct sm_state_t *sm_state;
> > +static int sm_inited;
>
> In case the driver cannot be loaded again, a comment about this should
> be here.
Better, could we remove global variables ?
[snip]
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support
2023-11-09 21:02 [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Umang Jain
` (15 preceding siblings ...)
2023-11-10 10:32 ` [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Stefan Wahren
@ 2023-11-15 19:59 ` Laurent Pinchart
2023-11-15 20:57 ` Stefan Wahren
16 siblings, 1 reply; 45+ messages in thread
From: Laurent Pinchart @ 2023-11-15 19:59 UTC (permalink / raw)
To: Umang Jain
Cc: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging, Dave Stevenson, Kieran Bingham,
Ricardo B . Marliere, Dan Carpenter, Stefan Wahren
Hello,
On Thu, Nov 09, 2023 at 04:02:52PM -0500, Umang Jain wrote:
> This series aims to upport bcm2835-isp from the RPi kernel.
> It is developed on top of staging-next which comprises many
> VC04 changes for it's de-staging. Hence, the merge of this
> driver is targeted when VC04 is de-staged completely (which I
> have been pushing), but it can be helped getting reviewed meanwhile.
> Hence, the reason for posting the series.
Related question, what do people think about dropping the legacy
firmware-based bcm2385-camera driver once this gets merged ?
firmware-based camera operation is deprecated by Raspberry Pi, and
doesn't work on the Pi 5.
> Patch (01-02)/15 adds a new driver named vc-sm-cma to handle memory sharing
> with the VC4 VPU.
>
> Patch 03/15 adds a small extension to videobuf2 to allow exporting as a
> dma_buf instead of a file-descriptor.
>
> Patch (04-05)/15 adds a couple of improvements/support for
> bcm2835-isp(event callback and zero-copy) to vchiq-mmal.
>
> Patch (06-10)/15 adds the core bcm2835-isp driver along with headers
> and format defintions. 09/15 is a standalone patch which can be merged
> independently I believe.
>
> Patch (11-12)/15 deals with the colorspace support.
>
> Patch 13/15 allows multiple instances of the ISP.
>
> Patch 14/15 adds a admin-guide document on bcm2835-isp.
>
> Patch 15/15 deals with driver registeration.
>
> Dave Stevenson (6):
> staging: vc04_services: Add new vc-sm-cma driver
> media: videobuf2: Allow exporting of a struct dmabuf
> staging: mmal-vchiq: Add support for event callbacks
> staging: mmal-vchiq: Use vc-sm-cma to support zero copy
> staging: mmal_vchiq: Add image formats to be used by bcm2835-isp
> uapi: bcm2835-isp: Add bcm2835-isp uapi header file
>
> David Plowman (2):
> vc04_services: bcm2835-isp: Allow formats with different colour spaces
> vc04_services: bcm2835-isp: Permit all sRGB colour spaces on ISP
> outputs
>
> Naushir Patuck (4):
> media: uapi: v4l2-core: Add ISP statistics output V4L2 fourcc type
> staging: vc04_services: bcm2835-isp: Add a more complex ISP processing
> component
> staging: vc04_services: bcm2835_isp: Allow multiple users
> docs: admin-guide: media: bcm2835-isp: Add documentation for
> bcm2835-isp
>
> Umang Jain (3):
> staging: vc04_services: vchiq_arm: Register vcsm-cma driver
> staging: vc04_services: Add helpers for vchiq driver data
> staging: vc04_services: vchiq: Register bcm2835-isp
>
> .../admin-guide/media/bcm2835-isp.rst | 127 ++
> .../userspace-api/media/drivers/index.rst | 1 +
> .../userspace-api/media/v4l/meta-formats.rst | 1 +
> .../v4l/pixfmt-meta-bcm2835-isp-stats.rst | 32 +
> MAINTAINERS | 9 +
> .../media/common/videobuf2/videobuf2-core.c | 36 +-
> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> drivers/staging/vc04_services/Kconfig | 4 +
> drivers/staging/vc04_services/Makefile | 3 +-
> .../staging/vc04_services/bcm2835-isp/Kconfig | 14 +
> .../vc04_services/bcm2835-isp/Makefile | 4 +
> .../bcm2835-isp/bcm2835-isp-ctrls.h | 72 +
> .../bcm2835-isp/bcm2835-isp-fmts.h | 559 +++++
> .../bcm2835-isp/bcm2835-v4l2-isp.c | 1822 +++++++++++++++++
> .../interface/vchiq_arm/vchiq_arm.c | 6 +
> .../interface/vchiq_arm/vchiq_bus.h | 10 +
> .../staging/vc04_services/vc-sm-cma/Kconfig | 10 +
> .../staging/vc04_services/vc-sm-cma/Makefile | 4 +
> .../staging/vc04_services/vc-sm-cma/vc_sm.c | 817 ++++++++
> .../staging/vc04_services/vc-sm-cma/vc_sm.h | 54 +
> .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c | 507 +++++
> .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h | 63 +
> .../vc04_services/vc-sm-cma/vc_sm_defs.h | 187 ++
> .../vc04_services/vc-sm-cma/vc_sm_knl.h | 28 +
> .../staging/vc04_services/vchiq-mmal/Kconfig | 1 +
> .../vc04_services/vchiq-mmal/mmal-common.h | 5 +
> .../vc04_services/vchiq-mmal/mmal-encodings.h | 66 +
> .../vc04_services/vchiq-mmal/mmal-msg.h | 35 +
> .../vchiq-mmal/mmal-parameters.h | 165 +-
> .../vc04_services/vchiq-mmal/mmal-vchiq.c | 253 ++-
> .../vc04_services/vchiq-mmal/mmal-vchiq.h | 5 +
> include/media/videobuf2-core.h | 15 +
> include/uapi/linux/bcm2835-isp.h | 347 ++++
> include/uapi/linux/v4l2-controls.h | 5 +
> include/uapi/linux/videodev2.h | 1 +
> 35 files changed, 5235 insertions(+), 34 deletions(-)
> create mode 100644 Documentation/admin-guide/media/bcm2835-isp.rst
> create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-ctrls.h
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
> create mode 100644 include/uapi/linux/bcm2835-isp.h
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 01/15] staging: vc04_services: Add new vc-sm-cma driver
2023-11-15 19:56 ` Laurent Pinchart
@ 2023-11-15 20:48 ` Stefan Wahren
0 siblings, 0 replies; 45+ messages in thread
From: Stefan Wahren @ 2023-11-15 20:48 UTC (permalink / raw)
To: Laurent Pinchart
Cc: Umang Jain, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging,
Dave Stevenson, Kieran Bingham, Ricardo B . Marliere,
Dan Carpenter
>
>>> +/* ---- Private Variables ----------------------------------------------- */
>>> +
>>> +static struct sm_state_t *sm_state;
>>> +static int sm_inited;
>>
>> In case the driver cannot be loaded again, a comment about this should
>> be here.
>
> Better, could we remove global variables ?
Yes
>
> [snip]
>
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support
2023-11-15 19:59 ` Laurent Pinchart
@ 2023-11-15 20:57 ` Stefan Wahren
2023-11-16 0:50 ` Laurent Pinchart
0 siblings, 1 reply; 45+ messages in thread
From: Stefan Wahren @ 2023-11-15 20:57 UTC (permalink / raw)
To: Laurent Pinchart, Umang Jain, Ivan T. Ivanov, Peter Robinson
Cc: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging, Dave Stevenson, Kieran Bingham,
Ricardo B . Marliere, Dan Carpenter
Hi Laurent,
[add Ivan & Peter]
Am 15.11.23 um 20:59 schrieb Laurent Pinchart:
> Hello,
>
> On Thu, Nov 09, 2023 at 04:02:52PM -0500, Umang Jain wrote:
>> This series aims to upport bcm2835-isp from the RPi kernel.
>> It is developed on top of staging-next which comprises many
>> VC04 changes for it's de-staging. Hence, the merge of this
>> driver is targeted when VC04 is de-staged completely (which I
>> have been pushing), but it can be helped getting reviewed meanwhile.
>> Hence, the reason for posting the series.
>
> Related question, what do people think about dropping the legacy
> firmware-based bcm2385-camera driver once this gets merged ?
> firmware-based camera operation is deprecated by Raspberry Pi, and
> doesn't work on the Pi 5
i don't remember exactly, but wasn't the bcm2835-camera required for Pi
Camera V1.3?
At the end cannot speak for the users. AFAIK OpenSuSE and Fedora use the
driver.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 03/15] media: videobuf2: Allow exporting of a struct dmabuf
2023-11-15 19:31 ` Laurent Pinchart
@ 2023-11-15 21:00 ` Stefan Wahren
0 siblings, 0 replies; 45+ messages in thread
From: Stefan Wahren @ 2023-11-15 21:00 UTC (permalink / raw)
To: Laurent Pinchart
Cc: Umang Jain, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging,
Dave Stevenson, Kieran Bingham, Ricardo B . Marliere,
Dan Carpenter, Dave Stevenson
Am 15.11.23 um 20:31 schrieb Laurent Pinchart:
> On Tue, Nov 14, 2023 at 08:41:11PM +0100, Stefan Wahren wrote:
>> Am 09.11.23 um 22:02 schrieb Umang Jain:
>>> From: Dave Stevenson <dave.stevenson@raspberrypi.org>
>>>
>>> videobuf2 only allowed exporting a dmabuf as a file descriptor,
>>> but there are instances where having the struct dma_buf is
>>> useful within the kernel.
>>>
>>> Split the current implementation into two, one step which
>>> exports a struct dma_buf, and the second which converts that
>>> into an fd.
>>>
>>> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
>>> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
>>> ---
>> ...
>>
>>> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
>>> index 4b6a9d2ea372..cba4e495f6a2 100644
>>> --- a/include/media/videobuf2-core.h
>>> +++ b/include/media/videobuf2-core.h
>>> @@ -925,6 +925,21 @@ int vb2_core_streamon(struct vb2_queue *q, unsigned int type);
>>> */
>>> int vb2_core_streamoff(struct vb2_queue *q, unsigned int type);
>>>
>>> +/**
>>> + * vb2_core_expbuf_dmabuf() - Export a buffer as a dma_buf structure
>>> + * @q: videobuf2 queue
>>> + * @type: buffer type
>>> + * @index: id number of the buffer
>>> + * @plane: index of the plane to be exported, 0 for single plane queues
>>> + * @flags: flags for newly created file, currently only O_CLOEXEC is
>>> + * supported, refer to manual of open syscall for more details
>> i think "newly created file" could be confusing here
> Would "newly created dmabuf file handle" be clearer and work for
> everybody ?
i'm fine with it
>
>>> + *
>>> + * Return: Returns the dmabuf pointer
>>> + */
>>> +struct dma_buf *vb2_core_expbuf_dmabuf(struct vb2_queue *q, unsigned int type,
>>> + unsigned int index, unsigned int plane,
>>> + unsigned int flags);
>>> +
>>> /**
>>> * vb2_core_expbuf() - Export a buffer as a file descriptor.
>>> * @q: pointer to &struct vb2_queue with videobuf2 queue.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support
2023-11-15 20:57 ` Stefan Wahren
@ 2023-11-16 0:50 ` Laurent Pinchart
2023-11-16 10:54 ` Ivan T. Ivanov
0 siblings, 1 reply; 45+ messages in thread
From: Laurent Pinchart @ 2023-11-16 0:50 UTC (permalink / raw)
To: Stefan Wahren
Cc: Umang Jain, Ivan T. Ivanov, Peter Robinson, linux-media,
kernel-list, linux-kernel, linux-rpi-kernel, linux-arm-kernel,
linux-staging, Dave Stevenson, Kieran Bingham,
Ricardo B . Marliere, Dan Carpenter
On Wed, Nov 15, 2023 at 09:57:52PM +0100, Stefan Wahren wrote:
> Hi Laurent,
>
> [add Ivan & Peter]
>
> Am 15.11.23 um 20:59 schrieb Laurent Pinchart:
> > Hello,
> >
> > On Thu, Nov 09, 2023 at 04:02:52PM -0500, Umang Jain wrote:
> >> This series aims to upport bcm2835-isp from the RPi kernel.
> >> It is developed on top of staging-next which comprises many
> >> VC04 changes for it's de-staging. Hence, the merge of this
> >> driver is targeted when VC04 is de-staged completely (which I
> >> have been pushing), but it can be helped getting reviewed meanwhile.
> >> Hence, the reason for posting the series.
> >
> > Related question, what do people think about dropping the legacy
> > firmware-based bcm2385-camera driver once this gets merged ?
> > firmware-based camera operation is deprecated by Raspberry Pi, and
> > doesn't work on the Pi 5
>
> i don't remember exactly, but wasn't the bcm2835-camera required for Pi
> Camera V1.3?
If I'm not mistaken (Dave can correct me), the legacy camera stack works
only with the Raspberry Pi official camera v1, v2 and HQ modules.
Raspberry Pi has switched to a new camera stack based on libcamera,
which works on the Pi Zero 2, Pi 3, Pi 4 and Pi 5. This new stack
supports the same camera modules as the legacy stack, and many more. The
legacy stack doesn't work on Pi 5 at all.
> At the end cannot speak for the users. AFAIK OpenSuSE and Fedora use the
> driver.
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support
2023-11-16 0:50 ` Laurent Pinchart
@ 2023-11-16 10:54 ` Ivan T. Ivanov
2023-11-16 10:57 ` Peter Robinson
0 siblings, 1 reply; 45+ messages in thread
From: Ivan T. Ivanov @ 2023-11-16 10:54 UTC (permalink / raw)
To: Laurent Pinchart
Cc: Stefan Wahren, Umang Jain, Peter Robinson, linux-media,
kernel-list, linux-kernel, linux-rpi-kernel, linux-arm-kernel,
linux-staging, Dave Stevenson, Kieran Bingham,
Ricardo B . Marliere, Dan Carpenter, Guillaume GARDET
Hi,
On 2023-11-16 02:50, Laurent Pinchart wrote:
> On Wed, Nov 15, 2023 at 09:57:52PM +0100, Stefan Wahren wrote:
>> Hi Laurent,
>>
>> [add Ivan & Peter]
>>
>> Am 15.11.23 um 20:59 schrieb Laurent Pinchart:
>> > Hello,
>> >
>> > On Thu, Nov 09, 2023 at 04:02:52PM -0500, Umang Jain wrote:
>> >> This series aims to upport bcm2835-isp from the RPi kernel.
>> >> It is developed on top of staging-next which comprises many
>> >> VC04 changes for it's de-staging. Hence, the merge of this
>> >> driver is targeted when VC04 is de-staged completely (which I
>> >> have been pushing), but it can be helped getting reviewed meanwhile.
>> >> Hence, the reason for posting the series.
>> >
>> > Related question, what do people think about dropping the legacy
>> > firmware-based bcm2385-camera driver once this gets merged ?
>> > firmware-based camera operation is deprecated by Raspberry Pi, and
>> > doesn't work on the Pi 5
>>
>> i don't remember exactly, but wasn't the bcm2835-camera required for
>> Pi
>> Camera V1.3?
>
> If I'm not mistaken (Dave can correct me), the legacy camera stack
> works
> only with the Raspberry Pi official camera v1, v2 and HQ modules.
> Raspberry Pi has switched to a new camera stack based on libcamera,
> which works on the Pi Zero 2, Pi 3, Pi 4 and Pi 5. This new stack
> supports the same camera modules as the legacy stack, and many more.
> The
> legacy stack doesn't work on Pi 5 at all.
>
>> At the end cannot speak for the users. AFAIK OpenSuSE and Fedora use
>> the
>> driver.
Guillaume (in CC) is more authoritative about this topic, but as long we
have
smooth migration plan I am more than happy to switch to proper camera
driver.
Regards,
Ivan
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support
2023-11-16 10:54 ` Ivan T. Ivanov
@ 2023-11-16 10:57 ` Peter Robinson
0 siblings, 0 replies; 45+ messages in thread
From: Peter Robinson @ 2023-11-16 10:57 UTC (permalink / raw)
To: Ivan T. Ivanov
Cc: Laurent Pinchart, Stefan Wahren, Umang Jain, linux-media,
kernel-list, linux-kernel, linux-rpi-kernel, linux-arm-kernel,
linux-staging, Dave Stevenson, Kieran Bingham,
Ricardo B . Marliere, Dan Carpenter, Guillaume GARDET
> >> > On Thu, Nov 09, 2023 at 04:02:52PM -0500, Umang Jain wrote:
> >> >> This series aims to upport bcm2835-isp from the RPi kernel.
> >> >> It is developed on top of staging-next which comprises many
> >> >> VC04 changes for it's de-staging. Hence, the merge of this
> >> >> driver is targeted when VC04 is de-staged completely (which I
> >> >> have been pushing), but it can be helped getting reviewed meanwhile.
> >> >> Hence, the reason for posting the series.
> >> >
> >> > Related question, what do people think about dropping the legacy
> >> > firmware-based bcm2385-camera driver once this gets merged ?
> >> > firmware-based camera operation is deprecated by Raspberry Pi, and
> >> > doesn't work on the Pi 5
> >>
> >> i don't remember exactly, but wasn't the bcm2835-camera required for
> >> Pi
> >> Camera V1.3?
> >
> > If I'm not mistaken (Dave can correct me), the legacy camera stack
> > works
> > only with the Raspberry Pi official camera v1, v2 and HQ modules.
> > Raspberry Pi has switched to a new camera stack based on libcamera,
> > which works on the Pi Zero 2, Pi 3, Pi 4 and Pi 5. This new stack
> > supports the same camera modules as the legacy stack, and many more.
> > The
> > legacy stack doesn't work on Pi 5 at all.
> >
> >> At the end cannot speak for the users. AFAIK OpenSuSE and Fedora use
> >> the
> >> driver.
>
> Guillaume (in CC) is more authoritative about this topic, but as long we
> have
> smooth migration plan I am more than happy to switch to proper camera
> driver.
From a Fedora PoV I am more than happy to move to the new libcamera
based stack for the RPi camera, IMO sooner the better :)
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 07/15] media: uapi: v4l2-core: Add ISP statistics output V4L2 fourcc type
2023-11-09 21:02 ` [PATCH v2 07/15] media: uapi: v4l2-core: Add ISP statistics output V4L2 fourcc type Umang Jain
@ 2023-11-18 13:11 ` Stefan Wahren
0 siblings, 0 replies; 45+ messages in thread
From: Stefan Wahren @ 2023-11-18 13:11 UTC (permalink / raw)
To: Umang Jain, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Naushir Patuck
Hi Umang,
Am 09.11.23 um 22:02 schrieb Umang Jain:
> From: Naushir Patuck <naush@raspberrypi.com>
>
> Add V4L2_META_FMT_BCM2835_ISP_STATS V4L2 format type.
>
> This new format will be used by the bcm2835-isp device to return
> out ISP statistics for 3A.
>
> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
> ---
> .../userspace-api/media/v4l/meta-formats.rst | 1 +
> .../v4l/pixfmt-meta-bcm2835-isp-stats.rst | 32 +++++++++++++++++++
> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> include/uapi/linux/videodev2.h | 1 +
> 4 files changed, 35 insertions(+)
> create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
>
> diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst
> index 919f595576b9..534542f235bc 100644
> --- a/Documentation/userspace-api/media/v4l/meta-formats.rst
> +++ b/Documentation/userspace-api/media/v4l/meta-formats.rst
> @@ -12,6 +12,7 @@ These formats are used for the :ref:`metadata` interface only.
> .. toctree::
> :maxdepth: 1
>
> + pixfmt-meta-bcm2835-isp-stats
> metafmt-d4xx
> metafmt-intel-ipu3
> metafmt-rkisp1
> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst b/Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
> new file mode 100644
> index 000000000000..2219235e0ecb
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
> @@ -0,0 +1,32 @@
> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> +.. _v4l2-meta-fmt-bcm2835-isp-stats:
> +
> ++*****************************************
> +V4L2_META_FMT_BCM2835_ISP_STATS ('BSTA')
> +*****************************************
> +
> +BCM2835 ISP Statistics
> +
> +Description
> +===========
> +
> +The BCM2835 ISP hardware calculate image statistics for an input Bayer frame.
> +These statistics are obtained from the "bcm2835-isp0-capture3" device node
> +using the :c:type:`v4l2_meta_format` interface. They are formatted as described
> +by the :c:type:`bcm2835_isp_stats` structure below.
> +
> +.. code-block:: c
> +
> + #define DEFAULT_AWB_REGIONS_X 16
> + #define DEFAULT_AWB_REGIONS_Y 12
> +
> + #define NUM_HISTOGRAMS 2
> + #define NUM_HISTOGRAM_BINS 128
> + #define AWB_REGIONS (DEFAULT_AWB_REGIONS_X * DEFAULT_AWB_REGIONS_Y)
> + #define FLOATING_REGIONS 16
> + #define AGC_REGIONS 16
> + #define FOCUS_REGIONS 12
> +
> +.. kernel-doc:: include/uapi/linux/bcm2835-isp.h
> + :functions: bcm2835_isp_stats_hist bcm2835_isp_stats_region
> + bcm2835_isp_stats_focus bcm2835_isp_stats
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 7f29886568b8..6f0f0f6526c9 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1463,6 +1463,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> case V4L2_META_FMT_GENERIC_CSI2_20: descr = "8b Generic Meta, 20b CSI-2"; break;
> case V4L2_META_FMT_GENERIC_CSI2_24: descr = "8b Generic Meta, 24b CSI-2"; break;
> case V4L2_META_FMT_GENERIC_CSI2_2_24: descr = "2x8b Generic Meta, 24b CSI-2"; break;
> + case V4L2_META_FMT_BCM2835_ISP_STATS: descr = "BCM2835 ISP Image Statistics"; break;
tab instead of space?
>
> default:
> /* Compressed formats */
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 85b60199c618..eddaea8598db 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -836,6 +836,7 @@ struct v4l2_pix_format {
> #define V4L2_META_FMT_UVC v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
> #define V4L2_META_FMT_D4XX v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
> #define V4L2_META_FMT_VIVID v4l2_fourcc('V', 'I', 'V', 'D') /* Vivid Metadata */
> +#define V4L2_META_FMT_BCM2835_ISP_STATS v4l2_fourcc('B', 'S', 'T', 'A') /* BCM2835 ISP image statistics output */
Shouldn't this go to the vendor specific ones?
>
> /* Vendor specific - used for RK_ISP1 camera sub-system */
> #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 09/15] staging: vc04_services: Add helpers for vchiq driver data
2023-11-09 21:03 ` [PATCH v2 09/15] staging: vc04_services: Add helpers for vchiq driver data Umang Jain
@ 2023-11-18 14:43 ` Stefan Wahren
0 siblings, 0 replies; 45+ messages in thread
From: Stefan Wahren @ 2023-11-18 14:43 UTC (permalink / raw)
To: Umang Jain, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter
Hi Umang,
Am 09.11.23 um 22:03 schrieb Umang Jain:
> Add helpers to set and get vchiq driver data.
> vchiq_set_drvdata() and vchiq_get_drvdata() wraps
> dev_set_drvdata() and dev_get_drvdata() respectively.
no strong opinion here, but is this change a real benefit?
Except of this such a change should make use of it.
>
> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
> ---
> .../vc04_services/interface/vchiq_arm/vchiq_bus.h | 10 ++++++++++
> 1 file changed, 10 insertions(+)
>
> diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_bus.h b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_bus.h
> index caa6fdf25bb1..800f53bb48c2 100644
> --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_bus.h
> +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_bus.h
> @@ -34,6 +34,16 @@ static inline struct vchiq_driver *to_vchiq_driver(struct device_driver *d)
> return container_of(d, struct vchiq_driver, driver);
> }
>
> +static inline void *vchiq_get_drvdata(const struct vchiq_device *device)
> +{
> + return dev_get_drvdata(&device->dev);
> +}
> +
> +static inline void vchiq_set_drvdata(struct vchiq_device *device, void *data)
> +{
> + dev_set_drvdata(&device->dev, data);
> +}
> +
> extern struct bus_type vchiq_bus_type;
>
> struct vchiq_device *
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 10/15] staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component
2023-11-09 21:03 ` [PATCH v2 10/15] staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component Umang Jain
2023-11-13 14:07 ` [PATCH] fixup! " Umang Jain
@ 2023-11-18 14:53 ` Stefan Wahren
2023-11-18 18:56 ` Laurent Pinchart
2023-11-20 10:33 ` Dan Carpenter
2 siblings, 1 reply; 45+ messages in thread
From: Stefan Wahren @ 2023-11-18 14:53 UTC (permalink / raw)
To: Umang Jain, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging
Cc: Dave Stevenson, Kieran Bingham, Laurent Pinchart,
Ricardo B . Marliere, Dan Carpenter, Naushir Patuck
Hi Umang,
Am 09.11.23 um 22:03 schrieb Umang Jain:
> From: Naushir Patuck <naush@raspberrypi.com>
>
> Driver for the BCM2835 ISP hardware block. This driver uses the MMAL
> component to program the ISP hardware through the VC firmware.
>
> The ISP component can produce two video stream outputs, and Bayer
> image statistics. This can't be encompassed in a simple V4L2
> M2M device, so create a new device that registers 4 video nodes.
sorry, i don't have much knowledge about V4L2, so someone else should
review this in depth.
But from my PoV, patch 11, 12, 13 and 15 should be squash into this one.
>
> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
> ---
> MAINTAINERS | 9 +
> drivers/staging/vc04_services/Kconfig | 2 +
> drivers/staging/vc04_services/Makefile | 1 +
> .../staging/vc04_services/bcm2835-isp/Kconfig | 14 +
> .../vc04_services/bcm2835-isp/Makefile | 4 +
> .../bcm2835-isp/bcm2835-isp-ctrls.h | 72 +
> .../bcm2835-isp/bcm2835-isp-fmts.h | 482 +++++
> .../bcm2835-isp/bcm2835-v4l2-isp.c | 1712 +++++++++++++++++
> .../vc04_services/vchiq-mmal/mmal-encodings.h | 4 +
> .../vchiq-mmal/mmal-parameters.h | 165 +-
> include/uapi/linux/bcm2835-isp.h | 27 +
> 11 files changed, 2491 insertions(+), 1 deletion(-)
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-ctrls.h
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 0748cc15588c..8670a8d95400 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -3976,6 +3976,15 @@ F: Documentation/devicetree/bindings/mips/brcm/
> F: arch/mips/bcm47xx/*
> F: arch/mips/include/asm/mach-bcm47xx/*
>
> +BROADCOM BCM2835 ISP DRIVER
> +M: Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> +L: linux-media@vger.kernel.org
> +S: Maintained
> +F: Documentation/media/uapi/v4l/pixfmt-meta-bcm2835-isp-stats.rst
> +F: Documentation/media/v4l-drivers/bcm2835-isp.rst
> +F: drivers/staging/vc04_services/bcm2835-isp
> +F: include/uapi/linux/bcm2835-isp.h
> +
> BROADCOM BCM4908 ETHERNET DRIVER
> M: Rafał Miłecki <rafal@milecki.pl>
> R: Broadcom internal kernel review list <bcm-kernel-feedback-list@broadcom.com>
> diff --git a/drivers/staging/vc04_services/Kconfig b/drivers/staging/vc04_services/Kconfig
> index 6c0e77d64376..e71e5a8b820a 100644
> --- a/drivers/staging/vc04_services/Kconfig
> +++ b/drivers/staging/vc04_services/Kconfig
> @@ -44,6 +44,8 @@ source "drivers/staging/vc04_services/bcm2835-audio/Kconfig"
>
> source "drivers/staging/vc04_services/bcm2835-camera/Kconfig"
>
> +source "drivers/staging/vc04_services/bcm2835-isp/Kconfig"
> +
> source "drivers/staging/vc04_services/vchiq-mmal/Kconfig"
>
> source "drivers/staging/vc04_services/vc-sm-cma/Kconfig"
> diff --git a/drivers/staging/vc04_services/Makefile b/drivers/staging/vc04_services/Makefile
> index 1f61a263f22d..2cebf92525a0 100644
> --- a/drivers/staging/vc04_services/Makefile
> +++ b/drivers/staging/vc04_services/Makefile
> @@ -16,3 +16,4 @@ obj-$(CONFIG_SND_BCM2835) += bcm2835-audio/
> obj-$(CONFIG_VIDEO_BCM2835) += bcm2835-camera/
> obj-$(CONFIG_BCM2835_VCHIQ_MMAL) += vchiq-mmal/
> obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma/
> +obj-$(CONFIG_VIDEO_ISP_BCM2835) += bcm2835-isp/
> diff --git a/drivers/staging/vc04_services/bcm2835-isp/Kconfig b/drivers/staging/vc04_services/bcm2835-isp/Kconfig
> new file mode 100644
> index 000000000000..6222799ebe16
> --- /dev/null
> +++ b/drivers/staging/vc04_services/bcm2835-isp/Kconfig
> @@ -0,0 +1,14 @@
> +config VIDEO_ISP_BCM2835
> + tristate "BCM2835 ISP support"
> + depends on MEDIA_SUPPORT
> + depends on VIDEO_DEV && (ARCH_BCM2835 || COMPILE_TEST)
> + depends on MEDIA_CONTROLLER
> + select BCM2835_VCHIQ_MMAL
> + select VIDEOBUF2_DMA_CONTIG
> + help
> + This is the V4L2 driver for the Broadcom BCM2835 ISP hardware.
I think the description is not helpful for all users. Not everybody
knows what ISP is for. Please describe it more for an end user.
Best regards
> + This operates over the VCHIQ interface to a service running on
> + VideoCore.
> +
> + To compile this driver as a module, choose M here: the module
> + will be called bcm2835-isp.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 10/15] staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component
2023-11-18 14:53 ` [PATCH v2 10/15] " Stefan Wahren
@ 2023-11-18 18:56 ` Laurent Pinchart
2023-11-19 12:02 ` Stefan Wahren
0 siblings, 1 reply; 45+ messages in thread
From: Laurent Pinchart @ 2023-11-18 18:56 UTC (permalink / raw)
To: Stefan Wahren
Cc: Umang Jain, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging,
Dave Stevenson, Kieran Bingham, Ricardo B . Marliere,
Dan Carpenter, Naushir Patuck
Hi Stefan,
On Sat, Nov 18, 2023 at 03:53:41PM +0100, Stefan Wahren wrote:
> Am 09.11.23 um 22:03 schrieb Umang Jain:
> > From: Naushir Patuck <naush@raspberrypi.com>
> >
> > Driver for the BCM2835 ISP hardware block. This driver uses the MMAL
> > component to program the ISP hardware through the VC firmware.
> >
> > The ISP component can produce two video stream outputs, and Bayer
> > image statistics. This can't be encompassed in a simple V4L2
> > M2M device, so create a new device that registers 4 video nodes.
>
> sorry, i don't have much knowledge about V4L2, so someone else should
> review this in depth.
>
> But from my PoV, patch 11, 12, 13 and 15 should be squash into this one.
For 11 and 12, I agree. 13 is a bit of a workaround for a missing
feature in V4L2, and while it's acceptable today, having it as a
separate commit may make it easier to revert it in the future when the
V4L2 API will be extended (that may or may not happen, and I wouldn't
count on it happening quickly in any case). I would keep patch 15
separate as it's not part of this driver.
> > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
> > ---
> > MAINTAINERS | 9 +
> > drivers/staging/vc04_services/Kconfig | 2 +
> > drivers/staging/vc04_services/Makefile | 1 +
> > .../staging/vc04_services/bcm2835-isp/Kconfig | 14 +
> > .../vc04_services/bcm2835-isp/Makefile | 4 +
> > .../bcm2835-isp/bcm2835-isp-ctrls.h | 72 +
> > .../bcm2835-isp/bcm2835-isp-fmts.h | 482 +++++
> > .../bcm2835-isp/bcm2835-v4l2-isp.c | 1712 +++++++++++++++++
> > .../vc04_services/vchiq-mmal/mmal-encodings.h | 4 +
> > .../vchiq-mmal/mmal-parameters.h | 165 +-
> > include/uapi/linux/bcm2835-isp.h | 27 +
> > 11 files changed, 2491 insertions(+), 1 deletion(-)
> > create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
> > create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
> > create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-ctrls.h
> > create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
> > create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 0748cc15588c..8670a8d95400 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -3976,6 +3976,15 @@ F: Documentation/devicetree/bindings/mips/brcm/
> > F: arch/mips/bcm47xx/*
> > F: arch/mips/include/asm/mach-bcm47xx/*
> >
> > +BROADCOM BCM2835 ISP DRIVER
> > +M: Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> > +L: linux-media@vger.kernel.org
> > +S: Maintained
> > +F: Documentation/media/uapi/v4l/pixfmt-meta-bcm2835-isp-stats.rst
> > +F: Documentation/media/v4l-drivers/bcm2835-isp.rst
> > +F: drivers/staging/vc04_services/bcm2835-isp
> > +F: include/uapi/linux/bcm2835-isp.h
> > +
> > BROADCOM BCM4908 ETHERNET DRIVER
> > M: Rafał Miłecki <rafal@milecki.pl>
> > R: Broadcom internal kernel review list <bcm-kernel-feedback-list@broadcom.com>
> > diff --git a/drivers/staging/vc04_services/Kconfig b/drivers/staging/vc04_services/Kconfig
> > index 6c0e77d64376..e71e5a8b820a 100644
> > --- a/drivers/staging/vc04_services/Kconfig
> > +++ b/drivers/staging/vc04_services/Kconfig
> > @@ -44,6 +44,8 @@ source "drivers/staging/vc04_services/bcm2835-audio/Kconfig"
> >
> > source "drivers/staging/vc04_services/bcm2835-camera/Kconfig"
> >
> > +source "drivers/staging/vc04_services/bcm2835-isp/Kconfig"
> > +
> > source "drivers/staging/vc04_services/vchiq-mmal/Kconfig"
> >
> > source "drivers/staging/vc04_services/vc-sm-cma/Kconfig"
> > diff --git a/drivers/staging/vc04_services/Makefile b/drivers/staging/vc04_services/Makefile
> > index 1f61a263f22d..2cebf92525a0 100644
> > --- a/drivers/staging/vc04_services/Makefile
> > +++ b/drivers/staging/vc04_services/Makefile
> > @@ -16,3 +16,4 @@ obj-$(CONFIG_SND_BCM2835) += bcm2835-audio/
> > obj-$(CONFIG_VIDEO_BCM2835) += bcm2835-camera/
> > obj-$(CONFIG_BCM2835_VCHIQ_MMAL) += vchiq-mmal/
> > obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma/
> > +obj-$(CONFIG_VIDEO_ISP_BCM2835) += bcm2835-isp/
> > diff --git a/drivers/staging/vc04_services/bcm2835-isp/Kconfig b/drivers/staging/vc04_services/bcm2835-isp/Kconfig
> > new file mode 100644
> > index 000000000000..6222799ebe16
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/bcm2835-isp/Kconfig
> > @@ -0,0 +1,14 @@
> > +config VIDEO_ISP_BCM2835
> > + tristate "BCM2835 ISP support"
> > + depends on MEDIA_SUPPORT
> > + depends on VIDEO_DEV && (ARCH_BCM2835 || COMPILE_TEST)
> > + depends on MEDIA_CONTROLLER
> > + select BCM2835_VCHIQ_MMAL
> > + select VIDEOBUF2_DMA_CONTIG
> > + help
> > + This is the V4L2 driver for the Broadcom BCM2835 ISP hardware.
>
> I think the description is not helpful for all users. Not everybody
> knows what ISP is for. Please describe it more for an end user.
>
> > + This operates over the VCHIQ interface to a service running on
> > + VideoCore.
> > +
> > + To compile this driver as a module, choose M here: the module
> > + will be called bcm2835-isp.
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 10/15] staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component
2023-11-18 18:56 ` Laurent Pinchart
@ 2023-11-19 12:02 ` Stefan Wahren
0 siblings, 0 replies; 45+ messages in thread
From: Stefan Wahren @ 2023-11-19 12:02 UTC (permalink / raw)
To: Laurent Pinchart
Cc: Umang Jain, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging,
Dave Stevenson, Kieran Bingham, Ricardo B . Marliere,
Dan Carpenter, Naushir Patuck
Am 18.11.23 um 19:56 schrieb Laurent Pinchart:
> Hi Stefan,
>
> On Sat, Nov 18, 2023 at 03:53:41PM +0100, Stefan Wahren wrote:
>> Am 09.11.23 um 22:03 schrieb Umang Jain:
>>> From: Naushir Patuck <naush@raspberrypi.com>
>>>
>>> Driver for the BCM2835 ISP hardware block. This driver uses the MMAL
>>> component to program the ISP hardware through the VC firmware.
>>>
>>> The ISP component can produce two video stream outputs, and Bayer
>>> image statistics. This can't be encompassed in a simple V4L2
>>> M2M device, so create a new device that registers 4 video nodes.
>> sorry, i don't have much knowledge about V4L2, so someone else should
>> review this in depth.
>>
>> But from my PoV, patch 11, 12, 13 and 15 should be squash into this one.
> For 11 and 12, I agree. 13 is a bit of a workaround for a missing
> feature in V4L2, and while it's acceptable today, having it as a
> separate commit may make it easier to revert it in the future when the
> V4L2 API will be extended (that may or may not happen, and I wouldn't
> count on it happening quickly in any case). I would keep patch 15
> separate as it's not part of this driver.
Okay, accepted
>
>>> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
>>> Signed-off-by: Umang Jain <umang.jain@ideasonboard.com>
>>> ---
>>> MAINTAINERS | 9 +
>>> drivers/staging/vc04_services/Kconfig | 2 +
>>> drivers/staging/vc04_services/Makefile | 1 +
>>> .../staging/vc04_services/bcm2835-isp/Kconfig | 14 +
>>> .../vc04_services/bcm2835-isp/Makefile | 4 +
>>> .../bcm2835-isp/bcm2835-isp-ctrls.h | 72 +
>>> .../bcm2835-isp/bcm2835-isp-fmts.h | 482 +++++
>>> .../bcm2835-isp/bcm2835-v4l2-isp.c | 1712 +++++++++++++++++
>>> .../vc04_services/vchiq-mmal/mmal-encodings.h | 4 +
>>> .../vchiq-mmal/mmal-parameters.h | 165 +-
>>> include/uapi/linux/bcm2835-isp.h | 27 +
>>> 11 files changed, 2491 insertions(+), 1 deletion(-)
>>> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
>>> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
>>> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-ctrls.h
>>> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-isp-fmts.h
>>> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
>>>
>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>> index 0748cc15588c..8670a8d95400 100644
>>> --- a/MAINTAINERS
>>> +++ b/MAINTAINERS
>>> @@ -3976,6 +3976,15 @@ F: Documentation/devicetree/bindings/mips/brcm/
>>> F: arch/mips/bcm47xx/*
>>> F: arch/mips/include/asm/mach-bcm47xx/*
>>>
>>> +BROADCOM BCM2835 ISP DRIVER
>>> +M: Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
>>> +L: linux-media@vger.kernel.org
>>> +S: Maintained
>>> +F: Documentation/media/uapi/v4l/pixfmt-meta-bcm2835-isp-stats.rst
>>> +F: Documentation/media/v4l-drivers/bcm2835-isp.rst
>>> +F: drivers/staging/vc04_services/bcm2835-isp
>>> +F: include/uapi/linux/bcm2835-isp.h
>>> +
>>> BROADCOM BCM4908 ETHERNET DRIVER
>>> M: Rafał Miłecki <rafal@milecki.pl>
>>> R: Broadcom internal kernel review list <bcm-kernel-feedback-list@broadcom.com>
>>> diff --git a/drivers/staging/vc04_services/Kconfig b/drivers/staging/vc04_services/Kconfig
>>> index 6c0e77d64376..e71e5a8b820a 100644
>>> --- a/drivers/staging/vc04_services/Kconfig
>>> +++ b/drivers/staging/vc04_services/Kconfig
>>> @@ -44,6 +44,8 @@ source "drivers/staging/vc04_services/bcm2835-audio/Kconfig"
>>>
>>> source "drivers/staging/vc04_services/bcm2835-camera/Kconfig"
>>>
>>> +source "drivers/staging/vc04_services/bcm2835-isp/Kconfig"
>>> +
>>> source "drivers/staging/vc04_services/vchiq-mmal/Kconfig"
>>>
>>> source "drivers/staging/vc04_services/vc-sm-cma/Kconfig"
>>> diff --git a/drivers/staging/vc04_services/Makefile b/drivers/staging/vc04_services/Makefile
>>> index 1f61a263f22d..2cebf92525a0 100644
>>> --- a/drivers/staging/vc04_services/Makefile
>>> +++ b/drivers/staging/vc04_services/Makefile
>>> @@ -16,3 +16,4 @@ obj-$(CONFIG_SND_BCM2835) += bcm2835-audio/
>>> obj-$(CONFIG_VIDEO_BCM2835) += bcm2835-camera/
>>> obj-$(CONFIG_BCM2835_VCHIQ_MMAL) += vchiq-mmal/
>>> obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma/
>>> +obj-$(CONFIG_VIDEO_ISP_BCM2835) += bcm2835-isp/
>>> diff --git a/drivers/staging/vc04_services/bcm2835-isp/Kconfig b/drivers/staging/vc04_services/bcm2835-isp/Kconfig
>>> new file mode 100644
>>> index 000000000000..6222799ebe16
>>> --- /dev/null
>>> +++ b/drivers/staging/vc04_services/bcm2835-isp/Kconfig
>>> @@ -0,0 +1,14 @@
>>> +config VIDEO_ISP_BCM2835
>>> + tristate "BCM2835 ISP support"
>>> + depends on MEDIA_SUPPORT
>>> + depends on VIDEO_DEV && (ARCH_BCM2835 || COMPILE_TEST)
>>> + depends on MEDIA_CONTROLLER
>>> + select BCM2835_VCHIQ_MMAL
>>> + select VIDEOBUF2_DMA_CONTIG
>>> + help
>>> + This is the V4L2 driver for the Broadcom BCM2835 ISP hardware.
>> I think the description is not helpful for all users. Not everybody
>> knows what ISP is for. Please describe it more for an end user.
>>
>>> + This operates over the VCHIQ interface to a service running on
>>> + VideoCore.
>>> +
>>> + To compile this driver as a module, choose M here: the module
>>> + will be called bcm2835-isp.
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 01/15] staging: vc04_services: Add new vc-sm-cma driver
2023-11-09 21:02 ` [PATCH v2 01/15] staging: vc04_services: Add new vc-sm-cma driver Umang Jain
2023-11-13 17:56 ` Stefan Wahren
@ 2023-11-20 9:52 ` Dan Carpenter
1 sibling, 0 replies; 45+ messages in thread
From: Dan Carpenter @ 2023-11-20 9:52 UTC (permalink / raw)
To: Umang Jain
Cc: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging, Dave Stevenson, Kieran Bingham,
Laurent Pinchart, Ricardo B . Marliere, Dan Carpenter,
Stefan Wahren
On Thu, Nov 09, 2023 at 04:02:53PM -0500, Umang Jain wrote:
> +/* Private file data associated with each opened device. */
> +struct vc_sm_privdata_t {
> +};
> +
> +typedef int (*VC_SM_SHOW) (struct seq_file *s, void *v);
> +struct sm_pde_t {
Get rid of all the _t suffixes. It's not a typedef.
> + VC_SM_SHOW show; /* Debug fs function hookup. */
> + struct dentry *dir_entry; /* Debug fs directory entry. */
> + void *priv_data; /* Private data */
> +};
> +
> +/* Global state information. */
> +struct sm_state_t {
Here too. Just search for "_t " and delete them all.
> + struct vchiq_device *device;
> +
> + struct miscdevice misc_dev;
> +
> + struct sm_instance *sm_handle; /* Handle for videocore service. */
> +
> + spinlock_t kernelid_map_lock; /* Spinlock protecting kernelid_map */
> + struct idr kernelid_map;
> +
> + struct mutex map_lock; /* Global map lock. */
> + struct list_head buffer_list; /* List of buffer. */
> +
> + struct dentry *dir_root; /* Debug fs entries root. */
> + struct sm_pde_t dir_state; /* Debug fs entries state sub-tree. */
> +
> + bool require_released_callback; /* VPU will send a released msg when it
> + * has finished with a resource.
> + */
> + /* State for transactions */
> + int restart_sys; /* Tracks restart on interrupt. */
> + enum vc_sm_msg_type int_action; /* Interrupted action. */
> +
> + u32 int_trans_id; /* Interrupted transaction. */
> + struct vchiq_instance *vchiq_instance;
> +};
> +
> +struct vc_sm_dma_buf_attachment {
> + struct device *dev;
> + struct sg_table sg_table;
> + struct list_head list;
> + enum dma_data_direction dma_dir;
> +};
> +
> +/* ---- Private Variables ----------------------------------------------- */
> +
> +static struct sm_state_t *sm_state;
> +static int sm_inited;
> +
> +/* ---- Private Function Prototypes -------------------------------------- */
> +
> +/* ---- Private Functions ------------------------------------------------ */
> +
> +static int get_kernel_id(struct vc_sm_buffer *buffer)
> +{
> + int handle;
> +
> + spin_lock(&sm_state->kernelid_map_lock);
> + handle = idr_alloc(&sm_state->kernelid_map, buffer, 0, 0, GFP_KERNEL);
> + spin_unlock(&sm_state->kernelid_map_lock);
> +
> + return handle;
> +}
> +
> +static struct vc_sm_buffer *lookup_kernel_id(int handle)
> +{
> + return idr_find(&sm_state->kernelid_map, handle);
This function would be better open coded. It adds nothing except a
global variable (which is bad).
> +}
> +
> +static void free_kernel_id(int handle)
> +{
> + spin_lock(&sm_state->kernelid_map_lock);
> + idr_remove(&sm_state->kernelid_map, handle);
> + spin_unlock(&sm_state->kernelid_map_lock);
> +}
> +
> +static int vc_sm_cma_seq_file_show(struct seq_file *s, void *v)
> +{
> + struct sm_pde_t *sm_pde;
> +
> + sm_pde = (struct sm_pde_t *)(s->private);
> +
> + if (sm_pde && sm_pde->show)
> + sm_pde->show(s, v);
> +
> + return 0;
> +}
> +
> +static int vc_sm_cma_single_open(struct inode *inode, struct file *file)
> +{
> + return single_open(file, vc_sm_cma_seq_file_show, inode->i_private);
> +}
> +
> +static const struct file_operations vc_sm_cma_debug_fs_fops = {
> + .open = vc_sm_cma_single_open,
> + .read = seq_read,
> + .llseek = seq_lseek,
> + .release = single_release,
> +};
> +
> +static int vc_sm_cma_global_state_show(struct seq_file *s, void *v)
> +{
> + struct vc_sm_buffer *resource = NULL;
> + int resource_count = 0;
> +
> + if (!sm_state)
> + return 0;
> +
> + seq_printf(s, "\nVC-ServiceHandle %p\n", sm_state->sm_handle);
> +
> + /* Log all applicable mapping(s). */
> +
> + mutex_lock(&sm_state->map_lock);
> + seq_puts(s, "\nResources\n");
> + if (!list_empty(&sm_state->buffer_list)) {
Delete this check. It's not required.
> + list_for_each_entry(resource, &sm_state->buffer_list,
> + global_buffer_list) {
> + resource_count++;
> +
> + seq_printf(s, "\nResource %p\n",
> + resource);
> + seq_printf(s, " NAME %s\n",
> + resource->name);
> + seq_printf(s, " SIZE %zu\n",
> + resource->size);
> + seq_printf(s, " DMABUF %p\n",
> + resource->dma_buf);
> + seq_printf(s, " IMPORTED_DMABUF %p\n",
> + resource->imported_dma_buf);
> + seq_printf(s, " ATTACH %p\n",
> + resource->attach);
> + seq_printf(s, " SGT %p\n",
> + resource->sgt);
> + seq_printf(s, " DMA_ADDR %pad\n",
> + &resource->dma_addr);
> + seq_printf(s, " VC_HANDLE %08x\n",
> + resource->vc_handle);
> + seq_printf(s, " VC_MAPPING %d\n",
> + resource->vpu_state);
Deleting the check will let you pull the code in a tab. I would just
delete the line breaks. The longest line will go up to 94 chars so it's
under the 100 character limit.
> + }
> + }
> + seq_printf(s, "\n\nTotal resource count: %d\n\n", resource_count);
> +
> + mutex_unlock(&sm_state->map_lock);
> +
> + return 0;
> +}
> +
> +/*
> + * Adds a buffer to the private data list which tracks all the allocated
> + * data.
> + */
> +static void vc_sm_add_resource(struct vc_sm_buffer *buffer)
> +{
> + mutex_lock(&sm_state->map_lock);
> + list_add(&buffer->global_buffer_list, &sm_state->buffer_list);
> + mutex_unlock(&sm_state->map_lock);
> +
> + dev_dbg(&sm_state->device->dev,"[%s]: added buffer %p (name %s, size %zu)\n",
> + __func__, buffer, buffer->name, buffer->size);
> +}
> +
> +/*
> + * Cleans up imported dmabuf.
> + * Should be called with mutex held.
> + */
> +static void vc_sm_clean_up_dmabuf(struct vc_sm_buffer *buffer)
> +{
> + /* Handle cleaning up imported dmabufs */
> + if (buffer->sgt) {
> + dma_buf_unmap_attachment_unlocked(buffer->attach, buffer->sgt,
> + DMA_BIDIRECTIONAL);
> + buffer->sgt = NULL;
> + }
> + if (buffer->attach) {
> + dma_buf_detach(buffer->imported_dma_buf, buffer->attach);
> + buffer->attach = NULL;
> + }
> +}
> +
> +/*
> + * Instructs VPU to decrement the refcount on a buffer.
> + */
> +static void vc_sm_vpu_free(struct vc_sm_buffer *buffer)
> +{
> + if (buffer->vc_handle && buffer->vpu_state == VPU_MAPPED) {
Reverse this test:
if (!buffer->vc_handle || buffer->vpu_state != VPU_MAPPED)
return;
> + struct vc_sm_free_t free = { buffer->vc_handle, 0 };
> + int status = vc_sm_cma_vchi_free(sm_state->sm_handle, &free,
> + &sm_state->int_trans_id);
> + if (status != 0 && status != -EINTR) {
> + dev_err(&sm_state->device->dev,
> + "[%s]: failed to free memory on videocore (status: %u, trans_id: %u)\n",
> + __func__, status, sm_state->int_trans_id);
> + }
> +
> + if (sm_state->require_released_callback) {
> + /* Need to wait for the VPU to confirm the free. */
> +
> + /* Retain a reference on this until the VPU has
> + * released it
> + */
> + buffer->vpu_state = VPU_UNMAPPING;
> + } else {
> + buffer->vpu_state = VPU_NOT_MAPPED;
> + buffer->vc_handle = 0;
> + }
> + }
> +}
> +
> +/*
> + * Release an allocation.
> + * All refcounting is done via the dma buf object.
> + *
> + * Must be called with the mutex held. The function will either release the
> + * mutex (if defering the release) or destroy it. The caller must therefore not
> + * reuse the buffer on return.
> + */
> +static void vc_sm_release_resource(struct vc_sm_buffer *buffer)
> +{
> + dev_dbg(&sm_state->device->dev, "[%s]: buffer %p (name %s, size %zu)\n",
> + __func__, buffer, buffer->name, buffer->size);
> +
> + if (buffer->vc_handle) {
> + /* We've sent the unmap request but not had the response. */
> + dev_dbg(&sm_state->device->dev, "[%s]: Waiting for VPU unmap response on %p\n",
> + __func__, buffer);
> + goto defer;
> + }
> + if (buffer->in_use) {
> + /* dmabuf still in use - we await the release */
> + dev_dbg(&sm_state->device->dev, "[%s]: buffer %p is still in use\n",
> + __func__, buffer);
> + goto defer;
> + }
> +
> + /* Release the allocation */
> + if (buffer->imported_dma_buf)
> + dma_buf_put(buffer->imported_dma_buf);
> + else
> + dev_err(&sm_state->device->dev, "%s: Imported dmabuf already been put for buf %p\n",
> + __func__, buffer);
> + buffer->imported_dma_buf = NULL;
> +
> + /* Free our buffer. Start by removing it from the list */
> + mutex_lock(&sm_state->map_lock);
> + list_del(&buffer->global_buffer_list);
> + mutex_unlock(&sm_state->map_lock);
> +
> + dev_dbg(&sm_state->device->dev, "%s: Release our allocation - done\n", __func__);
> + mutex_unlock(&buffer->lock);
> +
> + mutex_destroy(&buffer->lock);
> +
> + kfree(buffer);
> + return;
> +
> +defer:
> + mutex_unlock(&buffer->lock);
> +}
> +
> +/* Dma_buf operations for chaining through to an imported dma_buf */
> +
> +static void vc_sm_dma_buf_release(struct dma_buf *dmabuf)
> +{
> + struct vc_sm_buffer *buffer;
> +
> + if (!dmabuf)
> + return;
> +
> + buffer = (struct vc_sm_buffer *)dmabuf->priv;
> +
> + mutex_lock(&buffer->lock);
> +
> + dev_dbg(&sm_state->device->dev, "%s dmabuf %p, buffer %p\n", __func__, dmabuf, buffer);
> +
> + buffer->in_use = 0;
> +
> + /* Unmap on the VPU */
> + vc_sm_vpu_free(buffer);
> + dev_dbg(&sm_state->device->dev, "%s vpu_free done\n", __func__);
> +
> + /* Unmap our dma_buf object (the vc_sm_buffer remains until released
> + * on the VPU).
> + */
> + vc_sm_clean_up_dmabuf(buffer);
> + dev_dbg(&sm_state->device->dev, "%s clean_up dmabuf done\n", __func__);
> +
> + /* buffer->lock will be destroyed by vc_sm_release_resource if finished
> + * with, otherwise unlocked. Do NOT unlock here.
> + */
> + vc_sm_release_resource(buffer);
> + dev_dbg(&sm_state->device->dev, "%s done\n", __func__);
> +}
> +
> +static
> +int vc_sm_import_dma_buf_attach(struct dma_buf *dmabuf,
> + struct dma_buf_attachment *attachment)
> +{
> + struct vc_sm_buffer *buf = dmabuf->priv;
> +
> + return buf->imported_dma_buf->ops->attach(buf->imported_dma_buf,
> + attachment);
> +}
> +
> +static
> +void vc_sm_import_dma_buf_detatch(struct dma_buf *dmabuf,
> + struct dma_buf_attachment *attachment)
> +{
> + struct vc_sm_buffer *buf = dmabuf->priv;
> +
> + buf->imported_dma_buf->ops->detach(buf->imported_dma_buf, attachment);
> +}
> +
> +static
> +struct sg_table *vc_sm_import_map_dma_buf(struct dma_buf_attachment *attachment,
> + enum dma_data_direction direction)
> +{
> + struct vc_sm_buffer *buf = attachment->dmabuf->priv;
> +
> + return buf->imported_dma_buf->ops->map_dma_buf(attachment,
> + direction);
> +}
> +
> +static
> +void vc_sm_import_unmap_dma_buf(struct dma_buf_attachment *attachment,
> + struct sg_table *table,
> + enum dma_data_direction direction)
> +{
> + struct vc_sm_buffer *buf = attachment->dmabuf->priv;
> +
> + buf->imported_dma_buf->ops->unmap_dma_buf(attachment, table, direction);
> +}
> +
> +static
> +int vc_sm_import_dmabuf_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma)
> +{
> + struct vc_sm_buffer *buf = dmabuf->priv;
> +
> + dev_dbg(&sm_state->device->dev, "%s: mmap dma_buf %p, buf %p, imported db %p\n", __func__,
> + dmabuf, buf, buf->imported_dma_buf);
> +
> + return buf->imported_dma_buf->ops->mmap(buf->imported_dma_buf, vma);
> +}
> +
> +static
> +int vc_sm_import_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
> + enum dma_data_direction direction)
> +{
> + struct vc_sm_buffer *buf = dmabuf->priv;
> +
> + return buf->imported_dma_buf->ops->begin_cpu_access(buf->imported_dma_buf,
> + direction);
> +}
> +
> +static
> +int vc_sm_import_dma_buf_end_cpu_access(struct dma_buf *dmabuf,
> + enum dma_data_direction direction)
> +{
> + struct vc_sm_buffer *buf = dmabuf->priv;
> +
> + return buf->imported_dma_buf->ops->end_cpu_access(buf->imported_dma_buf,
> + direction);
> +}
> +
> +static const struct dma_buf_ops dma_buf_import_ops = {
> + .map_dma_buf = vc_sm_import_map_dma_buf,
> + .unmap_dma_buf = vc_sm_import_unmap_dma_buf,
> + .mmap = vc_sm_import_dmabuf_mmap,
> + .release = vc_sm_dma_buf_release,
> + .attach = vc_sm_import_dma_buf_attach,
> + .detach = vc_sm_import_dma_buf_detatch,
> + .begin_cpu_access = vc_sm_import_dma_buf_begin_cpu_access,
> + .end_cpu_access = vc_sm_import_dma_buf_end_cpu_access,
> +};
> +
> +/* Import a dma_buf to be shared with VC. */
> +int
> +vc_sm_cma_import_dmabuf_internal(struct sm_state_t *state,
> + struct dma_buf *dma_buf,
> + int fd,
> + struct dma_buf **imported_buf)
> +{
> + DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
> + struct vc_sm_buffer *buffer = NULL;
> + struct vc_sm_import import = { };
> + struct vc_sm_import_result result = { };
> + struct dma_buf_attachment *attach = NULL;
> + struct sg_table *sgt = NULL;
> + dma_addr_t dma_addr;
> + int ret = 0;
> + int status;
> +
> + /* Setup our allocation parameters */
> + dev_dbg(&sm_state->device->dev, "%s: importing dma_buf %p/fd %d\n", __func__, dma_buf, fd);
> +
> + if (fd < 0)
> + get_dma_buf(dma_buf);
> + else
> + dma_buf = dma_buf_get(fd);
> +
> + if (!dma_buf)
> + return -EINVAL;
> +
> + attach = dma_buf_attach(dma_buf, &sm_state->device->dev);
> + if (IS_ERR(attach)) {
> + ret = PTR_ERR(attach);
> + goto error;
This is One Err Style error handling. It's the second worst style of
error handling. The style which is worse than this is One Err Style
but with the error handling done in a separate function called One
Magical Cleanup Function style error handling.
The reason that it's buggy is because we free things that have not
been allocated. Better to re-write it in Free The Last Thing style.
https://staticthinking.wordpress.com/2022/04/28/free-the-last-thing-style/
> + }
> +
> + sgt = dma_buf_map_attachment_unlocked(attach, DMA_BIDIRECTIONAL);
> + if (IS_ERR(sgt)) {
> + ret = PTR_ERR(sgt);
> + goto error;
> + }
> +
> + /* Verify that the address block is contiguous */
> + if (sgt->nents != 1) {
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + /* Allocate local buffer to track this allocation. */
> + buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
> + if (!buffer) {
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + import.type = VC_SM_ALLOC_NON_CACHED;
> + dma_addr = sg_dma_address(sgt->sgl);
> + import.addr = (u32)dma_addr;
> + if ((import.addr & 0xC0000000) != 0xC0000000) {
> + dev_err(&sm_state->device->dev, "%s: Expecting an uncached alias for dma_addr %pad\n",
> + __func__, &dma_addr);
> + import.addr |= 0xC0000000;
> + }
> + import.size = sg_dma_len(sgt->sgl);
> + import.allocator = current->tgid;
> + import.kernel_id = get_kernel_id(buffer);
> +
> + memcpy(import.name, VC_SM_RESOURCE_NAME_DEFAULT,
> + sizeof(VC_SM_RESOURCE_NAME_DEFAULT));
> +
> + dev_dbg(&sm_state->device->dev,
> + "[%s]: attempt to import \"%s\" data - type %u, addr %pad, size %u.\n",
> + __func__, import.name, import.type, &dma_addr, import.size);
> +
> + /* Allocate the videocore buffer. */
> + status = vc_sm_cma_vchi_import(sm_state->sm_handle, &import, &result,
> + &sm_state->int_trans_id);
> + if (status == -EINTR) {
> + dev_dbg(&sm_state->device->dev,
> + "[%s]: requesting import memory action restart (trans_id: %u)\n",
> + __func__, sm_state->int_trans_id);
> + ret = -ERESTARTSYS;
> + sm_state->restart_sys = -EINTR;
> + sm_state->int_action = VC_SM_MSG_TYPE_IMPORT;
> + goto error;
> + } else if (status || !result.res_handle) {
> + dev_dbg(&sm_state->device->dev,
> + "[%s]: failed to import memory on videocore (status: %u, trans_id: %u)\n",
> + __func__, status, sm_state->int_trans_id);
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + mutex_init(&buffer->lock);
> + INIT_LIST_HEAD(&buffer->attachments);
> + memcpy(buffer->name, import.name,
> + min(sizeof(buffer->name), sizeof(import.name) - 1));
> +
> + /* Keep track of the buffer we created. */
> + buffer->vc_handle = result.res_handle;
> + buffer->size = import.size;
> + buffer->vpu_state = VPU_MAPPED;
> +
> + buffer->imported_dma_buf = dma_buf;
> +
> + buffer->attach = attach;
> + buffer->sgt = sgt;
> + buffer->dma_addr = dma_addr;
> + buffer->in_use = 1;
> + buffer->kernel_id = import.kernel_id;
> +
> + /*
> + * We're done - we need to export a new dmabuf chaining through most
> + * functions, but enabling us to release our own internal references
> + * here.
> + */
> + exp_info.ops = &dma_buf_import_ops;
> + exp_info.size = import.size;
> + exp_info.flags = O_RDWR;
> + exp_info.priv = buffer;
> +
> + buffer->dma_buf = dma_buf_export(&exp_info);
> + if (IS_ERR(buffer->dma_buf)) {
> + ret = PTR_ERR(buffer->dma_buf);
> + goto error;
> + }
> +
> + vc_sm_add_resource(buffer);
> +
> + *imported_buf = buffer->dma_buf;
> +
> + return 0;
> +
> +error:
> + if (result.res_handle) {
> + struct vc_sm_free_t free = { result.res_handle, 0 };
> +
> + vc_sm_cma_vchi_free(sm_state->sm_handle, &free,
> + &sm_state->int_trans_id);
> + }
> + free_kernel_id(import.kernel_id);
> + kfree(buffer);
> + if (sgt)
> + dma_buf_unmap_attachment_unlocked(attach, sgt, DMA_BIDIRECTIONAL);
> + if (attach)
> + dma_buf_detach(dma_buf, attach);
> + dma_buf_put(dma_buf);
> + return ret;
> +}
> +
> +static void
> +vc_sm_vpu_event(struct sm_instance *instance, struct vc_sm_result_t *reply,
> + int reply_len)
> +{
> + switch (reply->trans_id & ~0x80000000) {
> + case VC_SM_MSG_TYPE_CLIENT_VERSION:
> + {
> + /* Acknowledge that the firmware supports the version command */
> + dev_dbg(&sm_state->device->dev,
> + "%s: firmware acked version msg. Require release cb\n", __func__);
> + sm_state->require_released_callback = true;
> + }
> + break;
> + case VC_SM_MSG_TYPE_RELEASED:
> + {
> + struct vc_sm_released *release = (struct vc_sm_released *)reply;
> + struct vc_sm_buffer *buffer =
> + lookup_kernel_id(release->kernel_id);
> + if (!buffer) {
> + dev_err(&sm_state->device->dev,
> + "%s: VC released a buffer that is already released, kernel_id %d\n",
> + __func__, release->kernel_id);
> + break;
> + }
> + mutex_lock(&buffer->lock);
> +
> + dev_dbg(&sm_state->device->dev,
> + "%s: Released addr %08x, size %u, id %08x, mem_handle %08x\n",
> + __func__, release->addr, release->size,
> + release->kernel_id, release->vc_handle);
> +
> + buffer->vc_handle = 0;
> + buffer->vpu_state = VPU_NOT_MAPPED;
> + free_kernel_id(release->kernel_id);
> +
> + vc_sm_release_resource(buffer);
> + }
> + break;
> + default:
> + dev_err(&sm_state->device->dev, "%s: Unknown vpu cmd %x\n",
> + __func__, reply->trans_id);
> + break;
> + }
> +}
> +
> +/* Driver load/unload functions */
> +/* Videocore connected. */
> +static void vc_sm_connected_init(void)
> +{
> + int ret;
> + struct vc_sm_version version;
> + struct vc_sm_result_t version_result;
> +
> + pr_info("[%s]: start\n", __func__);
Delete this. Use ftrace instead.
> +
> + /*
> + * Initialize and create a VCHI connection for the shared memory service
> + * running on videocore.
> + */
> + ret = vchiq_initialise(&sm_state->vchiq_instance);
> + if (ret) {
> + dev_err(&sm_state->device->dev, "[%s]: failed to initialise VCHI instance (ret=%d)\n",
> + __func__, ret);
> +
> + return;
> + }
> +
> + ret = vchiq_connect(sm_state->vchiq_instance);
> + if (ret) {
> + dev_err(&sm_state->device->dev, "[%s]: failed to connect VCHI instance (ret=%d)\n",
> + __func__, ret);
> +
> + return;
> + }
> +
> + /* Initialize an instance of the shared memory service. */
> + sm_state->sm_handle = vc_sm_cma_vchi_init(sm_state->vchiq_instance, 1,
> + vc_sm_vpu_event);
> + if (!sm_state->sm_handle) {
> + dev_err(&sm_state->device->dev,
> + "[%s]: failed to initialize shared memory service\n", __func__);
> +
> + return;
> + }
> +
> + /* Create a debug fs directory entry (root). */
> + sm_state->dir_root = debugfs_create_dir(VC_SM_DIR_ROOT_NAME, NULL);
> +
> + sm_state->dir_state.show = &vc_sm_cma_global_state_show;
> + sm_state->dir_state.dir_entry =
> + debugfs_create_file(VC_SM_STATE, 0444, sm_state->dir_root,
> + &sm_state->dir_state,
> + &vc_sm_cma_debug_fs_fops);
> +
> + INIT_LIST_HEAD(&sm_state->buffer_list);
> +
> + version.version = 2;
> + ret = vc_sm_cma_vchi_client_version(sm_state->sm_handle, &version,
> + &version_result,
> + &sm_state->int_trans_id);
> + if (ret) {
> + dev_err(&sm_state->device->dev,
> + "[%s]: Failed to send version request %d\n", __func__, ret);
> + }
> +
> + /* Done! */
> + sm_inited = 1;
> + pr_info("[%s]: installed successfully\n", __func__);
> +}
> +
> +/* Driver loading. */
> +static int bcm2835_vc_sm_cma_probe(struct vchiq_device *device)
> +{
> + struct device *dev = &device->dev;
> + int err;
> +
> + err = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
> + if (err) {
> + dev_err(dev, "dma_set_mask_and_coherent failed: %d\n", err);
> + return err;
> + }
> +
> + sm_state = devm_kzalloc(dev, sizeof(*sm_state), GFP_KERNEL);
> + if (!sm_state)
> + return -ENOMEM;
> + sm_state->device = device;
> + mutex_init(&sm_state->map_lock);
> +
> + spin_lock_init(&sm_state->kernelid_map_lock);
> + idr_init_base(&sm_state->kernelid_map, 1);
> +
> + dev->dma_parms = devm_kzalloc(dev, sizeof(dev->dma_parms), GFP_KERNEL);
> +
> + /* dma_set_max_seg_size checks if dma_parms is NULL. */
> + dma_set_max_seg_size(dev, 0x3FFFFFFF);
> +
> + vchiq_add_connected_callback(vc_sm_connected_init);
> + return 0;
> +}
> +
> +/* Driver unloading. */
> +static void bcm2835_vc_sm_cma_remove(struct vchiq_device *device)
> +{
> + if (sm_inited) {
> + misc_deregister(&sm_state->misc_dev);
> +
> + /* Remove all proc entries. */
> + debugfs_remove_recursive(sm_state->dir_root);
> +
> + /* Stop the videocore shared memory service. */
> + vc_sm_cma_vchi_stop(sm_state->vchiq_instance, &sm_state->sm_handle);
> + }
> +
> + if (sm_state) {
> + idr_destroy(&sm_state->kernelid_map);
> +
> + /* Free the memory for the state structure. */
> + mutex_destroy(&sm_state->map_lock);
> + }
> +}
> +
> +/* Kernel API calls */
> +/* Get an internal resource handle mapped from the external one. */
> +int vc_sm_cma_int_handle(void *handle)
> +{
> + struct dma_buf *dma_buf = (struct dma_buf *)handle;
> + struct vc_sm_buffer *buf;
> +
> + /* Validate we can work with this device. */
> + if (!sm_state || !handle) {
> + dev_err(&sm_state->device->dev, "[%s]: invalid input\n", __func__);
> + return 0;
> + }
> +
> + buf = (struct vc_sm_buffer *)dma_buf->priv;
> + return buf->vc_handle;
> +}
> +EXPORT_SYMBOL_GPL(vc_sm_cma_int_handle);
> +
> +/* Free a previously allocated shared memory handle and block. */
> +int vc_sm_cma_free(void *handle)
> +{
> + struct dma_buf *dma_buf = (struct dma_buf *)handle;
> +
> + /* Validate we can work with this device. */
> + if (!sm_state || !handle) {
> + dev_err(&sm_state->device->dev, "[%s]: invalid input\n", __func__);
> + return -EPERM;
> + }
> +
> + dev_dbg(&sm_state->device->dev, "%s: handle %p/dmabuf %p\n", __func__, handle, dma_buf);
> +
> + dma_buf_put(dma_buf);
> +
> + return 0;
> +}
> +EXPORT_SYMBOL_GPL(vc_sm_cma_free);
> +
> +/* Import a dmabuf to be shared with VC. */
> +int vc_sm_cma_import_dmabuf(struct dma_buf *src_dmabuf, void **handle)
> +{
> + struct dma_buf *new_dma_buf;
> + int ret;
> +
> + /* Validate we can work with this device. */
> + if (!sm_state || !src_dmabuf || !handle) {
> + dev_err(&sm_state->device->dev, "[%s]: invalid input\n", __func__);
> + return -EPERM;
> + }
> +
> + ret = vc_sm_cma_import_dmabuf_internal(sm_state, src_dmabuf,
> + -1, &new_dma_buf);
> +
> + if (!ret) {
> + dev_dbg(&sm_state->device->dev, "%s: imported to ptr %p\n",
> + __func__, new_dma_buf);
> +
> + /* Assign valid handle at this time.*/
> + *handle = new_dma_buf;
> + } else {
> + /*
> + * succeeded in importing the dma_buf, but then
> + * failed to look it up again. How?
> + * Release the fd again.
> + */
> + dev_err(&sm_state->device->dev, "%s: imported vc_sm_cma_get_buffer failed %d\n",
> + __func__, ret);
> + }
> +
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(vc_sm_cma_import_dmabuf);
> +
> +static struct vchiq_device_id device_id_table[] = {
> + { .name = "vcsm-cma" },
> + {}
> +};
> +MODULE_DEVICE_TABLE(vchiq, device_id_table);
> +
> +static struct vchiq_driver bcm2835_vcsm_cma_driver = {
> + .probe = bcm2835_vc_sm_cma_probe,
> + .remove = bcm2835_vc_sm_cma_remove,
> + .id_table = device_id_table,
> + .driver = {
> + .name = "vcsm-cma",
> + },
> +};
> +
> +module_vchiq_driver(bcm2835_vcsm_cma_driver);
> +
> +MODULE_AUTHOR("Dave Stevenson");
> +MODULE_DESCRIPTION("VideoCore CMA Shared Memory Driver");
> +MODULE_LICENSE("GPL");
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> new file mode 100644
> index 000000000000..61e110ec414d
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> @@ -0,0 +1,54 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +/*
> + * VideoCore Shared Memory driver using CMA.
> + *
> + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> + *
> + */
> +
> +#ifndef VC_SM_H
> +#define VC_SM_H
> +
> +#define VC_SM_MAX_NAME_LEN 32
> +
> +enum vc_sm_vpu_mapping_state {
> + VPU_NOT_MAPPED,
> + VPU_MAPPED,
> + VPU_UNMAPPING
> +};
> +
> +struct vc_sm_buffer {
> + struct list_head global_buffer_list; /* Global list of buffers. */
> +
> + /* Index in the kernel_id idr so that we can find the
> + * mmal_msg_context again when servicing the VCHI reply.
> + */
> + int kernel_id;
> +
> + size_t size;
> +
> + /* Lock over all the following state for this buffer */
> + struct mutex lock;
> + struct list_head attachments;
> +
> + char name[VC_SM_MAX_NAME_LEN];
> +
> + int in_use:1; /* Kernel is still using this resource */
> +
> + enum vc_sm_vpu_mapping_state vpu_state;
> + u32 vc_handle; /* VideoCore handle for this buffer */
> +
> + /* DMABUF related fields */
> + struct dma_buf *dma_buf;
> + dma_addr_t dma_addr;
> + void *cookie;
> +
> + struct vc_sm_privdata_t *private;
> +
> + struct dma_buf *imported_dma_buf;
> + struct dma_buf_attachment *attach;
> + struct sg_table *sgt;
> +};
> +
> +#endif
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> new file mode 100644
> index 000000000000..08f70e955acf
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> @@ -0,0 +1,507 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * VideoCore Shared Memory CMA allocator
> + *
> + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> + * Copyright 2011-2012 Broadcom Corporation. All rights reserved.
> + *
> + * Based on vmcs_sm driver from Broadcom Corporation.
> + *
> + */
> +
> +/* ---- Include Files ----------------------------------------------------- */
> +#include <linux/kthread.h>
> +#include <linux/list.h>
> +#include <linux/semaphore.h>
> +#include <linux/slab.h>
> +#include <linux/types.h>
> +
> +#include "vc_sm_cma_vchi.h"
> +
> +#define VC_SM_VER 1
> +#define VC_SM_MIN_VER 0
> +
> +/* ---- Private Constants and Types -------------------------------------- */
> +
> +/* Command blocks come from a pool */
> +#define SM_MAX_NUM_CMD_RSP_BLKS 32
> +
> +/* The number of supported connections */
> +#define SM_MAX_NUM_CONNECTIONS 3
> +
> +struct sm_cmd_rsp_blk {
> + struct list_head head; /* To create lists */
> + /* To be signaled when the response is there */
> + struct completion cmplt;
> +
> + u32 id;
> + u16 length;
> +
> + u8 msg[VC_SM_MAX_MSG_LEN];
> +
> + uint32_t wait:1;
> + uint32_t sent:1;
> + uint32_t alloc:1;
> +
> +};
> +
> +struct sm_instance {
> + u32 num_connections;
> + unsigned int service_handle[SM_MAX_NUM_CONNECTIONS];
> + struct task_struct *io_thread;
> + struct completion io_cmplt;
> +
> + vpu_event_cb vpu_event;
> +
> + /* Mutex over the following lists */
> + struct mutex lock;
> + u32 trans_id;
> + struct list_head cmd_list;
> + struct list_head rsp_list;
> + struct list_head dead_list;
> +
> + struct sm_cmd_rsp_blk free_blk[SM_MAX_NUM_CMD_RSP_BLKS];
> +
> + /* Mutex over the free_list */
> + struct mutex free_lock;
> + struct list_head free_list;
> +
> + struct semaphore free_sema;
> + struct vchiq_instance *vchiq_instance;
> +};
> +
> +/* ---- Private Variables ------------------------------------------------ */
> +
> +/* ---- Private Function Prototypes -------------------------------------- */
> +
> +/* ---- Private Functions ------------------------------------------------ */
> +static int
> +bcm2835_vchi_msg_queue(struct vchiq_instance *vchiq_instance, unsigned int handle,
> + void *data,
> + unsigned int size)
> +{
> + return vchiq_queue_kernel_message(vchiq_instance, handle, data, size);
> +}
> +
> +static struct
> +sm_cmd_rsp_blk *vc_vchi_cmd_create(struct sm_instance *instance,
> + enum vc_sm_msg_type id, void *msg,
> + u32 size, int wait)
> +{
> + struct sm_cmd_rsp_blk *blk;
> + struct vc_sm_msg_hdr_t *hdr;
> +
> + if (down_interruptible(&instance->free_sema)) {
> + blk = kmalloc(sizeof(*blk), GFP_KERNEL);
> + if (!blk)
> + return NULL;
> +
> + blk->alloc = 1;
Make this bool. s/1/true/.
> + init_completion(&blk->cmplt);
> + } else {
> + mutex_lock(&instance->free_lock);
> + blk =
> + list_first_entry(&instance->free_list,
> + struct sm_cmd_rsp_blk, head);
How do we know that the ->free_list isn't empty?
> + list_del(&blk->head);
> + mutex_unlock(&instance->free_lock);
> + }
> +
> + blk->sent = 0;
> + blk->wait = wait;
> + blk->length = sizeof(*hdr) + size;
> +
> + hdr = (struct vc_sm_msg_hdr_t *)blk->msg;
> + hdr->type = id;
> + mutex_lock(&instance->lock);
> + instance->trans_id++;
> + /*
> + * Retain the top bit for identifying asynchronous events, or VPU cmds.
> + */
> + instance->trans_id &= ~0x80000000;
> + hdr->trans_id = instance->trans_id;
> + blk->id = instance->trans_id;
> + mutex_unlock(&instance->lock);
> +
> + if (size)
> + memcpy(hdr->body, msg, size);
If I remember right, this size check is there to silence a GCC false
positive about array overflows? If so it needs a comment, if not then
it can be removed.
> +
> + return blk;
> +}
> +
> +static void
> +vc_vchi_cmd_delete(struct sm_instance *instance, struct sm_cmd_rsp_blk *blk)
> +{
> + if (blk->alloc) {
> + kfree(blk);
> + return;
> + }
> +
> + mutex_lock(&instance->free_lock);
> + list_add(&blk->head, &instance->free_list);
> + mutex_unlock(&instance->free_lock);
> + up(&instance->free_sema);
> +}
> +
> +static void vc_sm_cma_vchi_rx_ack(struct sm_instance *instance,
> + struct sm_cmd_rsp_blk *cmd,
> + struct vc_sm_result_t *reply,
> + u32 reply_len)
> +{
> + mutex_lock(&instance->lock);
> + list_for_each_entry(cmd,
> + &instance->rsp_list,
> + head) {
> + if (cmd->id == reply->trans_id)
> + break;
> + }
> + mutex_unlock(&instance->lock);
> +
> + if (&cmd->head == &instance->rsp_list) {
> + //pr_debug("%s: received response %u, throw away...",
> + pr_err("%s: received response %u, throw away...",
> + __func__,
> + reply->trans_id);
> + } else if (reply_len > sizeof(cmd->msg)) {
> + pr_err("%s: reply too big (%u) %u, throw away...",
> + __func__, reply_len,
> + reply->trans_id);
> + } else {
> + memcpy(cmd->msg, reply,
> + reply_len);
Delete this unnecessary line break. Generally there are too many line
breaks in this code. See above as well.
> + complete(&cmd->cmplt);
> + }
> +}
> +
> +static int vc_sm_cma_vchi_videocore_io(void *arg)
> +{
> + struct sm_instance *instance = arg;
> + struct sm_cmd_rsp_blk *cmd = NULL, *cmd_tmp;
> + struct vc_sm_result_t *reply;
> + struct vchiq_header *header;
> + s32 status;
> + int svc_use = 1;
bool
> +
> + while (1) {
> + if (svc_use)
> + vchiq_release_service(instance->vchiq_instance,
> + instance->service_handle[0]);
> + svc_use = 0;
> +
> + if (wait_for_completion_interruptible(&instance->io_cmplt))
> + continue;
> + vchiq_use_service(instance->vchiq_instance, instance->service_handle[0]);
> + svc_use = 1;
> +
> + do {
Make this a while (1) loop and not a do while (1) loop.
> + /*
> + * Get new command and move it to response list
> + */
> + mutex_lock(&instance->lock);
> + if (list_empty(&instance->cmd_list)) {
> + /* no more commands to process */
> + mutex_unlock(&instance->lock);
> + break;
> + }
> + cmd = list_first_entry(&instance->cmd_list,
> + struct sm_cmd_rsp_blk, head);
> + list_move(&cmd->head, &instance->rsp_list);
> + cmd->sent = 1;
> + mutex_unlock(&instance->lock);
> + /* Send the command */
> + status =
> + bcm2835_vchi_msg_queue(instance->vchiq_instance,
> + instance->service_handle[0],
> + cmd->msg, cmd->length);
> + if (status) {
> + pr_err("%s: failed to queue message (%d)",
> + __func__, status);
> + }
> +
> + /* If no reply is needed then we're done */
> + if (!cmd->wait) {
> + mutex_lock(&instance->lock);
> + list_del(&cmd->head);
> + mutex_unlock(&instance->lock);
> + vc_vchi_cmd_delete(instance, cmd);
> + continue;
> + }
> +
> + if (status) {
> + complete(&cmd->cmplt);
> + continue;
> + }
> +
> + } while (1);
> +
> + while ((header = vchiq_msg_hold(instance->vchiq_instance,
> + instance->service_handle[0]))) {
> + reply = (struct vc_sm_result_t *)header->data;
> + if (reply->trans_id & 0x80000000) {
> + /* Async event or cmd from the VPU */
> + if (instance->vpu_event)
> + instance->vpu_event(instance, reply,
> + header->size);
> + } else {
> + vc_sm_cma_vchi_rx_ack(instance, cmd, reply,
> + header->size);
> + }
> +
> + vchiq_release_message(instance->vchiq_instance,
> + instance->service_handle[0],
> + header);
> + }
> +
> + /* Go through the dead list and free them */
> + mutex_lock(&instance->lock);
> + list_for_each_entry_safe(cmd, cmd_tmp, &instance->dead_list,
> + head) {
> + list_del(&cmd->head);
> + vc_vchi_cmd_delete(instance, cmd);
> + }
> + mutex_unlock(&instance->lock);
> + }
> +
> + return 0;
> +}
> +
> +static int vc_sm_cma_vchi_callback(struct vchiq_instance *vchiq_instance,
> + enum vchiq_reason reason,
> + struct vchiq_header *header,
> + unsigned int handle, void *userdata)
> +{
> + struct sm_instance *instance = vchiq_get_service_userdata(vchiq_instance, handle);
> +
> + switch (reason) {
> + case VCHIQ_MESSAGE_AVAILABLE:
> + vchiq_msg_queue_push(vchiq_instance, handle, header);
> + complete(&instance->io_cmplt);
> + break;
> +
> + case VCHIQ_SERVICE_CLOSED:
> + pr_info("%s: service CLOSED!!", __func__);
> + default:
> + break;
> + }
> +
> + return 0;
> +}
> +
> +struct sm_instance *vc_sm_cma_vchi_init(struct vchiq_instance *vchiq_instance,
> + unsigned int num_connections,
> + vpu_event_cb vpu_event)
> +{
> + u32 i;
> + struct sm_instance *instance;
> + int status;
> +
> + pr_debug("%s: start", __func__);
Delete. Use ftrace.
> +
> + if (num_connections > SM_MAX_NUM_CONNECTIONS) {
> + pr_err("%s: unsupported number of connections %u (max=%u)",
> + __func__, num_connections, SM_MAX_NUM_CONNECTIONS);
> +
> + goto err_null;
Just return NULL;
> + }
> + /* Allocate memory for this instance */
> + instance = kzalloc(sizeof(*instance), GFP_KERNEL);
> +
> + /* Misc initialisations */
> + mutex_init(&instance->lock);
Check for kzalloc() failure.
regards,
dan carpenter
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 04/15] staging: mmal-vchiq: Add support for event callbacks
2023-11-09 21:02 ` [PATCH v2 04/15] staging: mmal-vchiq: Add support for event callbacks Umang Jain
@ 2023-11-20 10:00 ` Dan Carpenter
0 siblings, 0 replies; 45+ messages in thread
From: Dan Carpenter @ 2023-11-20 10:00 UTC (permalink / raw)
To: Umang Jain
Cc: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging, Dave Stevenson, Kieran Bingham,
Laurent Pinchart, Ricardo B . Marliere, Dan Carpenter,
Stefan Wahren, Dave Stevenson
On Thu, Nov 09, 2023 at 04:02:56PM -0500, Umang Jain wrote:
> +static void event_to_host_cb(struct vchiq_mmal_instance *instance,
> + struct mmal_msg *msg, u32 msg_len)
> +{
> + int comp_idx = msg->u.event_to_host.client_component;
> + struct vchiq_mmal_component *component =
> + &instance->component[comp_idx];
> + struct vchiq_mmal_port *port = NULL;
> + struct mmal_msg_context *msg_context;
> + u32 port_num = msg->u.event_to_host.port_num;
> +
> + if (msg->u.buffer_from_host.drvbuf.magic == MMAL_MAGIC) {
> + pr_err("%s: MMAL_MSG_TYPE_BUFFER_TO_HOST with bad magic\n",
> + __func__);
> + return;
> + }
> +
> + switch (msg->u.event_to_host.port_type) {
> + case MMAL_PORT_TYPE_CONTROL:
> + if (port_num) {
> + pr_err("%s: port_num of %u >= number of ports 1",
This message is confusing. Perhaps:
pr_err("%s: invalid port_num %u (should be zero)", ...
> + __func__, port_num);
> + return;
> + }
> + port = &component->control;
> + break;
> + case MMAL_PORT_TYPE_INPUT:
> + if (port_num >= component->inputs) {
> + pr_err("%s: port_num of %u >= number of ports %u",
pr_err("%s: port_num (%u) >= component->inputs (%u)",
> + __func__, port_num,
> + port_num >= component->inputs);
> + return;
> + }
> + port = &component->input[port_num];
> + break;
> + case MMAL_PORT_TYPE_OUTPUT:
> + if (port_num >= component->outputs) {
> + pr_err("%s: port_num of %u >= number of ports %u",
pr_err("%s: port_num (%u) >= component->outputs (%u)",
> + __func__, port_num,
> + port_num >= component->outputs);
> + return;
> + }
> + port = &component->output[port_num];
> + break;
> + case MMAL_PORT_TYPE_CLOCK:
> + if (port_num >= component->clocks) {
> + pr_err("%s: port_num of %u >= number of ports %u",
> + __func__, port_num,
> + port_num >= component->clocks);
> + return;
> + }
> + port = &component->clock[port_num];
> + break;
> + default:
> + break;
> + }
> +
> + if (!mutex_trylock(&port->event_context_mutex)) {
> + pr_err("dropping event 0x%x\n", msg->u.event_to_host.cmd);
> + return;
> + }
> + msg_context = port->event_context;
> +
> + if (msg->h.status != MMAL_MSG_STATUS_SUCCESS) {
> + /* message reception had an error */
> + pr_err("%s: error %d in reply\n", __func__, msg->h.status);
> +
> + msg_context->u.bulk.status = msg->h.status;
> + } else if (msg->u.event_to_host.length > MMAL_WORKER_EVENT_SPACE) {
> + /* data is not in message, queue a bulk receive */
> + pr_err("%s: payload not in message - bulk receive??! NOT SUPPORTED\n",
> + __func__);
> + msg_context->u.bulk.status = -1;
> + } else {
> + memcpy(msg_context->u.bulk.buffer->buffer,
> + msg->u.event_to_host.data,
> + msg->u.event_to_host.length);
> +
> + msg_context->u.bulk.buffer_used =
> + msg->u.event_to_host.length;
> +
> + msg_context->u.bulk.mmal_flags = 0;
> + msg_context->u.bulk.dts = MMAL_TIME_UNKNOWN;
> + msg_context->u.bulk.pts = MMAL_TIME_UNKNOWN;
> + msg_context->u.bulk.cmd = msg->u.event_to_host.cmd;
> +
> + pr_debug("event component:%u port type:%d num:%d cmd:0x%x length:%d\n",
> + msg->u.event_to_host.client_component,
> + msg->u.event_to_host.port_type,
> + msg->u.event_to_host.port_num,
> + msg->u.event_to_host.cmd, msg->u.event_to_host.length);
> + }
> +
> + schedule_work(&msg_context->u.bulk.work);
> +}
> +
> /* deals with receipt of buffer to host message */
> static void buffer_to_host_cb(struct vchiq_mmal_instance *instance,
> struct mmal_msg *msg, u32 msg_len)
> @@ -1329,6 +1421,7 @@ static int port_disable(struct vchiq_mmal_instance *instance,
> mmalbuf->mmal_flags = 0;
> mmalbuf->dts = MMAL_TIME_UNKNOWN;
> mmalbuf->pts = MMAL_TIME_UNKNOWN;
> + mmalbuf->cmd = 0;
> port->buffer_cb(instance,
> port, 0, mmalbuf);
> }
> @@ -1630,6 +1723,43 @@ int mmal_vchi_buffer_cleanup(struct mmal_buffer *buf)
> }
> EXPORT_SYMBOL_GPL(mmal_vchi_buffer_cleanup);
>
> +static void init_event_context(struct vchiq_mmal_instance *instance,
> + struct vchiq_mmal_port *port)
> +{
> + struct mmal_msg_context *ctx = get_msg_context(instance);
> +
> + mutex_init(&port->event_context_mutex);
> +
> + port->event_context = ctx;
> + ctx->u.bulk.instance = instance;
> + ctx->u.bulk.port = port;
> + ctx->u.bulk.buffer =
> + kzalloc(sizeof(*ctx->u.bulk.buffer), GFP_KERNEL);
This is 79 chars. Delete the line break.
> + if (!ctx->u.bulk.buffer)
> + goto release_msg_context;
> + ctx->u.bulk.buffer->buffer = kzalloc(MMAL_WORKER_EVENT_SPACE,
> + GFP_KERNEL);
regards,
dan carpenter
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH v2 10/15] staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component
2023-11-09 21:03 ` [PATCH v2 10/15] staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component Umang Jain
2023-11-13 14:07 ` [PATCH] fixup! " Umang Jain
2023-11-18 14:53 ` [PATCH v2 10/15] " Stefan Wahren
@ 2023-11-20 10:33 ` Dan Carpenter
2 siblings, 0 replies; 45+ messages in thread
From: Dan Carpenter @ 2023-11-20 10:33 UTC (permalink / raw)
To: Umang Jain
Cc: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging, Dave Stevenson, Kieran Bingham,
Laurent Pinchart, Ricardo B . Marliere, Dan Carpenter,
Stefan Wahren, Naushir Patuck
On Thu, Nov 09, 2023 at 04:03:02PM -0500, Umang Jain wrote:
> +static int media_controller_register(struct bcm2835_isp_dev *dev)
> +{
> + char *name;
> + unsigned int i;
> + int ret;
> +
> + v4l2_dbg(2, debug, &dev->v4l2_dev, "Registering with media controller\n");
> + dev->mdev.dev = dev->dev;
> + strscpy(dev->mdev.model, "bcm2835-isp",
> + sizeof(dev->mdev.model));
> + strscpy(dev->mdev.bus_info, "platform:bcm2835-isp",
> + sizeof(dev->mdev.bus_info));
> + media_device_init(&dev->mdev);
> + dev->v4l2_dev.mdev = &dev->mdev;
> +
> + v4l2_dbg(2, debug, &dev->v4l2_dev, "Register entity for nodes\n");
> +
> + name = kmalloc(BCM2835_ISP_ENTITY_NAME_LEN, GFP_KERNEL);
> + if (!name) {
> + ret = -ENOMEM;
> + goto done;
Oh crap. This function doesn't clean up after itself, but instead
returns to a One Magical Cleanup Function... This style of error
handling is *ALWAYS* buggy. But in this case I only see a very minor
leak.
These two cleanups are under one ->registered flag but they are
allocated separately. So if we only complete one action and not the
second then the flag is not set and we don't call media_device_cleanup().
(This is one of many typical problems with One Magical Cleanup Function
Style).
> + if (dev->media_device_registered) {
> + media_device_unregister(&dev->mdev);
> + media_device_cleanup(&dev->mdev);
> + dev->media_device_registered = false;
> + }
It's just such a headache to review... (That's why it's so bug prone).
If you wrote it in Free the Last Thing Style then you could get rid
of a bunch of flags like ->media_device_registered and
->media_entity_registered because at that point you would always know
what you had done and hadn't done.
regards,
dan carpenter
> + }
> + snprintf(name, BCM2835_ISP_ENTITY_NAME_LEN, "bcm2835_isp0");
> + dev->entity.name = name;
> + dev->entity.obj_type = MEDIA_ENTITY_TYPE_BASE;
> + dev->entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
> +
> + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
> + dev->pad[i].flags = node_is_output(&dev->node[i]) ?
> + MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> + }
> +
> + ret = media_entity_pads_init(&dev->entity, BCM2835_ISP_NUM_NODES,
> + dev->pad);
> + if (ret)
> + goto done;
> +
> + ret = media_device_register_entity(&dev->mdev, &dev->entity);
> + if (ret)
> + goto done;
> +
> + dev->media_entity_registered = true;
> + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
> ret = media_controller_register_node(dev, i);
> + if (ret)
> + goto done;
> + }
> +
> + ret = media_device_register(&dev->mdev);
> + if (!ret)
> + dev->media_device_registered = true;
> +done:
> + return ret;
> +}
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH] fixup! staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component
2023-11-13 14:07 ` [PATCH] fixup! " Umang Jain
2023-11-13 15:25 ` Stefan Wahren
@ 2023-11-23 12:56 ` Greg KH
2023-11-23 13:53 ` Laurent Pinchart
1 sibling, 1 reply; 45+ messages in thread
From: Greg KH @ 2023-11-23 12:56 UTC (permalink / raw)
To: Umang Jain
Cc: linux-media, kernel-list, linux-kernel, linux-rpi-kernel,
linux-arm-kernel, linux-staging, Dave Stevenson,
Laurent Pinchart, Ricardo B . Marliere, Dan Carpenter,
Stefan Wahren
On Mon, Nov 13, 2023 at 09:07:55AM -0500, Umang Jain wrote:
> Setup DMA Mask
> ---
> I realised while (re)creating the test branches (sent in reply to the cover
> letter), I missed to squash this hunk in
> "staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component"
> before sending out the series.
>
> Sending it as a fixup! for now.
What does that mean? What can I do with this?
confused,
greg k-h
^ permalink raw reply [flat|nested] 45+ messages in thread
* Re: [PATCH] fixup! staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component
2023-11-23 12:56 ` Greg KH
@ 2023-11-23 13:53 ` Laurent Pinchart
0 siblings, 0 replies; 45+ messages in thread
From: Laurent Pinchart @ 2023-11-23 13:53 UTC (permalink / raw)
To: Umang Jain
Cc: Greg KH, linux-media, kernel-list, linux-kernel,
linux-rpi-kernel, linux-arm-kernel, linux-staging,
Dave Stevenson, Ricardo B . Marliere, Dan Carpenter,
Stefan Wahren
On Thu, Nov 23, 2023 at 12:56:07PM +0000, Greg KH wrote:
> On Mon, Nov 13, 2023 at 09:07:55AM -0500, Umang Jain wrote:
> > Setup DMA Mask
> > ---
> > I realised while (re)creating the test branches (sent in reply to the cover
> > letter), I missed to squash this hunk in
> > "staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component"
> > before sending out the series.
> >
> > Sending it as a fixup! for now.
>
> What does that mean? What can I do with this?
Umang, we don't do that for upstream kernel development. If a patch
needs a new version, you should send a new version. In some subsystems
maintainers will prefer if you only resubmit the offending patch when
the series is large (as a "vN.1" instead of a vN+1), while in other
cases resending the whole series is preferred.
Greg, I don't think Umang wanted you to do anything with this patch :-)
This was meant to tell reviewer that the issue was known already.
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 45+ messages in thread
end of thread, other threads:[~2023-11-23 13:53 UTC | newest]
Thread overview: 45+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-11-09 21:02 [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Umang Jain
2023-11-09 21:02 ` [PATCH v2 01/15] staging: vc04_services: Add new vc-sm-cma driver Umang Jain
2023-11-13 17:56 ` Stefan Wahren
2023-11-15 19:56 ` Laurent Pinchart
2023-11-15 20:48 ` Stefan Wahren
2023-11-20 9:52 ` Dan Carpenter
2023-11-09 21:02 ` [PATCH v2 02/15] staging: vc04_services: vchiq_arm: Register vcsm-cma driver Umang Jain
2023-11-14 19:32 ` Stefan Wahren
2023-11-09 21:02 ` [PATCH v2 03/15] media: videobuf2: Allow exporting of a struct dmabuf Umang Jain
2023-11-14 19:41 ` Stefan Wahren
2023-11-15 19:31 ` Laurent Pinchart
2023-11-15 21:00 ` Stefan Wahren
2023-11-09 21:02 ` [PATCH v2 04/15] staging: mmal-vchiq: Add support for event callbacks Umang Jain
2023-11-20 10:00 ` Dan Carpenter
2023-11-09 21:02 ` [PATCH v2 05/15] staging: mmal-vchiq: Use vc-sm-cma to support zero copy Umang Jain
2023-11-15 19:03 ` Stefan Wahren
2023-11-15 19:27 ` Laurent Pinchart
2023-11-09 21:02 ` [PATCH v2 06/15] staging: mmal_vchiq: Add image formats to be used by bcm2835-isp Umang Jain
2023-11-09 21:02 ` [PATCH v2 07/15] media: uapi: v4l2-core: Add ISP statistics output V4L2 fourcc type Umang Jain
2023-11-18 13:11 ` Stefan Wahren
2023-11-09 21:03 ` [PATCH v2 08/15] uapi: bcm2835-isp: Add bcm2835-isp uapi header file Umang Jain
2023-11-09 21:03 ` [PATCH v2 09/15] staging: vc04_services: Add helpers for vchiq driver data Umang Jain
2023-11-18 14:43 ` Stefan Wahren
2023-11-09 21:03 ` [PATCH v2 10/15] staging: vc04_services: bcm2835-isp: Add a more complex ISP processing component Umang Jain
2023-11-13 14:07 ` [PATCH] fixup! " Umang Jain
2023-11-13 15:25 ` Stefan Wahren
2023-11-23 12:56 ` Greg KH
2023-11-23 13:53 ` Laurent Pinchart
2023-11-18 14:53 ` [PATCH v2 10/15] " Stefan Wahren
2023-11-18 18:56 ` Laurent Pinchart
2023-11-19 12:02 ` Stefan Wahren
2023-11-20 10:33 ` Dan Carpenter
2023-11-09 21:03 ` [PATCH v2 11/15] vc04_services: bcm2835-isp: Allow formats with different colour spaces Umang Jain
2023-11-09 21:03 ` [PATCH v2 12/15] vc04_services: bcm2835-isp: Permit all sRGB colour spaces on ISP outputs Umang Jain
2023-11-09 21:03 ` [PATCH v2 13/15] staging: vc04_services: bcm2835_isp: Allow multiple users Umang Jain
2023-11-09 21:03 ` [PATCH v2 14/15] docs: admin-guide: media: bcm2835-isp: Add documentation for bcm2835-isp Umang Jain
2023-11-09 21:03 ` [PATCH v2 15/15] staging: vc04_services: vchiq: Register bcm2835-isp Umang Jain
2023-11-10 10:32 ` [PATCH v2 00/15] staging: vc04_services: bcm2835-isp support Stefan Wahren
2023-11-13 13:38 ` Umang Jain
2023-11-13 15:09 ` Stefan Wahren
2023-11-15 19:59 ` Laurent Pinchart
2023-11-15 20:57 ` Stefan Wahren
2023-11-16 0:50 ` Laurent Pinchart
2023-11-16 10:54 ` Ivan T. Ivanov
2023-11-16 10:57 ` Peter Robinson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).