* [PATCH 01/10] dt-bindings: media: Add bindings for Keem Bay Camera
2021-03-19 18:06 [PATCH 00/10] Keem Bay Camera Subsystem Martina Krasteva
@ 2021-03-19 18:06 ` Martina Krasteva
2021-03-19 21:49 ` Rob Herring
2021-03-19 18:06 ` [PATCH 02/10] media: Keem Bay Camera: Keem Bay camera driver Martina Krasteva
` (9 subsequent siblings)
10 siblings, 1 reply; 28+ messages in thread
From: Martina Krasteva @ 2021-03-19 18:06 UTC (permalink / raw)
To: linux-media
Cc: mchehab, robh+dt, devicetree, sakari.ailus,
daniele.alessandrelli, paul.j.murphy, gjorgjix.rosikopulos,
martinax.krasteva
From: Martina Krasteva <martinax.krasteva@intel.com>
- Add dt-bindings documentation for Intel Keem Bay Camera driver.
- Add MAINTAINERS entry for Intel Keem Bay Camera binding
documentation.
Co-developed-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
---
.../bindings/media/intel,keembay-camera.yaml | 98 ++++++++++++++++++++++
MAINTAINERS | 8 ++
2 files changed, 106 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/intel,keembay-camera.yaml
diff --git a/Documentation/devicetree/bindings/media/intel,keembay-camera.yaml b/Documentation/devicetree/bindings/media/intel,keembay-camera.yaml
new file mode 100644
index 000000000000..78242b05228d
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/intel,keembay-camera.yaml
@@ -0,0 +1,98 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+# Copyright (C) 2021 Intel Corporation
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/intel,keembay-camera.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Intel Keem Bay camera subsystem
+
+maintainers:
+ - Paul J. Murphy <paul.j.murphy@intel.com>
+ - Daniele Alessandrelli <daniele.alessandrelli@intel.com>
+
+properties:
+ compatible:
+ const: intel,keembay-camera
+ memory-region:
+ $ref: /schemas/types.yaml#/definitions/phandle
+
+ ports:
+ type: object
+ $ref: /schemas/graph.yaml#/properties/ports
+
+ "#address-cells":
+ const: 1
+ "#size-cells":
+ const: 0
+
+ properties:
+ port@[0-5]:
+ type: object
+ additionalProperties: false
+ $ref: /schemas/graph.yaml#/properties/port
+ description:
+ The port number matches the D-PHY number (D-PHY#0 - D-PHY#5).
+
+ properties:
+ endpoint:
+ type: object
+
+ properties:
+ data-lanes:
+ $ref: video-interfaces.yaml#/properties/data-lanes
+ description:
+ Six two-lane d-phys (D-PHY#0 - D-PHY#5) are available, which
+ can be used by six RX controllers (RX-CTRL#0 - RX-CTRL#5).
+ RX-CTRL#0, RX-CTRL#2, RX-CTRL#4 can be connected to two
+ D-PHY's and will be able to work with 3 and 4 lanes. In this
+ case the RX-CTRLs mapped to those D-PHYs cannot be used.
+
+ Clock and data lanes are defined as follows
+ D-PHY#0 - clock - 0, data - 1, 2
+ D-PHY#1 - clock - 3, data - 4, 5
+ D-PHY#2 - clock - 6, data - 7, 8
+ D-PHY#3 - clock - 9, data - 10, 11
+ D-PHY#4 - clock - 12, data - 13, 14
+ D-PHY#5 - clock - 15, data - 16, 17
+
+ required:
+ - data-lanes
+
+ required:
+ - reg
+ - endpoint
+
+ required:
+ - "#address-cells"
+ - "#size-cells"
+ - port@[0-5]
+
+required:
+ - compatible
+ - ports
+
+
+additionalProperties: false
+
+examples:
+ - |
+ keembay_camera {
+ compatible = "intel,keembay-camera";
+ memory-region = <&mem>;
+
+ ports {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ port@4 {
+ reg = <4>;
+
+ cam: endpoint {
+ remote-endpoint = <&imx334>;
+ data-lanes = <13 14 16 17>;
+ };
+ };
+ };
+ };
+...
diff --git a/MAINTAINERS b/MAINTAINERS
index 08f9c2b7f3b3..c3f583ef8e46 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1965,6 +1965,14 @@ M: Lennert Buytenhek <kernel@wantstofly.org>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
S: Maintained
+ARM/INTEL KEEM BAY CAMERA SUBSYSTEM
+M: Paul J. Murphy <paul.j.murphy@intel.com>
+M: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
+L: linux-media@vger.kernel.org
+S: Maintained
+T: git git://linuxtv.org/media_tree.git
+F: Documentation/devicetree/bindings/media/intel,keembay-camera.yaml
+
ARM/IP FABRICS DOUBLE ESPRESSO MACHINE SUPPORT
M: Lennert Buytenhek <kernel@wantstofly.org>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread* Re: [PATCH 01/10] dt-bindings: media: Add bindings for Keem Bay Camera
2021-03-19 18:06 ` [PATCH 01/10] dt-bindings: media: Add bindings for Keem Bay Camera Martina Krasteva
@ 2021-03-19 21:49 ` Rob Herring
0 siblings, 0 replies; 28+ messages in thread
From: Rob Herring @ 2021-03-19 21:49 UTC (permalink / raw)
To: Martina Krasteva
Cc: linux-media, gjorgjix.rosikopulos, robh+dt, paul.j.murphy,
mchehab, devicetree, sakari.ailus, daniele.alessandrelli
On Fri, 19 Mar 2021 18:06:23 +0000, Martina Krasteva wrote:
> From: Martina Krasteva <martinax.krasteva@intel.com>
>
> - Add dt-bindings documentation for Intel Keem Bay Camera driver.
> - Add MAINTAINERS entry for Intel Keem Bay Camera binding
> documentation.
>
> Co-developed-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
> Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
> Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
> Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
> Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
> ---
> .../bindings/media/intel,keembay-camera.yaml | 98 ++++++++++++++++++++++
> MAINTAINERS | 8 ++
> 2 files changed, 106 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/media/intel,keembay-camera.yaml
>
My bot found errors running 'make dt_binding_check' on your patch:
yamllint warnings/errors:
dtschema/dtc warnings/errors:
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/media/intel,keembay-camera.yaml: properties:ports:properties:required: ['reg', 'endpoint'] is not of type 'object', 'boolean'
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/media/intel,keembay-camera.yaml: properties:ports:required:2: 'port@[0-5]' does not match '^([a-zA-Z#][a-zA-Z0-9,+\\-._@]{0,63}|\\$nodename)$'
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/media/intel,keembay-camera.yaml: properties:ports: Additional properties are not allowed ('#size-cells', '#address-cells' were unexpected)
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/media/intel,keembay-camera.yaml: properties:ports: '#address-cells' is not one of ['$ref', 'additionalItems', 'additionalProperties', 'allOf', 'anyOf', 'const', 'contains', 'default', 'dependencies', 'deprecated', 'description', 'else', 'enum', 'exclusiveMaximum', 'exclusiveMinimum', 'items', 'if', 'minItems', 'minimum', 'maxItems', 'maximum', 'multipleOf', 'not', 'oneOf', 'pattern', 'patternProperties', 'properties', 'required', 'then', 'type', 'typeSize', 'unevaluatedProperties', 'uniqueItems']
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/media/intel,keembay-camera.yaml: properties:ports: '#size-cells' is not one of ['$ref', 'additionalItems', 'additionalProperties', 'allOf', 'anyOf', 'const', 'contains', 'default', 'dependencies', 'deprecated', 'description', 'else', 'enum', 'exclusiveMaximum', 'exclusiveMinimum', 'items', 'if', 'minItems', 'minimum', 'maxItems', 'maximum', 'multipleOf', 'not', 'oneOf', 'pattern', 'patternProperties', 'properties', 'required', 'then', 'type', 'typeSize', 'unevaluatedProperties', 'uniqueItems']
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/media/intel,keembay-camera.yaml: properties:ports:properties: {'enum': ['$ref', 'additionalItems', 'additionalProperties', 'allOf', 'anyOf', 'const', 'contains', 'default', 'dependencies', 'deprecated', 'description', 'else', 'enum', 'if', 'items', 'maxItems', 'maximum', 'minItems', 'minimum', 'multipleOf', 'not', 'oneOf', 'pattern', 'patternProperties', 'properties', 'propertyNames', 'required', 'then', 'unevaluatedProperties']} is not allowed for 'required'
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/media/intel,keembay-camera.yaml: properties:ports: '#address-cells' is not one of ['type', 'description', 'dependencies', 'properties', 'patternProperties', 'additionalProperties', 'unevaluatedProperties', 'deprecated', 'required', 'allOf', 'anyOf', 'oneOf', '$ref']
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/media/intel,keembay-camera.yaml: properties:ports: '#size-cells' is not one of ['type', 'description', 'dependencies', 'properties', 'patternProperties', 'additionalProperties', 'unevaluatedProperties', 'deprecated', 'required', 'allOf', 'anyOf', 'oneOf', '$ref']
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/media/intel,keembay-camera.yaml: ignoring, error in schema: properties: ports: properties: required
warning: no schema found in file: ./Documentation/devicetree/bindings/media/intel,keembay-camera.yaml
See https://patchwork.ozlabs.org/patch/1455960
This check can fail if there are any dependencies. The base for a patch
series is generally the most recent rc1.
If you already ran 'make dt_binding_check' and didn't see the above
error(s), then make sure 'yamllint' is installed and dt-schema is up to
date:
pip3 install dtschema --upgrade
Please check and re-submit.
^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 02/10] media: Keem Bay Camera: Keem Bay camera driver
2021-03-19 18:06 [PATCH 00/10] Keem Bay Camera Subsystem Martina Krasteva
2021-03-19 18:06 ` [PATCH 01/10] dt-bindings: media: Add bindings for Keem Bay Camera Martina Krasteva
@ 2021-03-19 18:06 ` Martina Krasteva
2021-03-19 18:06 ` [PATCH 03/10] media: Keem Bay Camera: Add VPU camera interface Martina Krasteva
` (8 subsequent siblings)
10 siblings, 0 replies; 28+ messages in thread
From: Martina Krasteva @ 2021-03-19 18:06 UTC (permalink / raw)
To: linux-media
Cc: mchehab, robh+dt, devicetree, sakari.ailus,
daniele.alessandrelli, paul.j.murphy, gjorgjix.rosikopulos,
martinax.krasteva
From: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
Keem Bay camera driver is a v4l2 media device
which implements media controller interface. It has one
sub-device (ISP sub-device) endpoint which can be connected
to a remote CSI2 TX endpoint. The camera supports up to 6 ports,
each of which is separate CSI2-PHY interface.
Camera driver uses Xlink for communication with remote VPU camera.
The Xlink camera protocol uses one control channel for pipeline
management and configuration, one ISP channel for ISP control and
one channel for each endpoint.
Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
Co-developed-by: Martina Krasteva <martinax.krasteva@intel.com>
Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
---
MAINTAINERS | 1 +
drivers/media/platform/Kconfig | 1 +
drivers/media/platform/Makefile | 2 +
drivers/media/platform/keembay-camera/Kconfig | 11 +
drivers/media/platform/keembay-camera/Makefile | 3 +
.../platform/keembay-camera/keembay-cam-xlink.c | 54 ++++
.../platform/keembay-camera/keembay-cam-xlink.h | 31 +++
.../media/platform/keembay-camera/keembay-camera.c | 287 +++++++++++++++++++++
.../media/platform/keembay-camera/keembay-camera.h | 43 +++
.../media/platform/keembay-camera/keembay-isp.c | 53 ++++
.../media/platform/keembay-camera/keembay-isp.h | 90 +++++++
11 files changed, 576 insertions(+)
create mode 100644 drivers/media/platform/keembay-camera/Kconfig
create mode 100644 drivers/media/platform/keembay-camera/Makefile
create mode 100644 drivers/media/platform/keembay-camera/keembay-cam-xlink.c
create mode 100644 drivers/media/platform/keembay-camera/keembay-cam-xlink.h
create mode 100644 drivers/media/platform/keembay-camera/keembay-camera.c
create mode 100644 drivers/media/platform/keembay-camera/keembay-camera.h
create mode 100644 drivers/media/platform/keembay-camera/keembay-isp.c
create mode 100644 drivers/media/platform/keembay-camera/keembay-isp.h
diff --git a/MAINTAINERS b/MAINTAINERS
index c3f583ef8e46..76082714a76f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1972,6 +1972,7 @@ L: linux-media@vger.kernel.org
S: Maintained
T: git git://linuxtv.org/media_tree.git
F: Documentation/devicetree/bindings/media/intel,keembay-camera.yaml
+F: drivers/media/platform/keembay-camera/
ARM/IP FABRICS DOUBLE ESPRESSO MACHINE SUPPORT
M: Lennert Buytenhek <kernel@wantstofly.org>
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 1ddb5d6354cf..6dedbf54d89a 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -171,6 +171,7 @@ source "drivers/media/platform/xilinx/Kconfig"
source "drivers/media/platform/rcar-vin/Kconfig"
source "drivers/media/platform/atmel/Kconfig"
source "drivers/media/platform/sunxi/Kconfig"
+source "drivers/media/platform/keembay-camera/Kconfig"
config VIDEO_TI_CAL
tristate "TI CAL (Camera Adaptation Layer) driver"
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 9d4d6370908d..f67db3de5864 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -84,3 +84,5 @@ obj-$(CONFIG_VIDEO_QCOM_VENUS) += qcom/venus/
obj-y += sunxi/
obj-$(CONFIG_VIDEO_MESON_GE2D) += meson/ge2d/
+
+obj-$(CONFIG_VIDEO_INTEL_KEEMBAY_CAMERA) += keembay-camera/
diff --git a/drivers/media/platform/keembay-camera/Kconfig b/drivers/media/platform/keembay-camera/Kconfig
new file mode 100644
index 000000000000..691348ed5d4a
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/Kconfig
@@ -0,0 +1,11 @@
+config VIDEO_INTEL_KEEMBAY_CAMERA
+ tristate "Intel Keem Bay camera subsystem"
+ depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
+ depends on XLINK_CORE && HAS_DMA && OF
+ select VIDEOBUF2_DMA_CONTIG
+ select V4L2_FWNODE
+ help
+ This is a V4L2 driver for the Intel Keem Bay VPU camera interface.
+
+ To compile this driver as a module, choose M here: the module
+ will be called keembay_cam
diff --git a/drivers/media/platform/keembay-camera/Makefile b/drivers/media/platform/keembay-camera/Makefile
new file mode 100644
index 000000000000..ea19b61e6469
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/Makefile
@@ -0,0 +1,3 @@
+keembay-cam-objs = keembay-camera.o keembay-cam-xlink.o keembay-isp.o
+
+obj-$(CONFIG_VIDEO_INTEL_KEEMBAY_CAMERA) += keembay-cam.o
diff --git a/drivers/media/platform/keembay-camera/keembay-cam-xlink.c b/drivers/media/platform/keembay-camera/keembay-cam-xlink.c
new file mode 100644
index 000000000000..e34516de0ff9
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-cam-xlink.c
@@ -0,0 +1,54 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Intel Keem Bay camera xlink
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+#include <linux/device.h>
+#include <linux/idr.h>
+
+#include "keembay-cam-xlink.h"
+
+/**
+ * kmb_cam_xlink_init - Initialize xlink for VPU camera communication
+ * @xlink_cam: Pointer to xlink camera handle
+ * @dev: Client device of the xlink
+ *
+ * Perform initialization and establish connection with xlink VPUIP device
+ *
+ * Return: 0 if successful, error code otherwise
+ */
+int kmb_cam_xlink_init(struct kmb_xlink_cam *xlink_cam, struct device *dev)
+{
+ int ret;
+
+ /* Connect to the device before opening channels */
+ memset(&xlink_cam->handle, 0, sizeof(xlink_cam->handle));
+ xlink_cam->handle.dev_type = VPUIP_DEVICE;
+ ret = xlink_connect(&xlink_cam->handle);
+ if (ret) {
+ dev_err(xlink_cam->dev, "Failed to connect: %d\n", ret);
+ return ret;
+ }
+
+ ida_init(&xlink_cam->channel_ids);
+ xlink_cam->ctrl_chan_refcnt = 0;
+
+ mutex_init(&xlink_cam->lock);
+ xlink_cam->dev = dev;
+
+ return 0;
+}
+
+/**
+ * kmb_cam_xlink_cleanup - Cleanup xlink camera communication
+ * @xlink_cam: Pointer to xlink camera handle
+ *
+ * Return: 0 if successful, error code otherwise
+ */
+void kmb_cam_xlink_cleanup(struct kmb_xlink_cam *xlink_cam)
+{
+ /* Disconnect from the device after closing channels */
+ xlink_disconnect(&xlink_cam->handle);
+ ida_destroy(&xlink_cam->channel_ids);
+}
diff --git a/drivers/media/platform/keembay-camera/keembay-cam-xlink.h b/drivers/media/platform/keembay-camera/keembay-cam-xlink.h
new file mode 100644
index 000000000000..d219d4298427
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-cam-xlink.h
@@ -0,0 +1,31 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Intel Keem Bay camera xlink
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+#ifndef KEEMBAY_CAM_XLINK_H
+#define KEEMBAY_CAM_XLINK_H
+
+#include <linux/xlink.h>
+
+/**
+ * struct kmb_xlink_cam - KMB Camera xlink communication
+ * @dev: Device client of the xlink
+ * @lock: Mutex to serialize access to kmb xlink communication channels
+ * @handle: Xlink handle
+ * @ctrl_chan_refcnt: Main control channel reference count
+ * @channel_ids: Channel IDs. Each channel should have unique ID
+ */
+struct kmb_xlink_cam {
+ struct device *dev;
+ struct mutex lock;
+ struct xlink_handle handle;
+ unsigned int ctrl_chan_refcnt;
+ struct ida channel_ids;
+};
+
+int kmb_cam_xlink_init(struct kmb_xlink_cam *xlink_cam, struct device *dev);
+void kmb_cam_xlink_cleanup(struct kmb_xlink_cam *xlink_cam);
+
+#endif /* KEEMBAY_CAM_XLINK_H */
diff --git a/drivers/media/platform/keembay-camera/keembay-camera.c b/drivers/media/platform/keembay-camera/keembay-camera.c
new file mode 100644
index 000000000000..bc23df53473e
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-camera.c
@@ -0,0 +1,287 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Intel Keem Bay camera driver.
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+#include <linux/dma-mapping.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/platform_device.h>
+
+#include <media/v4l2-fwnode.h>
+
+#include "keembay-camera.h"
+
+#define KMB_CAM_NUM_PORTS 6
+
+/* RX-CTRL to data lanes mapping
+ * 2-lanes
+ * RX-CTRL#0 - {1, 2}
+ * RX-CTRL#1 - {4, 5}
+ * RX-CTRL#2 - {7, 8}
+ * RX-CTRL#3 - {10, 11}
+ * RX-CTRL#4 - {13, 14}
+ * RX-CTRL#5 - {16, 17}
+ * 4-lanes
+ * RX-CTRL#0 - {1, 2, 4, 5}
+ * RX-CTRL#2 - {7, 8, 10, 11}
+ * RX-CTRL#4 - {13, 14, 16, 17}
+ */
+static const u8 rx_ctrl[KMB_CAM_NUM_PORTS][2][4] = {
+ { { 1, 2 }, { 1, 2, 4, 5 } },
+ { { 4, 5 }, {} },
+ { { 7, 8 }, { 7, 8, 10, 11 } },
+ { { 10, 11 }, {} },
+ { { 13, 14 }, { 13, 14, 16, 17 } },
+ { { 16, 17 }, {} }
+};
+
+static int get_rx_id(const u8 data_lanes[],
+ const unsigned short num_data_lanes)
+{
+ unsigned int i, j;
+
+ for (i = 0; i < KMB_CAM_NUM_PORTS; i++) {
+ for (j = 0; j < ARRAY_SIZE(rx_ctrl[i]); j++) {
+ if (!memcmp(data_lanes, rx_ctrl[i][j],
+ num_data_lanes * sizeof(u8)))
+ return i;
+ }
+ }
+
+ return -EINVAL;
+}
+
+static int kmb_cam_bound(struct v4l2_async_notifier *n,
+ struct v4l2_subdev *sd,
+ struct v4l2_async_subdev *asd)
+{
+ struct v4l2_device *v4l2_dev = n->v4l2_dev;
+ struct kmb_camera *kmb_cam =
+ container_of(v4l2_dev, struct kmb_camera, v4l2_dev);
+ struct kmb_camera_receiver *receiver =
+ container_of(asd, struct kmb_camera_receiver, asd);
+ int ret;
+
+ ret = kmb_isp_init(&receiver->isp, kmb_cam->dev,
+ &receiver->csi2_config, &kmb_cam->xlink_cam);
+ if (ret < 0)
+ return ret;
+
+ ret = kmb_isp_register_entities(&receiver->isp, &kmb_cam->v4l2_dev);
+ if (ret < 0)
+ goto error_isp_cleanup;
+
+ ret = media_create_pad_link(&sd->entity, 0,
+ &receiver->isp.subdev.entity,
+ KMB_ISP_SINK_PAD_SENSOR,
+ MEDIA_LNK_FL_IMMUTABLE |
+ MEDIA_LNK_FL_ENABLED);
+ if (ret < 0) {
+ dev_err(kmb_cam->dev, "Fail to link %s->%s entities",
+ sd->entity.name, receiver->isp.subdev.entity.name);
+ goto error_unregister_entities;
+ }
+
+ return 0;
+
+error_unregister_entities:
+ kmb_isp_unregister_entities(&receiver->isp);
+error_isp_cleanup:
+ kmb_isp_cleanup(&receiver->isp);
+
+ return ret;
+}
+
+static int kmb_cam_complete(struct v4l2_async_notifier *n)
+{
+ return v4l2_device_register_subdev_nodes(n->v4l2_dev);
+}
+
+static void kmb_cam_unbind(struct v4l2_async_notifier *n,
+ struct v4l2_subdev *sd,
+ struct v4l2_async_subdev *asd)
+{
+ struct kmb_camera_receiver *receiver =
+ container_of(asd, struct kmb_camera_receiver, asd);
+
+ kmb_isp_unregister_entities(&receiver->isp);
+ kmb_isp_cleanup(&receiver->isp);
+}
+
+static const struct v4l2_async_notifier_operations notifier_ops = {
+ .bound = kmb_cam_bound,
+ .complete = kmb_cam_complete,
+ .unbind = kmb_cam_unbind
+};
+
+static int kmb_cam_parse_nodes(struct kmb_camera *kmb_cam,
+ struct v4l2_async_notifier *n)
+{
+ struct fwnode_handle *fwnode = NULL;
+ unsigned int i;
+ int ret;
+
+ for (i = 0; i < KMB_CAM_NUM_PORTS; i++) {
+ struct v4l2_fwnode_endpoint ep_data = {
+ .bus_type = V4L2_MBUS_CSI2_DPHY,
+ };
+ struct kmb_camera_receiver *receiver;
+ int rx_id;
+
+ fwnode = fwnode_graph_get_endpoint_by_id(dev_fwnode(kmb_cam->dev),
+ i, 0,
+ FWNODE_GRAPH_ENDPOINT_NEXT);
+ if (!fwnode)
+ continue;
+
+ ret = v4l2_fwnode_endpoint_parse(fwnode, &ep_data);
+ if (ret < 0) {
+ dev_err(kmb_cam->dev,
+ "No endpoint to parse in this fwnode");
+ goto error_fwnode_handle_put;
+ }
+
+ rx_id = get_rx_id(ep_data.bus.mipi_csi2.data_lanes,
+ ep_data.bus.mipi_csi2.num_data_lanes);
+ if (rx_id < 0) {
+ dev_err(kmb_cam->dev, "Invalid RX ID");
+ ret = rx_id;
+ goto error_fwnode_handle_put;
+ }
+
+ receiver =
+ v4l2_async_notifier_add_fwnode_remote_subdev(&kmb_cam->v4l2_notifier,
+ fwnode,
+ struct kmb_camera_receiver);
+ if (IS_ERR(receiver)) {
+ ret = PTR_ERR(receiver);
+ goto error_fwnode_handle_put;
+ }
+
+ receiver->csi2_config.rx_id = rx_id;
+ receiver->csi2_config.num_lanes =
+ ep_data.bus.mipi_csi2.num_data_lanes;
+
+ fwnode_handle_put(fwnode);
+ }
+
+ return 0;
+
+error_fwnode_handle_put:
+ fwnode_handle_put(fwnode);
+
+ return ret;
+}
+
+static int kmb_cam_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct v4l2_device *v4l2_dev;
+ struct kmb_camera *kmb_cam;
+ int ret;
+
+ kmb_cam = devm_kzalloc(dev, sizeof(*kmb_cam), GFP_KERNEL);
+ if (!kmb_cam)
+ return -ENOMEM;
+
+ kmb_cam->dev = dev;
+
+ platform_set_drvdata(pdev, kmb_cam);
+
+ ret = kmb_cam_xlink_init(&kmb_cam->xlink_cam, dev);
+ if (ret < 0)
+ return ret;
+
+ strscpy(kmb_cam->media_dev.model, "Keem Bay camera",
+ sizeof(kmb_cam->media_dev.model));
+ kmb_cam->media_dev.dev = dev;
+ kmb_cam->media_dev.hw_revision = 0;
+ media_device_init(&kmb_cam->media_dev);
+
+ v4l2_dev = &kmb_cam->v4l2_dev;
+ v4l2_dev->mdev = &kmb_cam->media_dev;
+ strscpy(v4l2_dev->name, "keembay-camera", sizeof(v4l2_dev->name));
+
+ ret = v4l2_device_register(dev, &kmb_cam->v4l2_dev);
+ if (ret < 0) {
+ dev_err(kmb_cam->dev, "Fail to register v4l2_device: %d", ret);
+ goto error_xlink_cleanup;
+ }
+
+ ret = of_reserved_mem_device_init(dev);
+ if (ret)
+ dev_info(dev, "Default CMA memory region will be used!");
+
+ v4l2_async_notifier_init(&kmb_cam->v4l2_notifier);
+ ret = kmb_cam_parse_nodes(kmb_cam, &kmb_cam->v4l2_notifier);
+ if (ret < 0) {
+ dev_err(kmb_cam->dev, "Fail to parse nodes: %d", ret);
+ goto error_async_notifier_cleanup;
+ }
+
+ kmb_cam->v4l2_notifier.ops = ¬ifier_ops;
+ ret = v4l2_async_notifier_register(&kmb_cam->v4l2_dev,
+ &kmb_cam->v4l2_notifier);
+ if (ret < 0) {
+ dev_err(kmb_cam->dev, "Could not register notifier! %d", ret);
+ goto error_async_notifier_cleanup;
+ }
+
+ ret = media_device_register(&kmb_cam->media_dev);
+ if (ret < 0) {
+ dev_err(kmb_cam->dev, "Fail to register media device %d", ret);
+ goto error_async_notifier_unregister;
+ }
+
+ return 0;
+
+error_async_notifier_unregister:
+ v4l2_async_notifier_unregister(&kmb_cam->v4l2_notifier);
+error_async_notifier_cleanup:
+ v4l2_async_notifier_cleanup(&kmb_cam->v4l2_notifier);
+ v4l2_device_unregister(&kmb_cam->v4l2_dev);
+error_xlink_cleanup:
+ kmb_cam_xlink_cleanup(&kmb_cam->xlink_cam);
+
+ return ret;
+}
+
+static int kmb_cam_remove(struct platform_device *pdev)
+{
+ struct kmb_camera *kmb_cam = platform_get_drvdata(pdev);
+
+ v4l2_async_notifier_unregister(&kmb_cam->v4l2_notifier);
+ v4l2_async_notifier_cleanup(&kmb_cam->v4l2_notifier);
+
+ media_device_unregister(&kmb_cam->media_dev);
+ media_device_cleanup(&kmb_cam->media_dev);
+ v4l2_device_unregister(&kmb_cam->v4l2_dev);
+
+ kmb_cam_xlink_cleanup(&kmb_cam->xlink_cam);
+
+ return 0;
+}
+
+static const struct of_device_id kmb_cam_dt_match[] = {
+ {.compatible = "intel,keembay-camera"},
+ {}
+};
+MODULE_DEVICE_TABLE(of, kmb_cam_dt_match);
+
+static struct platform_driver keembay_camera_driver = {
+ .probe = kmb_cam_probe,
+ .remove = kmb_cam_remove,
+ .driver = {
+ .name = "keembay-camera",
+ .owner = THIS_MODULE,
+ .of_match_table = kmb_cam_dt_match,
+ }
+};
+
+module_platform_driver(keembay_camera_driver);
+
+MODULE_DESCRIPTION("Intel Keem Bay camera");
+MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/keembay-camera/keembay-camera.h b/drivers/media/platform/keembay-camera/keembay-camera.h
new file mode 100644
index 000000000000..c108c60fdc73
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-camera.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Intel Keem Bay camera driver.
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+#ifndef KEEMBAY_CAMERA_H
+#define KEEMBAY_CAMERA_H
+
+#include <media/v4l2-device.h>
+
+#include "keembay-cam-xlink.h"
+#include "keembay-isp.h"
+
+/**
+ * struct kmb_camera_receiver - Keem Bay camera receiver
+ * @asd: V4L2 asynchronous sub-device
+ * @csi2_config: CSI-2 configuration
+ * @isp: ISP device
+ */
+struct kmb_camera_receiver {
+ struct v4l2_async_subdev asd;
+ struct kmb_isp_csi2_config csi2_config;
+ struct kmb_isp isp;
+};
+
+/**
+ * struct kmb_cam - Keem Bay camera media device
+ * @dev: Pointer to basic device structure
+ * @media_dev: Media device
+ * @v4l2_dev: V4L2 device
+ * @v4l2_notifier: V4L2 async notifier
+ * @xlink_cam: Xlink camera communication handler
+ */
+struct kmb_camera {
+ struct device *dev;
+ struct media_device media_dev;
+ struct v4l2_device v4l2_dev;
+ struct v4l2_async_notifier v4l2_notifier;
+ struct kmb_xlink_cam xlink_cam;
+};
+
+#endif /* KEEMBAY_CAMERA_H */
diff --git a/drivers/media/platform/keembay-camera/keembay-isp.c b/drivers/media/platform/keembay-camera/keembay-isp.c
new file mode 100644
index 000000000000..79f6212e37d9
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-isp.c
@@ -0,0 +1,53 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Intel Keem Bay camera ISP driver.
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+#include "keembay-isp.h"
+
+/**
+ * kmb_isp_init - Initialize Kmb isp subdevice
+ * @kmb_isp: Pointer to kmb isp device
+ * @dev: Pointer to camera device for which isp will be associated with
+ * @csi2_config: Csi2 configuration
+ * @xlink_cam: Xlink camera communication handle
+ *
+ * Return: 0 if successful, error code otherwise.
+ */
+int kmb_isp_init(struct kmb_isp *kmb_isp, struct device *dev,
+ struct kmb_isp_csi2_config *csi2_config,
+ struct kmb_xlink_cam *xlink_cam)
+{
+ return 0;
+}
+
+/**
+ * kmb_isp_cleanup - Cleanup kmb isp sub-device resourcess allocated in init
+ * @kmb_isp: Pointer to kmb isp sub-device
+ */
+void kmb_isp_cleanup(struct kmb_isp *kmb_isp)
+{ }
+
+/**
+ * kmb_isp_register_entities - Register entities
+ * @kmb_isp: pointer to kmb isp device
+ * @v4l2_dev: pointer to V4L2 device drivers
+ *
+ * Register all entities in the pipeline and create
+ * links between them.
+ *
+ * Return: 0 if successful, error code otherwise.
+ */
+int kmb_isp_register_entities(struct kmb_isp *kmb_isp,
+ struct v4l2_device *v4l2_dev)
+{
+ return 0;
+}
+
+/**
+ * kmb_isp_unregister_entities - Unregister this media's entities
+ * @kmb_isp: pointer to kmb isp device
+ */
+void kmb_isp_unregister_entities(struct kmb_isp *kmb_isp)
+{ }
diff --git a/drivers/media/platform/keembay-camera/keembay-isp.h b/drivers/media/platform/keembay-camera/keembay-isp.h
new file mode 100644
index 000000000000..35af6c644676
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-isp.h
@@ -0,0 +1,90 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Intel Keem Bay camera ISP driver.
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+#ifndef KEEMBAY_ISP_H
+#define KEEMBAY_ISP_H
+
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-subdev.h>
+
+#define KMB_ISP_DRV_NAME "keembay-camera-isp"
+
+#define KMB_ISP_SINK_PAD_SENSOR 0
+#define KMB_ISP_SINK_PAD_CFG 1
+#define KMB_ISP_SRC_PAD_VID 2
+#define KMB_ISP_PADS_NUM 3
+
+/**
+ * struct kmb_isp_csi2_config - Isp csi2 configuration
+ * @rx_id: Source port id
+ * @num_lanes: Number of physical lanes
+ */
+struct kmb_isp_csi2_config {
+ u32 rx_id;
+ u32 num_lanes;
+};
+
+/**
+ * struct kmb_isp - Keem Bay camera ISP device structure
+ * @dev: Pointer to basic device structure
+ * @lock: Mutex serilizing access to ISP device
+ * @thread: Pointer to worker thread data
+ * @xlink_cam: Xlink camera communication handler
+ * @msg_phy_addr: ISP channel physical CMA address
+ * @msg_vaddr: ISP channel virtual CMA address
+ * @cfg_q_lock: Mutex to serialize access to isp cfg bufferss queue
+ * @isp_cfgs_queue: Isp cfg buffers queue
+ * @isp_streaming: Flag to indicate ISP state
+ * @source_streaming: Flag to indicate source state
+ * @source_stopped: Completion to wait until VPU source is stopped
+ * @subdev: V4L2 sub-device
+ * @pads: Array of supported isp pads
+ * @active_pad_fmt: Array holding active pad formats
+ * @try_pad_fmt: Array holding try pad formats
+ * @csi2_config: CSI2 configuration
+ * @source_fmt: Pointer to isp source format
+ * @sequence: frame sequence number
+ */
+struct kmb_isp {
+ struct device *dev;
+ struct mutex lock;
+ struct task_struct *thread;
+
+ struct kmb_xlink_cam *xlink_cam;
+
+ dma_addr_t msg_phy_addr;
+ void *msg_vaddr;
+
+ struct mutex cfg_q_lock;
+ struct list_head isp_cfgs_queue;
+
+ bool isp_streaming;
+ bool source_streaming;
+ struct completion source_stopped;
+
+ struct v4l2_subdev subdev;
+ struct media_pad pads[KMB_ISP_PADS_NUM];
+
+ struct v4l2_subdev_format active_pad_fmt[KMB_ISP_PADS_NUM];
+
+ struct v4l2_subdev_format try_pad_fmt[KMB_ISP_PADS_NUM];
+
+ struct kmb_isp_csi2_config csi2_config;
+ const struct kmb_isp_source_format *source_fmt;
+
+ u32 sequence;
+};
+
+int kmb_isp_init(struct kmb_isp *kmb_isp, struct device *dev,
+ struct kmb_isp_csi2_config *csi2_config,
+ struct kmb_xlink_cam *xlink_cam);
+void kmb_isp_cleanup(struct kmb_isp *kmb_isp);
+
+int kmb_isp_register_entities(struct kmb_isp *kmb_isp,
+ struct v4l2_device *v4l2_dev);
+void kmb_isp_unregister_entities(struct kmb_isp *kmb_isp);
+
+#endif /* KEEMBAY_ISP_H */
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread* [PATCH 03/10] media: Keem Bay Camera: Add VPU camera interface
2021-03-19 18:06 [PATCH 00/10] Keem Bay Camera Subsystem Martina Krasteva
2021-03-19 18:06 ` [PATCH 01/10] dt-bindings: media: Add bindings for Keem Bay Camera Martina Krasteva
2021-03-19 18:06 ` [PATCH 02/10] media: Keem Bay Camera: Keem Bay camera driver Martina Krasteva
@ 2021-03-19 18:06 ` Martina Krasteva
2021-04-09 12:01 ` Sakari Ailus
2021-03-19 18:06 ` [PATCH 04/10] uapi: Keem Bay ISP Parameters data types Martina Krasteva
` (7 subsequent siblings)
10 siblings, 1 reply; 28+ messages in thread
From: Martina Krasteva @ 2021-03-19 18:06 UTC (permalink / raw)
To: linux-media
Cc: mchehab, robh+dt, devicetree, sakari.ailus,
daniele.alessandrelli, paul.j.murphy, gjorgjix.rosikopulos,
martinax.krasteva
From: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
Communication with VPU firmware is over XLink.
XLink has a channel-based communication, each channel has a unique
ID. The communication between VPU FW and camera driver starts with
one channel with negotiated ID. Currently this ID it is hard-coded
on both sides and should not be changed.
Three main channel types are used for streaming session:
1. Pipeline management channel:
This is fixed channel used to configure/build/delete
the streaming pipelines. When pipeline is built the channel IDs
used for communication are provided from linux kernel camera
driver.
2. Isp control channel:
This channel is used for ISP and MIPI RX configuration. For each
pipeline a separate ISP channel is required.
3. Buffer pool channels:
Each endpoint and buffer pool from VPU FW is associated with
a separate XLink channel. This channel is used for buffer
management.
Messages in "1" and "2" are using cmd sturct as payload data which
contains message type and physical address containing message payload.
"3" messages are small and the whole message is in XLink payload data.
Pipeline management:
Each pipeline instance is created on pipeline XLink channel "1".
The pipeline lifecycle states are:
- Configuration: Pipeline mode and input resolution are sent,
as a result min/max resolutions for the available outputs in that mode
are received.
- Build: The returned pipeline configuration is passed to build
command in addition with filled output channel configurations for
each output endpoint. After this command pipeline is ready for
streaming and can accept messages on ISP and buffer pool channels.
- Delete: The command deletes constructed pipeline.
NOTE: Now pipeline lifecycle should be always
configuration->build->delete it is not allowed to mix
the states. However an request was sent to VPU firmware team to
be able to delete configured pipeline without going in build state.
ISP control:
ISP control channel is used for controlling VPU ISP. This includes:
- Isp source configuration: MIPI RX configuration
- Isp source start/stop: start/stop MIPI RX
- Sending ISP params for processing
Events from VPU ISP are also received on ISP control channel.
Those are MIPI RX events, ISP events and error events.
VPU ISP works in per-frame control mode - ISP configuration is
required for every processed endpoint.
ISP configuration has a lifecycle.
The following event sequence needs to be received for each ISP
configuration, then it is released by the VPU:
- Readout start - MIPI SOF
- Readout end - MIPI EOF
- Isp start - Isp processing SOF
- Isp end - Isp processing EOF
If VPU wants to discard ISP configuration because of some internal
error, ISP configuration skip event is sent.
NOTE: Received events' payload data contains an ISP configuration
address this event corresponds to or 0 if the event is not for
ISP configuration.
Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
Co-developed-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
Signed-off-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
---
.../platform/keembay-camera/keembay-vpu-cmd.h | 110 ++++
.../platform/keembay-camera/keembay-vpu-frame.h | 102 +++
.../platform/keembay-camera/keembay-vpu-isp.h | 724 +++++++++++++++++++++
.../platform/keembay-camera/keembay-vpu-pipe.h | 110 ++++
.../platform/keembay-camera/keembay-vpu-src.h | 193 ++++++
5 files changed, 1239 insertions(+)
create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-cmd.h
create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-frame.h
create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-isp.h
create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-pipe.h
create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-src.h
diff --git a/drivers/media/platform/keembay-camera/keembay-vpu-cmd.h b/drivers/media/platform/keembay-camera/keembay-vpu-cmd.h
new file mode 100644
index 000000000000..192deebf33c9
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-vpu-cmd.h
@@ -0,0 +1,110 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Intel Keem Bay camera VPU Commands
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+#ifndef KEEMBAY_VPU_CMD_H
+#define KEEMBAY_VPU_CMD_H
+
+enum {
+ /* IC_EVENT_TYPE enum to define event messages */
+ KMB_IC_EVENT_TYPE_SUCCESSFUL = 0,
+ KMB_IC_EVENT_TYPE_CONFIG_ISP_PIPE,
+ KMB_IC_EVENT_TYPE_BUILD_ISP_PIPE,
+ KMB_IC_EVENT_TYPE_DELETE_ISP_PIPE,
+
+ KMB_IC_EVENT_TYPE_INIT_MAX
+};
+
+enum {
+ /* Control -> Source */
+ KMB_IC_EVENT_TYPE_CONFIG_SOURCE = (KMB_IC_EVENT_TYPE_INIT_MAX + 1),
+ KMB_IC_EVENT_TYPE_START_SOURCE,
+ KMB_IC_EVENT_TYPE_STOP_SOURCE,
+ KMB_IC_EVENT_TYPE_CONFIG_SOURCE_DYNAMIC,
+ KMB_IC_EVENT_TYPE_SOURCE_SEND_USER_DATA,
+
+ /* Source -> Control */
+ KMB_IC_EVENT_TYPE_SOURCE_CONFIGURED,
+ KMB_IC_EVENT_TYPE_SOURCE_STARTED,
+ KMB_IC_EVENT_TYPE_SOURCE_STOPPED,
+ KMB_IC_EVENT_TYPE_SOURCE_DYN_CONFIGURED,
+
+ /* Source events */
+ KMB_IC_EVENT_TYPE_READOUT_START,
+ KMB_IC_EVENT_TYPE_READOUT_END,
+ KMB_IC_EVENT_TYPE_LINE_REACHED,
+
+ /* ISP events */
+ KMB_IC_EVENT_TYPE_ISP_START,
+ KMB_IC_EVENT_TYPE_ISP_END,
+ KMB_IC_EVENT_TYPE_STATS_READY,
+ KMB_IC_EVENT_TYPE_ISP_CONFIG_ACCEPTED,
+ KMB_IC_EVENT_TYPE_ZSL_LOCKED,
+ KMB_IC_EVENT_TYPE_CAPTURE_MADE,
+
+ /* Isp config events */
+ KMB_IC_EVENT_TYPE_CONFIG_ISP,
+ KMB_IC_EVENT_TYPE_LOCK_ZSL,
+ KMB_IC_EVENT_TYPE_CAPTURE,
+ KMB_IC_EVENT_TYPE_UNLOCK_ZSL,
+ KMB_IC_EVENT_TYPE_ZSL_ADD,
+ KMB_IC_EVENT_TYPE_ERROR,
+
+ KMB_IC_EVENT_MAX,
+};
+
+enum {
+ KMB_IC_ERROR_PIPE_INIT = (KMB_IC_EVENT_MAX + 1),
+ KMB_IC_ERROR_ISP_CONFIG,
+ KMB_IC_ERROR_YUV_BUFF_MISSING,
+
+ KMB_IC_ERROR_ISP_MAX,
+};
+
+enum {
+ KMB_IC_ERROR_SRC_MIPI_WRONG_STATE = (KMB_IC_ERROR_ISP_MAX + 1),
+ KMB_IC_ERROR_SRC_MIPI_BAD_PARAMETER,
+ KMB_IC_ERROR_SRC_MIPI_CFG_MISSING,
+ KMB_IC_ERROR_SRC_MIPI_CFG_SKIPPED,
+ KMB_IC_ERROR_SRC_MIPI_OUT_BUFFERS_NOT_AVAILABLE,
+ KMB_IC_ERROR_SRC_MIPI_EOF_TIMEOUT,
+ KMB_IC_ERROR_SRC_MIPI_LOC_BUF_NOT_AVAILABLE,
+ KMB_IC_ERROR_SRC_MIPI_INTERNAL_ERROR,
+ KMB_IC_ERROR_SRC_TRANSMISSION_ERROR,
+ KMB_IC_ERROR_SRC_DRIVER_UNEXPECTED,
+
+ KMB_IC_ERROR_SRC_MIPI_MAX,
+};
+
+enum {
+ KMB_IC_ERROR_NO_ZSL_BUFFS_AVAILABLE = (KMB_IC_ERROR_SRC_MIPI_MAX + 1),
+ KMB_IC_ERROR_TRIGGER_NOT_EXISTING_BUFF,
+
+ KMB_IC_ERROR_ISP_CTRL_MAX,
+};
+
+/**
+ * struct kmb_ic_ev - Event structure
+ *
+ * @ev_info: Describe ISP event
+ * @ev_info.inst_id: Pipe id
+ * @ev_info.seq_nr: Frame number
+ * @ev_info.user_data_base_addr01: Address of isp cfg buffer in CMA
+ * @ev_info.user_data_base_addr02: Address of isp cfg buffer in CMA
+ * @ev_info.ts: Timestamp in NS
+ * @ctrl: Value from the IC_EVENT_TYPE enum
+ */
+struct kmb_ic_ev {
+ struct {
+ u32 inst_id;
+ u32 seq_nr;
+ u32 user_data_base_addr01;
+ u32 user_data_base_addr02;
+ s64 ts;
+ } ev_info;
+ u32 ctrl;
+} __packed __aligned(64);
+
+#endif /* KEEMBAY_VPU_CMD_H */
diff --git a/drivers/media/platform/keembay-camera/keembay-vpu-frame.h b/drivers/media/platform/keembay-camera/keembay-vpu-frame.h
new file mode 100644
index 000000000000..aab99ab55077
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-vpu-frame.h
@@ -0,0 +1,102 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Intel Keem Bay camera VPU frame data
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+
+#ifndef KEEMBAY_VPU_FRAME_H_
+#define KEEMBAY_VPU_FRAME_H_
+
+/**
+ * enum kmb_frame_types - Frame types
+ *
+ * @KMB_FRAME_TYPE_YUV422I: Interleaved 8 bit
+ * @KMB_FRAME_TYPE_YUV444P: Planar 4:4:4 format
+ * @KMB_FRAME_TYPE_YUV420P: Planar 4:2:0 format
+ * @KMB_FRAME_TYPE_YUV422P: Planar 8-bit greyscale
+ * @KMB_FRAME_TYPE_YUV400P: 8-bit greyscale
+ * @KMB_FRAME_TYPE_RGBA8888: RGBA interleaved stored in 32 bit word
+ * @KMB_FRAME_TYPE_RGB888: Planar 8 bit RGB data
+ * @KMB_FRAME_TYPE_LUT2: 1 bit per pixel, Lookup table(used for graphics layers)
+ * @KMB_FRAME_TYPE_LUT4: 2 bit per pixel, Lookup table(used for graphics layers)
+ * @KMB_FRAME_TYPE_LUT16: 4 bit per pixel, Lookup table (used for
+ * graphics layers)
+ * @KMB_FRAME_TYPE_RAW16: Save any raw type (8, 10, 12bit) on 16 bits
+ * @KMB_FRAME_TYPE_RAW14: 14-bit value in 16-bit storage
+ * @KMB_FRAME_TYPE_RAW12: 12-bit value in 16-bit storage
+ * @KMB_FRAME_TYPE_RAW10: 10-bit value in 16-bit storage
+ * @KMB_FRAME_TYPE_RAW8: Raw 8 greyscale
+ * @KMB_FRAME_TYPE_PACK10: SIPP 10-bit packed format
+ * @KMB_FRAME_TYPE_PACK12: SIPP 12-bit packed format
+ * @KMB_FRAME_TYPE_YUV444I: Planar 4:4:4 interleaved format
+ * @KMB_FRAME_TYPE_NV12: Format NV12
+ * @KMB_FRAME_TYPE_NV21: Format NV21
+ * @KMB_FRAME_TYPE_BITSTREAM: Used for video encoder bitstream
+ * @KMB_FRAME_TYPE_HDR: Format HDR
+ * @KMB_FRAME_TYPE_NV12PACK10: NV12 format with pixels encoded in pack 10
+ * @KMB_FRAME_TYPE_NONE: Format None
+ */
+enum kmb_frame_types {
+ KMB_FRAME_TYPE_YUV422I,
+ KMB_FRAME_TYPE_YUV444P,
+ KMB_FRAME_TYPE_YUV420P,
+ KMB_FRAME_TYPE_YUV422P,
+ KMB_FRAME_TYPE_YUV400P,
+ KMB_FRAME_TYPE_RGBA8888,
+ KMB_FRAME_TYPE_RGB888,
+ KMB_FRAME_TYPE_LUT2,
+ KMB_FRAME_TYPE_LUT4,
+ KMB_FRAME_TYPE_LUT16,
+ KMB_FRAME_TYPE_RAW16,
+ KMB_FRAME_TYPE_RAW14,
+ KMB_FRAME_TYPE_RAW12,
+ KMB_FRAME_TYPE_RAW10,
+ KMB_FRAME_TYPE_RAW8,
+ KMB_FRAME_TYPE_PACK10,
+ KMB_FRAME_TYPE_PACK12,
+ KMB_FRAME_TYPE_YUV444I,
+ KMB_FRAME_TYPE_NV12,
+ KMB_FRAME_TYPE_NV21,
+ KMB_FRAME_TYPE_BITSTREAM,
+ KMB_FRAME_TYPE_HDR,
+ KMB_FRAME_TYPE_NV12PACK10,
+ KMB_FRAME_TYPE_NONE,
+};
+
+/**
+ * struct kmb_frame_spec - KMB frame specifications
+ *
+ * @type: Values from the enum kmb_frame_type
+ * @height: Height in pixels
+ * @width: Width in pixels
+ * @stride: Defines as distance in bytes from pix(y, x) to pix(y+1, x)
+ * @bpp: Bits per pixel (for unpacked types set to 8 or 16, for NV12 set only
+ * luma pixel size)
+ */
+struct kmb_frame_spec {
+ u16 type;
+ u16 height;
+ u16 width;
+ u16 stride;
+ u16 bpp;
+};
+
+/**
+ * struct kmb_vpu_frame_buffer - KMB frame buffer elements
+ *
+ * @spec: Frame specifications parameters
+ * @p1: Address to first image plane
+ * @p2: Address to second image plane (if used)
+ * @p3: Address to third image plane (if used)
+ * @ts: Timestamp in NS
+ */
+struct kmb_vpu_frame_buffer {
+ struct kmb_frame_spec spec;
+ u64 p1;
+ u64 p2;
+ u64 p3;
+ s64 ts;
+};
+
+#endif /* KEEMBAY_VPU_FRAME_H_ */
diff --git a/drivers/media/platform/keembay-camera/keembay-vpu-isp.h b/drivers/media/platform/keembay-camera/keembay-vpu-isp.h
new file mode 100644
index 000000000000..c8b35c8ffbb0
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-vpu-isp.h
@@ -0,0 +1,724 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Intel Keem Bay VPU ISP params
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+#ifndef KEEMBAY_VPU_ISP_H
+#define KEEMBAY_VPU_ISP_H
+
+/* Keembay VPU ISP Tables sizes and limits */
+#define KMB_VPU_MAX_EXPOSURES 3
+
+/**
+ * struct kmb_vpu_raw_stats - KMB Raw statisticsKMB
+ *
+ * @ae_awb_stats_addr: AE/AWB statistics addr
+ * @af_stats_addr: Base start offset for AF statistics addr
+ * @hist_luma_addr: Luma histogram addr
+ * @hist_rgb_addr: RGB histogram addr
+ * @flicker_rows_addr: Flicker detection raw addr
+ */
+struct kmb_vpu_raw_stats {
+ u64 ae_awb_stats_addr;
+ u64 af_stats_addr;
+ u64 hist_luma_addr;
+ u64 hist_rgb_addr;
+ u64 flicker_rows_addr;
+} __packed;
+
+/**
+ * struct kmb_vpu_blc_params - KMB Black Level Correction parameters
+ *
+ * @coeff1: Black level correction coefficient 1 parameter
+ * @coeff2: Black level correction coefficient 2 parameter
+ * @coeff3: Black level correction coefficient 3 parameter
+ * @coeff4: Black level correction coefficient 4 parameter
+ */
+struct kmb_vpu_blc_params {
+ u32 coeff1;
+ u32 coeff2;
+ u32 coeff3;
+ u32 coeff4;
+} __packed;
+
+/**
+ * struct kmb_vpu_sigma_dns_params - KMB Sigma Denoise parameters
+ *
+ * @noise: Sigma denoise noise parameter
+ * @threshold1: Sigma denoise min threshold1 parameter
+ * @threshold2: Sigma denoise max threshold2 parameter
+ * @threshold3: Sigma denoise min threshold3 parameter
+ * @threshold4: Sigma denoise max threshold4 parameter
+ * @threshold5: Sigma denoise min threshold5 parameter
+ * @threshold6: Sigma denoise max threshold6 parameter
+ * @threshold7: Sigma denoise min threshold7 parameter
+ * @threshold8: Sigma denoise max threshold8 parameter
+ */
+struct kmb_vpu_sigma_dns_params {
+ u32 noise;
+ u32 threshold1;
+ u32 threshold2;
+ u32 threshold3;
+ u32 threshold4;
+ u32 threshold5;
+ u32 threshold6;
+ u32 threshold7;
+ u32 threshold8;
+} __packed;
+
+/**
+ * struct kmb_vpu_lsc_params - KMB Lens Shading Correction parameters
+ *
+ * @threshold: Lens shading correction threshold parameter
+ * @width: Lens shading correction width parameter
+ * @height: Lens shading correction height parameter
+ * @reserved: Reserved for alignment purpose
+ * @addr: Lens shading correction table address
+ */
+struct kmb_vpu_lsc_params {
+ u32 threshold;
+ u32 width;
+ u32 height;
+ u8 reserved[4];
+ u64 addr;
+} __packed;
+
+/**
+ * struct kmb_vpu_raw_params - KMB Raw parameters
+ *
+ * @awb_stats_en: Enable AE/AWB stats output
+ * @awb_rgb_hist_en: Enable RGB histogram output
+ * @af_stats_en: Enable AF stats output
+ * @luma_hist_en: Enable Luma histogram output
+ * @flicker_accum_en: Enable flicker detection row accumulation output
+ * @bad_pixel_fix_en: Enable Hot/Cold pixel suppression
+ * @grgb_imb_en: Enable Gr/Gb imbalance correction
+ * @mono_imbalance_en: Enable mono imbalance correction
+ * @gain1: Raw gain1 parameter
+ * @gain2: Raw gain2 parameter
+ * @gain3: Raw gain3 parameter
+ * @gain4: Raw gain4 parameter
+ * @stop1: Raw stop1 parameter
+ * @stop2: Raw stop2 parameter
+ * @stop3: Raw stop3 parameter
+ * @stop4: Raw stop4 parameter
+ * @threshold1: Raw threshold1 parameter
+ * @alpha1: Raw alpha1 parameter
+ * @alpha2: Raw alpha2 parameter
+ * @alpha3: Raw alpha3 parameter
+ * @alpha4: Raw alpha4 parameter
+ * @threshold2: Raw threshold2 parameter
+ * @static_defect_size: Static defect data size
+ * @reserved: Reserved for alignment purpose
+ * @static_defect_addr: Static defect data address
+ * @flicker_first_row_acc: First row of flicker detection row accumulation
+ * @flicker_last_row_acc: First row of flicker detection row accumulation
+ * @stats: raw statistics buffers
+ */
+struct kmb_vpu_raw_params {
+ u32 awb_stats_en;
+ u32 awb_rgb_hist_en;
+ u32 af_stats_en;
+ u32 luma_hist_en;
+ u32 flicker_accum_en;
+ u32 bad_pixel_fix_en;
+ u32 grgb_imb_en;
+ u32 mono_imbalance_en;
+ u32 gain1;
+ u32 gain2;
+ u32 gain3;
+ u32 gain4;
+ u32 stop1;
+ u32 stop2;
+ u32 stop3;
+ u32 stop4;
+ u32 threshold1;
+ u32 alpha1;
+ u32 alpha2;
+ u32 alpha3;
+ u32 alpha4;
+ u32 threshold2;
+ u32 static_defect_size;
+ u8 reserved[4];
+ u64 static_defect_addr;
+ u32 flicker_first_row_acc;
+ u32 flicker_last_row_acc;
+ struct kmb_vpu_raw_stats stats[KMB_VPU_MAX_EXPOSURES];
+} __packed;
+
+/**
+ * struct kmb_vpu_ae_awb_params - KMB AE/AWB statistics parameters
+ *
+ * @start_x: AE/AWB start_x parameter
+ * @start_y: AE/AWB start_y parameter
+ * @width: AE/AWB width parameter
+ * @height: AE/AWB height parameter
+ * @skip_x: AE/AWB skip_x parameter
+ * @skip_y: AE/AWB skip_y parameter
+ * @patches_x: AE/AWB patches_x parameter
+ * @patches_y: AE/AWB patches_y parameter
+ * @threshold1: AE/AWB threshold1 parameter
+ * @threshold2: AE/AWB threshold2 parameter
+ */
+struct kmb_vpu_ae_awb_params {
+ u32 start_x;
+ u32 start_y;
+ u32 width;
+ u32 height;
+ u32 skip_x;
+ u32 skip_y;
+ u32 patches_x;
+ u32 patches_y;
+ u16 threshold1;
+ u16 threshold2;
+} __packed;
+
+/**
+ * struct kmb_vpu_af_params - KMB Auto Focus parameters
+ *
+ * @start_x: AF start_x parameter
+ * @start_y: AF start_y parameter
+ * @width: AF width parameter
+ * @height: AF height parameter
+ * @patches_x: AF patches_x parameter
+ * @patches_y: AF patches_y parameter
+ * @coeff: AF filter coeff parameter
+ * @threshold1: AF filer threshold1 parameter
+ * @threshold2: AF filer threshold2 parameter
+ * @coeffs1: AF filter coeffs1 parameter
+ * @coeffs2: AF filter coeffs2 parameter
+ */
+struct kmb_vpu_af_params {
+ u32 start_x;
+ u32 start_y;
+ u32 width;
+ u32 height;
+ u32 patches_x;
+ u32 patches_y;
+ s32 coeff;
+ s32 threshold1;
+ s32 threshold2;
+ s32 coeffs1[11];
+ s32 coeffs2[11];
+} __packed;
+
+/**
+ * struct kmb_vpu_hist_params - KMB Hist parameters
+ *
+ * @start_x: Hist start_x parameter
+ * @start_y: Hist start_y parameter
+ * @end_x: Hist end_x parameter
+ * @end_y: Hist end_y parameter
+ * @matrix: Hist matrix parameter
+ * @weight: Hist weight parameter
+ */
+struct kmb_vpu_hist_params {
+ u32 start_x;
+ u32 start_y;
+ u32 end_x;
+ u32 end_y;
+ u16 matrix[9];
+ u16 weight[3];
+} __packed;
+
+/**
+ * struct kmb_vpu_lca_params - KMB Lateral Chromatic Aberration parameters
+ *
+ * @addr: LCA table address
+ */
+struct kmb_vpu_lca_params {
+ u64 addr;
+} __packed;
+
+/**
+ * struct kmb_vpu_debayer_params - KMB Debayer parameters
+ *
+ * @coeff1: Filter coeff1 parameter
+ * @multiplier1: Filter multiplier1 parameter
+ * @multiplier2: Filter multiplier2 parameter
+ * @coeff2: Filter coeff2 parameter
+ * @coeff3: Filter coeff3 parameter
+ * @coeff4: Filter coeff4 parameter
+ */
+struct kmb_vpu_debayer_params {
+ s32 coeff1;
+ u32 multiplier1;
+ u32 multiplier2;
+ s32 coeff2;
+ s32 coeff3;
+ s32 coeff4;
+} __packed;
+
+/**
+ * struct kmb_vpu_hdr_params - KMB HDR parameters
+ *
+ * @ratio: HDR ratio parameter
+ * @scale: HDR scale parameter
+ * @offset1: HDR offset1 parameter
+ * @slope1: HDR slope1 parameter
+ * @offset2: HDR offset2 parameter
+ * @slope2: HDR slope2 parameter
+ * @offset3: HDR offset3 parameter
+ * @slope3: HDR slope3 parameter
+ * @offset4: HDR offset4 parameter
+ * @gain1: HDR gain1 parameter
+ * @blur1: HDR blur1 parameter
+ * @blur2: HDR blur2 parameter
+ * @contrast1: HDR contrast1 parameter
+ * @contrast2: HDR contrast2 parameter
+ * @enable1: HDR enable1 parameter
+ * @enable2: HDR enable2 parameter
+ * @offset5: HDR offset5 parameter
+ * @gain2: HDR gain2 parameter
+ * @offset6: HDR offset6 parameter
+ * @strength: HDR strength parameter
+ * @reserved1: Reserved for alignment purpose
+ * @luts_addr: HDR LUT address
+ * @offset7: HDR offset7 parameter
+ * @shift: HDR shift parameter
+ * @field1: HDR filed1 parameter
+ * @field2: HDR field2 parameter
+ * @gain3: HDR gain3 parameter
+ * @min: HDR min parameter
+ * @reserved2: Reserved for alignment purpose
+ */
+struct kmb_vpu_hdr_params {
+ u32 ratio[2];
+ u32 scale[3];
+ s32 offset1;
+ u32 slope1;
+ s32 offset2;
+ u32 slope2;
+ s32 offset3;
+ u32 slope3;
+ s32 offset4;
+ u32 gain1;
+ u32 blur1[3];
+ u32 blur2[5];
+ u32 contrast1;
+ u32 contrast2;
+ u32 enable1;
+ u32 enable2;
+ s32 offset5;
+ u32 gain2;
+ s32 offset6;
+ u32 strength;
+ u8 reserved1[4];
+ u64 luts_addr;
+ u16 offset7;
+ u32 shift;
+ u16 field1;
+ u16 field2;
+ u8 gain3;
+ u16 min;
+ u8 reserved2[3];
+} __packed;
+
+/**
+ * struct kmb_vpu_dog_dns_params - KMB Difference-of-Gaussians DNS parameters
+ *
+ * @threshold: Filter threshold parameter
+ * @strength: Filter strength parameter
+ * @coeffs11: Filter coeffs11 parameter
+ * @coeffs15: Filter coeffs15 parameter
+ * @reserved: Reserved for alignment purpose
+ */
+struct kmb_vpu_dog_dns_params {
+ u32 threshold;
+ u32 strength;
+ u8 coeffs11[6];
+ u8 coeffs15[8];
+ u8 reserved[2];
+} __packed;
+
+/**
+ * struct kmb_vpu_luma_dns_params - KMB Luma DNS parameters
+ *
+ * @threshold: Luma DNS threshold parameter
+ * @slope: Luma DNS slope parameter
+ * @shift: Luma DNS shift parameter
+ * @alpha: Luma DNS alpha parameter
+ * @weight: Luma DNS weight parameter
+ * @per_pixel_alpha_en: Enable adapt alpha
+ * @gain_bypass_en: Enable gain bypass
+ * @reserved: for alignment purpose
+ */
+struct kmb_vpu_luma_dns_params {
+ u32 threshold;
+ u32 slope;
+ u32 shift;
+ u32 alpha;
+ u32 weight;
+ u32 per_pixel_alpha_en;
+ u32 gain_bypass_en;
+ u8 reserved[4];
+} __packed;
+
+/**
+ * struct kmb_vpu_sharpen_params - KMB Sharpen parameters
+ *
+ * @coeffs1: Filter coeffs1 parameter
+ * @coeffs2: Filter coeffs2 parameter
+ * @coeffs3: Filter coeffs3 parameter
+ * @shift: Filter shift parameter
+ * @gain1: Filter gain1 parameter
+ * @gain2: Filter gain2 parameter
+ * @gain3: Filter gain3 parameter
+ * @gain4: Filter gain4 parameter
+ * @gain5: Filter gain5 parameter
+ * @stops1: Filter stops1 parameter
+ * @gains: Filter gains parameter
+ * @stops2: Filter stops2 parameter
+ * @overshoot: Filter overshoot parameter
+ * @undershoot: Filter undershoot parameter
+ * @alpha: Filter alpha parameter
+ * @gain6: Filter gain6 parameter
+ * @offset: Filter offset parameter
+ * @addr: Filter data address
+ */
+struct kmb_vpu_sharpen_params {
+ u16 coeffs1[6];
+ u16 coeffs2[6];
+ u16 coeffs3[6];
+ u32 shift;
+ u32 gain1;
+ u32 gain2;
+ u32 gain3;
+ u32 gain4;
+ u32 gain5;
+ u32 stops1[3];
+ u32 gains[3];
+ u32 stops2[4];
+ u32 overshoot;
+ u32 undershoot;
+ u32 alpha;
+ u32 gain6;
+ u32 offset;
+ u64 addr;
+} __packed;
+
+/**
+ * struct kmb_vpu_chroma_gen_params - KMB Chroma GEN parameters
+ *
+ * @epsilon: Chroma GEN epsilon parameter
+ * @coeff1: Chroma GEN coeff1 parameter
+ * @coeff2: Chroma GEN coeff2 parameter
+ * @coeff3: Chroma GEN coeff3 parameter
+ * @coeff4: Chroma GEN coeff4 parameter
+ * @coeff5: Chroma GEN coeff5 parameter
+ * @coeff6: Chroma GEN coeff6 parameter
+ * @strength1: Chroma GEN strength1 parameter
+ * @strength2: Chroma GEN strength2 parameter
+ * @coeffs: Chroma GEN coeffs parameter
+ * @offset1: Chroma GEN offset1 parameter
+ * @slope1: Chroma GEN slope1 parameter
+ * @slope2: Chroma GEN slope2 parameter
+ * @offset2: Chroma GEN offset2 parameter
+ * @limit: Chroma GEN limit parameter
+ */
+struct kmb_vpu_chroma_gen_params {
+ u32 epsilon;
+ u32 coeff1;
+ u32 coeff2;
+ u32 coeff3;
+ u32 coeff4;
+ u32 coeff5;
+ u32 coeff6;
+ u32 strength1;
+ u32 strength2;
+ u32 coeffs[3];
+ s32 offset1;
+ u32 slope1;
+ u32 slope2;
+ s32 offset2;
+ u32 limit;
+} __packed;
+
+/**
+ * struct kmb_vpu_median_params - KMB Median parameters
+ *
+ * @size: Filter size parameter
+ * @slope: Filter slope parameter
+ * @offset: Filter offset parameter
+ */
+struct kmb_vpu_median_params {
+ u32 size;
+ u32 slope;
+ s32 offset;
+} __packed;
+
+/**
+ * struct kmb_vpu_chroma_dns_params - KMB Chroma Denoise parameters
+ *
+ * @limit: Filter limit parameter
+ * @enable: Filter enable parameter
+ * @threshold1: Filter threshold1 parameter
+ * @threshold2: Filter threshold2 parameter
+ * @threshold3: Filter threshold3 parameter
+ * @threshold4: Filter threshold4 parameter
+ * @threshold5: Filter threshold5 parameter
+ * @threshold6: Filter threshold6 parameter
+ * @threshold7: Filter threshold7 parameter
+ * @threshold8: Filter threshold8 parameter
+ * @slope1: Filter slope1 parameter
+ * @offset1: Filter offset1 parameter
+ * @slope2: Filter slope2 parameter
+ * @offset2: Filter offset2 parameter
+ * @grey1: Filter grey1 parameter
+ * @grey2: Filter grey2 parameter
+ * @grey3: Filter grey3 parameter
+ * @coeff1: Filter coeff1 parameter
+ * @coeff2: Filter coeff2 parameter
+ * @coeff3: Filter coeff3 parameter
+ */
+struct kmb_vpu_chroma_dns_params {
+ u32 limit;
+ u32 enable;
+ u32 threshold1;
+ u32 threshold2;
+ u32 threshold3;
+ u32 threshold4;
+ u32 threshold5;
+ u32 threshold6;
+ u32 threshold7;
+ u32 threshold8;
+ u32 slope1;
+ s32 offset1;
+ u32 slope2;
+ s32 offset2;
+ u32 grey1;
+ u32 grey2;
+ u32 grey3;
+ u32 coeff1;
+ u32 coeff2;
+ u32 coeff3;
+} __packed;
+
+/**
+ * struct kmb_vpu_color_comb_params - KMB Color Combine parameters
+ *
+ * @matrix: Color combine matrix parameter
+ * @offsets:Color combine offsets parameter
+ * @coeff1: Color combine coeff1 parameter
+ * @coeff2: Color combine coeff2 parameter
+ * @coeff3: Color combine coeff3 parameter
+ * @reserved: Reserved for alignment purpose
+ * @addr: Color combine table address
+ * @enable: Color combine enable parameter
+ * @weight1: Color combine weight1 parameter
+ * @weight2: Color combine weight2 parameter
+ * @weight3: Color combine weight3 parameter
+ * @limit1: Color combine limit1 parameter
+ * @limit2: Color combine limit2 parameter
+ * @offset1: Color combine offset1 parameter
+ * @offset2: Color combine offset2 parameter
+ */
+struct kmb_vpu_color_comb_params {
+ u16 matrix[9];
+ u16 offsets[3];
+ u32 coeff1;
+ u32 coeff2;
+ u32 coeff3;
+ u8 reserved[4];
+ u64 addr;
+ u32 enable;
+ u32 weight1;
+ u32 weight2;
+ u32 weight3;
+ u32 limit1;
+ s32 limit2;
+ s32 offset1;
+ s32 offset2;
+} __packed;
+
+/**
+ * struct kmb_vpu_lut_params - KMB lut parameters
+ *
+ * @size: Lut size parameter
+ * @reserved: Reserved for alignment purpose
+ * @addr: Lut table address
+ * @matrix: Lut matrix parameter
+ * @offsets: Lut offsets parameter
+ */
+struct kmb_vpu_lut_params {
+ u32 size;
+ u8 reserved[4];
+ u64 addr;
+ u16 matrix[3 * 3];
+ u16 offsets[3];
+} __packed;
+
+/**
+ * struct kmb_vpu_tnf_params - KMB Temporal Noise Filter parameters
+ *
+ * @factor: Filter factor parameter
+ * @gain: Filter gain parameter
+ * @offset1: Filter offset1 parameter
+ * @slope1: Filter slope1 parameter
+ * @offset2: Filter offset2 parameter
+ * @slope2: Filter slope2 parameter
+ * @min1: Filter min1 parameter
+ * @min2: Filter min2 parameter
+ * @value: Filter value parameter
+ * @enable: Filter enable parameter
+ * @lut0_addr: Filter lut0 address
+ * @lut1_addr: Filter lut1 address
+ */
+struct kmb_vpu_tnf_params {
+ u32 factor;
+ u32 gain;
+ u32 offset1;
+ u32 slope1;
+ u32 offset2;
+ u32 slope2;
+ u32 min1;
+ u32 min2;
+ u32 value;
+ u32 enable;
+ u64 lut0_addr;
+ u64 lut1_addr;
+} __packed;
+
+/**
+ * struct kmb_vpu_dehaze_params - KMB dehaze parameters
+ *
+ * @gain1: Dehaze gain1 parameter
+ * @min: Dehaze min parameter
+ * @strength1: Dehaze strength1 parameter
+ * @strength2: Dehaze strength2 parameter
+ * @gain2: Dehaze gain2 parameter
+ * @saturation: Dehaze saturation parameter
+ * @value1: Dehaze value1 parameter
+ * @value2: Dehaze value2 parameter
+ * @value3: Dehaze value3 parameter
+ * @filter: Dehaze filter parameter
+ * @stats_addr: Dehaze statistics address
+ */
+struct kmb_vpu_dehaze_params {
+ u32 gain1;
+ u32 min;
+ u32 strength1;
+ u32 strength2;
+ u32 gain2;
+ u32 saturation;
+ u32 value1;
+ u32 value2;
+ u32 value3;
+ u32 filter[3];
+ u64 stats_addr;
+} __packed;
+
+/**
+ * struct kmb_vpu_warp_params - KMB Warp filter parameters
+ *
+ * @type: Warp filter type parameter
+ * @relative: Warp filter relative parameter
+ * @format: Warp filter format parameter
+ * @position: Warp filter position parameter
+ * @reserved: Reserved for alignment purposes
+ * @addr: Warp filter addr parameter
+ * @width: Warp filter width parameter
+ * @height: Warp filter height parameter
+ * @stride: Warp filter stride parameter
+ * @enable: Warp filter enable parameter
+ * @matrix: Warp matrix parameter
+ * @mode: Warp filter mode parameter
+ * @values: Warp filter values parameter
+ */
+struct kmb_vpu_warp_params {
+ u8 type;
+ u8 relative;
+ u8 format;
+ u8 position;
+ u8 reserved[4];
+ u64 addr;
+ u16 width;
+ u16 height;
+ u32 stride;
+ u8 enable;
+ u32 matrix[9];
+ u8 mode;
+ u16 values[3];
+} __packed;
+
+/**
+ * enum kmb_vpu_bayer_order - KMB sensor Bayer arrangement format types
+ *
+ * @KMB_ISP_BAYER_ORDER_GRBG: Gr R B Gr
+ * @KMB_ISP_BAYER_ORDER_RGGB: R Gr Gr B
+ * @KMB_ISP_BAYER_ORDER_GBRG: Gr B R Gr
+ * @KMB_ISP_BAYER_ORDER_BGGR: B Gr Gr R
+ */
+enum kmb_vpu_bayer_order {
+ KMB_VPU_ISP_BAYER_ORDER_GRBG = 0,
+ KMB_VPU_ISP_BAYER_ORDER_RGGB = 1,
+ KMB_VPU_ISP_BAYER_ORDER_GBRG = 2,
+ KMB_VPU_ISP_BAYER_ORDER_BGGR = 3,
+} __packed;
+
+/* Version of the VPU ISP ABI. It should be passed as
+ * first argument in the isp params struct
+ */
+#define KMB_VPU_ISP_ABI_VERSION 104
+
+/**
+ * struct kmb_vpu_isp_params - KMB VPU ISP parameters structure
+ *
+ * @header_version: Header Version
+ * @image_data_width: Image data width
+ * @num_exposures: Number of exposures
+ * @bayer_order: enum kmb_isp_bayer_order
+ * @user_data_key: Private key used for the client
+ * @blc: Black Level correction parameters
+ * @sigma_dns: Sigma denoise parameters
+ * @lsc: Lens Shading Correction parameters
+ * @raw: Raw parameters
+ * @ae_awb: Auto exposure/Auto white balance parameters
+ * @af: Auto focus parameters
+ * @histogram: Histogram parameters
+ * @lca: Lateral Chromatic Aberration filter parameters
+ * @debayer: SIPP Bayer demosaicing filter parameters
+ * @dog_dns: Difference-of-Gaussians filter parameters
+ * @luma_dns: Luma denoise parameters
+ * @sharpen: Sharpen filter parameters
+ * @chroma_gen: Chroma GEN parameters
+ * @median: Median hardware filter parameters
+ * @chroma_dns: Chroma Denoise hardware filter parameters
+ * @color_comb: Color Combine parameters
+ * @hdr: HDR parameters applied only in HDR mode
+ * @lut: lut parameters
+ * @tnf: Temporal Noise Filter parameters
+ * @dehaze: Dehaze parameters
+ * @warp: Warp filter parameters
+ */
+struct kmb_vpu_isp_params {
+ u32 header_version;
+ u32 image_data_width;
+ u32 num_exposures;
+ u32 bayer_order;
+ u32 user_data_key;
+ struct kmb_vpu_blc_params blc[KMB_VPU_MAX_EXPOSURES];
+ struct kmb_vpu_sigma_dns_params sigma_dns[KMB_VPU_MAX_EXPOSURES];
+ struct kmb_vpu_lsc_params lsc;
+ struct kmb_vpu_raw_params raw;
+ struct kmb_vpu_ae_awb_params ae_awb;
+ struct kmb_vpu_af_params af;
+ struct kmb_vpu_hist_params histogram;
+ struct kmb_vpu_lca_params lca;
+ struct kmb_vpu_debayer_params debayer;
+ struct kmb_vpu_dog_dns_params dog_dns;
+ struct kmb_vpu_luma_dns_params luma_dns;
+ struct kmb_vpu_sharpen_params sharpen;
+ struct kmb_vpu_chroma_gen_params chroma_gen;
+ struct kmb_vpu_median_params median;
+ struct kmb_vpu_chroma_dns_params chroma_dns;
+ struct kmb_vpu_color_comb_params color_comb;
+ struct kmb_vpu_hdr_params hdr;
+ struct kmb_vpu_lut_params lut;
+ struct kmb_vpu_tnf_params tnf;
+ struct kmb_vpu_dehaze_params dehaze;
+ struct kmb_vpu_warp_params warp;
+} __packed;
+
+#endif /* KEEMBAY_VPU_ISP */
diff --git a/drivers/media/platform/keembay-camera/keembay-vpu-pipe.h b/drivers/media/platform/keembay-camera/keembay-vpu-pipe.h
new file mode 100644
index 000000000000..d400b59938b2
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-vpu-pipe.h
@@ -0,0 +1,110 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Intel Keem Bay camera VPU pipe definitions
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+#ifndef KEEMBAY_VPU_PIPE_H
+#define KEEMBAY_VPU_PIPE_H
+
+#include "keembay-vpu-src.h"
+
+#define PIPE_TYPE_ISP_MAX_EXP 3
+
+enum {
+ PIPE_TYPE_ISP_ISP_ULL = 1,
+ PIPE_TYPE_ISP_ISP_2DOL,
+ PIPE_TYPE_ISP_ISP_3DOL,
+ PIPE_TYPE_ISP_ISP_MONO,
+
+ PIPE_TYPE_MAX,
+};
+
+enum {
+ SRC_TYPE_ALLOC_VPU_DATA_MIPI = 0,
+ SRC_TYPE_ALLOC_VPU_DATA_DBG,
+ SRC_TYPE_ALLOC_ARM_DATA_ARM,
+ SRC_TYPE_ALLOC_ARM_DATA_MIPI,
+ SRC_TYPE_ALLOC_ARM_DATA_DBG,
+
+ SRC_TYPE_ALLOC_DATA_MAX,
+};
+
+enum {
+ PIPE_TRANSFORM_HUB_NONE = 0,
+ PIPE_TRANSFORM_HUB_BASIC,
+ PIPE_TRANSFORM_HUB_FULL,
+ PIPE_TRANSFORM_HUB_STITCH,
+ PIPE_TRANSFORM_HUB_EPTZ,
+
+ PIPE_TRANSFORM_HUB_MAX,
+};
+
+enum {
+ PIPE_OUTPUT_ID_RAW = 0,
+ PIPE_OUTPUT_ID_ISP_CTRL,
+ PIPE_OUTPUT_ID_0,
+ PIPE_OUTPUT_ID_1,
+ PIPE_OUTPUT_ID_2,
+ PIPE_OUTPUT_ID_3,
+ PIPE_OUTPUT_ID_4,
+ PIPE_OUTPUT_ID_5,
+ PIPE_OUTPUT_ID_6,
+
+ PIPE_OUTPUT_ID_MAX,
+};
+
+/*
+ * struct kmb_channel_cfg - KMB channel configuration
+ *
+ * @id: Channel id
+ * @frm_res: Frame resolution
+ */
+struct kmb_channel_cfg {
+ u32 id;
+ struct kmb_ic_img_size frm_res;
+};
+
+/*
+ * struct kmb_pipe_config_evs - VPU pipeline configuration
+ *
+ * @pipe_id: Pipe id
+ * @pipe_type: Pipe type
+ * @src_type: Source type
+ * @pipe_trans_hub: Transform hub type
+ * @in_isp_res: Input ISP resolution
+ * @out_isp_res: Output isp resolution
+ * @in_isp_stride: ISP input stride used in DOL interleaved mode
+ * @in_exp_offsets: Long and short exp frames offsets used in interleaved mode
+ * @out_min_res: Output min resolution
+ * @out_max_res: Output max resolution
+ * @pipe_xlink_chann: Output channel id from the enum PIPE_OUTPUT_ID
+ * @keep_aspect_ratio: If enabled, aspect ratio must be kept when image is
+ * resized
+ * @in_data_width: Input bits per pixel
+ * @in_data_packed: Flag to enable packed mode
+ * @out_data_width: Output bits per pixel for first plane
+ * @internal_memory_addr: Internal memory pool address
+ * @internal_memory_size: Internal memory pool size
+ */
+struct kmb_pipe_config_evs {
+ u8 pipe_id;
+ u8 pipe_type;
+ u8 src_type;
+ u8 pipe_trans_hub;
+ struct kmb_ic_img_size in_isp_res;
+ struct kmb_ic_img_size out_isp_res;
+ u16 in_isp_stride;
+ u32 in_exp_offsets[PIPE_TYPE_ISP_MAX_EXP];
+ struct kmb_ic_img_size out_min_res[PIPE_OUTPUT_ID_MAX];
+ struct kmb_ic_img_size out_max_res[PIPE_OUTPUT_ID_MAX];
+ struct kmb_channel_cfg pipe_xlink_chann[PIPE_OUTPUT_ID_MAX];
+ u8 keep_aspect_ratio;
+ u8 in_data_width;
+ u8 in_data_packed;
+ u8 out_data_width;
+ u64 internal_memory_addr;
+ u32 internal_memory_size;
+} __aligned(64);
+
+#endif /* KEEMBAY_VPU_PIPE_H */
diff --git a/drivers/media/platform/keembay-camera/keembay-vpu-src.h b/drivers/media/platform/keembay-camera/keembay-vpu-src.h
new file mode 100644
index 000000000000..97f8febbc7e2
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-vpu-src.h
@@ -0,0 +1,193 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Intel Keem Bay camera VPU source configuration
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+
+#ifndef KEEMBAY_VPU_SRC_H
+#define KEEMBAY_VPU_SRC_H
+
+/*
+ * struct kmb_ic_img_size - The structure contains information about image size
+ *
+ * @w: Image width
+ * @h: Image height
+ */
+struct kmb_ic_img_size {
+ u32 w;
+ u32 h;
+};
+
+/*
+ * struct kmb_ic_img_rect - The struct represents the coordinates of a
+ * rectangular image
+ *
+ * @x1: Position of the bottom left corner
+ * @y1: Position of the top left corner
+ * @x2: Position of the bottom right corner
+ * @y2: Position of the top right corner
+ */
+struct kmb_ic_img_rect {
+ s32 x1;
+ s32 y1;
+ s32 x2;
+ s32 y2;
+};
+
+/*
+ * enum kmb_ic_source_instance - HW mipi/cif input devices
+ *
+ * @KMB_IC_SOURCE_0:
+ * @KMB_IC_SOURCE_1:
+ * @KMB_IC_SOURCE_2:
+ * @KMB_IC_SOURCE_3:
+ * @KMB_IC_SOURCE_4:
+ * @KMB_IC_SOURCE_5:
+ */
+enum kmb_ic_source_instance {
+ KMB_IC_SOURCE_0 = 0,
+ KMB_IC_SOURCE_1 = 1,
+ KMB_IC_SOURCE_2 = 2,
+ KMB_IC_SOURCE_3 = 3,
+ KMB_IC_SOURCE_4 = 4,
+ KMB_IC_SOURCE_5 = 5,
+};
+
+/*
+ * enum kmb_ic_bayer_format - Bayer pattern order
+ *
+ * @KMB_IC_BAYER_FORMAT_GRBG: Gr R B Gr
+ * @KMB_IC_BAYER_FORMAT_RGGB: R Gr Gr B
+ * @KMB_IC_BAYER_FORMAT_GBRG: Gr B R Gr
+ * @KMB_IC_BAYER_FORMAT_BGGR: B Gr Gr R
+ */
+enum kmb_ic_bayer_format {
+ KMB_IC_BAYER_FORMAT_GRBG = 0,
+ KMB_IC_BAYER_FORMAT_RGGB = 1,
+ KMB_IC_BAYER_FORMAT_GBRG = 2,
+ KMB_IC_BAYER_FORMAT_BGGR = 3,
+};
+
+/*
+ * enum kmb_ic_mipi_rx_ctrl_rec_not - List of receiver Id's for a specific
+ * sensor
+ *
+ * @KMB_IC_SIPP_DEVICE0:
+ * @KMB_IC_SIPP_DEVICE1:
+ * @KMB_IC_SIPP_DEVICE2:
+ * @KMB_IC_SIPP_DEVICE3:
+ * @KMB_IC_CIF0_DEVICE4:
+ * @KMB_IC_CIF1_DEVICE5:
+ */
+enum kmb_ic_mipi_rx_ctrl_rec_not {
+ KMB_IC_SIPP_DEVICE0 = 0,
+ KMB_IC_SIPP_DEVICE1 = 1,
+ KMB_IC_SIPP_DEVICE2 = 2,
+ KMB_IC_SIPP_DEVICE3 = 3,
+ KMB_IC_CIF0_DEVICE4 = 4,
+ KMB_IC_CIF1_DEVICE5 = 5,
+};
+
+/*
+ * enum kmb_ic_mipi_rx_ctrl_not - MIPI controller from chip
+ *
+ * @KMB_IC_MIPI_CTRL_0:
+ * @KMB_IC_MIPI_CTRL_1:
+ * @KMB_IC_MIPI_CTRL_2:
+ * @KMB_IC_MIPI_CTRL_3:
+ * @KMB_IC_MIPI_CTRL_4:
+ * @KMB_IC_MIPI_CTRL_5:
+ */
+enum kmb_ic_mipi_rx_ctrl_not {
+ KMB_IC_MIPI_CTRL_0 = 0,
+ KMB_IC_MIPI_CTRL_1 = 1,
+ KMB_IC_MIPI_CTRL_2 = 2,
+ KMB_IC_MIPI_CTRL_3 = 3,
+ KMB_IC_MIPI_CTRL_4 = 4,
+ KMB_IC_MIPI_CTRL_5 = 5,
+};
+
+/*
+ * enum kmb_ic_mipi_ex_data_type - All supported raw, sensor input formats
+ *
+ * @IC_IPIPE_YUV_420_B8:
+ * @IC_IPIPE_RAW_8:
+ * @IC_IPIPE_RAW_10:
+ * @IC_IPIPE_RAW_12:
+ * @IC_IPIPE_RAW_14:
+ * @IC_IPIPE_EMBEDDED_8BIT:
+ */
+enum kmb_ic_mipi_rx_data_type {
+ IC_IPIPE_YUV_420_B8 = 0x18,
+ IC_IPIPE_RAW_8 = 0x2A,
+ IC_IPIPE_RAW_10 = 0x2B,
+ IC_IPIPE_RAW_12 = 0x2C,
+ IC_IPIPE_RAW_14 = 0x2D,
+ IC_IPIPE_EMBEDDED_8BIT = 0x12
+};
+
+/*
+ * struct kmb_ic_source_config_dynamic - Per-source configuration of parameters
+ * which can be modified dynamically.
+ * Setting will take effect during the
+ * next blanking interval
+ *
+ * @notification_line: Line number upon which IC_EVENT_TYPE_LINE will be sent
+ * to the Lean OS. Set to -1 to disable notification
+ */
+struct kmb_ic_source_config_dynamic {
+ s32 notification_line;
+};
+
+/*
+ * struct kmb_ic_mipi_config - Mipi RX data configuration
+ *
+ * @no_controller: Number of controller
+ * @no_lanes: Number of lanes
+ * @lane_rate_mbps: Lane rate
+ * @data_type: Mipi RX data type
+ * @data_mode: Data mode
+ * @rec_nrl:
+ */
+struct kmb_ic_mipi_config {
+ u32 no_controller;
+ u32 no_lanes;
+ u32 lane_rate_mbps;
+ u32 data_type;
+ u32 data_mode;
+ u32 rec_nrl;
+};
+
+/*
+ * struct kmb_ic_source_config - Per-source configuration parameters - mostly
+ * information needed to configure the MIPI Rx
+ * filter
+ *
+ * @camera_output_size: Max frame size output by the camera
+ * @crop_window: Crop window coordinates
+ * @bayer_format: Bayer Format - Raw, Demosaic and LSC blocks should be
+ * programmed to match the Bayer order specified here.
+ * @bpp: Bits per pixel
+ * @mipi_rx_data: MIPI RX data configuration
+ * @no_exposure: Number of different exposure frames
+ * @metadata_width: Metadata width
+ * @metadata_height: Medata height
+ * @metadata_data_type: Metadata data type
+ */
+struct kmb_ic_source_config {
+ struct kmb_ic_img_size camera_output_size;
+ struct kmb_ic_img_rect crop_window;
+
+ u32 bayer_format;
+ u32 bpp;
+
+ struct kmb_ic_mipi_config mipi_rx_data;
+
+ u32 no_exposure;
+ u32 metadata_width;
+ u32 metadata_height;
+ u32 metadata_data_type;
+} __aligned(64);
+
+#endif /* KEEMBAY_VPU_SRC_H */
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread* Re: [PATCH 03/10] media: Keem Bay Camera: Add VPU camera interface
2021-03-19 18:06 ` [PATCH 03/10] media: Keem Bay Camera: Add VPU camera interface Martina Krasteva
@ 2021-04-09 12:01 ` Sakari Ailus
2021-04-09 14:39 ` Martina Krasteva
0 siblings, 1 reply; 28+ messages in thread
From: Sakari Ailus @ 2021-04-09 12:01 UTC (permalink / raw)
To: Martina Krasteva
Cc: linux-media, mchehab, robh+dt, devicetree, daniele.alessandrelli,
paul.j.murphy, gjorgjix.rosikopulos
Hi Martina,
On Fri, Mar 19, 2021 at 06:06:25PM +0000, Martina Krasteva wrote:
> From: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
>
> Communication with VPU firmware is over XLink.
> XLink has a channel-based communication, each channel has a unique
> ID. The communication between VPU FW and camera driver starts with
> one channel with negotiated ID. Currently this ID it is hard-coded
> on both sides and should not be changed.
>
> Three main channel types are used for streaming session:
>
> 1. Pipeline management channel:
> This is fixed channel used to configure/build/delete
> the streaming pipelines. When pipeline is built the channel IDs
> used for communication are provided from linux kernel camera
> driver.
>
> 2. Isp control channel:
> This channel is used for ISP and MIPI RX configuration. For each
> pipeline a separate ISP channel is required.
>
> 3. Buffer pool channels:
> Each endpoint and buffer pool from VPU FW is associated with
> a separate XLink channel. This channel is used for buffer
> management.
>
> Messages in "1" and "2" are using cmd sturct as payload data which
> contains message type and physical address containing message payload.
> "3" messages are small and the whole message is in XLink payload data.
>
> Pipeline management:
>
> Each pipeline instance is created on pipeline XLink channel "1".
> The pipeline lifecycle states are:
>
> - Configuration: Pipeline mode and input resolution are sent,
> as a result min/max resolutions for the available outputs in that mode
> are received.
>
> - Build: The returned pipeline configuration is passed to build
> command in addition with filled output channel configurations for
> each output endpoint. After this command pipeline is ready for
> streaming and can accept messages on ISP and buffer pool channels.
>
> - Delete: The command deletes constructed pipeline.
>
> NOTE: Now pipeline lifecycle should be always
> configuration->build->delete it is not allowed to mix
> the states. However an request was sent to VPU firmware team to
> be able to delete configured pipeline without going in build state.
>
> ISP control:
>
> ISP control channel is used for controlling VPU ISP. This includes:
> - Isp source configuration: MIPI RX configuration
> - Isp source start/stop: start/stop MIPI RX
> - Sending ISP params for processing
>
> Events from VPU ISP are also received on ISP control channel.
> Those are MIPI RX events, ISP events and error events.
>
> VPU ISP works in per-frame control mode - ISP configuration is
> required for every processed endpoint.
> ISP configuration has a lifecycle.
> The following event sequence needs to be received for each ISP
> configuration, then it is released by the VPU:
> - Readout start - MIPI SOF
> - Readout end - MIPI EOF
> - Isp start - Isp processing SOF
> - Isp end - Isp processing EOF
>
> If VPU wants to discard ISP configuration because of some internal
> error, ISP configuration skip event is sent.
>
> NOTE: Received events' payload data contains an ISP configuration
> address this event corresponds to or 0 if the event is not for
> ISP configuration.
>
> Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
> Co-developed-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
> Signed-off-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
> Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
> Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
> Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
> ---
> .../platform/keembay-camera/keembay-vpu-cmd.h | 110 ++++
> .../platform/keembay-camera/keembay-vpu-frame.h | 102 +++
> .../platform/keembay-camera/keembay-vpu-isp.h | 724 +++++++++++++++++++++
> .../platform/keembay-camera/keembay-vpu-pipe.h | 110 ++++
> .../platform/keembay-camera/keembay-vpu-src.h | 193 ++++++
> 5 files changed, 1239 insertions(+)
> create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-cmd.h
> create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-frame.h
> create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-isp.h
> create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-pipe.h
> create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-src.h
>
> diff --git a/drivers/media/platform/keembay-camera/keembay-vpu-cmd.h b/drivers/media/platform/keembay-camera/keembay-vpu-cmd.h
> new file mode 100644
> index 000000000000..192deebf33c9
> --- /dev/null
> +++ b/drivers/media/platform/keembay-camera/keembay-vpu-cmd.h
> @@ -0,0 +1,110 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Intel Keem Bay camera VPU Commands
> + *
> + * Copyright (C) 2021 Intel Corporation
> + */
> +#ifndef KEEMBAY_VPU_CMD_H
> +#define KEEMBAY_VPU_CMD_H
> +
> +enum {
> + /* IC_EVENT_TYPE enum to define event messages */
> + KMB_IC_EVENT_TYPE_SUCCESSFUL = 0,
> + KMB_IC_EVENT_TYPE_CONFIG_ISP_PIPE,
> + KMB_IC_EVENT_TYPE_BUILD_ISP_PIPE,
> + KMB_IC_EVENT_TYPE_DELETE_ISP_PIPE,
> +
> + KMB_IC_EVENT_TYPE_INIT_MAX
> +};
> +
> +enum {
> + /* Control -> Source */
> + KMB_IC_EVENT_TYPE_CONFIG_SOURCE = (KMB_IC_EVENT_TYPE_INIT_MAX + 1),
> + KMB_IC_EVENT_TYPE_START_SOURCE,
> + KMB_IC_EVENT_TYPE_STOP_SOURCE,
> + KMB_IC_EVENT_TYPE_CONFIG_SOURCE_DYNAMIC,
> + KMB_IC_EVENT_TYPE_SOURCE_SEND_USER_DATA,
> +
> + /* Source -> Control */
> + KMB_IC_EVENT_TYPE_SOURCE_CONFIGURED,
> + KMB_IC_EVENT_TYPE_SOURCE_STARTED,
> + KMB_IC_EVENT_TYPE_SOURCE_STOPPED,
> + KMB_IC_EVENT_TYPE_SOURCE_DYN_CONFIGURED,
> +
> + /* Source events */
> + KMB_IC_EVENT_TYPE_READOUT_START,
> + KMB_IC_EVENT_TYPE_READOUT_END,
> + KMB_IC_EVENT_TYPE_LINE_REACHED,
> +
> + /* ISP events */
> + KMB_IC_EVENT_TYPE_ISP_START,
> + KMB_IC_EVENT_TYPE_ISP_END,
> + KMB_IC_EVENT_TYPE_STATS_READY,
> + KMB_IC_EVENT_TYPE_ISP_CONFIG_ACCEPTED,
> + KMB_IC_EVENT_TYPE_ZSL_LOCKED,
> + KMB_IC_EVENT_TYPE_CAPTURE_MADE,
> +
> + /* Isp config events */
> + KMB_IC_EVENT_TYPE_CONFIG_ISP,
> + KMB_IC_EVENT_TYPE_LOCK_ZSL,
> + KMB_IC_EVENT_TYPE_CAPTURE,
> + KMB_IC_EVENT_TYPE_UNLOCK_ZSL,
> + KMB_IC_EVENT_TYPE_ZSL_ADD,
> + KMB_IC_EVENT_TYPE_ERROR,
> +
> + KMB_IC_EVENT_MAX,
> +};
> +
> +enum {
> + KMB_IC_ERROR_PIPE_INIT = (KMB_IC_EVENT_MAX + 1),
> + KMB_IC_ERROR_ISP_CONFIG,
> + KMB_IC_ERROR_YUV_BUFF_MISSING,
> +
> + KMB_IC_ERROR_ISP_MAX,
> +};
> +
> +enum {
> + KMB_IC_ERROR_SRC_MIPI_WRONG_STATE = (KMB_IC_ERROR_ISP_MAX + 1),
> + KMB_IC_ERROR_SRC_MIPI_BAD_PARAMETER,
> + KMB_IC_ERROR_SRC_MIPI_CFG_MISSING,
> + KMB_IC_ERROR_SRC_MIPI_CFG_SKIPPED,
> + KMB_IC_ERROR_SRC_MIPI_OUT_BUFFERS_NOT_AVAILABLE,
> + KMB_IC_ERROR_SRC_MIPI_EOF_TIMEOUT,
> + KMB_IC_ERROR_SRC_MIPI_LOC_BUF_NOT_AVAILABLE,
> + KMB_IC_ERROR_SRC_MIPI_INTERNAL_ERROR,
> + KMB_IC_ERROR_SRC_TRANSMISSION_ERROR,
> + KMB_IC_ERROR_SRC_DRIVER_UNEXPECTED,
> +
> + KMB_IC_ERROR_SRC_MIPI_MAX,
> +};
> +
> +enum {
> + KMB_IC_ERROR_NO_ZSL_BUFFS_AVAILABLE = (KMB_IC_ERROR_SRC_MIPI_MAX + 1),
> + KMB_IC_ERROR_TRIGGER_NOT_EXISTING_BUFF,
> +
> + KMB_IC_ERROR_ISP_CTRL_MAX,
> +};
> +
> +/**
> + * struct kmb_ic_ev - Event structure
> + *
> + * @ev_info: Describe ISP event
> + * @ev_info.inst_id: Pipe id
> + * @ev_info.seq_nr: Frame number
> + * @ev_info.user_data_base_addr01: Address of isp cfg buffer in CMA
> + * @ev_info.user_data_base_addr02: Address of isp cfg buffer in CMA
> + * @ev_info.ts: Timestamp in NS
> + * @ctrl: Value from the IC_EVENT_TYPE enum
> + */
> +struct kmb_ic_ev {
> + struct {
> + u32 inst_id;
> + u32 seq_nr;
> + u32 user_data_base_addr01;
> + u32 user_data_base_addr02;
> + s64 ts;
> + } ev_info;
> + u32 ctrl;
> +} __packed __aligned(64);
> +
> +#endif /* KEEMBAY_VPU_CMD_H */
> diff --git a/drivers/media/platform/keembay-camera/keembay-vpu-frame.h b/drivers/media/platform/keembay-camera/keembay-vpu-frame.h
> new file mode 100644
> index 000000000000..aab99ab55077
> --- /dev/null
> +++ b/drivers/media/platform/keembay-camera/keembay-vpu-frame.h
> @@ -0,0 +1,102 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Intel Keem Bay camera VPU frame data
> + *
> + * Copyright (C) 2021 Intel Corporation
> + */
> +
> +#ifndef KEEMBAY_VPU_FRAME_H_
> +#define KEEMBAY_VPU_FRAME_H_
> +
> +/**
> + * enum kmb_frame_types - Frame types
> + *
> + * @KMB_FRAME_TYPE_YUV422I: Interleaved 8 bit
> + * @KMB_FRAME_TYPE_YUV444P: Planar 4:4:4 format
> + * @KMB_FRAME_TYPE_YUV420P: Planar 4:2:0 format
> + * @KMB_FRAME_TYPE_YUV422P: Planar 8-bit greyscale
> + * @KMB_FRAME_TYPE_YUV400P: 8-bit greyscale
> + * @KMB_FRAME_TYPE_RGBA8888: RGBA interleaved stored in 32 bit word
> + * @KMB_FRAME_TYPE_RGB888: Planar 8 bit RGB data
> + * @KMB_FRAME_TYPE_LUT2: 1 bit per pixel, Lookup table(used for graphics layers)
> + * @KMB_FRAME_TYPE_LUT4: 2 bit per pixel, Lookup table(used for graphics layers)
> + * @KMB_FRAME_TYPE_LUT16: 4 bit per pixel, Lookup table (used for
> + * graphics layers)
> + * @KMB_FRAME_TYPE_RAW16: Save any raw type (8, 10, 12bit) on 16 bits
> + * @KMB_FRAME_TYPE_RAW14: 14-bit value in 16-bit storage
> + * @KMB_FRAME_TYPE_RAW12: 12-bit value in 16-bit storage
> + * @KMB_FRAME_TYPE_RAW10: 10-bit value in 16-bit storage
> + * @KMB_FRAME_TYPE_RAW8: Raw 8 greyscale
> + * @KMB_FRAME_TYPE_PACK10: SIPP 10-bit packed format
> + * @KMB_FRAME_TYPE_PACK12: SIPP 12-bit packed format
> + * @KMB_FRAME_TYPE_YUV444I: Planar 4:4:4 interleaved format
> + * @KMB_FRAME_TYPE_NV12: Format NV12
> + * @KMB_FRAME_TYPE_NV21: Format NV21
> + * @KMB_FRAME_TYPE_BITSTREAM: Used for video encoder bitstream
> + * @KMB_FRAME_TYPE_HDR: Format HDR
> + * @KMB_FRAME_TYPE_NV12PACK10: NV12 format with pixels encoded in pack 10
> + * @KMB_FRAME_TYPE_NONE: Format None
> + */
> +enum kmb_frame_types {
> + KMB_FRAME_TYPE_YUV422I,
> + KMB_FRAME_TYPE_YUV444P,
> + KMB_FRAME_TYPE_YUV420P,
> + KMB_FRAME_TYPE_YUV422P,
> + KMB_FRAME_TYPE_YUV400P,
> + KMB_FRAME_TYPE_RGBA8888,
> + KMB_FRAME_TYPE_RGB888,
> + KMB_FRAME_TYPE_LUT2,
> + KMB_FRAME_TYPE_LUT4,
> + KMB_FRAME_TYPE_LUT16,
> + KMB_FRAME_TYPE_RAW16,
> + KMB_FRAME_TYPE_RAW14,
> + KMB_FRAME_TYPE_RAW12,
> + KMB_FRAME_TYPE_RAW10,
> + KMB_FRAME_TYPE_RAW8,
> + KMB_FRAME_TYPE_PACK10,
> + KMB_FRAME_TYPE_PACK12,
> + KMB_FRAME_TYPE_YUV444I,
> + KMB_FRAME_TYPE_NV12,
> + KMB_FRAME_TYPE_NV21,
> + KMB_FRAME_TYPE_BITSTREAM,
> + KMB_FRAME_TYPE_HDR,
> + KMB_FRAME_TYPE_NV12PACK10,
> + KMB_FRAME_TYPE_NONE,
> +};
> +
> +/**
> + * struct kmb_frame_spec - KMB frame specifications
> + *
> + * @type: Values from the enum kmb_frame_type
> + * @height: Height in pixels
> + * @width: Width in pixels
> + * @stride: Defines as distance in bytes from pix(y, x) to pix(y+1, x)
> + * @bpp: Bits per pixel (for unpacked types set to 8 or 16, for NV12 set only
> + * luma pixel size)
> + */
> +struct kmb_frame_spec {
> + u16 type;
> + u16 height;
> + u16 width;
> + u16 stride;
> + u16 bpp;
> +};
> +
> +/**
> + * struct kmb_vpu_frame_buffer - KMB frame buffer elements
> + *
> + * @spec: Frame specifications parameters
> + * @p1: Address to first image plane
> + * @p2: Address to second image plane (if used)
> + * @p3: Address to third image plane (if used)
> + * @ts: Timestamp in NS
"ns"
> + */
> +struct kmb_vpu_frame_buffer {
> + struct kmb_frame_spec spec;
> + u64 p1;
> + u64 p2;
> + u64 p3;
> + s64 ts;
> +};
Messages in form of these structs are also sent over xlink. Shouldn't they
be __packed as well??
Also the size of kmb_frame_spec is not a multiple of 8 so it would seem to
require padding to be a member of struct kmb_vpu_frame_buffer.
> +
> +#endif /* KEEMBAY_VPU_FRAME_H_ */
> diff --git a/drivers/media/platform/keembay-camera/keembay-vpu-isp.h b/drivers/media/platform/keembay-camera/keembay-vpu-isp.h
> new file mode 100644
> index 000000000000..c8b35c8ffbb0
> --- /dev/null
> +++ b/drivers/media/platform/keembay-camera/keembay-vpu-isp.h
> @@ -0,0 +1,724 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Intel Keem Bay VPU ISP params
> + *
> + * Copyright (C) 2021 Intel Corporation
> + */
> +#ifndef KEEMBAY_VPU_ISP_H
> +#define KEEMBAY_VPU_ISP_H
> +
> +/* Keembay VPU ISP Tables sizes and limits */
> +#define KMB_VPU_MAX_EXPOSURES 3
> +
> +/**
> + * struct kmb_vpu_raw_stats - KMB Raw statisticsKMB
> + *
> + * @ae_awb_stats_addr: AE/AWB statistics addr
> + * @af_stats_addr: Base start offset for AF statistics addr
> + * @hist_luma_addr: Luma histogram addr
> + * @hist_rgb_addr: RGB histogram addr
> + * @flicker_rows_addr: Flicker detection raw addr
> + */
> +struct kmb_vpu_raw_stats {
> + u64 ae_awb_stats_addr;
> + u64 af_stats_addr;
> + u64 hist_luma_addr;
> + u64 hist_rgb_addr;
> + u64 flicker_rows_addr;
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_blc_params - KMB Black Level Correction parameters
> + *
> + * @coeff1: Black level correction coefficient 1 parameter
> + * @coeff2: Black level correction coefficient 2 parameter
> + * @coeff3: Black level correction coefficient 3 parameter
> + * @coeff4: Black level correction coefficient 4 parameter
> + */
> +struct kmb_vpu_blc_params {
> + u32 coeff1;
> + u32 coeff2;
> + u32 coeff3;
> + u32 coeff4;
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_sigma_dns_params - KMB Sigma Denoise parameters
> + *
> + * @noise: Sigma denoise noise parameter
> + * @threshold1: Sigma denoise min threshold1 parameter
> + * @threshold2: Sigma denoise max threshold2 parameter
> + * @threshold3: Sigma denoise min threshold3 parameter
> + * @threshold4: Sigma denoise max threshold4 parameter
> + * @threshold5: Sigma denoise min threshold5 parameter
> + * @threshold6: Sigma denoise max threshold6 parameter
> + * @threshold7: Sigma denoise min threshold7 parameter
> + * @threshold8: Sigma denoise max threshold8 parameter
> + */
> +struct kmb_vpu_sigma_dns_params {
> + u32 noise;
> + u32 threshold1;
> + u32 threshold2;
> + u32 threshold3;
> + u32 threshold4;
> + u32 threshold5;
> + u32 threshold6;
> + u32 threshold7;
> + u32 threshold8;
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_lsc_params - KMB Lens Shading Correction parameters
> + *
> + * @threshold: Lens shading correction threshold parameter
> + * @width: Lens shading correction width parameter
> + * @height: Lens shading correction height parameter
> + * @reserved: Reserved for alignment purpose
> + * @addr: Lens shading correction table address
> + */
> +struct kmb_vpu_lsc_params {
> + u32 threshold;
> + u32 width;
> + u32 height;
> + u8 reserved[4];
> + u64 addr;
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_raw_params - KMB Raw parameters
> + *
> + * @awb_stats_en: Enable AE/AWB stats output
> + * @awb_rgb_hist_en: Enable RGB histogram output
> + * @af_stats_en: Enable AF stats output
> + * @luma_hist_en: Enable Luma histogram output
> + * @flicker_accum_en: Enable flicker detection row accumulation output
> + * @bad_pixel_fix_en: Enable Hot/Cold pixel suppression
> + * @grgb_imb_en: Enable Gr/Gb imbalance correction
> + * @mono_imbalance_en: Enable mono imbalance correction
> + * @gain1: Raw gain1 parameter
> + * @gain2: Raw gain2 parameter
> + * @gain3: Raw gain3 parameter
> + * @gain4: Raw gain4 parameter
> + * @stop1: Raw stop1 parameter
> + * @stop2: Raw stop2 parameter
> + * @stop3: Raw stop3 parameter
> + * @stop4: Raw stop4 parameter
> + * @threshold1: Raw threshold1 parameter
> + * @alpha1: Raw alpha1 parameter
> + * @alpha2: Raw alpha2 parameter
> + * @alpha3: Raw alpha3 parameter
> + * @alpha4: Raw alpha4 parameter
> + * @threshold2: Raw threshold2 parameter
> + * @static_defect_size: Static defect data size
> + * @reserved: Reserved for alignment purpose
> + * @static_defect_addr: Static defect data address
> + * @flicker_first_row_acc: First row of flicker detection row accumulation
> + * @flicker_last_row_acc: First row of flicker detection row accumulation
> + * @stats: raw statistics buffers
> + */
> +struct kmb_vpu_raw_params {
> + u32 awb_stats_en;
> + u32 awb_rgb_hist_en;
> + u32 af_stats_en;
> + u32 luma_hist_en;
> + u32 flicker_accum_en;
> + u32 bad_pixel_fix_en;
> + u32 grgb_imb_en;
> + u32 mono_imbalance_en;
> + u32 gain1;
> + u32 gain2;
> + u32 gain3;
> + u32 gain4;
> + u32 stop1;
> + u32 stop2;
> + u32 stop3;
> + u32 stop4;
> + u32 threshold1;
> + u32 alpha1;
> + u32 alpha2;
> + u32 alpha3;
> + u32 alpha4;
> + u32 threshold2;
> + u32 static_defect_size;
> + u8 reserved[4];
> + u64 static_defect_addr;
> + u32 flicker_first_row_acc;
> + u32 flicker_last_row_acc;
> + struct kmb_vpu_raw_stats stats[KMB_VPU_MAX_EXPOSURES];
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_ae_awb_params - KMB AE/AWB statistics parameters
> + *
> + * @start_x: AE/AWB start_x parameter
> + * @start_y: AE/AWB start_y parameter
> + * @width: AE/AWB width parameter
> + * @height: AE/AWB height parameter
> + * @skip_x: AE/AWB skip_x parameter
> + * @skip_y: AE/AWB skip_y parameter
> + * @patches_x: AE/AWB patches_x parameter
> + * @patches_y: AE/AWB patches_y parameter
> + * @threshold1: AE/AWB threshold1 parameter
> + * @threshold2: AE/AWB threshold2 parameter
> + */
> +struct kmb_vpu_ae_awb_params {
> + u32 start_x;
> + u32 start_y;
> + u32 width;
> + u32 height;
> + u32 skip_x;
> + u32 skip_y;
> + u32 patches_x;
> + u32 patches_y;
> + u16 threshold1;
> + u16 threshold2;
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_af_params - KMB Auto Focus parameters
> + *
> + * @start_x: AF start_x parameter
> + * @start_y: AF start_y parameter
> + * @width: AF width parameter
> + * @height: AF height parameter
> + * @patches_x: AF patches_x parameter
> + * @patches_y: AF patches_y parameter
> + * @coeff: AF filter coeff parameter
> + * @threshold1: AF filer threshold1 parameter
> + * @threshold2: AF filer threshold2 parameter
> + * @coeffs1: AF filter coeffs1 parameter
> + * @coeffs2: AF filter coeffs2 parameter
> + */
> +struct kmb_vpu_af_params {
> + u32 start_x;
> + u32 start_y;
> + u32 width;
> + u32 height;
> + u32 patches_x;
> + u32 patches_y;
> + s32 coeff;
> + s32 threshold1;
> + s32 threshold2;
> + s32 coeffs1[11];
> + s32 coeffs2[11];
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_hist_params - KMB Hist parameters
> + *
> + * @start_x: Hist start_x parameter
> + * @start_y: Hist start_y parameter
> + * @end_x: Hist end_x parameter
> + * @end_y: Hist end_y parameter
> + * @matrix: Hist matrix parameter
> + * @weight: Hist weight parameter
> + */
> +struct kmb_vpu_hist_params {
> + u32 start_x;
> + u32 start_y;
> + u32 end_x;
> + u32 end_y;
> + u16 matrix[9];
> + u16 weight[3];
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_lca_params - KMB Lateral Chromatic Aberration parameters
> + *
> + * @addr: LCA table address
> + */
> +struct kmb_vpu_lca_params {
> + u64 addr;
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_debayer_params - KMB Debayer parameters
> + *
> + * @coeff1: Filter coeff1 parameter
> + * @multiplier1: Filter multiplier1 parameter
> + * @multiplier2: Filter multiplier2 parameter
> + * @coeff2: Filter coeff2 parameter
> + * @coeff3: Filter coeff3 parameter
> + * @coeff4: Filter coeff4 parameter
> + */
> +struct kmb_vpu_debayer_params {
> + s32 coeff1;
> + u32 multiplier1;
> + u32 multiplier2;
> + s32 coeff2;
> + s32 coeff3;
> + s32 coeff4;
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_hdr_params - KMB HDR parameters
> + *
> + * @ratio: HDR ratio parameter
> + * @scale: HDR scale parameter
> + * @offset1: HDR offset1 parameter
> + * @slope1: HDR slope1 parameter
> + * @offset2: HDR offset2 parameter
> + * @slope2: HDR slope2 parameter
> + * @offset3: HDR offset3 parameter
> + * @slope3: HDR slope3 parameter
> + * @offset4: HDR offset4 parameter
> + * @gain1: HDR gain1 parameter
> + * @blur1: HDR blur1 parameter
> + * @blur2: HDR blur2 parameter
> + * @contrast1: HDR contrast1 parameter
> + * @contrast2: HDR contrast2 parameter
> + * @enable1: HDR enable1 parameter
> + * @enable2: HDR enable2 parameter
> + * @offset5: HDR offset5 parameter
> + * @gain2: HDR gain2 parameter
> + * @offset6: HDR offset6 parameter
> + * @strength: HDR strength parameter
> + * @reserved1: Reserved for alignment purpose
> + * @luts_addr: HDR LUT address
> + * @offset7: HDR offset7 parameter
> + * @shift: HDR shift parameter
> + * @field1: HDR filed1 parameter
> + * @field2: HDR field2 parameter
> + * @gain3: HDR gain3 parameter
> + * @min: HDR min parameter
> + * @reserved2: Reserved for alignment purpose
> + */
> +struct kmb_vpu_hdr_params {
> + u32 ratio[2];
> + u32 scale[3];
> + s32 offset1;
> + u32 slope1;
> + s32 offset2;
> + u32 slope2;
> + s32 offset3;
> + u32 slope3;
> + s32 offset4;
> + u32 gain1;
> + u32 blur1[3];
> + u32 blur2[5];
> + u32 contrast1;
> + u32 contrast2;
> + u32 enable1;
> + u32 enable2;
> + s32 offset5;
> + u32 gain2;
> + s32 offset6;
> + u32 strength;
> + u8 reserved1[4];
> + u64 luts_addr;
> + u16 offset7;
Reserved field missing here?
> + u32 shift;
> + u16 field1;
> + u16 field2;
> + u8 gain3;
> + u16 min;
> + u8 reserved2[3];
Why three bytes? This would mean non-byte fields in kmb_vpu_isp_params will
be unaligned after the hdr field. I'd add reserved fields so that there
would be no need for unaligned accesses.
I do understand this may be part of a firmware interface so making such
changes likely would need to be propagated there, so just FYI.
There appear to be similar issues elsewhere in these structs.
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_dog_dns_params - KMB Difference-of-Gaussians DNS parameters
> + *
> + * @threshold: Filter threshold parameter
> + * @strength: Filter strength parameter
> + * @coeffs11: Filter coeffs11 parameter
> + * @coeffs15: Filter coeffs15 parameter
> + * @reserved: Reserved for alignment purpose
> + */
> +struct kmb_vpu_dog_dns_params {
> + u32 threshold;
> + u32 strength;
> + u8 coeffs11[6];
> + u8 coeffs15[8];
> + u8 reserved[2];
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_luma_dns_params - KMB Luma DNS parameters
> + *
> + * @threshold: Luma DNS threshold parameter
> + * @slope: Luma DNS slope parameter
> + * @shift: Luma DNS shift parameter
> + * @alpha: Luma DNS alpha parameter
> + * @weight: Luma DNS weight parameter
> + * @per_pixel_alpha_en: Enable adapt alpha
> + * @gain_bypass_en: Enable gain bypass
> + * @reserved: for alignment purpose
> + */
> +struct kmb_vpu_luma_dns_params {
> + u32 threshold;
> + u32 slope;
> + u32 shift;
> + u32 alpha;
> + u32 weight;
> + u32 per_pixel_alpha_en;
> + u32 gain_bypass_en;
> + u8 reserved[4];
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_sharpen_params - KMB Sharpen parameters
> + *
> + * @coeffs1: Filter coeffs1 parameter
> + * @coeffs2: Filter coeffs2 parameter
> + * @coeffs3: Filter coeffs3 parameter
> + * @shift: Filter shift parameter
> + * @gain1: Filter gain1 parameter
> + * @gain2: Filter gain2 parameter
> + * @gain3: Filter gain3 parameter
> + * @gain4: Filter gain4 parameter
> + * @gain5: Filter gain5 parameter
> + * @stops1: Filter stops1 parameter
> + * @gains: Filter gains parameter
> + * @stops2: Filter stops2 parameter
> + * @overshoot: Filter overshoot parameter
> + * @undershoot: Filter undershoot parameter
> + * @alpha: Filter alpha parameter
> + * @gain6: Filter gain6 parameter
> + * @offset: Filter offset parameter
> + * @addr: Filter data address
> + */
> +struct kmb_vpu_sharpen_params {
> + u16 coeffs1[6];
> + u16 coeffs2[6];
> + u16 coeffs3[6];
> + u32 shift;
> + u32 gain1;
> + u32 gain2;
> + u32 gain3;
> + u32 gain4;
> + u32 gain5;
> + u32 stops1[3];
> + u32 gains[3];
> + u32 stops2[4];
> + u32 overshoot;
> + u32 undershoot;
> + u32 alpha;
> + u32 gain6;
> + u32 offset;
> + u64 addr;
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_chroma_gen_params - KMB Chroma GEN parameters
> + *
> + * @epsilon: Chroma GEN epsilon parameter
> + * @coeff1: Chroma GEN coeff1 parameter
> + * @coeff2: Chroma GEN coeff2 parameter
> + * @coeff3: Chroma GEN coeff3 parameter
> + * @coeff4: Chroma GEN coeff4 parameter
> + * @coeff5: Chroma GEN coeff5 parameter
> + * @coeff6: Chroma GEN coeff6 parameter
> + * @strength1: Chroma GEN strength1 parameter
> + * @strength2: Chroma GEN strength2 parameter
> + * @coeffs: Chroma GEN coeffs parameter
> + * @offset1: Chroma GEN offset1 parameter
> + * @slope1: Chroma GEN slope1 parameter
> + * @slope2: Chroma GEN slope2 parameter
> + * @offset2: Chroma GEN offset2 parameter
> + * @limit: Chroma GEN limit parameter
> + */
> +struct kmb_vpu_chroma_gen_params {
> + u32 epsilon;
> + u32 coeff1;
> + u32 coeff2;
> + u32 coeff3;
> + u32 coeff4;
> + u32 coeff5;
> + u32 coeff6;
> + u32 strength1;
> + u32 strength2;
> + u32 coeffs[3];
> + s32 offset1;
> + u32 slope1;
> + u32 slope2;
> + s32 offset2;
> + u32 limit;
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_median_params - KMB Median parameters
> + *
> + * @size: Filter size parameter
> + * @slope: Filter slope parameter
> + * @offset: Filter offset parameter
> + */
> +struct kmb_vpu_median_params {
> + u32 size;
> + u32 slope;
> + s32 offset;
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_chroma_dns_params - KMB Chroma Denoise parameters
> + *
> + * @limit: Filter limit parameter
> + * @enable: Filter enable parameter
> + * @threshold1: Filter threshold1 parameter
> + * @threshold2: Filter threshold2 parameter
> + * @threshold3: Filter threshold3 parameter
> + * @threshold4: Filter threshold4 parameter
> + * @threshold5: Filter threshold5 parameter
> + * @threshold6: Filter threshold6 parameter
> + * @threshold7: Filter threshold7 parameter
> + * @threshold8: Filter threshold8 parameter
> + * @slope1: Filter slope1 parameter
> + * @offset1: Filter offset1 parameter
> + * @slope2: Filter slope2 parameter
> + * @offset2: Filter offset2 parameter
> + * @grey1: Filter grey1 parameter
> + * @grey2: Filter grey2 parameter
> + * @grey3: Filter grey3 parameter
> + * @coeff1: Filter coeff1 parameter
> + * @coeff2: Filter coeff2 parameter
> + * @coeff3: Filter coeff3 parameter
> + */
> +struct kmb_vpu_chroma_dns_params {
> + u32 limit;
> + u32 enable;
> + u32 threshold1;
> + u32 threshold2;
> + u32 threshold3;
> + u32 threshold4;
> + u32 threshold5;
> + u32 threshold6;
> + u32 threshold7;
> + u32 threshold8;
> + u32 slope1;
> + s32 offset1;
> + u32 slope2;
> + s32 offset2;
> + u32 grey1;
> + u32 grey2;
> + u32 grey3;
> + u32 coeff1;
> + u32 coeff2;
> + u32 coeff3;
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_color_comb_params - KMB Color Combine parameters
> + *
> + * @matrix: Color combine matrix parameter
> + * @offsets:Color combine offsets parameter
> + * @coeff1: Color combine coeff1 parameter
> + * @coeff2: Color combine coeff2 parameter
> + * @coeff3: Color combine coeff3 parameter
> + * @reserved: Reserved for alignment purpose
> + * @addr: Color combine table address
> + * @enable: Color combine enable parameter
> + * @weight1: Color combine weight1 parameter
> + * @weight2: Color combine weight2 parameter
> + * @weight3: Color combine weight3 parameter
> + * @limit1: Color combine limit1 parameter
> + * @limit2: Color combine limit2 parameter
> + * @offset1: Color combine offset1 parameter
> + * @offset2: Color combine offset2 parameter
> + */
> +struct kmb_vpu_color_comb_params {
> + u16 matrix[9];
> + u16 offsets[3];
> + u32 coeff1;
> + u32 coeff2;
> + u32 coeff3;
> + u8 reserved[4];
> + u64 addr;
> + u32 enable;
> + u32 weight1;
> + u32 weight2;
> + u32 weight3;
> + u32 limit1;
> + s32 limit2;
> + s32 offset1;
> + s32 offset2;
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_lut_params - KMB lut parameters
> + *
> + * @size: Lut size parameter
> + * @reserved: Reserved for alignment purpose
> + * @addr: Lut table address
> + * @matrix: Lut matrix parameter
> + * @offsets: Lut offsets parameter
> + */
> +struct kmb_vpu_lut_params {
> + u32 size;
> + u8 reserved[4];
> + u64 addr;
> + u16 matrix[3 * 3];
> + u16 offsets[3];
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_tnf_params - KMB Temporal Noise Filter parameters
> + *
> + * @factor: Filter factor parameter
> + * @gain: Filter gain parameter
> + * @offset1: Filter offset1 parameter
> + * @slope1: Filter slope1 parameter
> + * @offset2: Filter offset2 parameter
> + * @slope2: Filter slope2 parameter
> + * @min1: Filter min1 parameter
> + * @min2: Filter min2 parameter
> + * @value: Filter value parameter
> + * @enable: Filter enable parameter
> + * @lut0_addr: Filter lut0 address
> + * @lut1_addr: Filter lut1 address
> + */
> +struct kmb_vpu_tnf_params {
> + u32 factor;
> + u32 gain;
> + u32 offset1;
> + u32 slope1;
> + u32 offset2;
> + u32 slope2;
> + u32 min1;
> + u32 min2;
> + u32 value;
> + u32 enable;
> + u64 lut0_addr;
> + u64 lut1_addr;
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_dehaze_params - KMB dehaze parameters
> + *
> + * @gain1: Dehaze gain1 parameter
> + * @min: Dehaze min parameter
> + * @strength1: Dehaze strength1 parameter
> + * @strength2: Dehaze strength2 parameter
> + * @gain2: Dehaze gain2 parameter
> + * @saturation: Dehaze saturation parameter
> + * @value1: Dehaze value1 parameter
> + * @value2: Dehaze value2 parameter
> + * @value3: Dehaze value3 parameter
> + * @filter: Dehaze filter parameter
> + * @stats_addr: Dehaze statistics address
> + */
> +struct kmb_vpu_dehaze_params {
> + u32 gain1;
> + u32 min;
> + u32 strength1;
> + u32 strength2;
> + u32 gain2;
> + u32 saturation;
> + u32 value1;
> + u32 value2;
> + u32 value3;
> + u32 filter[3];
> + u64 stats_addr;
> +} __packed;
> +
> +/**
> + * struct kmb_vpu_warp_params - KMB Warp filter parameters
> + *
> + * @type: Warp filter type parameter
> + * @relative: Warp filter relative parameter
> + * @format: Warp filter format parameter
> + * @position: Warp filter position parameter
> + * @reserved: Reserved for alignment purposes
> + * @addr: Warp filter addr parameter
> + * @width: Warp filter width parameter
> + * @height: Warp filter height parameter
> + * @stride: Warp filter stride parameter
> + * @enable: Warp filter enable parameter
> + * @matrix: Warp matrix parameter
> + * @mode: Warp filter mode parameter
> + * @values: Warp filter values parameter
> + */
> +struct kmb_vpu_warp_params {
> + u8 type;
> + u8 relative;
> + u8 format;
> + u8 position;
> + u8 reserved[4];
> + u64 addr;
> + u16 width;
> + u16 height;
> + u32 stride;
> + u8 enable;
> + u32 matrix[9];
> + u8 mode;
> + u16 values[3];
> +} __packed;
> +
> +/**
> + * enum kmb_vpu_bayer_order - KMB sensor Bayer arrangement format types
> + *
> + * @KMB_ISP_BAYER_ORDER_GRBG: Gr R B Gr
> + * @KMB_ISP_BAYER_ORDER_RGGB: R Gr Gr B
> + * @KMB_ISP_BAYER_ORDER_GBRG: Gr B R Gr
> + * @KMB_ISP_BAYER_ORDER_BGGR: B Gr Gr R
> + */
> +enum kmb_vpu_bayer_order {
> + KMB_VPU_ISP_BAYER_ORDER_GRBG = 0,
> + KMB_VPU_ISP_BAYER_ORDER_RGGB = 1,
> + KMB_VPU_ISP_BAYER_ORDER_GBRG = 2,
> + KMB_VPU_ISP_BAYER_ORDER_BGGR = 3,
> +} __packed;
A packed enum? :-)
> +
> +/* Version of the VPU ISP ABI. It should be passed as
> + * first argument in the isp params struct
> + */
> +#define KMB_VPU_ISP_ABI_VERSION 104
> +
> +/**
> + * struct kmb_vpu_isp_params - KMB VPU ISP parameters structure
> + *
> + * @header_version: Header Version
> + * @image_data_width: Image data width
> + * @num_exposures: Number of exposures
> + * @bayer_order: enum kmb_isp_bayer_order
> + * @user_data_key: Private key used for the client
> + * @blc: Black Level correction parameters
> + * @sigma_dns: Sigma denoise parameters
> + * @lsc: Lens Shading Correction parameters
> + * @raw: Raw parameters
> + * @ae_awb: Auto exposure/Auto white balance parameters
> + * @af: Auto focus parameters
> + * @histogram: Histogram parameters
> + * @lca: Lateral Chromatic Aberration filter parameters
> + * @debayer: SIPP Bayer demosaicing filter parameters
> + * @dog_dns: Difference-of-Gaussians filter parameters
> + * @luma_dns: Luma denoise parameters
> + * @sharpen: Sharpen filter parameters
> + * @chroma_gen: Chroma GEN parameters
> + * @median: Median hardware filter parameters
> + * @chroma_dns: Chroma Denoise hardware filter parameters
> + * @color_comb: Color Combine parameters
> + * @hdr: HDR parameters applied only in HDR mode
> + * @lut: lut parameters
> + * @tnf: Temporal Noise Filter parameters
> + * @dehaze: Dehaze parameters
> + * @warp: Warp filter parameters
> + */
> +struct kmb_vpu_isp_params {
> + u32 header_version;
> + u32 image_data_width;
> + u32 num_exposures;
> + u32 bayer_order;
> + u32 user_data_key;
> + struct kmb_vpu_blc_params blc[KMB_VPU_MAX_EXPOSURES];
> + struct kmb_vpu_sigma_dns_params sigma_dns[KMB_VPU_MAX_EXPOSURES];
> + struct kmb_vpu_lsc_params lsc;
> + struct kmb_vpu_raw_params raw;
> + struct kmb_vpu_ae_awb_params ae_awb;
> + struct kmb_vpu_af_params af;
> + struct kmb_vpu_hist_params histogram;
> + struct kmb_vpu_lca_params lca;
> + struct kmb_vpu_debayer_params debayer;
> + struct kmb_vpu_dog_dns_params dog_dns;
> + struct kmb_vpu_luma_dns_params luma_dns;
> + struct kmb_vpu_sharpen_params sharpen;
> + struct kmb_vpu_chroma_gen_params chroma_gen;
> + struct kmb_vpu_median_params median;
> + struct kmb_vpu_chroma_dns_params chroma_dns;
> + struct kmb_vpu_color_comb_params color_comb;
> + struct kmb_vpu_hdr_params hdr;
> + struct kmb_vpu_lut_params lut;
> + struct kmb_vpu_tnf_params tnf;
> + struct kmb_vpu_dehaze_params dehaze;
> + struct kmb_vpu_warp_params warp;
> +} __packed;
> +
> +#endif /* KEEMBAY_VPU_ISP */
> diff --git a/drivers/media/platform/keembay-camera/keembay-vpu-pipe.h b/drivers/media/platform/keembay-camera/keembay-vpu-pipe.h
> new file mode 100644
> index 000000000000..d400b59938b2
> --- /dev/null
> +++ b/drivers/media/platform/keembay-camera/keembay-vpu-pipe.h
> @@ -0,0 +1,110 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Intel Keem Bay camera VPU pipe definitions
> + *
> + * Copyright (C) 2021 Intel Corporation
> + */
> +#ifndef KEEMBAY_VPU_PIPE_H
> +#define KEEMBAY_VPU_PIPE_H
> +
> +#include "keembay-vpu-src.h"
> +
> +#define PIPE_TYPE_ISP_MAX_EXP 3
> +
> +enum {
> + PIPE_TYPE_ISP_ISP_ULL = 1,
> + PIPE_TYPE_ISP_ISP_2DOL,
> + PIPE_TYPE_ISP_ISP_3DOL,
> + PIPE_TYPE_ISP_ISP_MONO,
> +
> + PIPE_TYPE_MAX,
> +};
> +
> +enum {
> + SRC_TYPE_ALLOC_VPU_DATA_MIPI = 0,
> + SRC_TYPE_ALLOC_VPU_DATA_DBG,
> + SRC_TYPE_ALLOC_ARM_DATA_ARM,
> + SRC_TYPE_ALLOC_ARM_DATA_MIPI,
> + SRC_TYPE_ALLOC_ARM_DATA_DBG,
> +
> + SRC_TYPE_ALLOC_DATA_MAX,
> +};
> +
> +enum {
> + PIPE_TRANSFORM_HUB_NONE = 0,
> + PIPE_TRANSFORM_HUB_BASIC,
> + PIPE_TRANSFORM_HUB_FULL,
> + PIPE_TRANSFORM_HUB_STITCH,
> + PIPE_TRANSFORM_HUB_EPTZ,
> +
> + PIPE_TRANSFORM_HUB_MAX,
> +};
> +
> +enum {
> + PIPE_OUTPUT_ID_RAW = 0,
> + PIPE_OUTPUT_ID_ISP_CTRL,
> + PIPE_OUTPUT_ID_0,
> + PIPE_OUTPUT_ID_1,
> + PIPE_OUTPUT_ID_2,
> + PIPE_OUTPUT_ID_3,
> + PIPE_OUTPUT_ID_4,
> + PIPE_OUTPUT_ID_5,
> + PIPE_OUTPUT_ID_6,
> +
> + PIPE_OUTPUT_ID_MAX,
> +};
> +
> +/*
> + * struct kmb_channel_cfg - KMB channel configuration
> + *
> + * @id: Channel id
> + * @frm_res: Frame resolution
> + */
> +struct kmb_channel_cfg {
> + u32 id;
> + struct kmb_ic_img_size frm_res;
> +};
> +
> +/*
> + * struct kmb_pipe_config_evs - VPU pipeline configuration
> + *
> + * @pipe_id: Pipe id
> + * @pipe_type: Pipe type
> + * @src_type: Source type
It'd be nice to name the enums and refer to them from here (as in "&enum
nameofenum").
> + * @pipe_trans_hub: Transform hub type
> + * @in_isp_res: Input ISP resolution
> + * @out_isp_res: Output isp resolution
> + * @in_isp_stride: ISP input stride used in DOL interleaved mode
> + * @in_exp_offsets: Long and short exp frames offsets used in interleaved mode
> + * @out_min_res: Output min resolution
> + * @out_max_res: Output max resolution
> + * @pipe_xlink_chann: Output channel id from the enum PIPE_OUTPUT_ID
> + * @keep_aspect_ratio: If enabled, aspect ratio must be kept when image is
> + * resized
> + * @in_data_width: Input bits per pixel
> + * @in_data_packed: Flag to enable packed mode
> + * @out_data_width: Output bits per pixel for first plane
> + * @internal_memory_addr: Internal memory pool address
> + * @internal_memory_size: Internal memory pool size
> + */
> +struct kmb_pipe_config_evs {
> + u8 pipe_id;
> + u8 pipe_type;
> + u8 src_type;
> + u8 pipe_trans_hub;
> + struct kmb_ic_img_size in_isp_res;
> + struct kmb_ic_img_size out_isp_res;
> + u16 in_isp_stride;
> + u32 in_exp_offsets[PIPE_TYPE_ISP_MAX_EXP];
> + struct kmb_ic_img_size out_min_res[PIPE_OUTPUT_ID_MAX];
> + struct kmb_ic_img_size out_max_res[PIPE_OUTPUT_ID_MAX];
> + struct kmb_channel_cfg pipe_xlink_chann[PIPE_OUTPUT_ID_MAX];
> + u8 keep_aspect_ratio;
> + u8 in_data_width;
> + u8 in_data_packed;
> + u8 out_data_width;
> + u64 internal_memory_addr;
> + u32 internal_memory_size;
> +} __aligned(64);
__packed as well?
> +
> +#endif /* KEEMBAY_VPU_PIPE_H */
> diff --git a/drivers/media/platform/keembay-camera/keembay-vpu-src.h b/drivers/media/platform/keembay-camera/keembay-vpu-src.h
> new file mode 100644
> index 000000000000..97f8febbc7e2
> --- /dev/null
> +++ b/drivers/media/platform/keembay-camera/keembay-vpu-src.h
> @@ -0,0 +1,193 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Intel Keem Bay camera VPU source configuration
> + *
> + * Copyright (C) 2021 Intel Corporation
> + */
> +
> +#ifndef KEEMBAY_VPU_SRC_H
> +#define KEEMBAY_VPU_SRC_H
> +
> +/*
> + * struct kmb_ic_img_size - The structure contains information about image size
> + *
> + * @w: Image width
> + * @h: Image height
> + */
> +struct kmb_ic_img_size {
> + u32 w;
> + u32 h;
> +};
> +
> +/*
> + * struct kmb_ic_img_rect - The struct represents the coordinates of a
> + * rectangular image
> + *
> + * @x1: Position of the bottom left corner
> + * @y1: Position of the top left corner
> + * @x2: Position of the bottom right corner
> + * @y2: Position of the top right corner
> + */
> +struct kmb_ic_img_rect {
> + s32 x1;
> + s32 y1;
> + s32 x2;
> + s32 y2;
> +};
> +
> +/*
> + * enum kmb_ic_source_instance - HW mipi/cif input devices
> + *
> + * @KMB_IC_SOURCE_0:
> + * @KMB_IC_SOURCE_1:
> + * @KMB_IC_SOURCE_2:
> + * @KMB_IC_SOURCE_3:
> + * @KMB_IC_SOURCE_4:
> + * @KMB_IC_SOURCE_5:
> + */
> +enum kmb_ic_source_instance {
> + KMB_IC_SOURCE_0 = 0,
> + KMB_IC_SOURCE_1 = 1,
> + KMB_IC_SOURCE_2 = 2,
> + KMB_IC_SOURCE_3 = 3,
> + KMB_IC_SOURCE_4 = 4,
> + KMB_IC_SOURCE_5 = 5,
> +};
> +
> +/*
> + * enum kmb_ic_bayer_format - Bayer pattern order
> + *
> + * @KMB_IC_BAYER_FORMAT_GRBG: Gr R B Gr
> + * @KMB_IC_BAYER_FORMAT_RGGB: R Gr Gr B
> + * @KMB_IC_BAYER_FORMAT_GBRG: Gr B R Gr
> + * @KMB_IC_BAYER_FORMAT_BGGR: B Gr Gr R
> + */
> +enum kmb_ic_bayer_format {
> + KMB_IC_BAYER_FORMAT_GRBG = 0,
> + KMB_IC_BAYER_FORMAT_RGGB = 1,
> + KMB_IC_BAYER_FORMAT_GBRG = 2,
> + KMB_IC_BAYER_FORMAT_BGGR = 3,
> +};
> +
> +/*
> + * enum kmb_ic_mipi_rx_ctrl_rec_not - List of receiver Id's for a specific
> + * sensor
> + *
> + * @KMB_IC_SIPP_DEVICE0:
> + * @KMB_IC_SIPP_DEVICE1:
> + * @KMB_IC_SIPP_DEVICE2:
> + * @KMB_IC_SIPP_DEVICE3:
> + * @KMB_IC_CIF0_DEVICE4:
> + * @KMB_IC_CIF1_DEVICE5:
> + */
> +enum kmb_ic_mipi_rx_ctrl_rec_not {
> + KMB_IC_SIPP_DEVICE0 = 0,
> + KMB_IC_SIPP_DEVICE1 = 1,
> + KMB_IC_SIPP_DEVICE2 = 2,
> + KMB_IC_SIPP_DEVICE3 = 3,
> + KMB_IC_CIF0_DEVICE4 = 4,
> + KMB_IC_CIF1_DEVICE5 = 5,
> +};
> +
> +/*
> + * enum kmb_ic_mipi_rx_ctrl_not - MIPI controller from chip
> + *
> + * @KMB_IC_MIPI_CTRL_0:
> + * @KMB_IC_MIPI_CTRL_1:
> + * @KMB_IC_MIPI_CTRL_2:
> + * @KMB_IC_MIPI_CTRL_3:
> + * @KMB_IC_MIPI_CTRL_4:
> + * @KMB_IC_MIPI_CTRL_5:
> + */
> +enum kmb_ic_mipi_rx_ctrl_not {
> + KMB_IC_MIPI_CTRL_0 = 0,
> + KMB_IC_MIPI_CTRL_1 = 1,
> + KMB_IC_MIPI_CTRL_2 = 2,
> + KMB_IC_MIPI_CTRL_3 = 3,
> + KMB_IC_MIPI_CTRL_4 = 4,
> + KMB_IC_MIPI_CTRL_5 = 5,
> +};
> +
> +/*
> + * enum kmb_ic_mipi_ex_data_type - All supported raw, sensor input formats
> + *
> + * @IC_IPIPE_YUV_420_B8:
> + * @IC_IPIPE_RAW_8:
> + * @IC_IPIPE_RAW_10:
> + * @IC_IPIPE_RAW_12:
> + * @IC_IPIPE_RAW_14:
> + * @IC_IPIPE_EMBEDDED_8BIT:
> + */
> +enum kmb_ic_mipi_rx_data_type {
> + IC_IPIPE_YUV_420_B8 = 0x18,
> + IC_IPIPE_RAW_8 = 0x2A,
> + IC_IPIPE_RAW_10 = 0x2B,
> + IC_IPIPE_RAW_12 = 0x2C,
> + IC_IPIPE_RAW_14 = 0x2D,
> + IC_IPIPE_EMBEDDED_8BIT = 0x12
> +};
> +
> +/*
> + * struct kmb_ic_source_config_dynamic - Per-source configuration of parameters
> + * which can be modified dynamically.
> + * Setting will take effect during the
> + * next blanking interval
> + *
> + * @notification_line: Line number upon which IC_EVENT_TYPE_LINE will be sent
> + * to the Lean OS. Set to -1 to disable notification
> + */
> +struct kmb_ic_source_config_dynamic {
> + s32 notification_line;
> +};
> +
> +/*
> + * struct kmb_ic_mipi_config - Mipi RX data configuration
> + *
> + * @no_controller: Number of controller
> + * @no_lanes: Number of lanes
> + * @lane_rate_mbps: Lane rate
> + * @data_type: Mipi RX data type
> + * @data_mode: Data mode
> + * @rec_nrl:
> + */
> +struct kmb_ic_mipi_config {
> + u32 no_controller;
> + u32 no_lanes;
> + u32 lane_rate_mbps;
> + u32 data_type;
> + u32 data_mode;
> + u32 rec_nrl;
> +};
> +
> +/*
> + * struct kmb_ic_source_config - Per-source configuration parameters - mostly
> + * information needed to configure the MIPI Rx
> + * filter
> + *
> + * @camera_output_size: Max frame size output by the camera
> + * @crop_window: Crop window coordinates
> + * @bayer_format: Bayer Format - Raw, Demosaic and LSC blocks should be
> + * programmed to match the Bayer order specified here.
> + * @bpp: Bits per pixel
> + * @mipi_rx_data: MIPI RX data configuration
> + * @no_exposure: Number of different exposure frames
> + * @metadata_width: Metadata width
> + * @metadata_height: Medata height
> + * @metadata_data_type: Metadata data type
> + */
> +struct kmb_ic_source_config {
> + struct kmb_ic_img_size camera_output_size;
> + struct kmb_ic_img_rect crop_window;
> +
> + u32 bayer_format;
> + u32 bpp;
> +
> + struct kmb_ic_mipi_config mipi_rx_data;
> +
> + u32 no_exposure;
> + u32 metadata_width;
> + u32 metadata_height;
> + u32 metadata_data_type;
> +} __aligned(64);
> +
> +#endif /* KEEMBAY_VPU_SRC_H */
--
Kind regards,
Sakari Ailus
^ permalink raw reply [flat|nested] 28+ messages in thread* RE: [PATCH 03/10] media: Keem Bay Camera: Add VPU camera interface
2021-04-09 12:01 ` Sakari Ailus
@ 2021-04-09 14:39 ` Martina Krasteva
0 siblings, 0 replies; 28+ messages in thread
From: Martina Krasteva @ 2021-04-09 14:39 UTC (permalink / raw)
To: 'Sakari Ailus'
Cc: linux-media, mchehab, robh+dt, devicetree, daniele.alessandrelli,
paul.j.murphy, gjorgjix.rosikopulos
Hi Sakari,
>
> Hi Martina,
>
> On Fri, Mar 19, 2021 at 06:06:25PM +0000, Martina Krasteva wrote:
> > From: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
> >
> > Communication with VPU firmware is over XLink.
> > XLink has a channel-based communication, each channel has a unique
> > ID. The communication between VPU FW and camera driver starts with
> > one channel with negotiated ID. Currently this ID it is hard-coded
> > on both sides and should not be changed.
> >
> > Three main channel types are used for streaming session:
> >
> > 1. Pipeline management channel:
> > This is fixed channel used to configure/build/delete
> > the streaming pipelines. When pipeline is built the channel IDs
> > used for communication are provided from linux kernel camera
> > driver.
> >
> > 2. Isp control channel:
> > This channel is used for ISP and MIPI RX configuration. For each
> > pipeline a separate ISP channel is required.
> >
> > 3. Buffer pool channels:
> > Each endpoint and buffer pool from VPU FW is associated with
> > a separate XLink channel. This channel is used for buffer
> > management.
> >
> > Messages in "1" and "2" are using cmd sturct as payload data which
> > contains message type and physical address containing message payload.
> > "3" messages are small and the whole message is in XLink payload data.
> >
> > Pipeline management:
> >
> > Each pipeline instance is created on pipeline XLink channel "1".
> > The pipeline lifecycle states are:
> >
> > - Configuration: Pipeline mode and input resolution are sent,
> > as a result min/max resolutions for the available outputs in that mode
> > are received.
> >
> > - Build: The returned pipeline configuration is passed to build
> > command in addition with filled output channel configurations for
> > each output endpoint. After this command pipeline is ready for
> > streaming and can accept messages on ISP and buffer pool channels.
> >
> > - Delete: The command deletes constructed pipeline.
> >
> > NOTE: Now pipeline lifecycle should be always
> > configuration->build->delete it is not allowed to mix
> > the states. However an request was sent to VPU firmware team to
> > be able to delete configured pipeline without going in build state.
> >
> > ISP control:
> >
> > ISP control channel is used for controlling VPU ISP. This includes:
> > - Isp source configuration: MIPI RX configuration
> > - Isp source start/stop: start/stop MIPI RX
> > - Sending ISP params for processing
> >
> > Events from VPU ISP are also received on ISP control channel.
> > Those are MIPI RX events, ISP events and error events.
> >
> > VPU ISP works in per-frame control mode - ISP configuration is
> > required for every processed endpoint.
> > ISP configuration has a lifecycle.
> > The following event sequence needs to be received for each ISP
> > configuration, then it is released by the VPU:
> > - Readout start - MIPI SOF
> > - Readout end - MIPI EOF
> > - Isp start - Isp processing SOF
> > - Isp end - Isp processing EOF
> >
> > If VPU wants to discard ISP configuration because of some internal
> > error, ISP configuration skip event is sent.
> >
> > NOTE: Received events' payload data contains an ISP configuration
> > address this event corresponds to or 0 if the event is not for
> > ISP configuration.
> >
> > Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
> > Co-developed-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
> > Signed-off-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
> > Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
> > Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
> > Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
> > ---
> > .../platform/keembay-camera/keembay-vpu-cmd.h | 110 ++++
> > .../platform/keembay-camera/keembay-vpu-frame.h | 102 +++
> > .../platform/keembay-camera/keembay-vpu-isp.h | 724 +++++++++++++++++++++
> > .../platform/keembay-camera/keembay-vpu-pipe.h | 110 ++++
> > .../platform/keembay-camera/keembay-vpu-src.h | 193 ++++++
> > 5 files changed, 1239 insertions(+)
> > create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-cmd.h
> > create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-frame.h
> > create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-isp.h
> > create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-pipe.h
> > create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-src.h
> >
> > diff --git a/drivers/media/platform/keembay-camera/keembay-vpu-cmd.h b/drivers/media/platform/keembay-camera/keembay-vpu-
> cmd.h
> > new file mode 100644
> > index 000000000000..192deebf33c9
> > --- /dev/null
> > +++ b/drivers/media/platform/keembay-camera/keembay-vpu-cmd.h
> > @@ -0,0 +1,110 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * Intel Keem Bay camera VPU Commands
> > + *
> > + * Copyright (C) 2021 Intel Corporation
> > + */
> > +#ifndef KEEMBAY_VPU_CMD_H
> > +#define KEEMBAY_VPU_CMD_H
> > +
> > +enum {
> > + /* IC_EVENT_TYPE enum to define event messages */
> > + KMB_IC_EVENT_TYPE_SUCCESSFUL = 0,
> > + KMB_IC_EVENT_TYPE_CONFIG_ISP_PIPE,
> > + KMB_IC_EVENT_TYPE_BUILD_ISP_PIPE,
> > + KMB_IC_EVENT_TYPE_DELETE_ISP_PIPE,
> > +
> > + KMB_IC_EVENT_TYPE_INIT_MAX
> > +};
> > +
> > +enum {
> > + /* Control -> Source */
> > + KMB_IC_EVENT_TYPE_CONFIG_SOURCE = (KMB_IC_EVENT_TYPE_INIT_MAX + 1),
> > + KMB_IC_EVENT_TYPE_START_SOURCE,
> > + KMB_IC_EVENT_TYPE_STOP_SOURCE,
> > + KMB_IC_EVENT_TYPE_CONFIG_SOURCE_DYNAMIC,
> > + KMB_IC_EVENT_TYPE_SOURCE_SEND_USER_DATA,
> > +
> > + /* Source -> Control */
> > + KMB_IC_EVENT_TYPE_SOURCE_CONFIGURED,
> > + KMB_IC_EVENT_TYPE_SOURCE_STARTED,
> > + KMB_IC_EVENT_TYPE_SOURCE_STOPPED,
> > + KMB_IC_EVENT_TYPE_SOURCE_DYN_CONFIGURED,
> > +
> > + /* Source events */
> > + KMB_IC_EVENT_TYPE_READOUT_START,
> > + KMB_IC_EVENT_TYPE_READOUT_END,
> > + KMB_IC_EVENT_TYPE_LINE_REACHED,
> > +
> > + /* ISP events */
> > + KMB_IC_EVENT_TYPE_ISP_START,
> > + KMB_IC_EVENT_TYPE_ISP_END,
> > + KMB_IC_EVENT_TYPE_STATS_READY,
> > + KMB_IC_EVENT_TYPE_ISP_CONFIG_ACCEPTED,
> > + KMB_IC_EVENT_TYPE_ZSL_LOCKED,
> > + KMB_IC_EVENT_TYPE_CAPTURE_MADE,
> > +
> > + /* Isp config events */
> > + KMB_IC_EVENT_TYPE_CONFIG_ISP,
> > + KMB_IC_EVENT_TYPE_LOCK_ZSL,
> > + KMB_IC_EVENT_TYPE_CAPTURE,
> > + KMB_IC_EVENT_TYPE_UNLOCK_ZSL,
> > + KMB_IC_EVENT_TYPE_ZSL_ADD,
> > + KMB_IC_EVENT_TYPE_ERROR,
> > +
> > + KMB_IC_EVENT_MAX,
> > +};
> > +
> > +enum {
> > + KMB_IC_ERROR_PIPE_INIT = (KMB_IC_EVENT_MAX + 1),
> > + KMB_IC_ERROR_ISP_CONFIG,
> > + KMB_IC_ERROR_YUV_BUFF_MISSING,
> > +
> > + KMB_IC_ERROR_ISP_MAX,
> > +};
> > +
> > +enum {
> > + KMB_IC_ERROR_SRC_MIPI_WRONG_STATE = (KMB_IC_ERROR_ISP_MAX + 1),
> > + KMB_IC_ERROR_SRC_MIPI_BAD_PARAMETER,
> > + KMB_IC_ERROR_SRC_MIPI_CFG_MISSING,
> > + KMB_IC_ERROR_SRC_MIPI_CFG_SKIPPED,
> > + KMB_IC_ERROR_SRC_MIPI_OUT_BUFFERS_NOT_AVAILABLE,
> > + KMB_IC_ERROR_SRC_MIPI_EOF_TIMEOUT,
> > + KMB_IC_ERROR_SRC_MIPI_LOC_BUF_NOT_AVAILABLE,
> > + KMB_IC_ERROR_SRC_MIPI_INTERNAL_ERROR,
> > + KMB_IC_ERROR_SRC_TRANSMISSION_ERROR,
> > + KMB_IC_ERROR_SRC_DRIVER_UNEXPECTED,
> > +
> > + KMB_IC_ERROR_SRC_MIPI_MAX,
> > +};
> > +
> > +enum {
> > + KMB_IC_ERROR_NO_ZSL_BUFFS_AVAILABLE = (KMB_IC_ERROR_SRC_MIPI_MAX + 1),
> > + KMB_IC_ERROR_TRIGGER_NOT_EXISTING_BUFF,
> > +
> > + KMB_IC_ERROR_ISP_CTRL_MAX,
> > +};
> > +
> > +/**
> > + * struct kmb_ic_ev - Event structure
> > + *
> > + * @ev_info: Describe ISP event
> > + * @ev_info.inst_id: Pipe id
> > + * @ev_info.seq_nr: Frame number
> > + * @ev_info.user_data_base_addr01: Address of isp cfg buffer in CMA
> > + * @ev_info.user_data_base_addr02: Address of isp cfg buffer in CMA
> > + * @ev_info.ts: Timestamp in NS
> > + * @ctrl: Value from the IC_EVENT_TYPE enum
> > + */
> > +struct kmb_ic_ev {
> > + struct {
> > + u32 inst_id;
> > + u32 seq_nr;
> > + u32 user_data_base_addr01;
> > + u32 user_data_base_addr02;
> > + s64 ts;
> > + } ev_info;
> > + u32 ctrl;
> > +} __packed __aligned(64);
> > +
> > +#endif /* KEEMBAY_VPU_CMD_H */
> > diff --git a/drivers/media/platform/keembay-camera/keembay-vpu-frame.h b/drivers/media/platform/keembay-camera/keembay-vpu-
> frame.h
> > new file mode 100644
> > index 000000000000..aab99ab55077
> > --- /dev/null
> > +++ b/drivers/media/platform/keembay-camera/keembay-vpu-frame.h
> > @@ -0,0 +1,102 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * Intel Keem Bay camera VPU frame data
> > + *
> > + * Copyright (C) 2021 Intel Corporation
> > + */
> > +
> > +#ifndef KEEMBAY_VPU_FRAME_H_
> > +#define KEEMBAY_VPU_FRAME_H_
> > +
> > +/**
> > + * enum kmb_frame_types - Frame types
> > + *
> > + * @KMB_FRAME_TYPE_YUV422I: Interleaved 8 bit
> > + * @KMB_FRAME_TYPE_YUV444P: Planar 4:4:4 format
> > + * @KMB_FRAME_TYPE_YUV420P: Planar 4:2:0 format
> > + * @KMB_FRAME_TYPE_YUV422P: Planar 8-bit greyscale
> > + * @KMB_FRAME_TYPE_YUV400P: 8-bit greyscale
> > + * @KMB_FRAME_TYPE_RGBA8888: RGBA interleaved stored in 32 bit word
> > + * @KMB_FRAME_TYPE_RGB888: Planar 8 bit RGB data
> > + * @KMB_FRAME_TYPE_LUT2: 1 bit per pixel, Lookup table(used for graphics layers)
> > + * @KMB_FRAME_TYPE_LUT4: 2 bit per pixel, Lookup table(used for graphics layers)
> > + * @KMB_FRAME_TYPE_LUT16: 4 bit per pixel, Lookup table (used for
> > + * graphics layers)
> > + * @KMB_FRAME_TYPE_RAW16: Save any raw type (8, 10, 12bit) on 16 bits
> > + * @KMB_FRAME_TYPE_RAW14: 14-bit value in 16-bit storage
> > + * @KMB_FRAME_TYPE_RAW12: 12-bit value in 16-bit storage
> > + * @KMB_FRAME_TYPE_RAW10: 10-bit value in 16-bit storage
> > + * @KMB_FRAME_TYPE_RAW8: Raw 8 greyscale
> > + * @KMB_FRAME_TYPE_PACK10: SIPP 10-bit packed format
> > + * @KMB_FRAME_TYPE_PACK12: SIPP 12-bit packed format
> > + * @KMB_FRAME_TYPE_YUV444I: Planar 4:4:4 interleaved format
> > + * @KMB_FRAME_TYPE_NV12: Format NV12
> > + * @KMB_FRAME_TYPE_NV21: Format NV21
> > + * @KMB_FRAME_TYPE_BITSTREAM: Used for video encoder bitstream
> > + * @KMB_FRAME_TYPE_HDR: Format HDR
> > + * @KMB_FRAME_TYPE_NV12PACK10: NV12 format with pixels encoded in pack 10
> > + * @KMB_FRAME_TYPE_NONE: Format None
> > + */
> > +enum kmb_frame_types {
> > + KMB_FRAME_TYPE_YUV422I,
> > + KMB_FRAME_TYPE_YUV444P,
> > + KMB_FRAME_TYPE_YUV420P,
> > + KMB_FRAME_TYPE_YUV422P,
> > + KMB_FRAME_TYPE_YUV400P,
> > + KMB_FRAME_TYPE_RGBA8888,
> > + KMB_FRAME_TYPE_RGB888,
> > + KMB_FRAME_TYPE_LUT2,
> > + KMB_FRAME_TYPE_LUT4,
> > + KMB_FRAME_TYPE_LUT16,
> > + KMB_FRAME_TYPE_RAW16,
> > + KMB_FRAME_TYPE_RAW14,
> > + KMB_FRAME_TYPE_RAW12,
> > + KMB_FRAME_TYPE_RAW10,
> > + KMB_FRAME_TYPE_RAW8,
> > + KMB_FRAME_TYPE_PACK10,
> > + KMB_FRAME_TYPE_PACK12,
> > + KMB_FRAME_TYPE_YUV444I,
> > + KMB_FRAME_TYPE_NV12,
> > + KMB_FRAME_TYPE_NV21,
> > + KMB_FRAME_TYPE_BITSTREAM,
> > + KMB_FRAME_TYPE_HDR,
> > + KMB_FRAME_TYPE_NV12PACK10,
> > + KMB_FRAME_TYPE_NONE,
> > +};
> > +
> > +/**
> > + * struct kmb_frame_spec - KMB frame specifications
> > + *
> > + * @type: Values from the enum kmb_frame_type
> > + * @height: Height in pixels
> > + * @width: Width in pixels
> > + * @stride: Defines as distance in bytes from pix(y, x) to pix(y+1, x)
> > + * @bpp: Bits per pixel (for unpacked types set to 8 or 16, for NV12 set only
> > + * luma pixel size)
> > + */
> > +struct kmb_frame_spec {
> > + u16 type;
> > + u16 height;
> > + u16 width;
> > + u16 stride;
> > + u16 bpp;
> > +};
> > +
> > +/**
> > + * struct kmb_vpu_frame_buffer - KMB frame buffer elements
> > + *
> > + * @spec: Frame specifications parameters
> > + * @p1: Address to first image plane
> > + * @p2: Address to second image plane (if used)
> > + * @p3: Address to third image plane (if used)
> > + * @ts: Timestamp in NS
>
> "ns"
>
Will be fixed
> > + */
> > +struct kmb_vpu_frame_buffer {
> > + struct kmb_frame_spec spec;
> > + u64 p1;
> > + u64 p2;
> > + u64 p3;
> > + s64 ts;
> > +};
>
> Messages in form of these structs are also sent over xlink. Shouldn't they
> be __packed as well??
>
> Also the size of kmb_frame_spec is not a multiple of 8 so it would seem to
> require padding to be a member of struct kmb_vpu_frame_buffer.
>
Hm, I think you are right but I need to double check that
> > +
> > +#endif /* KEEMBAY_VPU_FRAME_H_ */
> > diff --git a/drivers/media/platform/keembay-camera/keembay-vpu-isp.h b/drivers/media/platform/keembay-camera/keembay-vpu-isp.h
> > new file mode 100644
> > index 000000000000..c8b35c8ffbb0
> > --- /dev/null
> > +++ b/drivers/media/platform/keembay-camera/keembay-vpu-isp.h
> > @@ -0,0 +1,724 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * Intel Keem Bay VPU ISP params
> > + *
> > + * Copyright (C) 2021 Intel Corporation
> > + */
> > +#ifndef KEEMBAY_VPU_ISP_H
> > +#define KEEMBAY_VPU_ISP_H
> > +
> > +/* Keembay VPU ISP Tables sizes and limits */
> > +#define KMB_VPU_MAX_EXPOSURES 3
> > +
> > +/**
> > + * struct kmb_vpu_raw_stats - KMB Raw statisticsKMB
> > + *
> > + * @ae_awb_stats_addr: AE/AWB statistics addr
> > + * @af_stats_addr: Base start offset for AF statistics addr
> > + * @hist_luma_addr: Luma histogram addr
> > + * @hist_rgb_addr: RGB histogram addr
> > + * @flicker_rows_addr: Flicker detection raw addr
> > + */
> > +struct kmb_vpu_raw_stats {
> > + u64 ae_awb_stats_addr;
> > + u64 af_stats_addr;
> > + u64 hist_luma_addr;
> > + u64 hist_rgb_addr;
> > + u64 flicker_rows_addr;
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_blc_params - KMB Black Level Correction parameters
> > + *
> > + * @coeff1: Black level correction coefficient 1 parameter
> > + * @coeff2: Black level correction coefficient 2 parameter
> > + * @coeff3: Black level correction coefficient 3 parameter
> > + * @coeff4: Black level correction coefficient 4 parameter
> > + */
> > +struct kmb_vpu_blc_params {
> > + u32 coeff1;
> > + u32 coeff2;
> > + u32 coeff3;
> > + u32 coeff4;
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_sigma_dns_params - KMB Sigma Denoise parameters
> > + *
> > + * @noise: Sigma denoise noise parameter
> > + * @threshold1: Sigma denoise min threshold1 parameter
> > + * @threshold2: Sigma denoise max threshold2 parameter
> > + * @threshold3: Sigma denoise min threshold3 parameter
> > + * @threshold4: Sigma denoise max threshold4 parameter
> > + * @threshold5: Sigma denoise min threshold5 parameter
> > + * @threshold6: Sigma denoise max threshold6 parameter
> > + * @threshold7: Sigma denoise min threshold7 parameter
> > + * @threshold8: Sigma denoise max threshold8 parameter
> > + */
> > +struct kmb_vpu_sigma_dns_params {
> > + u32 noise;
> > + u32 threshold1;
> > + u32 threshold2;
> > + u32 threshold3;
> > + u32 threshold4;
> > + u32 threshold5;
> > + u32 threshold6;
> > + u32 threshold7;
> > + u32 threshold8;
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_lsc_params - KMB Lens Shading Correction parameters
> > + *
> > + * @threshold: Lens shading correction threshold parameter
> > + * @width: Lens shading correction width parameter
> > + * @height: Lens shading correction height parameter
> > + * @reserved: Reserved for alignment purpose
> > + * @addr: Lens shading correction table address
> > + */
> > +struct kmb_vpu_lsc_params {
> > + u32 threshold;
> > + u32 width;
> > + u32 height;
> > + u8 reserved[4];
> > + u64 addr;
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_raw_params - KMB Raw parameters
> > + *
> > + * @awb_stats_en: Enable AE/AWB stats output
> > + * @awb_rgb_hist_en: Enable RGB histogram output
> > + * @af_stats_en: Enable AF stats output
> > + * @luma_hist_en: Enable Luma histogram output
> > + * @flicker_accum_en: Enable flicker detection row accumulation output
> > + * @bad_pixel_fix_en: Enable Hot/Cold pixel suppression
> > + * @grgb_imb_en: Enable Gr/Gb imbalance correction
> > + * @mono_imbalance_en: Enable mono imbalance correction
> > + * @gain1: Raw gain1 parameter
> > + * @gain2: Raw gain2 parameter
> > + * @gain3: Raw gain3 parameter
> > + * @gain4: Raw gain4 parameter
> > + * @stop1: Raw stop1 parameter
> > + * @stop2: Raw stop2 parameter
> > + * @stop3: Raw stop3 parameter
> > + * @stop4: Raw stop4 parameter
> > + * @threshold1: Raw threshold1 parameter
> > + * @alpha1: Raw alpha1 parameter
> > + * @alpha2: Raw alpha2 parameter
> > + * @alpha3: Raw alpha3 parameter
> > + * @alpha4: Raw alpha4 parameter
> > + * @threshold2: Raw threshold2 parameter
> > + * @static_defect_size: Static defect data size
> > + * @reserved: Reserved for alignment purpose
> > + * @static_defect_addr: Static defect data address
> > + * @flicker_first_row_acc: First row of flicker detection row accumulation
> > + * @flicker_last_row_acc: First row of flicker detection row accumulation
> > + * @stats: raw statistics buffers
> > + */
> > +struct kmb_vpu_raw_params {
> > + u32 awb_stats_en;
> > + u32 awb_rgb_hist_en;
> > + u32 af_stats_en;
> > + u32 luma_hist_en;
> > + u32 flicker_accum_en;
> > + u32 bad_pixel_fix_en;
> > + u32 grgb_imb_en;
> > + u32 mono_imbalance_en;
> > + u32 gain1;
> > + u32 gain2;
> > + u32 gain3;
> > + u32 gain4;
> > + u32 stop1;
> > + u32 stop2;
> > + u32 stop3;
> > + u32 stop4;
> > + u32 threshold1;
> > + u32 alpha1;
> > + u32 alpha2;
> > + u32 alpha3;
> > + u32 alpha4;
> > + u32 threshold2;
> > + u32 static_defect_size;
> > + u8 reserved[4];
> > + u64 static_defect_addr;
> > + u32 flicker_first_row_acc;
> > + u32 flicker_last_row_acc;
> > + struct kmb_vpu_raw_stats stats[KMB_VPU_MAX_EXPOSURES];
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_ae_awb_params - KMB AE/AWB statistics parameters
> > + *
> > + * @start_x: AE/AWB start_x parameter
> > + * @start_y: AE/AWB start_y parameter
> > + * @width: AE/AWB width parameter
> > + * @height: AE/AWB height parameter
> > + * @skip_x: AE/AWB skip_x parameter
> > + * @skip_y: AE/AWB skip_y parameter
> > + * @patches_x: AE/AWB patches_x parameter
> > + * @patches_y: AE/AWB patches_y parameter
> > + * @threshold1: AE/AWB threshold1 parameter
> > + * @threshold2: AE/AWB threshold2 parameter
> > + */
> > +struct kmb_vpu_ae_awb_params {
> > + u32 start_x;
> > + u32 start_y;
> > + u32 width;
> > + u32 height;
> > + u32 skip_x;
> > + u32 skip_y;
> > + u32 patches_x;
> > + u32 patches_y;
> > + u16 threshold1;
> > + u16 threshold2;
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_af_params - KMB Auto Focus parameters
> > + *
> > + * @start_x: AF start_x parameter
> > + * @start_y: AF start_y parameter
> > + * @width: AF width parameter
> > + * @height: AF height parameter
> > + * @patches_x: AF patches_x parameter
> > + * @patches_y: AF patches_y parameter
> > + * @coeff: AF filter coeff parameter
> > + * @threshold1: AF filer threshold1 parameter
> > + * @threshold2: AF filer threshold2 parameter
> > + * @coeffs1: AF filter coeffs1 parameter
> > + * @coeffs2: AF filter coeffs2 parameter
> > + */
> > +struct kmb_vpu_af_params {
> > + u32 start_x;
> > + u32 start_y;
> > + u32 width;
> > + u32 height;
> > + u32 patches_x;
> > + u32 patches_y;
> > + s32 coeff;
> > + s32 threshold1;
> > + s32 threshold2;
> > + s32 coeffs1[11];
> > + s32 coeffs2[11];
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_hist_params - KMB Hist parameters
> > + *
> > + * @start_x: Hist start_x parameter
> > + * @start_y: Hist start_y parameter
> > + * @end_x: Hist end_x parameter
> > + * @end_y: Hist end_y parameter
> > + * @matrix: Hist matrix parameter
> > + * @weight: Hist weight parameter
> > + */
> > +struct kmb_vpu_hist_params {
> > + u32 start_x;
> > + u32 start_y;
> > + u32 end_x;
> > + u32 end_y;
> > + u16 matrix[9];
> > + u16 weight[3];
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_lca_params - KMB Lateral Chromatic Aberration parameters
> > + *
> > + * @addr: LCA table address
> > + */
> > +struct kmb_vpu_lca_params {
> > + u64 addr;
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_debayer_params - KMB Debayer parameters
> > + *
> > + * @coeff1: Filter coeff1 parameter
> > + * @multiplier1: Filter multiplier1 parameter
> > + * @multiplier2: Filter multiplier2 parameter
> > + * @coeff2: Filter coeff2 parameter
> > + * @coeff3: Filter coeff3 parameter
> > + * @coeff4: Filter coeff4 parameter
> > + */
> > +struct kmb_vpu_debayer_params {
> > + s32 coeff1;
> > + u32 multiplier1;
> > + u32 multiplier2;
> > + s32 coeff2;
> > + s32 coeff3;
> > + s32 coeff4;
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_hdr_params - KMB HDR parameters
> > + *
> > + * @ratio: HDR ratio parameter
> > + * @scale: HDR scale parameter
> > + * @offset1: HDR offset1 parameter
> > + * @slope1: HDR slope1 parameter
> > + * @offset2: HDR offset2 parameter
> > + * @slope2: HDR slope2 parameter
> > + * @offset3: HDR offset3 parameter
> > + * @slope3: HDR slope3 parameter
> > + * @offset4: HDR offset4 parameter
> > + * @gain1: HDR gain1 parameter
> > + * @blur1: HDR blur1 parameter
> > + * @blur2: HDR blur2 parameter
> > + * @contrast1: HDR contrast1 parameter
> > + * @contrast2: HDR contrast2 parameter
> > + * @enable1: HDR enable1 parameter
> > + * @enable2: HDR enable2 parameter
> > + * @offset5: HDR offset5 parameter
> > + * @gain2: HDR gain2 parameter
> > + * @offset6: HDR offset6 parameter
> > + * @strength: HDR strength parameter
> > + * @reserved1: Reserved for alignment purpose
> > + * @luts_addr: HDR LUT address
> > + * @offset7: HDR offset7 parameter
> > + * @shift: HDR shift parameter
> > + * @field1: HDR filed1 parameter
> > + * @field2: HDR field2 parameter
> > + * @gain3: HDR gain3 parameter
> > + * @min: HDR min parameter
> > + * @reserved2: Reserved for alignment purpose
> > + */
> > +struct kmb_vpu_hdr_params {
> > + u32 ratio[2];
> > + u32 scale[3];
> > + s32 offset1;
> > + u32 slope1;
> > + s32 offset2;
> > + u32 slope2;
> > + s32 offset3;
> > + u32 slope3;
> > + s32 offset4;
> > + u32 gain1;
> > + u32 blur1[3];
> > + u32 blur2[5];
> > + u32 contrast1;
> > + u32 contrast2;
> > + u32 enable1;
> > + u32 enable2;
> > + s32 offset5;
> > + u32 gain2;
> > + s32 offset6;
> > + u32 strength;
> > + u8 reserved1[4];
> > + u64 luts_addr;
> > + u16 offset7;
>
> Reserved field missing here?
>
I will double-check
> > + u32 shift;
> > + u16 field1;
> > + u16 field2;
> > + u8 gain3;
> > + u16 min;
> > + u8 reserved2[3];
>
> Why three bytes? This would mean non-byte fields in kmb_vpu_isp_params will
> be unaligned after the hdr field. I'd add reserved fields so that there
> would be no need for unaligned accesses.
>
> I do understand this may be part of a firmware interface so making such
> changes likely would need to be propagated there, so just FYI.
>
> There appear to be similar issues elsewhere in these structs.
>
Hm, I will check if this is due to the fw interface or a typo
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_dog_dns_params - KMB Difference-of-Gaussians DNS parameters
> > + *
> > + * @threshold: Filter threshold parameter
> > + * @strength: Filter strength parameter
> > + * @coeffs11: Filter coeffs11 parameter
> > + * @coeffs15: Filter coeffs15 parameter
> > + * @reserved: Reserved for alignment purpose
> > + */
> > +struct kmb_vpu_dog_dns_params {
> > + u32 threshold;
> > + u32 strength;
> > + u8 coeffs11[6];
> > + u8 coeffs15[8];
> > + u8 reserved[2];
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_luma_dns_params - KMB Luma DNS parameters
> > + *
> > + * @threshold: Luma DNS threshold parameter
> > + * @slope: Luma DNS slope parameter
> > + * @shift: Luma DNS shift parameter
> > + * @alpha: Luma DNS alpha parameter
> > + * @weight: Luma DNS weight parameter
> > + * @per_pixel_alpha_en: Enable adapt alpha
> > + * @gain_bypass_en: Enable gain bypass
> > + * @reserved: for alignment purpose
> > + */
> > +struct kmb_vpu_luma_dns_params {
> > + u32 threshold;
> > + u32 slope;
> > + u32 shift;
> > + u32 alpha;
> > + u32 weight;
> > + u32 per_pixel_alpha_en;
> > + u32 gain_bypass_en;
> > + u8 reserved[4];
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_sharpen_params - KMB Sharpen parameters
> > + *
> > + * @coeffs1: Filter coeffs1 parameter
> > + * @coeffs2: Filter coeffs2 parameter
> > + * @coeffs3: Filter coeffs3 parameter
> > + * @shift: Filter shift parameter
> > + * @gain1: Filter gain1 parameter
> > + * @gain2: Filter gain2 parameter
> > + * @gain3: Filter gain3 parameter
> > + * @gain4: Filter gain4 parameter
> > + * @gain5: Filter gain5 parameter
> > + * @stops1: Filter stops1 parameter
> > + * @gains: Filter gains parameter
> > + * @stops2: Filter stops2 parameter
> > + * @overshoot: Filter overshoot parameter
> > + * @undershoot: Filter undershoot parameter
> > + * @alpha: Filter alpha parameter
> > + * @gain6: Filter gain6 parameter
> > + * @offset: Filter offset parameter
> > + * @addr: Filter data address
> > + */
> > +struct kmb_vpu_sharpen_params {
> > + u16 coeffs1[6];
> > + u16 coeffs2[6];
> > + u16 coeffs3[6];
> > + u32 shift;
> > + u32 gain1;
> > + u32 gain2;
> > + u32 gain3;
> > + u32 gain4;
> > + u32 gain5;
> > + u32 stops1[3];
> > + u32 gains[3];
> > + u32 stops2[4];
> > + u32 overshoot;
> > + u32 undershoot;
> > + u32 alpha;
> > + u32 gain6;
> > + u32 offset;
> > + u64 addr;
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_chroma_gen_params - KMB Chroma GEN parameters
> > + *
> > + * @epsilon: Chroma GEN epsilon parameter
> > + * @coeff1: Chroma GEN coeff1 parameter
> > + * @coeff2: Chroma GEN coeff2 parameter
> > + * @coeff3: Chroma GEN coeff3 parameter
> > + * @coeff4: Chroma GEN coeff4 parameter
> > + * @coeff5: Chroma GEN coeff5 parameter
> > + * @coeff6: Chroma GEN coeff6 parameter
> > + * @strength1: Chroma GEN strength1 parameter
> > + * @strength2: Chroma GEN strength2 parameter
> > + * @coeffs: Chroma GEN coeffs parameter
> > + * @offset1: Chroma GEN offset1 parameter
> > + * @slope1: Chroma GEN slope1 parameter
> > + * @slope2: Chroma GEN slope2 parameter
> > + * @offset2: Chroma GEN offset2 parameter
> > + * @limit: Chroma GEN limit parameter
> > + */
> > +struct kmb_vpu_chroma_gen_params {
> > + u32 epsilon;
> > + u32 coeff1;
> > + u32 coeff2;
> > + u32 coeff3;
> > + u32 coeff4;
> > + u32 coeff5;
> > + u32 coeff6;
> > + u32 strength1;
> > + u32 strength2;
> > + u32 coeffs[3];
> > + s32 offset1;
> > + u32 slope1;
> > + u32 slope2;
> > + s32 offset2;
> > + u32 limit;
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_median_params - KMB Median parameters
> > + *
> > + * @size: Filter size parameter
> > + * @slope: Filter slope parameter
> > + * @offset: Filter offset parameter
> > + */
> > +struct kmb_vpu_median_params {
> > + u32 size;
> > + u32 slope;
> > + s32 offset;
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_chroma_dns_params - KMB Chroma Denoise parameters
> > + *
> > + * @limit: Filter limit parameter
> > + * @enable: Filter enable parameter
> > + * @threshold1: Filter threshold1 parameter
> > + * @threshold2: Filter threshold2 parameter
> > + * @threshold3: Filter threshold3 parameter
> > + * @threshold4: Filter threshold4 parameter
> > + * @threshold5: Filter threshold5 parameter
> > + * @threshold6: Filter threshold6 parameter
> > + * @threshold7: Filter threshold7 parameter
> > + * @threshold8: Filter threshold8 parameter
> > + * @slope1: Filter slope1 parameter
> > + * @offset1: Filter offset1 parameter
> > + * @slope2: Filter slope2 parameter
> > + * @offset2: Filter offset2 parameter
> > + * @grey1: Filter grey1 parameter
> > + * @grey2: Filter grey2 parameter
> > + * @grey3: Filter grey3 parameter
> > + * @coeff1: Filter coeff1 parameter
> > + * @coeff2: Filter coeff2 parameter
> > + * @coeff3: Filter coeff3 parameter
> > + */
> > +struct kmb_vpu_chroma_dns_params {
> > + u32 limit;
> > + u32 enable;
> > + u32 threshold1;
> > + u32 threshold2;
> > + u32 threshold3;
> > + u32 threshold4;
> > + u32 threshold5;
> > + u32 threshold6;
> > + u32 threshold7;
> > + u32 threshold8;
> > + u32 slope1;
> > + s32 offset1;
> > + u32 slope2;
> > + s32 offset2;
> > + u32 grey1;
> > + u32 grey2;
> > + u32 grey3;
> > + u32 coeff1;
> > + u32 coeff2;
> > + u32 coeff3;
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_color_comb_params - KMB Color Combine parameters
> > + *
> > + * @matrix: Color combine matrix parameter
> > + * @offsets:Color combine offsets parameter
> > + * @coeff1: Color combine coeff1 parameter
> > + * @coeff2: Color combine coeff2 parameter
> > + * @coeff3: Color combine coeff3 parameter
> > + * @reserved: Reserved for alignment purpose
> > + * @addr: Color combine table address
> > + * @enable: Color combine enable parameter
> > + * @weight1: Color combine weight1 parameter
> > + * @weight2: Color combine weight2 parameter
> > + * @weight3: Color combine weight3 parameter
> > + * @limit1: Color combine limit1 parameter
> > + * @limit2: Color combine limit2 parameter
> > + * @offset1: Color combine offset1 parameter
> > + * @offset2: Color combine offset2 parameter
> > + */
> > +struct kmb_vpu_color_comb_params {
> > + u16 matrix[9];
> > + u16 offsets[3];
> > + u32 coeff1;
> > + u32 coeff2;
> > + u32 coeff3;
> > + u8 reserved[4];
> > + u64 addr;
> > + u32 enable;
> > + u32 weight1;
> > + u32 weight2;
> > + u32 weight3;
> > + u32 limit1;
> > + s32 limit2;
> > + s32 offset1;
> > + s32 offset2;
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_lut_params - KMB lut parameters
> > + *
> > + * @size: Lut size parameter
> > + * @reserved: Reserved for alignment purpose
> > + * @addr: Lut table address
> > + * @matrix: Lut matrix parameter
> > + * @offsets: Lut offsets parameter
> > + */
> > +struct kmb_vpu_lut_params {
> > + u32 size;
> > + u8 reserved[4];
> > + u64 addr;
> > + u16 matrix[3 * 3];
> > + u16 offsets[3];
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_tnf_params - KMB Temporal Noise Filter parameters
> > + *
> > + * @factor: Filter factor parameter
> > + * @gain: Filter gain parameter
> > + * @offset1: Filter offset1 parameter
> > + * @slope1: Filter slope1 parameter
> > + * @offset2: Filter offset2 parameter
> > + * @slope2: Filter slope2 parameter
> > + * @min1: Filter min1 parameter
> > + * @min2: Filter min2 parameter
> > + * @value: Filter value parameter
> > + * @enable: Filter enable parameter
> > + * @lut0_addr: Filter lut0 address
> > + * @lut1_addr: Filter lut1 address
> > + */
> > +struct kmb_vpu_tnf_params {
> > + u32 factor;
> > + u32 gain;
> > + u32 offset1;
> > + u32 slope1;
> > + u32 offset2;
> > + u32 slope2;
> > + u32 min1;
> > + u32 min2;
> > + u32 value;
> > + u32 enable;
> > + u64 lut0_addr;
> > + u64 lut1_addr;
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_dehaze_params - KMB dehaze parameters
> > + *
> > + * @gain1: Dehaze gain1 parameter
> > + * @min: Dehaze min parameter
> > + * @strength1: Dehaze strength1 parameter
> > + * @strength2: Dehaze strength2 parameter
> > + * @gain2: Dehaze gain2 parameter
> > + * @saturation: Dehaze saturation parameter
> > + * @value1: Dehaze value1 parameter
> > + * @value2: Dehaze value2 parameter
> > + * @value3: Dehaze value3 parameter
> > + * @filter: Dehaze filter parameter
> > + * @stats_addr: Dehaze statistics address
> > + */
> > +struct kmb_vpu_dehaze_params {
> > + u32 gain1;
> > + u32 min;
> > + u32 strength1;
> > + u32 strength2;
> > + u32 gain2;
> > + u32 saturation;
> > + u32 value1;
> > + u32 value2;
> > + u32 value3;
> > + u32 filter[3];
> > + u64 stats_addr;
> > +} __packed;
> > +
> > +/**
> > + * struct kmb_vpu_warp_params - KMB Warp filter parameters
> > + *
> > + * @type: Warp filter type parameter
> > + * @relative: Warp filter relative parameter
> > + * @format: Warp filter format parameter
> > + * @position: Warp filter position parameter
> > + * @reserved: Reserved for alignment purposes
> > + * @addr: Warp filter addr parameter
> > + * @width: Warp filter width parameter
> > + * @height: Warp filter height parameter
> > + * @stride: Warp filter stride parameter
> > + * @enable: Warp filter enable parameter
> > + * @matrix: Warp matrix parameter
> > + * @mode: Warp filter mode parameter
> > + * @values: Warp filter values parameter
> > + */
> > +struct kmb_vpu_warp_params {
> > + u8 type;
> > + u8 relative;
> > + u8 format;
> > + u8 position;
> > + u8 reserved[4];
> > + u64 addr;
> > + u16 width;
> > + u16 height;
> > + u32 stride;
> > + u8 enable;
> > + u32 matrix[9];
> > + u8 mode;
> > + u16 values[3];
> > +} __packed;
> > +
> > +/**
> > + * enum kmb_vpu_bayer_order - KMB sensor Bayer arrangement format types
> > + *
> > + * @KMB_ISP_BAYER_ORDER_GRBG: Gr R B Gr
> > + * @KMB_ISP_BAYER_ORDER_RGGB: R Gr Gr B
> > + * @KMB_ISP_BAYER_ORDER_GBRG: Gr B R Gr
> > + * @KMB_ISP_BAYER_ORDER_BGGR: B Gr Gr R
> > + */
> > +enum kmb_vpu_bayer_order {
> > + KMB_VPU_ISP_BAYER_ORDER_GRBG = 0,
> > + KMB_VPU_ISP_BAYER_ORDER_RGGB = 1,
> > + KMB_VPU_ISP_BAYER_ORDER_GBRG = 2,
> > + KMB_VPU_ISP_BAYER_ORDER_BGGR = 3,
> > +} __packed;
>
> A packed enum? :-)
>
Oops 😃 will be fixed
> > +
> > +/* Version of the VPU ISP ABI. It should be passed as
> > + * first argument in the isp params struct
> > + */
> > +#define KMB_VPU_ISP_ABI_VERSION 104
> > +
> > +/**
> > + * struct kmb_vpu_isp_params - KMB VPU ISP parameters structure
> > + *
> > + * @header_version: Header Version
> > + * @image_data_width: Image data width
> > + * @num_exposures: Number of exposures
> > + * @bayer_order: enum kmb_isp_bayer_order
> > + * @user_data_key: Private key used for the client
> > + * @blc: Black Level correction parameters
> > + * @sigma_dns: Sigma denoise parameters
> > + * @lsc: Lens Shading Correction parameters
> > + * @raw: Raw parameters
> > + * @ae_awb: Auto exposure/Auto white balance parameters
> > + * @af: Auto focus parameters
> > + * @histogram: Histogram parameters
> > + * @lca: Lateral Chromatic Aberration filter parameters
> > + * @debayer: SIPP Bayer demosaicing filter parameters
> > + * @dog_dns: Difference-of-Gaussians filter parameters
> > + * @luma_dns: Luma denoise parameters
> > + * @sharpen: Sharpen filter parameters
> > + * @chroma_gen: Chroma GEN parameters
> > + * @median: Median hardware filter parameters
> > + * @chroma_dns: Chroma Denoise hardware filter parameters
> > + * @color_comb: Color Combine parameters
> > + * @hdr: HDR parameters applied only in HDR mode
> > + * @lut: lut parameters
> > + * @tnf: Temporal Noise Filter parameters
> > + * @dehaze: Dehaze parameters
> > + * @warp: Warp filter parameters
> > + */
> > +struct kmb_vpu_isp_params {
> > + u32 header_version;
> > + u32 image_data_width;
> > + u32 num_exposures;
> > + u32 bayer_order;
> > + u32 user_data_key;
> > + struct kmb_vpu_blc_params blc[KMB_VPU_MAX_EXPOSURES];
> > + struct kmb_vpu_sigma_dns_params sigma_dns[KMB_VPU_MAX_EXPOSURES];
> > + struct kmb_vpu_lsc_params lsc;
> > + struct kmb_vpu_raw_params raw;
> > + struct kmb_vpu_ae_awb_params ae_awb;
> > + struct kmb_vpu_af_params af;
> > + struct kmb_vpu_hist_params histogram;
> > + struct kmb_vpu_lca_params lca;
> > + struct kmb_vpu_debayer_params debayer;
> > + struct kmb_vpu_dog_dns_params dog_dns;
> > + struct kmb_vpu_luma_dns_params luma_dns;
> > + struct kmb_vpu_sharpen_params sharpen;
> > + struct kmb_vpu_chroma_gen_params chroma_gen;
> > + struct kmb_vpu_median_params median;
> > + struct kmb_vpu_chroma_dns_params chroma_dns;
> > + struct kmb_vpu_color_comb_params color_comb;
> > + struct kmb_vpu_hdr_params hdr;
> > + struct kmb_vpu_lut_params lut;
> > + struct kmb_vpu_tnf_params tnf;
> > + struct kmb_vpu_dehaze_params dehaze;
> > + struct kmb_vpu_warp_params warp;
> > +} __packed;
> > +
> > +#endif /* KEEMBAY_VPU_ISP */
> > diff --git a/drivers/media/platform/keembay-camera/keembay-vpu-pipe.h b/drivers/media/platform/keembay-camera/keembay-vpu-
> pipe.h
> > new file mode 100644
> > index 000000000000..d400b59938b2
> > --- /dev/null
> > +++ b/drivers/media/platform/keembay-camera/keembay-vpu-pipe.h
> > @@ -0,0 +1,110 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * Intel Keem Bay camera VPU pipe definitions
> > + *
> > + * Copyright (C) 2021 Intel Corporation
> > + */
> > +#ifndef KEEMBAY_VPU_PIPE_H
> > +#define KEEMBAY_VPU_PIPE_H
> > +
> > +#include "keembay-vpu-src.h"
> > +
> > +#define PIPE_TYPE_ISP_MAX_EXP 3
> > +
> > +enum {
> > + PIPE_TYPE_ISP_ISP_ULL = 1,
> > + PIPE_TYPE_ISP_ISP_2DOL,
> > + PIPE_TYPE_ISP_ISP_3DOL,
> > + PIPE_TYPE_ISP_ISP_MONO,
> > +
> > + PIPE_TYPE_MAX,
> > +};
> > +
> > +enum {
> > + SRC_TYPE_ALLOC_VPU_DATA_MIPI = 0,
> > + SRC_TYPE_ALLOC_VPU_DATA_DBG,
> > + SRC_TYPE_ALLOC_ARM_DATA_ARM,
> > + SRC_TYPE_ALLOC_ARM_DATA_MIPI,
> > + SRC_TYPE_ALLOC_ARM_DATA_DBG,
> > +
> > + SRC_TYPE_ALLOC_DATA_MAX,
> > +};
> > +
> > +enum {
> > + PIPE_TRANSFORM_HUB_NONE = 0,
> > + PIPE_TRANSFORM_HUB_BASIC,
> > + PIPE_TRANSFORM_HUB_FULL,
> > + PIPE_TRANSFORM_HUB_STITCH,
> > + PIPE_TRANSFORM_HUB_EPTZ,
> > +
> > + PIPE_TRANSFORM_HUB_MAX,
> > +};
> > +
> > +enum {
> > + PIPE_OUTPUT_ID_RAW = 0,
> > + PIPE_OUTPUT_ID_ISP_CTRL,
> > + PIPE_OUTPUT_ID_0,
> > + PIPE_OUTPUT_ID_1,
> > + PIPE_OUTPUT_ID_2,
> > + PIPE_OUTPUT_ID_3,
> > + PIPE_OUTPUT_ID_4,
> > + PIPE_OUTPUT_ID_5,
> > + PIPE_OUTPUT_ID_6,
> > +
> > + PIPE_OUTPUT_ID_MAX,
> > +};
> > +
> > +/*
> > + * struct kmb_channel_cfg - KMB channel configuration
> > + *
> > + * @id: Channel id
> > + * @frm_res: Frame resolution
> > + */
> > +struct kmb_channel_cfg {
> > + u32 id;
> > + struct kmb_ic_img_size frm_res;
> > +};
> > +
> > +/*
> > + * struct kmb_pipe_config_evs - VPU pipeline configuration
> > + *
> > + * @pipe_id: Pipe id
> > + * @pipe_type: Pipe type
> > + * @src_type: Source type
>
> It'd be nice to name the enums and refer to them from here (as in "&enum
> nameofenum").
>
Yes, will be fixed
> > + * @pipe_trans_hub: Transform hub type
> > + * @in_isp_res: Input ISP resolution
> > + * @out_isp_res: Output isp resolution
> > + * @in_isp_stride: ISP input stride used in DOL interleaved mode
> > + * @in_exp_offsets: Long and short exp frames offsets used in interleaved mode
> > + * @out_min_res: Output min resolution
> > + * @out_max_res: Output max resolution
> > + * @pipe_xlink_chann: Output channel id from the enum PIPE_OUTPUT_ID
> > + * @keep_aspect_ratio: If enabled, aspect ratio must be kept when image is
> > + * resized
> > + * @in_data_width: Input bits per pixel
> > + * @in_data_packed: Flag to enable packed mode
> > + * @out_data_width: Output bits per pixel for first plane
> > + * @internal_memory_addr: Internal memory pool address
> > + * @internal_memory_size: Internal memory pool size
> > + */
> > +struct kmb_pipe_config_evs {
> > + u8 pipe_id;
> > + u8 pipe_type;
> > + u8 src_type;
> > + u8 pipe_trans_hub;
> > + struct kmb_ic_img_size in_isp_res;
> > + struct kmb_ic_img_size out_isp_res;
> > + u16 in_isp_stride;
> > + u32 in_exp_offsets[PIPE_TYPE_ISP_MAX_EXP];
> > + struct kmb_ic_img_size out_min_res[PIPE_OUTPUT_ID_MAX];
> > + struct kmb_ic_img_size out_max_res[PIPE_OUTPUT_ID_MAX];
> > + struct kmb_channel_cfg pipe_xlink_chann[PIPE_OUTPUT_ID_MAX];
> > + u8 keep_aspect_ratio;
> > + u8 in_data_width;
> > + u8 in_data_packed;
> > + u8 out_data_width;
> > + u64 internal_memory_addr;
> > + u32 internal_memory_size;
> > +} __aligned(64);
>
> __packed as well?
>
> > +
> > +#endif /* KEEMBAY_VPU_PIPE_H */
> > diff --git a/drivers/media/platform/keembay-camera/keembay-vpu-src.h b/drivers/media/platform/keembay-camera/keembay-vpu-src.h
> > new file mode 100644
> > index 000000000000..97f8febbc7e2
> > --- /dev/null
> > +++ b/drivers/media/platform/keembay-camera/keembay-vpu-src.h
> > @@ -0,0 +1,193 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * Intel Keem Bay camera VPU source configuration
> > + *
> > + * Copyright (C) 2021 Intel Corporation
> > + */
> > +
> > +#ifndef KEEMBAY_VPU_SRC_H
> > +#define KEEMBAY_VPU_SRC_H
> > +
> > +/*
> > + * struct kmb_ic_img_size - The structure contains information about image size
> > + *
> > + * @w: Image width
> > + * @h: Image height
> > + */
> > +struct kmb_ic_img_size {
> > + u32 w;
> > + u32 h;
> > +};
> > +
> > +/*
> > + * struct kmb_ic_img_rect - The struct represents the coordinates of a
> > + * rectangular image
> > + *
> > + * @x1: Position of the bottom left corner
> > + * @y1: Position of the top left corner
> > + * @x2: Position of the bottom right corner
> > + * @y2: Position of the top right corner
> > + */
> > +struct kmb_ic_img_rect {
> > + s32 x1;
> > + s32 y1;
> > + s32 x2;
> > + s32 y2;
> > +};
> > +
> > +/*
> > + * enum kmb_ic_source_instance - HW mipi/cif input devices
> > + *
> > + * @KMB_IC_SOURCE_0:
> > + * @KMB_IC_SOURCE_1:
> > + * @KMB_IC_SOURCE_2:
> > + * @KMB_IC_SOURCE_3:
> > + * @KMB_IC_SOURCE_4:
> > + * @KMB_IC_SOURCE_5:
> > + */
> > +enum kmb_ic_source_instance {
> > + KMB_IC_SOURCE_0 = 0,
> > + KMB_IC_SOURCE_1 = 1,
> > + KMB_IC_SOURCE_2 = 2,
> > + KMB_IC_SOURCE_3 = 3,
> > + KMB_IC_SOURCE_4 = 4,
> > + KMB_IC_SOURCE_5 = 5,
> > +};
> > +
> > +/*
> > + * enum kmb_ic_bayer_format - Bayer pattern order
> > + *
> > + * @KMB_IC_BAYER_FORMAT_GRBG: Gr R B Gr
> > + * @KMB_IC_BAYER_FORMAT_RGGB: R Gr Gr B
> > + * @KMB_IC_BAYER_FORMAT_GBRG: Gr B R Gr
> > + * @KMB_IC_BAYER_FORMAT_BGGR: B Gr Gr R
> > + */
> > +enum kmb_ic_bayer_format {
> > + KMB_IC_BAYER_FORMAT_GRBG = 0,
> > + KMB_IC_BAYER_FORMAT_RGGB = 1,
> > + KMB_IC_BAYER_FORMAT_GBRG = 2,
> > + KMB_IC_BAYER_FORMAT_BGGR = 3,
> > +};
> > +
> > +/*
> > + * enum kmb_ic_mipi_rx_ctrl_rec_not - List of receiver Id's for a specific
> > + * sensor
> > + *
> > + * @KMB_IC_SIPP_DEVICE0:
> > + * @KMB_IC_SIPP_DEVICE1:
> > + * @KMB_IC_SIPP_DEVICE2:
> > + * @KMB_IC_SIPP_DEVICE3:
> > + * @KMB_IC_CIF0_DEVICE4:
> > + * @KMB_IC_CIF1_DEVICE5:
> > + */
> > +enum kmb_ic_mipi_rx_ctrl_rec_not {
> > + KMB_IC_SIPP_DEVICE0 = 0,
> > + KMB_IC_SIPP_DEVICE1 = 1,
> > + KMB_IC_SIPP_DEVICE2 = 2,
> > + KMB_IC_SIPP_DEVICE3 = 3,
> > + KMB_IC_CIF0_DEVICE4 = 4,
> > + KMB_IC_CIF1_DEVICE5 = 5,
> > +};
> > +
> > +/*
> > + * enum kmb_ic_mipi_rx_ctrl_not - MIPI controller from chip
> > + *
> > + * @KMB_IC_MIPI_CTRL_0:
> > + * @KMB_IC_MIPI_CTRL_1:
> > + * @KMB_IC_MIPI_CTRL_2:
> > + * @KMB_IC_MIPI_CTRL_3:
> > + * @KMB_IC_MIPI_CTRL_4:
> > + * @KMB_IC_MIPI_CTRL_5:
> > + */
> > +enum kmb_ic_mipi_rx_ctrl_not {
> > + KMB_IC_MIPI_CTRL_0 = 0,
> > + KMB_IC_MIPI_CTRL_1 = 1,
> > + KMB_IC_MIPI_CTRL_2 = 2,
> > + KMB_IC_MIPI_CTRL_3 = 3,
> > + KMB_IC_MIPI_CTRL_4 = 4,
> > + KMB_IC_MIPI_CTRL_5 = 5,
> > +};
> > +
> > +/*
> > + * enum kmb_ic_mipi_ex_data_type - All supported raw, sensor input formats
> > + *
> > + * @IC_IPIPE_YUV_420_B8:
> > + * @IC_IPIPE_RAW_8:
> > + * @IC_IPIPE_RAW_10:
> > + * @IC_IPIPE_RAW_12:
> > + * @IC_IPIPE_RAW_14:
> > + * @IC_IPIPE_EMBEDDED_8BIT:
> > + */
> > +enum kmb_ic_mipi_rx_data_type {
> > + IC_IPIPE_YUV_420_B8 = 0x18,
> > + IC_IPIPE_RAW_8 = 0x2A,
> > + IC_IPIPE_RAW_10 = 0x2B,
> > + IC_IPIPE_RAW_12 = 0x2C,
> > + IC_IPIPE_RAW_14 = 0x2D,
> > + IC_IPIPE_EMBEDDED_8BIT = 0x12
> > +};
> > +
> > +/*
> > + * struct kmb_ic_source_config_dynamic - Per-source configuration of parameters
> > + * which can be modified dynamically.
> > + * Setting will take effect during the
> > + * next blanking interval
> > + *
> > + * @notification_line: Line number upon which IC_EVENT_TYPE_LINE will be sent
> > + * to the Lean OS. Set to -1 to disable notification
> > + */
> > +struct kmb_ic_source_config_dynamic {
> > + s32 notification_line;
> > +};
> > +
> > +/*
> > + * struct kmb_ic_mipi_config - Mipi RX data configuration
> > + *
> > + * @no_controller: Number of controller
> > + * @no_lanes: Number of lanes
> > + * @lane_rate_mbps: Lane rate
> > + * @data_type: Mipi RX data type
> > + * @data_mode: Data mode
> > + * @rec_nrl:
> > + */
> > +struct kmb_ic_mipi_config {
> > + u32 no_controller;
> > + u32 no_lanes;
> > + u32 lane_rate_mbps;
> > + u32 data_type;
> > + u32 data_mode;
> > + u32 rec_nrl;
> > +};
> > +
> > +/*
> > + * struct kmb_ic_source_config - Per-source configuration parameters - mostly
> > + * information needed to configure the MIPI Rx
> > + * filter
> > + *
> > + * @camera_output_size: Max frame size output by the camera
> > + * @crop_window: Crop window coordinates
> > + * @bayer_format: Bayer Format - Raw, Demosaic and LSC blocks should be
> > + * programmed to match the Bayer order specified here.
> > + * @bpp: Bits per pixel
> > + * @mipi_rx_data: MIPI RX data configuration
> > + * @no_exposure: Number of different exposure frames
> > + * @metadata_width: Metadata width
> > + * @metadata_height: Medata height
> > + * @metadata_data_type: Metadata data type
> > + */
> > +struct kmb_ic_source_config {
> > + struct kmb_ic_img_size camera_output_size;
> > + struct kmb_ic_img_rect crop_window;
> > +
> > + u32 bayer_format;
> > + u32 bpp;
> > +
> > + struct kmb_ic_mipi_config mipi_rx_data;
> > +
> > + u32 no_exposure;
> > + u32 metadata_width;
> > + u32 metadata_height;
> > + u32 metadata_data_type;
> > +} __aligned(64);
> > +
> > +#endif /* KEEMBAY_VPU_SRC_H */
>
> --
> Kind regards,
>
> Sakari Ailus
^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 04/10] uapi: Keem Bay ISP Parameters data types
2021-03-19 18:06 [PATCH 00/10] Keem Bay Camera Subsystem Martina Krasteva
` (2 preceding siblings ...)
2021-03-19 18:06 ` [PATCH 03/10] media: Keem Bay Camera: Add VPU camera interface Martina Krasteva
@ 2021-03-19 18:06 ` Martina Krasteva
2021-03-19 20:58 ` kernel test robot
2021-03-22 13:32 ` Sakari Ailus
2021-03-19 18:06 ` [PATCH 05/10] media: v4l: Add Keem Bay Camera meta buffer formats Martina Krasteva
` (6 subsequent siblings)
10 siblings, 2 replies; 28+ messages in thread
From: Martina Krasteva @ 2021-03-19 18:06 UTC (permalink / raw)
To: linux-media
Cc: mchehab, robh+dt, devicetree, sakari.ailus,
daniele.alessandrelli, paul.j.murphy, gjorgjix.rosikopulos,
martinax.krasteva
From: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
ISP parameters passed to the “keembay-metadata-params”
metadata output video node
Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
Co-developed-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
Signed-off-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
Co-developed-by: Martina Krasteva <martinax.krasteva@intel.com>
Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
---
MAINTAINERS | 1 +
include/uapi/linux/keembay-isp-ctl.h | 796 +++++++++++++++++++++++++++++++++++
2 files changed, 797 insertions(+)
create mode 100644 include/uapi/linux/keembay-isp-ctl.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 76082714a76f..955f9f6a195d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1973,6 +1973,7 @@ S: Maintained
T: git git://linuxtv.org/media_tree.git
F: Documentation/devicetree/bindings/media/intel,keembay-camera.yaml
F: drivers/media/platform/keembay-camera/
+F: include/uapi/linux/keembay-isp-ctl.h
ARM/IP FABRICS DOUBLE ESPRESSO MACHINE SUPPORT
M: Lennert Buytenhek <kernel@wantstofly.org>
diff --git a/include/uapi/linux/keembay-isp-ctl.h b/include/uapi/linux/keembay-isp-ctl.h
new file mode 100644
index 000000000000..86e1654067f0
--- /dev/null
+++ b/include/uapi/linux/keembay-isp-ctl.h
@@ -0,0 +1,796 @@
+/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */
+/*
+ * Intel Keem Bay camera control parameters and statistics
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+#ifndef KEEMBAY_ISP_CTL_H
+#define KEEMBAY_ISP_CTL_H
+
+#include <linux/types.h>
+#include <linux/videodev2.h>
+
+#define KMB_CAM_MAX_EXPOSURES 3
+
+/* Table max sizes */
+#define KMB_CAM_LCA_MESH_SIZE (32 * 24)
+#define KMB_CAM_SHARPEN_RADIAL_SIZE 256
+#define KMB_CAM_LUT3D_SIZE (16 * 16 * 16 * 4)
+#define KMB_CAM_LSC_SIZE ((64 + 4) * 64 * 4)
+#define KMB_CAM_GAMMA_SIZE (512 * 4)
+#define KMB_CAM_STATIC_DEFECT_SIZE 1000
+#define KMB_CAM_CHROMA_LUT_SIZE 1024
+#define KMB_CAM_WARP_MESH_SIZE (480 * 270)
+#define KMB_CAM_HDR_TM_LUTS_SIZE (((67 * 16) + 63) & ~63)
+
+/* Statistics max sizes */
+#define KMB_CAM_AE_AWB_STATS_SIZE (64 * 64)
+#define KMB_CAM_AF_STATS_SIZE (64 * 64)
+#define KMB_CAM_HIST_LUMA_SIZE 256
+#define KMB_CAM_HIST_RGB_SIZE (3 * 128)
+#define KMB_CAM_FLICKER_ROWS_SIZE (65535 + 1) /* align to 64 */
+#define MAX_DHZ_AIRLIGHT_STATS_SIZE (4 * 128)
+
+/**
+ * enum kmb_vpu_isp_bayer_order - KMB sensor Bayer arrangement format types
+ *
+ * @KMB_ISP_BAYER_ORDER_GRBG: Gr R B Gr
+ * @KMB_ISP_BAYER_ORDER_RGGB: R Gr Gr B
+ * @KMB_ISP_BAYER_ORDER_GBRG: Gr B R Gr
+ * @KMB_ISP_BAYER_ORDER_BGGR: B Gr Gr R
+ */
+enum kmb_isp_bayer_order {
+ KMB_ISP_BAYER_ORDER_GRBG = 0,
+ KMB_ISP_BAYER_ORDER_RGGB = 1,
+ KMB_ISP_BAYER_ORDER_GBRG = 2,
+ KMB_ISP_BAYER_ORDER_BGGR = 3,
+} __packed;
+
+/**
+ * struct kmb_blc_params - KMB Black Level Correction parameters
+ *
+ * @coeff1: Black level correction coefficient parameter. Range [0 - 4096]
+ * @coeff2: Black level correction coefficient parameter. Range [0 - 4096]
+ * @coeff3: Black level correction coefficient parameter. Range [0 - 4096]
+ * @coeff4: Black level correction coefficient parameter. Range [0 - 4096]
+ */
+struct kmb_blc_params {
+ __u32 coeff1;
+ __u32 coeff2;
+ __u32 coeff3;
+ __u32 coeff4;
+} __packed;
+
+/**
+ * struct kmb_sigma_dns_params - KMB Sigma Denoise parameters
+ *
+ * @noise: Sigma denoise noise parameter. Range [0 - 65535]
+ * @threshold1: Sigma denoise min threshold1 parameter. Range [0 - 255]
+ * @threshold2: Sigma denoise max threshold2 parameter. Range [0 - 255]
+ * @threshold3: Sigma denoise min threshold3 parameter. Range [0 - 255]
+ * @threshold4: Sigma denoise max threshold4 parameter. Range [0 - 255]
+ * @threshold5: Sigma denoise min threshold5 parameter. Range [0 - 255]
+ * @threshold6: Sigma denoise max threshold6 parameter. Range [0 - 255]
+ * @threshold7: Sigma denoise min threshold7 parameter. Range [0 - 255]
+ * @threshold8: Sigma denoise max threshold8 parameter. Range [0 - 255]
+ */
+struct kmb_sigma_dns_params {
+ __u32 noise;
+ __u32 threshold1;
+ __u32 threshold2;
+ __u32 threshold3;
+ __u32 threshold4;
+ __u32 threshold5;
+ __u32 threshold6;
+ __u32 threshold7;
+ __u32 threshold8;
+} __packed;
+
+/**
+ * struct kmb_lsc_params - KMB Lens Shading Correction parameters
+ *
+ * @threshold: Lens shading correction threshold parameter
+ * @width: Lens shading correction gain1 parameter. Range [1 - 64].
+ * Must be a multiple of 4
+ * @height: Lens shading correction gain2 parameter. Range [1 - 64].
+ * Must be an even number
+ * @gain_mesh: Lens shading correction gain mesh table
+ */
+struct kmb_lsc_params {
+ __u32 threshold;
+ __u32 width;
+ __u32 height;
+ __u8 gain_mesh[KMB_CAM_LSC_SIZE];
+} __packed;
+
+/**
+ * struct kmb_raw_params - KMB Raw parameters
+ *
+ * @awb_stats_en: Enable AE/AWB stats output
+ * @awb_rgb_hist_en: Enable RGB histogram output
+ * @af_stats_en: Enable AF stats output
+ * @luma_hist_en: Enable Luma histogram output
+ * @flicker_accum_en: Enable flicker detection row accumulation output
+ * @bad_pixel_fix_en: Enable Hot/Cold pixel suppression
+ * @grgb_imb_en: Enable Gr/Gb imbalance correction
+ * @mono_imbalance_en: Enable mono imbalance correction
+ * @gain1: Raw gain1 parameter
+ * @gain2: Raw gain2 parameter
+ * @gain3: Raw gain3 parameter
+ * @gain4: Raw gain4 parameter
+ * @stop1: Raw stop1 parameter
+ * @stop2: Raw stop2 parameter
+ * @stop3: Raw stop3 parameter
+ * @stop4: Raw stop4 parameter
+ * @threshold1: Raw threshold1 parameter
+ * @alpha1: Raw alpha1 parameter. Range [0 - 15]
+ * @alpha2: Raw alpha2 parameter. Range [0 - 15]
+ * @alpha3: Raw alpha3 parameter. Range [0 - 15]
+ * @alpha4: Raw alpha4 parameter. Range [0 - 15]
+ * @threshold2: Raw threshold2 parameter. Range [0 - 2047]
+ * @static_defect_size: static_defect_map size parameter. Range [0 - 65536]
+ * @static_defect_map: Static defect map
+ * @start_row: Raw start_row parameter
+ * @end_row: Raw end_row parameter
+ */
+struct kmb_raw_params {
+ __u32 awb_stats_en;
+ __u32 awb_rgb_hist_en;
+ __u32 af_stats_en;
+ __u32 luma_hist_en;
+ __u32 flicker_accum_en;
+ __u32 bad_pixel_fix_en;
+ __u32 grgb_imb_en;
+ __u32 mono_imbalance_en;
+ __u32 gain1;
+ __u32 gain2;
+ __u32 gain3;
+ __u32 gain4;
+ __u32 stop1;
+ __u32 stop2;
+ __u32 stop3;
+ __u32 stop4;
+ __u32 threshold1;
+ __u32 alpha1;
+ __u32 alpha2;
+ __u32 alpha3;
+ __u32 alpha4;
+ __u32 threshold2;
+ __u32 static_defect_size;
+ __u8 static_defect_map[KMB_CAM_STATIC_DEFECT_SIZE];
+ __u32 start_row;
+ __u32 end_row;
+} __packed;
+
+/**
+ * struct kmb_ae_awb_params - KMB AE/AWB statistics parameters
+ *
+ * @start_x: AE/AWB start_x parameter
+ * @start_y: AE/AWB start_y parameter
+ * @width: AE/AWB width parameter
+ * @height: AE/AWB height parameter
+ * @skip_x: AE/AWB skip_x parameter
+ * @skip_y: AE/AWB skip_y parameter
+ * @patches_x: AE/AWB patches_x parameter
+ * @patches_y: AE/AWB patches_y parameter
+ * @threshold1: AE/AWB threshold1 parameter
+ * @threshold2: AE/AWB threshold2 parameter
+ */
+struct kmb_ae_awb_params {
+ __u32 start_x;
+ __u32 start_y;
+ __u32 width;
+ __u32 height;
+ __u32 skip_x;
+ __u32 skip_y;
+ __u32 patches_x;
+ __u32 patches_y;
+ __u16 threshold1;
+ __u16 threshold2;
+} __packed;
+
+/**
+ * struct kmb_af_params - KMB Auto Focus parameters
+ *
+ * @start_x: AF start_x parameter
+ * @start_y: AF start_y parameter
+ * @width: AF width parameter
+ * @height: AF height parameter
+ * @patches_x: AF patches_x parameter
+ * @patches_y: AF patches_y parameter
+ * @coeff: AF filter coeff parameter
+ * @threshold1: AF filer threshold1 parameter
+ * @threshold2: AF filer threshold2 parameter
+ * @coeffs1: AF filter coeffs1 parameter
+ * @coeffs2: AF filter coeffs2 parameter
+ */
+struct kmb_af_params {
+ __u32 start_x;
+ __u32 start_y;
+ __u32 width;
+ __u32 height;
+ __u32 patches_x;
+ __u32 patches_y;
+ __s32 coeff;
+ __s32 threshold1;
+ __s32 threshold2;
+ __s32 coeffs1[11];
+ __s32 coeffs2[11];
+} __packed;
+
+/**
+ * struct kmb_hist_params - KMB Hist parameters
+ *
+ * @start_x: Hist start_x parameter. Range [0 - 1]
+ * @start_y: Hist start_y parameter. Range [0 - 1]
+ * @end_x: Hist end_x parameter. Range [0 - 1]
+ * @end_y: Hist end_y parameter. Range [0 - 1]
+ * @matrix: Hist matrix parameter. Range [0.0 - 8.0]
+ * @weight: Hist weight parameter. Range [0.0 - 1.0]
+ */
+struct kmb_hist_params {
+ __u32 start_x;
+ __u32 start_y;
+ __u32 end_x;
+ __u32 end_y;
+ __u16 matrix[9];
+ __u16 weight[3];
+} __packed;
+
+/**
+ * struct kmb_lca_params - KMB Lateral Chromatic Aberration parameters
+ *
+ * @coeff: LCA coeff parameter
+ */
+struct kmb_lca_params {
+ __u8 coeff[KMB_CAM_LCA_MESH_SIZE];
+} __packed;
+
+/**
+ * struct kmb_debayer_params - KMB Debayer parameters
+ *
+ * @coeff1: Filter coeff1 parameter
+ * @multiplier1: Filter multiplier1 parameter
+ * @multiplier2: Filter multiplier2 parameter
+ * @coeff2: Filter coeff2 parameter
+ * @coeff3: Filter coeff3 parameter
+ * @coeff4: Filter coeff4 parameter
+ */
+struct kmb_debayer_params {
+ __s32 coeff1;
+ __u32 multiplier1;
+ __u32 multiplier2;
+ __s32 coeff2;
+ __s32 coeff3;
+ __s32 coeff4;
+} __packed;
+
+/**
+ * struct kmb_hdr_params - KMB HDR parameters
+ *
+ * @ratio: HDR ratio parameter. Range [0 - 65536]
+ * @scale: HDR scale parameter
+ * @offset1: HDR offset1 parameter. Range [-4095 - 0]
+ * @slope1: HDR slope1 parameter. Range [0 - 4095]
+ * @offset2: HDR offset2 parameter. Range [-4095 - 0]
+ * @slope2: HDR slope2 parameter. Range [0 - 4095]
+ * @offset3: HDR offset3 parameter. Range [0 - 4095]
+ * @slope3: HDR slope3 parameter. Range [0 - 4095]
+ * @offset4: HDR offset4 parameter. Range [0 - 4095]
+ * @gain1: HDR gain1 parameter. Range [0 - 4095]
+ * @blur1: HDR blur1 parameter. Range [0.0 - 255.0]
+ * @blur2: HDR blur2 parameter. Range [0 - 255]
+ * @contrast1: HDR contrast1 parameter
+ * @contrast2: HDR contrast2 parameter
+ * @enable1: HDR enable1 parameter
+ * @enable2: HDR enable2 parameter
+ * @offset5: HDR offset5 parameter
+ * @gain2: HDR gain2 parameter
+ * @offset6: HDR offset6 parameter. Range [0 - 1024]
+ * @strength: HDR strength parameter. Range [0 - 65536]
+ * @tm_lut: HDR tm lut parameter
+ * @offset7: HDR offset7 parameter. Range [0 - 65536]
+ * @shift: HDR shift parameter
+ * @field1: HDR filed1 parameter
+ * @field2: HDR field2 parameter
+ * @gain3: HDR gain3 parameter. Range [0 - 255]
+ * @min: HDR min parameter. Range [0 - 4095]
+ */
+struct kmb_hdr_params {
+ __u32 ratio[2];
+ __u32 scale[3];
+ __s32 offset1;
+ __u32 slope1;
+ __s32 offset2;
+ __u32 slope2;
+ __s32 offset3;
+ __u32 slope3;
+ __s32 offset4;
+ __u32 gain1;
+ __u32 blur1[3];
+ __u32 blur2[5];
+ __u32 contrast1;
+ __u32 contrast2;
+ __u32 enable1;
+ __u32 enable2;
+ __s32 offset5;
+ __u32 gain2;
+ __s32 offset6;
+ __u32 strength;
+ __u8 tm_lut[KMB_CAM_HDR_TM_LUTS_SIZE];
+ __u16 offset7;
+ __u32 shift;
+ __u16 field1;
+ __u16 field2;
+ __u8 gain3;
+ __u16 min;
+} __packed;
+
+/**
+ * struct kmb_dog_dns_params - KMB Difference-of-Gaussians DNS parameters
+ *
+ * @threshold: Filter threshold parameter. Range [0 - 255]
+ * @strength: Filter strength parameter. Range [0 - 255]
+ * @coeffs11: Filter coeffs11 parameter. Range [0 - 1023]
+ * @coeffs15: Filter coeffs15 parameter. Range [0 - 1023]
+ */
+struct kmb_dog_dns_params {
+ __u32 threshold;
+ __u32 strength;
+ __u8 coeffs11[6];
+ __u8 coeffs15[8];
+} __packed;
+
+/**
+ * struct kmb_luma_dns_params - KMB Luma DNS parameters
+ *
+ * @threshold: Luma DNS threshold parameter. Range [0 - 32768]
+ * @slope: Luma DNS slope parameter. Range [0 - 2048]
+ * @shift: Luma DNS shift parameter. Range [0 - 255]
+ * @alpha: Luma DNS alpha parameter. Range [0 - 127]
+ * @weight: Luma DNS weight parameter. Range [0 - 4294967295]
+ * @per_pixel_alpha_en: Enable adapt alpha
+ * @gain_bypass_en: Enable gain bypass
+ */
+struct kmb_luma_dns_params {
+ __u32 threshold;
+ __u32 slope;
+ __u32 shift;
+ __u32 alpha;
+ __u32 weight;
+ __u32 per_pixel_alpha_en;
+ __u32 gain_bypass_en;
+} __packed;
+
+/**
+ * struct kmb_sharpen_params - KMB Sharpen parameters
+ *
+ * @coeffs1: Filter coeffs1 parameter. Range [0 - 255]
+ * @coeffs2: Filter coeffs2 parameter. Range [0 - 255]
+ * @coeffs3: Filter coeffs3 parameter. Range [0 - 255]
+ * @shift: Filter shift parameter. Range [0 - 255]
+ * @gain1: Filter gain1 parameter. Range [0 - 2047]
+ * @gain2: Filter gain2 parameter. Range [0 - 2047]
+ * @gain3: Filter gain3 parameter. Range [0 - 2047]
+ * @gain4: Filter gain4 parameter. Range [0 - 2047]
+ * @gain5: Filter gain5 parameter. Range [0 - 255]
+ * @stops1: Filter stops1 parameter. Range [0 - 4095]
+ * @gains: Filter gains parameter. Range [0 - 256]
+ * @stops2: Filter stops2 parameter. Range [0 - 4095]
+ * @overshoot: Filter overshoot parameter. Range [0 - 256]
+ * @undershoot: Filter undershoot parameter. Range [0 - 256]
+ * @alpha: Filter alpha parameter. Range [0 - 256]
+ * @gain6: Filter gain6 parameter. Range [0 - 255]
+ * @offset: Filter offset parameter. Range [0 - 1023]
+ * @radial_lut: Sharpen radial LUT parameter. Range [0 - 255]
+ */
+struct kmb_sharpen_params {
+ __u16 coeffs1[6];
+ __u16 coeffs2[6];
+ __u16 coeffs3[6];
+ __u32 shift;
+ __u32 gain1;
+ __u32 gain2;
+ __u32 gain3;
+ __u32 gain4;
+ __u32 gain5;
+ __u32 stops1[3];
+ __u32 gains[3];
+ __u32 stops2[4];
+ __u32 overshoot;
+ __u32 undershoot;
+ __u32 alpha;
+ __u32 gain6;
+ __u32 offset;
+ __u8 radial_lut[KMB_CAM_SHARPEN_RADIAL_SIZE];
+} __packed;
+
+/**
+ * struct kmb_chroma_gen_params - KMB Chroma GEN parameters
+ *
+ * @epsilon: Chroma GEN epsilon parameter. Range [0 - 255]
+ * @coeff1: Chroma GEN coeff1 parameter. Range [0 - 1024]
+ * @coeff2: Chroma GEN coeff2 parameter. Range [0 - 1024]
+ * @coeff3: Chroma GEN coeff3 parameter. Range [0 - 1024]
+ * @coeff4: Chroma GEN coeff4 parameter. Range [0 - 255]
+ * @coeff5: Chroma GEN coeff5 parameter. Range [0 - 255]
+ * @coeff6: Chroma GEN coeff6 parameter. Range [0 - 255]
+ * @strength1: Chroma GEN strength1 parameter. Range [0 - 255]
+ * @strength2: Chroma GEN strength2 parameter. Range [0 - 255]
+ * @coeffs: Chroma GEN coeffs parameter . Range [0 - 255]
+ * @offset1: Chroma GEN offset1 parameter. Range [0 - 255]
+ * @slope1: Chroma GEN slope1 parameter. Range [0 - 255]
+ * @slope2: Chroma GEN slope2 parameter. Range [0 - 255]
+ * @offset2: Chroma GEN offset2 parameter. Range [0 - 255]
+ * @limit: Chroma GEN limit parameter. Range [0 - 767]
+ */
+struct kmb_chroma_gen_params {
+ __u32 epsilon;
+ __u32 coeff1;
+ __u32 coeff2;
+ __u32 coeff3;
+ __u32 coeff4;
+ __u32 coeff5;
+ __u32 coeff6;
+ __u32 strength1;
+ __u32 strength2;
+ __u32 coeffs[3];
+ __s32 offset1;
+ __u32 slope1;
+ __u32 slope2;
+ __s32 offset2;
+ __u32 limit;
+} __packed;
+
+/**
+ * struct kmb_median_params - KMB Median parameters
+ *
+ * @size: Filter size parameter. Range [1;3;5;7]
+ * @slope: Filter slope parameter. Range [0 - 128]
+ * @offset: Filter offset parameter. Range [-32 - 32]
+ */
+struct kmb_median_params {
+ __u32 size;
+ __u32 slope;
+ __s32 offset;
+} __packed;
+
+/**
+ * struct kmb_chroma_dns_params - KMB Chroma Denoise parameters
+ *
+ * @limit: Filter limit parameter
+ * @enable: Filter enable parameter
+ * @threshold1: Filter threshold1 parameter
+ * @threshold2: Filter threshold2 parameter
+ * @threshold3: Filter threshold3 parameter. Range [0 - 255]
+ * @threshold4: Filter threshold4 parameter. Range [0 - 255]
+ * @threshold5: Filter threshold5 parameter. Range [0 - 255]
+ * @threshold6: Filter threshold6 parameter. Range [0 - 255]
+ * @threshold7: Filter threshold7 parameter. Range [0 - 255]
+ * @threshold8: Filter threshold8 parameter. Range [0 - 255]
+ * @slope1: Filter slope1 parameter. Range [0 - 255]
+ * @offset1: Filter offset1 parameter. Range [0 - 255]
+ * @slope2: Filter slope2 parameter. Range [0 - 255]
+ * @offset2: Filter offset2 parameter. Range [0 - 255]
+ * @grey1: Filter grey1 parameter. Range [0 - 255]
+ * @grey2: Filter grey2 parameter. Range [0 - 255]
+ * @grey3: Filter grey3 parameter. Range [0 - 255]
+ * @coeff1: Filter coeff1 parameter. Range [0 - 255]
+ * @coeff2: Filter coeff2 parameter. Range [0 - 255]
+ * @coeff3: Filter coeff3 parameter. Range [0 - 255]
+ */
+struct kmb_chroma_dns_params {
+ __u32 limit;
+ __u32 enable;
+ __u32 threshold1;
+ __u32 threshold2;
+ __u32 threshold3;
+ __u32 threshold4;
+ __u32 threshold5;
+ __u32 threshold6;
+ __u32 threshold7;
+ __u32 threshold8;
+ __u32 slope1;
+ __s32 offset1;
+ __u32 slope2;
+ __s32 offset2;
+ __u32 grey1;
+ __u32 grey2;
+ __u32 grey3;
+ __u32 coeff1;
+ __u32 coeff2;
+ __u32 coeff3;
+} __packed;
+
+/**
+ * struct kmb_color_comb_params - KMB Color Combine parameters
+ *
+ * @matrix: Color combine matrix parameter. Range [-8.0 - 8.0]
+ * @offsets:Color combine offsets parameter. Range [-1.0 - 1.0]
+ * @coeff1: Color combine coeff1 parameter. Range [0 - 1023]
+ * @coeff2: Color combine coeff2 parameter. Range [0 - 1023]
+ * @coeff3: Color combine coeff3 parameter. Range [0 - 1023]
+ * @lut_3d: Color combine LUT 3D parameter. Range [0 - 4095]
+ * @enable: Color combine enable parameter
+ * @weight1: Color combine weight1 parameter. Range [0 - 255]
+ * @weight2: Color combine weight2 parameter. Range [0 - 255]
+ * @weight3: Color combine weight3 parameter. Range [0 - 255]
+ * @limit1: Color combine limit1 parameter. Range [0 - 32766]
+ * @limit2: Color combine limit2 parameter. Range [-32766 - 0]
+ * @offset1: Color combine offset1 parameter. Range [0 - 8192]
+ * @offset2: Color combine offset2 parameter. Range [0 - 8192]
+ */
+struct kmb_color_comb_params {
+ __u16 matrix[9];
+ __u16 offsets[3];
+ __u32 coeff1;
+ __u32 coeff2;
+ __u32 coeff3;
+ __u8 lut_3d[KMB_CAM_LUT3D_SIZE];
+ __u32 enable;
+ __u32 weight1;
+ __u32 weight2;
+ __u32 weight3;
+ __u32 limit1;
+ __s32 limit2;
+ __s32 offset1;
+ __s32 offset2;
+} __packed;
+
+/**
+ * struct kmb_lut_params - KMB lut parameters
+ *
+ * @size: Lut size parameter. Range [0 - 128]
+ * @table: Lut table parameter
+ * @matriix: Lut matrix parameter
+ * @offsets: Lut offsets pparameter
+ */
+struct kmb_lut_params {
+ __u32 size;
+ __u8 table[KMB_CAM_GAMMA_SIZE];
+ __u16 matrix[3 * 3];
+ __u16 offsets[3];
+} __packed;
+
+/**
+ * struct kmb_tnf_params - KMB Temporal Noise Filter parameters
+ *
+ * @factor: Filter factor parameter. Range [0 - 255]
+ * @gain: Filter gain parameter. Range [0 - 4095]
+ * @offset1: Filter offset1 parameter. Range [0 - 8192]
+ * @slope1: Filter slope1 parameter. Range [0 - 512]
+ * @offset2: Filter offset2 parameter. Range [0 - 8192]
+ * @slope2: Filter slope2 parameter. Range [0 - 512]
+ * @min1: Filter min1 parameter. Range [0 - 65535]
+ * @min2: Filter min2 parameter. Range [0 - 65535]
+ * @value: Filter value parameter. Range [0 - 255]
+ * @enable: Filter enable parameter
+ * @chroma_lut0: First chroma LUT. Range [0 - 4095]
+ * @chroma_lut1: Second chroma LUT. Range [0 - 4095]
+ */
+struct kmb_tnf_params {
+ __u32 factor;
+ __u32 gain;
+ __u32 offset1;
+ __u32 slope1;
+ __u32 offset2;
+ __u32 slope2;
+ __u32 min1;
+ __u32 min2;
+ __u32 value;
+ __u32 enable;
+ __u8 chroma_lut0[KMB_CAM_CHROMA_LUT_SIZE];
+ __u8 chroma_lut1[KMB_CAM_CHROMA_LUT_SIZE];
+} __packed;
+
+/**
+ * struct kmb_dehaze_params - KMB dehaze parameters
+ *
+ * @gain1: Dehaze gain1 parameter. Range [0 - 1023]
+ * @min: Dehaze min parameter. Range [0 - 255]
+ * @strength1: Dehaze strength1 parameter. Range [0 - 255]
+ * @strength2: Dehaze strength2 parameter. Range [0 - 65535]
+ * @gain2: Dehaze gain2 parameter. Range [0 - 255]
+ * @saturation: Dehaze saturation parameter. Range [0 - 127]
+ * @value1: Dehaze value1 parameter. Range [0 - 4095]
+ * @value2: Dehaze value2 parameter. Range [0 - 4095]
+ * @value3: Dehaze value3 parameter. Range [0 - 4095]
+ * @filter: Dehaze filter parameter. Range [0 - 255]
+ */
+struct kmb_dehaze_params {
+ __u32 gain1;
+ __u32 min;
+ __u32 strength1;
+ __u32 strength2;
+ __u32 gain2;
+ __u32 saturation;
+ __u32 value1;
+ __u32 value2;
+ __u32 value3;
+ __u32 filter[3];
+} __packed;
+
+/**
+ * struct kmb_warp_params - KMB Warp filter parameters
+ *
+ * @type: Warp filter type parameter. Range [0 - 1]
+ * @relative: Warp filter relative parameter. Range [0 - 1]
+ * @format: Warp filter format parameter. Range [0 - 1]
+ * @position: Warp filter position parameter
+ * @mesh_grid: Warp mesh grid
+ * @width: Warp filter width parameter
+ * @height: Warp filter height parameter
+ * @stride: Warp filter stride parameter
+ * @enable: Warp filter enable parameter
+ * @matrix: Warp matrix parameter
+ * @mode: Warp filter mode parameter. Range [0 - 1]
+ * @values: Warp filter values parameter. Range [0 - 128]
+ */
+struct kmb_warp_params {
+ __u8 type;
+ __u8 relative;
+ __u8 format;
+ __u8 position;
+ __u8 mesh_grid[KMB_CAM_WARP_MESH_SIZE];
+ __u16 width;
+ __u16 height;
+ __u32 stride;
+ __u8 enable;
+ __u32 matrix[9];
+ __u8 mode;
+ __u16 values[3];
+} __packed;
+
+/**
+ * struct kmb_isp_params_flags - Bits to indicate which params
+ * need to be updated
+ *
+ * @blc: 1 = update, 0 = do not update.
+ * @sigma_dns: 1 = update, 0 = do not update.
+ * @lsc: 1 = update, 0 = do not update.
+ * @raw: 1 = update, 0 = do not update.
+ * @ae_awb: 1 = update, 0 = do not update.
+ * @af: 1 = update, 0 = do not update.
+ * @histogram: 1 = update, 0 = do not update.
+ * @lca: 1 = update, 0 = do not update.
+ * @debayer: 1 = update, 0 = do not update.
+ * @dog_dns: 1 = update, 0 = do not update.
+ * @luma_dns: 1 = update, 0 = do not update.
+ * @sharpen: 1 = update, 0 = do not update.
+ * @chroma_gen: 1 = update, 0 = do not update.
+ * @median: 1 = update, 0 = do not update.
+ * @chroma_dns: 1 = update, 0 = do not update.
+ * @color_comb: 1 = update, 0 = do not update.
+ * @hdr: 1 = update, 0 = do not update.
+ * @lut: 1 = update, 0 = do not update.
+ * @tnf: 1 = update, 0 = do not update.
+ * @dehaze: 1 = update, 0 = do not update.
+ * @warp: 1 = update, 0 = do not update.
+ * @reserved: reserved for future use and for alignment
+ */
+struct kmb_isp_params_flags {
+ __u32 blc:1;
+ __u32 sigma_dns:1;
+ __u32 lsc:1;
+ __u32 raw:1;
+ __u32 ae_awb:1;
+ __u32 af:1;
+ __u32 histogram:1;
+ __u32 lca:1;
+ __u32 debayer:1;
+ __u32 dog_dns:1;
+ __u32 luma_dns:1;
+ __u32 sharpen:1;
+ __u32 chroma_gen:1;
+ __u32 median:1;
+ __u32 chroma_dns:1;
+ __u32 color_comb:1;
+ __u32 hdr:1;
+ __u32 lut:1;
+ __u32 tnf:1;
+ __u32 dehaze:1;
+ __u32 warp:1;
+ __u32 reserved:11;
+} __packed;
+
+/**
+ * struct kmb_isp_params - KMB ISP parameters structure
+ *
+ * @update: Select which parameters to apply, see kmb_vpu_isp_params_flags
+ * @blc: Black Level correction parameters
+ * @sigma_dns: Sigma denoise parameters
+ * @lsc: Lens Shading Correction parameters
+ * @raw: Raw parameters
+ * @ae_awb: Auto exposure/Auto white balance parameters
+ * @af: Auto focus parameters
+ * @histogram: Histogram parameters
+ * @lca: Lateral Chromatic Aberration filter parameters
+ * @debayer: SIPP Bayer demosaicing filter parameters
+ * @dog_dns: Difference-of-Gaussians filter parameters
+ * @luma_dns: Luma denoise parameters
+ * @sharpen: Sharpen filter parameters
+ * @chroma_gen: Chroma GEN parameters
+ * @median: Median hardware filter parameters
+ * @chroma_dns: Chroma Denoise hardware filter parameters
+ * @color_comb: Color Combine parameters
+ * @hdr: HDR parameters applied only in HDR mode
+ * @lut: LUT parameters
+ * @tnf: Temporal Noise Filter parameters
+ * @dehaze: Dehaze parameters
+ * @warp: Warp filter parameters
+ *
+ * Each struct represents a filter and its settings which are applied on the raw
+ * image.
+ */
+struct kmb_isp_params {
+ struct kmb_isp_params_flags update;
+ struct kmb_blc_params blc[KMB_CAM_MAX_EXPOSURES];
+ struct kmb_sigma_dns_params sigma_dns[KMB_CAM_MAX_EXPOSURES];
+ struct kmb_lsc_params lsc;
+ struct kmb_raw_params raw;
+ struct kmb_ae_awb_params ae_awb;
+ struct kmb_af_params af;
+ struct kmb_hist_params histogram;
+ struct kmb_lca_params lca;
+ struct kmb_debayer_params debayer;
+ struct kmb_dog_dns_params dog_dns;
+ struct kmb_luma_dns_params luma_dns;
+ struct kmb_sharpen_params sharpen;
+ struct kmb_chroma_gen_params chroma_gen;
+ struct kmb_median_params median;
+ struct kmb_chroma_dns_params chroma_dns;
+ struct kmb_color_comb_params color_comb;
+ struct kmb_hdr_params hdr;
+ struct kmb_lut_params lut;
+ struct kmb_tnf_params tnf;
+ struct kmb_dehaze_params dehaze;
+ struct kmb_warp_params warp;
+} __packed;
+
+/**
+ * struct kmb_isp_stats_flags - Bits to indicate which stats need to be updated
+ *
+ * @ae_awb: 1 = updated, 0 = not updated.
+ * @af: 1 = updated, 0 = not updated.
+ * @luma_hist: 1 = updated, 0 = not updated.
+ * @rgb_hist: 1 = updated, 0 = not updated.
+ * @flicker_rows: 1 = updated, 0 = not updated.
+ * @dehaze: 1 = updated, 0 = not updated.
+ * @reserved: reserved for future use and for alignment
+ */
+struct kmb_isp_stats_flags {
+ __u32 ae_awb:1;
+ __u32 af:1;
+ __u32 luma_hist:1;
+ __u32 rgb_hist:1;
+ __u32 flicker_rows:1;
+ __u32 dehaze:1;
+ __u32 reserved:26;
+} __packed;
+
+/**
+ * struct kmb_isp_stats - KMB ISP raw statistics
+ *
+ * @exposure: Array with exposure statistics blocks. When HDR mode is not
+ * enable only statistics with index 0 are valid. Included stats:
+ * @exposure.ae_awb_stats: Raw AE/AWB statistics.
+ * @exposure.af_stats: Raw AF statistics.
+ * @exposure.hist_luma: Luma histogram.
+ * @exposure.hist_rgb: RGB histogram.
+ * @exposure.flicker_rows: Flicker detection rows.
+ * @dehaze: Dehaze statistics it is collected after HDR Fusion in HDR case.
+ * @update: Select which stats to update, see kmb_vpu_isp_stats_flags
+ */
+struct kmb_isp_stats {
+ struct {
+ __u8 ae_awb_stats[KMB_CAM_AE_AWB_STATS_SIZE];
+ __u8 af_stats[KMB_CAM_AF_STATS_SIZE];
+ __u8 hist_luma[KMB_CAM_HIST_LUMA_SIZE];
+ __u8 hist_rgb[KMB_CAM_HIST_RGB_SIZE];
+ __u8 flicker_rows[KMB_CAM_FLICKER_ROWS_SIZE];
+ } exposure[KMB_CAM_MAX_EXPOSURES];
+ __u8 dehaze[MAX_DHZ_AIRLIGHT_STATS_SIZE];
+ struct kmb_isp_stats_flags update;
+} __packed;
+
+#endif /* KEEMBAY_ISP_CTL_H */
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread* Re: [PATCH 04/10] uapi: Keem Bay ISP Parameters data types
2021-03-19 18:06 ` [PATCH 04/10] uapi: Keem Bay ISP Parameters data types Martina Krasteva
@ 2021-03-19 20:58 ` kernel test robot
2021-03-22 13:32 ` Sakari Ailus
1 sibling, 0 replies; 28+ messages in thread
From: kernel test robot @ 2021-03-19 20:58 UTC (permalink / raw)
To: Martina Krasteva, linux-media
Cc: kbuild-all, mchehab, robh+dt, devicetree, sakari.ailus,
daniele.alessandrelli, paul.j.murphy, gjorgjix.rosikopulos,
martinax.krasteva
[-- Attachment #1: Type: text/plain, Size: 1487 bytes --]
Hi Martina,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on f00397ee41c79b6155b9b44abd0055b2c0621349]
url: https://github.com/0day-ci/linux/commits/Martina-Krasteva/Keem-Bay-Camera-Subsystem/20210320-020904
base: f00397ee41c79b6155b9b44abd0055b2c0621349
config: i386-randconfig-s002-20210318 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-22) 9.3.0
reproduce:
# apt-get install sparse
# sparse version: v0.6.3-277-gc089cd2d-dirty
# https://github.com/0day-ci/linux/commit/f90a61aa54b5ce4a5d3876ebf2a229ed7708af4e
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Martina-Krasteva/Keem-Bay-Camera-Subsystem/20210320-020904
git checkout f90a61aa54b5ce4a5d3876ebf2a229ed7708af4e
# save the attached .config to linux build tree
make W=1 C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' ARCH=i386
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
In file included from ./usr/include/linux/keembay-isp-ctl.h:11,
from <command-line>:32:
>> ./usr/include/linux/videodev2.h:2370:20: error: field 'timestamp' has incomplete type
2370 | struct timespec timestamp;
| ^~~~~~~~~
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 36669 bytes --]
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH 04/10] uapi: Keem Bay ISP Parameters data types
2021-03-19 18:06 ` [PATCH 04/10] uapi: Keem Bay ISP Parameters data types Martina Krasteva
2021-03-19 20:58 ` kernel test robot
@ 2021-03-22 13:32 ` Sakari Ailus
1 sibling, 0 replies; 28+ messages in thread
From: Sakari Ailus @ 2021-03-22 13:32 UTC (permalink / raw)
To: Martina Krasteva
Cc: linux-media, mchehab, robh+dt, devicetree, daniele.alessandrelli,
paul.j.murphy, gjorgjix.rosikopulos
Hi Martina,
Thanks for the patchset. A few more comments below.
On Fri, Mar 19, 2021 at 06:06:26PM +0000, Martina Krasteva wrote:
> From: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
>
> ISP parameters passed to the “keembay-metadata-params”
> metadata output video node
>
> Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
> Co-developed-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
> Signed-off-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
> Co-developed-by: Martina Krasteva <martinax.krasteva@intel.com>
> Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
> Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
> Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
> ---
> MAINTAINERS | 1 +
> include/uapi/linux/keembay-isp-ctl.h | 796 +++++++++++++++++++++++++++++++++++
> 2 files changed, 797 insertions(+)
> create mode 100644 include/uapi/linux/keembay-isp-ctl.h
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 76082714a76f..955f9f6a195d 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1973,6 +1973,7 @@ S: Maintained
> T: git git://linuxtv.org/media_tree.git
> F: Documentation/devicetree/bindings/media/intel,keembay-camera.yaml
> F: drivers/media/platform/keembay-camera/
> +F: include/uapi/linux/keembay-isp-ctl.h
>
> ARM/IP FABRICS DOUBLE ESPRESSO MACHINE SUPPORT
> M: Lennert Buytenhek <kernel@wantstofly.org>
> diff --git a/include/uapi/linux/keembay-isp-ctl.h b/include/uapi/linux/keembay-isp-ctl.h
> new file mode 100644
> index 000000000000..86e1654067f0
> --- /dev/null
> +++ b/include/uapi/linux/keembay-isp-ctl.h
> @@ -0,0 +1,796 @@
> +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */
> +/*
> + * Intel Keem Bay camera control parameters and statistics
> + *
> + * Copyright (C) 2021 Intel Corporation
> + */
> +#ifndef KEEMBAY_ISP_CTL_H
> +#define KEEMBAY_ISP_CTL_H
> +
> +#include <linux/types.h>
> +#include <linux/videodev2.h>
> +
> +#define KMB_CAM_MAX_EXPOSURES 3
> +
> +/* Table max sizes */
> +#define KMB_CAM_LCA_MESH_SIZE (32 * 24)
> +#define KMB_CAM_SHARPEN_RADIAL_SIZE 256
> +#define KMB_CAM_LUT3D_SIZE (16 * 16 * 16 * 4)
> +#define KMB_CAM_LSC_SIZE ((64 + 4) * 64 * 4)
> +#define KMB_CAM_GAMMA_SIZE (512 * 4)
> +#define KMB_CAM_STATIC_DEFECT_SIZE 1000
> +#define KMB_CAM_CHROMA_LUT_SIZE 1024
> +#define KMB_CAM_WARP_MESH_SIZE (480 * 270)
> +#define KMB_CAM_HDR_TM_LUTS_SIZE (((67 * 16) + 63) & ~63)
> +
> +/* Statistics max sizes */
> +#define KMB_CAM_AE_AWB_STATS_SIZE (64 * 64)
> +#define KMB_CAM_AF_STATS_SIZE (64 * 64)
> +#define KMB_CAM_HIST_LUMA_SIZE 256
> +#define KMB_CAM_HIST_RGB_SIZE (3 * 128)
> +#define KMB_CAM_FLICKER_ROWS_SIZE (65535 + 1) /* align to 64 */
> +#define MAX_DHZ_AIRLIGHT_STATS_SIZE (4 * 128)
> +
> +/**
> + * enum kmb_vpu_isp_bayer_order - KMB sensor Bayer arrangement format types
> + *
> + * @KMB_ISP_BAYER_ORDER_GRBG: Gr R B Gr
> + * @KMB_ISP_BAYER_ORDER_RGGB: R Gr Gr B
> + * @KMB_ISP_BAYER_ORDER_GBRG: Gr B R Gr
> + * @KMB_ISP_BAYER_ORDER_BGGR: B Gr Gr R
> + */
> +enum kmb_isp_bayer_order {
> + KMB_ISP_BAYER_ORDER_GRBG = 0,
> + KMB_ISP_BAYER_ORDER_RGGB = 1,
> + KMB_ISP_BAYER_ORDER_GBRG = 2,
> + KMB_ISP_BAYER_ORDER_BGGR = 3,
> +} __packed;
Please replace with __attribute__((packed)). The userspace headers aren't
converted to use the GCC attributes.
> +
> +/**
> + * struct kmb_blc_params - KMB Black Level Correction parameters
> + *
> + * @coeff1: Black level correction coefficient parameter. Range [0 - 4096]
> + * @coeff2: Black level correction coefficient parameter. Range [0 - 4096]
> + * @coeff3: Black level correction coefficient parameter. Range [0 - 4096]
> + * @coeff4: Black level correction coefficient parameter. Range [0 - 4096]
> + */
> +struct kmb_blc_params {
> + __u32 coeff1;
> + __u32 coeff2;
> + __u32 coeff3;
> + __u32 coeff4;
> +} __packed;
> +
> +/**
> + * struct kmb_sigma_dns_params - KMB Sigma Denoise parameters
> + *
> + * @noise: Sigma denoise noise parameter. Range [0 - 65535]
> + * @threshold1: Sigma denoise min threshold1 parameter. Range [0 - 255]
> + * @threshold2: Sigma denoise max threshold2 parameter. Range [0 - 255]
> + * @threshold3: Sigma denoise min threshold3 parameter. Range [0 - 255]
> + * @threshold4: Sigma denoise max threshold4 parameter. Range [0 - 255]
> + * @threshold5: Sigma denoise min threshold5 parameter. Range [0 - 255]
> + * @threshold6: Sigma denoise max threshold6 parameter. Range [0 - 255]
> + * @threshold7: Sigma denoise min threshold7 parameter. Range [0 - 255]
> + * @threshold8: Sigma denoise max threshold8 parameter. Range [0 - 255]
> + */
> +struct kmb_sigma_dns_params {
> + __u32 noise;
> + __u32 threshold1;
> + __u32 threshold2;
> + __u32 threshold3;
> + __u32 threshold4;
> + __u32 threshold5;
> + __u32 threshold6;
> + __u32 threshold7;
> + __u32 threshold8;
> +} __packed;
> +
> +/**
> + * struct kmb_lsc_params - KMB Lens Shading Correction parameters
> + *
> + * @threshold: Lens shading correction threshold parameter
> + * @width: Lens shading correction gain1 parameter. Range [1 - 64].
> + * Must be a multiple of 4
> + * @height: Lens shading correction gain2 parameter. Range [1 - 64].
> + * Must be an even number
> + * @gain_mesh: Lens shading correction gain mesh table
> + */
> +struct kmb_lsc_params {
> + __u32 threshold;
> + __u32 width;
> + __u32 height;
> + __u8 gain_mesh[KMB_CAM_LSC_SIZE];
> +} __packed;
> +
> +/**
> + * struct kmb_raw_params - KMB Raw parameters
> + *
> + * @awb_stats_en: Enable AE/AWB stats output
> + * @awb_rgb_hist_en: Enable RGB histogram output
> + * @af_stats_en: Enable AF stats output
> + * @luma_hist_en: Enable Luma histogram output
> + * @flicker_accum_en: Enable flicker detection row accumulation output
> + * @bad_pixel_fix_en: Enable Hot/Cold pixel suppression
> + * @grgb_imb_en: Enable Gr/Gb imbalance correction
> + * @mono_imbalance_en: Enable mono imbalance correction
> + * @gain1: Raw gain1 parameter
> + * @gain2: Raw gain2 parameter
> + * @gain3: Raw gain3 parameter
> + * @gain4: Raw gain4 parameter
> + * @stop1: Raw stop1 parameter
> + * @stop2: Raw stop2 parameter
> + * @stop3: Raw stop3 parameter
> + * @stop4: Raw stop4 parameter
> + * @threshold1: Raw threshold1 parameter
> + * @alpha1: Raw alpha1 parameter. Range [0 - 15]
> + * @alpha2: Raw alpha2 parameter. Range [0 - 15]
> + * @alpha3: Raw alpha3 parameter. Range [0 - 15]
> + * @alpha4: Raw alpha4 parameter. Range [0 - 15]
> + * @threshold2: Raw threshold2 parameter. Range [0 - 2047]
> + * @static_defect_size: static_defect_map size parameter. Range [0 - 65536]
> + * @static_defect_map: Static defect map
> + * @start_row: Raw start_row parameter
> + * @end_row: Raw end_row parameter
> + */
> +struct kmb_raw_params {
> + __u32 awb_stats_en;
> + __u32 awb_rgb_hist_en;
> + __u32 af_stats_en;
> + __u32 luma_hist_en;
> + __u32 flicker_accum_en;
> + __u32 bad_pixel_fix_en;
> + __u32 grgb_imb_en;
> + __u32 mono_imbalance_en;
> + __u32 gain1;
> + __u32 gain2;
> + __u32 gain3;
> + __u32 gain4;
> + __u32 stop1;
> + __u32 stop2;
> + __u32 stop3;
> + __u32 stop4;
> + __u32 threshold1;
> + __u32 alpha1;
> + __u32 alpha2;
> + __u32 alpha3;
> + __u32 alpha4;
> + __u32 threshold2;
> + __u32 static_defect_size;
> + __u8 static_defect_map[KMB_CAM_STATIC_DEFECT_SIZE];
> + __u32 start_row;
> + __u32 end_row;
> +} __packed;
> +
> +/**
> + * struct kmb_ae_awb_params - KMB AE/AWB statistics parameters
> + *
> + * @start_x: AE/AWB start_x parameter
> + * @start_y: AE/AWB start_y parameter
> + * @width: AE/AWB width parameter
> + * @height: AE/AWB height parameter
> + * @skip_x: AE/AWB skip_x parameter
> + * @skip_y: AE/AWB skip_y parameter
> + * @patches_x: AE/AWB patches_x parameter
> + * @patches_y: AE/AWB patches_y parameter
> + * @threshold1: AE/AWB threshold1 parameter
> + * @threshold2: AE/AWB threshold2 parameter
> + */
> +struct kmb_ae_awb_params {
> + __u32 start_x;
> + __u32 start_y;
> + __u32 width;
> + __u32 height;
> + __u32 skip_x;
> + __u32 skip_y;
> + __u32 patches_x;
> + __u32 patches_y;
> + __u16 threshold1;
> + __u16 threshold2;
> +} __packed;
> +
> +/**
> + * struct kmb_af_params - KMB Auto Focus parameters
> + *
> + * @start_x: AF start_x parameter
> + * @start_y: AF start_y parameter
> + * @width: AF width parameter
> + * @height: AF height parameter
> + * @patches_x: AF patches_x parameter
> + * @patches_y: AF patches_y parameter
> + * @coeff: AF filter coeff parameter
> + * @threshold1: AF filer threshold1 parameter
> + * @threshold2: AF filer threshold2 parameter
> + * @coeffs1: AF filter coeffs1 parameter
> + * @coeffs2: AF filter coeffs2 parameter
> + */
> +struct kmb_af_params {
> + __u32 start_x;
> + __u32 start_y;
> + __u32 width;
> + __u32 height;
> + __u32 patches_x;
> + __u32 patches_y;
> + __s32 coeff;
> + __s32 threshold1;
> + __s32 threshold2;
> + __s32 coeffs1[11];
> + __s32 coeffs2[11];
> +} __packed;
> +
> +/**
> + * struct kmb_hist_params - KMB Hist parameters
> + *
> + * @start_x: Hist start_x parameter. Range [0 - 1]
> + * @start_y: Hist start_y parameter. Range [0 - 1]
> + * @end_x: Hist end_x parameter. Range [0 - 1]
> + * @end_y: Hist end_y parameter. Range [0 - 1]
> + * @matrix: Hist matrix parameter. Range [0.0 - 8.0]
> + * @weight: Hist weight parameter. Range [0.0 - 1.0]
> + */
> +struct kmb_hist_params {
> + __u32 start_x;
> + __u32 start_y;
> + __u32 end_x;
> + __u32 end_y;
> + __u16 matrix[9];
> + __u16 weight[3];
> +} __packed;
> +
> +/**
> + * struct kmb_lca_params - KMB Lateral Chromatic Aberration parameters
> + *
> + * @coeff: LCA coeff parameter
> + */
> +struct kmb_lca_params {
> + __u8 coeff[KMB_CAM_LCA_MESH_SIZE];
> +} __packed;
> +
> +/**
> + * struct kmb_debayer_params - KMB Debayer parameters
> + *
> + * @coeff1: Filter coeff1 parameter
> + * @multiplier1: Filter multiplier1 parameter
> + * @multiplier2: Filter multiplier2 parameter
> + * @coeff2: Filter coeff2 parameter
> + * @coeff3: Filter coeff3 parameter
> + * @coeff4: Filter coeff4 parameter
> + */
> +struct kmb_debayer_params {
> + __s32 coeff1;
> + __u32 multiplier1;
> + __u32 multiplier2;
> + __s32 coeff2;
> + __s32 coeff3;
> + __s32 coeff4;
> +} __packed;
> +
> +/**
> + * struct kmb_hdr_params - KMB HDR parameters
> + *
> + * @ratio: HDR ratio parameter. Range [0 - 65536]
> + * @scale: HDR scale parameter
> + * @offset1: HDR offset1 parameter. Range [-4095 - 0]
> + * @slope1: HDR slope1 parameter. Range [0 - 4095]
> + * @offset2: HDR offset2 parameter. Range [-4095 - 0]
> + * @slope2: HDR slope2 parameter. Range [0 - 4095]
> + * @offset3: HDR offset3 parameter. Range [0 - 4095]
> + * @slope3: HDR slope3 parameter. Range [0 - 4095]
> + * @offset4: HDR offset4 parameter. Range [0 - 4095]
> + * @gain1: HDR gain1 parameter. Range [0 - 4095]
> + * @blur1: HDR blur1 parameter. Range [0.0 - 255.0]
> + * @blur2: HDR blur2 parameter. Range [0 - 255]
> + * @contrast1: HDR contrast1 parameter
> + * @contrast2: HDR contrast2 parameter
> + * @enable1: HDR enable1 parameter
> + * @enable2: HDR enable2 parameter
> + * @offset5: HDR offset5 parameter
> + * @gain2: HDR gain2 parameter
> + * @offset6: HDR offset6 parameter. Range [0 - 1024]
> + * @strength: HDR strength parameter. Range [0 - 65536]
> + * @tm_lut: HDR tm lut parameter
> + * @offset7: HDR offset7 parameter. Range [0 - 65536]
> + * @shift: HDR shift parameter
> + * @field1: HDR filed1 parameter
> + * @field2: HDR field2 parameter
> + * @gain3: HDR gain3 parameter. Range [0 - 255]
> + * @min: HDR min parameter. Range [0 - 4095]
> + */
> +struct kmb_hdr_params {
> + __u32 ratio[2];
> + __u32 scale[3];
> + __s32 offset1;
> + __u32 slope1;
> + __s32 offset2;
> + __u32 slope2;
> + __s32 offset3;
> + __u32 slope3;
> + __s32 offset4;
> + __u32 gain1;
> + __u32 blur1[3];
> + __u32 blur2[5];
> + __u32 contrast1;
> + __u32 contrast2;
> + __u32 enable1;
> + __u32 enable2;
> + __s32 offset5;
> + __u32 gain2;
> + __s32 offset6;
> + __u32 strength;
> + __u8 tm_lut[KMB_CAM_HDR_TM_LUTS_SIZE];
> + __u16 offset7;
> + __u32 shift;
> + __u16 field1;
> + __u16 field2;
> + __u8 gain3;
> + __u16 min;
If this struct is only used on UAPI, could you use reserved fields (or
change types) so that the ABI alignment rules would be fulfilled without
packing the struct? The above works, but the unaligned accesses are costing
cycles if nothing else.
Please also check the rest of the file for the same.
> +} __packed;
> +
> +/**
> + * struct kmb_dog_dns_params - KMB Difference-of-Gaussians DNS parameters
> + *
> + * @threshold: Filter threshold parameter. Range [0 - 255]
> + * @strength: Filter strength parameter. Range [0 - 255]
> + * @coeffs11: Filter coeffs11 parameter. Range [0 - 1023]
> + * @coeffs15: Filter coeffs15 parameter. Range [0 - 1023]
> + */
> +struct kmb_dog_dns_params {
> + __u32 threshold;
> + __u32 strength;
> + __u8 coeffs11[6];
> + __u8 coeffs15[8];
> +} __packed;
> +
> +/**
> + * struct kmb_luma_dns_params - KMB Luma DNS parameters
> + *
> + * @threshold: Luma DNS threshold parameter. Range [0 - 32768]
> + * @slope: Luma DNS slope parameter. Range [0 - 2048]
> + * @shift: Luma DNS shift parameter. Range [0 - 255]
> + * @alpha: Luma DNS alpha parameter. Range [0 - 127]
> + * @weight: Luma DNS weight parameter. Range [0 - 4294967295]
> + * @per_pixel_alpha_en: Enable adapt alpha
> + * @gain_bypass_en: Enable gain bypass
> + */
> +struct kmb_luma_dns_params {
> + __u32 threshold;
> + __u32 slope;
> + __u32 shift;
> + __u32 alpha;
> + __u32 weight;
> + __u32 per_pixel_alpha_en;
> + __u32 gain_bypass_en;
> +} __packed;
> +
> +/**
> + * struct kmb_sharpen_params - KMB Sharpen parameters
> + *
> + * @coeffs1: Filter coeffs1 parameter. Range [0 - 255]
> + * @coeffs2: Filter coeffs2 parameter. Range [0 - 255]
> + * @coeffs3: Filter coeffs3 parameter. Range [0 - 255]
> + * @shift: Filter shift parameter. Range [0 - 255]
> + * @gain1: Filter gain1 parameter. Range [0 - 2047]
> + * @gain2: Filter gain2 parameter. Range [0 - 2047]
> + * @gain3: Filter gain3 parameter. Range [0 - 2047]
> + * @gain4: Filter gain4 parameter. Range [0 - 2047]
> + * @gain5: Filter gain5 parameter. Range [0 - 255]
> + * @stops1: Filter stops1 parameter. Range [0 - 4095]
> + * @gains: Filter gains parameter. Range [0 - 256]
> + * @stops2: Filter stops2 parameter. Range [0 - 4095]
> + * @overshoot: Filter overshoot parameter. Range [0 - 256]
> + * @undershoot: Filter undershoot parameter. Range [0 - 256]
> + * @alpha: Filter alpha parameter. Range [0 - 256]
> + * @gain6: Filter gain6 parameter. Range [0 - 255]
> + * @offset: Filter offset parameter. Range [0 - 1023]
> + * @radial_lut: Sharpen radial LUT parameter. Range [0 - 255]
> + */
> +struct kmb_sharpen_params {
> + __u16 coeffs1[6];
> + __u16 coeffs2[6];
> + __u16 coeffs3[6];
> + __u32 shift;
> + __u32 gain1;
> + __u32 gain2;
> + __u32 gain3;
> + __u32 gain4;
> + __u32 gain5;
> + __u32 stops1[3];
> + __u32 gains[3];
> + __u32 stops2[4];
> + __u32 overshoot;
> + __u32 undershoot;
> + __u32 alpha;
> + __u32 gain6;
> + __u32 offset;
> + __u8 radial_lut[KMB_CAM_SHARPEN_RADIAL_SIZE];
> +} __packed;
> +
> +/**
> + * struct kmb_chroma_gen_params - KMB Chroma GEN parameters
> + *
> + * @epsilon: Chroma GEN epsilon parameter. Range [0 - 255]
> + * @coeff1: Chroma GEN coeff1 parameter. Range [0 - 1024]
> + * @coeff2: Chroma GEN coeff2 parameter. Range [0 - 1024]
> + * @coeff3: Chroma GEN coeff3 parameter. Range [0 - 1024]
> + * @coeff4: Chroma GEN coeff4 parameter. Range [0 - 255]
> + * @coeff5: Chroma GEN coeff5 parameter. Range [0 - 255]
> + * @coeff6: Chroma GEN coeff6 parameter. Range [0 - 255]
> + * @strength1: Chroma GEN strength1 parameter. Range [0 - 255]
> + * @strength2: Chroma GEN strength2 parameter. Range [0 - 255]
> + * @coeffs: Chroma GEN coeffs parameter . Range [0 - 255]
> + * @offset1: Chroma GEN offset1 parameter. Range [0 - 255]
> + * @slope1: Chroma GEN slope1 parameter. Range [0 - 255]
> + * @slope2: Chroma GEN slope2 parameter. Range [0 - 255]
> + * @offset2: Chroma GEN offset2 parameter. Range [0 - 255]
> + * @limit: Chroma GEN limit parameter. Range [0 - 767]
> + */
> +struct kmb_chroma_gen_params {
> + __u32 epsilon;
> + __u32 coeff1;
> + __u32 coeff2;
> + __u32 coeff3;
> + __u32 coeff4;
> + __u32 coeff5;
> + __u32 coeff6;
> + __u32 strength1;
> + __u32 strength2;
> + __u32 coeffs[3];
> + __s32 offset1;
> + __u32 slope1;
> + __u32 slope2;
> + __s32 offset2;
> + __u32 limit;
> +} __packed;
> +
> +/**
> + * struct kmb_median_params - KMB Median parameters
> + *
> + * @size: Filter size parameter. Range [1;3;5;7]
> + * @slope: Filter slope parameter. Range [0 - 128]
> + * @offset: Filter offset parameter. Range [-32 - 32]
> + */
> +struct kmb_median_params {
> + __u32 size;
> + __u32 slope;
> + __s32 offset;
> +} __packed;
> +
> +/**
> + * struct kmb_chroma_dns_params - KMB Chroma Denoise parameters
> + *
> + * @limit: Filter limit parameter
> + * @enable: Filter enable parameter
> + * @threshold1: Filter threshold1 parameter
> + * @threshold2: Filter threshold2 parameter
> + * @threshold3: Filter threshold3 parameter. Range [0 - 255]
> + * @threshold4: Filter threshold4 parameter. Range [0 - 255]
> + * @threshold5: Filter threshold5 parameter. Range [0 - 255]
> + * @threshold6: Filter threshold6 parameter. Range [0 - 255]
> + * @threshold7: Filter threshold7 parameter. Range [0 - 255]
> + * @threshold8: Filter threshold8 parameter. Range [0 - 255]
Such documentation isn't telling much to the programmer considering using
the API. :-( The same applies to much of the the kerneldoc documentation in
this file.
Could you use the usual range notation e.g. [0, 1] for closed ranges and
(0, 1) for open ones.
> + * @slope1: Filter slope1 parameter. Range [0 - 255]
> + * @offset1: Filter offset1 parameter. Range [0 - 255]
> + * @slope2: Filter slope2 parameter. Range [0 - 255]
> + * @offset2: Filter offset2 parameter. Range [0 - 255]
> + * @grey1: Filter grey1 parameter. Range [0 - 255]
> + * @grey2: Filter grey2 parameter. Range [0 - 255]
> + * @grey3: Filter grey3 parameter. Range [0 - 255]
> + * @coeff1: Filter coeff1 parameter. Range [0 - 255]
> + * @coeff2: Filter coeff2 parameter. Range [0 - 255]
> + * @coeff3: Filter coeff3 parameter. Range [0 - 255]
> + */
> +struct kmb_chroma_dns_params {
> + __u32 limit;
> + __u32 enable;
> + __u32 threshold1;
> + __u32 threshold2;
> + __u32 threshold3;
> + __u32 threshold4;
> + __u32 threshold5;
> + __u32 threshold6;
> + __u32 threshold7;
> + __u32 threshold8;
> + __u32 slope1;
> + __s32 offset1;
> + __u32 slope2;
> + __s32 offset2;
> + __u32 grey1;
> + __u32 grey2;
> + __u32 grey3;
> + __u32 coeff1;
> + __u32 coeff2;
> + __u32 coeff3;
> +} __packed;
> +
> +/**
> + * struct kmb_color_comb_params - KMB Color Combine parameters
> + *
> + * @matrix: Color combine matrix parameter. Range [-8.0 - 8.0]
> + * @offsets:Color combine offsets parameter. Range [-1.0 - 1.0]
These are integers. Is the field a Q number?
> + * @coeff1: Color combine coeff1 parameter. Range [0 - 1023]
> + * @coeff2: Color combine coeff2 parameter. Range [0 - 1023]
> + * @coeff3: Color combine coeff3 parameter. Range [0 - 1023]
> + * @lut_3d: Color combine LUT 3D parameter. Range [0 - 4095]
> + * @enable: Color combine enable parameter
> + * @weight1: Color combine weight1 parameter. Range [0 - 255]
> + * @weight2: Color combine weight2 parameter. Range [0 - 255]
> + * @weight3: Color combine weight3 parameter. Range [0 - 255]
> + * @limit1: Color combine limit1 parameter. Range [0 - 32766]
> + * @limit2: Color combine limit2 parameter. Range [-32766 - 0]
> + * @offset1: Color combine offset1 parameter. Range [0 - 8192]
> + * @offset2: Color combine offset2 parameter. Range [0 - 8192]
> + */
> +struct kmb_color_comb_params {
> + __u16 matrix[9];
> + __u16 offsets[3];
> + __u32 coeff1;
> + __u32 coeff2;
> + __u32 coeff3;
> + __u8 lut_3d[KMB_CAM_LUT3D_SIZE];
> + __u32 enable;
> + __u32 weight1;
> + __u32 weight2;
> + __u32 weight3;
> + __u32 limit1;
> + __s32 limit2;
> + __s32 offset1;
> + __s32 offset2;
> +} __packed;
> +
> +/**
> + * struct kmb_lut_params - KMB lut parameters
> + *
> + * @size: Lut size parameter. Range [0 - 128]
> + * @table: Lut table parameter
> + * @matriix: Lut matrix parameter
> + * @offsets: Lut offsets pparameter
> + */
> +struct kmb_lut_params {
> + __u32 size;
> + __u8 table[KMB_CAM_GAMMA_SIZE];
> + __u16 matrix[3 * 3];
> + __u16 offsets[3];
> +} __packed;
> +
> +/**
> + * struct kmb_tnf_params - KMB Temporal Noise Filter parameters
> + *
> + * @factor: Filter factor parameter. Range [0 - 255]
> + * @gain: Filter gain parameter. Range [0 - 4095]
> + * @offset1: Filter offset1 parameter. Range [0 - 8192]
> + * @slope1: Filter slope1 parameter. Range [0 - 512]
> + * @offset2: Filter offset2 parameter. Range [0 - 8192]
> + * @slope2: Filter slope2 parameter. Range [0 - 512]
> + * @min1: Filter min1 parameter. Range [0 - 65535]
> + * @min2: Filter min2 parameter. Range [0 - 65535]
> + * @value: Filter value parameter. Range [0 - 255]
> + * @enable: Filter enable parameter
> + * @chroma_lut0: First chroma LUT. Range [0 - 4095]
> + * @chroma_lut1: Second chroma LUT. Range [0 - 4095]
> + */
> +struct kmb_tnf_params {
> + __u32 factor;
> + __u32 gain;
> + __u32 offset1;
> + __u32 slope1;
> + __u32 offset2;
> + __u32 slope2;
> + __u32 min1;
> + __u32 min2;
> + __u32 value;
> + __u32 enable;
> + __u8 chroma_lut0[KMB_CAM_CHROMA_LUT_SIZE];
> + __u8 chroma_lut1[KMB_CAM_CHROMA_LUT_SIZE];
> +} __packed;
> +
> +/**
> + * struct kmb_dehaze_params - KMB dehaze parameters
> + *
> + * @gain1: Dehaze gain1 parameter. Range [0 - 1023]
> + * @min: Dehaze min parameter. Range [0 - 255]
> + * @strength1: Dehaze strength1 parameter. Range [0 - 255]
> + * @strength2: Dehaze strength2 parameter. Range [0 - 65535]
> + * @gain2: Dehaze gain2 parameter. Range [0 - 255]
> + * @saturation: Dehaze saturation parameter. Range [0 - 127]
> + * @value1: Dehaze value1 parameter. Range [0 - 4095]
> + * @value2: Dehaze value2 parameter. Range [0 - 4095]
> + * @value3: Dehaze value3 parameter. Range [0 - 4095]
> + * @filter: Dehaze filter parameter. Range [0 - 255]
> + */
> +struct kmb_dehaze_params {
> + __u32 gain1;
> + __u32 min;
> + __u32 strength1;
> + __u32 strength2;
> + __u32 gain2;
> + __u32 saturation;
> + __u32 value1;
> + __u32 value2;
> + __u32 value3;
> + __u32 filter[3];
> +} __packed;
> +
> +/**
> + * struct kmb_warp_params - KMB Warp filter parameters
> + *
> + * @type: Warp filter type parameter. Range [0 - 1]
> + * @relative: Warp filter relative parameter. Range [0 - 1]
> + * @format: Warp filter format parameter. Range [0 - 1]
> + * @position: Warp filter position parameter
> + * @mesh_grid: Warp mesh grid
> + * @width: Warp filter width parameter
> + * @height: Warp filter height parameter
> + * @stride: Warp filter stride parameter
> + * @enable: Warp filter enable parameter
> + * @matrix: Warp matrix parameter
> + * @mode: Warp filter mode parameter. Range [0 - 1]
> + * @values: Warp filter values parameter. Range [0 - 128]
> + */
> +struct kmb_warp_params {
> + __u8 type;
> + __u8 relative;
> + __u8 format;
> + __u8 position;
> + __u8 mesh_grid[KMB_CAM_WARP_MESH_SIZE];
> + __u16 width;
> + __u16 height;
> + __u32 stride;
> + __u8 enable;
> + __u32 matrix[9];
> + __u8 mode;
> + __u16 values[3];
> +} __packed;
> +
> +/**
> + * struct kmb_isp_params_flags - Bits to indicate which params
> + * need to be updated
> + *
> + * @blc: 1 = update, 0 = do not update.
> + * @sigma_dns: 1 = update, 0 = do not update.
> + * @lsc: 1 = update, 0 = do not update.
> + * @raw: 1 = update, 0 = do not update.
> + * @ae_awb: 1 = update, 0 = do not update.
> + * @af: 1 = update, 0 = do not update.
> + * @histogram: 1 = update, 0 = do not update.
> + * @lca: 1 = update, 0 = do not update.
> + * @debayer: 1 = update, 0 = do not update.
> + * @dog_dns: 1 = update, 0 = do not update.
> + * @luma_dns: 1 = update, 0 = do not update.
> + * @sharpen: 1 = update, 0 = do not update.
> + * @chroma_gen: 1 = update, 0 = do not update.
> + * @median: 1 = update, 0 = do not update.
> + * @chroma_dns: 1 = update, 0 = do not update.
> + * @color_comb: 1 = update, 0 = do not update.
> + * @hdr: 1 = update, 0 = do not update.
> + * @lut: 1 = update, 0 = do not update.
> + * @tnf: 1 = update, 0 = do not update.
> + * @dehaze: 1 = update, 0 = do not update.
> + * @warp: 1 = update, 0 = do not update.
> + * @reserved: reserved for future use and for alignment
> + */
> +struct kmb_isp_params_flags {
> + __u32 blc:1;
> + __u32 sigma_dns:1;
> + __u32 lsc:1;
> + __u32 raw:1;
> + __u32 ae_awb:1;
> + __u32 af:1;
> + __u32 histogram:1;
> + __u32 lca:1;
> + __u32 debayer:1;
> + __u32 dog_dns:1;
> + __u32 luma_dns:1;
> + __u32 sharpen:1;
> + __u32 chroma_gen:1;
> + __u32 median:1;
> + __u32 chroma_dns:1;
> + __u32 color_comb:1;
> + __u32 hdr:1;
> + __u32 lut:1;
> + __u32 tnf:1;
> + __u32 dehaze:1;
> + __u32 warp:1;
> + __u32 reserved:11;
> +} __packed;
> +
> +/**
> + * struct kmb_isp_params - KMB ISP parameters structure
> + *
> + * @update: Select which parameters to apply, see kmb_vpu_isp_params_flags
> + * @blc: Black Level correction parameters
> + * @sigma_dns: Sigma denoise parameters
> + * @lsc: Lens Shading Correction parameters
> + * @raw: Raw parameters
> + * @ae_awb: Auto exposure/Auto white balance parameters
> + * @af: Auto focus parameters
> + * @histogram: Histogram parameters
> + * @lca: Lateral Chromatic Aberration filter parameters
> + * @debayer: SIPP Bayer demosaicing filter parameters
> + * @dog_dns: Difference-of-Gaussians filter parameters
> + * @luma_dns: Luma denoise parameters
> + * @sharpen: Sharpen filter parameters
> + * @chroma_gen: Chroma GEN parameters
> + * @median: Median hardware filter parameters
> + * @chroma_dns: Chroma Denoise hardware filter parameters
> + * @color_comb: Color Combine parameters
> + * @hdr: HDR parameters applied only in HDR mode
> + * @lut: LUT parameters
> + * @tnf: Temporal Noise Filter parameters
> + * @dehaze: Dehaze parameters
> + * @warp: Warp filter parameters
> + *
> + * Each struct represents a filter and its settings which are applied on the raw
> + * image.
> + */
> +struct kmb_isp_params {
> + struct kmb_isp_params_flags update;
> + struct kmb_blc_params blc[KMB_CAM_MAX_EXPOSURES];
> + struct kmb_sigma_dns_params sigma_dns[KMB_CAM_MAX_EXPOSURES];
> + struct kmb_lsc_params lsc;
> + struct kmb_raw_params raw;
> + struct kmb_ae_awb_params ae_awb;
> + struct kmb_af_params af;
> + struct kmb_hist_params histogram;
> + struct kmb_lca_params lca;
> + struct kmb_debayer_params debayer;
> + struct kmb_dog_dns_params dog_dns;
> + struct kmb_luma_dns_params luma_dns;
> + struct kmb_sharpen_params sharpen;
> + struct kmb_chroma_gen_params chroma_gen;
> + struct kmb_median_params median;
> + struct kmb_chroma_dns_params chroma_dns;
> + struct kmb_color_comb_params color_comb;
> + struct kmb_hdr_params hdr;
> + struct kmb_lut_params lut;
> + struct kmb_tnf_params tnf;
> + struct kmb_dehaze_params dehaze;
> + struct kmb_warp_params warp;
> +} __packed;
> +
> +/**
> + * struct kmb_isp_stats_flags - Bits to indicate which stats need to be updated
> + *
> + * @ae_awb: 1 = updated, 0 = not updated.
> + * @af: 1 = updated, 0 = not updated.
> + * @luma_hist: 1 = updated, 0 = not updated.
> + * @rgb_hist: 1 = updated, 0 = not updated.
> + * @flicker_rows: 1 = updated, 0 = not updated.
> + * @dehaze: 1 = updated, 0 = not updated.
> + * @reserved: reserved for future use and for alignment
> + */
> +struct kmb_isp_stats_flags {
> + __u32 ae_awb:1;
> + __u32 af:1;
> + __u32 luma_hist:1;
> + __u32 rgb_hist:1;
> + __u32 flicker_rows:1;
> + __u32 dehaze:1;
> + __u32 reserved:26;
> +} __packed;
> +
> +/**
> + * struct kmb_isp_stats - KMB ISP raw statistics
> + *
> + * @exposure: Array with exposure statistics blocks. When HDR mode is not
> + * enable only statistics with index 0 are valid. Included stats:
> + * @exposure.ae_awb_stats: Raw AE/AWB statistics.
> + * @exposure.af_stats: Raw AF statistics.
> + * @exposure.hist_luma: Luma histogram.
> + * @exposure.hist_rgb: RGB histogram.
> + * @exposure.flicker_rows: Flicker detection rows.
> + * @dehaze: Dehaze statistics it is collected after HDR Fusion in HDR case.
> + * @update: Select which stats to update, see kmb_vpu_isp_stats_flags
> + */
> +struct kmb_isp_stats {
> + struct {
> + __u8 ae_awb_stats[KMB_CAM_AE_AWB_STATS_SIZE];
> + __u8 af_stats[KMB_CAM_AF_STATS_SIZE];
> + __u8 hist_luma[KMB_CAM_HIST_LUMA_SIZE];
> + __u8 hist_rgb[KMB_CAM_HIST_RGB_SIZE];
> + __u8 flicker_rows[KMB_CAM_FLICKER_ROWS_SIZE];
> + } exposure[KMB_CAM_MAX_EXPOSURES];
> + __u8 dehaze[MAX_DHZ_AIRLIGHT_STATS_SIZE];
> + struct kmb_isp_stats_flags update;
> +} __packed;
> +
> +#endif /* KEEMBAY_ISP_CTL_H */
--
Kind regards,
Sakari Ailus
^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 05/10] media: v4l: Add Keem Bay Camera meta buffer formats
2021-03-19 18:06 [PATCH 00/10] Keem Bay Camera Subsystem Martina Krasteva
` (3 preceding siblings ...)
2021-03-19 18:06 ` [PATCH 04/10] uapi: Keem Bay ISP Parameters data types Martina Krasteva
@ 2021-03-19 18:06 ` Martina Krasteva
2021-03-22 18:27 ` Sakari Ailus
2021-03-19 18:06 ` [PATCH 06/10] media: Keem Bay Camera: Add ISP sub-device Martina Krasteva
` (5 subsequent siblings)
10 siblings, 1 reply; 28+ messages in thread
From: Martina Krasteva @ 2021-03-19 18:06 UTC (permalink / raw)
To: linux-media
Cc: mchehab, robh+dt, devicetree, sakari.ailus,
daniele.alessandrelli, paul.j.murphy, gjorgjix.rosikopulos,
martinax.krasteva
From: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
Add Keem Bay Camera specific meta formats for processing
parameters and statistics:
V4L2_META_FMT_KMB_PARAMS
V4L2_META_FMT_KMB_STATS
Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
Co-developed-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
Signed-off-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
Co-developed-by: Martina Krasteva <martinax.krasteva@intel.com>
Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
---
.../userspace-api/media/v4l/meta-formats.rst | 1 +
.../media/v4l/pixfmt-meta-intel-kmb.rst | 98 ++++++++++++++++++++++
MAINTAINERS | 2 +
include/uapi/linux/videodev2.h | 4 +
4 files changed, 105 insertions(+)
create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst
diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst
index fff25357fe86..cb85161dc1ae 100644
--- a/Documentation/userspace-api/media/v4l/meta-formats.rst
+++ b/Documentation/userspace-api/media/v4l/meta-formats.rst
@@ -14,6 +14,7 @@ These formats are used for the :ref:`metadata` interface only.
pixfmt-meta-d4xx
pixfmt-meta-intel-ipu3
+ pixfmt-meta-intel-kmb
pixfmt-meta-rkisp1
pixfmt-meta-uvc
pixfmt-meta-vsp1-hgo
diff --git a/Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst b/Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst
new file mode 100644
index 000000000000..99615bbed106
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst
@@ -0,0 +1,98 @@
+.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
+
+.. _v4l2-meta-fmt-params:
+.. _v4l2-meta-fmt-stats:
+
+*******************************************************************
+V4L2_META_FMT_KMB_PARAMS ('kmbp'), V4L2_META_FMT_KMB_STATS ('kmbs')
+*******************************************************************
+
+.. kmb_isp_stats
+
+ISP statistics
+==============
+
+The Keembay ISP statistics blocks collect different statistics over
+an input Bayer frame in non-HDR mode, or up to three input Bayer frames
+in HDR mode. Those statistics are obtained from the "keembay-metadata-stats"
+metadata capture video node, using the :c:type:`v4l2_meta_format` interface.
+They are formatted as described by the :c:type:`kmb_isp_stats` structure.
+
+The statistics collected are AE/AWB (Auto-exposure/Auto-white balance),
+AF (Auto-focus) filter response, luma histogram, rgb histograms and dehaze statistics.
+Dehaze statistic are collected after HDR fusion in HDR mode.
+
+The struct :c:type:`kmb_isp_params` contain all configurable parameters for the
+statistics:
+
+- The struct :c:type:`kmb_raw_params` contain enable flags for all
+ statistics except dehaze (always enabled) and configuration for flicker rows
+ statistics.
+- The struct :c:type:`kmb_ae_awb_params` contain configuration parameters for AE/AWB
+ statistics.
+- The struct :c:type:`kmb_af_params` contain configuration for AF (Auto-focus) filter
+ response statistics.
+- The struct :c:type:`kmb_hist_params` contain configuration for luma and rgb histograms.
+- The struct :c:type:`kmb_hist_params` contain configuration for luma and rgb histograms.
+- The struct :c:type:`kmb_dehaze_params` contain configuration for dehaze statistics.
+
+.. code-block:: c
+
+ struct kmb_isp_stats {
+ struct {
+ __u8 ae_awb_stats[KMB_CAM_AE_AWB_STATS_SIZE];
+ __u8 af_stats[KMB_CAM_AF_STATS_SIZE];
+ __u8 hist_luma[KMB_CAM_HIST_LUMA_SIZE];
+ __u8 hist_rgb[KMB_CAM_HIST_RGB_SIZE];
+ __u8 flicker_rows[KMB_CAM_FLICKER_ROWS_SIZE];
+ } exposure[KMB_CAM_MAX_EXPOSURES];
+ __u8 dehaze[MAX_DHZ_AIRLIGHT_STATS_SIZE];
+ struct kmb_isp_stats_flags update;
+ };
+
+.. kmb_isp_stats
+
+ISP parameters
+==============
+
+The ISP parameters are passed to the "keembay-metadata-params" metadata
+output video node, using the :c:type:`v4l2_meta_format` interface. They are
+formatted as described by the :c:type:`kmb_isp_params` structure.
+
+Both ISP statistics and ISP parameters described here are closely tied to
+the underlying camera sub-system (VPU Camera) APIs. They are usually consumed
+and produced by dedicated user space libraries that comprise the important
+tuning tools, thus freeing the developers from being bothered with the low
+level hardware and algorithm details.
+
+.. code-block:: c
+
+ struct kmb_isp_params {
+ struct kmb_isp_params_flags update;
+ struct kmb_blc_params blc[KMB_CAM_MAX_EXPOSURES];
+ struct kmb_sigma_dns_params sigma_dns[KMB_CAM_MAX_EXPOSURES];
+ struct kmb_lsc_params lsc;
+ struct kmb_raw_params raw;
+ struct kmb_ae_awb_params ae_awb;
+ struct kmb_af_params af;
+ struct kmb_hist_params histogram;
+ struct kmb_lca_params lca;
+ struct kmb_debayer_params debayer;
+ struct kmb_dog_dns_params dog_dns;
+ struct kmb_luma_dns_params luma_dns;
+ struct kmb_sharpen_params sharpen;
+ struct kmb_chroma_gen_params chroma_gen;
+ struct kmb_median_params median;
+ struct kmb_chroma_dns_params chroma_dns;
+ struct kmb_color_comb_params color_comb;
+ struct kmb_hdr_params hdr;
+ struct kmb_lut_params lut;
+ struct kmb_tnf_params tnf;
+ struct kmb_dehaze_params dehaze;
+ struct kmb_warp_params warp;
+ };
+
+Keembay ISP uAPI data types
+===============================
+
+.. kernel-doc:: include/uapi/linux/keembay-isp-ctl.h
diff --git a/MAINTAINERS b/MAINTAINERS
index 955f9f6a195d..d90eaf453012 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1972,6 +1972,8 @@ L: linux-media@vger.kernel.org
S: Maintained
T: git git://linuxtv.org/media_tree.git
F: Documentation/devicetree/bindings/media/intel,keembay-camera.yaml
+F: Documentation/media/uapi/v4l/meta-formats.rst
+F: Documentation/media/uapi/v4l/pixfmt-meta-intel-kmb.rst
F: drivers/media/platform/keembay-camera/
F: include/uapi/linux/keembay-isp-ctl.h
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 79dbde3bcf8d..0d32269638f6 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -769,6 +769,10 @@ struct v4l2_pix_format {
#define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
#define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
+/* Vendor specific - used for Keem Bay camera sub-system */
+#define V4L2_META_FMT_KMB_PARAMS v4l2_fourcc('K', 'M', 'B', 'P') /* Keem Bay parameters */
+#define V4L2_META_FMT_KMB_STATS v4l2_fourcc('K', 'M', 'B', 'S') /* Keem Bay statistics */
+
/* priv field value to indicates that subsequent fields are valid. */
#define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread* Re: [PATCH 05/10] media: v4l: Add Keem Bay Camera meta buffer formats
2021-03-19 18:06 ` [PATCH 05/10] media: v4l: Add Keem Bay Camera meta buffer formats Martina Krasteva
@ 2021-03-22 18:27 ` Sakari Ailus
2021-03-24 17:20 ` Rosikopulos, GjorgjiX
2021-03-24 17:23 ` Rosikopulos, GjorgjiX
0 siblings, 2 replies; 28+ messages in thread
From: Sakari Ailus @ 2021-03-22 18:27 UTC (permalink / raw)
To: Martina Krasteva
Cc: linux-media, mchehab, robh+dt, devicetree, daniele.alessandrelli,
paul.j.murphy, gjorgjix.rosikopulos
Hi Martian and Gjorgji,
On Fri, Mar 19, 2021 at 06:06:27PM +0000, Martina Krasteva wrote:
> From: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
>
> Add Keem Bay Camera specific meta formats for processing
> parameters and statistics:
>
> V4L2_META_FMT_KMB_PARAMS
> V4L2_META_FMT_KMB_STATS
>
> Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
> Co-developed-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
> Signed-off-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
> Co-developed-by: Martina Krasteva <martinax.krasteva@intel.com>
> Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
> Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
> Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
> ---
> .../userspace-api/media/v4l/meta-formats.rst | 1 +
> .../media/v4l/pixfmt-meta-intel-kmb.rst | 98 ++++++++++++++++++++++
> MAINTAINERS | 2 +
> include/uapi/linux/videodev2.h | 4 +
> 4 files changed, 105 insertions(+)
> create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst
>
> diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst
> index fff25357fe86..cb85161dc1ae 100644
> --- a/Documentation/userspace-api/media/v4l/meta-formats.rst
> +++ b/Documentation/userspace-api/media/v4l/meta-formats.rst
> @@ -14,6 +14,7 @@ These formats are used for the :ref:`metadata` interface only.
>
> pixfmt-meta-d4xx
> pixfmt-meta-intel-ipu3
> + pixfmt-meta-intel-kmb
> pixfmt-meta-rkisp1
> pixfmt-meta-uvc
> pixfmt-meta-vsp1-hgo
> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst b/Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst
> new file mode 100644
> index 000000000000..99615bbed106
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst
> @@ -0,0 +1,98 @@
> +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
> +
> +.. _v4l2-meta-fmt-params:
> +.. _v4l2-meta-fmt-stats:
> +
> +*******************************************************************
> +V4L2_META_FMT_KMB_PARAMS ('kmbp'), V4L2_META_FMT_KMB_STATS ('kmbs')
> +*******************************************************************
> +
> +.. kmb_isp_stats
> +
> +ISP statistics
> +==============
> +
> +The Keembay ISP statistics blocks collect different statistics over
> +an input Bayer frame in non-HDR mode, or up to three input Bayer frames
> +in HDR mode. Those statistics are obtained from the "keembay-metadata-stats"
> +metadata capture video node, using the :c:type:`v4l2_meta_format` interface.
> +They are formatted as described by the :c:type:`kmb_isp_stats` structure.
> +
> +The statistics collected are AE/AWB (Auto-exposure/Auto-white balance),
> +AF (Auto-focus) filter response, luma histogram, rgb histograms and dehaze statistics.
> +Dehaze statistic are collected after HDR fusion in HDR mode.
> +
> +The struct :c:type:`kmb_isp_params` contain all configurable parameters for the
The syntax has changed recently regarding references to structs, which now
are simply "struct nameofthestruct".
> +statistics:
> +
> +- The struct :c:type:`kmb_raw_params` contain enable flags for all
> + statistics except dehaze (always enabled) and configuration for flicker rows
> + statistics.
> +- The struct :c:type:`kmb_ae_awb_params` contain configuration parameters for AE/AWB
> + statistics.
> +- The struct :c:type:`kmb_af_params` contain configuration for AF (Auto-focus) filter
> + response statistics.
> +- The struct :c:type:`kmb_hist_params` contain configuration for luma and rgb histograms.
> +- The struct :c:type:`kmb_hist_params` contain configuration for luma and rgb histograms.
> +- The struct :c:type:`kmb_dehaze_params` contain configuration for dehaze statistics.
Please wrap before 80 characters per line unless there's a reason to do
otherwise.
> +
> +.. code-block:: c
> +
> + struct kmb_isp_stats {
> + struct {
> + __u8 ae_awb_stats[KMB_CAM_AE_AWB_STATS_SIZE];
> + __u8 af_stats[KMB_CAM_AF_STATS_SIZE];
> + __u8 hist_luma[KMB_CAM_HIST_LUMA_SIZE];
> + __u8 hist_rgb[KMB_CAM_HIST_RGB_SIZE];
> + __u8 flicker_rows[KMB_CAM_FLICKER_ROWS_SIZE];
> + } exposure[KMB_CAM_MAX_EXPOSURES];
> + __u8 dehaze[MAX_DHZ_AIRLIGHT_STATS_SIZE];
> + struct kmb_isp_stats_flags update;
> + };
> +
> +.. kmb_isp_stats
> +
> +ISP parameters
> +==============
> +
> +The ISP parameters are passed to the "keembay-metadata-params" metadata
> +output video node, using the :c:type:`v4l2_meta_format` interface. They are
> +formatted as described by the :c:type:`kmb_isp_params` structure.
> +
> +Both ISP statistics and ISP parameters described here are closely tied to
> +the underlying camera sub-system (VPU Camera) APIs. They are usually consumed
> +and produced by dedicated user space libraries that comprise the important
> +tuning tools, thus freeing the developers from being bothered with the low
> +level hardware and algorithm details.
> +
> +.. code-block:: c
> +
> + struct kmb_isp_params {
> + struct kmb_isp_params_flags update;
> + struct kmb_blc_params blc[KMB_CAM_MAX_EXPOSURES];
> + struct kmb_sigma_dns_params sigma_dns[KMB_CAM_MAX_EXPOSURES];
> + struct kmb_lsc_params lsc;
> + struct kmb_raw_params raw;
> + struct kmb_ae_awb_params ae_awb;
> + struct kmb_af_params af;
> + struct kmb_hist_params histogram;
> + struct kmb_lca_params lca;
> + struct kmb_debayer_params debayer;
> + struct kmb_dog_dns_params dog_dns;
> + struct kmb_luma_dns_params luma_dns;
> + struct kmb_sharpen_params sharpen;
> + struct kmb_chroma_gen_params chroma_gen;
> + struct kmb_median_params median;
> + struct kmb_chroma_dns_params chroma_dns;
> + struct kmb_color_comb_params color_comb;
> + struct kmb_hdr_params hdr;
> + struct kmb_lut_params lut;
> + struct kmb_tnf_params tnf;
> + struct kmb_dehaze_params dehaze;
> + struct kmb_warp_params warp;
> + };
As this is already part of the UAPI header you don't need to repeat it
here.
> +
> +Keembay ISP uAPI data types
> +===============================
> +
> +.. kernel-doc:: include/uapi/linux/keembay-isp-ctl.h
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 955f9f6a195d..d90eaf453012 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1972,6 +1972,8 @@ L: linux-media@vger.kernel.org
> S: Maintained
> T: git git://linuxtv.org/media_tree.git
> F: Documentation/devicetree/bindings/media/intel,keembay-camera.yaml
> +F: Documentation/media/uapi/v4l/meta-formats.rst
> +F: Documentation/media/uapi/v4l/pixfmt-meta-intel-kmb.rst
The files are under Documentation/userspace-api/media/v4l/ .
> F: drivers/media/platform/keembay-camera/
> F: include/uapi/linux/keembay-isp-ctl.h
>
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 79dbde3bcf8d..0d32269638f6 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -769,6 +769,10 @@ struct v4l2_pix_format {
> #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
> #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
>
> +/* Vendor specific - used for Keem Bay camera sub-system */
> +#define V4L2_META_FMT_KMB_PARAMS v4l2_fourcc('K', 'M', 'B', 'P') /* Keem Bay parameters */
> +#define V4L2_META_FMT_KMB_STATS v4l2_fourcc('K', 'M', 'B', 'S') /* Keem Bay statistics */
> +
> /* priv field value to indicates that subsequent fields are valid. */
> #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe
>
--
Kind regards,
Sakari Ailus
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH 05/10] media: v4l: Add Keem Bay Camera meta buffer formats
2021-03-22 18:27 ` Sakari Ailus
@ 2021-03-24 17:20 ` Rosikopulos, GjorgjiX
2021-03-24 17:23 ` Rosikopulos, GjorgjiX
1 sibling, 0 replies; 28+ messages in thread
From: Rosikopulos, GjorgjiX @ 2021-03-24 17:20 UTC (permalink / raw)
To: Sakari Ailus, Martina Krasteva
Cc: linux-media, mchehab, robh+dt, devicetree, daniele.alessandrelli,
paul.j.murphy
Hi Sakari,
Thank you for the review
On 22/03/2021 18:27, Sakari Ailus wrote:
> Hi Martian and Gjorgji,
>
> On Fri, Mar 19, 2021 at 06:06:27PM +0000, Martina Krasteva wrote:
>> From: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
>>
>> Add Keem Bay Camera specific meta formats for processing
>> parameters and statistics:
>>
>> V4L2_META_FMT_KMB_PARAMS
>> V4L2_META_FMT_KMB_STATS
>>
>> Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
>> Co-developed-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
>> Signed-off-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
>> Co-developed-by: Martina Krasteva <martinax.krasteva@intel.com>
>> Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
>> Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
>> Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
>> ---
>> .../userspace-api/media/v4l/meta-formats.rst | 1 +
>> .../media/v4l/pixfmt-meta-intel-kmb.rst | 98 ++++++++++++++++++++++
>> MAINTAINERS | 2 +
>> include/uapi/linux/videodev2.h | 4 +
>> 4 files changed, 105 insertions(+)
>> create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst
>>
>> diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst
>> index fff25357fe86..cb85161dc1ae 100644
>> --- a/Documentation/userspace-api/media/v4l/meta-formats.rst
>> +++ b/Documentation/userspace-api/media/v4l/meta-formats.rst
>> @@ -14,6 +14,7 @@ These formats are used for the :ref:`metadata` interface only.
>>
>> pixfmt-meta-d4xx
>> pixfmt-meta-intel-ipu3
>> + pixfmt-meta-intel-kmb
>> pixfmt-meta-rkisp1
>> pixfmt-meta-uvc
>> pixfmt-meta-vsp1-hgo
>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst b/Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst
>> new file mode 100644
>> index 000000000000..99615bbed106
>> --- /dev/null
>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst
>> @@ -0,0 +1,98 @@
>> +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
>> +
>> +.. _v4l2-meta-fmt-params:
>> +.. _v4l2-meta-fmt-stats:
>> +
>> +*******************************************************************
>> +V4L2_META_FMT_KMB_PARAMS ('kmbp'), V4L2_META_FMT_KMB_STATS ('kmbs')
>> +*******************************************************************
>> +
>> +.. kmb_isp_stats
>> +
>> +ISP statistics
>> +==============
>> +
>> +The Keembay ISP statistics blocks collect different statistics over
>> +an input Bayer frame in non-HDR mode, or up to three input Bayer frames
>> +in HDR mode. Those statistics are obtained from the "keembay-metadata-stats"
>> +metadata capture video node, using the :c:type:`v4l2_meta_format` interface.
>> +They are formatted as described by the :c:type:`kmb_isp_stats` structure.
>> +
>> +The statistics collected are AE/AWB (Auto-exposure/Auto-white balance),
>> +AF (Auto-focus) filter response, luma histogram, rgb histograms and dehaze statistics.
>> +Dehaze statistic are collected after HDR fusion in HDR mode.
>> +
>> +The struct :c:type:`kmb_isp_params` contain all configurable parameters for the
> The syntax has changed recently regarding references to structs, which now
> are simply "struct nameofthestruct".
Thanks i have missed that, it will be fixed in next patchset.
>
>> +statistics:
>> +
>> +- The struct :c:type:`kmb_raw_params` contain enable flags for all
>> + statistics except dehaze (always enabled) and configuration for flicker rows
>> + statistics.
>> +- The struct :c:type:`kmb_ae_awb_params` contain configuration parameters for AE/AWB
>> + statistics.
>> +- The struct :c:type:`kmb_af_params` contain configuration for AF (Auto-focus) filter
>> + response statistics.
>> +- The struct :c:type:`kmb_hist_params` contain configuration for luma and rgb histograms.
>> +- The struct :c:type:`kmb_hist_params` contain configuration for luma and rgb histograms.
>> +- The struct :c:type:`kmb_dehaze_params` contain configuration for dehaze statistics.
> Please wrap before 80 characters per line unless there's a reason to do
> otherwise.
Ok
>
>> +
>> +.. code-block:: c
>> +
>> + struct kmb_isp_stats {
>> + struct {
>> + __u8 ae_awb_stats[KMB_CAM_AE_AWB_STATS_SIZE];
>> + __u8 af_stats[KMB_CAM_AF_STATS_SIZE];
>> + __u8 hist_luma[KMB_CAM_HIST_LUMA_SIZE];
>> + __u8 hist_rgb[KMB_CAM_HIST_RGB_SIZE];
>> + __u8 flicker_rows[KMB_CAM_FLICKER_ROWS_SIZE];
>> + } exposure[KMB_CAM_MAX_EXPOSURES];
>> + __u8 dehaze[MAX_DHZ_AIRLIGHT_STATS_SIZE];
>> + struct kmb_isp_stats_flags update;
>> + };
>> +
>> +.. kmb_isp_stats
>> +
>> +ISP parameters
>> +==============
>> +
>> +The ISP parameters are passed to the "keembay-metadata-params" metadata
>> +output video node, using the :c:type:`v4l2_meta_format` interface. They are
>> +formatted as described by the :c:type:`kmb_isp_params` structure.
>> +
>> +Both ISP statistics and ISP parameters described here are closely tied to
>> +the underlying camera sub-system (VPU Camera) APIs. They are usually consumed
>> +and produced by dedicated user space libraries that comprise the important
>> +tuning tools, thus freeing the developers from being bothered with the low
>> +level hardware and algorithm details.
>> +
>> +.. code-block:: c
>> +
>> + struct kmb_isp_params {
>> + struct kmb_isp_params_flags update;
>> + struct kmb_blc_params blc[KMB_CAM_MAX_EXPOSURES];
>> + struct kmb_sigma_dns_params sigma_dns[KMB_CAM_MAX_EXPOSURES];
>> + struct kmb_lsc_params lsc;
>> + struct kmb_raw_params raw;
>> + struct kmb_ae_awb_params ae_awb;
>> + struct kmb_af_params af;
>> + struct kmb_hist_params histogram;
>> + struct kmb_lca_params lca;
>> + struct kmb_debayer_params debayer;
>> + struct kmb_dog_dns_params dog_dns;
>> + struct kmb_luma_dns_params luma_dns;
>> + struct kmb_sharpen_params sharpen;
>> + struct kmb_chroma_gen_params chroma_gen;
>> + struct kmb_median_params median;
>> + struct kmb_chroma_dns_params chroma_dns;
>> + struct kmb_color_comb_params color_comb;
>> + struct kmb_hdr_params hdr;
>> + struct kmb_lut_params lut;
>> + struct kmb_tnf_params tnf;
>> + struct kmb_dehaze_params dehaze;
>> + struct kmb_warp_params warp;
>> + };
> As this is already part of the UAPI header you don't need to repeat it
> here.
I Agree it will be removed in next patchset.
>
>> +
>> +Keembay ISP uAPI data types
>> +===============================
>> +
>> +.. kernel-doc:: include/uapi/linux/keembay-isp-ctl.h
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 955f9f6a195d..d90eaf453012 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -1972,6 +1972,8 @@ L: linux-media@vger.kernel.org
>> S: Maintained
>> T: git git://linuxtv.org/media_tree.git
>> F: Documentation/devicetree/bindings/media/intel,keembay-camera.yaml
>> +F: Documentation/media/uapi/v4l/meta-formats.rst
>> +F: Documentation/media/uapi/v4l/pixfmt-meta-intel-kmb.rst
> The files are under Documentation/userspace-api/media/v4l/ .
Thanks it will be fixed.
>
>> F: drivers/media/platform/keembay-camera/
>> F: include/uapi/linux/keembay-isp-ctl.h
>>
>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>> index 79dbde3bcf8d..0d32269638f6 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -769,6 +769,10 @@ struct v4l2_pix_format {
>> #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
>> #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
>>
>> +/* Vendor specific - used for Keem Bay camera sub-system */
>> +#define V4L2_META_FMT_KMB_PARAMS v4l2_fourcc('K', 'M', 'B', 'P') /* Keem Bay parameters */
>> +#define V4L2_META_FMT_KMB_STATS v4l2_fourcc('K', 'M', 'B', 'S') /* Keem Bay statistics */
>> +
>> /* priv field value to indicates that subsequent fields are valid. */
>> #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe
>>
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH 05/10] media: v4l: Add Keem Bay Camera meta buffer formats
2021-03-22 18:27 ` Sakari Ailus
2021-03-24 17:20 ` Rosikopulos, GjorgjiX
@ 2021-03-24 17:23 ` Rosikopulos, GjorgjiX
1 sibling, 0 replies; 28+ messages in thread
From: Rosikopulos, GjorgjiX @ 2021-03-24 17:23 UTC (permalink / raw)
To: Sakari Ailus, Martina Krasteva
Cc: linux-media, mchehab, robh+dt, devicetree, daniele.alessandrelli,
paul.j.murphy
Hi Sakari,
Thank you for the review,
On 22/03/2021 18:27, Sakari Ailus wrote:
> Hi Martian and Gjorgji,
>
> On Fri, Mar 19, 2021 at 06:06:27PM +0000, Martina Krasteva wrote:
>> From: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
>>
>> Add Keem Bay Camera specific meta formats for processing
>> parameters and statistics:
>>
>> V4L2_META_FMT_KMB_PARAMS
>> V4L2_META_FMT_KMB_STATS
>>
>> Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
>> Co-developed-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
>> Signed-off-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
>> Co-developed-by: Martina Krasteva <martinax.krasteva@intel.com>
>> Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
>> Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
>> Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
>> ---
>> .../userspace-api/media/v4l/meta-formats.rst | 1 +
>> .../media/v4l/pixfmt-meta-intel-kmb.rst | 98 ++++++++++++++++++++++
>> MAINTAINERS | 2 +
>> include/uapi/linux/videodev2.h | 4 +
>> 4 files changed, 105 insertions(+)
>> create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst
>>
>> diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst
>> index fff25357fe86..cb85161dc1ae 100644
>> --- a/Documentation/userspace-api/media/v4l/meta-formats.rst
>> +++ b/Documentation/userspace-api/media/v4l/meta-formats.rst
>> @@ -14,6 +14,7 @@ These formats are used for the :ref:`metadata` interface only.
>>
>> pixfmt-meta-d4xx
>> pixfmt-meta-intel-ipu3
>> + pixfmt-meta-intel-kmb
>> pixfmt-meta-rkisp1
>> pixfmt-meta-uvc
>> pixfmt-meta-vsp1-hgo
>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst b/Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst
>> new file mode 100644
>> index 000000000000..99615bbed106
>> --- /dev/null
>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst
>> @@ -0,0 +1,98 @@
>> +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later
>> +
>> +.. _v4l2-meta-fmt-params:
>> +.. _v4l2-meta-fmt-stats:
>> +
>> +*******************************************************************
>> +V4L2_META_FMT_KMB_PARAMS ('kmbp'), V4L2_META_FMT_KMB_STATS ('kmbs')
>> +*******************************************************************
>> +
>> +.. kmb_isp_stats
>> +
>> +ISP statistics
>> +==============
>> +
>> +The Keembay ISP statistics blocks collect different statistics over
>> +an input Bayer frame in non-HDR mode, or up to three input Bayer frames
>> +in HDR mode. Those statistics are obtained from the "keembay-metadata-stats"
>> +metadata capture video node, using the :c:type:`v4l2_meta_format` interface.
>> +They are formatted as described by the :c:type:`kmb_isp_stats` structure.
>> +
>> +The statistics collected are AE/AWB (Auto-exposure/Auto-white balance),
>> +AF (Auto-focus) filter response, luma histogram, rgb histograms and dehaze statistics.
>> +Dehaze statistic are collected after HDR fusion in HDR mode.
>> +
>> +The struct :c:type:`kmb_isp_params` contain all configurable parameters for the
> The syntax has changed recently regarding references to structs, which now
> are simply "struct nameofthestruct".
Thanks i have missed that, it will be fixed in next patchset.
>
>> +statistics:
>> +
>> +- The struct :c:type:`kmb_raw_params` contain enable flags for all
>> + statistics except dehaze (always enabled) and configuration for flicker rows
>> + statistics.
>> +- The struct :c:type:`kmb_ae_awb_params` contain configuration parameters for AE/AWB
>> + statistics.
>> +- The struct :c:type:`kmb_af_params` contain configuration for AF (Auto-focus) filter
>> + response statistics.
>> +- The struct :c:type:`kmb_hist_params` contain configuration for luma and rgb histograms.
>> +- The struct :c:type:`kmb_hist_params` contain configuration for luma and rgb histograms.
>> +- The struct :c:type:`kmb_dehaze_params` contain configuration for dehaze statistics.
> Please wrap before 80 characters per line unless there's a reason to do
> otherwise.
Ok
>
>> +
>> +.. code-block:: c
>> +
>> + struct kmb_isp_stats {
>> + struct {
>> + __u8 ae_awb_stats[KMB_CAM_AE_AWB_STATS_SIZE];
>> + __u8 af_stats[KMB_CAM_AF_STATS_SIZE];
>> + __u8 hist_luma[KMB_CAM_HIST_LUMA_SIZE];
>> + __u8 hist_rgb[KMB_CAM_HIST_RGB_SIZE];
>> + __u8 flicker_rows[KMB_CAM_FLICKER_ROWS_SIZE];
>> + } exposure[KMB_CAM_MAX_EXPOSURES];
>> + __u8 dehaze[MAX_DHZ_AIRLIGHT_STATS_SIZE];
>> + struct kmb_isp_stats_flags update;
>> + };
>> +
>> +.. kmb_isp_stats
>> +
>> +ISP parameters
>> +==============
>> +
>> +The ISP parameters are passed to the "keembay-metadata-params" metadata
>> +output video node, using the :c:type:`v4l2_meta_format` interface. They are
>> +formatted as described by the :c:type:`kmb_isp_params` structure.
>> +
>> +Both ISP statistics and ISP parameters described here are closely tied to
>> +the underlying camera sub-system (VPU Camera) APIs. They are usually consumed
>> +and produced by dedicated user space libraries that comprise the important
>> +tuning tools, thus freeing the developers from being bothered with the low
>> +level hardware and algorithm details.
>> +
>> +.. code-block:: c
>> +
>> + struct kmb_isp_params {
>> + struct kmb_isp_params_flags update;
>> + struct kmb_blc_params blc[KMB_CAM_MAX_EXPOSURES];
>> + struct kmb_sigma_dns_params sigma_dns[KMB_CAM_MAX_EXPOSURES];
>> + struct kmb_lsc_params lsc;
>> + struct kmb_raw_params raw;
>> + struct kmb_ae_awb_params ae_awb;
>> + struct kmb_af_params af;
>> + struct kmb_hist_params histogram;
>> + struct kmb_lca_params lca;
>> + struct kmb_debayer_params debayer;
>> + struct kmb_dog_dns_params dog_dns;
>> + struct kmb_luma_dns_params luma_dns;
>> + struct kmb_sharpen_params sharpen;
>> + struct kmb_chroma_gen_params chroma_gen;
>> + struct kmb_median_params median;
>> + struct kmb_chroma_dns_params chroma_dns;
>> + struct kmb_color_comb_params color_comb;
>> + struct kmb_hdr_params hdr;
>> + struct kmb_lut_params lut;
>> + struct kmb_tnf_params tnf;
>> + struct kmb_dehaze_params dehaze;
>> + struct kmb_warp_params warp;
>> + };
> As this is already part of the UAPI header you don't need to repeat it
> here.
I Agree it will be removed in next patchset.
>
>> +
>> +Keembay ISP uAPI data types
>> +===============================
>> +
>> +.. kernel-doc:: include/uapi/linux/keembay-isp-ctl.h
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 955f9f6a195d..d90eaf453012 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -1972,6 +1972,8 @@ L: linux-media@vger.kernel.org
>> S: Maintained
>> T: git git://linuxtv.org/media_tree.git
>> F: Documentation/devicetree/bindings/media/intel,keembay-camera.yaml
>> +F: Documentation/media/uapi/v4l/meta-formats.rst
>> +F: Documentation/media/uapi/v4l/pixfmt-meta-intel-kmb.rst
> The files are under Documentation/userspace-api/media/v4l/ .
Thanks it will be fixed.
>
>> F: drivers/media/platform/keembay-camera/
>> F: include/uapi/linux/keembay-isp-ctl.h
>>
>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>> index 79dbde3bcf8d..0d32269638f6 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -769,6 +769,10 @@ struct v4l2_pix_format {
>> #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
>> #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
>>
>> +/* Vendor specific - used for Keem Bay camera sub-system */
>> +#define V4L2_META_FMT_KMB_PARAMS v4l2_fourcc('K', 'M', 'B', 'P') /* Keem Bay parameters */
>> +#define V4L2_META_FMT_KMB_STATS v4l2_fourcc('K', 'M', 'B', 'S') /* Keem Bay statistics */
>> +
>> /* priv field value to indicates that subsequent fields are valid. */
>> #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe
>>
^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 06/10] media: Keem Bay Camera: Add ISP sub-device
2021-03-19 18:06 [PATCH 00/10] Keem Bay Camera Subsystem Martina Krasteva
` (4 preceding siblings ...)
2021-03-19 18:06 ` [PATCH 05/10] media: v4l: Add Keem Bay Camera meta buffer formats Martina Krasteva
@ 2021-03-19 18:06 ` Martina Krasteva
2021-04-09 8:31 ` Sakari Ailus
2021-03-19 18:06 ` [PATCH 07/10] media: Keem Bay Camera: Add pipeline support Martina Krasteva
` (4 subsequent siblings)
10 siblings, 1 reply; 28+ messages in thread
From: Martina Krasteva @ 2021-03-19 18:06 UTC (permalink / raw)
To: linux-media
Cc: mchehab, robh+dt, devicetree, sakari.ailus,
daniele.alessandrelli, paul.j.murphy, gjorgjix.rosikopulos,
martinax.krasteva
From: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
ISP sub-device has two sink pads and two source pads.
Sink pads are connected to sensor and metadata params output video node.
Source pads are connected to capture video node and capture metadata.
All links are immutable active and are required for streaming session.
Isp sub-device implements v4l2 event interface, each event
received on VPU ISP channel is also sent as event to userspace.
ISP sub-device is managing ISP configuration buffers, the reason
for that is they are sent through XLink ISP channel.
ISP sub-device is also owner of VPU pipeline. It is responsible
for the pipeline configuration, which is executed on set_fmt pad op
called on sensor source pad.
The source is started/stopped on s_stream video operation.
Build/delete pipeline are executed on video nodes
streaming operations. When all video nodes with active linked
are in stream on state the pipeline is started.
Isp sub-device is using metadata output video nodes for parameters
and filling the statistics to metadata capture video node.
The statistics can be matched with params using buffer sequence number.
V4L2_EVENT_FRAME_SYNC is also support. frame_sequence is matching with
metadata video buffer sequence number.
Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
Co-developed-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
Signed-off-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
Co-developed-by: Martina Krasteva <martinax.krasteva@intel.com>
Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
---
drivers/media/platform/keembay-camera/Makefile | 4 +-
.../platform/keembay-camera/keembay-cam-xlink.c | 140 ++
.../platform/keembay-camera/keembay-cam-xlink.h | 11 +
.../media/platform/keembay-camera/keembay-isp.c | 1348 +++++++++++++++++++-
.../media/platform/keembay-camera/keembay-isp.h | 66 +-
.../platform/keembay-camera/keembay-metadata.c | 45 +
.../platform/keembay-camera/keembay-metadata.h | 150 +++
.../platform/keembay-camera/keembay-pipeline.c | 76 ++
.../platform/keembay-camera/keembay-pipeline.h | 71 ++
.../media/platform/keembay-camera/keembay-video.c | 46 +
.../media/platform/keembay-camera/keembay-video.h | 85 ++
11 files changed, 2029 insertions(+), 13 deletions(-)
create mode 100644 drivers/media/platform/keembay-camera/keembay-metadata.c
create mode 100644 drivers/media/platform/keembay-camera/keembay-metadata.h
create mode 100644 drivers/media/platform/keembay-camera/keembay-pipeline.c
create mode 100644 drivers/media/platform/keembay-camera/keembay-pipeline.h
create mode 100644 drivers/media/platform/keembay-camera/keembay-video.c
create mode 100644 drivers/media/platform/keembay-camera/keembay-video.h
diff --git a/drivers/media/platform/keembay-camera/Makefile b/drivers/media/platform/keembay-camera/Makefile
index ea19b61e6469..8b3ad715c5c4 100644
--- a/drivers/media/platform/keembay-camera/Makefile
+++ b/drivers/media/platform/keembay-camera/Makefile
@@ -1,3 +1,5 @@
-keembay-cam-objs = keembay-camera.o keembay-cam-xlink.o keembay-isp.o
+keembay-cam-objs = keembay-camera.o keembay-pipeline.o \
+ keembay-cam-xlink.o keembay-isp.o \
+ keembay-metadata.o keembay-video.o
obj-$(CONFIG_VIDEO_INTEL_KEEMBAY_CAMERA) += keembay-cam.o
diff --git a/drivers/media/platform/keembay-camera/keembay-cam-xlink.c b/drivers/media/platform/keembay-camera/keembay-cam-xlink.c
index e34516de0ff9..49a0937bc9fc 100644
--- a/drivers/media/platform/keembay-camera/keembay-cam-xlink.c
+++ b/drivers/media/platform/keembay-camera/keembay-cam-xlink.c
@@ -6,8 +6,19 @@
*/
#include <linux/device.h>
#include <linux/idr.h>
+#include <linux/kernel.h>
+#include <linux/xlink.h>
#include "keembay-cam-xlink.h"
+#include "keembay-vpu-cmd.h"
+
+/* Do not change: it is negotiated on both sides */
+#define KMB_CAM_XLINK_CTRL_CHAN_ID 27
+#define KMB_CAM_XLINK_CHAN_ID_BASE 28
+
+/* Control channel */
+#define KMB_CAM_XLINK_CH_MAX_DATA_SIZE 1024
+#define KMB_CAM_XLINK_CH_TIMEOUT_MS 1000
/**
* kmb_cam_xlink_init - Initialize xlink for VPU camera communication
@@ -52,3 +63,132 @@ void kmb_cam_xlink_cleanup(struct kmb_xlink_cam *xlink_cam)
xlink_disconnect(&xlink_cam->handle);
ida_destroy(&xlink_cam->channel_ids);
}
+
+/**
+ * kmb_cam_xlink_alloc_channel - Allocate xlink camera channel id
+ * @xlink_cam: Pointer to xlink camera handle
+ *
+ * Each xlink channel (except main control) should have unieque id
+ *
+ * Return: Channel id, negative error otherwise
+ */
+int kmb_cam_xlink_alloc_channel(struct kmb_xlink_cam *xlink_cam)
+{
+ int chan_id;
+
+ chan_id = ida_alloc_range(&xlink_cam->channel_ids,
+ KMB_CAM_XLINK_CHAN_ID_BASE,
+ U16_MAX, GFP_KERNEL);
+
+ return chan_id;
+}
+
+/**
+ * kmb_cam_xlink_free_channel - Free xlink camera channel id
+ * @xlink_cam: Pointer to xlink camera handle
+ * @chan_id: XLink channel ID
+ */
+void kmb_cam_xlink_free_channel(struct kmb_xlink_cam *xlink_cam, int chan_id)
+{
+ ida_free(&xlink_cam->channel_ids, chan_id);
+}
+
+/**
+ * kmb_cam_xlink_open_channel - Open xlink channel for communication
+ * @xlink_cam: Pointer to xlink camera handle
+ * @chan_id: Xlink channel id to be opened
+ *
+ * Each xlink channel should be open first, to establish communication channel.
+ *
+ * Return: 0 if successful, error code otherwise
+ */
+int kmb_cam_xlink_open_channel(struct kmb_xlink_cam *xlink_cam, int chan_id)
+{
+ int ret;
+
+ ret = xlink_open_channel(&xlink_cam->handle,
+ chan_id, RXB_TXB,
+ KMB_CAM_XLINK_CH_MAX_DATA_SIZE,
+ KMB_CAM_XLINK_CH_TIMEOUT_MS);
+ if (ret) {
+ dev_err(xlink_cam->dev, "Failed to close xlink channel %d", ret);
+ return -ENODEV;
+ }
+
+ return 0;
+}
+
+/**
+ * kmb_cam_xlink_close_channel - Close xlink channel
+ * @xlink_cam: Pointer to xlink camera handle
+ * @chan_id: Xlink channel id to be closed
+ *
+ * Return: 0 if successful, error code otherwise
+ */
+int kmb_cam_xlink_close_channel(struct kmb_xlink_cam *xlink_cam, int chan_id)
+{
+ int ret;
+
+ ret = xlink_close_channel(&xlink_cam->handle, chan_id);
+ if (ret) {
+ dev_err(xlink_cam->dev, "Failed to close xlink channel %d", ret);
+ return -ENODEV;
+ }
+
+ return 0;
+}
+
+/**
+ * kmb_cam_xlink_write_msg - Write xlink message
+ * @xlink_cam: Pointer to xlink camera handle
+ * @chan_id: Xlink channel id where message should be writted
+ * @message: Pointer to message to be written in to xlink channel
+ * @msg_size: Size of the message
+ *
+ * Return: 0 if successful, error code otherwise
+ */
+int kmb_cam_xlink_write_msg(struct kmb_xlink_cam *xlink_cam, int chan_id,
+ u8 *message, u32 msg_size)
+{
+ int ret;
+
+ if (msg_size > KMB_CAM_XLINK_CH_MAX_DATA_SIZE)
+ return -EINVAL;
+
+ ret = xlink_write_volatile(&xlink_cam->handle, chan_id,
+ message, msg_size);
+ if (ret) {
+ dev_err(xlink_cam->dev, "Fail to write xlink message %d", ret);
+ return -ENODEV;
+ }
+
+ return 0;
+}
+
+/**
+ * kmb_cam_xlink_read_msg - Read xlink message
+ * @xlink_cam: Pointer to xlink camera handle
+ * @chan_id: Xlink channel id from where message will be read
+ * @message: Pointer to data where read message will be placed
+ * @msg_size: Mas size of data to be read
+ *
+ * Return: Actual size read, negative error code otherwise
+ */
+int kmb_cam_xlink_read_msg(struct kmb_xlink_cam *xlink_cam, int chan_id,
+ u8 *message, u32 msg_size)
+{
+ u32 written_size = msg_size;
+ int ret;
+
+ if (msg_size > KMB_CAM_XLINK_CH_MAX_DATA_SIZE)
+ return -EINVAL;
+
+ ret = xlink_read_data_to_buffer(&xlink_cam->handle, chan_id,
+ message, &written_size);
+ if (ret) {
+ dev_err(xlink_cam->dev, "Fail to read xlink message %d", ret);
+ return -ENODEV;
+ }
+
+ return written_size;
+}
diff --git a/drivers/media/platform/keembay-camera/keembay-cam-xlink.h b/drivers/media/platform/keembay-camera/keembay-cam-xlink.h
index d219d4298427..d9a78d847a4b 100644
--- a/drivers/media/platform/keembay-camera/keembay-cam-xlink.h
+++ b/drivers/media/platform/keembay-camera/keembay-cam-xlink.h
@@ -28,4 +28,15 @@ struct kmb_xlink_cam {
int kmb_cam_xlink_init(struct kmb_xlink_cam *xlink_cam, struct device *dev);
void kmb_cam_xlink_cleanup(struct kmb_xlink_cam *xlink_cam);
+int kmb_cam_xlink_alloc_channel(struct kmb_xlink_cam *xlink_cam);
+void kmb_cam_xlink_free_channel(struct kmb_xlink_cam *xlink_cam, int chan_id);
+
+int kmb_cam_xlink_open_channel(struct kmb_xlink_cam *xlink_cam, int chan_id);
+int kmb_cam_xlink_close_channel(struct kmb_xlink_cam *xlink_cam, int chan_id);
+
+int kmb_cam_xlink_write_msg(struct kmb_xlink_cam *xlink_cam, int chan_id,
+ u8 *message, u32 msg_size);
+int kmb_cam_xlink_read_msg(struct kmb_xlink_cam *xlink_cam, int chan_id,
+ u8 *message, u32 msg_size);
+
#endif /* KEEMBAY_CAM_XLINK_H */
diff --git a/drivers/media/platform/keembay-camera/keembay-isp.c b/drivers/media/platform/keembay-camera/keembay-isp.c
index 79f6212e37d9..54e9c70cd46d 100644
--- a/drivers/media/platform/keembay-camera/keembay-isp.c
+++ b/drivers/media/platform/keembay-camera/keembay-isp.c
@@ -4,7 +4,1146 @@
*
* Copyright (C) 2021 Intel Corporation
*/
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/freezer.h>
+#include <linux/keembay-isp-ctl.h>
+#include <linux/kthread.h>
+
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ioctl.h>
+
#include "keembay-isp.h"
+#include "keembay-cam-xlink.h"
+#include "keembay-vpu-cmd.h"
+#include "keembay-vpu-pipe.h"
+#include "keembay-vpu-src.h"
+
+#define KMB_ISP_DRV_NAME "keembay-camera-isp"
+
+/* Xlink channel configuration */
+#define KMB_ISP_CH_DATA_SIZE 1024
+#define KMB_ISP_CH_TIMEOUT_MS 5000
+
+/* Predefined event queue length */
+#define KMB_ISP_EVT_Q_LEN 8
+
+/* Wait timeout for stopping isp source */
+#define KMB_STOP_SOURCE_TIMEOUT_MS 1000
+
+enum kmb_isp_stop_method {
+ KMB_ISP_STOP_SYNC = 0,
+ KMB_ISP_STOP_FORCE = 1,
+};
+
+static const struct kmb_isp_source_format source_fmts[] = {
+ {
+ .code = MEDIA_BUS_FMT_SRGGB8_1X8,
+ .bayer_pattern = KMB_IC_BAYER_FORMAT_RGGB,
+ .bpp = 8,
+ .rx_data_type = IC_IPIPE_RAW_8,
+ .dest_fmts = {
+ MEDIA_BUS_FMT_YUYV8_1_5X8,
+ MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+ MEDIA_BUS_FMT_YUV8_1X24,
+ },
+ },
+ {
+ .code = MEDIA_BUS_FMT_SGRBG8_1X8,
+ .bayer_pattern = KMB_IC_BAYER_FORMAT_GRBG,
+ .bpp = 8,
+ .rx_data_type = IC_IPIPE_RAW_8,
+ .dest_fmts = {
+ MEDIA_BUS_FMT_YUYV8_1_5X8,
+ MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+ MEDIA_BUS_FMT_YUV8_1X24,
+ },
+ },
+ {
+ .code = MEDIA_BUS_FMT_SGBRG8_1X8,
+ .bayer_pattern = KMB_IC_BAYER_FORMAT_GBRG,
+ .bpp = 8,
+ .rx_data_type = IC_IPIPE_RAW_8,
+ .dest_fmts = {
+ MEDIA_BUS_FMT_YUYV8_1_5X8,
+ MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+ MEDIA_BUS_FMT_YUV8_1X24,
+ },
+ },
+ {
+ .code = MEDIA_BUS_FMT_SBGGR8_1X8,
+ .bayer_pattern = KMB_IC_BAYER_FORMAT_BGGR,
+ .bpp = 8,
+ .rx_data_type = IC_IPIPE_RAW_8,
+ .dest_fmts = {
+ MEDIA_BUS_FMT_YUYV8_1_5X8,
+ MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+ MEDIA_BUS_FMT_YUV8_1X24,
+ },
+ },
+ {
+ .code = MEDIA_BUS_FMT_SRGGB10_1X10,
+ .bayer_pattern = KMB_IC_BAYER_FORMAT_RGGB,
+ .bpp = 10,
+ .rx_data_type = IC_IPIPE_RAW_10,
+ .dest_fmts = {
+ MEDIA_BUS_FMT_YUYV8_1_5X8,
+ MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+ MEDIA_BUS_FMT_YUV8_1X24,
+ },
+ },
+ {
+ .code = MEDIA_BUS_FMT_SGRBG10_1X10,
+ .bayer_pattern = KMB_IC_BAYER_FORMAT_GRBG,
+ .bpp = 10,
+ .rx_data_type = IC_IPIPE_RAW_10,
+ .dest_fmts = {
+ MEDIA_BUS_FMT_YUYV8_1_5X8,
+ MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+ MEDIA_BUS_FMT_YUV8_1X24,
+ },
+ },
+ {
+ .code = MEDIA_BUS_FMT_SGBRG10_1X10,
+ .bayer_pattern = KMB_IC_BAYER_FORMAT_GBRG,
+ .bpp = 10,
+ .rx_data_type = IC_IPIPE_RAW_10,
+ .dest_fmts = {
+ MEDIA_BUS_FMT_YUYV8_1_5X8,
+ MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+ MEDIA_BUS_FMT_YUV8_1X24,
+ },
+ },
+ {
+ .code = MEDIA_BUS_FMT_SBGGR10_1X10,
+ .bayer_pattern = KMB_IC_BAYER_FORMAT_BGGR,
+ .bpp = 10,
+ .rx_data_type = IC_IPIPE_RAW_10,
+ .dest_fmts = {
+ MEDIA_BUS_FMT_YUYV8_1_5X8,
+ MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+ MEDIA_BUS_FMT_YUV8_1X24,
+ },
+ },
+ {
+ .code = MEDIA_BUS_FMT_SRGGB12_1X12,
+ .bayer_pattern = KMB_IC_BAYER_FORMAT_RGGB,
+ .bpp = 12,
+ .rx_data_type = IC_IPIPE_RAW_12,
+ .dest_fmts = {
+ MEDIA_BUS_FMT_YUYV8_1_5X8,
+ MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+ MEDIA_BUS_FMT_YUV8_1X24,
+ },
+ },
+ {
+ .code = MEDIA_BUS_FMT_SGRBG12_1X12,
+ .bayer_pattern = KMB_IC_BAYER_FORMAT_GRBG,
+ .bpp = 12,
+ .rx_data_type = IC_IPIPE_RAW_12,
+ .dest_fmts = {
+ MEDIA_BUS_FMT_YUYV8_1_5X8,
+ MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+ MEDIA_BUS_FMT_YUV8_1X24,
+ },
+ },
+ {
+ .code = MEDIA_BUS_FMT_SGBRG12_1X12,
+ .bayer_pattern = KMB_IC_BAYER_FORMAT_GBRG,
+ .bpp = 12,
+ .rx_data_type = IC_IPIPE_RAW_12,
+ .dest_fmts = {
+ MEDIA_BUS_FMT_YUYV8_1_5X8,
+ MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+ MEDIA_BUS_FMT_YUV8_1X24,
+ },
+ },
+ {
+ .code = MEDIA_BUS_FMT_SBGGR12_1X12,
+ .bayer_pattern = KMB_IC_BAYER_FORMAT_BGGR,
+ .bpp = 12,
+ .rx_data_type = IC_IPIPE_RAW_12,
+ .dest_fmts = {
+ MEDIA_BUS_FMT_YUYV8_1_5X8,
+ MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+ MEDIA_BUS_FMT_YUV8_1X24,
+ },
+ },
+ {
+ .code = MEDIA_BUS_FMT_YUYV8_1_5X8,
+ .bayer_pattern = KMB_IC_BAYER_FORMAT_BGGR,
+ .bpp = 8,
+ .rx_data_type = IC_IPIPE_YUV_420_B8,
+ .dest_fmts = {
+ MEDIA_BUS_FMT_YUYV8_1_5X8,
+ MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+ MEDIA_BUS_FMT_YUV8_1X24,
+ },
+ },
+ {
+ .code = MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+ .bayer_pattern = KMB_IC_BAYER_FORMAT_BGGR,
+ .bpp = 8,
+ .rx_data_type = IC_IPIPE_YUV_420_B8,
+ .dest_fmts = {
+ MEDIA_BUS_FMT_YUYV8_1_5X8,
+ MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+ MEDIA_BUS_FMT_YUV8_1X24,
+ },
+ },
+ {
+ .code = MEDIA_BUS_FMT_YUV8_1X24,
+ .bayer_pattern = KMB_IC_BAYER_FORMAT_BGGR,
+ .bpp = 8,
+ .rx_data_type = IC_IPIPE_YUV_420_B8,
+ .dest_fmts = {
+ MEDIA_BUS_FMT_YUYV8_1_5X8,
+ MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+ MEDIA_BUS_FMT_YUV8_1X24,
+ },
+ },
+ {
+ .code = MEDIA_BUS_FMT_Y8_1X8,
+ .bayer_pattern = KMB_IC_BAYER_FORMAT_BGGR,
+ .bpp = 8,
+ .rx_data_type = IC_IPIPE_RAW_8,
+ .dest_fmts = {
+ MEDIA_BUS_FMT_Y8_1X8,
+ MEDIA_BUS_FMT_Y10_1X10,
+ },
+ },
+ {
+ .code = MEDIA_BUS_FMT_Y10_1X10,
+ .bayer_pattern = KMB_IC_BAYER_FORMAT_BGGR,
+ .bpp = 10,
+ .rx_data_type = IC_IPIPE_RAW_10,
+ .dest_fmts = {
+ MEDIA_BUS_FMT_Y8_1X8,
+ MEDIA_BUS_FMT_Y10_1X10,
+ },
+ },
+};
+
+static inline const struct kmb_isp_source_format *
+kmb_isp_get_src_fmt_by_code(u32 code)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(source_fmts); i++) {
+ if (source_fmts[i].code == code)
+ return &source_fmts[i];
+ }
+
+ return NULL;
+}
+
+static inline bool
+kmb_isp_valid_destination_format(struct v4l2_mbus_framefmt *mbus_fmt, u32 code)
+{
+ const struct kmb_isp_source_format *src_fmt =
+ kmb_isp_get_src_fmt_by_code(mbus_fmt->code);
+ unsigned int i;
+
+ if (!src_fmt)
+ return false;
+
+ for (i = 0; i < KMB_ISP_MAX_DEST_FMTS; i++)
+ if (src_fmt->dest_fmts[i] == code)
+ return true;
+
+ return false;
+}
+
+static void kmb_isp_meta_buf_done(struct kmb_isp *kmb_isp,
+ struct kmb_metadata_buf *meta_buf,
+ enum vb2_buffer_state state)
+{
+ /* Remove isp config on buf done */
+ mutex_lock(&kmb_isp->meta_q_lock);
+ list_del(&meta_buf->list);
+ mutex_unlock(&kmb_isp->meta_q_lock);
+
+ vb2_buffer_done(&meta_buf->vb.vb2_buf, state);
+ dev_dbg(kmb_isp->dev, "Meta buf done %u state %d",
+ meta_buf->vb.sequence, state);
+}
+
+static struct kmb_metadata_buf *
+kmb_isp_find_params_by_addr(struct kmb_isp *kmb_isp, dma_addr_t addr)
+{
+ struct kmb_metadata_buf *meta_buf;
+ struct list_head *node = NULL;
+
+ mutex_lock(&kmb_isp->meta_q_lock);
+
+ list_for_each(node, &kmb_isp->meta_params_process_q) {
+ meta_buf = list_entry(node, struct kmb_metadata_buf, list);
+ if (meta_buf->params.dma_addr_isp == addr) {
+ mutex_unlock(&kmb_isp->meta_q_lock);
+ return meta_buf;
+ }
+ }
+
+ mutex_unlock(&kmb_isp->meta_q_lock);
+
+ return NULL;
+}
+
+static struct kmb_metadata_buf *
+kmb_isp_find_stats_by_seq(struct kmb_isp *kmb_isp, u32 sequence)
+{
+ struct kmb_metadata_buf *meta_buf;
+ struct list_head *node = NULL;
+
+ mutex_lock(&kmb_isp->meta_q_lock);
+
+ list_for_each(node, &kmb_isp->meta_stats_process_q) {
+ meta_buf = list_entry(node, struct kmb_metadata_buf, list);
+ if (meta_buf->vb.sequence == sequence) {
+ mutex_unlock(&kmb_isp->meta_q_lock);
+ return meta_buf;
+ }
+ }
+
+ mutex_unlock(&kmb_isp->meta_q_lock);
+
+ return NULL;
+}
+
+static void kmb_isp_fill_stats_update_flags(struct kmb_metadata_buf *stats_buf,
+ struct kmb_metadata_buf *param_buf)
+{
+ struct kmb_isp_stats *user_stats =
+ vb2_plane_vaddr(&stats_buf->vb.vb2_buf, 0);
+
+ user_stats->update.ae_awb =
+ param_buf->params.isp->raw.awb_stats_en;
+ user_stats->update.af =
+ param_buf->params.isp->raw.af_stats_en;
+ user_stats->update.luma_hist =
+ param_buf->params.isp->raw.luma_hist_en;
+ user_stats->update.rgb_hist =
+ param_buf->params.isp->raw.awb_rgb_hist_en;
+ user_stats->update.flicker_rows =
+ param_buf->params.isp->raw.flicker_accum_en;
+ /* Dehaze stats is always enabled */
+ user_stats->update.dehaze = true;
+}
+
+static int kmb_isp_process_config(struct kmb_isp *kmb_isp)
+{
+ struct kmb_metadata_buf *param_buf;
+ struct kmb_metadata_buf *stats_buf;
+ struct kmb_ic_ev cfg_evt;
+ int ret;
+
+ mutex_lock(&kmb_isp->meta_q_lock);
+
+ if (list_empty(&kmb_isp->meta_params_pending_q)) {
+ mutex_unlock(&kmb_isp->meta_q_lock);
+ return -EAGAIN;
+ }
+ param_buf = list_first_entry(&kmb_isp->meta_params_pending_q,
+ struct kmb_metadata_buf, list);
+
+ if (list_empty(&kmb_isp->meta_stats_pending_q)) {
+ mutex_unlock(&kmb_isp->meta_q_lock);
+ return -EAGAIN;
+ }
+ stats_buf = list_first_entry(&kmb_isp->meta_stats_pending_q,
+ struct kmb_metadata_buf, list);
+
+ list_del(&stats_buf->list);
+ list_del(¶m_buf->list);
+
+ mutex_unlock(&kmb_isp->meta_q_lock);
+
+ param_buf->vb.sequence = kmb_isp->sequence++;
+ stats_buf->vb.sequence = param_buf->vb.sequence;
+
+ /* Update header version, user data key and image width */
+ param_buf->params.isp->header_version = KMB_VPU_ISP_ABI_VERSION;
+ param_buf->params.isp->num_exposures = 1;
+ param_buf->params.isp->user_data_key =
+ param_buf->vb.sequence;
+ param_buf->params.isp->image_data_width =
+ kmb_isp->source_fmt->bpp;
+ param_buf->params.isp->bayer_order =
+ kmb_isp->source_fmt->bayer_pattern;
+
+ /* Set stats addresses */
+ memcpy(param_buf->params.isp->raw.stats, stats_buf->stats.raw,
+ sizeof(param_buf->params.isp->raw.stats));
+ param_buf->params.isp->dehaze.stats_addr =
+ stats_buf->stats.dehaze_stats_addr;
+
+ memset(&cfg_evt, 0, sizeof(cfg_evt));
+ cfg_evt.ctrl = KMB_IC_EVENT_TYPE_CONFIG_ISP;
+ cfg_evt.ev_info.seq_nr = param_buf->vb.sequence;
+ cfg_evt.ev_info.user_data_base_addr01 = param_buf->params.dma_addr_isp;
+ dev_dbg(kmb_isp->dev, "Process config addr %llx",
+ param_buf->params.dma_addr_isp);
+ ret = kmb_cam_xlink_write_msg(kmb_isp->xlink_cam,
+ kmb_isp->config_chan_id,
+ (u8 *)&cfg_evt, sizeof(cfg_evt));
+ if (ret < 0) {
+ dev_err(kmb_isp->dev, "Error on process config %d", ret);
+ vb2_buffer_done(¶m_buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+ vb2_buffer_done(&stats_buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+ return ret;
+ }
+
+ /* Update stats update flags */
+ kmb_isp_fill_stats_update_flags(stats_buf, param_buf);
+
+ /* Add to items to the processing list */
+ mutex_lock(&kmb_isp->meta_q_lock);
+ list_add_tail(¶m_buf->list, &kmb_isp->meta_params_process_q);
+ list_add_tail(&stats_buf->list, &kmb_isp->meta_stats_process_q);
+ mutex_unlock(&kmb_isp->meta_q_lock);
+
+ return 0;
+}
+
+static int kmb_isp_worker_thread(void *isp)
+{
+ struct kmb_metadata_buf *meta_params;
+ struct kmb_metadata_buf *meta_stats;
+ struct kmb_isp *kmb_isp = isp;
+ struct v4l2_event v4l2_evt;
+ struct kmb_ic_ev cfg_evt;
+ bool stopped = false;
+ u32 base_addr;
+ int ret;
+
+ memset(&v4l2_evt, 0, sizeof(v4l2_evt));
+
+ set_freezable();
+
+ while (!kthread_should_stop()) {
+ try_to_freeze();
+
+ if (stopped) {
+ set_current_state(TASK_INTERRUPTIBLE);
+ schedule();
+ continue;
+ }
+
+ memset(&cfg_evt, 0x00, sizeof(cfg_evt));
+ cfg_evt.ctrl = KMB_IC_EVENT_MAX;
+ ret = kmb_cam_xlink_read_msg(kmb_isp->xlink_cam,
+ kmb_isp->config_chan_id,
+ (u8 *)&cfg_evt, sizeof(cfg_evt));
+ if (ret < 0) {
+ stopped = true;
+ complete_all(&kmb_isp->source_stopped);
+ continue;
+ }
+ base_addr = cfg_evt.ev_info.user_data_base_addr01;
+
+ meta_params = kmb_isp_find_params_by_addr(kmb_isp, base_addr);
+
+ switch (cfg_evt.ctrl) {
+ case KMB_IC_EVENT_TYPE_READOUT_START:
+ if (meta_params) {
+ v4l2_evt.type = V4L2_EVENT_FRAME_SYNC;
+ v4l2_evt.u.frame_sync.frame_sequence =
+ meta_params->vb.sequence;
+ v4l2_subdev_notify_event(&kmb_isp->subdev,
+ &v4l2_evt);
+ } else {
+ dev_err(kmb_isp->dev, "Ouch readout no buf");
+ }
+ /* Process next isp configuration on readout start */
+ kmb_isp_process_config(kmb_isp);
+ break;
+ case KMB_IC_EVENT_TYPE_ISP_END:
+ if (meta_params)
+ kmb_isp_meta_buf_done(kmb_isp, meta_params,
+ VB2_BUF_STATE_DONE);
+ else
+ dev_err(kmb_isp->dev, "Ouch no params buf");
+ break;
+ case KMB_IC_EVENT_TYPE_STATS_READY:
+ meta_stats = NULL;
+ if (meta_params)
+ meta_stats =
+ kmb_isp_find_stats_by_seq(kmb_isp,
+ meta_params->vb.sequence);
+
+ if (meta_stats)
+ kmb_isp_meta_buf_done(kmb_isp, meta_stats,
+ VB2_BUF_STATE_DONE);
+ else
+ dev_err(kmb_isp->dev, "Ouch no stats buf");
+ break;
+ case KMB_IC_ERROR_SRC_MIPI_CFG_SKIPPED:
+ if (meta_params) {
+ kmb_isp_meta_buf_done(kmb_isp, meta_params,
+ VB2_BUF_STATE_ERROR);
+ meta_stats =
+ kmb_isp_find_stats_by_seq(kmb_isp,
+ meta_params->vb.sequence);
+ if (meta_stats)
+ kmb_isp_meta_buf_done(kmb_isp,
+ meta_stats,
+ VB2_BUF_STATE_ERROR);
+ }
+ break;
+ case KMB_IC_ERROR_SRC_MIPI_CFG_MISSING:
+ /* Process new configuration when vpu is starving */
+ kmb_isp_process_config(kmb_isp);
+ break;
+ case KMB_IC_EVENT_TYPE_SOURCE_STOPPED:
+ complete_all(&kmb_isp->source_stopped);
+ stopped = true;
+ break;
+ default:
+ dev_dbg(kmb_isp->dev, "Received event %d",
+ cfg_evt.ctrl);
+ break;
+ }
+ }
+
+ return 0;
+}
+
+static int kmb_isp_configure_vpu_source(struct kmb_isp *kmb_isp)
+{
+ struct kmb_ic_source_config *src_cfg;
+ struct v4l2_subdev_format *src_fmt;
+ struct kmb_ic_ev mipi_cfg_evt;
+ struct v4l2_subdev *subdev;
+ struct media_pad *rpd;
+ s64 link_freq;
+ int ret;
+
+ if (WARN_ON(!kmb_isp->source_fmt))
+ return -EINVAL;
+
+ /* Get sensor remote pad we need information for pixel clock */
+ rpd = media_entity_remote_pad(&kmb_isp->pads[KMB_ISP_SINK_PAD_SENSOR]);
+ if (!rpd || !is_media_entity_v4l2_subdev(rpd->entity))
+ return -EINVAL;
+
+ subdev = media_entity_to_v4l2_subdev(rpd->entity);
+ if (!subdev)
+ return -EINVAL;
+
+ src_cfg = kmb_isp->msg_vaddr;
+ memset(src_cfg, 0, sizeof(*src_cfg));
+
+ src_fmt = &kmb_isp->active_pad_fmt[KMB_ISP_SINK_PAD_SENSOR];
+
+ /* Full size isp destination is always set on first src pad */
+ src_cfg->camera_output_size.w = src_fmt->format.width;
+ src_cfg->camera_output_size.h = src_fmt->format.height;
+ src_cfg->no_exposure = 1;
+
+ src_cfg->crop_window.x1 = 0;
+ src_cfg->crop_window.x2 = src_cfg->camera_output_size.w;
+ src_cfg->crop_window.y1 = 0;
+ src_cfg->crop_window.y2 = src_cfg->camera_output_size.h;
+
+ src_cfg->bayer_format = kmb_isp->source_fmt->bayer_pattern;
+ src_cfg->bpp = kmb_isp->source_fmt->bpp;
+
+ src_cfg->mipi_rx_data.no_controller = kmb_isp->csi2_config.rx_id;
+ src_cfg->mipi_rx_data.data_mode = 1;
+ src_cfg->mipi_rx_data.no_lanes = kmb_isp->csi2_config.num_lanes;
+ src_cfg->mipi_rx_data.data_type = kmb_isp->source_fmt->rx_data_type;
+
+ link_freq = v4l2_get_link_freq(subdev->ctrl_handler, src_cfg->bpp,
+ src_cfg->mipi_rx_data.no_lanes * 2);
+ if (link_freq < 0)
+ return link_freq;
+
+ src_cfg->mipi_rx_data.lane_rate_mbps = link_freq * 2;
+
+ src_cfg->metadata_width = src_fmt->format.width;
+ src_cfg->metadata_height = 0;
+ src_cfg->metadata_data_type = IC_IPIPE_EMBEDDED_8BIT;
+
+ memset(&mipi_cfg_evt, 0, sizeof(mipi_cfg_evt));
+ mipi_cfg_evt.ctrl = KMB_IC_EVENT_TYPE_CONFIG_SOURCE;
+ mipi_cfg_evt.ev_info.user_data_base_addr01 = kmb_isp->msg_phy_addr;
+ ret = kmb_cam_xlink_write_msg(kmb_isp->xlink_cam,
+ kmb_isp->config_chan_id,
+ (u8 *)&mipi_cfg_evt,
+ sizeof(mipi_cfg_evt));
+ if (ret < 0) {
+ dev_err(kmb_isp->dev, "Error config source xlink msg %d", ret);
+ return ret;
+ }
+
+ ret = kmb_cam_xlink_read_msg(kmb_isp->xlink_cam,
+ kmb_isp->config_chan_id,
+ (u8 *)&mipi_cfg_evt,
+ sizeof(mipi_cfg_evt));
+ if (ret < 0) {
+ dev_err(kmb_isp->dev, "Error source xlink msg ack %d", ret);
+ return ret;
+ }
+ if (mipi_cfg_evt.ctrl != KMB_IC_EVENT_TYPE_SOURCE_CONFIGURED) {
+ dev_err(kmb_isp->dev, "Error source configured %d",
+ mipi_cfg_evt.ctrl);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int kmb_isp_start_source(struct kmb_isp *kmb_isp)
+{
+ struct kmb_ic_ev cfg_evt;
+ int ret;
+
+ if (WARN_ON(kmb_isp->source_streaming))
+ return -EINVAL;
+
+ memset(&cfg_evt, 0, sizeof(cfg_evt));
+ cfg_evt.ctrl = KMB_IC_EVENT_TYPE_START_SOURCE;
+ cfg_evt.ev_info.inst_id = 0;
+ ret = kmb_cam_xlink_write_msg(kmb_isp->xlink_cam,
+ kmb_isp->config_chan_id,
+ (u8 *)&cfg_evt, sizeof(cfg_evt));
+ if (ret < 0) {
+ dev_err(kmb_isp->dev, "Error start source xlink msg %d", ret);
+ return ret;
+ }
+
+ ret = kmb_cam_xlink_read_msg(kmb_isp->xlink_cam,
+ kmb_isp->config_chan_id,
+ (u8 *)&cfg_evt, sizeof(cfg_evt));
+ if (ret < 0) {
+ dev_err(kmb_isp->dev, "Error start source msg ack %d", ret);
+ return ret;
+ }
+ if (cfg_evt.ctrl != KMB_IC_EVENT_TYPE_SOURCE_STARTED) {
+ dev_err(kmb_isp->dev, "Error source started ack %d",
+ cfg_evt.ctrl);
+ return -EINVAL;
+ }
+
+ init_completion(&kmb_isp->source_stopped);
+ kmb_isp->thread = kthread_run(kmb_isp_worker_thread, kmb_isp,
+ "kmb_isp_thread");
+ if (IS_ERR(kmb_isp->thread)) {
+ ret = PTR_ERR(kmb_isp->thread);
+ kmb_isp->thread = NULL;
+ dev_err(kmb_isp->dev, "Thread run failed %d", ret);
+ return ret;
+ }
+
+ kmb_isp->source_streaming = true;
+
+ return 0;
+}
+
+static int kmb_isp_stop_source(struct kmb_isp *kmb_isp,
+ enum kmb_isp_stop_method method)
+{
+ struct kmb_ic_ev cfg_evt;
+ unsigned long timeout;
+ int ret;
+
+ if (WARN_ON(!kmb_isp->source_streaming))
+ return -EINVAL;
+
+ switch (method) {
+ case KMB_ISP_STOP_SYNC:
+ memset(&cfg_evt, 0, sizeof(cfg_evt));
+ cfg_evt.ctrl = KMB_IC_EVENT_TYPE_STOP_SOURCE;
+ cfg_evt.ev_info.inst_id = 0;
+
+ ret = kmb_cam_xlink_write_msg(kmb_isp->xlink_cam,
+ kmb_isp->config_chan_id,
+ (u8 *)&cfg_evt,
+ sizeof(cfg_evt));
+ if (ret < 0) {
+ dev_err(kmb_isp->dev,
+ "Error stop source xlink msg %d", ret);
+ return ret;
+ }
+
+ timeout = msecs_to_jiffies(KMB_STOP_SOURCE_TIMEOUT_MS);
+ ret = wait_for_completion_timeout(&kmb_isp->source_stopped,
+ timeout);
+ if (ret == 0) {
+ dev_err(kmb_isp->dev, "Source stopped timeout");
+ return -ETIMEDOUT;
+ }
+ break;
+ case KMB_ISP_STOP_FORCE:
+ /* Stop ISP without notifying VPU. */
+ break;
+ default:
+ dev_err(kmb_isp->dev, "Invalid stop method %d", method);
+ return -EINVAL;
+ }
+
+ ret = kthread_stop(kmb_isp->thread);
+ if (ret < 0) {
+ dev_err(kmb_isp->dev, "Thread stop failed %d", ret);
+ return ret;
+ }
+ kmb_isp->thread = NULL;
+
+ kmb_isp->source_streaming = false;
+
+ return 0;
+}
+
+static void kmb_isp_discard_all_params(struct kmb_isp *kmb_isp)
+{
+ struct kmb_metadata_buf *meta_buf;
+ struct list_head *next = NULL;
+ struct list_head *pos = NULL;
+
+ mutex_lock(&kmb_isp->meta_q_lock);
+ list_for_each_safe(pos, next, &kmb_isp->meta_params_pending_q) {
+ meta_buf = list_entry(pos, struct kmb_metadata_buf, list);
+ list_del(&meta_buf->list);
+ vb2_buffer_done(&meta_buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+ }
+ list_for_each_safe(pos, next, &kmb_isp->meta_params_process_q) {
+ meta_buf = list_entry(pos, struct kmb_metadata_buf, list);
+ list_del(&meta_buf->list);
+ vb2_buffer_done(&meta_buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+ }
+ mutex_unlock(&kmb_isp->meta_q_lock);
+}
+
+/* Params metadata buffer ops */
+static int kmb_isp_queue_params_buf(void *priv, struct kmb_metadata_buf *meta_buf)
+{
+ struct kmb_isp *kmb_isp = priv;
+
+ if (WARN_ON(!priv || !meta_buf))
+ return -EINVAL;
+
+ INIT_LIST_HEAD(&meta_buf->list);
+
+ mutex_lock(&kmb_isp->meta_q_lock);
+ list_add_tail(&meta_buf->list, &kmb_isp->meta_params_pending_q);
+ mutex_unlock(&kmb_isp->meta_q_lock);
+
+ return 0;
+}
+
+static void kmb_isp_queue_params_flush(void *priv)
+{
+ struct kmb_isp *kmb_isp = priv;
+
+ kmb_isp_discard_all_params(kmb_isp);
+}
+
+/* Statistics metadata buffer ops */
+static const struct kmb_metabuf_queue_ops isp_params_queue_ops = {
+ .queue = kmb_isp_queue_params_buf,
+ .flush = kmb_isp_queue_params_flush,
+};
+
+static void kmb_isp_discard_all_stats(struct kmb_isp *kmb_isp)
+{
+ struct kmb_metadata_buf *meta_buf;
+ struct list_head *next = NULL;
+ struct list_head *pos = NULL;
+
+ mutex_lock(&kmb_isp->meta_q_lock);
+ list_for_each_safe(pos, next, &kmb_isp->meta_stats_pending_q) {
+ meta_buf = list_entry(pos, struct kmb_metadata_buf, list);
+ list_del(&meta_buf->list);
+ vb2_buffer_done(&meta_buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+ }
+ list_for_each_safe(pos, next, &kmb_isp->meta_stats_process_q) {
+ meta_buf = list_entry(pos, struct kmb_metadata_buf, list);
+ list_del(&meta_buf->list);
+ vb2_buffer_done(&meta_buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+ }
+ mutex_unlock(&kmb_isp->meta_q_lock);
+}
+
+static int kmb_isp_queue_stats_buf(void *priv, struct kmb_metadata_buf *meta_buf)
+{
+ struct kmb_isp *kmb_isp = priv;
+
+ if (WARN_ON(!priv || !meta_buf))
+ return -EINVAL;
+
+ INIT_LIST_HEAD(&meta_buf->list);
+
+ mutex_lock(&kmb_isp->meta_q_lock);
+ list_add_tail(&meta_buf->list, &kmb_isp->meta_stats_pending_q);
+ mutex_unlock(&kmb_isp->meta_q_lock);
+
+ return 0;
+}
+
+static void kmb_isp_queue_stats_flush(void *priv)
+{
+ struct kmb_isp *kmb_isp = priv;
+
+ kmb_isp_discard_all_stats(kmb_isp);
+}
+
+static const struct kmb_metabuf_queue_ops isp_stats_queue_ops = {
+ .queue = kmb_isp_queue_stats_buf,
+ .flush = kmb_isp_queue_stats_flush,
+};
+
+static int kmb_isp_subdev_get_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *fmt)
+{
+ struct kmb_isp *kmb_isp = v4l2_get_subdevdata(sd);
+
+ if (fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
+ fmt->format = *v4l2_subdev_get_try_format(sd, cfg, fmt->pad);
+ } else {
+ mutex_lock(&kmb_isp->lock);
+ fmt->format = kmb_isp->active_pad_fmt[fmt->pad].format;
+ mutex_unlock(&kmb_isp->lock);
+ }
+
+ return 0;
+}
+
+static int kmb_isp_config_pipe_src(struct kmb_isp *kmb_isp,
+ struct kmb_pipeline *pipe,
+ struct v4l2_mbus_framefmt *mbus_fmt)
+{
+ const struct kmb_isp_source_format *fmt_info;
+ int ret;
+
+ fmt_info = kmb_isp_get_src_fmt_by_code(mbus_fmt->code);
+ if (!fmt_info) {
+ dev_err(kmb_isp->dev, "Format code not supported %d",
+ mbus_fmt->code);
+ return -EINVAL;
+ }
+
+ /* Clean any previous configurations */
+ memset(&kmb_isp->pipe_cfg, 0x00, sizeof(kmb_isp->pipe_cfg));
+ kmb_isp->pipe_cfg.pipe_type = PIPE_TYPE_ISP_ISP_ULL;
+ kmb_isp->pipe_cfg.src_type = SRC_TYPE_ALLOC_VPU_DATA_MIPI;
+ kmb_isp->pipe_cfg.pipe_trans_hub = PIPE_TRANSFORM_HUB_NONE;
+
+ kmb_isp->pipe_cfg.in_isp_res.w = mbus_fmt->width;
+ kmb_isp->pipe_cfg.in_isp_res.h = mbus_fmt->height;
+
+ kmb_isp->pipe_cfg.in_data_width = fmt_info->bpp;
+ kmb_isp->pipe_cfg.in_data_packed = 1;
+
+ kmb_isp->pipe_cfg.in_isp_stride = (kmb_isp->pipe_cfg.in_isp_res.w *
+ kmb_isp->pipe_cfg.in_isp_res.h *
+ kmb_isp->pipe_cfg.in_data_width) / 8;
+
+ /* Always set to 8 required by the vpu firmware */
+ kmb_isp->pipe_cfg.out_data_width = 8;
+
+ /* Isp does not have scaler */
+ kmb_isp->pipe_cfg.out_isp_res = kmb_isp->pipe_cfg.in_isp_res;
+
+ ret = kmb_pipe_config_src(pipe, &kmb_isp->pipe_cfg);
+ if (ret < 0)
+ return ret;
+
+ kmb_isp->source_fmt = fmt_info;
+
+ return 0;
+}
+
+static int kmb_isp_subdev_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *sd_fmt)
+{
+ struct kmb_isp *kmb_isp = v4l2_get_subdevdata(sd);
+ struct v4l2_mbus_framefmt *mbus_fmt;
+ struct kmb_channel_cfg channel_cfg;
+ struct kmb_pipeline *pipe;
+ int ret;
+
+ mutex_lock(&kmb_isp->lock);
+ if (sd_fmt->which == V4L2_SUBDEV_FORMAT_TRY) {
+ pipe = &kmb_isp->try_pipe;
+ mbus_fmt = v4l2_subdev_get_try_format(sd, cfg, sd_fmt->pad);
+ } else {
+ pipe = &kmb_isp->active_pipe;
+ mbus_fmt = &kmb_isp->active_pad_fmt[sd_fmt->pad].format;
+ }
+ mutex_unlock(&kmb_isp->lock);
+
+ switch (sd_fmt->pad) {
+ case KMB_ISP_SINK_PAD_SENSOR:
+ ret = kmb_isp_config_pipe_src(kmb_isp, pipe, &sd_fmt->format);
+ if (ret < 0)
+ return ret;
+
+ /* Configure first isp control channel */
+ channel_cfg.frm_res.w = sd_fmt->format.width;
+ channel_cfg.frm_res.h = sd_fmt->format.height;
+ channel_cfg.id = kmb_isp->config_chan_id;
+ kmb_pipe_config_dest(pipe, PIPE_OUTPUT_ID_ISP_CTRL,
+ &channel_cfg);
+
+ /* Set default resolution of destination channel */
+ channel_cfg.frm_res.w = sd_fmt->format.width;
+ channel_cfg.frm_res.h = sd_fmt->format.height;
+ channel_cfg.id = kmb_isp->capture_chan_id;
+ kmb_pipe_config_dest(pipe, PIPE_OUTPUT_ID_0, &channel_cfg);
+
+ sd_fmt->format.width = channel_cfg.frm_res.w;
+ sd_fmt->format.height = channel_cfg.frm_res.h;
+ break;
+ case KMB_ISP_SRC_PAD_VID: {
+ struct v4l2_mbus_framefmt *mbus_src_fmt;
+
+ mutex_lock(&kmb_isp->lock);
+ if (sd_fmt->which == V4L2_SUBDEV_FORMAT_TRY)
+ mbus_src_fmt =
+ v4l2_subdev_get_try_format(sd, cfg,
+ KMB_ISP_SINK_PAD_SENSOR);
+ else
+ mbus_src_fmt =
+ &kmb_isp->active_pad_fmt[KMB_ISP_SINK_PAD_SENSOR].format;
+ mutex_unlock(&kmb_isp->lock);
+
+ if (!kmb_isp_valid_destination_format(mbus_src_fmt,
+ sd_fmt->format.code))
+ return -EINVAL;
+
+ channel_cfg.frm_res.w = sd_fmt->format.width;
+ channel_cfg.frm_res.h = sd_fmt->format.height;
+ channel_cfg.id = kmb_isp->capture_chan_id;
+ kmb_pipe_config_dest(pipe, PIPE_OUTPUT_ID_0, &channel_cfg);
+
+ sd_fmt->format.width = channel_cfg.frm_res.w;
+ sd_fmt->format.height = channel_cfg.frm_res.h;
+ break;
+ }
+ case KMB_ISP_SINK_PAD_PARAM:
+ case KMB_ISP_SRC_PAD_STATS:
+ /* Isp config metadata sink format can be just fixed */
+ if (sd_fmt->format.code != MEDIA_BUS_FMT_FIXED)
+ return -EINVAL;
+ break;
+ }
+
+ mutex_lock(&kmb_isp->lock);
+ *mbus_fmt = sd_fmt->format;
+ mutex_unlock(&kmb_isp->lock);
+
+ return 0;
+}
+
+static int
+kmb_isp_subdev_enum_mbus_code(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_mbus_code_enum *code)
+{
+ switch (code->pad) {
+ case KMB_ISP_SINK_PAD_SENSOR:
+ if (code->index >= ARRAY_SIZE(source_fmts))
+ return -EINVAL;
+
+ code->code = source_fmts[code->index].code;
+ break;
+ case KMB_ISP_SRC_PAD_VID: {
+ struct kmb_isp *kmb_isp = v4l2_get_subdevdata(sd);
+ const struct kmb_isp_source_format *src_fmt;
+ struct v4l2_mbus_framefmt *mbus_src_fmt;
+
+ mutex_lock(&kmb_isp->lock);
+
+ if (code->which == V4L2_SUBDEV_FORMAT_TRY)
+ mbus_src_fmt =
+ v4l2_subdev_get_try_format(sd, cfg,
+ KMB_ISP_SINK_PAD_SENSOR);
+ else
+ mbus_src_fmt =
+ &kmb_isp->active_pad_fmt[KMB_ISP_SINK_PAD_SENSOR].format;
+
+ mutex_unlock(&kmb_isp->lock);
+
+ src_fmt = kmb_isp_get_src_fmt_by_code(mbus_src_fmt->code);
+ if (!src_fmt)
+ return -EINVAL;
+
+ if (code->index >= ARRAY_SIZE(src_fmt->dest_fmts))
+ return -EINVAL;
+
+ if (!src_fmt->dest_fmts[code->index])
+ return -EINVAL;
+
+ code->code = src_fmt->dest_fmts[code->index];
+ break;
+ }
+ case KMB_ISP_SINK_PAD_PARAM:
+ case KMB_ISP_SRC_PAD_STATS:
+ if (code->index > 0)
+ return -EINVAL;
+
+ code->code = MEDIA_BUS_FMT_FIXED;
+ break;
+ }
+
+ return 0;
+}
+
+static int kmb_isp_src_s_stream(struct kmb_isp *kmb_isp, int enable)
+{
+ struct v4l2_subdev *subdev;
+ struct media_pad *remote;
+ int ret;
+
+ remote = media_entity_remote_pad(&kmb_isp->pads[KMB_ISP_SINK_PAD_SENSOR]);
+ if (!remote || !is_media_entity_v4l2_subdev(remote->entity))
+ return -EINVAL;
+
+ subdev = media_entity_to_v4l2_subdev(remote->entity);
+ if (!subdev)
+ return -EINVAL;
+
+ ret = v4l2_subdev_call(subdev, video, s_stream, enable);
+ if (ret < 0 && ret != -ENOIOCTLCMD)
+ dev_err(kmb_isp->dev, "Cannot set source stream %d", enable);
+
+ return ret != -ENOIOCTLCMD ? ret : 0;
+}
+
+static int kmb_isp_s_stream(struct v4l2_subdev *sd, int enable)
+{
+ struct kmb_isp *kmb_isp = v4l2_get_subdevdata(sd);
+ int ret;
+
+ mutex_lock(&kmb_isp->lock);
+
+ /* Don't send isp config on stream disable */
+ if (enable) {
+ ret = kmb_cam_xlink_open_channel(kmb_isp->xlink_cam,
+ kmb_isp->config_chan_id);
+ if (ret < 0) {
+ dev_err(kmb_isp->dev,
+ "Fail to open xlink channel %d", ret);
+ goto error_unlock;
+ }
+
+ ret = kmb_isp_configure_vpu_source(kmb_isp);
+ if (ret)
+ goto error_close_xlink_channel;
+
+ /* Process first configuration on stream enable */
+ ret = kmb_isp_process_config(kmb_isp);
+ if (ret)
+ goto error_close_xlink_channel;
+
+ ret = kmb_isp_start_source(kmb_isp);
+ if (ret)
+ goto error_discard_metadata;
+
+ ret = kmb_isp_src_s_stream(kmb_isp, enable);
+ if (ret)
+ goto error_isp_stop_source;
+
+ kmb_isp->isp_streaming = true;
+ } else {
+ /* Try top to stop the source synchronized */
+ if (kmb_isp->source_streaming)
+ kmb_isp_stop_source(kmb_isp, KMB_ISP_STOP_SYNC);
+
+ kmb_cam_xlink_close_channel(kmb_isp->xlink_cam,
+ kmb_isp->config_chan_id);
+
+ /* Force stop isp if still streaming after channel is closed */
+ if (kmb_isp->source_streaming)
+ kmb_isp_stop_source(kmb_isp, KMB_ISP_STOP_FORCE);
+
+ /* Discard all unprocessed params and statistics */
+ kmb_isp_discard_all_params(kmb_isp);
+ kmb_isp_discard_all_stats(kmb_isp);
+
+ kmb_isp_src_s_stream(kmb_isp, enable);
+
+ kmb_isp->isp_streaming = false;
+ kmb_isp->sequence = 0;
+ }
+
+ mutex_unlock(&kmb_isp->lock);
+
+ return 0;
+
+error_isp_stop_source:
+ kmb_isp_stop_source(kmb_isp, KMB_ISP_STOP_FORCE);
+error_discard_metadata:
+ kmb_isp_discard_all_params(kmb_isp);
+ kmb_isp_discard_all_stats(kmb_isp);
+error_close_xlink_channel:
+ xlink_close_channel(&kmb_isp->xlink_cam->handle,
+ kmb_isp->config_chan_id);
+error_unlock:
+ mutex_unlock(&kmb_isp->lock);
+
+ return ret;
+}
+
+static int kmb_isp_subscribe_event(struct v4l2_subdev *sd,
+ struct v4l2_fh *fh,
+ struct v4l2_event_subscription *sub)
+{
+ return v4l2_event_subscribe(fh, sub, KMB_ISP_EVT_Q_LEN, NULL);
+}
+
+/* sub-device core operations */
+static struct v4l2_subdev_core_ops kmb_isp_subdev_core_ops = {
+ .subscribe_event = kmb_isp_subscribe_event,
+ .unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
+/* sub-device video operations */
+static struct v4l2_subdev_video_ops kmb_isp_subdev_video_ops = {
+ .s_stream = kmb_isp_s_stream,
+};
+
+/* sub-device pad operations */
+static struct v4l2_subdev_pad_ops kmb_isp_subdev_pad_ops = {
+ .set_fmt = kmb_isp_subdev_set_fmt,
+ .get_fmt = kmb_isp_subdev_get_fmt,
+ .enum_mbus_code = kmb_isp_subdev_enum_mbus_code,
+};
+
+/* sub-device operations */
+static const struct v4l2_subdev_ops kmb_isp_subdev_ops = {
+ .core = &kmb_isp_subdev_core_ops,
+ .video = &kmb_isp_subdev_video_ops,
+ .pad = &kmb_isp_subdev_pad_ops,
+};
+
+/* sub-device internal operations */
+static int kmb_isp_open(struct v4l2_subdev *sd,
+ struct v4l2_subdev_fh *fh)
+{
+ struct kmb_isp *kmb_isp = v4l2_get_subdevdata(sd);
+
+ return kmb_pipe_request(&kmb_isp->active_pipe);
+}
+
+static int kmb_isp_close(struct v4l2_subdev *sd,
+ struct v4l2_subdev_fh *fh)
+{
+ struct kmb_isp *kmb_isp = v4l2_get_subdevdata(sd);
+
+ kmb_pipe_release(&kmb_isp->active_pipe);
+
+ return 0;
+}
+
+static const struct v4l2_subdev_internal_ops kmb_isp_internal_ops = {
+ .open = kmb_isp_open,
+ .close = kmb_isp_close,
+};
/**
* kmb_isp_init - Initialize Kmb isp subdevice
@@ -19,7 +1158,90 @@ int kmb_isp_init(struct kmb_isp *kmb_isp, struct device *dev,
struct kmb_isp_csi2_config *csi2_config,
struct kmb_xlink_cam *xlink_cam)
{
+ int ret;
+
+ v4l2_subdev_init(&kmb_isp->subdev, &kmb_isp_subdev_ops);
+ v4l2_set_subdevdata(&kmb_isp->subdev, kmb_isp);
+
+ ret = kmb_pipe_init(&kmb_isp->active_pipe, dev, xlink_cam);
+ if (ret < 0)
+ return ret;
+
+ ret = kmb_pipe_init(&kmb_isp->try_pipe, dev, xlink_cam);
+ if (ret < 0)
+ goto error_cleanup_active_pipeline;
+
+ INIT_LIST_HEAD(&kmb_isp->meta_params_pending_q);
+ INIT_LIST_HEAD(&kmb_isp->meta_params_process_q);
+ INIT_LIST_HEAD(&kmb_isp->meta_stats_pending_q);
+ INIT_LIST_HEAD(&kmb_isp->meta_stats_process_q);
+
+ kmb_isp->isp_streaming = false;
+
+ kmb_isp->dev = dev;
+ kmb_isp->xlink_cam = xlink_cam;
+
+ ret = kmb_cam_xlink_alloc_channel(kmb_isp->xlink_cam);
+ if (ret < 0)
+ goto error_cleanup_try_pipeline;
+
+ kmb_isp->config_chan_id = ret;
+
+ /* Video nodes are connected only to active pipes */
+ kmb_isp->params.dma_dev = dev;
+ kmb_isp->params.pipe = &kmb_isp->active_pipe;
+ kmb_isp->params.queue_ops = &isp_params_queue_ops;
+ kmb_isp->params.priv = kmb_isp;
+ kmb_isp->params.type = KMB_METADATA_PARAMS;
+ ret = kmb_metadata_init(&kmb_isp->params);
+ if (ret < 0)
+ goto error_free_config_channel_id;
+
+ kmb_isp->stats.dma_dev = dev;
+ kmb_isp->stats.pipe = &kmb_isp->active_pipe;
+ kmb_isp->stats.queue_ops = &isp_stats_queue_ops;
+ kmb_isp->stats.priv = kmb_isp;
+ kmb_isp->stats.type = KMB_METADATA_STATS;
+ ret = kmb_metadata_init(&kmb_isp->stats);
+ if (ret < 0)
+ goto error_metadata_params_cleanup;
+
+ ret = kmb_cam_xlink_alloc_channel(kmb_isp->xlink_cam);
+ if (ret < 0)
+ goto error_metadata_stats_cleanup;
+
+ kmb_isp->capture_chan_id = ret;
+ kmb_isp->capture.dma_dev = dev;
+ kmb_isp->capture.pipe = &kmb_isp->active_pipe;
+ kmb_isp->capture.chan_id = kmb_isp->capture_chan_id;
+ kmb_isp->capture.xlink_cam = kmb_isp->xlink_cam;
+ ret = kmb_video_init(&kmb_isp->capture, "kmb-video-capture");
+ if (ret < 0)
+ goto error_free_capture_channel_id;
+
+ kmb_isp->csi2_config = *csi2_config;
+
+ mutex_init(&kmb_isp->lock);
+ mutex_init(&kmb_isp->meta_q_lock);
+
return 0;
+
+error_free_capture_channel_id:
+ kmb_cam_xlink_free_channel(kmb_isp->xlink_cam,
+ kmb_isp->capture_chan_id);
+error_metadata_stats_cleanup:
+ kmb_metadata_cleanup(&kmb_isp->stats);
+error_metadata_params_cleanup:
+ kmb_metadata_cleanup(&kmb_isp->params);
+error_free_config_channel_id:
+ kmb_cam_xlink_free_channel(kmb_isp->xlink_cam,
+ kmb_isp->config_chan_id);
+error_cleanup_try_pipeline:
+ kmb_pipe_cleanup(&kmb_isp->try_pipe);
+error_cleanup_active_pipeline:
+ kmb_pipe_cleanup(&kmb_isp->active_pipe);
+
+ return ret;
}
/**
@@ -27,7 +1249,20 @@ int kmb_isp_init(struct kmb_isp *kmb_isp, struct device *dev,
* @kmb_isp: Pointer to kmb isp sub-device
*/
void kmb_isp_cleanup(struct kmb_isp *kmb_isp)
-{ }
+{
+ kmb_video_cleanup(&kmb_isp->capture);
+ kmb_cam_xlink_free_channel(kmb_isp->xlink_cam,
+ kmb_isp->capture_chan_id);
+
+ kmb_metadata_cleanup(&kmb_isp->stats);
+ kmb_metadata_cleanup(&kmb_isp->params);
+
+ kmb_cam_xlink_free_channel(kmb_isp->xlink_cam,
+ kmb_isp->config_chan_id);
+
+ mutex_destroy(&kmb_isp->meta_q_lock);
+ mutex_destroy(&kmb_isp->lock);
+}
/**
* kmb_isp_register_entities - Register entities
@@ -42,7 +1277,107 @@ void kmb_isp_cleanup(struct kmb_isp *kmb_isp)
int kmb_isp_register_entities(struct kmb_isp *kmb_isp,
struct v4l2_device *v4l2_dev)
{
+ struct media_pad *pads = kmb_isp->pads;
+ struct device *dev = kmb_isp->dev;
+ int ret;
+
+ /* Memory for xlink messages */
+ kmb_isp->msg_vaddr = NULL;
+ kmb_isp->msg_phy_addr = 0;
+ kmb_isp->msg_vaddr = dma_alloc_coherent(kmb_isp->dev,
+ KMB_ISP_CH_DATA_SIZE,
+ &kmb_isp->msg_phy_addr, 0);
+ if (!kmb_isp->msg_vaddr) {
+ dev_err(dev, "Fail to allocate msg dma memory");
+ return -ENOMEM;
+ }
+
+ kmb_isp->subdev.internal_ops = &kmb_isp_internal_ops;
+ kmb_isp->subdev.flags |= V4L2_SUBDEV_FL_HAS_DEVNODE |
+ V4L2_SUBDEV_FL_HAS_EVENTS;
+ kmb_isp->subdev.entity.function = MEDIA_ENT_F_PROC_VIDEO_ISP;
+ strscpy(kmb_isp->subdev.name, KMB_ISP_DRV_NAME,
+ sizeof(kmb_isp->subdev.name));
+
+ pads[KMB_ISP_SINK_PAD_SENSOR].flags = MEDIA_PAD_FL_SINK;
+ pads[KMB_ISP_SINK_PAD_PARAM].flags = MEDIA_PAD_FL_SINK;
+ pads[KMB_ISP_SRC_PAD_STATS].flags = MEDIA_PAD_FL_SOURCE;
+ pads[KMB_ISP_SRC_PAD_VID].flags = MEDIA_PAD_FL_SOURCE;
+ ret = media_entity_pads_init(&kmb_isp->subdev.entity,
+ KMB_ISP_PADS_NUM, pads);
+ if (ret < 0) {
+ dev_err(dev, "Fail to init media entity");
+ return ret;
+ }
+
+ ret = v4l2_device_register_subdev(v4l2_dev, &kmb_isp->subdev);
+ if (ret < 0) {
+ dev_err(dev, "Fail to register media entity");
+ return ret;
+ }
+
+ /* Register video nodes */
+ ret = kmb_metadata_register(&kmb_isp->params, v4l2_dev);
+ if (ret < 0)
+ goto error_unregister_subdev;
+
+ ret = media_create_pad_link(&kmb_isp->params.video.entity,
+ 0,
+ &kmb_isp->subdev.entity,
+ KMB_ISP_SINK_PAD_PARAM,
+ MEDIA_LNK_FL_IMMUTABLE |
+ MEDIA_LNK_FL_ENABLED);
+ if (ret < 0) {
+ dev_err(kmb_isp->dev, "Fail to link %s->%s entities",
+ kmb_isp->params.video.entity.name,
+ kmb_isp->subdev.entity.name);
+ goto error_unregister_params;
+ }
+
+ ret = kmb_metadata_register(&kmb_isp->stats, v4l2_dev);
+ if (ret < 0)
+ goto error_unregister_params;
+
+ ret = media_create_pad_link(&kmb_isp->subdev.entity,
+ KMB_ISP_SRC_PAD_STATS,
+ &kmb_isp->stats.video.entity, 0,
+ MEDIA_LNK_FL_IMMUTABLE |
+ MEDIA_LNK_FL_ENABLED);
+ if (ret < 0) {
+ dev_err(kmb_isp->dev, "Fail to link %s->%s entities",
+ kmb_isp->stats.video.entity.name,
+ kmb_isp->subdev.entity.name);
+ goto error_unregister_stats;
+ }
+
+ ret = kmb_video_register(&kmb_isp->capture, v4l2_dev);
+ if (ret < 0)
+ goto error_unregister_stats;
+
+ ret = media_create_pad_link(&kmb_isp->subdev.entity,
+ KMB_ISP_SRC_PAD_VID,
+ &kmb_isp->capture.video->entity, 0,
+ MEDIA_LNK_FL_IMMUTABLE |
+ MEDIA_LNK_FL_ENABLED);
+ if (ret < 0) {
+ dev_err(kmb_isp->dev, "Fail to link %s->%s entities",
+ kmb_isp->subdev.entity.name,
+ kmb_isp->capture.video->entity.name);
+ goto error_unregister_video;
+ }
+
return 0;
+
+error_unregister_video:
+ kmb_video_unregister(&kmb_isp->capture);
+error_unregister_stats:
+ kmb_metadata_unregister(&kmb_isp->stats);
+error_unregister_params:
+ kmb_metadata_unregister(&kmb_isp->params);
+error_unregister_subdev:
+ v4l2_device_unregister_subdev(&kmb_isp->subdev);
+
+ return ret;
}
/**
@@ -50,4 +1385,13 @@ int kmb_isp_register_entities(struct kmb_isp *kmb_isp,
* @kmb_isp: pointer to kmb isp device
*/
void kmb_isp_unregister_entities(struct kmb_isp *kmb_isp)
-{ }
+{
+ dma_free_coherent(kmb_isp->dev, KMB_ISP_CH_DATA_SIZE,
+ kmb_isp->msg_vaddr, kmb_isp->msg_phy_addr);
+
+ kmb_video_unregister(&kmb_isp->capture);
+ kmb_metadata_unregister(&kmb_isp->stats);
+ kmb_metadata_unregister(&kmb_isp->params);
+
+ v4l2_device_unregister_subdev(&kmb_isp->subdev);
+}
diff --git a/drivers/media/platform/keembay-camera/keembay-isp.h b/drivers/media/platform/keembay-camera/keembay-isp.h
index 35af6c644676..883eafada164 100644
--- a/drivers/media/platform/keembay-camera/keembay-isp.h
+++ b/drivers/media/platform/keembay-camera/keembay-isp.h
@@ -10,12 +10,17 @@
#include <media/v4l2-ctrls.h>
#include <media/v4l2-subdev.h>
-#define KMB_ISP_DRV_NAME "keembay-camera-isp"
+#include "keembay-metadata.h"
+#include "keembay-pipeline.h"
+#include "keembay-video.h"
#define KMB_ISP_SINK_PAD_SENSOR 0
-#define KMB_ISP_SINK_PAD_CFG 1
-#define KMB_ISP_SRC_PAD_VID 2
-#define KMB_ISP_PADS_NUM 3
+#define KMB_ISP_SINK_PAD_PARAM 1
+#define KMB_ISP_SRC_PAD_STATS 2
+#define KMB_ISP_SRC_PAD_VID 3
+#define KMB_ISP_PADS_NUM 4
+
+#define KMB_ISP_MAX_DEST_FMTS 5
/**
* struct kmb_isp_csi2_config - Isp csi2 configuration
@@ -28,25 +33,52 @@ struct kmb_isp_csi2_config {
};
/**
+ * struct kmb_isp_source_format - Isp source format
+ * @code: V4l2 media bus code for the format
+ * @bayer_pattern: Bayer format
+ * @bpp: Bits per pixel
+ * @rx_data_type: Receiver data type
+ * @dest_fmts: Supported destination formats
+ */
+struct kmb_isp_source_format {
+ u32 code;
+ u32 bayer_pattern;
+ u32 bpp;
+ enum kmb_ic_mipi_rx_data_type rx_data_type;
+ u32 dest_fmts[KMB_ISP_MAX_DEST_FMTS];
+};
+
+/**
* struct kmb_isp - Keem Bay camera ISP device structure
* @dev: Pointer to basic device structure
* @lock: Mutex serilizing access to ISP device
* @thread: Pointer to worker thread data
- * @xlink_cam: Xlink camera communication handler
+ * @xlink_cam: Pointer to xlink camera communication handler
* @msg_phy_addr: ISP channel physical CMA address
* @msg_vaddr: ISP channel virtual CMA address
- * @cfg_q_lock: Mutex to serialize access to isp cfg bufferss queue
- * @isp_cfgs_queue: Isp cfg buffers queue
+ * @meta_q_lock: Mutex to protect metadata buffers queues
+ * @meta_params_pending_q: Metadata params pending queue
+ * @meta_params_process_q: Metadata params processing queue
+ * @meta_stats_pending_q: Metadata statistics pending queue
+ * @meta_stats_process_q: Metadata statistics processing queue
* @isp_streaming: Flag to indicate ISP state
* @source_streaming: Flag to indicate source state
* @source_stopped: Completion to wait until VPU source is stopped
* @subdev: V4L2 sub-device
* @pads: Array of supported isp pads
+ * @active_pipe: VPU pipeline instance used for active format and streaming
* @active_pad_fmt: Array holding active pad formats
+ * @try_pipe: VPU pipeline instance used for try format
* @try_pad_fmt: Array holding try pad formats
* @csi2_config: CSI2 configuration
* @source_fmt: Pointer to isp source format
- * @sequence: frame sequence number
+ * @pipe_cfg: VPU pipeline configuration structure
+ * @config_chan_id: Isp config channel id
+ * @stats: Statistics video node
+ * @params: Params video node
+ * @capture_chan_id: Capture xlink channel id
+ * @capture: Isp capture video node
+ * @sequence: Frame sequence number
*/
struct kmb_isp {
struct device *dev;
@@ -58,8 +90,11 @@ struct kmb_isp {
dma_addr_t msg_phy_addr;
void *msg_vaddr;
- struct mutex cfg_q_lock;
- struct list_head isp_cfgs_queue;
+ struct mutex meta_q_lock;
+ struct list_head meta_params_pending_q;
+ struct list_head meta_params_process_q;
+ struct list_head meta_stats_pending_q;
+ struct list_head meta_stats_process_q;
bool isp_streaming;
bool source_streaming;
@@ -68,13 +103,24 @@ struct kmb_isp {
struct v4l2_subdev subdev;
struct media_pad pads[KMB_ISP_PADS_NUM];
+ struct kmb_pipeline active_pipe;
struct v4l2_subdev_format active_pad_fmt[KMB_ISP_PADS_NUM];
+ struct kmb_pipeline try_pipe;
struct v4l2_subdev_format try_pad_fmt[KMB_ISP_PADS_NUM];
struct kmb_isp_csi2_config csi2_config;
const struct kmb_isp_source_format *source_fmt;
+ struct kmb_pipe_config_evs pipe_cfg;
+
+ unsigned int config_chan_id;
+ struct kmb_metadata stats;
+ struct kmb_metadata params;
+
+ unsigned int capture_chan_id;
+ struct kmb_video capture;
+
u32 sequence;
};
diff --git a/drivers/media/platform/keembay-camera/keembay-metadata.c b/drivers/media/platform/keembay-camera/keembay-metadata.c
new file mode 100644
index 000000000000..a1df746d9582
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-metadata.c
@@ -0,0 +1,45 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Intel Keem Bay camera ISP metadata video node.
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+#include "keembay-metadata.h"
+
+/**
+ * kmb_video_init - Initialize entity
+ * @kmb_meta: pointer to kmb isp config device
+ *
+ * Return: 0 if successful, error code otherwise.
+ */
+int kmb_metadata_init(struct kmb_metadata *kmb_meta)
+{
+ return 0;
+}
+
+/**
+ * kmb_metadata_cleanup - Free resources associated with entity
+ * @kmb_meta: pointer to kmb isp config device
+ */
+void kmb_metadata_cleanup(struct kmb_metadata *kmb_meta)
+{ }
+
+/**
+ * kmb_metadata_register - Register V4L2 device
+ * @kmb_meta: pointer to kmb isp config device
+ * @v4l2_dev: pointer to V4L2 device drivers
+ *
+ * Return: 0 if successful, error code otherwise.
+ */
+int kmb_metadata_register(struct kmb_metadata *kmb_meta,
+ struct v4l2_device *v4l2_dev)
+{
+ return 0;
+}
+
+/**
+ * kmb_metadata_unregister - Unregister V4L device
+ * @kmb_meta: pointer to kmb isp config device
+ */
+void kmb_metadata_unregister(struct kmb_metadata *kmb_meta)
+{ }
diff --git a/drivers/media/platform/keembay-camera/keembay-metadata.h b/drivers/media/platform/keembay-camera/keembay-metadata.h
new file mode 100644
index 000000000000..88e85d3caba0
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-metadata.h
@@ -0,0 +1,150 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Intel Keem Bay camera ISP metadata video node.
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+#ifndef KEEMBAY_METADATA_H
+#define KEEMBAY_METADATA_H
+
+#include <media/v4l2-dev.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "keembay-vpu-isp.h"
+
+/**
+ * enum kmb_metadata_table_type - Keembay metadata table type
+ * @KMB_METADATA_TABLE_LSC: Lens shading table
+ * @KMB_METADATA_TABLE_SDEFECT: Static defect pixel table
+ * @KMB_METADATA_TABLE_LCA: Lateral цhroma аberration table
+ * @KMB_METADATA_TABLE_HDR: HDR table
+ * @KMB_METADATA_TABLE_SHARP: Shartpnes table
+ * @KMB_METADATA_TABLE_COLOR_CUMB: Color combination table
+ * @KMB_METADATA_TABLE_TNF0: Temporal denoise first table
+ * @KMB_METADATA_TABLE_TNF1: Temporal denoise second table
+ * @KMB_METADATA_TABLE_WARP: Warp mesh table
+ */
+enum kmb_metadata_table_type {
+ KMB_METADATA_TABLE_LSC = 0,
+ KMB_METADATA_TABLE_SDEFECT = 1,
+ KMB_METADATA_TABLE_LCA = 2,
+ KMB_METADATA_TABLE_HDR = 3,
+ KMB_METADATA_TABLE_SHARP = 4,
+ KMB_METADATA_TABLE_COLOR_CUMB = 5,
+ KMB_METADATA_TABLE_LUT = 6,
+ KMB_METADATA_TABLE_TNF0 = 7,
+ KMB_METADATA_TABLE_TNF1 = 8,
+ KMB_METADATA_TABLE_WARP = 10,
+ KMB_METADATA_TABLE_MAX = 11,
+};
+
+/**
+ * enum kmb_metadata_type - Keembay metadata type
+ * @KMB_METADATA_PARAMS: Keembay metadata parameters
+ * @KMB_METADATA_STATS: Keembay metadata statistics
+ */
+enum kmb_metadata_type {
+ KMB_METADATA_PARAMS,
+ KMB_METADATA_STATS,
+};
+
+/**
+ * struct kmb_metadata_table - Keembay metadata table
+ * @refcount: Metadata table reference count
+ * @dma_addr: Physical address of the table
+ * @cpu_addr: Virtual address of the table
+ * @pool: Dma pool from where table was allocated
+ */
+struct kmb_metadata_table {
+ struct kref refcount;
+ dma_addr_t dma_addr;
+ void *cpu_addr;
+ struct dma_pool *pool;
+};
+
+/**
+ * struct kmb_metadata_buf - Keembay metadata buffer handle
+ * @vb: Video buffer for v4l2
+ * @type: Metadata type
+ * @stats: Statistics physical addresses
+ * @raw: VPU raw statistics physical addresses
+ * @dehaze_stats_addr: VPU dehaze statistics physical address
+ * @params: VPU ISP parameters
+ * @isp: VPU ISP parameters virtual address
+ * @dma_addr_isp: VPU ISP parameters physical address
+ * @tab: Metadata tables
+ * @list: List for buffer queue
+ */
+struct kmb_metadata_buf {
+ struct vb2_v4l2_buffer vb;
+ enum kmb_metadata_type type;
+ union {
+ struct {
+ struct kmb_vpu_raw_stats raw[KMB_VPU_MAX_EXPOSURES];
+ u64 dehaze_stats_addr;
+ } stats;
+ struct {
+ struct kmb_vpu_isp_params *isp;
+ dma_addr_t dma_addr_isp;
+ struct kmb_metadata_table *tab[KMB_METADATA_TABLE_MAX];
+ } params;
+ };
+ struct list_head list;
+};
+
+/**
+ * struct kmb_metabuf_queue_ops - Keembay metadata queue operations
+ * @queue: queue an metadata buffer
+ * @flish: discard all metadata buffers
+ */
+struct kmb_metabuf_queue_ops {
+ int (*queue)(void *priv, struct kmb_metadata_buf *buf);
+ void (*flush)(void *priv);
+};
+
+/**
+ * struct kmb_metadata - Keembay metadata device
+ * @lock: mutex to protect keembay metadata device
+ * @video: pointer to V4L2 video device node
+ * @dma_dev: pointer to dma device
+ * @pad: media pad graph objects
+ * @vb2_q: V4L2 Video buffer queue
+ * @type: Metadata type
+ * @pipe: pointer to KMB pipeline object
+ * @priv: pointer to private data
+ * @queue_ops: Metadata buffer queue operations
+ * @table_pools_refcnt: Table pool reference count
+ * @table_pool: ISP tables dma pool
+ * @last_buf: Pointer to last enqueued buffer
+ * @format: Active format
+ */
+struct kmb_metadata {
+ struct mutex lock;
+ struct video_device video;
+ struct device *dma_dev;
+ struct media_pad pad;
+ struct vb2_queue vb2_q;
+ enum kmb_metadata_type type;
+
+ struct kmb_pipeline *pipe;
+
+ void *priv;
+ const struct kmb_metabuf_queue_ops *queue_ops;
+
+ unsigned int table_pools_refcnt;
+ struct dma_pool *table_pool[KMB_METADATA_TABLE_MAX];
+
+ struct kmb_metadata_buf *last_buf;
+
+ struct v4l2_meta_format format;
+};
+
+int kmb_metadata_init(struct kmb_metadata *kmb_meta);
+void kmb_metadata_cleanup(struct kmb_metadata *kmb_meta);
+
+int kmb_metadata_register(struct kmb_metadata *kmb_meta,
+ struct v4l2_device *v4l2_dev);
+void kmb_metadata_unregister(struct kmb_metadata *kmb_meta);
+
+#endif /* KEEMBAY_METADATA_H */
diff --git a/drivers/media/platform/keembay-camera/keembay-pipeline.c b/drivers/media/platform/keembay-camera/keembay-pipeline.c
new file mode 100644
index 000000000000..1ebaa900e4dc
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-pipeline.c
@@ -0,0 +1,76 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Intel Keem Bay camera pipeline.
+ *
+ * Copyright (C) 2020 Intel Corporation
+ */
+#include <media/v4l2-device.h>
+
+#include "keembay-pipeline.h"
+#include "keembay-vpu-cmd.h"
+
+/**
+ * kmb_pipe_init - Initialize KMB Pipeline
+ * @pipe: pointer to pipeline object
+ * @dev: pointer to device
+ * @xlink_cam: pointer to xlink cam handle
+ *
+ * Return: 0 if successful, error code otherwise.
+ */
+int kmb_pipe_init(struct kmb_pipeline *pipe, struct device *dev,
+ struct kmb_xlink_cam *xlink_cam)
+{
+ return 0;
+}
+
+/**
+ * kmb_pipe_cleanup - Cleanup KMB Pipeline
+ * @pipe: pointer to pipeline object
+ */
+void kmb_pipe_cleanup(struct kmb_pipeline *pipe)
+{ }
+
+/**
+ * kmb_pipe_request - Request a pipeline
+ * @pipe: pointer to pipeline object
+ *
+ * Return: 0 if successful, error code otherwise.
+ */
+int kmb_pipe_request(struct kmb_pipeline *pipe)
+{
+ return 0;
+}
+
+/**
+ * kmb_pipe_release - Release a pipeline
+ * @pipe: pointer to pipeline object
+ */
+void kmb_pipe_release(struct kmb_pipeline *pipe)
+{ }
+
+/**
+ * kmb_pipe_config_dest - Configure pipeline destination information
+ * @pipe: pointer to pipeline object
+ * @output_id: destination id
+ * @channel_cfg: pointer to channel configuration
+ */
+void kmb_pipe_config_dest(struct kmb_pipeline *pipe, unsigned int output_id,
+ struct kmb_channel_cfg *channel_cfg)
+{ }
+
+/**
+ * kmb_pipe_config_src - Configure pipeline source information
+ * @pipe: pointer to pipeline object
+ * @pipe_cfg: pointer to pipeline configuration
+ *
+ * Configure pipeline source information. Sending source configuration and
+ * getting destination restrictions. After this call all destination data is
+ * initialized. Changing state to CONFIGURED.
+ *
+ * Return: 0 if successful, error code otherwise.
+ */
+int kmb_pipe_config_src(struct kmb_pipeline *pipe,
+ struct kmb_pipe_config_evs *pipe_cfg)
+{
+ return 0;
+}
diff --git a/drivers/media/platform/keembay-camera/keembay-pipeline.h b/drivers/media/platform/keembay-camera/keembay-pipeline.h
new file mode 100644
index 000000000000..83ff94d11b34
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-pipeline.h
@@ -0,0 +1,71 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Intel Keem Bay camera pipeline.
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+#ifndef KEEMBAY_PIPELINE_H
+#define KEEMBAY_PIPELINE_H
+
+#include "keembay-vpu-pipe.h"
+
+/**
+ * enum kmb_pipe_state - KMB pipeline state
+ * @KMB_PIPE_STATE_UNCONFIGURED: Pipeline is unconfigured only configure can be
+ * called in this state
+ * @KMB_PIPE_STATE_CONFIGURED: Pipeline is configured. Pipeline can be
+ * re-configured, build or destroyed
+ * @KMB_PIPE_STATE_BUILT: Pipeline is built and ready for streaming.
+ * Pipeline destroy or start stream can be called
+ * @KMB_PIPE_STATE_STREAMING: Pipeline is in streaming state only stop
+ * stream can be called.
+ */
+enum kmb_pipe_state {
+ KMB_PIPE_STATE_UNCONFIGURED,
+ KMB_PIPE_STATE_CONFIGURED,
+ KMB_PIPE_STATE_BUILT,
+ KMB_PIPE_STATE_STREAMING,
+};
+
+/**
+ * struct kmb_pipeline - KMB Pipeline
+ * @lock: Mutex to serialize access to kmb pipeline object
+ * @dev: Pointer to device
+ * @media_pipe: Media pipeline
+ * @state: Pipeline state
+ * @pipe_cfg: VPU pipeline configuration
+ * @pipe_cfg_paddr: VPU pipeline configuration physical address
+ * @pending: Number of media graph entities expected on streaming
+ * @streaming: Number of entities in streaming state
+ * @xlink_cam: Pointer to xlink camera communication handler
+ */
+struct kmb_pipeline {
+ struct mutex lock;
+ struct device *dev;
+ struct media_pipeline media_pipe;
+
+ enum kmb_pipe_state state;
+
+ struct kmb_pipe_config_evs *pipe_cfg;
+ dma_addr_t pipe_cfg_paddr;
+
+ unsigned int pending;
+ unsigned int streaming;
+
+ struct kmb_xlink_cam *xlink_cam;
+};
+
+int kmb_pipe_init(struct kmb_pipeline *pipe,
+ struct device *dev,
+ struct kmb_xlink_cam *xlink_cam);
+void kmb_pipe_cleanup(struct kmb_pipeline *pipe);
+
+int kmb_pipe_request(struct kmb_pipeline *pipe);
+void kmb_pipe_release(struct kmb_pipeline *pipe);
+
+void kmb_pipe_config_dest(struct kmb_pipeline *pipe, unsigned int output_id,
+ struct kmb_channel_cfg *channel_cfg);
+int kmb_pipe_config_src(struct kmb_pipeline *pipe,
+ struct kmb_pipe_config_evs *pipe_cfg);
+
+#endif /* KEEMBAY_PIPELINE_H */
diff --git a/drivers/media/platform/keembay-camera/keembay-video.c b/drivers/media/platform/keembay-camera/keembay-video.c
new file mode 100644
index 000000000000..02f4d97e16fb
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-video.c
@@ -0,0 +1,46 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Intel Keem Bay camera Video node.
+ *
+ * Copyright (C) 2018-2020 Intel Corporation
+ */
+#include "keembay-video.h"
+
+/**
+ * kmb_video_init - Initialize entity
+ * @kmb_vid: pointer to kmb video device
+ * @name: entity name
+ *
+ * Return: 0 if successful, error code otherwise.
+ */
+int kmb_video_init(struct kmb_video *kmb_vid, const char *name)
+{
+ return 0;
+}
+
+/**
+ * kmb_video_cleanup - Free resources associated with entity
+ * @kmb_vid: pointer to kmb video device
+ */
+void kmb_video_cleanup(struct kmb_video *kmb_vid)
+{ }
+
+/**
+ * kmb_video_register - Register V4L2 device
+ * @kmb_vid: pointer to kmb video device
+ * @v4l2_dev: pointer to V4L2 device drivers
+ *
+ * Return: 0 if successful, error code otherwise.
+ */
+int kmb_video_register(struct kmb_video *kmb_vid,
+ struct v4l2_device *v4l2_dev)
+{
+ return 0;
+}
+
+/**
+ * kmb_video_unregister - Unregister V4L device
+ * @kmb_vid: pointer to kmb video device
+ */
+void kmb_video_unregister(struct kmb_video *kmb_vid)
+{ }
diff --git a/drivers/media/platform/keembay-camera/keembay-video.h b/drivers/media/platform/keembay-camera/keembay-video.h
new file mode 100644
index 000000000000..2aebbb37424b
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-video.h
@@ -0,0 +1,85 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Intel Keem Bay camera video node.
+ *
+ * Copyright (C) 2020 Intel Corporation
+ */
+#ifndef KEEMBAY_VIDEO_H
+#define KEEMBAY_VIDEO_H
+
+#include <media/v4l2-dev.h>
+#include <media/v4l2-subdev.h>
+#include <media/videobuf2-v4l2.h>
+
+#include "keembay-cam-xlink.h"
+
+/**
+ * struct kmb_frame_buffer - KMB frame buffer structure
+ * @vb: Video buffer for v4l2
+ * @addr: Array of dma buffer plane address
+ * @list: Frame buffer list
+ */
+struct kmb_frame_buffer {
+ struct vb2_v4l2_buffer vb;
+ dma_addr_t addr[3];
+ struct list_head list;
+};
+
+/**
+ * struct kmb_video - KMB Video device structure
+ * @lock: Mutex serializing kmb video device ops
+ * @video_lock: Mutex serializing video operations
+ * @video: Pointer to V4L2 sub-device
+ * @pad: Media pad graph objects
+ * @dma_dev: Pointer to dma device
+ * @pipe: Pointer to kmb media pipeline
+ * @chan: Pointer to xlink channel
+ */
+struct kmb_video {
+ struct mutex lock; /* Lock protecting kmb video device */
+ struct mutex video_lock; /* Lock serializing video device operations */
+ struct video_device *video;
+ struct media_pad pad;
+ struct device *dma_dev;
+ struct kmb_pipeline *pipe;
+ struct kmb_xlink_cam *xlink_cam;
+ unsigned int chan_id;
+};
+
+/**
+ * struct kmb_video_fh - KMB video file handler
+ * @fh: V4L2 file handler
+ * @kmb_vid: Pointer to KMB video device
+ * @lock: Mutex serializing access to fh
+ * @vb2_lock: Mutex serializing access to vb2 queue
+ * @vb2_q: Video buffer queue
+ * @active_fmt: Active format
+ @pix: Mplane active pixel format
+ @info: Active kmb format info
+ * @contiguous_memory: Flag to enable contiguous memory allocation
+ * @dma_queue: DMA buffers queue
+ * @thread: Pointer to worker thread data
+ */
+struct kmb_video_fh {
+ struct v4l2_fh fh;
+ struct kmb_video *kmb_vid;
+ struct mutex lock; /* Lock protecting fh operations */
+ struct mutex vb2_lock; /* Lock protecting video buffer queue */
+ struct vb2_queue vb2_q;
+ struct {
+ struct v4l2_pix_format_mplane pix;
+ const struct kmb_video_fmt_info *info;
+ } active_fmt;
+ bool contiguous_memory;
+ struct list_head dma_queue;
+ struct task_struct *thread;
+};
+
+int kmb_video_init(struct kmb_video *kmb_vid, const char *name);
+void kmb_video_cleanup(struct kmb_video *kmb_vid);
+
+int kmb_video_register(struct kmb_video *kmb_vid,
+ struct v4l2_device *v4l2_dev);
+void kmb_video_unregister(struct kmb_video *kmb_vid);
+
+#endif /* KEEMBAY_VIDEO_H */
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread* Re: [PATCH 06/10] media: Keem Bay Camera: Add ISP sub-device
2021-03-19 18:06 ` [PATCH 06/10] media: Keem Bay Camera: Add ISP sub-device Martina Krasteva
@ 2021-04-09 8:31 ` Sakari Ailus
2021-04-09 10:17 ` Martina Krasteva
0 siblings, 1 reply; 28+ messages in thread
From: Sakari Ailus @ 2021-04-09 8:31 UTC (permalink / raw)
To: Martina Krasteva
Cc: linux-media, mchehab, robh+dt, devicetree, daniele.alessandrelli,
paul.j.murphy, gjorgjix.rosikopulos
Hi Martina,
On Fri, Mar 19, 2021 at 06:06:28PM +0000, Martina Krasteva wrote:
...
> +/**
> + * kmb_cam_xlink_alloc_channel - Allocate xlink camera channel id
> + * @xlink_cam: Pointer to xlink camera handle
> + *
> + * Each xlink channel (except main control) should have unieque id
> + *
> + * Return: Channel id, negative error otherwise
> + */
> +int kmb_cam_xlink_alloc_channel(struct kmb_xlink_cam *xlink_cam)
> +{
> + int chan_id;
> +
> + chan_id = ida_alloc_range(&xlink_cam->channel_ids,
> + KMB_CAM_XLINK_CHAN_ID_BASE,
> + U16_MAX, GFP_KERNEL);
return ida_alloc_range(...);
> +
> + return chan_id;
> +}
...
> +
> +/**
> + * struct kmb_video_fh - KMB video file handler
> + * @fh: V4L2 file handler
> + * @kmb_vid: Pointer to KMB video device
> + * @lock: Mutex serializing access to fh
> + * @vb2_lock: Mutex serializing access to vb2 queue
> + * @vb2_q: Video buffer queue
> + * @active_fmt: Active format
> + @pix: Mplane active pixel format
> + @info: Active kmb format info
@active_fmt.info: ...
etc.
> + * @contiguous_memory: Flag to enable contiguous memory allocation
> + * @dma_queue: DMA buffers queue
> + * @thread: Pointer to worker thread data
> + */
> +struct kmb_video_fh {
> + struct v4l2_fh fh;
> + struct kmb_video *kmb_vid;
> + struct mutex lock; /* Lock protecting fh operations */
> + struct mutex vb2_lock; /* Lock protecting video buffer queue */
> + struct vb2_queue vb2_q;
> + struct {
> + struct v4l2_pix_format_mplane pix;
> + const struct kmb_video_fmt_info *info;
> + } active_fmt;
> + bool contiguous_memory;
> + struct list_head dma_queue;
> + struct task_struct *thread;
> +};
> +
> +int kmb_video_init(struct kmb_video *kmb_vid, const char *name);
> +void kmb_video_cleanup(struct kmb_video *kmb_vid);
> +
> +int kmb_video_register(struct kmb_video *kmb_vid,
> + struct v4l2_device *v4l2_dev);
> +void kmb_video_unregister(struct kmb_video *kmb_vid);
> +
> +#endif /* KEEMBAY_VIDEO_H */
--
Kind regards,
Sakari Ailus
^ permalink raw reply [flat|nested] 28+ messages in thread* RE: [PATCH 06/10] media: Keem Bay Camera: Add ISP sub-device
2021-04-09 8:31 ` Sakari Ailus
@ 2021-04-09 10:17 ` Martina Krasteva
0 siblings, 0 replies; 28+ messages in thread
From: Martina Krasteva @ 2021-04-09 10:17 UTC (permalink / raw)
To: 'Sakari Ailus'
Cc: linux-media, mchehab, robh+dt, devicetree, daniele.alessandrelli,
paul.j.murphy, gjorgjix.rosikopulos
Hi Sakari,
Thank you for the review
>
> Hi Martina,
>
> On Fri, Mar 19, 2021 at 06:06:28PM +0000, Martina Krasteva wrote:
> ...
> > +/**
> > + * kmb_cam_xlink_alloc_channel - Allocate xlink camera channel id
> > + * @xlink_cam: Pointer to xlink camera handle
> > + *
> > + * Each xlink channel (except main control) should have unieque id
> > + *
> > + * Return: Channel id, negative error otherwise */ int
> > +kmb_cam_xlink_alloc_channel(struct kmb_xlink_cam *xlink_cam) {
> > + int chan_id;
> > +
> > + chan_id = ida_alloc_range(&xlink_cam->channel_ids,
> > + KMB_CAM_XLINK_CHAN_ID_BASE,
> > + U16_MAX, GFP_KERNEL);
>
> return ida_alloc_range(...);
>
Will be fixed in next version
> > +
> > + return chan_id;
> > +}
>
> ...
>
> > +
> > +/**
> > + * struct kmb_video_fh - KMB video file handler
> > + * @fh: V4L2 file handler
> > + * @kmb_vid: Pointer to KMB video device
> > + * @lock: Mutex serializing access to fh
> > + * @vb2_lock: Mutex serializing access to vb2 queue
> > + * @vb2_q: Video buffer queue
> > + * @active_fmt: Active format
> > + @pix: Mplane active pixel format
> > + @info: Active kmb format info
>
> @active_fmt.info: ...
>
> etc.
>
Oops, will be fixed
> > + * @contiguous_memory: Flag to enable contiguous memory allocation
> > + * @dma_queue: DMA buffers queue
> > + * @thread: Pointer to worker thread data */ struct kmb_video_fh {
> > + struct v4l2_fh fh;
> > + struct kmb_video *kmb_vid;
> > + struct mutex lock; /* Lock protecting fh operations */
> > + struct mutex vb2_lock; /* Lock protecting video buffer queue */
> > + struct vb2_queue vb2_q;
> > + struct {
> > + struct v4l2_pix_format_mplane pix;
> > + const struct kmb_video_fmt_info *info;
> > + } active_fmt;
> > + bool contiguous_memory;
> > + struct list_head dma_queue;
> > + struct task_struct *thread;
> > +};
> > +
> > +int kmb_video_init(struct kmb_video *kmb_vid, const char *name); void
> > +kmb_video_cleanup(struct kmb_video *kmb_vid);
> > +
> > +int kmb_video_register(struct kmb_video *kmb_vid,
> > + struct v4l2_device *v4l2_dev); void
> > +kmb_video_unregister(struct kmb_video *kmb_vid);
> > +
> > +#endif /* KEEMBAY_VIDEO_H */
>
> --
> Kind regards,
>
> Sakari Ailus
Best Regards,
Martina
^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 07/10] media: Keem Bay Camera: Add pipeline support
2021-03-19 18:06 [PATCH 00/10] Keem Bay Camera Subsystem Martina Krasteva
` (5 preceding siblings ...)
2021-03-19 18:06 ` [PATCH 06/10] media: Keem Bay Camera: Add ISP sub-device Martina Krasteva
@ 2021-03-19 18:06 ` Martina Krasteva
2021-03-19 18:06 ` [PATCH 08/10] media: Keem Bay Camera: Add capture video node Martina Krasteva
` (3 subsequent siblings)
10 siblings, 0 replies; 28+ messages in thread
From: Martina Krasteva @ 2021-03-19 18:06 UTC (permalink / raw)
To: linux-media
Cc: mchehab, robh+dt, devicetree, sakari.ailus,
daniele.alessandrelli, paul.j.murphy, gjorgjix.rosikopulos,
martinax.krasteva
From: Martina Krasteva <martinax.krasteva@intel.com>
Keem Bay pipeline object is responsible for handling
of all pipeline management operations. It handles
pipeline lifecycle states and configuration.
Co-developed-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
---
.../platform/keembay-camera/keembay-cam-xlink.c | 133 +++++++++++++++++++
.../platform/keembay-camera/keembay-cam-xlink.h | 7 +
.../platform/keembay-camera/keembay-pipeline.c | 145 ++++++++++++++++++++-
3 files changed, 279 insertions(+), 6 deletions(-)
diff --git a/drivers/media/platform/keembay-camera/keembay-cam-xlink.c b/drivers/media/platform/keembay-camera/keembay-cam-xlink.c
index 49a0937bc9fc..5b403874c4a1 100644
--- a/drivers/media/platform/keembay-camera/keembay-cam-xlink.c
+++ b/drivers/media/platform/keembay-camera/keembay-cam-xlink.c
@@ -192,3 +192,136 @@ int kmb_cam_xlink_read_msg(struct kmb_xlink_cam *xlink_cam, int chan_id,
return written_size;
}
+
+/**
+ * kmb_cam_xlink_open_ctrl_channel - Open xlink control channel for communication
+ * @xlink_cam: Pointer to xlink camera handle
+ *
+ * There is only one control channel for xlink camera communication.
+ * NOTE: The channel is serialized and reference counted.
+ *
+ * Return: 0 if successful, error code otherwise
+ */
+int kmb_cam_xlink_open_ctrl_channel(struct kmb_xlink_cam *xlink_cam)
+{
+ int ret;
+
+ mutex_lock(&xlink_cam->lock);
+
+ if (xlink_cam->ctrl_chan_refcnt) {
+ xlink_cam->ctrl_chan_refcnt++;
+ mutex_unlock(&xlink_cam->lock);
+ return 0;
+ }
+
+ ret = xlink_open_channel(&xlink_cam->handle,
+ KMB_CAM_XLINK_CTRL_CHAN_ID, RXB_TXB,
+ KMB_CAM_XLINK_CH_MAX_DATA_SIZE,
+ KMB_CAM_XLINK_CH_TIMEOUT_MS);
+ if (ret) {
+ dev_err(xlink_cam->dev, "Failed to open xlink control channel %d", ret);
+ mutex_unlock(&xlink_cam->lock);
+ return -ENODEV;
+ }
+
+ xlink_cam->ctrl_chan_refcnt++;
+
+ mutex_unlock(&xlink_cam->lock);
+
+ return 0;
+}
+
+/**
+ * kmb_cam_xlink_close_ctrl_channel - Close xlink control channel
+ * @xlink_cam: Pointer to xlink camera handle
+ *
+ * There is only one control channel for xlink camera communication.
+ * NOTE: The channel is serialized and reference counted.
+ *
+ * Return: 0 if successful, error code otherwise
+ */
+void kmb_cam_xlink_close_ctrl_channel(struct kmb_xlink_cam *xlink_cam)
+{
+ int ret;
+
+ mutex_lock(&xlink_cam->lock);
+
+ if (WARN_ON(!xlink_cam->ctrl_chan_refcnt)) {
+ mutex_unlock(&xlink_cam->lock);
+ return;
+ }
+
+ if (--xlink_cam->ctrl_chan_refcnt) {
+ mutex_unlock(&xlink_cam->lock);
+ return;
+ }
+
+ ret = xlink_close_channel(&xlink_cam->handle, KMB_CAM_XLINK_CTRL_CHAN_ID);
+ if (ret)
+ dev_err(xlink_cam->dev, "Failed to close xlink channel %d", ret);
+
+ mutex_unlock(&xlink_cam->lock);
+}
+
+/**
+ * kmb_cam_xlink_write_ctrl_msg - Write xlink control message
+ * @xlink_cam: Pointer to xlink camera handle
+ * @ctrl_paddr: Physical address of the control message
+ * @ctrl_type: Control message type
+ * @expected_result: Control message expected result
+ *
+ * For each control message there is ack from the VPU camera.
+ * This function check the error against expected result.
+ * NOTE: Because there is only one control channel, the msg/ack
+ * is sequence serliazed.
+ *
+ * Return: 0 if successful, error code otherwise
+ */
+int kmb_cam_xlink_write_ctrl_msg(struct kmb_xlink_cam *xlink_cam,
+ dma_addr_t ctrl_paddr, u32 ctrl_type,
+ u32 expected_result)
+{
+ size_t init_evt_size = sizeof(struct kmb_ic_ev);
+ struct kmb_ic_ev init_evt;
+ int ret;
+
+ mutex_lock(&xlink_cam->lock);
+
+ memset(&init_evt, 0, sizeof(init_evt));
+ init_evt.ctrl = ctrl_type;
+ init_evt.ev_info.user_data_base_addr01 = ctrl_paddr;
+ ret = xlink_write_volatile(&xlink_cam->handle,
+ KMB_CAM_XLINK_CTRL_CHAN_ID,
+ (u8 *)&init_evt,
+ init_evt_size);
+ if (ret) {
+ dev_err(xlink_cam->dev, "Error ret %d ctrl type %d",
+ ret, ctrl_type);
+ ret = -ENODEV;
+ goto error_unlock;
+ }
+
+ ret = xlink_read_data_to_buffer(&xlink_cam->handle,
+ KMB_CAM_XLINK_CTRL_CHAN_ID,
+ (u8 *)&init_evt,
+ (u32 *)&init_evt_size);
+ if (ret) {
+ dev_err(xlink_cam->dev, "Error read ack ret %d", ret);
+ ret = -ENODEV;
+ goto error_unlock;
+ }
+ if (init_evt.ctrl != expected_result) {
+ dev_err(xlink_cam->dev, "Error ctrl type %d evt ctrl %d",
+ ctrl_type, init_evt.ctrl);
+ ret = -EINVAL;
+ goto error_unlock;
+ }
+
+ mutex_unlock(&xlink_cam->lock);
+
+ return 0;
+
+error_unlock:
+ mutex_unlock(&xlink_cam->lock);
+ return ret;
+}
diff --git a/drivers/media/platform/keembay-camera/keembay-cam-xlink.h b/drivers/media/platform/keembay-camera/keembay-cam-xlink.h
index d9a78d847a4b..45e2c003cd33 100644
--- a/drivers/media/platform/keembay-camera/keembay-cam-xlink.h
+++ b/drivers/media/platform/keembay-camera/keembay-cam-xlink.h
@@ -39,4 +39,11 @@ int kmb_cam_xlink_write_msg(struct kmb_xlink_cam *xlink_cam, int chan_id,
int kmb_cam_xlink_read_msg(struct kmb_xlink_cam *xlink_cam, int chan_id,
u8 *message, u32 msg_size);
+int kmb_cam_xlink_open_ctrl_channel(struct kmb_xlink_cam *xlink_cam);
+void kmb_cam_xlink_close_ctrl_channel(struct kmb_xlink_cam *xlink_cam);
+
+int kmb_cam_xlink_write_ctrl_msg(struct kmb_xlink_cam *xlink_cam,
+ dma_addr_t ctrl_paddr, u32 ctrl_type,
+ u32 expected_result);
+
#endif /* KEEMBAY_CAM_XLINK_H */
diff --git a/drivers/media/platform/keembay-camera/keembay-pipeline.c b/drivers/media/platform/keembay-camera/keembay-pipeline.c
index 1ebaa900e4dc..78b2fffa42ee 100644
--- a/drivers/media/platform/keembay-camera/keembay-pipeline.c
+++ b/drivers/media/platform/keembay-camera/keembay-pipeline.c
@@ -2,13 +2,64 @@
/*
* Intel Keem Bay camera pipeline.
*
- * Copyright (C) 2020 Intel Corporation
+ * Copyright (C) 2021 Intel Corporation
*/
+#include <linux/dma-mapping.h>
+#include <linux/idr.h>
+#include <linux/kernel.h>
+#include <linux/xlink.h>
+
#include <media/v4l2-device.h>
+#include "keembay-cam-xlink.h"
#include "keembay-pipeline.h"
#include "keembay-vpu-cmd.h"
+static void kmb_pipe_print_config(struct kmb_pipeline *pipe)
+{
+ struct kmb_pipe_config_evs *cfg = pipe->pipe_cfg;
+ struct device *dev = pipe->dev;
+ unsigned int i;
+
+ dev_dbg(dev, "\tpipe_id %u\n", cfg->pipe_id);
+ dev_dbg(dev, "\tpipe_type %u\n", cfg->pipe_type);
+ dev_dbg(dev, "\tsrc_type %u\n", cfg->src_type);
+ dev_dbg(dev, "\tpipe_trans_hub %u\n", cfg->pipe_trans_hub);
+ dev_dbg(dev, "\tin_isp_res %ux%u\n",
+ cfg->in_isp_res.w, cfg->in_isp_res.h);
+ dev_dbg(dev, "\tout_isp_res %ux%u\n",
+ cfg->out_isp_res.w, cfg->out_isp_res.h);
+ dev_dbg(dev, "\tin_isp_stride %u\n", cfg->in_isp_stride);
+ dev_dbg(dev, "\tin_exp_offsets[0] %u\n\tin_exp_offsets[1] %u\n"
+ "\tin_exp_offsets[2] %u\n",
+ cfg->in_exp_offsets[0], cfg->in_exp_offsets[1],
+ cfg->in_exp_offsets[2]);
+
+ for (i = 0; i < PIPE_OUTPUT_ID_MAX; i++) {
+ dev_dbg(dev, "\tOUTPUT ID: %d\n", i);
+ dev_dbg(dev, "\t\tout_min_res %ux%u\n",
+ cfg->out_min_res[i].w, cfg->out_min_res[i].h);
+ dev_dbg(dev, "\t\tout_max_res %ux%u\n",
+ cfg->out_max_res[i].w, cfg->out_max_res[i].h);
+ }
+
+ for (i = 0; i < PIPE_OUTPUT_ID_MAX; i++) {
+ dev_dbg(dev, "\tpipe_xlink_chann: %d\n", i);
+ dev_dbg(dev, "\t\tid: %u %ux%u\n",
+ cfg->pipe_xlink_chann[i].id,
+ cfg->pipe_xlink_chann[i].frm_res.w,
+ cfg->pipe_xlink_chann[i].frm_res.h);
+ }
+
+ dev_dbg(dev, "\tkeep_aspect_ratio %u\n", cfg->keep_aspect_ratio);
+ dev_dbg(dev, "\tin_data_width %u\n", cfg->in_data_width);
+ dev_dbg(dev, "\tin_data_packed %u\n", cfg->in_data_packed);
+ dev_dbg(dev, "\tout_data_width %u\n", cfg->out_data_width);
+ dev_dbg(dev, "\tinternal_memory_addr 0x%llx\n",
+ cfg->internal_memory_addr);
+ dev_dbg(dev, "\tinternal_memory_size %u\n", cfg->internal_memory_size);
+}
+
/**
* kmb_pipe_init - Initialize KMB Pipeline
* @pipe: pointer to pipeline object
@@ -20,6 +71,20 @@
int kmb_pipe_init(struct kmb_pipeline *pipe, struct device *dev,
struct kmb_xlink_cam *xlink_cam)
{
+ pipe->pipe_cfg = dma_alloc_coherent(dev,
+ sizeof(*pipe->pipe_cfg),
+ &pipe->pipe_cfg_paddr, 0);
+ if (!pipe->pipe_cfg)
+ return -ENOMEM;
+
+ mutex_init(&pipe->lock);
+ pipe->pending = 0;
+ pipe->streaming = 0;
+ pipe->state = KMB_PIPE_STATE_UNCONFIGURED;
+
+ pipe->dev = dev;
+ pipe->xlink_cam = xlink_cam;
+
return 0;
}
@@ -28,7 +93,10 @@ int kmb_pipe_init(struct kmb_pipeline *pipe, struct device *dev,
* @pipe: pointer to pipeline object
*/
void kmb_pipe_cleanup(struct kmb_pipeline *pipe)
-{ }
+{
+ dma_free_coherent(pipe->dev, sizeof(struct kmb_pipe_config_evs),
+ pipe->pipe_cfg, pipe->pipe_cfg_paddr);
+}
/**
* kmb_pipe_request - Request a pipeline
@@ -38,7 +106,13 @@ void kmb_pipe_cleanup(struct kmb_pipeline *pipe)
*/
int kmb_pipe_request(struct kmb_pipeline *pipe)
{
- return 0;
+ int ret;
+
+ ret = kmb_cam_xlink_open_ctrl_channel(pipe->xlink_cam);
+ if (ret < 0)
+ dev_err(pipe->dev, "Failed to request control channel");
+
+ return ret;
}
/**
@@ -46,7 +120,9 @@ int kmb_pipe_request(struct kmb_pipeline *pipe)
* @pipe: pointer to pipeline object
*/
void kmb_pipe_release(struct kmb_pipeline *pipe)
-{ }
+{
+ kmb_cam_xlink_close_ctrl_channel(pipe->xlink_cam);
+}
/**
* kmb_pipe_config_dest - Configure pipeline destination information
@@ -56,7 +132,23 @@ void kmb_pipe_release(struct kmb_pipeline *pipe)
*/
void kmb_pipe_config_dest(struct kmb_pipeline *pipe, unsigned int output_id,
struct kmb_channel_cfg *channel_cfg)
-{ }
+{
+ mutex_lock(&pipe->lock);
+
+ channel_cfg->frm_res.w =
+ clamp_val(channel_cfg->frm_res.w,
+ pipe->pipe_cfg->out_min_res[output_id].w,
+ pipe->pipe_cfg->out_max_res[output_id].w);
+
+ channel_cfg->frm_res.h =
+ clamp_val(channel_cfg->frm_res.h,
+ pipe->pipe_cfg->out_min_res[output_id].h,
+ pipe->pipe_cfg->out_max_res[output_id].h);
+
+ pipe->pipe_cfg->pipe_xlink_chann[output_id] = *channel_cfg;
+
+ mutex_unlock(&pipe->lock);
+}
/**
* kmb_pipe_config_src - Configure pipeline source information
@@ -72,5 +164,46 @@ void kmb_pipe_config_dest(struct kmb_pipeline *pipe, unsigned int output_id,
int kmb_pipe_config_src(struct kmb_pipeline *pipe,
struct kmb_pipe_config_evs *pipe_cfg)
{
- return 0;
+ int ret = 0;
+
+ mutex_lock(&pipe->lock);
+
+ switch (pipe->state) {
+ case KMB_PIPE_STATE_CONFIGURED:
+ case KMB_PIPE_STATE_UNCONFIGURED:
+ /* Initialize pipeline configuration and counters */
+ pipe->pending = 0;
+ pipe->streaming = 0;
+
+ /* Store pipeline configuration */
+ *pipe->pipe_cfg = *pipe_cfg;
+
+ /*
+ * For some reason vpu firmware is returning config pipe as
+ * result for config pipe control.
+ */
+ ret = kmb_cam_xlink_write_ctrl_msg(pipe->xlink_cam,
+ pipe->pipe_cfg_paddr,
+ KMB_IC_EVENT_TYPE_CONFIG_ISP_PIPE,
+ KMB_IC_EVENT_TYPE_CONFIG_ISP_PIPE);
+ if (ret < 0) {
+ dev_err(pipe->dev, "Failed to reconfigure pipeline!");
+ break;
+ }
+ kmb_pipe_print_config(pipe);
+
+ pipe->state = KMB_PIPE_STATE_CONFIGURED;
+ break;
+ case KMB_PIPE_STATE_BUILT:
+ dev_err(pipe->dev, "Invalid state transition, already built");
+ break;
+ default:
+ dev_err(pipe->dev,
+ "Config pipe in invalid state %d", pipe->state);
+ ret = -EINVAL;
+ break;
+ }
+
+ mutex_unlock(&pipe->lock);
+ return ret;
}
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread* [PATCH 08/10] media: Keem Bay Camera: Add capture video node
2021-03-19 18:06 [PATCH 00/10] Keem Bay Camera Subsystem Martina Krasteva
` (6 preceding siblings ...)
2021-03-19 18:06 ` [PATCH 07/10] media: Keem Bay Camera: Add pipeline support Martina Krasteva
@ 2021-03-19 18:06 ` Martina Krasteva
2021-04-09 14:32 ` Sakari Ailus
2021-03-19 18:06 ` [PATCH 09/10] media: Keem Bay Camera: Add metadata " Martina Krasteva
` (2 subsequent siblings)
10 siblings, 1 reply; 28+ messages in thread
From: Martina Krasteva @ 2021-03-19 18:06 UTC (permalink / raw)
To: linux-media
Cc: mchehab, robh+dt, devicetree, sakari.ailus,
daniele.alessandrelli, paul.j.murphy, gjorgjix.rosikopulos,
martinax.krasteva
From: Martina Krasteva <martinax.krasteva@intel.com>
Capture video node implements v4l2 capture
interface and XLink VPU Camera buffer pool operations.
Build and set stream pipeline operations are also executed
from capture video node. Resolution depends on remote
entity pad connected to this video node.
Co-developed-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
---
.../platform/keembay-camera/keembay-pipeline.c | 192 +++++
.../platform/keembay-camera/keembay-pipeline.h | 4 +
.../media/platform/keembay-camera/keembay-video.c | 884 ++++++++++++++++++++-
.../media/platform/keembay-camera/keembay-video.h | 51 +-
4 files changed, 1096 insertions(+), 35 deletions(-)
diff --git a/drivers/media/platform/keembay-camera/keembay-pipeline.c b/drivers/media/platform/keembay-camera/keembay-pipeline.c
index 78b2fffa42ee..0050361ef3c0 100644
--- a/drivers/media/platform/keembay-camera/keembay-pipeline.c
+++ b/drivers/media/platform/keembay-camera/keembay-pipeline.c
@@ -60,6 +60,39 @@ static void kmb_pipe_print_config(struct kmb_pipeline *pipe)
dev_dbg(dev, "\tinternal_memory_size %u\n", cfg->internal_memory_size);
}
+static unsigned int kmb_pipe_get_pending(struct media_entity *entity)
+{
+ struct media_device *mdev = entity->graph_obj.mdev;
+ unsigned int num_vdevs = 0;
+ struct media_entity *next;
+ struct media_graph graph;
+ int ret;
+
+ /* Walk through graph to count the connected video node entities */
+ mutex_lock(&mdev->graph_mutex);
+
+ ret = media_graph_walk_init(&graph, mdev);
+ if (ret) {
+ mutex_unlock(&mdev->graph_mutex);
+ return -EINVAL;
+ }
+
+ media_graph_walk_start(&graph, entity);
+
+ while ((next = media_graph_walk_next(&graph))) {
+ if (!is_media_entity_v4l2_video_device(next))
+ continue;
+
+ num_vdevs++;
+ }
+
+ mutex_unlock(&mdev->graph_mutex);
+
+ media_graph_walk_cleanup(&graph);
+
+ return num_vdevs;
+}
+
/**
* kmb_pipe_init - Initialize KMB Pipeline
* @pipe: pointer to pipeline object
@@ -207,3 +240,162 @@ int kmb_pipe_config_src(struct kmb_pipeline *pipe,
mutex_unlock(&pipe->lock);
return ret;
}
+
+/**
+ * kmb_pipe_prepare - Prepare VPU pipeline for streaming
+ * @pipe: pointer to pipeline object
+ *
+ * Prepare pipeline for streaming by sending negotiated configuration to VPU
+ * and changing state to BUILT.
+ *
+ * Return: 0 if successful, error code otherwise.
+ */
+int kmb_pipe_prepare(struct kmb_pipeline *pipe)
+{
+ int ret = 0;
+
+ mutex_lock(&pipe->lock);
+
+ /* build only if all outputs are configured */
+ switch (pipe->state) {
+ case KMB_PIPE_STATE_UNCONFIGURED:
+ /* Call config and continue */
+ ret = kmb_cam_xlink_write_ctrl_msg(pipe->xlink_cam,
+ pipe->pipe_cfg_paddr,
+ KMB_IC_EVENT_TYPE_CONFIG_ISP_PIPE,
+ KMB_IC_EVENT_TYPE_SUCCESSFUL);
+ if (ret < 0) {
+ dev_err(pipe->dev, "Failed to reconfigure pipeline!");
+ break;
+ }
+ fallthrough;
+ case KMB_PIPE_STATE_CONFIGURED:
+ ret = kmb_cam_xlink_write_ctrl_msg(pipe->xlink_cam,
+ pipe->pipe_cfg_paddr,
+ KMB_IC_EVENT_TYPE_BUILD_ISP_PIPE,
+ KMB_IC_EVENT_TYPE_SUCCESSFUL);
+ if (ret < 0) {
+ dev_err(pipe->dev, "Failed to build pipeline!");
+ break;
+ }
+ pipe->state = KMB_PIPE_STATE_BUILT;
+ break;
+ case KMB_PIPE_STATE_BUILT:
+ /* Pipeline is already built ignore */
+ break;
+ default:
+ dev_err(pipe->dev,
+ "Build pipe in invalid state %d", pipe->state);
+ ret = -EINVAL;
+ break;
+ }
+
+ mutex_unlock(&pipe->lock);
+
+ return ret;
+}
+
+static int kmb_pipe_s_stream(struct kmb_pipeline *pipe,
+ struct media_entity *entity, int enable)
+{
+ struct v4l2_subdev *subdev;
+ struct media_pad *remote;
+ int ret;
+
+ remote = media_entity_remote_pad(entity->pads);
+ if (!remote || !is_media_entity_v4l2_subdev(remote->entity))
+ return -EINVAL;
+
+ subdev = media_entity_to_v4l2_subdev(remote->entity);
+ if (!subdev)
+ return -EINVAL;
+
+ ret = v4l2_subdev_call(subdev, video, s_stream, enable);
+ if (ret < 0 && ret != -ENOIOCTLCMD)
+ dev_err(pipe->dev, "Cannot set stream %d", enable);
+
+ return ret != -ENOIOCTLCMD ? ret : 0;
+}
+
+/**
+ * kmb_pipe_stop - Set stream off and stop media pipeline
+ * @pipe: KMB pipeline object
+ * @entity: media entity
+ */
+void kmb_pipe_stop(struct kmb_pipeline *pipe, struct media_entity *entity)
+{
+ mutex_lock(&pipe->lock);
+
+ if (WARN_ON(!pipe->streaming)) {
+ dev_err(pipe->dev, "Calling stop on already stopped pipeline");
+ mutex_unlock(&pipe->lock);
+ return;
+ }
+
+ if (pipe->state == KMB_PIPE_STATE_STREAMING) {
+ kmb_pipe_s_stream(pipe, entity, 0);
+ media_pipeline_stop(entity);
+ pipe->state = KMB_PIPE_STATE_BUILT;
+ }
+
+ if (pipe->state == KMB_PIPE_STATE_BUILT ||
+ pipe->state == KMB_PIPE_STATE_CONFIGURED) {
+ kmb_cam_xlink_write_ctrl_msg(pipe->xlink_cam,
+ pipe->pipe_cfg_paddr,
+ KMB_IC_EVENT_TYPE_DELETE_ISP_PIPE,
+ KMB_IC_EVENT_TYPE_SUCCESSFUL);
+
+ pipe->state = KMB_PIPE_STATE_UNCONFIGURED;
+ }
+
+ pipe->streaming--;
+
+ mutex_unlock(&pipe->lock);
+}
+
+/**
+ * kmb_pipe_run - Run media pipeline and start streaming
+ * @pipe: KMB pipeline object
+ * @entity: media entity
+ *
+ * Return: 0 if successful, error code otherwise.
+ */
+int kmb_pipe_run(struct kmb_pipeline *pipe, struct media_entity *entity)
+{
+ int ret = 0;
+
+ mutex_lock(&pipe->lock);
+
+ if (!pipe->streaming)
+ pipe->pending = kmb_pipe_get_pending(entity);
+
+ pipe->streaming++;
+
+ if (pipe->streaming != pipe->pending)
+ goto done_unlock;
+
+ if (pipe->state != KMB_PIPE_STATE_BUILT) {
+ ret = -EINVAL;
+ goto done_unlock;
+ }
+
+ ret = media_pipeline_start(entity, &pipe->media_pipe);
+ if (ret < 0) {
+ dev_err(pipe->dev, "Failed to start media pipeline");
+ goto done_unlock;
+ }
+
+ ret = kmb_pipe_s_stream(pipe, entity, 1);
+ if (ret < 0 && ret != -ENOIOCTLCMD) {
+ mutex_unlock(&pipe->lock);
+ kmb_pipe_stop(pipe, entity);
+ return ret;
+ }
+
+ pipe->state = KMB_PIPE_STATE_STREAMING;
+
+done_unlock:
+ mutex_unlock(&pipe->lock);
+
+ return ret;
+}
diff --git a/drivers/media/platform/keembay-camera/keembay-pipeline.h b/drivers/media/platform/keembay-camera/keembay-pipeline.h
index 83ff94d11b34..60ba99e9a73c 100644
--- a/drivers/media/platform/keembay-camera/keembay-pipeline.h
+++ b/drivers/media/platform/keembay-camera/keembay-pipeline.h
@@ -68,4 +68,8 @@ void kmb_pipe_config_dest(struct kmb_pipeline *pipe, unsigned int output_id,
int kmb_pipe_config_src(struct kmb_pipeline *pipe,
struct kmb_pipe_config_evs *pipe_cfg);
+int kmb_pipe_prepare(struct kmb_pipeline *pipe);
+int kmb_pipe_run(struct kmb_pipeline *pipe, struct media_entity *entity);
+void kmb_pipe_stop(struct kmb_pipeline *pipe, struct media_entity *entity);
+
#endif /* KEEMBAY_PIPELINE_H */
diff --git a/drivers/media/platform/keembay-camera/keembay-video.c b/drivers/media/platform/keembay-camera/keembay-video.c
index 02f4d97e16fb..a92cfbeffea9 100644
--- a/drivers/media/platform/keembay-camera/keembay-video.c
+++ b/drivers/media/platform/keembay-camera/keembay-video.c
@@ -2,9 +2,816 @@
/*
* Intel Keem Bay camera Video node.
*
- * Copyright (C) 2018-2020 Intel Corporation
+ * Copyright (C) 2021 Intel Corporation
*/
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/freezer.h>
+#include <linux/kthread.h>
+
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-mc.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "keembay-cam-xlink.h"
+#include "keembay-pipeline.h"
#include "keembay-video.h"
+#include "keembay-vpu-frame.h"
+
+#define KMB_CAM_VIDEO_NAME "keembay-video"
+
+/* Xlink data channel size and timeout */
+#define KMB_VID_CH_DATA_SIZE 1024
+#define KMB_VID_CH_TIMEOUT_MS 5000
+
+#define KMB_VID_MIN_WIDTH 16
+#define KMB_VID_MIN_HEIGHT 16
+#define KMB_VID_MAX_WIDTH U16_MAX
+#define KMB_VID_MAX_HEIGHT U16_MAX
+#define KMB_VID_STEP_WIDTH 8
+#define KMB_VID_STEP_HEIGHT 8
+
+#define to_kmb_video_buf(vbuf) container_of(vbuf, struct kmb_frame_buffer, vb)
+
+/* Kmb video format info structure */
+struct kmb_video_fmt_info {
+ const char *description;
+ u32 code;
+ u32 pixelformat;
+ enum kmb_frame_types type;
+ u32 colorspace;
+ unsigned int planes;
+ unsigned int bpp;
+ unsigned int h_subsample;
+ unsigned int v_subsample;
+ bool contiguous_memory;
+};
+
+/* Supported video formats */
+static const struct kmb_video_fmt_info video_formats[] = {
+ {
+ .description = "NV12",
+ .code = MEDIA_BUS_FMT_YUYV8_1_5X8,
+ .pixelformat = V4L2_PIX_FMT_NV12,
+ .type = KMB_FRAME_TYPE_NV12,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+ .planes = 2,
+ .bpp = 8,
+ .h_subsample = 1,
+ .v_subsample = 2,
+ .contiguous_memory = true,
+ },
+ {
+ .description = "Planar YUV 4:2:0",
+ .code = MEDIA_BUS_FMT_UYYVYY8_0_5X24,
+ .pixelformat = V4L2_PIX_FMT_YUV420,
+ .type = KMB_FRAME_TYPE_YUV420P,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+ .planes = 3,
+ .bpp = 8,
+ .h_subsample = 2,
+ .v_subsample = 2,
+ .contiguous_memory = false,
+ },
+ {
+ .description = "Planar YUV 4:4:4",
+ .code = MEDIA_BUS_FMT_YUV8_1X24,
+ .pixelformat = V4L2_PIX_FMT_YUV444,
+ .type = KMB_FRAME_TYPE_YUV444P,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+ .planes = 3,
+ .bpp = 8,
+ .h_subsample = 1,
+ .v_subsample = 1,
+ .contiguous_memory = false,
+ },
+ {
+ .description = "RAW 8 Garyscale",
+ .code = MEDIA_BUS_FMT_Y8_1X8,
+ .pixelformat = V4L2_PIX_FMT_GREY,
+ .type = KMB_FRAME_TYPE_RAW8,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .planes = 1,
+ .bpp = 8,
+ .h_subsample = 1,
+ .v_subsample = 1,
+ .contiguous_memory = false,
+ },
+ {
+ .description = "RAW 10 Grayscale",
+ .code = MEDIA_BUS_FMT_Y10_1X10,
+ .pixelformat = V4L2_PIX_FMT_Y10,
+ .type = KMB_FRAME_TYPE_RAW10,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .planes = 1,
+ .bpp = 10,
+ .h_subsample = 1,
+ .v_subsample = 1,
+ .contiguous_memory = false,
+ }
+};
+
+static const struct kmb_video_fmt_info *
+kmb_video_get_fmt_info_by_code(u32 code)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(video_formats); i++)
+ if (video_formats[i].code == code)
+ return &video_formats[i];
+
+ return NULL;
+}
+
+static const struct kmb_video_fmt_info *
+kmb_video_get_fmt_info_by_pixfmt(u32 pix_fmt)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(video_formats); i++)
+ if (video_formats[i].pixelformat == pix_fmt)
+ return &video_formats[i];
+
+ return NULL;
+}
+
+/* Buffer processing operations */
+static void kmb_video_insert_buf(struct kmb_video *kmb_vid,
+ struct kmb_frame_buffer *buf)
+{
+ INIT_LIST_HEAD(&buf->list);
+
+ mutex_lock(&kmb_vid->dma_lock);
+ list_add_tail(&buf->list, &kmb_vid->dma_queue);
+ mutex_unlock(&kmb_vid->dma_lock);
+}
+
+static void __kmb_video_buf_discard(struct kmb_video *kmb_vid,
+ struct kmb_frame_buffer *buf)
+{
+ lockdep_assert_held(&kmb_vid->dma_lock);
+
+ list_del(&buf->list);
+ vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+}
+
+static int kmb_video_process_buf(struct kmb_video *kmb_vid,
+ struct kmb_frame_buffer *buf)
+{
+ const struct kmb_video_fmt_info *info = kmb_vid->active_fmt.info;
+ struct v4l2_pix_format_mplane *pix = &kmb_vid->active_fmt.pix;
+ struct kmb_vpu_frame_buffer rt_frame_buf;
+ int ret;
+
+ lockdep_assert_held(&kmb_vid->lock);
+
+ memset(&rt_frame_buf, 0, sizeof(rt_frame_buf));
+ rt_frame_buf.spec.bpp = info->bpp;
+ rt_frame_buf.spec.type = info->type;
+ rt_frame_buf.spec.width = pix->width;
+ rt_frame_buf.spec.height = pix->height;
+ rt_frame_buf.spec.stride = pix->plane_fmt[0].bytesperline;
+ rt_frame_buf.p1 = buf->addr[0];
+
+ /* Planes not used by the VPU should be set with addr 0 */
+ if (pix->num_planes > 1)
+ rt_frame_buf.p2 = buf->addr[1];
+ if (pix->num_planes > 2)
+ rt_frame_buf.p3 = buf->addr[2];
+
+ ret = kmb_cam_xlink_write_msg(kmb_vid->xlink_cam,
+ kmb_vid->chan_id,
+ (u8 *)&rt_frame_buf,
+ sizeof(rt_frame_buf));
+ if (ret < 0) {
+ dev_err(kmb_vid->dma_dev, "Error on buffer queue %d", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static void kmb_video_process_all_bufs(struct kmb_video *kmb_vid)
+{
+ struct kmb_frame_buffer *buf;
+ struct list_head *next;
+ struct list_head *pos;
+ int ret;
+
+ mutex_lock(&kmb_vid->dma_lock);
+
+ /* Discard buf is removing buffer from the list */
+ list_for_each_safe(pos, next, &kmb_vid->dma_queue) {
+ buf = list_entry(pos, struct kmb_frame_buffer, list);
+
+ ret = kmb_video_process_buf(kmb_vid, buf);
+ if (ret) {
+ dev_err(&kmb_vid->video->dev,
+ "Cannot process output buf 0x%pad",
+ &buf->addr[0]);
+ __kmb_video_buf_discard(kmb_vid, buf);
+ continue;
+ }
+ }
+
+ mutex_unlock(&kmb_vid->dma_lock);
+}
+
+static int kmb_video_queue_output_buf(struct kmb_video *kmb_vid,
+ struct kmb_frame_buffer *buf)
+{
+ int ret = 0;
+
+ kmb_video_insert_buf(kmb_vid, buf);
+
+ mutex_lock(&kmb_vid->dma_lock);
+
+ /* Process buffers only when device is streaming */
+ if (vb2_is_streaming(&kmb_vid->vb2_q)) {
+ ret = kmb_video_process_buf(kmb_vid, buf);
+ if (ret) {
+ dev_err(&kmb_vid->video->dev,
+ "Fail to process output buf 0x%pad",
+ &buf->addr[0]);
+ __kmb_video_buf_discard(kmb_vid, buf);
+ }
+ }
+
+ mutex_unlock(&kmb_vid->dma_lock);
+
+ return ret;
+}
+
+static void kmb_video_release_all_bufs(struct kmb_video *kmb_vid,
+ enum vb2_buffer_state state)
+{
+ struct list_head *next = NULL;
+ struct list_head *pos = NULL;
+ struct kmb_frame_buffer *buf;
+
+ mutex_lock(&kmb_vid->dma_lock);
+ list_for_each_safe(pos, next, &kmb_vid->dma_queue) {
+ buf = list_entry(pos, struct kmb_frame_buffer, list);
+ list_del(&buf->list);
+ vb2_buffer_done(&buf->vb.vb2_buf, state);
+ }
+ mutex_unlock(&kmb_vid->dma_lock);
+}
+
+static void kmb_video_remove_buf(struct kmb_video *kmb_vid,
+ struct kmb_frame_buffer *buf)
+{
+ mutex_lock(&kmb_vid->dma_lock);
+ list_del(&buf->list);
+ mutex_unlock(&kmb_vid->dma_lock);
+}
+
+static struct kmb_frame_buffer *
+kmb_video_find_buf_by_addr(struct kmb_video *kmb_vid, uint64_t addr)
+{
+ struct kmb_frame_buffer *buf = NULL;
+ struct list_head *node = NULL;
+
+ mutex_lock(&kmb_vid->dma_lock);
+
+ list_for_each(node, &kmb_vid->dma_queue) {
+ buf = list_entry(node, struct kmb_frame_buffer, list);
+ if (buf->addr[0] == addr) {
+ mutex_unlock(&kmb_vid->dma_lock);
+ return buf;
+ }
+ }
+
+ mutex_unlock(&kmb_vid->dma_lock);
+
+ return NULL;
+}
+
+static void kmb_video_fmt_info_to_pix(const struct kmb_video_fmt_info *info,
+ struct v4l2_mbus_framefmt *mbus_fmt,
+ struct v4l2_pix_format_mplane *pix)
+{
+ u32 bytesperline;
+ u32 sizeimage;
+ u32 v_sub = 1;
+ u32 h_sub = 1;
+ unsigned int i;
+
+ pix->width = mbus_fmt->width;
+ pix->height = mbus_fmt->height;
+
+ pix->pixelformat = info->pixelformat;
+ pix->colorspace = info->colorspace;
+ pix->num_planes = info->planes;
+
+ for (i = 0; i < pix->num_planes; i++) {
+ bytesperline = pix->width * info->bpp / 8 / h_sub;
+
+ if (pix->plane_fmt[i].bytesperline < bytesperline)
+ pix->plane_fmt[i].bytesperline = bytesperline;
+
+ sizeimage = pix->plane_fmt[i].bytesperline *
+ pix->height / v_sub;
+
+ if (pix->plane_fmt[i].sizeimage < sizeimage)
+ pix->plane_fmt[i].sizeimage = sizeimage;
+
+ h_sub = info->h_subsample;
+ v_sub = info->v_subsample;
+ }
+}
+
+static int kmb_video_get_subdev_fmt(struct kmb_video *kmb_vid,
+ struct v4l2_pix_format_mplane *pix)
+{
+ const struct kmb_video_fmt_info *fmt_info;
+ struct v4l2_subdev_format sd_fmt;
+ struct v4l2_subdev *subdev;
+ struct media_pad *remote;
+ int ret;
+
+ remote = media_entity_remote_pad(&kmb_vid->pad);
+ if (!remote || !is_media_entity_v4l2_subdev(remote->entity))
+ return -EINVAL;
+
+ subdev = media_entity_to_v4l2_subdev(remote->entity);
+ if (!subdev)
+ return -EINVAL;
+
+ memset(&sd_fmt, 0, sizeof(sd_fmt));
+ sd_fmt.pad = remote->index;
+ sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ ret = v4l2_subdev_call(subdev, pad, get_fmt, NULL, &sd_fmt);
+ if (ret < 0)
+ return ret;
+
+ fmt_info = kmb_video_get_fmt_info_by_code(sd_fmt.format.code);
+ if (!fmt_info)
+ return -EINVAL;
+
+ kmb_video_fmt_info_to_pix(fmt_info, &sd_fmt.format, pix);
+
+ return 0;
+}
+
+static int kmb_video_queue_setup(struct vb2_queue *q,
+ unsigned int *num_buffers,
+ unsigned int *num_planes,
+ unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct kmb_video *kmb_vid = vb2_get_drv_priv(q);
+ struct v4l2_pix_format_mplane *pix = &kmb_vid->active_fmt.pix;
+ unsigned int i;
+
+ if (kmb_vid->active_fmt.info->contiguous_memory) {
+ *num_planes = 1;
+ for (i = 0; i < pix->num_planes; i++)
+ sizes[0] += pix->plane_fmt[i].sizeimage;
+ } else {
+ *num_planes = pix->num_planes;
+ for (i = 0; i < pix->num_planes; i++)
+ sizes[i] = pix->plane_fmt[i].sizeimage;
+ }
+
+ return 0;
+}
+
+static int kmb_video_buffer_prepare(struct vb2_buffer *vb)
+{
+ struct kmb_video *kmb_vid = vb2_get_drv_priv(vb->vb2_queue);
+ struct v4l2_pix_format_mplane *pix = &kmb_vid->active_fmt.pix;
+ unsigned int size_image = 0;
+ unsigned int i;
+
+ if (kmb_vid->active_fmt.info->contiguous_memory) {
+ for (i = 0; i < pix->num_planes; i++)
+ size_image += pix->plane_fmt[i].sizeimage;
+
+ vb2_set_plane_payload(vb, 0, size_image);
+ } else {
+ for (i = 0; i < pix->num_planes; i++)
+ vb2_set_plane_payload(vb, i,
+ pix->plane_fmt[i].sizeimage);
+ }
+
+ return 0;
+}
+
+static int kmb_video_buf_init(struct vb2_buffer *vb)
+{
+ struct kmb_video *kmb_vid = vb2_get_drv_priv(vb->vb2_queue);
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct kmb_frame_buffer *buf = to_kmb_video_buf(vbuf);
+ struct v4l2_pix_format_mplane *pix = &kmb_vid->active_fmt.pix;
+ unsigned int i;
+
+ if (kmb_vid->active_fmt.info->contiguous_memory) {
+ buf->addr[0] = vb2_dma_contig_plane_dma_addr(vb, 0);
+ for (i = 1; i < pix->num_planes; i++) {
+ buf->addr[i] = buf->addr[i - 1] +
+ pix->plane_fmt[i - 1].sizeimage;
+ }
+ } else {
+ for (i = 0; i < pix->num_planes; i++)
+ buf->addr[i] = vb2_dma_contig_plane_dma_addr(vb, i);
+ }
+
+ return 0;
+}
+
+static void kmb_video_buf_queue(struct vb2_buffer *vb)
+{
+ struct kmb_video *kmb_vid = vb2_get_drv_priv(vb->vb2_queue);
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct kmb_frame_buffer *buf = to_kmb_video_buf(vbuf);
+ int ret;
+
+ ret = kmb_video_queue_output_buf(kmb_vid, buf);
+ if (ret)
+ dev_err(kmb_vid->dma_dev, "Fail output buf queue %d", ret);
+}
+
+static int kmb_video_worker_thread(void *video)
+{
+ struct kmb_vpu_frame_buffer rt_frame_buf;
+ struct kmb_video *kmb_vid = video;
+ struct kmb_frame_buffer *buf = NULL;
+ bool stopped = false;
+ int ret;
+
+ set_freezable();
+
+ while (!kthread_should_stop()) {
+ try_to_freeze();
+
+ if (stopped) {
+ set_current_state(TASK_INTERRUPTIBLE);
+ schedule();
+ continue;
+ }
+
+ memset(&rt_frame_buf, 0, sizeof(rt_frame_buf));
+ ret = kmb_cam_xlink_read_msg(kmb_vid->xlink_cam,
+ kmb_vid->chan_id,
+ (u8 *)&rt_frame_buf,
+ sizeof(rt_frame_buf));
+ if (ret < 0) {
+ stopped = true;
+ /* Continue here to enter in freeze state */
+ continue;
+ }
+
+ buf = kmb_video_find_buf_by_addr(kmb_vid, rt_frame_buf.p1);
+ if (buf) {
+ kmb_video_remove_buf(kmb_vid, buf);
+
+ buf->vb.vb2_buf.timestamp = rt_frame_buf.ts;
+ vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
+ } else {
+ dev_err(kmb_vid->dma_dev, "Ouch cannot find buff %llx",
+ rt_frame_buf.p1);
+ }
+ }
+
+ return 0;
+}
+
+static int kmb_video_worker_start(struct kmb_video *kmb_vid)
+{
+ int ret;
+
+ ret = kmb_cam_xlink_open_channel(kmb_vid->xlink_cam, kmb_vid->chan_id);
+ if (ret)
+ return ret;
+
+ kmb_vid->thread = kthread_run(kmb_video_worker_thread,
+ kmb_vid, "kmb_vnode_thread");
+ if (IS_ERR(kmb_vid->thread)) {
+ dev_err(&kmb_vid->video->dev, "Cannot start thread");
+ ret = -ENOMEM;
+ kmb_vid->thread = NULL;
+ goto error_close_xlink_channel;
+ }
+
+ return 0;
+
+error_close_xlink_channel:
+ kmb_cam_xlink_close_channel(kmb_vid->xlink_cam, kmb_vid->chan_id);
+
+ return ret;
+}
+
+static int kmb_video_worker_stop(struct kmb_video *kmb_vid)
+{
+ int ret;
+
+ /*
+ * Xlink has no functionality to unblock read volatile function,
+ * only way to unblock is to close the channel.
+ */
+ kmb_cam_xlink_close_channel(kmb_vid->xlink_cam, kmb_vid->chan_id);
+ if (!kmb_vid->thread) {
+ dev_warn(&kmb_vid->video->dev, "No thread running");
+ return 0;
+ }
+
+ ret = kthread_stop(kmb_vid->thread);
+ if (ret < 0)
+ dev_err(&kmb_vid->video->dev, "Thread stop failed %d", ret);
+
+ kmb_vid->thread = NULL;
+
+ return ret;
+}
+
+static int kmb_video_capture_start_streaming(struct vb2_queue *q,
+ unsigned int count)
+{
+ struct kmb_video *kmb_vid = vb2_get_drv_priv(q);
+ int ret;
+
+ ret = kmb_pipe_prepare(kmb_vid->pipe);
+ if (ret < 0)
+ goto error_discard_all_bufs;
+
+ ret = kmb_video_worker_start(kmb_vid);
+ if (ret < 0)
+ goto error_pipeline_stop;
+
+ /* Process all pending buffers after worker is started */
+ kmb_video_process_all_bufs(kmb_vid);
+
+ /*
+ * Run the pipeline after all buffers are provided for processing,
+ * the main reason is to not skip any frame from the source.
+ */
+ ret = kmb_pipe_run(kmb_vid->pipe, &kmb_vid->video->entity);
+ if (ret < 0)
+ goto error_pipeline_stop;
+
+ return 0;
+
+error_pipeline_stop:
+ kmb_pipe_stop(kmb_vid->pipe, &kmb_vid->video->entity);
+error_discard_all_bufs:
+ kmb_video_release_all_bufs(kmb_vid, VB2_BUF_STATE_QUEUED);
+
+ return ret;
+}
+
+static void kmb_video_capture_stop_streaming(struct vb2_queue *q)
+{
+ struct kmb_video *kmb_vid = vb2_get_drv_priv(q);
+
+ kmb_pipe_stop(kmb_vid->pipe, &kmb_vid->video->entity);
+
+ kmb_video_worker_stop(kmb_vid);
+
+ kmb_video_release_all_bufs(kmb_vid, VB2_BUF_STATE_ERROR);
+}
+
+/* driver-specific operations */
+static const struct vb2_ops kmb_video_vb2_q_capture_ops = {
+ .queue_setup = kmb_video_queue_setup,
+ .buf_prepare = kmb_video_buffer_prepare,
+ .buf_init = kmb_video_buf_init,
+ .buf_queue = kmb_video_buf_queue,
+ .start_streaming = kmb_video_capture_start_streaming,
+ .stop_streaming = kmb_video_capture_stop_streaming,
+};
+
+static int kmb_video_querycap(struct file *file, void *fh,
+ struct v4l2_capability *cap)
+{
+ cap->bus_info[0] = 0;
+ strscpy(cap->driver, KMB_CAM_VIDEO_NAME, sizeof(cap->driver));
+ strscpy(cap->card, KMB_CAM_VIDEO_NAME, sizeof(cap->card));
+
+ return 0;
+}
+
+static int kmb_video_enum_fmt(struct file *file, void *fh,
+ struct v4l2_fmtdesc *f)
+{
+ const struct kmb_video_fmt_info *info;
+
+ if (!V4L2_TYPE_IS_MULTIPLANAR(f->type))
+ return -EINVAL;
+
+ if (f->mbus_code) {
+ if (f->index != 0)
+ return -EINVAL;
+
+ info = kmb_video_get_fmt_info_by_code(f->mbus_code);
+ if (!info)
+ return -EINVAL;
+ } else {
+ info = &video_formats[f->index];
+ if (!info)
+ return -EINVAL;
+ }
+
+ f->pixelformat = info->pixelformat;
+ f->mbus_code = info->code;
+ strscpy(f->description, info->description, sizeof(f->description));
+
+ return 0;
+}
+
+static int kmb_video_enum_framesizes(struct file *file, void *fh,
+ struct v4l2_frmsizeenum *fsize)
+{
+ const struct kmb_video_fmt_info *info;
+
+ if (fsize->index != 0)
+ return -EINVAL;
+
+ info = kmb_video_get_fmt_info_by_pixfmt(fsize->pixel_format);
+ if (!info)
+ return -EINVAL;
+
+ fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+
+ fsize->stepwise.min_width = KMB_VID_MIN_WIDTH;
+ fsize->stepwise.max_width = KMB_VID_MAX_WIDTH;
+ fsize->stepwise.step_width = KMB_VID_STEP_WIDTH;
+ fsize->stepwise.min_height = KMB_VID_MIN_HEIGHT;
+ fsize->stepwise.max_height = KMB_VID_MAX_HEIGHT;
+ fsize->stepwise.step_height = KMB_VID_STEP_HEIGHT;
+
+ return 0;
+}
+
+static int kmb_video_try_fmt(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ const struct kmb_video_fmt_info *info;
+ struct v4l2_mbus_framefmt mbus_fmt;
+
+ info = kmb_video_get_fmt_info_by_pixfmt(f->fmt.pix_mp.pixelformat);
+ if (!info)
+ info = &video_formats[0];
+
+ mbus_fmt.width = f->fmt.pix_mp.width;
+ mbus_fmt.height = f->fmt.pix_mp.height;
+ kmb_video_fmt_info_to_pix(info, &mbus_fmt, &f->fmt.pix_mp);
+
+ return 0;
+}
+
+static int kmb_video_set_fmt(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ struct kmb_video *kmb_vid = video_drvdata(file);
+ const struct kmb_video_fmt_info *info;
+ struct v4l2_mbus_framefmt mbus_fmt;
+
+ info = kmb_video_get_fmt_info_by_pixfmt(f->fmt.pix_mp.pixelformat);
+ if (!info)
+ info = &video_formats[0];
+
+ mbus_fmt.width = f->fmt.pix_mp.width;
+ mbus_fmt.height = f->fmt.pix_mp.height;
+ kmb_video_fmt_info_to_pix(info, &mbus_fmt, &f->fmt.pix_mp);
+
+ kmb_vid->active_fmt.pix = f->fmt.pix_mp;
+ kmb_vid->active_fmt.info = info;
+
+ return 0;
+}
+
+static int kmb_video_get_fmt(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ struct kmb_video *kmb_vid = video_drvdata(file);
+
+ f->fmt.pix_mp = kmb_vid->active_fmt.pix;
+
+ return 0;
+}
+
+static int kmb_video_check_format(struct kmb_video *kmb_vid)
+{
+ int ret;
+ struct v4l2_pix_format_mplane pix;
+
+ ret = kmb_video_get_subdev_fmt(kmb_vid, &pix);
+ if (ret < 0)
+ return ret;
+
+ if (kmb_vid->active_fmt.pix.pixelformat != pix.pixelformat ||
+ kmb_vid->active_fmt.pix.height != pix.height ||
+ kmb_vid->active_fmt.pix.width != pix.width ||
+ kmb_vid->active_fmt.pix.num_planes != pix.num_planes) {
+ dev_err(&kmb_vid->video->dev, "Pix fmt mismatch:\n\t"
+ "pix_fmt %u %u\n\theight %u %u\n\twidth %u %u\n\t"
+ "num_planes %u %u",
+ kmb_vid->active_fmt.pix.pixelformat, pix.pixelformat,
+ kmb_vid->active_fmt.pix.height, pix.height,
+ kmb_vid->active_fmt.pix.width, pix.width,
+ kmb_vid->active_fmt.pix.num_planes, pix.num_planes);
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
+static int kmb_video_streamon(struct file *file, void *fh,
+ enum v4l2_buf_type type)
+{
+ struct kmb_video *kmb_vid = video_drvdata(file);
+ int ret;
+
+ if (type != kmb_vid->vb2_q.type)
+ return -EINVAL;
+
+ ret = kmb_video_check_format(kmb_vid);
+ if (ret < 0)
+ return ret;
+
+ return vb2_streamon(&kmb_vid->vb2_q, type);
+}
+
+/* V4L2 ioctl operations */
+static const struct v4l2_ioctl_ops kmb_vid_ioctl_ops = {
+ .vidioc_querycap = kmb_video_querycap,
+ .vidioc_enum_fmt_vid_cap = kmb_video_enum_fmt,
+ .vidioc_enum_framesizes = kmb_video_enum_framesizes,
+ .vidioc_g_fmt_vid_cap_mplane = kmb_video_get_fmt,
+ .vidioc_try_fmt_vid_cap_mplane = kmb_video_try_fmt,
+ .vidioc_s_fmt_vid_cap_mplane = kmb_video_set_fmt,
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_qbuf = vb2_ioctl_qbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_streamon = kmb_video_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+ .vidioc_expbuf = vb2_ioctl_expbuf,
+};
+
+static int kmb_video_open(struct file *file)
+{
+ struct kmb_video *kmb_vid = video_drvdata(file);
+ struct v4l2_mbus_framefmt fmt;
+ int ret;
+
+ mutex_lock(&kmb_vid->lock);
+ ret = v4l2_fh_open(file);
+ if (ret) {
+ mutex_unlock(&kmb_vid->lock);
+ return ret;
+ }
+
+ INIT_LIST_HEAD(&kmb_vid->dma_queue);
+
+ ret = kmb_pipe_request(kmb_vid->pipe);
+ if (ret < 0)
+ goto error_fh_release;
+
+ /* Fill default format. */
+ memset(&fmt, 0, sizeof(fmt));
+ kmb_video_fmt_info_to_pix(&video_formats[0], &fmt,
+ &kmb_vid->active_fmt.pix);
+ kmb_vid->active_fmt.info = &video_formats[0];
+
+ mutex_unlock(&kmb_vid->lock);
+
+ return 0;
+
+error_fh_release:
+ _vb2_fop_release(file, NULL);
+ mutex_unlock(&kmb_vid->lock);
+
+ return ret;
+}
+
+static int kmb_video_release(struct file *file)
+{
+ struct kmb_video *kmb_vid = video_drvdata(file);
+ int ret;
+
+ mutex_lock(&kmb_vid->lock);
+
+ kmb_pipe_release(kmb_vid->pipe);
+
+ ret = _vb2_fop_release(file, NULL);
+
+ mutex_unlock(&kmb_vid->lock);
+
+ return ret;
+}
+
+/* FS operations for V4L2 device */
+static const struct v4l2_file_operations kmb_vid_fops = {
+ .owner = THIS_MODULE,
+ .unlocked_ioctl = video_ioctl2,
+ .open = kmb_video_open,
+ .release = kmb_video_release,
+ .poll = vb2_fop_poll,
+ .mmap = vb2_fop_mmap,
+};
/**
* kmb_video_init - Initialize entity
@@ -15,7 +822,63 @@
*/
int kmb_video_init(struct kmb_video *kmb_vid, const char *name)
{
+ int ret;
+
+ kmb_vid->video = video_device_alloc();
+ if (!kmb_vid->video) {
+ dev_err(&kmb_vid->video->dev,
+ "Failed to allocate video device");
+ return -ENOMEM;
+ }
+
+ mutex_init(&kmb_vid->lock);
+ mutex_init(&kmb_vid->dma_lock);
+
+ kmb_vid->video->fops = &kmb_vid_fops;
+ kmb_vid->video->ioctl_ops = &kmb_vid_ioctl_ops;
+ kmb_vid->video->minor = -1;
+ kmb_vid->video->release = video_device_release;
+ kmb_vid->video->vfl_type = VFL_TYPE_VIDEO;
+ kmb_vid->video->lock = &kmb_vid->lock;
+ kmb_vid->video->queue = &kmb_vid->vb2_q;
+ video_set_drvdata(kmb_vid->video, kmb_vid);
+ snprintf(kmb_vid->video->name, sizeof(kmb_vid->video->name),
+ "kmb_video %s", name);
+
+ kmb_vid->vb2_q.drv_priv = kmb_vid;
+ kmb_vid->vb2_q.ops = &kmb_video_vb2_q_capture_ops;
+ kmb_vid->vb2_q.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+ kmb_vid->vb2_q.buf_struct_size = sizeof(struct kmb_frame_buffer);
+ kmb_vid->vb2_q.io_modes = VB2_MMAP | VB2_DMABUF;
+ kmb_vid->vb2_q.timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+ kmb_vid->vb2_q.mem_ops = &vb2_dma_contig_memops;
+ kmb_vid->vb2_q.dev = kmb_vid->dma_dev;
+ kmb_vid->vb2_q.lock = &kmb_vid->lock;
+ kmb_vid->vb2_q.min_buffers_needed = 1;
+
+ kmb_vid->pad.flags = MEDIA_PAD_FL_SINK;
+ kmb_vid->video->device_caps = V4L2_CAP_VIDEO_CAPTURE_MPLANE |
+ V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
+
+ ret = media_entity_pads_init(&kmb_vid->video->entity, 1, &kmb_vid->pad);
+ if (ret < 0)
+ goto error_mutex_destroy;
+
+ ret = vb2_queue_init(&kmb_vid->vb2_q);
+ if (ret < 0) {
+ dev_err(&kmb_vid->video->dev, "Failed to init vb2 queue");
+ goto error_video_cleanup;
+ }
+
return 0;
+
+error_video_cleanup:
+ kmb_video_cleanup(kmb_vid);
+error_mutex_destroy:
+ mutex_destroy(&kmb_vid->lock);
+ mutex_destroy(&kmb_vid->dma_lock);
+
+ return ret;
}
/**
@@ -23,7 +886,11 @@ int kmb_video_init(struct kmb_video *kmb_vid, const char *name)
* @kmb_vid: pointer to kmb video device
*/
void kmb_video_cleanup(struct kmb_video *kmb_vid)
-{ }
+{
+ media_entity_cleanup(&kmb_vid->video->entity);
+ mutex_destroy(&kmb_vid->lock);
+ mutex_destroy(&kmb_vid->dma_lock);
+}
/**
* kmb_video_register - Register V4L2 device
@@ -35,7 +902,14 @@ void kmb_video_cleanup(struct kmb_video *kmb_vid)
int kmb_video_register(struct kmb_video *kmb_vid,
struct v4l2_device *v4l2_dev)
{
- return 0;
+ int ret;
+
+ kmb_vid->video->v4l2_dev = v4l2_dev;
+ ret = video_register_device(kmb_vid->video, VFL_TYPE_VIDEO, -1);
+ if (ret < 0)
+ dev_err(&kmb_vid->video->dev, "Failed to register video device");
+
+ return ret;
}
/**
@@ -43,4 +917,6 @@ int kmb_video_register(struct kmb_video *kmb_vid,
* @kmb_vid: pointer to kmb video device
*/
void kmb_video_unregister(struct kmb_video *kmb_vid)
-{ }
+{
+ video_unregister_device(kmb_vid->video);
+}
diff --git a/drivers/media/platform/keembay-camera/keembay-video.h b/drivers/media/platform/keembay-camera/keembay-video.h
index 2aebbb37424b..de25dfe3d684 100644
--- a/drivers/media/platform/keembay-camera/keembay-video.h
+++ b/drivers/media/platform/keembay-camera/keembay-video.h
@@ -16,7 +16,7 @@
/**
* struct kmb_frame_buffer - KMB frame buffer structure
* @vb: Video buffer for v4l2
- * @addr: Array of dma buffer plane address
+ * @addr: Array of dma buffer plane addresses
* @list: Frame buffer list
*/
struct kmb_frame_buffer {
@@ -28,50 +28,39 @@ struct kmb_frame_buffer {
/**
* struct kmb_video - KMB Video device structure
* @lock: Mutex serializing kmb video device ops
- * @video_lock: Mutex serializing video operations
* @video: Pointer to V4L2 sub-device
+ * @vb2_q: Video buffer queue
* @pad: Media pad graph objects
* @dma_dev: Pointer to dma device
+ * @dma_queue: DMA buffers queue
+ * @dma_lock: Mutex serializing dma queue ops
+ * @active_fmt: Active format
+ * @active_fmt.pix: Mplane active pixel format
+ * @active_fmt.info: Active kmb format info
* @pipe: Pointer to kmb media pipeline
- * @chan: Pointer to xlink channel
+ * @xlink_cam: Pointer to xlink camera communication handler
+ * @chan_id: Channel ID
+ * @thread: Pointer to worker thread data
*/
struct kmb_video {
- struct mutex lock; /* Lock protecting kmb video device */
- struct mutex video_lock; /* Lock serializing video device operations */
+ struct mutex lock;
struct video_device *video;
+ struct vb2_queue vb2_q;
struct media_pad pad;
+
struct device *dma_dev;
- struct kmb_pipeline *pipe;
- struct kmb_xlink_cam *xlink_cam;
- unsigned int chan_id;
-};
+ struct list_head dma_queue;
+ struct mutex dma_lock;
-/**
- * struct kmb_video_fh - KMB video file handler
- * @fh: V4L2 file handler
- * @kmb_vid: Pointer to KMB video device
- * @lock: Mutex serializing access to fh
- * @vb2_lock: Mutex serializing access to vb2 queue
- * @vb2_q: Video buffer queue
- * @active_fmt: Active format
- @pix: Mplane active pixel format
- @info: Active kmb format info
- * @contiguous_memory: Flag to enable contiguous memory allocation
- * @dma_queue: DMA buffers queue
- * @thread: Pointer to worker thread data
- */
-struct kmb_video_fh {
- struct v4l2_fh fh;
- struct kmb_video *kmb_vid;
- struct mutex lock; /* Lock protecting fh operations */
- struct mutex vb2_lock; /* Lock protecting video buffer queue */
- struct vb2_queue vb2_q;
struct {
struct v4l2_pix_format_mplane pix;
const struct kmb_video_fmt_info *info;
} active_fmt;
- bool contiguous_memory;
- struct list_head dma_queue;
+
+ struct kmb_pipeline *pipe;
+ struct kmb_xlink_cam *xlink_cam;
+ unsigned int chan_id;
+
struct task_struct *thread;
};
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread* Re: [PATCH 08/10] media: Keem Bay Camera: Add capture video node
2021-03-19 18:06 ` [PATCH 08/10] media: Keem Bay Camera: Add capture video node Martina Krasteva
@ 2021-04-09 14:32 ` Sakari Ailus
0 siblings, 0 replies; 28+ messages in thread
From: Sakari Ailus @ 2021-04-09 14:32 UTC (permalink / raw)
To: Martina Krasteva
Cc: linux-media, mchehab, robh+dt, devicetree, daniele.alessandrelli,
paul.j.murphy, gjorgjix.rosikopulos
Hi Martina,
A few more comments below.
On Fri, Mar 19, 2021 at 06:06:30PM +0000, Martina Krasteva wrote:
> From: Martina Krasteva <martinax.krasteva@intel.com>
>
> Capture video node implements v4l2 capture
> interface and XLink VPU Camera buffer pool operations.
>
> Build and set stream pipeline operations are also executed
> from capture video node. Resolution depends on remote
> entity pad connected to this video node.
>
> Co-developed-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
> Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
> Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
> Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
> Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
> ---
> .../platform/keembay-camera/keembay-pipeline.c | 192 +++++
> .../platform/keembay-camera/keembay-pipeline.h | 4 +
> .../media/platform/keembay-camera/keembay-video.c | 884 ++++++++++++++++++++-
> .../media/platform/keembay-camera/keembay-video.h | 51 +-
> 4 files changed, 1096 insertions(+), 35 deletions(-)
>
> diff --git a/drivers/media/platform/keembay-camera/keembay-pipeline.c b/drivers/media/platform/keembay-camera/keembay-pipeline.c
> index 78b2fffa42ee..0050361ef3c0 100644
> --- a/drivers/media/platform/keembay-camera/keembay-pipeline.c
> +++ b/drivers/media/platform/keembay-camera/keembay-pipeline.c
> @@ -60,6 +60,39 @@ static void kmb_pipe_print_config(struct kmb_pipeline *pipe)
> dev_dbg(dev, "\tinternal_memory_size %u\n", cfg->internal_memory_size);
> }
>
> +static unsigned int kmb_pipe_get_pending(struct media_entity *entity)
> +{
> + struct media_device *mdev = entity->graph_obj.mdev;
> + unsigned int num_vdevs = 0;
> + struct media_entity *next;
> + struct media_graph graph;
> + int ret;
> +
> + /* Walk through graph to count the connected video node entities */
> + mutex_lock(&mdev->graph_mutex);
> +
> + ret = media_graph_walk_init(&graph, mdev);
> + if (ret) {
> + mutex_unlock(&mdev->graph_mutex);
> + return -EINVAL;
> + }
> +
> + media_graph_walk_start(&graph, entity);
> +
> + while ((next = media_graph_walk_next(&graph))) {
> + if (!is_media_entity_v4l2_video_device(next))
> + continue;
> +
> + num_vdevs++;
> + }
> +
> + mutex_unlock(&mdev->graph_mutex);
> +
> + media_graph_walk_cleanup(&graph);
> +
> + return num_vdevs;
> +}
> +
> /**
> * kmb_pipe_init - Initialize KMB Pipeline
> * @pipe: pointer to pipeline object
> @@ -207,3 +240,162 @@ int kmb_pipe_config_src(struct kmb_pipeline *pipe,
> mutex_unlock(&pipe->lock);
> return ret;
> }
> +
> +/**
> + * kmb_pipe_prepare - Prepare VPU pipeline for streaming
> + * @pipe: pointer to pipeline object
> + *
> + * Prepare pipeline for streaming by sending negotiated configuration to VPU
> + * and changing state to BUILT.
> + *
> + * Return: 0 if successful, error code otherwise.
> + */
> +int kmb_pipe_prepare(struct kmb_pipeline *pipe)
> +{
> + int ret = 0;
> +
> + mutex_lock(&pipe->lock);
> +
> + /* build only if all outputs are configured */
> + switch (pipe->state) {
> + case KMB_PIPE_STATE_UNCONFIGURED:
> + /* Call config and continue */
> + ret = kmb_cam_xlink_write_ctrl_msg(pipe->xlink_cam,
> + pipe->pipe_cfg_paddr,
> + KMB_IC_EVENT_TYPE_CONFIG_ISP_PIPE,
> + KMB_IC_EVENT_TYPE_SUCCESSFUL);
> + if (ret < 0) {
> + dev_err(pipe->dev, "Failed to reconfigure pipeline!");
> + break;
> + }
> + fallthrough;
> + case KMB_PIPE_STATE_CONFIGURED:
> + ret = kmb_cam_xlink_write_ctrl_msg(pipe->xlink_cam,
> + pipe->pipe_cfg_paddr,
> + KMB_IC_EVENT_TYPE_BUILD_ISP_PIPE,
> + KMB_IC_EVENT_TYPE_SUCCESSFUL);
> + if (ret < 0) {
> + dev_err(pipe->dev, "Failed to build pipeline!");
> + break;
> + }
> + pipe->state = KMB_PIPE_STATE_BUILT;
> + break;
> + case KMB_PIPE_STATE_BUILT:
> + /* Pipeline is already built ignore */
> + break;
> + default:
> + dev_err(pipe->dev,
> + "Build pipe in invalid state %d", pipe->state);
> + ret = -EINVAL;
> + break;
> + }
> +
> + mutex_unlock(&pipe->lock);
> +
> + return ret;
> +}
> +
> +static int kmb_pipe_s_stream(struct kmb_pipeline *pipe,
> + struct media_entity *entity, int enable)
> +{
> + struct v4l2_subdev *subdev;
> + struct media_pad *remote;
> + int ret;
> +
> + remote = media_entity_remote_pad(entity->pads);
> + if (!remote || !is_media_entity_v4l2_subdev(remote->entity))
> + return -EINVAL;
> +
> + subdev = media_entity_to_v4l2_subdev(remote->entity);
> + if (!subdev)
> + return -EINVAL;
> +
> + ret = v4l2_subdev_call(subdev, video, s_stream, enable);
> + if (ret < 0 && ret != -ENOIOCTLCMD)
> + dev_err(pipe->dev, "Cannot set stream %d", enable);
> +
> + return ret != -ENOIOCTLCMD ? ret : 0;
> +}
> +
> +/**
> + * kmb_pipe_stop - Set stream off and stop media pipeline
> + * @pipe: KMB pipeline object
> + * @entity: media entity
> + */
> +void kmb_pipe_stop(struct kmb_pipeline *pipe, struct media_entity *entity)
> +{
> + mutex_lock(&pipe->lock);
> +
> + if (WARN_ON(!pipe->streaming)) {
> + dev_err(pipe->dev, "Calling stop on already stopped pipeline");
> + mutex_unlock(&pipe->lock);
> + return;
> + }
> +
> + if (pipe->state == KMB_PIPE_STATE_STREAMING) {
> + kmb_pipe_s_stream(pipe, entity, 0);
> + media_pipeline_stop(entity);
> + pipe->state = KMB_PIPE_STATE_BUILT;
> + }
> +
> + if (pipe->state == KMB_PIPE_STATE_BUILT ||
> + pipe->state == KMB_PIPE_STATE_CONFIGURED) {
> + kmb_cam_xlink_write_ctrl_msg(pipe->xlink_cam,
> + pipe->pipe_cfg_paddr,
> + KMB_IC_EVENT_TYPE_DELETE_ISP_PIPE,
> + KMB_IC_EVENT_TYPE_SUCCESSFUL);
> +
> + pipe->state = KMB_PIPE_STATE_UNCONFIGURED;
> + }
> +
> + pipe->streaming--;
> +
> + mutex_unlock(&pipe->lock);
> +}
> +
> +/**
> + * kmb_pipe_run - Run media pipeline and start streaming
> + * @pipe: KMB pipeline object
> + * @entity: media entity
> + *
> + * Return: 0 if successful, error code otherwise.
> + */
> +int kmb_pipe_run(struct kmb_pipeline *pipe, struct media_entity *entity)
> +{
> + int ret = 0;
> +
> + mutex_lock(&pipe->lock);
> +
> + if (!pipe->streaming)
> + pipe->pending = kmb_pipe_get_pending(entity);
> +
> + pipe->streaming++;
> +
> + if (pipe->streaming != pipe->pending)
> + goto done_unlock;
> +
> + if (pipe->state != KMB_PIPE_STATE_BUILT) {
> + ret = -EINVAL;
> + goto done_unlock;
> + }
> +
> + ret = media_pipeline_start(entity, &pipe->media_pipe);
> + if (ret < 0) {
> + dev_err(pipe->dev, "Failed to start media pipeline");
> + goto done_unlock;
> + }
> +
> + ret = kmb_pipe_s_stream(pipe, entity, 1);
> + if (ret < 0 && ret != -ENOIOCTLCMD) {
> + mutex_unlock(&pipe->lock);
> + kmb_pipe_stop(pipe, entity);
The caller kmb_metadata_start_streaming() already calls kmb_pipe_stop() if
kmb_pipe_run() returns an error.
> + return ret;
> + }
> +
> + pipe->state = KMB_PIPE_STATE_STREAMING;
> +
> +done_unlock:
> + mutex_unlock(&pipe->lock);
> +
> + return ret;
> +}
> diff --git a/drivers/media/platform/keembay-camera/keembay-pipeline.h b/drivers/media/platform/keembay-camera/keembay-pipeline.h
> index 83ff94d11b34..60ba99e9a73c 100644
> --- a/drivers/media/platform/keembay-camera/keembay-pipeline.h
> +++ b/drivers/media/platform/keembay-camera/keembay-pipeline.h
> @@ -68,4 +68,8 @@ void kmb_pipe_config_dest(struct kmb_pipeline *pipe, unsigned int output_id,
> int kmb_pipe_config_src(struct kmb_pipeline *pipe,
> struct kmb_pipe_config_evs *pipe_cfg);
>
> +int kmb_pipe_prepare(struct kmb_pipeline *pipe);
> +int kmb_pipe_run(struct kmb_pipeline *pipe, struct media_entity *entity);
> +void kmb_pipe_stop(struct kmb_pipeline *pipe, struct media_entity *entity);
> +
> #endif /* KEEMBAY_PIPELINE_H */
> diff --git a/drivers/media/platform/keembay-camera/keembay-video.c b/drivers/media/platform/keembay-camera/keembay-video.c
> index 02f4d97e16fb..a92cfbeffea9 100644
> --- a/drivers/media/platform/keembay-camera/keembay-video.c
> +++ b/drivers/media/platform/keembay-camera/keembay-video.c
> @@ -2,9 +2,816 @@
> /*
> * Intel Keem Bay camera Video node.
> *
> - * Copyright (C) 2018-2020 Intel Corporation
> + * Copyright (C) 2021 Intel Corporation
This should be in an earlier patch.
> */
> +#include <linux/delay.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/freezer.h>
> +#include <linux/kthread.h>
> +
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-mc.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "keembay-cam-xlink.h"
> +#include "keembay-pipeline.h"
> #include "keembay-video.h"
> +#include "keembay-vpu-frame.h"
> +
> +#define KMB_CAM_VIDEO_NAME "keembay-video"
> +
> +/* Xlink data channel size and timeout */
> +#define KMB_VID_CH_DATA_SIZE 1024
> +#define KMB_VID_CH_TIMEOUT_MS 5000
> +
> +#define KMB_VID_MIN_WIDTH 16
> +#define KMB_VID_MIN_HEIGHT 16
> +#define KMB_VID_MAX_WIDTH U16_MAX
> +#define KMB_VID_MAX_HEIGHT U16_MAX
> +#define KMB_VID_STEP_WIDTH 8
> +#define KMB_VID_STEP_HEIGHT 8
> +
> +#define to_kmb_video_buf(vbuf) container_of(vbuf, struct kmb_frame_buffer, vb)
> +
> +/* Kmb video format info structure */
> +struct kmb_video_fmt_info {
> + const char *description;
> + u32 code;
> + u32 pixelformat;
> + enum kmb_frame_types type;
> + u32 colorspace;
> + unsigned int planes;
> + unsigned int bpp;
> + unsigned int h_subsample;
> + unsigned int v_subsample;
> + bool contiguous_memory;
> +};
> +
> +/* Supported video formats */
> +static const struct kmb_video_fmt_info video_formats[] = {
> + {
> + .description = "NV12",
You can drop this field and the related assignment for format enumeration
IOCTL handling. It's already filled by v4l2_fill_fmtdesc in v4l2-ioctl.c.
> + .code = MEDIA_BUS_FMT_YUYV8_1_5X8,
> + .pixelformat = V4L2_PIX_FMT_NV12,
> + .type = KMB_FRAME_TYPE_NV12,
> + .colorspace = V4L2_COLORSPACE_SRGB,
> + .planes = 2,
> + .bpp = 8,
> + .h_subsample = 1,
> + .v_subsample = 2,
> + .contiguous_memory = true,
> + },
> + {
> + .description = "Planar YUV 4:2:0",
> + .code = MEDIA_BUS_FMT_UYYVYY8_0_5X24,
> + .pixelformat = V4L2_PIX_FMT_YUV420,
> + .type = KMB_FRAME_TYPE_YUV420P,
> + .colorspace = V4L2_COLORSPACE_SRGB,
> + .planes = 3,
> + .bpp = 8,
> + .h_subsample = 2,
> + .v_subsample = 2,
> + .contiguous_memory = false,
> + },
> + {
> + .description = "Planar YUV 4:4:4",
> + .code = MEDIA_BUS_FMT_YUV8_1X24,
> + .pixelformat = V4L2_PIX_FMT_YUV444,
> + .type = KMB_FRAME_TYPE_YUV444P,
> + .colorspace = V4L2_COLORSPACE_SRGB,
> + .planes = 3,
> + .bpp = 8,
> + .h_subsample = 1,
> + .v_subsample = 1,
> + .contiguous_memory = false,
> + },
> + {
> + .description = "RAW 8 Garyscale",
> + .code = MEDIA_BUS_FMT_Y8_1X8,
> + .pixelformat = V4L2_PIX_FMT_GREY,
> + .type = KMB_FRAME_TYPE_RAW8,
> + .colorspace = V4L2_COLORSPACE_RAW,
> + .planes = 1,
> + .bpp = 8,
> + .h_subsample = 1,
> + .v_subsample = 1,
> + .contiguous_memory = false,
> + },
> + {
> + .description = "RAW 10 Grayscale",
> + .code = MEDIA_BUS_FMT_Y10_1X10,
> + .pixelformat = V4L2_PIX_FMT_Y10,
> + .type = KMB_FRAME_TYPE_RAW10,
> + .colorspace = V4L2_COLORSPACE_RAW,
> + .planes = 1,
> + .bpp = 10,
> + .h_subsample = 1,
> + .v_subsample = 1,
> + .contiguous_memory = false,
> + }
> +};
> +
> +static const struct kmb_video_fmt_info *
> +kmb_video_get_fmt_info_by_code(u32 code)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(video_formats); i++)
> + if (video_formats[i].code == code)
> + return &video_formats[i];
> +
> + return NULL;
> +}
> +
> +static const struct kmb_video_fmt_info *
> +kmb_video_get_fmt_info_by_pixfmt(u32 pix_fmt)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(video_formats); i++)
> + if (video_formats[i].pixelformat == pix_fmt)
> + return &video_formats[i];
> +
> + return NULL;
> +}
> +
> +/* Buffer processing operations */
> +static void kmb_video_insert_buf(struct kmb_video *kmb_vid,
> + struct kmb_frame_buffer *buf)
> +{
> + INIT_LIST_HEAD(&buf->list);
> +
> + mutex_lock(&kmb_vid->dma_lock);
> + list_add_tail(&buf->list, &kmb_vid->dma_queue);
> + mutex_unlock(&kmb_vid->dma_lock);
> +}
> +
> +static void __kmb_video_buf_discard(struct kmb_video *kmb_vid,
> + struct kmb_frame_buffer *buf)
> +{
> + lockdep_assert_held(&kmb_vid->dma_lock);
> +
> + list_del(&buf->list);
> + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
> +}
> +
> +static int kmb_video_process_buf(struct kmb_video *kmb_vid,
> + struct kmb_frame_buffer *buf)
> +{
> + const struct kmb_video_fmt_info *info = kmb_vid->active_fmt.info;
> + struct v4l2_pix_format_mplane *pix = &kmb_vid->active_fmt.pix;
> + struct kmb_vpu_frame_buffer rt_frame_buf;
> + int ret;
> +
> + lockdep_assert_held(&kmb_vid->lock);
> +
> + memset(&rt_frame_buf, 0, sizeof(rt_frame_buf));
> + rt_frame_buf.spec.bpp = info->bpp;
> + rt_frame_buf.spec.type = info->type;
> + rt_frame_buf.spec.width = pix->width;
> + rt_frame_buf.spec.height = pix->height;
> + rt_frame_buf.spec.stride = pix->plane_fmt[0].bytesperline;
> + rt_frame_buf.p1 = buf->addr[0];
> +
> + /* Planes not used by the VPU should be set with addr 0 */
> + if (pix->num_planes > 1)
> + rt_frame_buf.p2 = buf->addr[1];
> + if (pix->num_planes > 2)
> + rt_frame_buf.p3 = buf->addr[2];
> +
> + ret = kmb_cam_xlink_write_msg(kmb_vid->xlink_cam,
> + kmb_vid->chan_id,
Unwrap, maybe?
> + (u8 *)&rt_frame_buf,
Could the function use a void pointer instead so you could remove the cast
(also other instances)?
> + sizeof(rt_frame_buf));
> + if (ret < 0) {
> + dev_err(kmb_vid->dma_dev, "Error on buffer queue %d", ret);
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> +static void kmb_video_process_all_bufs(struct kmb_video *kmb_vid)
> +{
> + struct kmb_frame_buffer *buf;
> + struct list_head *next;
> + struct list_head *pos;
> + int ret;
> +
> + mutex_lock(&kmb_vid->dma_lock);
> +
> + /* Discard buf is removing buffer from the list */
> + list_for_each_safe(pos, next, &kmb_vid->dma_queue) {
> + buf = list_entry(pos, struct kmb_frame_buffer, list);
> +
> + ret = kmb_video_process_buf(kmb_vid, buf);
> + if (ret) {
> + dev_err(&kmb_vid->video->dev,
> + "Cannot process output buf 0x%pad",
> + &buf->addr[0]);
> + __kmb_video_buf_discard(kmb_vid, buf);
> + continue;
> + }
> + }
> +
> + mutex_unlock(&kmb_vid->dma_lock);
> +}
> +
> +static int kmb_video_queue_output_buf(struct kmb_video *kmb_vid,
> + struct kmb_frame_buffer *buf)
> +{
> + int ret = 0;
> +
> + kmb_video_insert_buf(kmb_vid, buf);
> +
> + mutex_lock(&kmb_vid->dma_lock);
> +
> + /* Process buffers only when device is streaming */
> + if (vb2_is_streaming(&kmb_vid->vb2_q)) {
> + ret = kmb_video_process_buf(kmb_vid, buf);
> + if (ret) {
> + dev_err(&kmb_vid->video->dev,
> + "Fail to process output buf 0x%pad",
> + &buf->addr[0]);
> + __kmb_video_buf_discard(kmb_vid, buf);
> + }
> + }
> +
> + mutex_unlock(&kmb_vid->dma_lock);
> +
> + return ret;
> +}
> +
> +static void kmb_video_release_all_bufs(struct kmb_video *kmb_vid,
> + enum vb2_buffer_state state)
> +{
> + struct list_head *next = NULL;
> + struct list_head *pos = NULL;
> + struct kmb_frame_buffer *buf;
> +
> + mutex_lock(&kmb_vid->dma_lock);
> + list_for_each_safe(pos, next, &kmb_vid->dma_queue) {
> + buf = list_entry(pos, struct kmb_frame_buffer, list);
> + list_del(&buf->list);
> + vb2_buffer_done(&buf->vb.vb2_buf, state);
> + }
> + mutex_unlock(&kmb_vid->dma_lock);
> +}
> +
> +static void kmb_video_remove_buf(struct kmb_video *kmb_vid,
> + struct kmb_frame_buffer *buf)
> +{
> + mutex_lock(&kmb_vid->dma_lock);
> + list_del(&buf->list);
> + mutex_unlock(&kmb_vid->dma_lock);
> +}
> +
> +static struct kmb_frame_buffer *
> +kmb_video_find_buf_by_addr(struct kmb_video *kmb_vid, uint64_t addr)
> +{
> + struct kmb_frame_buffer *buf = NULL;
> + struct list_head *node = NULL;
> +
> + mutex_lock(&kmb_vid->dma_lock);
> +
> + list_for_each(node, &kmb_vid->dma_queue) {
> + buf = list_entry(node, struct kmb_frame_buffer, list);
> + if (buf->addr[0] == addr) {
> + mutex_unlock(&kmb_vid->dma_lock);
> + return buf;
> + }
> + }
> +
> + mutex_unlock(&kmb_vid->dma_lock);
> +
> + return NULL;
> +}
> +
> +static void kmb_video_fmt_info_to_pix(const struct kmb_video_fmt_info *info,
> + struct v4l2_mbus_framefmt *mbus_fmt,
> + struct v4l2_pix_format_mplane *pix)
> +{
> + u32 bytesperline;
> + u32 sizeimage;
> + u32 v_sub = 1;
> + u32 h_sub = 1;
> + unsigned int i;
> +
> + pix->width = mbus_fmt->width;
> + pix->height = mbus_fmt->height;
> +
> + pix->pixelformat = info->pixelformat;
> + pix->colorspace = info->colorspace;
> + pix->num_planes = info->planes;
> +
> + for (i = 0; i < pix->num_planes; i++) {
> + bytesperline = pix->width * info->bpp / 8 / h_sub;
> +
> + if (pix->plane_fmt[i].bytesperline < bytesperline)
> + pix->plane_fmt[i].bytesperline = bytesperline;
> +
> + sizeimage = pix->plane_fmt[i].bytesperline *
> + pix->height / v_sub;
> +
> + if (pix->plane_fmt[i].sizeimage < sizeimage)
> + pix->plane_fmt[i].sizeimage = sizeimage;
> +
> + h_sub = info->h_subsample;
> + v_sub = info->v_subsample;
> + }
> +}
> +
> +static int kmb_video_get_subdev_fmt(struct kmb_video *kmb_vid,
> + struct v4l2_pix_format_mplane *pix)
> +{
> + const struct kmb_video_fmt_info *fmt_info;
> + struct v4l2_subdev_format sd_fmt;
> + struct v4l2_subdev *subdev;
> + struct media_pad *remote;
> + int ret;
> +
> + remote = media_entity_remote_pad(&kmb_vid->pad);
> + if (!remote || !is_media_entity_v4l2_subdev(remote->entity))
> + return -EINVAL;
> +
> + subdev = media_entity_to_v4l2_subdev(remote->entity);
> + if (!subdev)
> + return -EINVAL;
> +
> + memset(&sd_fmt, 0, sizeof(sd_fmt));
> + sd_fmt.pad = remote->index;
> + sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> + ret = v4l2_subdev_call(subdev, pad, get_fmt, NULL, &sd_fmt);
> + if (ret < 0)
> + return ret;
> +
> + fmt_info = kmb_video_get_fmt_info_by_code(sd_fmt.format.code);
> + if (!fmt_info)
> + return -EINVAL;
> +
> + kmb_video_fmt_info_to_pix(fmt_info, &sd_fmt.format, pix);
> +
> + return 0;
> +}
> +
> +static int kmb_video_queue_setup(struct vb2_queue *q,
> + unsigned int *num_buffers,
> + unsigned int *num_planes,
> + unsigned int sizes[],
> + struct device *alloc_devs[])
> +{
> + struct kmb_video *kmb_vid = vb2_get_drv_priv(q);
> + struct v4l2_pix_format_mplane *pix = &kmb_vid->active_fmt.pix;
> + unsigned int i;
> +
> + if (kmb_vid->active_fmt.info->contiguous_memory) {
> + *num_planes = 1;
> + for (i = 0; i < pix->num_planes; i++)
> + sizes[0] += pix->plane_fmt[i].sizeimage;
> + } else {
> + *num_planes = pix->num_planes;
> + for (i = 0; i < pix->num_planes; i++)
> + sizes[i] = pix->plane_fmt[i].sizeimage;
> + }
> +
> + return 0;
> +}
> +
> +static int kmb_video_buffer_prepare(struct vb2_buffer *vb)
> +{
> + struct kmb_video *kmb_vid = vb2_get_drv_priv(vb->vb2_queue);
> + struct v4l2_pix_format_mplane *pix = &kmb_vid->active_fmt.pix;
> + unsigned int size_image = 0;
> + unsigned int i;
> +
> + if (kmb_vid->active_fmt.info->contiguous_memory) {
> + for (i = 0; i < pix->num_planes; i++)
> + size_image += pix->plane_fmt[i].sizeimage;
> +
> + vb2_set_plane_payload(vb, 0, size_image);
> + } else {
> + for (i = 0; i < pix->num_planes; i++)
> + vb2_set_plane_payload(vb, i,
> + pix->plane_fmt[i].sizeimage);
> + }
> +
> + return 0;
> +}
> +
> +static int kmb_video_buf_init(struct vb2_buffer *vb)
> +{
> + struct kmb_video *kmb_vid = vb2_get_drv_priv(vb->vb2_queue);
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + struct kmb_frame_buffer *buf = to_kmb_video_buf(vbuf);
> + struct v4l2_pix_format_mplane *pix = &kmb_vid->active_fmt.pix;
> + unsigned int i;
> +
> + if (kmb_vid->active_fmt.info->contiguous_memory) {
> + buf->addr[0] = vb2_dma_contig_plane_dma_addr(vb, 0);
> + for (i = 1; i < pix->num_planes; i++) {
> + buf->addr[i] = buf->addr[i - 1] +
> + pix->plane_fmt[i - 1].sizeimage;
> + }
> + } else {
> + for (i = 0; i < pix->num_planes; i++)
> + buf->addr[i] = vb2_dma_contig_plane_dma_addr(vb, i);
> + }
> +
> + return 0;
> +}
> +
> +static void kmb_video_buf_queue(struct vb2_buffer *vb)
> +{
> + struct kmb_video *kmb_vid = vb2_get_drv_priv(vb->vb2_queue);
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + struct kmb_frame_buffer *buf = to_kmb_video_buf(vbuf);
> + int ret;
> +
> + ret = kmb_video_queue_output_buf(kmb_vid, buf);
> + if (ret)
> + dev_err(kmb_vid->dma_dev, "Fail output buf queue %d", ret);
> +}
> +
> +static int kmb_video_worker_thread(void *video)
> +{
> + struct kmb_vpu_frame_buffer rt_frame_buf;
> + struct kmb_video *kmb_vid = video;
> + struct kmb_frame_buffer *buf = NULL;
> + bool stopped = false;
> + int ret;
> +
> + set_freezable();
> +
> + while (!kthread_should_stop()) {
> + try_to_freeze();
> +
> + if (stopped) {
> + set_current_state(TASK_INTERRUPTIBLE);
> + schedule();
> + continue;
> + }
> +
> + memset(&rt_frame_buf, 0, sizeof(rt_frame_buf));
> + ret = kmb_cam_xlink_read_msg(kmb_vid->xlink_cam,
> + kmb_vid->chan_id,
> + (u8 *)&rt_frame_buf,
> + sizeof(rt_frame_buf));
> + if (ret < 0) {
> + stopped = true;
> + /* Continue here to enter in freeze state */
> + continue;
> + }
> +
> + buf = kmb_video_find_buf_by_addr(kmb_vid, rt_frame_buf.p1);
> + if (buf) {
> + kmb_video_remove_buf(kmb_vid, buf);
> +
> + buf->vb.vb2_buf.timestamp = rt_frame_buf.ts;
> + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
> + } else {
> + dev_err(kmb_vid->dma_dev, "Ouch cannot find buff %llx",
> + rt_frame_buf.p1);
> + }
> + }
> +
> + return 0;
> +}
> +
> +static int kmb_video_worker_start(struct kmb_video *kmb_vid)
> +{
> + int ret;
> +
> + ret = kmb_cam_xlink_open_channel(kmb_vid->xlink_cam, kmb_vid->chan_id);
> + if (ret)
> + return ret;
> +
> + kmb_vid->thread = kthread_run(kmb_video_worker_thread,
> + kmb_vid, "kmb_vnode_thread");
> + if (IS_ERR(kmb_vid->thread)) {
> + dev_err(&kmb_vid->video->dev, "Cannot start thread");
> + ret = -ENOMEM;
> + kmb_vid->thread = NULL;
> + goto error_close_xlink_channel;
> + }
> +
> + return 0;
> +
> +error_close_xlink_channel:
You have a single label here, I'd move this to where the label is and
remove the goto.
> + kmb_cam_xlink_close_channel(kmb_vid->xlink_cam, kmb_vid->chan_id);
> +
> + return ret;
> +}
> +
> +static int kmb_video_worker_stop(struct kmb_video *kmb_vid)
> +{
> + int ret;
> +
> + /*
> + * Xlink has no functionality to unblock read volatile function,
> + * only way to unblock is to close the channel.
> + */
> + kmb_cam_xlink_close_channel(kmb_vid->xlink_cam, kmb_vid->chan_id);
> + if (!kmb_vid->thread) {
> + dev_warn(&kmb_vid->video->dev, "No thread running");
> + return 0;
> + }
> +
> + ret = kthread_stop(kmb_vid->thread);
> + if (ret < 0)
> + dev_err(&kmb_vid->video->dev, "Thread stop failed %d", ret);
> +
> + kmb_vid->thread = NULL;
> +
> + return ret;
> +}
> +
> +static int kmb_video_capture_start_streaming(struct vb2_queue *q,
> + unsigned int count)
> +{
> + struct kmb_video *kmb_vid = vb2_get_drv_priv(q);
> + int ret;
> +
> + ret = kmb_pipe_prepare(kmb_vid->pipe);
> + if (ret < 0)
> + goto error_discard_all_bufs;
> +
> + ret = kmb_video_worker_start(kmb_vid);
> + if (ret < 0)
> + goto error_pipeline_stop;
> +
> + /* Process all pending buffers after worker is started */
> + kmb_video_process_all_bufs(kmb_vid);
> +
> + /*
> + * Run the pipeline after all buffers are provided for processing,
> + * the main reason is to not skip any frame from the source.
> + */
> + ret = kmb_pipe_run(kmb_vid->pipe, &kmb_vid->video->entity);
> + if (ret < 0)
> + goto error_pipeline_stop;
> +
> + return 0;
> +
> +error_pipeline_stop:
> + kmb_pipe_stop(kmb_vid->pipe, &kmb_vid->video->entity);
> +error_discard_all_bufs:
> + kmb_video_release_all_bufs(kmb_vid, VB2_BUF_STATE_QUEUED);
> +
> + return ret;
> +}
> +
> +static void kmb_video_capture_stop_streaming(struct vb2_queue *q)
> +{
> + struct kmb_video *kmb_vid = vb2_get_drv_priv(q);
> +
> + kmb_pipe_stop(kmb_vid->pipe, &kmb_vid->video->entity);
> +
> + kmb_video_worker_stop(kmb_vid);
> +
> + kmb_video_release_all_bufs(kmb_vid, VB2_BUF_STATE_ERROR);
> +}
> +
> +/* driver-specific operations */
> +static const struct vb2_ops kmb_video_vb2_q_capture_ops = {
> + .queue_setup = kmb_video_queue_setup,
> + .buf_prepare = kmb_video_buffer_prepare,
> + .buf_init = kmb_video_buf_init,
> + .buf_queue = kmb_video_buf_queue,
> + .start_streaming = kmb_video_capture_start_streaming,
> + .stop_streaming = kmb_video_capture_stop_streaming,
> +};
> +
> +static int kmb_video_querycap(struct file *file, void *fh,
> + struct v4l2_capability *cap)
> +{
> + cap->bus_info[0] = 0;
> + strscpy(cap->driver, KMB_CAM_VIDEO_NAME, sizeof(cap->driver));
> + strscpy(cap->card, KMB_CAM_VIDEO_NAME, sizeof(cap->card));
> +
> + return 0;
> +}
> +
> +static int kmb_video_enum_fmt(struct file *file, void *fh,
> + struct v4l2_fmtdesc *f)
> +{
> + const struct kmb_video_fmt_info *info;
> +
> + if (!V4L2_TYPE_IS_MULTIPLANAR(f->type))
> + return -EINVAL;
> +
> + if (f->mbus_code) {
> + if (f->index != 0)
> + return -EINVAL;
> +
> + info = kmb_video_get_fmt_info_by_code(f->mbus_code);
> + if (!info)
> + return -EINVAL;
> + } else {
> + info = &video_formats[f->index];
> + if (!info)
> + return -EINVAL;
> + }
> +
> + f->pixelformat = info->pixelformat;
> + f->mbus_code = info->code;
> + strscpy(f->description, info->description, sizeof(f->description));
> +
> + return 0;
> +}
> +
> +static int kmb_video_enum_framesizes(struct file *file, void *fh,
> + struct v4l2_frmsizeenum *fsize)
> +{
> + const struct kmb_video_fmt_info *info;
> +
> + if (fsize->index != 0)
> + return -EINVAL;
> +
> + info = kmb_video_get_fmt_info_by_pixfmt(fsize->pixel_format);
> + if (!info)
> + return -EINVAL;
> +
> + fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> +
> + fsize->stepwise.min_width = KMB_VID_MIN_WIDTH;
> + fsize->stepwise.max_width = KMB_VID_MAX_WIDTH;
> + fsize->stepwise.step_width = KMB_VID_STEP_WIDTH;
> + fsize->stepwise.min_height = KMB_VID_MIN_HEIGHT;
> + fsize->stepwise.max_height = KMB_VID_MAX_HEIGHT;
> + fsize->stepwise.step_height = KMB_VID_STEP_HEIGHT;
> +
> + return 0;
> +}
> +
> +static int kmb_video_try_fmt(struct file *file, void *fh,
> + struct v4l2_format *f)
> +{
> + const struct kmb_video_fmt_info *info;
> + struct v4l2_mbus_framefmt mbus_fmt;
> +
> + info = kmb_video_get_fmt_info_by_pixfmt(f->fmt.pix_mp.pixelformat);
> + if (!info)
> + info = &video_formats[0];
> +
> + mbus_fmt.width = f->fmt.pix_mp.width;
> + mbus_fmt.height = f->fmt.pix_mp.height;
> + kmb_video_fmt_info_to_pix(info, &mbus_fmt, &f->fmt.pix_mp);
> +
> + return 0;
> +}
> +
> +static int kmb_video_set_fmt(struct file *file, void *fh,
> + struct v4l2_format *f)
> +{
> + struct kmb_video *kmb_vid = video_drvdata(file);
> + const struct kmb_video_fmt_info *info;
> + struct v4l2_mbus_framefmt mbus_fmt;
> +
> + info = kmb_video_get_fmt_info_by_pixfmt(f->fmt.pix_mp.pixelformat);
> + if (!info)
> + info = &video_formats[0];
> +
> + mbus_fmt.width = f->fmt.pix_mp.width;
> + mbus_fmt.height = f->fmt.pix_mp.height;
> + kmb_video_fmt_info_to_pix(info, &mbus_fmt, &f->fmt.pix_mp);
> +
> + kmb_vid->active_fmt.pix = f->fmt.pix_mp;
> + kmb_vid->active_fmt.info = info;
> +
> + return 0;
> +}
> +
> +static int kmb_video_get_fmt(struct file *file, void *fh,
> + struct v4l2_format *f)
> +{
> + struct kmb_video *kmb_vid = video_drvdata(file);
> +
> + f->fmt.pix_mp = kmb_vid->active_fmt.pix;
> +
> + return 0;
> +}
> +
> +static int kmb_video_check_format(struct kmb_video *kmb_vid)
> +{
> + int ret;
> + struct v4l2_pix_format_mplane pix;
> +
> + ret = kmb_video_get_subdev_fmt(kmb_vid, &pix);
> + if (ret < 0)
> + return ret;
> +
> + if (kmb_vid->active_fmt.pix.pixelformat != pix.pixelformat ||
> + kmb_vid->active_fmt.pix.height != pix.height ||
> + kmb_vid->active_fmt.pix.width != pix.width ||
> + kmb_vid->active_fmt.pix.num_planes != pix.num_planes) {
> + dev_err(&kmb_vid->video->dev, "Pix fmt mismatch:\n\t"
> + "pix_fmt %u %u\n\theight %u %u\n\twidth %u %u\n\t"
> + "num_planes %u %u",
> + kmb_vid->active_fmt.pix.pixelformat, pix.pixelformat,
> + kmb_vid->active_fmt.pix.height, pix.height,
> + kmb_vid->active_fmt.pix.width, pix.width,
> + kmb_vid->active_fmt.pix.num_planes, pix.num_planes);
> + ret = -EINVAL;
> + }
> +
> + return ret;
> +}
> +
> +static int kmb_video_streamon(struct file *file, void *fh,
> + enum v4l2_buf_type type)
> +{
> + struct kmb_video *kmb_vid = video_drvdata(file);
> + int ret;
> +
> + if (type != kmb_vid->vb2_q.type)
> + return -EINVAL;
> +
> + ret = kmb_video_check_format(kmb_vid);
> + if (ret < 0)
> + return ret;
> +
> + return vb2_streamon(&kmb_vid->vb2_q, type);
> +}
> +
> +/* V4L2 ioctl operations */
> +static const struct v4l2_ioctl_ops kmb_vid_ioctl_ops = {
> + .vidioc_querycap = kmb_video_querycap,
> + .vidioc_enum_fmt_vid_cap = kmb_video_enum_fmt,
> + .vidioc_enum_framesizes = kmb_video_enum_framesizes,
> + .vidioc_g_fmt_vid_cap_mplane = kmb_video_get_fmt,
> + .vidioc_try_fmt_vid_cap_mplane = kmb_video_try_fmt,
> + .vidioc_s_fmt_vid_cap_mplane = kmb_video_set_fmt,
> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> + .vidioc_querybuf = vb2_ioctl_querybuf,
> + .vidioc_qbuf = vb2_ioctl_qbuf,
> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> + .vidioc_streamon = kmb_video_streamon,
> + .vidioc_streamoff = vb2_ioctl_streamoff,
> + .vidioc_expbuf = vb2_ioctl_expbuf,
> +};
> +
> +static int kmb_video_open(struct file *file)
> +{
> + struct kmb_video *kmb_vid = video_drvdata(file);
> + struct v4l2_mbus_framefmt fmt;
> + int ret;
> +
> + mutex_lock(&kmb_vid->lock);
> + ret = v4l2_fh_open(file);
> + if (ret) {
> + mutex_unlock(&kmb_vid->lock);
> + return ret;
> + }
> +
> + INIT_LIST_HEAD(&kmb_vid->dma_queue);
> +
> + ret = kmb_pipe_request(kmb_vid->pipe);
If there are n file handles open to this device node, how does it affect
the xlink channels? Is there a counter on a givne channel, or will there be
as many channels?
> + if (ret < 0)
> + goto error_fh_release;
> +
> + /* Fill default format. */
> + memset(&fmt, 0, sizeof(fmt));
> + kmb_video_fmt_info_to_pix(&video_formats[0], &fmt,
> + &kmb_vid->active_fmt.pix);
> + kmb_vid->active_fmt.info = &video_formats[0];
> +
> + mutex_unlock(&kmb_vid->lock);
> +
> + return 0;
> +
> +error_fh_release:
> + _vb2_fop_release(file, NULL);
Shouldn't this be v4l2_fh_release(file)?
You could also add a label for unlocking the mutex here instead of doing it
separately in the begining of the function.
> + mutex_unlock(&kmb_vid->lock);
> +
> + return ret;
> +}
> +
> +static int kmb_video_release(struct file *file)
> +{
> + struct kmb_video *kmb_vid = video_drvdata(file);
> + int ret;
> +
> + mutex_lock(&kmb_vid->lock);
> +
> + kmb_pipe_release(kmb_vid->pipe);
> +
> + ret = _vb2_fop_release(file, NULL);
> +
> + mutex_unlock(&kmb_vid->lock);
> +
> + return ret;
> +}
> +
> +/* FS operations for V4L2 device */
> +static const struct v4l2_file_operations kmb_vid_fops = {
> + .owner = THIS_MODULE,
> + .unlocked_ioctl = video_ioctl2,
> + .open = kmb_video_open,
> + .release = kmb_video_release,
> + .poll = vb2_fop_poll,
> + .mmap = vb2_fop_mmap,
> +};
>
> /**
> * kmb_video_init - Initialize entity
> @@ -15,7 +822,63 @@
> */
> int kmb_video_init(struct kmb_video *kmb_vid, const char *name)
> {
> + int ret;
> +
> + kmb_vid->video = video_device_alloc();
> + if (!kmb_vid->video) {
> + dev_err(&kmb_vid->video->dev,
> + "Failed to allocate video device");
> + return -ENOMEM;
> + }
> +
> + mutex_init(&kmb_vid->lock);
> + mutex_init(&kmb_vid->dma_lock);
> +
> + kmb_vid->video->fops = &kmb_vid_fops;
> + kmb_vid->video->ioctl_ops = &kmb_vid_ioctl_ops;
> + kmb_vid->video->minor = -1;
> + kmb_vid->video->release = video_device_release;
> + kmb_vid->video->vfl_type = VFL_TYPE_VIDEO;
> + kmb_vid->video->lock = &kmb_vid->lock;
> + kmb_vid->video->queue = &kmb_vid->vb2_q;
> + video_set_drvdata(kmb_vid->video, kmb_vid);
> + snprintf(kmb_vid->video->name, sizeof(kmb_vid->video->name),
> + "kmb_video %s", name);
> +
> + kmb_vid->vb2_q.drv_priv = kmb_vid;
> + kmb_vid->vb2_q.ops = &kmb_video_vb2_q_capture_ops;
> + kmb_vid->vb2_q.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
> + kmb_vid->vb2_q.buf_struct_size = sizeof(struct kmb_frame_buffer);
> + kmb_vid->vb2_q.io_modes = VB2_MMAP | VB2_DMABUF;
> + kmb_vid->vb2_q.timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> + kmb_vid->vb2_q.mem_ops = &vb2_dma_contig_memops;
> + kmb_vid->vb2_q.dev = kmb_vid->dma_dev;
> + kmb_vid->vb2_q.lock = &kmb_vid->lock;
> + kmb_vid->vb2_q.min_buffers_needed = 1;
> +
> + kmb_vid->pad.flags = MEDIA_PAD_FL_SINK;
> + kmb_vid->video->device_caps = V4L2_CAP_VIDEO_CAPTURE_MPLANE |
> + V4L2_CAP_STREAMING | V4L2_CAP_IO_MC;
> +
> + ret = media_entity_pads_init(&kmb_vid->video->entity, 1, &kmb_vid->pad);
> + if (ret < 0)
> + goto error_mutex_destroy;
> +
> + ret = vb2_queue_init(&kmb_vid->vb2_q);
> + if (ret < 0) {
> + dev_err(&kmb_vid->video->dev, "Failed to init vb2 queue");
> + goto error_video_cleanup;
> + }
> +
> return 0;
> +
> +error_video_cleanup:
> + kmb_video_cleanup(kmb_vid);
> +error_mutex_destroy:
> + mutex_destroy(&kmb_vid->lock);
> + mutex_destroy(&kmb_vid->dma_lock);
> +
> + return ret;
> }
>
> /**
> @@ -23,7 +886,11 @@ int kmb_video_init(struct kmb_video *kmb_vid, const char *name)
> * @kmb_vid: pointer to kmb video device
> */
> void kmb_video_cleanup(struct kmb_video *kmb_vid)
> -{ }
> +{
> + media_entity_cleanup(&kmb_vid->video->entity);
> + mutex_destroy(&kmb_vid->lock);
> + mutex_destroy(&kmb_vid->dma_lock);
> +}
>
> /**
> * kmb_video_register - Register V4L2 device
> @@ -35,7 +902,14 @@ void kmb_video_cleanup(struct kmb_video *kmb_vid)
> int kmb_video_register(struct kmb_video *kmb_vid,
> struct v4l2_device *v4l2_dev)
> {
> - return 0;
> + int ret;
> +
> + kmb_vid->video->v4l2_dev = v4l2_dev;
> + ret = video_register_device(kmb_vid->video, VFL_TYPE_VIDEO, -1);
> + if (ret < 0)
> + dev_err(&kmb_vid->video->dev, "Failed to register video device");
> +
> + return ret;
> }
>
> /**
> @@ -43,4 +917,6 @@ int kmb_video_register(struct kmb_video *kmb_vid,
> * @kmb_vid: pointer to kmb video device
> */
> void kmb_video_unregister(struct kmb_video *kmb_vid)
> -{ }
> +{
> + video_unregister_device(kmb_vid->video);
> +}
> diff --git a/drivers/media/platform/keembay-camera/keembay-video.h b/drivers/media/platform/keembay-camera/keembay-video.h
> index 2aebbb37424b..de25dfe3d684 100644
> --- a/drivers/media/platform/keembay-camera/keembay-video.h
> +++ b/drivers/media/platform/keembay-camera/keembay-video.h
> @@ -16,7 +16,7 @@
> /**
> * struct kmb_frame_buffer - KMB frame buffer structure
> * @vb: Video buffer for v4l2
> - * @addr: Array of dma buffer plane address
> + * @addr: Array of dma buffer plane addresses
> * @list: Frame buffer list
> */
> struct kmb_frame_buffer {
> @@ -28,50 +28,39 @@ struct kmb_frame_buffer {
> /**
> * struct kmb_video - KMB Video device structure
> * @lock: Mutex serializing kmb video device ops
> - * @video_lock: Mutex serializing video operations
> * @video: Pointer to V4L2 sub-device
> + * @vb2_q: Video buffer queue
> * @pad: Media pad graph objects
> * @dma_dev: Pointer to dma device
> + * @dma_queue: DMA buffers queue
> + * @dma_lock: Mutex serializing dma queue ops
> + * @active_fmt: Active format
> + * @active_fmt.pix: Mplane active pixel format
> + * @active_fmt.info: Active kmb format info
> * @pipe: Pointer to kmb media pipeline
> - * @chan: Pointer to xlink channel
> + * @xlink_cam: Pointer to xlink camera communication handler
> + * @chan_id: Channel ID
> + * @thread: Pointer to worker thread data
> */
> struct kmb_video {
> - struct mutex lock; /* Lock protecting kmb video device */
> - struct mutex video_lock; /* Lock serializing video device operations */
Ideally lines would be only added here, not removed.
Should this perhaps belong to an earlier patch?
> + struct mutex lock;
> struct video_device *video;
> + struct vb2_queue vb2_q;
> struct media_pad pad;
> +
> struct device *dma_dev;
> - struct kmb_pipeline *pipe;
> - struct kmb_xlink_cam *xlink_cam;
> - unsigned int chan_id;
> -};
> + struct list_head dma_queue;
> + struct mutex dma_lock;
>
> -/**
> - * struct kmb_video_fh - KMB video file handler
> - * @fh: V4L2 file handler
> - * @kmb_vid: Pointer to KMB video device
> - * @lock: Mutex serializing access to fh
> - * @vb2_lock: Mutex serializing access to vb2 queue
> - * @vb2_q: Video buffer queue
> - * @active_fmt: Active format
> - @pix: Mplane active pixel format
> - @info: Active kmb format info
> - * @contiguous_memory: Flag to enable contiguous memory allocation
> - * @dma_queue: DMA buffers queue
> - * @thread: Pointer to worker thread data
> - */
> -struct kmb_video_fh {
> - struct v4l2_fh fh;
> - struct kmb_video *kmb_vid;
> - struct mutex lock; /* Lock protecting fh operations */
> - struct mutex vb2_lock; /* Lock protecting video buffer queue */
> - struct vb2_queue vb2_q;
> struct {
> struct v4l2_pix_format_mplane pix;
> const struct kmb_video_fmt_info *info;
> } active_fmt;
> - bool contiguous_memory;
> - struct list_head dma_queue;
> +
> + struct kmb_pipeline *pipe;
> + struct kmb_xlink_cam *xlink_cam;
> + unsigned int chan_id;
> +
> struct task_struct *thread;
> };
>
--
Kind regards,
Sakari Ailus
^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 09/10] media: Keem Bay Camera: Add metadata video node
2021-03-19 18:06 [PATCH 00/10] Keem Bay Camera Subsystem Martina Krasteva
` (7 preceding siblings ...)
2021-03-19 18:06 ` [PATCH 08/10] media: Keem Bay Camera: Add capture video node Martina Krasteva
@ 2021-03-19 18:06 ` Martina Krasteva
2021-04-09 10:24 ` Sakari Ailus
2021-03-19 18:06 ` [PATCH 10/10] media: admin-guide: Add documentation for Keem Bay Camera Martina Krasteva
2021-04-16 9:37 ` [PATCH 00/10] Keem Bay Camera Subsystem Laurent Pinchart
10 siblings, 1 reply; 28+ messages in thread
From: Martina Krasteva @ 2021-03-19 18:06 UTC (permalink / raw)
To: linux-media
Cc: mchehab, robh+dt, devicetree, sakari.ailus,
daniele.alessandrelli, paul.j.murphy, gjorgjix.rosikopulos,
martinax.krasteva
From: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
Metadata video node implements output and capture meta type
interface.
- Output video node is used to provide isp parameters for processing.
Each buffer internally has real vpu isp params structure
allocated. User space params are copied on every qbuf based on
update flags. Since vpu need every time all parameters to be provided,
params are copied on every qbuf. Based on update flags they are copied
from userspace buffer or last buffer processed.
To reduce coping of the tables, they are allocated separately
in table buffer pool.
The tables are copied only when there is update from the userspace,
otherwise they are only reference from last processed frame.
This is possible because vpu interface has separate address for each table.
- Capture video node is used to provide statistics to userspace.
Capture video node statistics memory addresses are copied to isp
params before processing, and corresponding update flags are set
based on statistics availability.
Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
Co-developed-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
Signed-off-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
Co-developed-by: Martina Krasteva <martinax.krasteva@intel.com>
Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
---
drivers/media/platform/keembay-camera/Makefile | 4 +-
.../platform/keembay-camera/keembay-metadata.c | 1823 +++++++++++++++++++-
.../platform/keembay-camera/keembay-metadata.h | 14 +-
.../keembay-camera/keembay-params-defaults.c | 326 ++++
.../keembay-camera/keembay-params-defaults.h | 38 +
5 files changed, 2194 insertions(+), 11 deletions(-)
create mode 100644 drivers/media/platform/keembay-camera/keembay-params-defaults.c
create mode 100644 drivers/media/platform/keembay-camera/keembay-params-defaults.h
diff --git a/drivers/media/platform/keembay-camera/Makefile b/drivers/media/platform/keembay-camera/Makefile
index 8b3ad715c5c4..1b949cf009ef 100644
--- a/drivers/media/platform/keembay-camera/Makefile
+++ b/drivers/media/platform/keembay-camera/Makefile
@@ -1,5 +1,5 @@
keembay-cam-objs = keembay-camera.o keembay-pipeline.o \
- keembay-cam-xlink.o keembay-isp.o \
- keembay-metadata.o keembay-video.o
+ keembay-cam-xlink.o keembay-params-defaults.o \
+ keembay-isp.o keembay-metadata.o keembay-video.o
obj-$(CONFIG_VIDEO_INTEL_KEEMBAY_CAMERA) += keembay-cam.o
diff --git a/drivers/media/platform/keembay-camera/keembay-metadata.c b/drivers/media/platform/keembay-camera/keembay-metadata.c
index a1df746d9582..8807e3f322c5 100644
--- a/drivers/media/platform/keembay-camera/keembay-metadata.c
+++ b/drivers/media/platform/keembay-camera/keembay-metadata.c
@@ -4,17 +4,1818 @@
*
* Copyright (C) 2021 Intel Corporation
*/
+
+#include <linux/keembay-isp-ctl.h>
+#include <linux/dmapool.h>
+
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-dma-contig.h>
+#include <media/videobuf2-vmalloc.h>
+
+#include "keembay-pipeline.h"
#include "keembay-metadata.h"
+#define KMB_CAM_METADATA_STATS_NAME "keembay-metadata-stats"
+#define KMB_CAM_METADATA_PARAMS_NAME "keembay-metadata-params"
+
+#define KMB_TABLE_ALIGN 64
+
+/* Table names map */
+static const char *table_name[KMB_METADATA_TABLE_MAX] = {
+ "LSC",
+ "StaticDefect",
+ "LCA",
+ "HDR",
+ "Sharpness",
+ "Color cumb",
+ "LUT",
+ "TNF1",
+ "TNF2",
+ "Dehaze",
+ "Warp",
+};
+
+static void
+kmb_metadata_copy_blc(struct kmb_vpu_blc_params *dst,
+ struct kmb_blc_params *src)
+{
+ int i;
+
+ for (i = 0; i < KMB_CAM_MAX_EXPOSURES; i++) {
+ dst[i].coeff1 = src[i].coeff1;
+ dst[i].coeff2 = src[i].coeff2;
+ dst[i].coeff3 = src[i].coeff3;
+ dst[i].coeff4 = src[i].coeff4;
+ }
+}
+
+static void
+kmb_metadata_copy_sigma_dns(struct kmb_vpu_sigma_dns_params *dst,
+ struct kmb_sigma_dns_params *src)
+{
+ int i;
+
+ for (i = 0; i < KMB_CAM_MAX_EXPOSURES; i++) {
+ dst[i].noise = src[i].noise;
+ dst[i].threshold1 = src[i].threshold1;
+ dst[i].threshold2 = src[i].threshold2;
+ dst[i].threshold3 = src[i].threshold3;
+ dst[i].threshold4 = src[i].threshold4;
+ dst[i].threshold5 = src[i].threshold5;
+ dst[i].threshold6 = src[i].threshold6;
+ dst[i].threshold7 = src[i].threshold7;
+ dst[i].threshold8 = src[i].threshold8;
+ }
+}
+
+static void
+kmb_metadata_copy_lsc(struct kmb_vpu_lsc_params *dst,
+ struct kmb_lsc_params *src)
+{
+ dst->threshold = src->threshold;
+ dst->width = src->width;
+ dst->height = src->height;
+}
+
+static void
+kmb_metadata_copy_raw(struct kmb_vpu_raw_params *dst,
+ struct kmb_raw_params *src)
+{
+ dst->awb_stats_en = src->awb_stats_en;
+ dst->awb_rgb_hist_en = src->awb_rgb_hist_en;
+ dst->af_stats_en = src->af_stats_en;
+ dst->luma_hist_en = src->luma_hist_en;
+ dst->flicker_accum_en = src->flicker_accum_en;
+ dst->bad_pixel_fix_en = src->bad_pixel_fix_en;
+ dst->grgb_imb_en = src->grgb_imb_en;
+ dst->mono_imbalance_en = src->mono_imbalance_en;
+ dst->gain1 = src->gain1;
+ dst->gain2 = src->gain2;
+ dst->gain3 = src->gain3;
+ dst->gain4 = src->gain4;
+ dst->stop1 = src->stop1;
+ dst->stop2 = src->stop2;
+ dst->stop3 = src->stop3;
+ dst->stop4 = src->stop4;
+ dst->threshold1 = src->threshold1;
+ dst->alpha1 = src->alpha1;
+ dst->alpha2 = src->alpha2;
+ dst->alpha3 = src->alpha3;
+ dst->alpha4 = src->alpha4;
+ dst->threshold2 = src->threshold2;
+ dst->static_defect_size = src->static_defect_size;
+ dst->flicker_first_row_acc = src->start_row;
+ dst->flicker_last_row_acc = src->end_row;
+}
+
+static void
+kmb_metadata_copy_ae_awb(struct kmb_vpu_ae_awb_params *dst,
+ struct kmb_ae_awb_params *src)
+{
+ dst->start_x = src->start_x;
+ dst->start_y = src->start_y;
+ dst->width = src->width;
+ dst->height = src->height;
+ dst->skip_x = src->skip_x;
+ dst->skip_y = src->skip_y;
+ dst->patches_x = src->patches_x;
+ dst->patches_y = src->patches_y;
+ dst->threshold1 = src->threshold1;
+ dst->threshold2 = src->threshold2;
+}
+
+static void
+kmb_metadata_copy_af(struct kmb_vpu_af_params *dst,
+ struct kmb_af_params *src)
+{
+ int i;
+
+ dst->start_x = src->start_x;
+ dst->start_y = src->start_y;
+ dst->width = src->width;
+ dst->height = src->height;
+ dst->patches_x = src->patches_x;
+ dst->patches_y = src->patches_y;
+ dst->coeff = src->coeff;
+ dst->threshold1 = src->threshold1;
+ dst->threshold2 = src->threshold2;
+
+ for (i = 0; i < ARRAY_SIZE(dst->coeffs1); i++) {
+ dst->coeffs1[i] = src->coeffs1[i];
+ dst->coeffs2[i] = src->coeffs2[i];
+ }
+}
+
+static void
+kmb_metadata_copy_histogram(struct kmb_vpu_hist_params *dst,
+ struct kmb_hist_params *src)
+{
+ int i;
+
+ dst->start_x = src->start_x;
+ dst->start_y = src->start_y;
+ dst->end_x = src->end_x;
+ dst->end_y = src->end_y;
+
+ for (i = 0; i < ARRAY_SIZE(dst->matrix); i++)
+ dst->matrix[i] = src->matrix[i];
+
+ for (i = 0; i < ARRAY_SIZE(dst->weight); i++)
+ dst->weight[i] = src->weight[i];
+}
+
+static void
+kmb_metadata_copy_debayer(struct kmb_vpu_debayer_params *dst,
+ struct kmb_debayer_params *src)
+{
+ dst->coeff1 = src->coeff1;
+ dst->multiplier1 = src->multiplier1;
+ dst->multiplier2 = src->multiplier2;
+ dst->coeff2 = src->coeff2;
+ dst->coeff3 = src->coeff3;
+ dst->coeff4 = src->coeff4;
+}
+
+static void
+kmb_metadata_copy_dog_dns(struct kmb_vpu_dog_dns_params *dst,
+ struct kmb_dog_dns_params *src)
+{
+ int i;
+
+ dst->threshold = src->threshold;
+ dst->strength = src->strength;
+
+ for (i = 0; i < ARRAY_SIZE(dst->coeffs11); i++)
+ dst->coeffs11[i] = src->coeffs11[i];
+
+ for (i = 0; i < ARRAY_SIZE(dst->coeffs15); i++)
+ dst->coeffs15[i] = src->coeffs15[i];
+}
+
+static void
+kmb_metadata_copy_luma_dns(struct kmb_vpu_luma_dns_params *dst,
+ struct kmb_luma_dns_params *src)
+{
+ dst->threshold = src->threshold;
+ dst->slope = src->slope;
+ dst->shift = src->shift;
+ dst->alpha = src->alpha;
+ dst->weight = src->weight;
+ dst->per_pixel_alpha_en = src->per_pixel_alpha_en;
+ dst->gain_bypass_en = src->gain_bypass_en;
+}
+
+static void
+kmb_metadata_copy_sharpen(struct kmb_vpu_sharpen_params *dst,
+ struct kmb_sharpen_params *src)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(dst->coeffs1); i++) {
+ dst->coeffs1[i] = src->coeffs1[i];
+ dst->coeffs2[i] = src->coeffs2[i];
+ dst->coeffs3[i] = src->coeffs3[i];
+ }
+
+ dst->shift = src->shift;
+ dst->gain1 = src->gain1;
+ dst->gain2 = src->gain2;
+ dst->gain3 = src->gain3;
+ dst->gain4 = src->gain4;
+ dst->gain5 = src->gain5;
+
+ for (i = 0; i < ARRAY_SIZE(dst->stops1); i++) {
+ dst->stops1[i] = src->stops1[i];
+ dst->gains[i] = src->gains[i];
+ }
+
+ for (i = 0; i < ARRAY_SIZE(dst->stops2); i++)
+ dst->stops2[i] = src->stops2[i];
+
+ dst->overshoot = src->overshoot;
+ dst->undershoot = src->undershoot;
+ dst->alpha = src->alpha;
+ dst->gain6 = src->gain6;
+ dst->offset = src->offset;
+}
+
+static void
+kmb_metadata_copy_chroma_gen(struct kmb_vpu_chroma_gen_params *dst,
+ struct kmb_chroma_gen_params *src)
+{
+ int i;
+
+ dst->epsilon = src->epsilon;
+ dst->coeff1 = src->coeff1;
+ dst->coeff2 = src->coeff2;
+ dst->coeff3 = src->coeff3;
+ dst->coeff4 = src->coeff4;
+ dst->coeff5 = src->coeff5;
+ dst->coeff6 = src->coeff6;
+ dst->strength1 = src->strength1;
+ dst->strength2 = src->strength2;
+
+ for (i = 0; i < ARRAY_SIZE(dst->coeffs); i++)
+ dst->coeffs[i] = src->coeffs[i];
+
+ dst->offset1 = src->offset1;
+ dst->slope1 = src->slope1;
+ dst->slope2 = src->slope2;
+ dst->offset2 = src->offset2;
+ dst->limit = src->limit;
+}
+
+static void
+kmb_metadata_copy_median(struct kmb_vpu_median_params *dst,
+ struct kmb_median_params *src)
+{
+ dst->size = src->size;
+ dst->slope = src->slope;
+ dst->offset = src->offset;
+}
+
+static void
+kmb_metadata_copy_chroma_dns(struct kmb_vpu_chroma_dns_params *dst,
+ struct kmb_chroma_dns_params *src)
+{
+ dst->limit = src->limit;
+ dst->enable = src->enable;
+ dst->threshold1 = src->threshold1;
+ dst->threshold2 = src->threshold2;
+ dst->threshold3 = src->threshold3;
+ dst->threshold4 = src->threshold4;
+ dst->threshold5 = src->threshold5;
+ dst->threshold6 = src->threshold6;
+ dst->threshold7 = src->threshold7;
+ dst->threshold8 = src->threshold8;
+ dst->slope1 = src->slope1;
+ dst->offset1 = src->offset1;
+ dst->slope2 = src->slope2;
+ dst->offset2 = src->offset2;
+ dst->grey1 = src->grey1;
+ dst->grey2 = src->grey2;
+ dst->grey3 = src->grey3;
+ dst->coeff1 = src->coeff1;
+ dst->coeff2 = src->coeff2;
+ dst->coeff3 = src->coeff3;
+}
+
+static void
+kmb_metadata_copy_color_comb(struct kmb_vpu_color_comb_params *dst,
+ struct kmb_color_comb_params *src)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(dst->matrix); i++)
+ dst->matrix[i] = src->matrix[i];
+
+ for (i = 0; i < ARRAY_SIZE(dst->offsets); i++)
+ dst->offsets[i] = src->offsets[i];
+
+ dst->coeff1 = src->coeff1;
+ dst->coeff2 = src->coeff2;
+ dst->coeff3 = src->coeff3;
+ dst->enable = src->enable;
+ dst->weight1 = src->weight1;
+ dst->weight2 = src->weight2;
+ dst->weight3 = src->weight3;
+ dst->limit1 = src->limit1;
+ dst->limit2 = src->limit2;
+ dst->offset1 = src->offset1;
+ dst->offset2 = src->offset2;
+}
+
+static void
+kmb_metadata_copy_hdr(struct kmb_vpu_hdr_params *dst,
+ struct kmb_hdr_params *src)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(dst->ratio); i++)
+ dst->ratio[i] = src->ratio[i];
+
+ for (i = 0; i < ARRAY_SIZE(dst->scale); i++)
+ dst->scale[i] = src->scale[i];
+
+ dst->offset1 = src->offset1;
+ dst->slope1 = src->slope1;
+ dst->offset2 = src->offset2;
+ dst->slope2 = src->slope2;
+ dst->offset3 = src->offset3;
+ dst->slope3 = src->slope3;
+ dst->offset4 = src->offset4;
+ dst->gain1 = src->gain1;
+
+ for (i = 0; i < ARRAY_SIZE(dst->blur1); i++)
+ dst->blur1[i] = src->blur1[i];
+
+ for (i = 0; i < ARRAY_SIZE(dst->blur2); i++)
+ dst->blur2[i] = src->blur2[i];
+
+ dst->contrast1 = src->contrast1;
+ dst->contrast2 = src->contrast2;
+ dst->enable1 = src->enable1;
+ dst->enable2 = src->enable2;
+ dst->offset5 = src->offset5;
+ dst->gain2 = src->gain2;
+ dst->offset6 = src->offset6;
+ dst->strength = src->strength;
+ dst->offset7 = src->offset7;
+ dst->shift = src->shift;
+ dst->field1 = src->field1;
+ dst->field2 = src->field2;
+ dst->gain3 = src->gain3;
+ dst->min = src->min;
+}
+
+static void
+kmb_metadata_copy_lut(struct kmb_vpu_lut_params *dst,
+ struct kmb_lut_params *src)
+{
+ int i;
+
+ dst->size = src->size;
+ for (i = 0; i < ARRAY_SIZE(dst->matrix); i++)
+ dst->matrix[i] = src->matrix[i];
+
+ for (i = 0; i < ARRAY_SIZE(dst->offsets); i++)
+ dst->offsets[i] = src->offsets[i];
+}
+
+static void
+kmb_metadata_copy_tnf(struct kmb_vpu_tnf_params *dst,
+ struct kmb_tnf_params *src)
+{
+ dst->factor = src->factor;
+ dst->gain = src->gain;
+ dst->offset1 = src->offset1;
+ dst->slope1 = src->slope1;
+ dst->offset2 = src->offset2;
+ dst->slope2 = src->slope2;
+ dst->min1 = src->min1;
+ dst->min2 = src->min2;
+ dst->value = src->value;
+ dst->enable = src->enable;
+}
+
+static void
+kmb_metadata_copy_dehaze(struct kmb_vpu_dehaze_params *dst,
+ struct kmb_dehaze_params *src)
+{
+ int i;
+
+ dst->gain1 = src->gain1;
+ dst->min = src->min;
+ dst->strength1 = src->strength1;
+ dst->strength2 = src->strength2;
+ dst->gain2 = src->gain2;
+ dst->saturation = src->saturation;
+ dst->value1 = src->value1;
+ dst->value2 = src->value2;
+ dst->value3 = src->value3;
+
+ for (i = 0; i < ARRAY_SIZE(dst->filter); i++)
+ dst->filter[i] = src->filter[i];
+}
+
+static void
+kmb_metadata_copy_warp(struct kmb_vpu_warp_params *dst,
+ struct kmb_warp_params *src)
+{
+ int i;
+
+ dst->type = src->type;
+ dst->relative = src->relative;
+ dst->format = src->format;
+ dst->position = src->position;
+ dst->width = src->width;
+ dst->height = src->height;
+ dst->stride = src->stride;
+ dst->enable = src->enable;
+
+ for (i = 0; i < ARRAY_SIZE(dst->matrix); i++)
+ dst->matrix[i] = src->matrix[i];
+
+ dst->mode = src->mode;
+
+ for (i = 0; i < ARRAY_SIZE(dst->values); i++)
+ dst->values[i] = src->values[i];
+}
+
+/* VPU Params tables */
+static struct kmb_metadata_table *
+kmb_metadata_cpalloc_table(struct kmb_metadata *kmb_meta,
+ enum kmb_metadata_table_type type,
+ size_t src_table_size)
+{
+ struct kmb_metadata_table *table;
+
+ lockdep_assert_held(&kmb_meta->lock);
+
+ /* First create pool if needed */
+ if (!kmb_meta->table_pool[type]) {
+ kmb_meta->table_pool[type] =
+ dma_pool_create(table_name[type],
+ kmb_meta->dma_dev,
+ src_table_size + sizeof(*table),
+ KMB_TABLE_ALIGN, 0);
+ if (!kmb_meta->table_pool[type]) {
+ dev_err(kmb_meta->dma_dev,
+ "Fail to create %s pool", table_name[type]);
+ return NULL;
+ }
+ }
+
+ table = kmalloc(sizeof(*table), GFP_KERNEL);
+ if (!table)
+ return NULL;
+
+ kref_init(&table->refcount);
+ table->pool = kmb_meta->table_pool[type];
+
+ table->cpu_addr = dma_pool_alloc(kmb_meta->table_pool[type],
+ GFP_KERNEL,
+ &table->dma_addr);
+ if (!table->cpu_addr) {
+ kfree(table);
+ return NULL;
+ }
+
+ return table;
+}
+
+static void kmb_metadata_free_table(struct kref *ref)
+{
+ struct kmb_metadata_table *table =
+ container_of(ref, struct kmb_metadata_table, refcount);
+
+ dma_pool_free(table->pool, table->cpu_addr, table->dma_addr);
+ kfree(table);
+}
+
+static void
+kmb_metadata_release_tables(struct kmb_metadata_buf *meta_buf)
+{
+ int i;
+
+ for (i = 0; i < KMB_METADATA_TABLE_MAX; i++) {
+ if (meta_buf->params.tab[i]) {
+ kref_put(&meta_buf->params.tab[i]->refcount,
+ kmb_metadata_free_table);
+ meta_buf->params.tab[i] = NULL;
+ }
+ }
+}
+
+static void
+kmb_metadata_destroy_table_pools(struct kmb_metadata *kmb_meta)
+{
+ int i;
+
+ /* Release allocated pools during streaming */
+ for (i = 0; i < KMB_METADATA_TABLE_MAX; i++) {
+ dma_pool_destroy(kmb_meta->table_pool[i]);
+ kmb_meta->table_pool[i] = NULL;
+ }
+}
+
+static dma_addr_t
+kmb_metadata_get_table_addr(struct kmb_metadata_buf *meta_buf,
+ enum kmb_metadata_table_type type)
+{
+ struct kmb_metadata_table *table = meta_buf->params.tab[type];
+
+ if (!table)
+ return 0;
+
+ return table->dma_addr;
+}
+
+static struct kmb_metadata_table *
+kmb_metadata_create_table(struct kmb_metadata *kmb_meta,
+ struct kmb_metadata_buf *meta_buf,
+ enum kmb_metadata_table_type type,
+ size_t user_table_size)
+{
+ struct kmb_metadata_table *table;
+
+ lockdep_assert_held(&kmb_meta->lock);
+
+ table = kmb_metadata_cpalloc_table(kmb_meta,
+ type,
+ user_table_size);
+ if (!table)
+ return NULL;
+
+ if (meta_buf->params.tab[type])
+ kref_put(&meta_buf->params.tab[type]->refcount,
+ kmb_metadata_free_table);
+
+ meta_buf->params.tab[type] = table;
+
+ return table;
+}
+
+static int
+kmb_metadata_copy_table_usr(struct kmb_metadata *kmb_meta,
+ struct kmb_metadata_buf *meta_buf,
+ enum kmb_metadata_table_type type,
+ u8 *user_table, size_t user_table_size)
+{
+ struct kmb_metadata_table *table;
+
+ table = kmb_metadata_create_table(kmb_meta, meta_buf,
+ type, user_table_size);
+ if (!table)
+ return -ENOMEM;
+
+ memcpy(table->cpu_addr, user_table, user_table_size);
+
+ return 0;
+}
+
+static int kmb_metadata_create_default_table(struct kmb_metadata *kmb_meta,
+ struct kmb_metadata_buf *meta_buf,
+ enum kmb_metadata_table_type type,
+ u8 *user_table,
+ size_t user_table_size)
+{
+ struct kmb_metadata_table *table;
+
+ table = kmb_metadata_create_table(kmb_meta, meta_buf,
+ type, user_table_size);
+ if (!table)
+ return -ENOMEM;
+
+ memset(table->cpu_addr, 0, user_table_size);
+
+ return 0;
+}
+
+static void
+kmb_metadata_copy_table_vpu(struct kmb_metadata_buf *meta_buf,
+ struct kmb_metadata_buf *last_meta_buf,
+ enum kmb_metadata_table_type type)
+{
+ /* Do nothing if params are the same */
+ if (WARN_ON(meta_buf->params.isp == last_meta_buf->params.isp))
+ return;
+
+ meta_buf->params.tab[type] = last_meta_buf->params.tab[type];
+ if (meta_buf->params.tab[type])
+ kref_get(&meta_buf->params.tab[type]->refcount);
+}
+
+static void
+kmb_metadata_fill_blc(struct kmb_vpu_isp_params *params,
+ struct kmb_isp_params *user_params,
+ struct kmb_vpu_isp_params *last_params,
+ struct kmb_vpu_isp_params_defaults *def_params)
+{
+ if (user_params->update.blc) {
+ kmb_metadata_copy_blc(params->blc, user_params->blc);
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(params->blc, last_params->blc,
+ sizeof(params->blc));
+ } else {
+ memcpy(params->blc, def_params->blc, sizeof(params->blc));
+ }
+}
+
+static void
+kmb_metadata_fill_signma_dns(struct kmb_vpu_isp_params *params,
+ struct kmb_isp_params *user_params,
+ struct kmb_vpu_isp_params *last_params,
+ struct kmb_vpu_isp_params_defaults *def_params)
+{
+ if (user_params->update.sigma_dns) {
+ kmb_metadata_copy_sigma_dns(params->sigma_dns,
+ user_params->sigma_dns);
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(params->sigma_dns, last_params->sigma_dns,
+ sizeof(params->sigma_dns));
+ } else {
+ memcpy(params->sigma_dns, def_params->sigma_dns,
+ sizeof(params->sigma_dns));
+ }
+}
+
+static void
+kmb_metadata_fill_ae_awb(struct kmb_vpu_isp_params *params,
+ struct kmb_isp_params *user_params,
+ struct kmb_vpu_isp_params *last_params,
+ struct kmb_vpu_isp_params_defaults *def_params)
+{
+ if (user_params->update.ae_awb) {
+ kmb_metadata_copy_ae_awb(¶ms->ae_awb,
+ &user_params->ae_awb);
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(¶ms->ae_awb, &last_params->ae_awb,
+ sizeof(params->ae_awb));
+ } else {
+ memcpy(¶ms->ae_awb, def_params->ae_awb,
+ sizeof(params->ae_awb));
+ }
+}
+
+static void
+kmb_metadata_fill_af(struct kmb_vpu_isp_params *params,
+ struct kmb_isp_params *user_params,
+ struct kmb_vpu_isp_params *last_params,
+ struct kmb_vpu_isp_params_defaults *def_params)
+{
+ if (user_params->update.af) {
+ kmb_metadata_copy_af(¶ms->af, &user_params->af);
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(¶ms->af, &last_params->af,
+ sizeof(params->af));
+ } else {
+ memcpy(¶ms->af, def_params->af, sizeof(params->af));
+ }
+}
+
+static void
+kmb_metadata_fill_histogram(struct kmb_vpu_isp_params *params,
+ struct kmb_isp_params *user_params,
+ struct kmb_vpu_isp_params *last_params,
+ struct kmb_vpu_isp_params_defaults *def_params)
+{
+ if (user_params->update.histogram) {
+ kmb_metadata_copy_histogram(¶ms->histogram,
+ &user_params->histogram);
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(¶ms->histogram, &last_params->histogram,
+ sizeof(params->histogram));
+ } else {
+ memcpy(¶ms->histogram, def_params->histogram,
+ sizeof(params->histogram));
+ }
+}
+
+static void
+kmb_metadata_fill_debayer(struct kmb_vpu_isp_params *params,
+ struct kmb_isp_params *user_params,
+ struct kmb_vpu_isp_params *last_params,
+ struct kmb_vpu_isp_params_defaults *def_params)
+{
+ if (user_params->update.debayer) {
+ kmb_metadata_copy_debayer(¶ms->debayer,
+ &user_params->debayer);
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(¶ms->debayer, &last_params->debayer,
+ sizeof(params->debayer));
+ } else {
+ memcpy(¶ms->debayer, def_params->debayer,
+ sizeof(params->debayer));
+ }
+}
+
+static void
+kmb_metadata_fill_dog_dns(struct kmb_vpu_isp_params *params,
+ struct kmb_isp_params *user_params,
+ struct kmb_vpu_isp_params *last_params,
+ struct kmb_vpu_isp_params_defaults *def_params)
+{
+ if (user_params->update.dog_dns) {
+ kmb_metadata_copy_dog_dns(¶ms->dog_dns,
+ &user_params->dog_dns);
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(¶ms->dog_dns, &last_params->dog_dns,
+ sizeof(params->dog_dns));
+ } else {
+ memcpy(¶ms->dog_dns, def_params->dog_dns,
+ sizeof(params->dog_dns));
+ }
+}
+
+static void
+kmb_metadata_fill_luma_dns(struct kmb_vpu_isp_params *params,
+ struct kmb_isp_params *user_params,
+ struct kmb_vpu_isp_params *last_params,
+ struct kmb_vpu_isp_params_defaults *def_params)
+{
+ if (user_params->update.luma_dns) {
+ kmb_metadata_copy_luma_dns(¶ms->luma_dns,
+ &user_params->luma_dns);
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(¶ms->luma_dns, &last_params->luma_dns,
+ sizeof(params->luma_dns));
+ } else {
+ memcpy(¶ms->luma_dns, def_params->luma_dns,
+ sizeof(params->luma_dns));
+ }
+}
+
+static void
+kmb_metadata_fill_chroma_gen(struct kmb_vpu_isp_params *params,
+ struct kmb_isp_params *user_params,
+ struct kmb_vpu_isp_params *last_params,
+ struct kmb_vpu_isp_params_defaults *def_params)
+{
+ if (user_params->update.chroma_gen) {
+ kmb_metadata_copy_chroma_gen(¶ms->chroma_gen,
+ &user_params->chroma_gen);
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(¶ms->chroma_gen, &last_params->chroma_gen,
+ sizeof(params->chroma_gen));
+ } else {
+ memcpy(¶ms->chroma_gen, def_params->chroma_gen,
+ sizeof(params->chroma_gen));
+ }
+}
+
+static void
+kmb_metadata_fill_median(struct kmb_vpu_isp_params *params,
+ struct kmb_isp_params *user_params,
+ struct kmb_vpu_isp_params *last_params,
+ struct kmb_vpu_isp_params_defaults *def_params)
+{
+ if (user_params->update.median) {
+ kmb_metadata_copy_median(¶ms->median,
+ &user_params->median);
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(¶ms->median, &last_params->median,
+ sizeof(params->median));
+ } else {
+ memcpy(¶ms->median, def_params->median,
+ sizeof(params->median));
+ }
+}
+
+static void
+kmb_metadata_fill_chroma_dns(struct kmb_vpu_isp_params *params,
+ struct kmb_isp_params *user_params,
+ struct kmb_vpu_isp_params *last_params,
+ struct kmb_vpu_isp_params_defaults *def_params)
+{
+ if (user_params->update.chroma_dns) {
+ kmb_metadata_copy_chroma_dns(¶ms->chroma_dns,
+ &user_params->chroma_dns);
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(¶ms->chroma_dns, &last_params->chroma_dns,
+ sizeof(params->chroma_dns));
+ } else {
+ memcpy(¶ms->chroma_dns, def_params->chroma_dns,
+ sizeof(params->chroma_dns));
+ }
+}
+
+static void
+kmb_metadata_fill_dehaze(struct kmb_vpu_isp_params *params,
+ struct kmb_isp_params *user_params,
+ struct kmb_vpu_isp_params *last_params,
+ struct kmb_vpu_isp_params_defaults *def_params)
+{
+ if (user_params->update.dehaze) {
+ kmb_metadata_copy_dehaze(¶ms->dehaze,
+ &user_params->dehaze);
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(¶ms->dehaze, &last_params->dehaze,
+ sizeof(params->dehaze));
+ } else {
+ memcpy(¶ms->dehaze, def_params->dehaze,
+ sizeof(params->dehaze));
+ }
+}
+
+static int
+kmb_metadata_fill_lsc(struct kmb_metadata *kmb_meta,
+ struct kmb_metadata_buf *meta_buf,
+ struct kmb_isp_params *user_params)
+{
+ struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
+ struct kmb_vpu_isp_params *params = meta_buf->params.isp;
+ struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
+ struct kmb_vpu_isp_params *last_params = NULL;
+ int ret = 0;
+
+ if (last_buf)
+ last_params = last_buf->params.isp;
+
+ if (user_params->update.lsc) {
+ kmb_metadata_copy_lsc(¶ms->lsc,
+ &user_params->lsc);
+ if (params->lsc.width && params->lsc.height) {
+ ret = kmb_metadata_copy_table_usr(kmb_meta,
+ meta_buf,
+ KMB_METADATA_TABLE_LSC,
+ user_params->lsc.gain_mesh,
+ params->lsc.width *
+ params->lsc.height);
+ if (ret < 0)
+ return ret;
+ }
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(¶ms->lsc, &last_params->lsc,
+ sizeof(params->lsc));
+
+ if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
+ kmb_metadata_copy_table_vpu(meta_buf, last_buf,
+ KMB_METADATA_TABLE_LSC);
+ } else {
+ memcpy(¶ms->lsc, def_params->lsc, sizeof(params->lsc));
+ kmb_metadata_create_default_table(kmb_meta,
+ meta_buf,
+ KMB_METADATA_TABLE_LSC,
+ user_params->lsc.gain_mesh,
+ ARRAY_SIZE(user_params->lsc.gain_mesh));
+ }
+
+ if (params->lsc.width && params->lsc.height) {
+ params->lsc.addr =
+ kmb_metadata_get_table_addr(meta_buf,
+ KMB_METADATA_TABLE_LSC);
+ if (!params->lsc.addr)
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
+static int
+kmb_metadata_fill_raw(struct kmb_metadata *kmb_meta,
+ struct kmb_metadata_buf *meta_buf,
+ struct kmb_isp_params *user_params)
+{
+ struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
+ struct kmb_vpu_isp_params *params = meta_buf->params.isp;
+ struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
+ struct kmb_vpu_isp_params *last_params = NULL;
+ int ret = 0;
+
+ if (last_buf)
+ last_params = last_buf->params.isp;
+
+ if (user_params->update.raw) {
+ kmb_metadata_copy_raw(¶ms->raw,
+ &user_params->raw);
+ if (params->raw.static_defect_size) {
+ ret = kmb_metadata_copy_table_usr(kmb_meta,
+ meta_buf,
+ KMB_METADATA_TABLE_SDEFECT,
+ user_params->raw.static_defect_map,
+ params->raw.static_defect_size);
+ if (ret < 0)
+ return ret;
+ }
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(¶ms->raw, &last_params->raw,
+ sizeof(params->raw));
+
+ if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
+ kmb_metadata_copy_table_vpu(meta_buf, last_buf,
+ KMB_METADATA_TABLE_SDEFECT);
+ } else {
+ memcpy(¶ms->raw, def_params->raw, sizeof(params->raw));
+ kmb_metadata_create_default_table(kmb_meta,
+ meta_buf,
+ KMB_METADATA_TABLE_SDEFECT,
+ user_params->raw.static_defect_map,
+ ARRAY_SIZE(user_params->raw.static_defect_map));
+ }
+
+ if (params->raw.static_defect_size) {
+ params->raw.static_defect_addr =
+ kmb_metadata_get_table_addr(meta_buf,
+ KMB_METADATA_TABLE_SDEFECT);
+ if (!params->raw.static_defect_addr)
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
+static int
+kmb_metadata_fill_lca(struct kmb_metadata *kmb_meta,
+ struct kmb_metadata_buf *meta_buf,
+ struct kmb_isp_params *user_params)
+{
+ struct kmb_vpu_isp_params *params = meta_buf->params.isp;
+ struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
+ struct kmb_vpu_isp_params *last_params = NULL;
+ int ret = 0;
+
+ if (last_buf)
+ last_params = last_buf->params.isp;
+
+ if (user_params->update.lca) {
+ ret = kmb_metadata_copy_table_usr(kmb_meta,
+ meta_buf,
+ KMB_METADATA_TABLE_LCA,
+ user_params->lca.coeff,
+ ARRAY_SIZE(user_params->lca.coeff));
+ if (ret < 0)
+ return ret;
+ } else if (last_params) {
+ if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
+ kmb_metadata_copy_table_vpu(meta_buf, last_buf,
+ KMB_METADATA_TABLE_LCA);
+ } else {
+ kmb_metadata_create_default_table(kmb_meta,
+ meta_buf,
+ KMB_METADATA_TABLE_LCA,
+ user_params->lca.coeff,
+ ARRAY_SIZE(user_params->lca.coeff));
+ }
+
+ params->lca.addr = kmb_metadata_get_table_addr(meta_buf,
+ KMB_METADATA_TABLE_LCA);
+ if (!params->lca.addr)
+ ret = -EINVAL;
+
+ return ret;
+}
+
+static int
+kmb_metadata_fill_sharpen(struct kmb_metadata *kmb_meta,
+ struct kmb_metadata_buf *meta_buf,
+ struct kmb_isp_params *user_params)
+{
+ struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
+ struct kmb_vpu_isp_params *params = meta_buf->params.isp;
+ struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
+ struct kmb_vpu_isp_params *last_params = NULL;
+ int ret = 0;
+
+ if (last_buf)
+ last_params = last_buf->params.isp;
+
+ if (user_params->update.sharpen) {
+ kmb_metadata_copy_sharpen(¶ms->sharpen,
+ &user_params->sharpen);
+ ret = kmb_metadata_copy_table_usr(kmb_meta,
+ meta_buf,
+ KMB_METADATA_TABLE_SHARP,
+ user_params->sharpen.radial_lut,
+ ARRAY_SIZE(user_params->sharpen.radial_lut));
+ if (ret < 0)
+ return ret;
+
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(¶ms->sharpen, &last_params->sharpen,
+ sizeof(params->sharpen));
+
+ if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
+ kmb_metadata_copy_table_vpu(meta_buf, last_buf,
+ KMB_METADATA_TABLE_SHARP);
+ } else {
+ memcpy(¶ms->sharpen, def_params->sharpen,
+ sizeof(params->sharpen));
+
+ kmb_metadata_create_default_table(kmb_meta,
+ meta_buf,
+ KMB_METADATA_TABLE_SHARP,
+ user_params->sharpen.radial_lut,
+ ARRAY_SIZE(user_params->sharpen.radial_lut));
+ }
+
+ params->sharpen.addr =
+ kmb_metadata_get_table_addr(meta_buf,
+ KMB_METADATA_TABLE_SHARP);
+ if (!params->sharpen.addr)
+ ret = -EINVAL;
+
+ return ret;
+}
+
+static int
+kmb_metadata_fill_color_comb(struct kmb_metadata *kmb_meta,
+ struct kmb_metadata_buf *meta_buf,
+ struct kmb_isp_params *user_params)
+{
+ struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
+ struct kmb_vpu_isp_params *params = meta_buf->params.isp;
+ struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
+ struct kmb_vpu_isp_params *last_params = NULL;
+ struct kmb_color_comb_params *col = NULL;
+ int ret = 0;
+
+ if (last_buf)
+ last_params = last_buf->params.isp;
+
+ if (user_params->update.color_comb) {
+ col = &user_params->color_comb;
+ kmb_metadata_copy_color_comb(¶ms->color_comb,
+ &user_params->color_comb);
+ if (params->color_comb.enable) {
+ ret = kmb_metadata_copy_table_usr(kmb_meta,
+ meta_buf,
+ KMB_METADATA_TABLE_COLOR_CUMB,
+ col->lut_3d,
+ ARRAY_SIZE(col->lut_3d));
+ if (ret < 0)
+ return ret;
+ }
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(¶ms->color_comb, &last_params->color_comb,
+ sizeof(params->color_comb));
+
+ if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
+ kmb_metadata_copy_table_vpu(meta_buf, last_buf,
+ KMB_METADATA_TABLE_COLOR_CUMB);
+ } else {
+ memcpy(¶ms->color_comb, def_params->color_comb,
+ sizeof(params->color_comb));
+ }
+
+ if (params->color_comb.enable) {
+ params->color_comb.addr =
+ kmb_metadata_get_table_addr(meta_buf,
+ KMB_METADATA_TABLE_COLOR_CUMB);
+ if (!params->color_comb.addr)
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
+static int
+kmb_metadata_fill_hdr(struct kmb_metadata *kmb_meta,
+ struct kmb_metadata_buf *meta_buf,
+ struct kmb_isp_params *user_params)
+{
+ struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
+ struct kmb_vpu_isp_params *params = meta_buf->params.isp;
+ struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
+ struct kmb_vpu_isp_params *last_params = NULL;
+ int ret = 0;
+
+ if (last_buf)
+ last_params = last_buf->params.isp;
+
+ if (user_params->update.hdr) {
+ kmb_metadata_copy_hdr(¶ms->hdr,
+ &user_params->hdr);
+ if (params->hdr.enable1 || params->hdr.enable2) {
+ ret = kmb_metadata_copy_table_usr(kmb_meta,
+ meta_buf,
+ KMB_METADATA_TABLE_HDR,
+ user_params->hdr.tm_lut,
+ ARRAY_SIZE(user_params->hdr.tm_lut));
+ if (ret < 0)
+ return ret;
+ }
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(¶ms->hdr, &last_params->hdr,
+ sizeof(params->hdr));
+
+ if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
+ kmb_metadata_copy_table_vpu(meta_buf, last_buf,
+ KMB_METADATA_TABLE_HDR);
+ } else {
+ memcpy(¶ms->hdr, def_params->hdr, sizeof(params->hdr));
+ }
+
+ if (params->hdr.enable1 || params->hdr.enable2) {
+ params->hdr.luts_addr =
+ kmb_metadata_get_table_addr(meta_buf,
+ KMB_METADATA_TABLE_HDR);
+ if (!params->hdr.luts_addr)
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
+static int
+kmb_metadata_fill_lut(struct kmb_metadata *kmb_meta,
+ struct kmb_metadata_buf *meta_buf,
+ struct kmb_isp_params *user_params)
+{
+ struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
+ struct kmb_vpu_isp_params *params = meta_buf->params.isp;
+ struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
+ struct kmb_vpu_isp_params *last_params = NULL;
+ int ret = 0;
+
+ if (last_buf)
+ last_params = last_buf->params.isp;
+
+ if (user_params->update.lut) {
+ kmb_metadata_copy_lut(¶ms->lut, &user_params->lut);
+ if (params->lut.size) {
+ ret = kmb_metadata_copy_table_usr(kmb_meta,
+ meta_buf,
+ KMB_METADATA_TABLE_LUT,
+ user_params->lut.table,
+ ARRAY_SIZE(user_params->lut.table));
+ if (ret < 0)
+ return ret;
+ }
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(¶ms->lut, &last_params->lut,
+ sizeof(params->lut));
+
+ if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
+ kmb_metadata_copy_table_vpu(meta_buf, last_buf,
+ KMB_METADATA_TABLE_LUT);
+ } else {
+ memcpy(¶ms->lut, def_params->lut, sizeof(params->lut));
+ kmb_metadata_create_default_table(kmb_meta,
+ meta_buf,
+ KMB_METADATA_TABLE_LUT,
+ user_params->lut.table,
+ ARRAY_SIZE(user_params->lut.table));
+ }
+
+ if (params->lut.size) {
+ params->lut.addr =
+ kmb_metadata_get_table_addr(meta_buf,
+ KMB_METADATA_TABLE_LUT);
+ if (!params->lut.size)
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
+static int
+kmb_metadata_fill_warp(struct kmb_metadata *kmb_meta,
+ struct kmb_metadata_buf *meta_buf,
+ struct kmb_isp_params *user_params)
+{
+ struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
+ struct kmb_vpu_isp_params *params = meta_buf->params.isp;
+ struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
+ struct kmb_vpu_isp_params *last_params = NULL;
+ int ret = 0;
+
+ if (last_buf)
+ last_params = last_buf->params.isp;
+
+ if (user_params->update.warp) {
+ kmb_metadata_copy_warp(¶ms->warp, &user_params->warp);
+ if (params->warp.enable) {
+ ret = kmb_metadata_copy_table_usr(kmb_meta,
+ meta_buf,
+ KMB_METADATA_TABLE_WARP,
+ user_params->warp.mesh_grid,
+ ARRAY_SIZE(user_params->warp.mesh_grid));
+ if (ret < 0)
+ return ret;
+ }
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(¶ms->warp, &last_params->warp,
+ sizeof(params->warp));
+
+ if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
+ kmb_metadata_copy_table_vpu(meta_buf, last_buf,
+ KMB_METADATA_TABLE_WARP);
+ } else {
+ memcpy(¶ms->warp, def_params->warp, sizeof(params->warp));
+ }
+
+ if (params->warp.enable) {
+ params->warp.addr =
+ kmb_metadata_get_table_addr(meta_buf,
+ KMB_METADATA_TABLE_WARP);
+ if (!params->warp.addr)
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
+static int
+kmb_metadata_fill_tnf(struct kmb_metadata *kmb_meta,
+ struct kmb_metadata_buf *meta_buf,
+ struct kmb_isp_params *user_params)
+{
+ struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
+ struct kmb_vpu_isp_params *params = meta_buf->params.isp;
+ struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
+ struct kmb_vpu_isp_params *last_params = NULL;
+ struct kmb_tnf_params *tnf = NULL;
+ int ret = 0;
+
+ if (last_buf)
+ last_params = last_buf->params.isp;
+
+ if (user_params->update.tnf) {
+ kmb_metadata_copy_tnf(¶ms->tnf, &user_params->tnf);
+ if (params->tnf.enable) {
+ tnf = &user_params->tnf;
+ ret = kmb_metadata_copy_table_usr(kmb_meta,
+ meta_buf,
+ KMB_METADATA_TABLE_TNF0,
+ tnf->chroma_lut0,
+ ARRAY_SIZE(tnf->chroma_lut0));
+ if (ret < 0)
+ return ret;
+
+ ret = kmb_metadata_copy_table_usr(kmb_meta,
+ meta_buf,
+ KMB_METADATA_TABLE_TNF1,
+ tnf->chroma_lut1,
+ ARRAY_SIZE(tnf->chroma_lut1));
+ if (ret < 0)
+ return ret;
+ }
+ } else if (last_params) {
+ if (last_params != params)
+ memcpy(¶ms->tnf, &last_params->tnf,
+ sizeof(params->tnf));
+
+ if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf) {
+ kmb_metadata_copy_table_vpu(meta_buf, last_buf,
+ KMB_METADATA_TABLE_TNF0);
+ kmb_metadata_copy_table_vpu(meta_buf, last_buf,
+ KMB_METADATA_TABLE_TNF1);
+ }
+ } else {
+ memcpy(¶ms->tnf, def_params->tnf, sizeof(params->tnf));
+ }
+
+ if (params->tnf.enable) {
+ params->tnf.lut0_addr =
+ kmb_metadata_get_table_addr(meta_buf,
+ KMB_METADATA_TABLE_TNF0);
+ if (!params->tnf.lut0_addr)
+ return -EINVAL;
+
+ params->tnf.lut1_addr =
+ kmb_metadata_get_table_addr(meta_buf,
+ KMB_METADATA_TABLE_TNF1);
+ if (!params->tnf.lut1_addr)
+ return -EINVAL;
+ }
+
+ return ret;
+}
+
+/* Fill static functions for conversions here */
+static int kmb_metadata_fill_isp_params(struct kmb_metadata *kmb_meta,
+ struct kmb_metadata_buf *meta_buf,
+ struct kmb_isp_params *user_params)
+{
+ struct kmb_vpu_isp_params *params = meta_buf->params.isp;
+ struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
+ struct kmb_vpu_isp_params *last_params = NULL;
+ struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
+ int ret;
+
+ if (last_buf)
+ last_params = last_buf->params.isp;
+
+ kmb_metadata_fill_blc(params, user_params, last_params, def_params);
+
+ kmb_metadata_fill_signma_dns(params, user_params, last_params,
+ def_params);
+
+ kmb_metadata_fill_ae_awb(params, user_params, last_params, def_params);
+
+ kmb_metadata_fill_af(params, user_params, last_params, def_params);
+
+ kmb_metadata_fill_histogram(params, user_params, last_params,
+ def_params);
+
+ kmb_metadata_fill_debayer(params, user_params, last_params,
+ def_params);
+
+ kmb_metadata_fill_dog_dns(params, user_params, last_params,
+ def_params);
+
+ kmb_metadata_fill_luma_dns(params, user_params, last_params,
+ def_params);
+
+ kmb_metadata_fill_chroma_gen(params, user_params, last_params,
+ def_params);
+
+ kmb_metadata_fill_median(params, user_params, last_params, def_params);
+
+ kmb_metadata_fill_chroma_dns(params, user_params, last_params,
+ def_params);
+
+ kmb_metadata_fill_dehaze(params, user_params, last_params, def_params);
+
+ /* Copy params with tables */
+ ret = kmb_metadata_fill_lsc(kmb_meta, meta_buf, user_params);
+ if (ret < 0)
+ goto error_release_tables;
+
+ ret = kmb_metadata_fill_raw(kmb_meta, meta_buf, user_params);
+ if (ret < 0)
+ goto error_release_tables;
+
+ ret = kmb_metadata_fill_lca(kmb_meta, meta_buf, user_params);
+ if (ret < 0)
+ goto error_release_tables;
+
+ ret = kmb_metadata_fill_sharpen(kmb_meta, meta_buf, user_params);
+ if (ret < 0)
+ goto error_release_tables;
+
+ ret = kmb_metadata_fill_color_comb(kmb_meta, meta_buf, user_params);
+ if (ret < 0)
+ goto error_release_tables;
+
+ ret = kmb_metadata_fill_hdr(kmb_meta, meta_buf, user_params);
+ if (ret < 0)
+ goto error_release_tables;
+
+ ret = kmb_metadata_fill_lut(kmb_meta, meta_buf, user_params);
+ if (ret < 0)
+ goto error_release_tables;
+
+ ret = kmb_metadata_fill_warp(kmb_meta, meta_buf, user_params);
+ if (ret < 0)
+ goto error_release_tables;
+
+ ret = kmb_metadata_fill_tnf(kmb_meta, meta_buf, user_params);
+ if (ret < 0)
+ goto error_release_tables;
+
+ /* Store last buffer */
+ kmb_meta->last_buf = meta_buf;
+
+ return 0;
+
+error_release_tables:
+ kmb_metadata_release_tables(meta_buf);
+ return ret;
+}
+
+static int kmb_metadata_queue_setup(struct vb2_queue *q,
+ unsigned int *num_buffers,
+ unsigned int *num_planes,
+ unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct kmb_metadata *kmb_meta = vb2_get_drv_priv(q);
+
+ *num_planes = 1;
+ sizes[0] = kmb_meta->format.buffersize;
+
+ return 0;
+}
+
+#define to_kmb_meta_buf(vbuf) container_of(vbuf, struct kmb_metadata_buf, vb)
+
+static int kmb_metadata_buf_params_init(struct vb2_buffer *vb)
+{
+ struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue);
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf);
+
+ buf->type = KMB_METADATA_PARAMS;
+ buf->params.isp = dma_alloc_coherent(kmb_meta->dma_dev,
+ sizeof(*buf->params.isp),
+ &buf->params.dma_addr_isp, 0);
+ if (!buf->params.isp)
+ return -ENOMEM;
+
+ memset(buf->params.isp, 0, sizeof(*buf->params.isp));
+ /*
+ * Table pools will be allocated per need.
+ * The pools need to be released when last buffer is finished.
+ * Use table reference count for that purpose
+ */
+ kmb_meta->table_pools_refcnt++;
+
+ return 0;
+}
+
+static int kmb_metadata_buf_params_prepare(struct vb2_buffer *vb)
+{
+ struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue);
+ struct kmb_isp_params *user_params = vb2_plane_vaddr(vb, 0);
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf);
+
+ vb2_set_plane_payload(vb, 0, kmb_meta->format.buffersize);
+ return kmb_metadata_fill_isp_params(kmb_meta, buf, user_params);
+}
+
+static void kmb_metadata_buf_params_cleanup(struct vb2_buffer *vb)
+{
+ struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue);
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf);
+
+ if (buf == kmb_meta->last_buf)
+ kmb_meta->last_buf = NULL;
+
+ kmb_metadata_release_tables(buf);
+ dma_free_coherent(kmb_meta->dma_dev, sizeof(*buf->params.isp),
+ buf->params.isp, buf->params.dma_addr_isp);
+
+ /* Destroy allocated table pools on last finish */
+ if (kmb_meta->table_pools_refcnt-- == 1)
+ kmb_metadata_destroy_table_pools(kmb_meta);
+}
+
+static int kmb_metadata_buf_stats_init(struct vb2_buffer *vb)
+{
+ dma_addr_t stats_addr = vb2_dma_contig_plane_dma_addr(vb, 0);
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf);
+ int i;
+
+ buf->type = KMB_METADATA_STATS;
+ memset(&buf->stats.raw, 0, sizeof(buf->stats.raw));
+ buf->stats.dehaze_stats_addr = 0;
+
+ /* Fill statistics addresses */
+ for (i = 0; i < KMB_CAM_MAX_EXPOSURES; i++) {
+ buf->stats.raw[i].ae_awb_stats_addr = stats_addr +
+ offsetof(struct kmb_isp_stats,
+ exposure[i].ae_awb_stats[0]);
+
+ buf->stats.raw[i].af_stats_addr = stats_addr +
+ offsetof(struct kmb_isp_stats,
+ exposure[i].af_stats[0]);
+
+ buf->stats.raw[i].hist_luma_addr = stats_addr +
+ offsetof(struct kmb_isp_stats,
+ exposure[i].hist_luma[0]);
+
+ buf->stats.raw[i].hist_rgb_addr = stats_addr +
+ offsetof(struct kmb_isp_stats,
+ exposure[i].hist_rgb[0]);
+
+ buf->stats.raw[i].flicker_rows_addr = stats_addr +
+ offsetof(struct kmb_isp_stats,
+ exposure[i].flicker_rows[0]);
+ }
+
+ buf->stats.dehaze_stats_addr = stats_addr +
+ offsetof(struct kmb_isp_stats, dehaze);
+
+ return 0;
+}
+
+static int kmb_metadata_buf_stats_prepare(struct vb2_buffer *vb)
+{
+ struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue);
+
+ vb2_set_plane_payload(vb, 0, kmb_meta->format.buffersize);
+
+ return 0;
+}
+
+static void kmb_metadata_buf_queue(struct vb2_buffer *vb)
+{
+ struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue);
+ struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+ struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf);
+ int ret;
+
+ ret = kmb_meta->queue_ops->queue(kmb_meta->priv, buf);
+ if (ret)
+ dev_err(&kmb_meta->video.dev, "Fail metadata queue %d", ret);
+}
+
+static int kmb_metadata_start_streaming(struct vb2_queue *q,
+ unsigned int count)
+{
+ struct kmb_metadata *kmb_meta = vb2_get_drv_priv(q);
+ int ret;
+
+ ret = kmb_pipe_prepare(kmb_meta->pipe);
+ if (ret < 0)
+ goto error_discard_all_bufs;
+
+ ret = kmb_pipe_run(kmb_meta->pipe, &kmb_meta->video.entity);
+ if (ret < 0)
+ goto error_pipeline_stop;
+
+ return 0;
+
+error_pipeline_stop:
+ kmb_pipe_stop(kmb_meta->pipe, &kmb_meta->video.entity);
+error_discard_all_bufs:
+ kmb_meta->queue_ops->flush(kmb_meta->priv);
+ return 0;
+}
+
+static void kmb_metadata_stop_streaming(struct vb2_queue *q)
+{
+ struct kmb_metadata *kmb_meta = vb2_get_drv_priv(q);
+
+ kmb_pipe_stop(kmb_meta->pipe, &kmb_meta->video.entity);
+
+ kmb_meta->queue_ops->flush(kmb_meta->priv);
+}
+
+/* driver-specific operations */
+static struct vb2_ops kmb_meta_params_vb2_q_ops = {
+ .queue_setup = kmb_metadata_queue_setup,
+ .buf_init = kmb_metadata_buf_params_init,
+ .buf_prepare = kmb_metadata_buf_params_prepare,
+ .buf_cleanup = kmb_metadata_buf_params_cleanup,
+ .start_streaming = kmb_metadata_start_streaming,
+ .stop_streaming = kmb_metadata_stop_streaming,
+ .buf_queue = kmb_metadata_buf_queue,
+};
+
+static struct vb2_ops kmb_meta_stats_vb2_q_ops = {
+ .queue_setup = kmb_metadata_queue_setup,
+ .buf_init = kmb_metadata_buf_stats_init,
+ .buf_prepare = kmb_metadata_buf_stats_prepare,
+ .start_streaming = kmb_metadata_start_streaming,
+ .stop_streaming = kmb_metadata_stop_streaming,
+ .buf_queue = kmb_metadata_buf_queue,
+};
+
+#define to_kmb_meta_dev(vdev) container_of(vdev, struct kmb_metadata, video)
+
+static int kmb_metadata_querycap(struct file *file, void *fh,
+ struct v4l2_capability *cap)
+{
+ struct v4l2_fh *vfh = file->private_data;
+ struct kmb_metadata *kmb_meta =
+ to_kmb_meta_dev(vfh->vdev);
+
+ cap->bus_info[0] = 0;
+ strscpy(cap->driver, kmb_meta->video.name, sizeof(cap->driver));
+ snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+ kmb_meta->video.name);
+
+ return 0;
+}
+
+static int kmb_metadata_get_fmt(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ struct v4l2_fh *vfh = file->private_data;
+ struct kmb_metadata *kmb_meta =
+ to_kmb_meta_dev(vfh->vdev);
+
+ f->fmt.meta = kmb_meta->format;
+
+ return 0;
+}
+
+static int kmb_metadata_try_fmt_cap(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ f->fmt.meta.dataformat = V4L2_META_FMT_KMB_STATS;
+ if (f->fmt.meta.buffersize < sizeof(struct kmb_isp_stats))
+ f->fmt.meta.buffersize = sizeof(struct kmb_isp_stats);
+
+ return 0;
+}
+
+static int kmb_metadata_set_fmt_cap(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ struct v4l2_fh *vfh = file->private_data;
+ struct kmb_metadata *kmb_meta =
+ to_kmb_meta_dev(vfh->vdev);
+ int ret;
+
+ ret = kmb_metadata_try_fmt_cap(file, fh, f);
+ if (ret < 0)
+ return ret;
+
+ kmb_meta->format = f->fmt.meta;
+
+ return 0;
+}
+
+static int kmb_metadata_try_fmt_out(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ f->fmt.meta.dataformat = V4L2_META_FMT_KMB_PARAMS;
+ if (f->fmt.meta.buffersize < sizeof(struct kmb_isp_params))
+ f->fmt.meta.buffersize = sizeof(struct kmb_isp_params);
+
+ return 0;
+}
+
+static int kmb_metadata_set_fmt_out(struct file *file, void *fh,
+ struct v4l2_format *f)
+{
+ struct v4l2_fh *vfh = file->private_data;
+ struct kmb_metadata *kmb_meta =
+ to_kmb_meta_dev(vfh->vdev);
+ int ret;
+
+ ret = kmb_metadata_try_fmt_out(file, fh, f);
+ if (ret < 0)
+ return ret;
+
+ kmb_meta->format = f->fmt.meta;
+
+ return 0;
+}
+
+/* V4L2 ioctl operations */
+static const struct v4l2_ioctl_ops kmb_vid_ioctl_ops = {
+ .vidioc_querycap = kmb_metadata_querycap,
+ .vidioc_g_fmt_meta_out = kmb_metadata_get_fmt,
+ .vidioc_s_fmt_meta_out = kmb_metadata_set_fmt_out,
+ .vidioc_try_fmt_meta_out = kmb_metadata_try_fmt_out,
+ .vidioc_g_fmt_meta_cap = kmb_metadata_get_fmt,
+ .vidioc_s_fmt_meta_cap = kmb_metadata_set_fmt_cap,
+ .vidioc_try_fmt_meta_cap = kmb_metadata_try_fmt_cap,
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_qbuf = vb2_ioctl_qbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+};
+
+static int kmb_metadata_open(struct file *file)
+{
+ struct kmb_metadata *kmb_meta = video_drvdata(file);
+ int ret;
+
+ mutex_lock(&kmb_meta->lock);
+
+ ret = v4l2_fh_open(file);
+ if (ret) {
+ mutex_unlock(&kmb_meta->lock);
+ return ret;
+ }
+
+ ret = kmb_pipe_request(kmb_meta->pipe);
+ if (ret < 0)
+ goto error_fh_release;
+
+ mutex_unlock(&kmb_meta->lock);
+
+ return 0;
+
+error_fh_release:
+ _vb2_fop_release(file, NULL);
+ mutex_unlock(&kmb_meta->lock);
+ return ret;
+}
+
+static int kmb_metadata_release(struct file *file)
+{
+ struct kmb_metadata *kmb_meta = video_drvdata(file);
+ int ret;
+
+ mutex_lock(&kmb_meta->lock);
+
+ kmb_pipe_release(kmb_meta->pipe);
+
+ ret = _vb2_fop_release(file, NULL);
+
+ mutex_unlock(&kmb_meta->lock);
+
+ return ret;
+}
+
+/* V4L2 file operations */
+static const struct v4l2_file_operations kmb_vid_output_fops = {
+ .owner = THIS_MODULE,
+ .unlocked_ioctl = video_ioctl2,
+ .open = kmb_metadata_open,
+ .release = kmb_metadata_release,
+ .poll = vb2_fop_poll,
+ .mmap = vb2_fop_mmap,
+};
+
/**
- * kmb_video_init - Initialize entity
+ * kmb_metadata_init - Initialize entity
* @kmb_meta: pointer to kmb isp config device
*
* Return: 0 if successful, error code otherwise.
*/
int kmb_metadata_init(struct kmb_metadata *kmb_meta)
{
+ int ret;
+
+ mutex_init(&kmb_meta->lock);
+
+ kmb_meta->table_pools_refcnt = 0;
+ memset(kmb_meta->table_pool, 0, sizeof(kmb_meta->table_pool));
+
+ kmb_meta->video.fops = &kmb_vid_output_fops;
+ kmb_meta->video.ioctl_ops = &kmb_vid_ioctl_ops;
+ kmb_meta->video.minor = -1;
+ kmb_meta->video.release = video_device_release;
+ kmb_meta->video.vfl_type = VFL_TYPE_VIDEO;
+ kmb_meta->video.lock = &kmb_meta->lock;
+ kmb_meta->video.queue = &kmb_meta->vb2_q;
+ video_set_drvdata(&kmb_meta->video, kmb_meta);
+
+ kmb_meta->vb2_q.drv_priv = kmb_meta;
+ kmb_meta->vb2_q.buf_struct_size = sizeof(struct kmb_metadata_buf);
+ kmb_meta->vb2_q.io_modes = VB2_DMABUF | VB2_MMAP;
+ kmb_meta->vb2_q.timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ kmb_meta->vb2_q.dev = kmb_meta->dma_dev;
+ kmb_meta->vb2_q.lock = &kmb_meta->lock;
+ kmb_meta->vb2_q.min_buffers_needed = 1;
+
+ /* Initialize per type variables */
+ kmb_meta->video.device_caps = V4L2_CAP_STREAMING;
+ if (kmb_meta->type == KMB_METADATA_PARAMS) {
+ kmb_meta->video.device_caps |= V4L2_CAP_META_OUTPUT;
+ kmb_meta->video.vfl_dir = VFL_DIR_TX;
+ snprintf(kmb_meta->video.name, sizeof(kmb_meta->video.name),
+ KMB_CAM_METADATA_PARAMS_NAME);
+
+ kmb_meta->vb2_q.ops = &kmb_meta_params_vb2_q_ops;
+ kmb_meta->vb2_q.mem_ops = &vb2_dma_contig_memops;
+ kmb_meta->vb2_q.type = V4L2_BUF_TYPE_META_OUTPUT;
+
+ kmb_meta->pad.flags = MEDIA_PAD_FL_SOURCE;
+
+ kmb_meta->format.dataformat = V4L2_META_FMT_KMB_PARAMS;
+ kmb_meta->format.buffersize = sizeof(struct kmb_isp_params);
+ } else {
+ kmb_meta->video.device_caps |= V4L2_CAP_META_CAPTURE;
+ kmb_meta->video.vfl_dir = VFL_DIR_RX;
+
+ snprintf(kmb_meta->video.name, sizeof(kmb_meta->video.name),
+ KMB_CAM_METADATA_STATS_NAME);
+
+ kmb_meta->vb2_q.ops = &kmb_meta_stats_vb2_q_ops;
+ kmb_meta->vb2_q.mem_ops = &vb2_dma_contig_memops;
+ kmb_meta->vb2_q.type = V4L2_BUF_TYPE_META_CAPTURE;
+
+ kmb_meta->pad.flags = MEDIA_PAD_FL_SINK;
+
+ kmb_meta->format.dataformat = V4L2_META_FMT_KMB_STATS;
+ kmb_meta->format.buffersize = sizeof(struct kmb_isp_stats);
+ }
+
+ ret = media_entity_pads_init(&kmb_meta->video.entity,
+ 1, &kmb_meta->pad);
+ if (ret < 0)
+ goto error_mutex_destroy;
+
+ ret = vb2_queue_init(&kmb_meta->vb2_q);
+ if (ret < 0) {
+ dev_err(&kmb_meta->video.dev, "Error vb2 queue init");
+ goto error_metadata_cleanup;
+ }
+
+ kmb_params_get_defaults(&kmb_meta->def);
+
return 0;
+
+error_metadata_cleanup:
+ kmb_metadata_cleanup(kmb_meta);
+error_mutex_destroy:
+ mutex_destroy(&kmb_meta->lock);
+
+ return ret;
}
/**
@@ -22,7 +1823,10 @@ int kmb_metadata_init(struct kmb_metadata *kmb_meta)
* @kmb_meta: pointer to kmb isp config device
*/
void kmb_metadata_cleanup(struct kmb_metadata *kmb_meta)
-{ }
+{
+ media_entity_cleanup(&kmb_meta->video.entity);
+ mutex_destroy(&kmb_meta->lock);
+}
/**
* kmb_metadata_register - Register V4L2 device
@@ -34,7 +1838,15 @@ void kmb_metadata_cleanup(struct kmb_metadata *kmb_meta)
int kmb_metadata_register(struct kmb_metadata *kmb_meta,
struct v4l2_device *v4l2_dev)
{
- return 0;
+ int ret;
+
+ kmb_meta->video.v4l2_dev = v4l2_dev;
+
+ ret = video_register_device(&kmb_meta->video, VFL_TYPE_VIDEO, -1);
+ if (ret < 0)
+ dev_err(&kmb_meta->video.dev, "Failed to register video device");
+
+ return ret;
}
/**
@@ -42,4 +1854,7 @@ int kmb_metadata_register(struct kmb_metadata *kmb_meta,
* @kmb_meta: pointer to kmb isp config device
*/
void kmb_metadata_unregister(struct kmb_metadata *kmb_meta)
-{ }
+{
+ mutex_destroy(&kmb_meta->lock);
+ video_unregister_device(&kmb_meta->video);
+}
diff --git a/drivers/media/platform/keembay-camera/keembay-metadata.h b/drivers/media/platform/keembay-camera/keembay-metadata.h
index 88e85d3caba0..ab77ed11bd15 100644
--- a/drivers/media/platform/keembay-camera/keembay-metadata.h
+++ b/drivers/media/platform/keembay-camera/keembay-metadata.h
@@ -12,6 +12,7 @@
#include <media/videobuf2-v4l2.h>
#include "keembay-vpu-isp.h"
+#include "keembay-params-defaults.h"
/**
* enum kmb_metadata_table_type - Keembay metadata table type
@@ -68,12 +69,12 @@ struct kmb_metadata_table {
* @vb: Video buffer for v4l2
* @type: Metadata type
* @stats: Statistics physical addresses
- * @raw: VPU raw statistics physical addresses
- * @dehaze_stats_addr: VPU dehaze statistics physical address
+ * @stats.raw: VPU raw statistics physical addresses
+ * @stats.dehaze_stats_addr: VPU dehaze statistics physical address
* @params: VPU ISP parameters
- * @isp: VPU ISP parameters virtual address
- * @dma_addr_isp: VPU ISP parameters physical address
- * @tab: Metadata tables
+ * @params.isp: VPU ISP parameters virtual address
+ * @params.dma_addr_isp: VPU ISP parameters physical address
+ * @params.tab: Metadata tables
* @list: List for buffer queue
*/
struct kmb_metadata_buf {
@@ -118,6 +119,7 @@ struct kmb_metabuf_queue_ops {
* @table_pool: ISP tables dma pool
* @last_buf: Pointer to last enqueued buffer
* @format: Active format
+ * @def: Default ISP params
*/
struct kmb_metadata {
struct mutex lock;
@@ -138,6 +140,8 @@ struct kmb_metadata {
struct kmb_metadata_buf *last_buf;
struct v4l2_meta_format format;
+
+ struct kmb_vpu_isp_params_defaults def;
};
int kmb_metadata_init(struct kmb_metadata *kmb_meta);
diff --git a/drivers/media/platform/keembay-camera/keembay-params-defaults.c b/drivers/media/platform/keembay-camera/keembay-params-defaults.c
new file mode 100644
index 000000000000..a2dd7888375e
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-params-defaults.c
@@ -0,0 +1,326 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Intel Keem Bay camera ISP parameter defaults.
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+#include <linux/stddef.h>
+#include <linux/types.h>
+
+#include "keembay-params-defaults.h"
+
+static const struct kmb_vpu_blc_params blc_default[KMB_VPU_MAX_EXPOSURES] = {
+ {
+ .coeff1 = 800,
+ .coeff2 = 800,
+ .coeff3 = 800,
+ .coeff4 = 800,
+ },
+ {
+ .coeff1 = 800,
+ .coeff2 = 800,
+ .coeff3 = 800,
+ .coeff4 = 800,
+ },
+ {
+ .coeff1 = 800,
+ .coeff2 = 800,
+ .coeff3 = 800,
+ .coeff4 = 800,
+ }
+
+};
+
+static const struct kmb_vpu_sigma_dns_params
+ sigma_dns_default[KMB_VPU_MAX_EXPOSURES] = { 0 };
+
+static const struct kmb_vpu_lsc_params lsc_default = {
+ .threshold = 2048,
+ .width = 64,
+ .height = 44,
+ .reserved = { 0 },
+};
+
+static const struct kmb_vpu_raw_params raw_default = {
+ .awb_stats_en = 0,
+ .awb_rgb_hist_en = 0,
+ .af_stats_en = 0,
+ .luma_hist_en = 0,
+ .flicker_accum_en = 0,
+ .bad_pixel_fix_en = 0,
+ .grgb_imb_en = 1,
+ .mono_imbalance_en = 0,
+ .gain1 = 269,
+ .gain2 = 452,
+ .gain3 = 634,
+ .gain4 = 269,
+ .stop1 = 400,
+ .stop2 = 450,
+ .stop3 = 700,
+ .stop4 = 800,
+ .threshold1 = 128,
+ .alpha1 = 12,
+ .alpha2 = 12,
+ .alpha3 = 12,
+ .alpha4 = 12,
+ .threshold2 = 53,
+ .static_defect_size = 1,
+ .reserved = { 0 },
+ .flicker_first_row_acc = 0,
+ .flicker_last_row_acc = 0,
+};
+
+static const struct kmb_vpu_ae_awb_params ae_awb_default = {
+ .start_x = 0,
+ .start_y = 0,
+ .width = 100,
+ .height = 98,
+ .skip_x = 100,
+ .skip_y = 98,
+ .patches_x = 38,
+ .patches_y = 22,
+ .threshold1 = 0,
+ .threshold2 = 4095,
+};
+
+static const struct kmb_vpu_af_params af_default = {
+ .start_x = 0,
+ .start_y = 0,
+ .width = 192,
+ .height = 144,
+ .patches_x = 20,
+ .patches_y = 15,
+ .coeff = 0,
+ .threshold1 = 0,
+ .threshold2 = 0,
+ .coeffs1 = {31, 19, -32, 31, 63, 31, -50, -35, 35, -70, 35},
+ .coeffs2 = {35, 11, -29, 8, 17, 8, 78, -39, 119, -238, 119},
+};
+
+static const struct kmb_vpu_hist_params histogram_default = {
+ .start_x = 0,
+ .start_y = 0,
+ .end_x = 3839,
+ .end_y = 2156,
+ .matrix = {1719, 0, 0, 0, 1024, 0, 0, 0, 2414},
+ .weight = {64, 128, 64},
+};
+
+// only address - nothing to init...
+static const struct kmb_vpu_lca_params lca_default = { 0 };
+
+static const struct kmb_vpu_debayer_params debayer_default = {
+ .coeff1 = 51,
+ .multiplier1 = 13107,
+ .multiplier2 = 13107,
+ .coeff2 = 77,
+ .coeff3 = 150,
+ .coeff4 = 29,
+};
+
+static const struct kmb_vpu_dog_dns_params dog_dns_default = {
+ .threshold = 0,
+ .strength = 0,
+ .coeffs11 = {0, 0, 0, 0, 0, 255},
+ .coeffs15 = {0, 0, 0, 0, 0, 0, 0, 255},
+ .reserved = { 0 },
+};
+
+static const struct kmb_vpu_luma_dns_params luma_dns_default = {
+ .threshold = 13094,
+ .slope = 967,
+ .shift = 7,
+ .alpha = 50,
+ .weight = 0,
+ .per_pixel_alpha_en = 0,
+ .gain_bypass_en = 0,
+ .reserved = { 0 },
+};
+
+static const struct kmb_vpu_sharpen_params sharpen_default = {
+ .coeffs1 = {0, 0, 0, 4, 182, 396},
+ .coeffs2 = {0, 0, 0, 1, 141, 740},
+ .coeffs3 = {0, 0, 2, 42, 246, 444},
+ .shift = 15,
+ .gain1 = 3396,
+ .gain2 = 3378,
+ .gain3 = 3270,
+ .gain4 = 3400,
+ .gain5 = 207,
+ .stops1 = {20, 40, 605},
+ .gains = {10, 120, 60},
+ .stops2 = {11, 100, 2500, 4000},
+ .overshoot = 359,
+ .undershoot = 146,
+ .alpha = 36,
+ .gain6 = 128,
+ .offset = 637,
+};
+
+static const struct kmb_vpu_chroma_gen_params chroma_gen_default = {
+ .epsilon = 2,
+ .coeff1 = 426,
+ .coeff2 = 767,
+ .coeff3 = 597,
+ .coeff4 = 77,
+ .coeff5 = 150,
+ .coeff6 = 29,
+ .strength1 = 0,
+ .strength2 = 32,
+ .coeffs = {33, 59, 71},
+ .offset1 = 2,
+ .slope1 = 230,
+ .slope2 = 256,
+ .offset2 = 0,
+ .limit = 767,
+};
+
+static const struct kmb_vpu_median_params median_default = {
+ .size = 7,
+ .slope = 32,
+ .offset = -19,
+};
+
+static const struct kmb_vpu_chroma_dns_params chroma_dns_default = {
+ .limit = 255,
+ .enable = 0,
+ .threshold1 = 30,
+ .threshold2 = 30,
+ .threshold3 = 30,
+ .threshold4 = 30,
+ .threshold5 = 45,
+ .threshold6 = 45,
+ .threshold7 = 45,
+ .threshold8 = 45,
+ .slope1 = 77,
+ .offset1 = -15,
+ .slope2 = 255,
+ .offset2 = 127,
+ .grey1 = 421,
+ .grey2 = 758,
+ .grey3 = 590,
+ .coeff1 = 52,
+ .coeff2 = 32,
+ .coeff3 = 19,
+};
+
+static const struct kmb_vpu_color_comb_params color_comb_default = {
+ .matrix = {1303, 65427, 65367, 65172, 1463, 65462, 55, 65034, 1459},
+ .offsets = { 0 },
+ .coeff1 = 615,
+ .coeff2 = 342,
+ .coeff3 = 439,
+ .reserved = { 0 },
+ .enable = 0,
+ .weight1 = 85,
+ .weight2 = 86,
+ .weight3 = 85,
+ .limit1 = 512,
+ .limit2 = -8192,
+ .offset1 = 0,
+ .offset2 = 0,
+};
+
+static const struct kmb_vpu_hdr_params hdr_default = {
+ .ratio = {256, 256},
+ .scale = {262143, 262143, 262143},
+ .offset1 = -3275,
+ .slope1 = 320,
+ .offset2 = -3685,
+ .slope2 = 641,
+ .offset3 = -4054,
+ .slope3 = 4095,
+ .offset4 = 3686,
+ .gain1 = 16,
+ .blur1 = {0, 0, 255},
+ .blur2 = {0, 0, 0, 0, 255},
+ .contrast1 = 20,
+ .contrast2 = 16,
+ .enable1 = 0,
+ .enable2 = 0,
+ .offset5 = 0,
+ .offset6 = 0,
+ .strength = 0,
+ .reserved1 = { 0 },
+ .offset7 = 15,
+ .shift = 1702133760,
+ .field1 = 16,
+ .field2 = 123,
+ .gain3 = 0,
+ .min = 0,
+ .reserved2 = { 0 },
+};
+
+static const struct kmb_vpu_lut_params lut_default = {
+ .size = 512,
+ .reserved = { 0 },
+ .matrix = {262, 516, 100, 3945, 3799, 449, 449, 3720, 4023},
+ .offsets = {256, 2048, 2048},
+};
+
+static const struct kmb_vpu_tnf_params tnf_default = {
+ .factor = 179,
+ .gain = 0,
+ .offset1 = 217,
+ .slope1 = 162,
+ .offset2 = 299,
+ .slope2 = 121,
+ .min1 = 0,
+ .min2 = 40,
+ .value = 128,
+ .enable = 0,
+};
+
+static const struct kmb_vpu_dehaze_params dehaze_default = {
+ .gain1 = 512,
+ .min = 70,
+ .strength1 = 0,
+ .strength2 = 0,
+ .gain2 = 128,
+ .saturation = 127,
+ .value1 = 2048,
+ .value2 = 2048,
+ .value3 = 2048,
+ .filter = {0, 0, 255},
+};
+
+static const struct kmb_vpu_warp_params warp_default = {
+ .type = 0,
+ .relative = 0,
+ .format = 0,
+ .position = 0,
+ .reserved = { 0 },
+ .width = 8,
+ .height = 4,
+ .stride = 128,
+ .enable = 0,
+ .matrix = {1, 0, 0, 0, 1, 0, 0, 0, 1},
+ .mode = 1,
+ .values = {0, 128, 128},
+};
+
+void kmb_params_get_defaults(struct kmb_vpu_isp_params_defaults *defaults)
+{
+ defaults->blc = blc_default;
+ defaults->sigma_dns = sigma_dns_default;
+ defaults->lsc = &lsc_default;
+ defaults->raw = &raw_default;
+ defaults->ae_awb = &ae_awb_default;
+ defaults->af = &af_default;
+ defaults->histogram = &histogram_default;
+ defaults->lca = &lca_default;
+ defaults->debayer = &debayer_default;
+ defaults->dog_dns = &dog_dns_default;
+ defaults->luma_dns = &luma_dns_default;
+ defaults->sharpen = &sharpen_default;
+ defaults->chroma_gen = &chroma_gen_default;
+ defaults->median = &median_default;
+ defaults->chroma_dns = &chroma_dns_default;
+ defaults->color_comb = &color_comb_default;
+ defaults->hdr = &hdr_default;
+ defaults->lut = &lut_default;
+ defaults->tnf = &tnf_default;
+ defaults->dehaze = &dehaze_default;
+ defaults->warp = &warp_default;
+}
+
diff --git a/drivers/media/platform/keembay-camera/keembay-params-defaults.h b/drivers/media/platform/keembay-camera/keembay-params-defaults.h
new file mode 100644
index 000000000000..d6134d64be7c
--- /dev/null
+++ b/drivers/media/platform/keembay-camera/keembay-params-defaults.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Intel Keem Bay camera ISP parameter defaults.
+ *
+ * Copyright (C) 2021 Intel Corporation
+ */
+#ifndef KEEMBAY_DEFAULTS_H
+#define KEEMBAY_DEFAULTS_H
+
+#include "keembay-vpu-isp.h"
+
+struct kmb_vpu_isp_params_defaults {
+ const struct kmb_vpu_blc_params *blc;
+ const struct kmb_vpu_sigma_dns_params *sigma_dns;
+ const struct kmb_vpu_lsc_params *lsc;
+ const struct kmb_vpu_raw_params *raw;
+ const struct kmb_vpu_ae_awb_params *ae_awb;
+ const struct kmb_vpu_af_params *af;
+ const struct kmb_vpu_hist_params *histogram;
+ const struct kmb_vpu_lca_params *lca;
+ const struct kmb_vpu_debayer_params *debayer;
+ const struct kmb_vpu_dog_dns_params *dog_dns;
+ const struct kmb_vpu_luma_dns_params *luma_dns;
+ const struct kmb_vpu_sharpen_params *sharpen;
+ const struct kmb_vpu_chroma_gen_params *chroma_gen;
+ const struct kmb_vpu_median_params *median;
+ const struct kmb_vpu_chroma_dns_params *chroma_dns;
+ const struct kmb_vpu_color_comb_params *color_comb;
+ const struct kmb_vpu_hdr_params *hdr;
+ const struct kmb_vpu_lut_params *lut;
+ const struct kmb_vpu_tnf_params *tnf;
+ const struct kmb_vpu_dehaze_params *dehaze;
+ const struct kmb_vpu_warp_params *warp;
+};
+
+void kmb_params_get_defaults(struct kmb_vpu_isp_params_defaults *defaults);
+
+#endif /* KEEMBAY_DEFAULTS_H */
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread* Re: [PATCH 09/10] media: Keem Bay Camera: Add metadata video node
2021-03-19 18:06 ` [PATCH 09/10] media: Keem Bay Camera: Add metadata " Martina Krasteva
@ 2021-04-09 10:24 ` Sakari Ailus
2021-04-09 14:19 ` Martina Krasteva
0 siblings, 1 reply; 28+ messages in thread
From: Sakari Ailus @ 2021-04-09 10:24 UTC (permalink / raw)
To: Martina Krasteva
Cc: linux-media, mchehab, robh+dt, devicetree, daniele.alessandrelli,
paul.j.murphy, gjorgjix.rosikopulos
Hi Martina,
On Fri, Mar 19, 2021 at 06:06:31PM +0000, Martina Krasteva wrote:
> From: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
>
> Metadata video node implements output and capture meta type
> interface.
>
> - Output video node is used to provide isp parameters for processing.
>
> Each buffer internally has real vpu isp params structure
> allocated. User space params are copied on every qbuf based on
> update flags. Since vpu need every time all parameters to be provided,
> params are copied on every qbuf. Based on update flags they are copied
> from userspace buffer or last buffer processed.
> To reduce coping of the tables, they are allocated separately
> in table buffer pool.
> The tables are copied only when there is update from the userspace,
> otherwise they are only reference from last processed frame.
> This is possible because vpu interface has separate address for each table.
>
> - Capture video node is used to provide statistics to userspace.
> Capture video node statistics memory addresses are copied to isp
> params before processing, and corresponding update flags are set
> based on statistics availability.
>
> Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
> Co-developed-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
> Signed-off-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
> Co-developed-by: Martina Krasteva <martinax.krasteva@intel.com>
> Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
> Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
> Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
> ---
> drivers/media/platform/keembay-camera/Makefile | 4 +-
> .../platform/keembay-camera/keembay-metadata.c | 1823 +++++++++++++++++++-
> .../platform/keembay-camera/keembay-metadata.h | 14 +-
> .../keembay-camera/keembay-params-defaults.c | 326 ++++
> .../keembay-camera/keembay-params-defaults.h | 38 +
> 5 files changed, 2194 insertions(+), 11 deletions(-)
> create mode 100644 drivers/media/platform/keembay-camera/keembay-params-defaults.c
> create mode 100644 drivers/media/platform/keembay-camera/keembay-params-defaults.h
>
> diff --git a/drivers/media/platform/keembay-camera/Makefile b/drivers/media/platform/keembay-camera/Makefile
> index 8b3ad715c5c4..1b949cf009ef 100644
> --- a/drivers/media/platform/keembay-camera/Makefile
> +++ b/drivers/media/platform/keembay-camera/Makefile
> @@ -1,5 +1,5 @@
> keembay-cam-objs = keembay-camera.o keembay-pipeline.o \
> - keembay-cam-xlink.o keembay-isp.o \
> - keembay-metadata.o keembay-video.o
> + keembay-cam-xlink.o keembay-params-defaults.o \
> + keembay-isp.o keembay-metadata.o keembay-video.o
>
> obj-$(CONFIG_VIDEO_INTEL_KEEMBAY_CAMERA) += keembay-cam.o
> diff --git a/drivers/media/platform/keembay-camera/keembay-metadata.c b/drivers/media/platform/keembay-camera/keembay-metadata.c
> index a1df746d9582..8807e3f322c5 100644
> --- a/drivers/media/platform/keembay-camera/keembay-metadata.c
> +++ b/drivers/media/platform/keembay-camera/keembay-metadata.c
> @@ -4,17 +4,1818 @@
> *
> * Copyright (C) 2021 Intel Corporation
> */
> +
> +#include <linux/keembay-isp-ctl.h>
> +#include <linux/dmapool.h>
> +
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/videobuf2-dma-contig.h>
> +#include <media/videobuf2-vmalloc.h>
> +
> +#include "keembay-pipeline.h"
> #include "keembay-metadata.h"
>
> +#define KMB_CAM_METADATA_STATS_NAME "keembay-metadata-stats"
> +#define KMB_CAM_METADATA_PARAMS_NAME "keembay-metadata-params"
> +
> +#define KMB_TABLE_ALIGN 64
> +
> +/* Table names map */
> +static const char *table_name[KMB_METADATA_TABLE_MAX] = {
> + "LSC",
> + "StaticDefect",
> + "LCA",
> + "HDR",
> + "Sharpness",
> + "Color cumb",
> + "LUT",
> + "TNF1",
> + "TNF2",
> + "Dehaze",
> + "Warp",
> +};
> +
> +static void
> +kmb_metadata_copy_blc(struct kmb_vpu_blc_params *dst,
> + struct kmb_blc_params *src)
> +{
> + int i;
unsigned int would be preferred. The same for functions with similar loops
below.
> +
> + for (i = 0; i < KMB_CAM_MAX_EXPOSURES; i++) {
> + dst[i].coeff1 = src[i].coeff1;
> + dst[i].coeff2 = src[i].coeff2;
> + dst[i].coeff3 = src[i].coeff3;
> + dst[i].coeff4 = src[i].coeff4;
> + }
> +}
> +
> +static void
> +kmb_metadata_copy_sigma_dns(struct kmb_vpu_sigma_dns_params *dst,
> + struct kmb_sigma_dns_params *src)
> +{
> + int i;
> +
> + for (i = 0; i < KMB_CAM_MAX_EXPOSURES; i++) {
> + dst[i].noise = src[i].noise;
> + dst[i].threshold1 = src[i].threshold1;
> + dst[i].threshold2 = src[i].threshold2;
> + dst[i].threshold3 = src[i].threshold3;
> + dst[i].threshold4 = src[i].threshold4;
> + dst[i].threshold5 = src[i].threshold5;
> + dst[i].threshold6 = src[i].threshold6;
> + dst[i].threshold7 = src[i].threshold7;
> + dst[i].threshold8 = src[i].threshold8;
> + }
> +}
> +
> +static void
> +kmb_metadata_copy_lsc(struct kmb_vpu_lsc_params *dst,
> + struct kmb_lsc_params *src)
> +{
> + dst->threshold = src->threshold;
> + dst->width = src->width;
> + dst->height = src->height;
> +}
> +
> +static void
> +kmb_metadata_copy_raw(struct kmb_vpu_raw_params *dst,
> + struct kmb_raw_params *src)
> +{
> + dst->awb_stats_en = src->awb_stats_en;
> + dst->awb_rgb_hist_en = src->awb_rgb_hist_en;
> + dst->af_stats_en = src->af_stats_en;
> + dst->luma_hist_en = src->luma_hist_en;
> + dst->flicker_accum_en = src->flicker_accum_en;
> + dst->bad_pixel_fix_en = src->bad_pixel_fix_en;
> + dst->grgb_imb_en = src->grgb_imb_en;
> + dst->mono_imbalance_en = src->mono_imbalance_en;
> + dst->gain1 = src->gain1;
> + dst->gain2 = src->gain2;
> + dst->gain3 = src->gain3;
> + dst->gain4 = src->gain4;
> + dst->stop1 = src->stop1;
> + dst->stop2 = src->stop2;
> + dst->stop3 = src->stop3;
> + dst->stop4 = src->stop4;
> + dst->threshold1 = src->threshold1;
> + dst->alpha1 = src->alpha1;
> + dst->alpha2 = src->alpha2;
> + dst->alpha3 = src->alpha3;
> + dst->alpha4 = src->alpha4;
> + dst->threshold2 = src->threshold2;
> + dst->static_defect_size = src->static_defect_size;
> + dst->flicker_first_row_acc = src->start_row;
> + dst->flicker_last_row_acc = src->end_row;
> +}
> +
> +static void
> +kmb_metadata_copy_ae_awb(struct kmb_vpu_ae_awb_params *dst,
> + struct kmb_ae_awb_params *src)
> +{
> + dst->start_x = src->start_x;
> + dst->start_y = src->start_y;
> + dst->width = src->width;
> + dst->height = src->height;
> + dst->skip_x = src->skip_x;
> + dst->skip_y = src->skip_y;
> + dst->patches_x = src->patches_x;
> + dst->patches_y = src->patches_y;
> + dst->threshold1 = src->threshold1;
> + dst->threshold2 = src->threshold2;
> +}
> +
> +static void
> +kmb_metadata_copy_af(struct kmb_vpu_af_params *dst,
> + struct kmb_af_params *src)
> +{
> + int i;
> +
> + dst->start_x = src->start_x;
> + dst->start_y = src->start_y;
> + dst->width = src->width;
> + dst->height = src->height;
> + dst->patches_x = src->patches_x;
> + dst->patches_y = src->patches_y;
> + dst->coeff = src->coeff;
> + dst->threshold1 = src->threshold1;
> + dst->threshold2 = src->threshold2;
> +
> + for (i = 0; i < ARRAY_SIZE(dst->coeffs1); i++) {
> + dst->coeffs1[i] = src->coeffs1[i];
> + dst->coeffs2[i] = src->coeffs2[i];
> + }
> +}
> +
> +static void
> +kmb_metadata_copy_histogram(struct kmb_vpu_hist_params *dst,
> + struct kmb_hist_params *src)
> +{
> + int i;
> +
> + dst->start_x = src->start_x;
> + dst->start_y = src->start_y;
> + dst->end_x = src->end_x;
> + dst->end_y = src->end_y;
> +
> + for (i = 0; i < ARRAY_SIZE(dst->matrix); i++)
> + dst->matrix[i] = src->matrix[i];
> +
> + for (i = 0; i < ARRAY_SIZE(dst->weight); i++)
> + dst->weight[i] = src->weight[i];
> +}
> +
> +static void
> +kmb_metadata_copy_debayer(struct kmb_vpu_debayer_params *dst,
> + struct kmb_debayer_params *src)
> +{
> + dst->coeff1 = src->coeff1;
> + dst->multiplier1 = src->multiplier1;
> + dst->multiplier2 = src->multiplier2;
> + dst->coeff2 = src->coeff2;
> + dst->coeff3 = src->coeff3;
> + dst->coeff4 = src->coeff4;
> +}
> +
> +static void
> +kmb_metadata_copy_dog_dns(struct kmb_vpu_dog_dns_params *dst,
> + struct kmb_dog_dns_params *src)
> +{
> + int i;
> +
> + dst->threshold = src->threshold;
> + dst->strength = src->strength;
> +
> + for (i = 0; i < ARRAY_SIZE(dst->coeffs11); i++)
> + dst->coeffs11[i] = src->coeffs11[i];
> +
> + for (i = 0; i < ARRAY_SIZE(dst->coeffs15); i++)
> + dst->coeffs15[i] = src->coeffs15[i];
> +}
> +
> +static void
> +kmb_metadata_copy_luma_dns(struct kmb_vpu_luma_dns_params *dst,
> + struct kmb_luma_dns_params *src)
> +{
> + dst->threshold = src->threshold;
> + dst->slope = src->slope;
> + dst->shift = src->shift;
> + dst->alpha = src->alpha;
> + dst->weight = src->weight;
> + dst->per_pixel_alpha_en = src->per_pixel_alpha_en;
> + dst->gain_bypass_en = src->gain_bypass_en;
> +}
> +
> +static void
> +kmb_metadata_copy_sharpen(struct kmb_vpu_sharpen_params *dst,
> + struct kmb_sharpen_params *src)
> +{
> + int i;
> +
> + for (i = 0; i < ARRAY_SIZE(dst->coeffs1); i++) {
> + dst->coeffs1[i] = src->coeffs1[i];
> + dst->coeffs2[i] = src->coeffs2[i];
> + dst->coeffs3[i] = src->coeffs3[i];
> + }
> +
> + dst->shift = src->shift;
> + dst->gain1 = src->gain1;
> + dst->gain2 = src->gain2;
> + dst->gain3 = src->gain3;
> + dst->gain4 = src->gain4;
> + dst->gain5 = src->gain5;
> +
> + for (i = 0; i < ARRAY_SIZE(dst->stops1); i++) {
> + dst->stops1[i] = src->stops1[i];
> + dst->gains[i] = src->gains[i];
> + }
> +
> + for (i = 0; i < ARRAY_SIZE(dst->stops2); i++)
> + dst->stops2[i] = src->stops2[i];
> +
> + dst->overshoot = src->overshoot;
> + dst->undershoot = src->undershoot;
> + dst->alpha = src->alpha;
> + dst->gain6 = src->gain6;
> + dst->offset = src->offset;
> +}
> +
> +static void
> +kmb_metadata_copy_chroma_gen(struct kmb_vpu_chroma_gen_params *dst,
> + struct kmb_chroma_gen_params *src)
> +{
> + int i;
> +
> + dst->epsilon = src->epsilon;
> + dst->coeff1 = src->coeff1;
> + dst->coeff2 = src->coeff2;
> + dst->coeff3 = src->coeff3;
> + dst->coeff4 = src->coeff4;
> + dst->coeff5 = src->coeff5;
> + dst->coeff6 = src->coeff6;
> + dst->strength1 = src->strength1;
> + dst->strength2 = src->strength2;
> +
> + for (i = 0; i < ARRAY_SIZE(dst->coeffs); i++)
> + dst->coeffs[i] = src->coeffs[i];
> +
> + dst->offset1 = src->offset1;
> + dst->slope1 = src->slope1;
> + dst->slope2 = src->slope2;
> + dst->offset2 = src->offset2;
> + dst->limit = src->limit;
> +}
> +
> +static void
> +kmb_metadata_copy_median(struct kmb_vpu_median_params *dst,
> + struct kmb_median_params *src)
> +{
> + dst->size = src->size;
> + dst->slope = src->slope;
> + dst->offset = src->offset;
> +}
> +
> +static void
> +kmb_metadata_copy_chroma_dns(struct kmb_vpu_chroma_dns_params *dst,
> + struct kmb_chroma_dns_params *src)
> +{
> + dst->limit = src->limit;
> + dst->enable = src->enable;
> + dst->threshold1 = src->threshold1;
> + dst->threshold2 = src->threshold2;
> + dst->threshold3 = src->threshold3;
> + dst->threshold4 = src->threshold4;
> + dst->threshold5 = src->threshold5;
> + dst->threshold6 = src->threshold6;
> + dst->threshold7 = src->threshold7;
> + dst->threshold8 = src->threshold8;
> + dst->slope1 = src->slope1;
> + dst->offset1 = src->offset1;
> + dst->slope2 = src->slope2;
> + dst->offset2 = src->offset2;
> + dst->grey1 = src->grey1;
> + dst->grey2 = src->grey2;
> + dst->grey3 = src->grey3;
> + dst->coeff1 = src->coeff1;
> + dst->coeff2 = src->coeff2;
> + dst->coeff3 = src->coeff3;
> +}
> +
> +static void
> +kmb_metadata_copy_color_comb(struct kmb_vpu_color_comb_params *dst,
> + struct kmb_color_comb_params *src)
> +{
> + int i;
> +
> + for (i = 0; i < ARRAY_SIZE(dst->matrix); i++)
> + dst->matrix[i] = src->matrix[i];
> +
> + for (i = 0; i < ARRAY_SIZE(dst->offsets); i++)
> + dst->offsets[i] = src->offsets[i];
> +
> + dst->coeff1 = src->coeff1;
> + dst->coeff2 = src->coeff2;
> + dst->coeff3 = src->coeff3;
> + dst->enable = src->enable;
> + dst->weight1 = src->weight1;
> + dst->weight2 = src->weight2;
> + dst->weight3 = src->weight3;
> + dst->limit1 = src->limit1;
> + dst->limit2 = src->limit2;
> + dst->offset1 = src->offset1;
> + dst->offset2 = src->offset2;
> +}
> +
> +static void
> +kmb_metadata_copy_hdr(struct kmb_vpu_hdr_params *dst,
> + struct kmb_hdr_params *src)
> +{
> + int i;
> +
> + for (i = 0; i < ARRAY_SIZE(dst->ratio); i++)
> + dst->ratio[i] = src->ratio[i];
> +
> + for (i = 0; i < ARRAY_SIZE(dst->scale); i++)
> + dst->scale[i] = src->scale[i];
> +
> + dst->offset1 = src->offset1;
> + dst->slope1 = src->slope1;
> + dst->offset2 = src->offset2;
> + dst->slope2 = src->slope2;
> + dst->offset3 = src->offset3;
> + dst->slope3 = src->slope3;
> + dst->offset4 = src->offset4;
> + dst->gain1 = src->gain1;
> +
> + for (i = 0; i < ARRAY_SIZE(dst->blur1); i++)
> + dst->blur1[i] = src->blur1[i];
> +
> + for (i = 0; i < ARRAY_SIZE(dst->blur2); i++)
> + dst->blur2[i] = src->blur2[i];
> +
> + dst->contrast1 = src->contrast1;
> + dst->contrast2 = src->contrast2;
> + dst->enable1 = src->enable1;
> + dst->enable2 = src->enable2;
> + dst->offset5 = src->offset5;
> + dst->gain2 = src->gain2;
> + dst->offset6 = src->offset6;
> + dst->strength = src->strength;
> + dst->offset7 = src->offset7;
> + dst->shift = src->shift;
> + dst->field1 = src->field1;
> + dst->field2 = src->field2;
> + dst->gain3 = src->gain3;
> + dst->min = src->min;
> +}
> +
> +static void
> +kmb_metadata_copy_lut(struct kmb_vpu_lut_params *dst,
> + struct kmb_lut_params *src)
> +{
> + int i;
> +
> + dst->size = src->size;
> + for (i = 0; i < ARRAY_SIZE(dst->matrix); i++)
> + dst->matrix[i] = src->matrix[i];
> +
> + for (i = 0; i < ARRAY_SIZE(dst->offsets); i++)
> + dst->offsets[i] = src->offsets[i];
> +}
> +
> +static void
> +kmb_metadata_copy_tnf(struct kmb_vpu_tnf_params *dst,
> + struct kmb_tnf_params *src)
> +{
> + dst->factor = src->factor;
> + dst->gain = src->gain;
> + dst->offset1 = src->offset1;
> + dst->slope1 = src->slope1;
> + dst->offset2 = src->offset2;
> + dst->slope2 = src->slope2;
> + dst->min1 = src->min1;
> + dst->min2 = src->min2;
> + dst->value = src->value;
> + dst->enable = src->enable;
> +}
> +
> +static void
> +kmb_metadata_copy_dehaze(struct kmb_vpu_dehaze_params *dst,
> + struct kmb_dehaze_params *src)
> +{
> + int i;
> +
> + dst->gain1 = src->gain1;
> + dst->min = src->min;
> + dst->strength1 = src->strength1;
> + dst->strength2 = src->strength2;
> + dst->gain2 = src->gain2;
> + dst->saturation = src->saturation;
> + dst->value1 = src->value1;
> + dst->value2 = src->value2;
> + dst->value3 = src->value3;
> +
> + for (i = 0; i < ARRAY_SIZE(dst->filter); i++)
> + dst->filter[i] = src->filter[i];
> +}
> +
> +static void
> +kmb_metadata_copy_warp(struct kmb_vpu_warp_params *dst,
> + struct kmb_warp_params *src)
> +{
> + int i;
> +
> + dst->type = src->type;
> + dst->relative = src->relative;
> + dst->format = src->format;
> + dst->position = src->position;
> + dst->width = src->width;
> + dst->height = src->height;
> + dst->stride = src->stride;
> + dst->enable = src->enable;
> +
> + for (i = 0; i < ARRAY_SIZE(dst->matrix); i++)
> + dst->matrix[i] = src->matrix[i];
> +
> + dst->mode = src->mode;
> +
> + for (i = 0; i < ARRAY_SIZE(dst->values); i++)
> + dst->values[i] = src->values[i];
> +}
> +
> +/* VPU Params tables */
> +static struct kmb_metadata_table *
> +kmb_metadata_cpalloc_table(struct kmb_metadata *kmb_meta,
> + enum kmb_metadata_table_type type,
> + size_t src_table_size)
> +{
> + struct kmb_metadata_table *table;
> +
> + lockdep_assert_held(&kmb_meta->lock);
> +
> + /* First create pool if needed */
> + if (!kmb_meta->table_pool[type]) {
> + kmb_meta->table_pool[type] =
> + dma_pool_create(table_name[type],
> + kmb_meta->dma_dev,
> + src_table_size + sizeof(*table),
> + KMB_TABLE_ALIGN, 0);
> + if (!kmb_meta->table_pool[type]) {
> + dev_err(kmb_meta->dma_dev,
> + "Fail to create %s pool", table_name[type]);
> + return NULL;
> + }
> + }
> +
> + table = kmalloc(sizeof(*table), GFP_KERNEL);
> + if (!table)
> + return NULL;
> +
> + kref_init(&table->refcount);
> + table->pool = kmb_meta->table_pool[type];
> +
> + table->cpu_addr = dma_pool_alloc(kmb_meta->table_pool[type],
> + GFP_KERNEL,
> + &table->dma_addr);
> + if (!table->cpu_addr) {
> + kfree(table);
> + return NULL;
> + }
> +
> + return table;
> +}
> +
> +static void kmb_metadata_free_table(struct kref *ref)
> +{
> + struct kmb_metadata_table *table =
> + container_of(ref, struct kmb_metadata_table, refcount);
> +
> + dma_pool_free(table->pool, table->cpu_addr, table->dma_addr);
> + kfree(table);
> +}
> +
> +static void
> +kmb_metadata_release_tables(struct kmb_metadata_buf *meta_buf)
How about calling this kmb_metadata_put_tables()? It puts the tables that
may result in releasing them.
> +{
> + int i;
> +
> + for (i = 0; i < KMB_METADATA_TABLE_MAX; i++) {
> + if (meta_buf->params.tab[i]) {
> + kref_put(&meta_buf->params.tab[i]->refcount,
> + kmb_metadata_free_table);
> + meta_buf->params.tab[i] = NULL;
> + }
> + }
> +}
> +
> +static void
> +kmb_metadata_destroy_table_pools(struct kmb_metadata *kmb_meta)
> +{
> + int i;
> +
> + /* Release allocated pools during streaming */
> + for (i = 0; i < KMB_METADATA_TABLE_MAX; i++) {
> + dma_pool_destroy(kmb_meta->table_pool[i]);
> + kmb_meta->table_pool[i] = NULL;
> + }
> +}
> +
> +static dma_addr_t
> +kmb_metadata_get_table_addr(struct kmb_metadata_buf *meta_buf,
> + enum kmb_metadata_table_type type)
> +{
> + struct kmb_metadata_table *table = meta_buf->params.tab[type];
> +
> + if (!table)
> + return 0;
> +
> + return table->dma_addr;
> +}
> +
> +static struct kmb_metadata_table *
> +kmb_metadata_create_table(struct kmb_metadata *kmb_meta,
> + struct kmb_metadata_buf *meta_buf,
> + enum kmb_metadata_table_type type,
> + size_t user_table_size)
> +{
> + struct kmb_metadata_table *table;
> +
> + lockdep_assert_held(&kmb_meta->lock);
> +
> + table = kmb_metadata_cpalloc_table(kmb_meta,
> + type,
> + user_table_size);
Fits on fewer lines.
> + if (!table)
> + return NULL;
> +
> + if (meta_buf->params.tab[type])
> + kref_put(&meta_buf->params.tab[type]->refcount,
> + kmb_metadata_free_table);
> +
> + meta_buf->params.tab[type] = table;
> +
> + return table;
> +}
> +
> +static int
> +kmb_metadata_copy_table_usr(struct kmb_metadata *kmb_meta,
> + struct kmb_metadata_buf *meta_buf,
> + enum kmb_metadata_table_type type,
> + u8 *user_table, size_t user_table_size)
> +{
> + struct kmb_metadata_table *table;
> +
> + table = kmb_metadata_create_table(kmb_meta, meta_buf,
> + type, user_table_size);
> + if (!table)
> + return -ENOMEM;
> +
> + memcpy(table->cpu_addr, user_table, user_table_size);
> +
> + return 0;
> +}
> +
> +static int kmb_metadata_create_default_table(struct kmb_metadata *kmb_meta,
> + struct kmb_metadata_buf *meta_buf,
> + enum kmb_metadata_table_type type,
> + u8 *user_table,
> + size_t user_table_size)
> +{
> + struct kmb_metadata_table *table;
> +
> + table = kmb_metadata_create_table(kmb_meta, meta_buf,
> + type, user_table_size);
> + if (!table)
> + return -ENOMEM;
> +
> + memset(table->cpu_addr, 0, user_table_size);
> +
> + return 0;
> +}
> +
> +static void
> +kmb_metadata_copy_table_vpu(struct kmb_metadata_buf *meta_buf,
> + struct kmb_metadata_buf *last_meta_buf,
> + enum kmb_metadata_table_type type)
> +{
> + /* Do nothing if params are the same */
> + if (WARN_ON(meta_buf->params.isp == last_meta_buf->params.isp))
> + return;
> +
> + meta_buf->params.tab[type] = last_meta_buf->params.tab[type];
> + if (meta_buf->params.tab[type])
> + kref_get(&meta_buf->params.tab[type]->refcount);
> +}
> +
> +static void
> +kmb_metadata_fill_blc(struct kmb_vpu_isp_params *params,
> + struct kmb_isp_params *user_params,
> + struct kmb_vpu_isp_params *last_params,
> + struct kmb_vpu_isp_params_defaults *def_params)
> +{
> + if (user_params->update.blc) {
> + kmb_metadata_copy_blc(params->blc, user_params->blc);
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(params->blc, last_params->blc,
> + sizeof(params->blc));
> + } else {
> + memcpy(params->blc, def_params->blc, sizeof(params->blc));
> + }
> +}
> +
> +static void
> +kmb_metadata_fill_signma_dns(struct kmb_vpu_isp_params *params,
> + struct kmb_isp_params *user_params,
> + struct kmb_vpu_isp_params *last_params,
> + struct kmb_vpu_isp_params_defaults *def_params)
> +{
> + if (user_params->update.sigma_dns) {
> + kmb_metadata_copy_sigma_dns(params->sigma_dns,
> + user_params->sigma_dns);
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(params->sigma_dns, last_params->sigma_dns,
> + sizeof(params->sigma_dns));
> + } else {
> + memcpy(params->sigma_dns, def_params->sigma_dns,
> + sizeof(params->sigma_dns));
> + }
> +}
> +
> +static void
> +kmb_metadata_fill_ae_awb(struct kmb_vpu_isp_params *params,
> + struct kmb_isp_params *user_params,
> + struct kmb_vpu_isp_params *last_params,
> + struct kmb_vpu_isp_params_defaults *def_params)
> +{
> + if (user_params->update.ae_awb) {
> + kmb_metadata_copy_ae_awb(¶ms->ae_awb,
> + &user_params->ae_awb);
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(¶ms->ae_awb, &last_params->ae_awb,
> + sizeof(params->ae_awb));
> + } else {
> + memcpy(¶ms->ae_awb, def_params->ae_awb,
> + sizeof(params->ae_awb));
> + }
> +}
> +
> +static void
> +kmb_metadata_fill_af(struct kmb_vpu_isp_params *params,
> + struct kmb_isp_params *user_params,
> + struct kmb_vpu_isp_params *last_params,
> + struct kmb_vpu_isp_params_defaults *def_params)
> +{
> + if (user_params->update.af) {
> + kmb_metadata_copy_af(¶ms->af, &user_params->af);
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(¶ms->af, &last_params->af,
> + sizeof(params->af));
> + } else {
> + memcpy(¶ms->af, def_params->af, sizeof(params->af));
> + }
> +}
> +
> +static void
> +kmb_metadata_fill_histogram(struct kmb_vpu_isp_params *params,
> + struct kmb_isp_params *user_params,
> + struct kmb_vpu_isp_params *last_params,
> + struct kmb_vpu_isp_params_defaults *def_params)
> +{
> + if (user_params->update.histogram) {
> + kmb_metadata_copy_histogram(¶ms->histogram,
> + &user_params->histogram);
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(¶ms->histogram, &last_params->histogram,
> + sizeof(params->histogram));
> + } else {
> + memcpy(¶ms->histogram, def_params->histogram,
> + sizeof(params->histogram));
> + }
> +}
> +
> +static void
> +kmb_metadata_fill_debayer(struct kmb_vpu_isp_params *params,
> + struct kmb_isp_params *user_params,
> + struct kmb_vpu_isp_params *last_params,
> + struct kmb_vpu_isp_params_defaults *def_params)
> +{
> + if (user_params->update.debayer) {
> + kmb_metadata_copy_debayer(¶ms->debayer,
> + &user_params->debayer);
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(¶ms->debayer, &last_params->debayer,
> + sizeof(params->debayer));
> + } else {
> + memcpy(¶ms->debayer, def_params->debayer,
> + sizeof(params->debayer));
> + }
> +}
> +
> +static void
> +kmb_metadata_fill_dog_dns(struct kmb_vpu_isp_params *params,
> + struct kmb_isp_params *user_params,
> + struct kmb_vpu_isp_params *last_params,
> + struct kmb_vpu_isp_params_defaults *def_params)
> +{
> + if (user_params->update.dog_dns) {
> + kmb_metadata_copy_dog_dns(¶ms->dog_dns,
> + &user_params->dog_dns);
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(¶ms->dog_dns, &last_params->dog_dns,
> + sizeof(params->dog_dns));
> + } else {
> + memcpy(¶ms->dog_dns, def_params->dog_dns,
> + sizeof(params->dog_dns));
> + }
> +}
> +
> +static void
> +kmb_metadata_fill_luma_dns(struct kmb_vpu_isp_params *params,
> + struct kmb_isp_params *user_params,
> + struct kmb_vpu_isp_params *last_params,
> + struct kmb_vpu_isp_params_defaults *def_params)
> +{
> + if (user_params->update.luma_dns) {
> + kmb_metadata_copy_luma_dns(¶ms->luma_dns,
> + &user_params->luma_dns);
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(¶ms->luma_dns, &last_params->luma_dns,
> + sizeof(params->luma_dns));
> + } else {
> + memcpy(¶ms->luma_dns, def_params->luma_dns,
> + sizeof(params->luma_dns));
> + }
> +}
> +
> +static void
> +kmb_metadata_fill_chroma_gen(struct kmb_vpu_isp_params *params,
> + struct kmb_isp_params *user_params,
> + struct kmb_vpu_isp_params *last_params,
> + struct kmb_vpu_isp_params_defaults *def_params)
> +{
> + if (user_params->update.chroma_gen) {
> + kmb_metadata_copy_chroma_gen(¶ms->chroma_gen,
> + &user_params->chroma_gen);
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(¶ms->chroma_gen, &last_params->chroma_gen,
> + sizeof(params->chroma_gen));
> + } else {
> + memcpy(¶ms->chroma_gen, def_params->chroma_gen,
> + sizeof(params->chroma_gen));
> + }
> +}
> +
> +static void
> +kmb_metadata_fill_median(struct kmb_vpu_isp_params *params,
> + struct kmb_isp_params *user_params,
> + struct kmb_vpu_isp_params *last_params,
> + struct kmb_vpu_isp_params_defaults *def_params)
> +{
> + if (user_params->update.median) {
> + kmb_metadata_copy_median(¶ms->median,
> + &user_params->median);
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(¶ms->median, &last_params->median,
> + sizeof(params->median));
> + } else {
> + memcpy(¶ms->median, def_params->median,
> + sizeof(params->median));
> + }
> +}
> +
> +static void
> +kmb_metadata_fill_chroma_dns(struct kmb_vpu_isp_params *params,
> + struct kmb_isp_params *user_params,
> + struct kmb_vpu_isp_params *last_params,
> + struct kmb_vpu_isp_params_defaults *def_params)
> +{
> + if (user_params->update.chroma_dns) {
> + kmb_metadata_copy_chroma_dns(¶ms->chroma_dns,
> + &user_params->chroma_dns);
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(¶ms->chroma_dns, &last_params->chroma_dns,
> + sizeof(params->chroma_dns));
> + } else {
> + memcpy(¶ms->chroma_dns, def_params->chroma_dns,
> + sizeof(params->chroma_dns));
> + }
> +}
> +
> +static void
> +kmb_metadata_fill_dehaze(struct kmb_vpu_isp_params *params,
> + struct kmb_isp_params *user_params,
> + struct kmb_vpu_isp_params *last_params,
> + struct kmb_vpu_isp_params_defaults *def_params)
> +{
> + if (user_params->update.dehaze) {
> + kmb_metadata_copy_dehaze(¶ms->dehaze,
> + &user_params->dehaze);
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(¶ms->dehaze, &last_params->dehaze,
> + sizeof(params->dehaze));
> + } else {
> + memcpy(¶ms->dehaze, def_params->dehaze,
> + sizeof(params->dehaze));
> + }
> +}
> +
> +static int
> +kmb_metadata_fill_lsc(struct kmb_metadata *kmb_meta,
> + struct kmb_metadata_buf *meta_buf,
> + struct kmb_isp_params *user_params)
> +{
> + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
> + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> + struct kmb_vpu_isp_params *last_params = NULL;
> + int ret = 0;
> +
> + if (last_buf)
> + last_params = last_buf->params.isp;
> +
> + if (user_params->update.lsc) {
> + kmb_metadata_copy_lsc(¶ms->lsc,
> + &user_params->lsc);
> + if (params->lsc.width && params->lsc.height) {
> + ret = kmb_metadata_copy_table_usr(kmb_meta,
> + meta_buf,
> + KMB_METADATA_TABLE_LSC,
> + user_params->lsc.gain_mesh,
> + params->lsc.width *
> + params->lsc.height);
> + if (ret < 0)
> + return ret;
> + }
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(¶ms->lsc, &last_params->lsc,
> + sizeof(params->lsc));
> +
> + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
> + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> + KMB_METADATA_TABLE_LSC);
> + } else {
> + memcpy(¶ms->lsc, def_params->lsc, sizeof(params->lsc));
> + kmb_metadata_create_default_table(kmb_meta,
> + meta_buf,
> + KMB_METADATA_TABLE_LSC,
> + user_params->lsc.gain_mesh,
> + ARRAY_SIZE(user_params->lsc.gain_mesh));
> + }
> +
> + if (params->lsc.width && params->lsc.height) {
> + params->lsc.addr =
> + kmb_metadata_get_table_addr(meta_buf,
> + KMB_METADATA_TABLE_LSC);
> + if (!params->lsc.addr)
> + ret = -EINVAL;
> + }
> +
> + return ret;
> +}
There seems to be a few groups of functions that look very similar to each
other, with the difference that they just operate on different struct
fields. I wonder if these could be refactored into one (or a few) function
per group that would just operate on different data. If you need to
differentiate more, you can use smaller functions to do a particular part
of the job that is different between these functions (within a group) and
use the offsetof() and sizeof() macros.
> +
> +static int
> +kmb_metadata_fill_raw(struct kmb_metadata *kmb_meta,
> + struct kmb_metadata_buf *meta_buf,
> + struct kmb_isp_params *user_params)
> +{
> + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
> + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> + struct kmb_vpu_isp_params *last_params = NULL;
> + int ret = 0;
> +
> + if (last_buf)
> + last_params = last_buf->params.isp;
> +
> + if (user_params->update.raw) {
> + kmb_metadata_copy_raw(¶ms->raw,
> + &user_params->raw);
> + if (params->raw.static_defect_size) {
> + ret = kmb_metadata_copy_table_usr(kmb_meta,
> + meta_buf,
> + KMB_METADATA_TABLE_SDEFECT,
> + user_params->raw.static_defect_map,
> + params->raw.static_defect_size);
> + if (ret < 0)
> + return ret;
> + }
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(¶ms->raw, &last_params->raw,
> + sizeof(params->raw));
> +
> + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
> + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> + KMB_METADATA_TABLE_SDEFECT);
> + } else {
> + memcpy(¶ms->raw, def_params->raw, sizeof(params->raw));
> + kmb_metadata_create_default_table(kmb_meta,
> + meta_buf,
> + KMB_METADATA_TABLE_SDEFECT,
> + user_params->raw.static_defect_map,
> + ARRAY_SIZE(user_params->raw.static_defect_map));
> + }
> +
> + if (params->raw.static_defect_size) {
> + params->raw.static_defect_addr =
> + kmb_metadata_get_table_addr(meta_buf,
> + KMB_METADATA_TABLE_SDEFECT);
> + if (!params->raw.static_defect_addr)
> + ret = -EINVAL;
> + }
> +
> + return ret;
> +}
> +
> +static int
> +kmb_metadata_fill_lca(struct kmb_metadata *kmb_meta,
> + struct kmb_metadata_buf *meta_buf,
> + struct kmb_isp_params *user_params)
> +{
> + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> + struct kmb_vpu_isp_params *last_params = NULL;
> + int ret = 0;
> +
> + if (last_buf)
> + last_params = last_buf->params.isp;
> +
> + if (user_params->update.lca) {
> + ret = kmb_metadata_copy_table_usr(kmb_meta,
> + meta_buf,
> + KMB_METADATA_TABLE_LCA,
> + user_params->lca.coeff,
> + ARRAY_SIZE(user_params->lca.coeff));
> + if (ret < 0)
> + return ret;
> + } else if (last_params) {
> + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
> + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> + KMB_METADATA_TABLE_LCA);
> + } else {
> + kmb_metadata_create_default_table(kmb_meta,
> + meta_buf,
> + KMB_METADATA_TABLE_LCA,
> + user_params->lca.coeff,
> + ARRAY_SIZE(user_params->lca.coeff));
> + }
> +
> + params->lca.addr = kmb_metadata_get_table_addr(meta_buf,
> + KMB_METADATA_TABLE_LCA);
> + if (!params->lca.addr)
> + ret = -EINVAL;
> +
> + return ret;
> +}
> +
> +static int
> +kmb_metadata_fill_sharpen(struct kmb_metadata *kmb_meta,
> + struct kmb_metadata_buf *meta_buf,
> + struct kmb_isp_params *user_params)
> +{
> + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
> + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> + struct kmb_vpu_isp_params *last_params = NULL;
> + int ret = 0;
> +
> + if (last_buf)
> + last_params = last_buf->params.isp;
> +
> + if (user_params->update.sharpen) {
> + kmb_metadata_copy_sharpen(¶ms->sharpen,
> + &user_params->sharpen);
> + ret = kmb_metadata_copy_table_usr(kmb_meta,
> + meta_buf,
> + KMB_METADATA_TABLE_SHARP,
> + user_params->sharpen.radial_lut,
> + ARRAY_SIZE(user_params->sharpen.radial_lut));
> + if (ret < 0)
> + return ret;
> +
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(¶ms->sharpen, &last_params->sharpen,
> + sizeof(params->sharpen));
> +
> + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
> + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> + KMB_METADATA_TABLE_SHARP);
> + } else {
> + memcpy(¶ms->sharpen, def_params->sharpen,
> + sizeof(params->sharpen));
> +
> + kmb_metadata_create_default_table(kmb_meta,
> + meta_buf,
> + KMB_METADATA_TABLE_SHARP,
> + user_params->sharpen.radial_lut,
> + ARRAY_SIZE(user_params->sharpen.radial_lut));
> + }
> +
> + params->sharpen.addr =
> + kmb_metadata_get_table_addr(meta_buf,
> + KMB_METADATA_TABLE_SHARP);
> + if (!params->sharpen.addr)
> + ret = -EINVAL;
> +
> + return ret;
> +}
> +
> +static int
> +kmb_metadata_fill_color_comb(struct kmb_metadata *kmb_meta,
> + struct kmb_metadata_buf *meta_buf,
> + struct kmb_isp_params *user_params)
> +{
> + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
> + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> + struct kmb_vpu_isp_params *last_params = NULL;
> + struct kmb_color_comb_params *col = NULL;
> + int ret = 0;
> +
> + if (last_buf)
> + last_params = last_buf->params.isp;
> +
> + if (user_params->update.color_comb) {
> + col = &user_params->color_comb;
> + kmb_metadata_copy_color_comb(¶ms->color_comb,
> + &user_params->color_comb);
> + if (params->color_comb.enable) {
> + ret = kmb_metadata_copy_table_usr(kmb_meta,
> + meta_buf,
> + KMB_METADATA_TABLE_COLOR_CUMB,
> + col->lut_3d,
> + ARRAY_SIZE(col->lut_3d));
> + if (ret < 0)
> + return ret;
> + }
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(¶ms->color_comb, &last_params->color_comb,
> + sizeof(params->color_comb));
> +
> + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
> + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> + KMB_METADATA_TABLE_COLOR_CUMB);
> + } else {
> + memcpy(¶ms->color_comb, def_params->color_comb,
> + sizeof(params->color_comb));
> + }
> +
> + if (params->color_comb.enable) {
> + params->color_comb.addr =
> + kmb_metadata_get_table_addr(meta_buf,
> + KMB_METADATA_TABLE_COLOR_CUMB);
> + if (!params->color_comb.addr)
> + ret = -EINVAL;
> + }
> +
> + return ret;
> +}
> +
> +static int
> +kmb_metadata_fill_hdr(struct kmb_metadata *kmb_meta,
> + struct kmb_metadata_buf *meta_buf,
> + struct kmb_isp_params *user_params)
> +{
> + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
> + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> + struct kmb_vpu_isp_params *last_params = NULL;
> + int ret = 0;
> +
> + if (last_buf)
> + last_params = last_buf->params.isp;
> +
> + if (user_params->update.hdr) {
> + kmb_metadata_copy_hdr(¶ms->hdr,
> + &user_params->hdr);
> + if (params->hdr.enable1 || params->hdr.enable2) {
> + ret = kmb_metadata_copy_table_usr(kmb_meta,
> + meta_buf,
> + KMB_METADATA_TABLE_HDR,
> + user_params->hdr.tm_lut,
> + ARRAY_SIZE(user_params->hdr.tm_lut));
> + if (ret < 0)
> + return ret;
> + }
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(¶ms->hdr, &last_params->hdr,
> + sizeof(params->hdr));
> +
> + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
> + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> + KMB_METADATA_TABLE_HDR);
> + } else {
> + memcpy(¶ms->hdr, def_params->hdr, sizeof(params->hdr));
> + }
> +
> + if (params->hdr.enable1 || params->hdr.enable2) {
> + params->hdr.luts_addr =
> + kmb_metadata_get_table_addr(meta_buf,
> + KMB_METADATA_TABLE_HDR);
> + if (!params->hdr.luts_addr)
> + ret = -EINVAL;
> + }
> +
> + return ret;
> +}
> +
> +static int
> +kmb_metadata_fill_lut(struct kmb_metadata *kmb_meta,
> + struct kmb_metadata_buf *meta_buf,
> + struct kmb_isp_params *user_params)
> +{
> + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
> + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> + struct kmb_vpu_isp_params *last_params = NULL;
> + int ret = 0;
> +
> + if (last_buf)
> + last_params = last_buf->params.isp;
> +
> + if (user_params->update.lut) {
> + kmb_metadata_copy_lut(¶ms->lut, &user_params->lut);
> + if (params->lut.size) {
> + ret = kmb_metadata_copy_table_usr(kmb_meta,
> + meta_buf,
> + KMB_METADATA_TABLE_LUT,
> + user_params->lut.table,
> + ARRAY_SIZE(user_params->lut.table));
> + if (ret < 0)
> + return ret;
> + }
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(¶ms->lut, &last_params->lut,
> + sizeof(params->lut));
> +
> + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
> + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> + KMB_METADATA_TABLE_LUT);
> + } else {
> + memcpy(¶ms->lut, def_params->lut, sizeof(params->lut));
> + kmb_metadata_create_default_table(kmb_meta,
> + meta_buf,
> + KMB_METADATA_TABLE_LUT,
> + user_params->lut.table,
> + ARRAY_SIZE(user_params->lut.table));
> + }
> +
> + if (params->lut.size) {
> + params->lut.addr =
> + kmb_metadata_get_table_addr(meta_buf,
> + KMB_METADATA_TABLE_LUT);
> + if (!params->lut.size)
> + ret = -EINVAL;
> + }
> +
> + return ret;
> +}
> +
> +static int
> +kmb_metadata_fill_warp(struct kmb_metadata *kmb_meta,
> + struct kmb_metadata_buf *meta_buf,
> + struct kmb_isp_params *user_params)
> +{
> + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
> + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> + struct kmb_vpu_isp_params *last_params = NULL;
> + int ret = 0;
> +
> + if (last_buf)
> + last_params = last_buf->params.isp;
> +
> + if (user_params->update.warp) {
> + kmb_metadata_copy_warp(¶ms->warp, &user_params->warp);
> + if (params->warp.enable) {
> + ret = kmb_metadata_copy_table_usr(kmb_meta,
> + meta_buf,
> + KMB_METADATA_TABLE_WARP,
> + user_params->warp.mesh_grid,
> + ARRAY_SIZE(user_params->warp.mesh_grid));
> + if (ret < 0)
> + return ret;
> + }
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(¶ms->warp, &last_params->warp,
> + sizeof(params->warp));
> +
> + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
> + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> + KMB_METADATA_TABLE_WARP);
> + } else {
> + memcpy(¶ms->warp, def_params->warp, sizeof(params->warp));
> + }
> +
> + if (params->warp.enable) {
> + params->warp.addr =
> + kmb_metadata_get_table_addr(meta_buf,
> + KMB_METADATA_TABLE_WARP);
> + if (!params->warp.addr)
> + ret = -EINVAL;
> + }
> +
> + return ret;
> +}
> +
> +static int
> +kmb_metadata_fill_tnf(struct kmb_metadata *kmb_meta,
> + struct kmb_metadata_buf *meta_buf,
> + struct kmb_isp_params *user_params)
> +{
> + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
> + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> + struct kmb_vpu_isp_params *last_params = NULL;
> + struct kmb_tnf_params *tnf = NULL;
> + int ret = 0;
> +
> + if (last_buf)
> + last_params = last_buf->params.isp;
> +
> + if (user_params->update.tnf) {
> + kmb_metadata_copy_tnf(¶ms->tnf, &user_params->tnf);
> + if (params->tnf.enable) {
> + tnf = &user_params->tnf;
> + ret = kmb_metadata_copy_table_usr(kmb_meta,
> + meta_buf,
> + KMB_METADATA_TABLE_TNF0,
> + tnf->chroma_lut0,
> + ARRAY_SIZE(tnf->chroma_lut0));
> + if (ret < 0)
> + return ret;
> +
> + ret = kmb_metadata_copy_table_usr(kmb_meta,
> + meta_buf,
> + KMB_METADATA_TABLE_TNF1,
> + tnf->chroma_lut1,
> + ARRAY_SIZE(tnf->chroma_lut1));
> + if (ret < 0)
> + return ret;
> + }
> + } else if (last_params) {
> + if (last_params != params)
> + memcpy(¶ms->tnf, &last_params->tnf,
> + sizeof(params->tnf));
> +
> + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf) {
> + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> + KMB_METADATA_TABLE_TNF0);
> + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> + KMB_METADATA_TABLE_TNF1);
> + }
> + } else {
> + memcpy(¶ms->tnf, def_params->tnf, sizeof(params->tnf));
> + }
> +
> + if (params->tnf.enable) {
> + params->tnf.lut0_addr =
> + kmb_metadata_get_table_addr(meta_buf,
> + KMB_METADATA_TABLE_TNF0);
> + if (!params->tnf.lut0_addr)
> + return -EINVAL;
> +
> + params->tnf.lut1_addr =
> + kmb_metadata_get_table_addr(meta_buf,
> + KMB_METADATA_TABLE_TNF1);
> + if (!params->tnf.lut1_addr)
> + return -EINVAL;
> + }
> +
> + return ret;
This is a successful case, so you could return 0 and omit assigning ret in
declaration.
You could do the same in the functions above this one, too (and change one
ret assignment to return).
> +}
> +
> +/* Fill static functions for conversions here */
> +static int kmb_metadata_fill_isp_params(struct kmb_metadata *kmb_meta,
> + struct kmb_metadata_buf *meta_buf,
> + struct kmb_isp_params *user_params)
> +{
> + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> + struct kmb_vpu_isp_params *last_params = NULL;
> + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
> + int ret;
> +
> + if (last_buf)
> + last_params = last_buf->params.isp;
> +
> + kmb_metadata_fill_blc(params, user_params, last_params, def_params);
> +
> + kmb_metadata_fill_signma_dns(params, user_params, last_params,
> + def_params);
> +
> + kmb_metadata_fill_ae_awb(params, user_params, last_params, def_params);
> +
> + kmb_metadata_fill_af(params, user_params, last_params, def_params);
> +
> + kmb_metadata_fill_histogram(params, user_params, last_params,
> + def_params);
> +
> + kmb_metadata_fill_debayer(params, user_params, last_params,
> + def_params);
> +
> + kmb_metadata_fill_dog_dns(params, user_params, last_params,
> + def_params);
> +
> + kmb_metadata_fill_luma_dns(params, user_params, last_params,
> + def_params);
> +
> + kmb_metadata_fill_chroma_gen(params, user_params, last_params,
> + def_params);
> +
> + kmb_metadata_fill_median(params, user_params, last_params, def_params);
> +
> + kmb_metadata_fill_chroma_dns(params, user_params, last_params,
> + def_params);
> +
> + kmb_metadata_fill_dehaze(params, user_params, last_params, def_params);
> +
> + /* Copy params with tables */
> + ret = kmb_metadata_fill_lsc(kmb_meta, meta_buf, user_params);
> + if (ret < 0)
> + goto error_release_tables;
> +
> + ret = kmb_metadata_fill_raw(kmb_meta, meta_buf, user_params);
> + if (ret < 0)
> + goto error_release_tables;
> +
> + ret = kmb_metadata_fill_lca(kmb_meta, meta_buf, user_params);
> + if (ret < 0)
> + goto error_release_tables;
> +
> + ret = kmb_metadata_fill_sharpen(kmb_meta, meta_buf, user_params);
> + if (ret < 0)
> + goto error_release_tables;
> +
> + ret = kmb_metadata_fill_color_comb(kmb_meta, meta_buf, user_params);
> + if (ret < 0)
> + goto error_release_tables;
> +
> + ret = kmb_metadata_fill_hdr(kmb_meta, meta_buf, user_params);
> + if (ret < 0)
> + goto error_release_tables;
> +
> + ret = kmb_metadata_fill_lut(kmb_meta, meta_buf, user_params);
> + if (ret < 0)
> + goto error_release_tables;
> +
> + ret = kmb_metadata_fill_warp(kmb_meta, meta_buf, user_params);
> + if (ret < 0)
> + goto error_release_tables;
> +
> + ret = kmb_metadata_fill_tnf(kmb_meta, meta_buf, user_params);
> + if (ret < 0)
> + goto error_release_tables;
> +
> + /* Store last buffer */
> + kmb_meta->last_buf = meta_buf;
> +
> + return 0;
> +
> +error_release_tables:
> + kmb_metadata_release_tables(meta_buf);
> + return ret;
> +}
> +
> +static int kmb_metadata_queue_setup(struct vb2_queue *q,
> + unsigned int *num_buffers,
> + unsigned int *num_planes,
> + unsigned int sizes[],
> + struct device *alloc_devs[])
> +{
> + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(q);
> +
> + *num_planes = 1;
> + sizes[0] = kmb_meta->format.buffersize;
> +
> + return 0;
> +}
> +
> +#define to_kmb_meta_buf(vbuf) container_of(vbuf, struct kmb_metadata_buf, vb)
> +
> +static int kmb_metadata_buf_params_init(struct vb2_buffer *vb)
> +{
> + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue);
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf);
> +
> + buf->type = KMB_METADATA_PARAMS;
> + buf->params.isp = dma_alloc_coherent(kmb_meta->dma_dev,
> + sizeof(*buf->params.isp),
> + &buf->params.dma_addr_isp, 0);
> + if (!buf->params.isp)
> + return -ENOMEM;
> +
> + memset(buf->params.isp, 0, sizeof(*buf->params.isp));
> + /*
> + * Table pools will be allocated per need.
> + * The pools need to be released when last buffer is finished.
> + * Use table reference count for that purpose
> + */
> + kmb_meta->table_pools_refcnt++;
> +
> + return 0;
> +}
> +
> +static int kmb_metadata_buf_params_prepare(struct vb2_buffer *vb)
> +{
> + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue);
> + struct kmb_isp_params *user_params = vb2_plane_vaddr(vb, 0);
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf);
> +
> + vb2_set_plane_payload(vb, 0, kmb_meta->format.buffersize);
> + return kmb_metadata_fill_isp_params(kmb_meta, buf, user_params);
> +}
> +
> +static void kmb_metadata_buf_params_cleanup(struct vb2_buffer *vb)
> +{
> + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue);
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf);
> +
> + if (buf == kmb_meta->last_buf)
> + kmb_meta->last_buf = NULL;
> +
> + kmb_metadata_release_tables(buf);
> + dma_free_coherent(kmb_meta->dma_dev, sizeof(*buf->params.isp),
> + buf->params.isp, buf->params.dma_addr_isp);
> +
> + /* Destroy allocated table pools on last finish */
> + if (kmb_meta->table_pools_refcnt-- == 1)
> + kmb_metadata_destroy_table_pools(kmb_meta);
> +}
> +
> +static int kmb_metadata_buf_stats_init(struct vb2_buffer *vb)
> +{
> + dma_addr_t stats_addr = vb2_dma_contig_plane_dma_addr(vb, 0);
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf);
> + int i;
> +
> + buf->type = KMB_METADATA_STATS;
> + memset(&buf->stats.raw, 0, sizeof(buf->stats.raw));
> + buf->stats.dehaze_stats_addr = 0;
> +
> + /* Fill statistics addresses */
> + for (i = 0; i < KMB_CAM_MAX_EXPOSURES; i++) {
> + buf->stats.raw[i].ae_awb_stats_addr = stats_addr +
> + offsetof(struct kmb_isp_stats,
> + exposure[i].ae_awb_stats[0]);
> +
> + buf->stats.raw[i].af_stats_addr = stats_addr +
> + offsetof(struct kmb_isp_stats,
> + exposure[i].af_stats[0]);
> +
> + buf->stats.raw[i].hist_luma_addr = stats_addr +
> + offsetof(struct kmb_isp_stats,
> + exposure[i].hist_luma[0]);
> +
> + buf->stats.raw[i].hist_rgb_addr = stats_addr +
> + offsetof(struct kmb_isp_stats,
> + exposure[i].hist_rgb[0]);
> +
> + buf->stats.raw[i].flicker_rows_addr = stats_addr +
> + offsetof(struct kmb_isp_stats,
> + exposure[i].flicker_rows[0]);
> + }
> +
> + buf->stats.dehaze_stats_addr = stats_addr +
> + offsetof(struct kmb_isp_stats, dehaze);
> +
> + return 0;
> +}
> +
> +static int kmb_metadata_buf_stats_prepare(struct vb2_buffer *vb)
> +{
> + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue);
> +
> + vb2_set_plane_payload(vb, 0, kmb_meta->format.buffersize);
> +
> + return 0;
> +}
> +
> +static void kmb_metadata_buf_queue(struct vb2_buffer *vb)
> +{
> + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue);
> + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> + struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf);
> + int ret;
> +
> + ret = kmb_meta->queue_ops->queue(kmb_meta->priv, buf);
> + if (ret)
> + dev_err(&kmb_meta->video.dev, "Fail metadata queue %d", ret);
> +}
> +
> +static int kmb_metadata_start_streaming(struct vb2_queue *q,
> + unsigned int count)
> +{
> + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(q);
> + int ret;
> +
> + ret = kmb_pipe_prepare(kmb_meta->pipe);
> + if (ret < 0)
> + goto error_discard_all_bufs;
> +
> + ret = kmb_pipe_run(kmb_meta->pipe, &kmb_meta->video.entity);
> + if (ret < 0)
> + goto error_pipeline_stop;
> +
> + return 0;
> +
> +error_pipeline_stop:
> + kmb_pipe_stop(kmb_meta->pipe, &kmb_meta->video.entity);
> +error_discard_all_bufs:
> + kmb_meta->queue_ops->flush(kmb_meta->priv);
> + return 0;
> +}
> +
> +static void kmb_metadata_stop_streaming(struct vb2_queue *q)
> +{
> + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(q);
> +
> + kmb_pipe_stop(kmb_meta->pipe, &kmb_meta->video.entity);
> +
> + kmb_meta->queue_ops->flush(kmb_meta->priv);
> +}
> +
> +/* driver-specific operations */
> +static struct vb2_ops kmb_meta_params_vb2_q_ops = {
> + .queue_setup = kmb_metadata_queue_setup,
> + .buf_init = kmb_metadata_buf_params_init,
> + .buf_prepare = kmb_metadata_buf_params_prepare,
> + .buf_cleanup = kmb_metadata_buf_params_cleanup,
> + .start_streaming = kmb_metadata_start_streaming,
> + .stop_streaming = kmb_metadata_stop_streaming,
> + .buf_queue = kmb_metadata_buf_queue,
> +};
> +
> +static struct vb2_ops kmb_meta_stats_vb2_q_ops = {
> + .queue_setup = kmb_metadata_queue_setup,
> + .buf_init = kmb_metadata_buf_stats_init,
> + .buf_prepare = kmb_metadata_buf_stats_prepare,
> + .start_streaming = kmb_metadata_start_streaming,
> + .stop_streaming = kmb_metadata_stop_streaming,
> + .buf_queue = kmb_metadata_buf_queue,
> +};
> +
> +#define to_kmb_meta_dev(vdev) container_of(vdev, struct kmb_metadata, video)
> +
> +static int kmb_metadata_querycap(struct file *file, void *fh,
> + struct v4l2_capability *cap)
> +{
> + struct v4l2_fh *vfh = file->private_data;
> + struct kmb_metadata *kmb_meta =
> + to_kmb_meta_dev(vfh->vdev);
> +
> + cap->bus_info[0] = 0;
> + strscpy(cap->driver, kmb_meta->video.name, sizeof(cap->driver));
> + snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> + kmb_meta->video.name);
> +
> + return 0;
> +}
> +
> +static int kmb_metadata_get_fmt(struct file *file, void *fh,
> + struct v4l2_format *f)
> +{
> + struct v4l2_fh *vfh = file->private_data;
> + struct kmb_metadata *kmb_meta =
> + to_kmb_meta_dev(vfh->vdev);
> +
> + f->fmt.meta = kmb_meta->format;
> +
> + return 0;
> +}
> +
> +static int kmb_metadata_try_fmt_cap(struct file *file, void *fh,
> + struct v4l2_format *f)
> +{
> + f->fmt.meta.dataformat = V4L2_META_FMT_KMB_STATS;
> + if (f->fmt.meta.buffersize < sizeof(struct kmb_isp_stats))
> + f->fmt.meta.buffersize = sizeof(struct kmb_isp_stats);
> +
> + return 0;
> +}
> +
> +static int kmb_metadata_set_fmt_cap(struct file *file, void *fh,
> + struct v4l2_format *f)
> +{
> + struct v4l2_fh *vfh = file->private_data;
> + struct kmb_metadata *kmb_meta =
> + to_kmb_meta_dev(vfh->vdev);
> + int ret;
> +
> + ret = kmb_metadata_try_fmt_cap(file, fh, f);
> + if (ret < 0)
> + return ret;
> +
> + kmb_meta->format = f->fmt.meta;
> +
> + return 0;
> +}
> +
> +static int kmb_metadata_try_fmt_out(struct file *file, void *fh,
> + struct v4l2_format *f)
> +{
> + f->fmt.meta.dataformat = V4L2_META_FMT_KMB_PARAMS;
> + if (f->fmt.meta.buffersize < sizeof(struct kmb_isp_params))
> + f->fmt.meta.buffersize = sizeof(struct kmb_isp_params);
> +
> + return 0;
> +}
> +
> +static int kmb_metadata_set_fmt_out(struct file *file, void *fh,
> + struct v4l2_format *f)
> +{
> + struct v4l2_fh *vfh = file->private_data;
> + struct kmb_metadata *kmb_meta =
> + to_kmb_meta_dev(vfh->vdev);
> + int ret;
> +
> + ret = kmb_metadata_try_fmt_out(file, fh, f);
> + if (ret < 0)
> + return ret;
> +
> + kmb_meta->format = f->fmt.meta;
> +
> + return 0;
> +}
> +
> +/* V4L2 ioctl operations */
> +static const struct v4l2_ioctl_ops kmb_vid_ioctl_ops = {
> + .vidioc_querycap = kmb_metadata_querycap,
> + .vidioc_g_fmt_meta_out = kmb_metadata_get_fmt,
> + .vidioc_s_fmt_meta_out = kmb_metadata_set_fmt_out,
> + .vidioc_try_fmt_meta_out = kmb_metadata_try_fmt_out,
> + .vidioc_g_fmt_meta_cap = kmb_metadata_get_fmt,
> + .vidioc_s_fmt_meta_cap = kmb_metadata_set_fmt_cap,
> + .vidioc_try_fmt_meta_cap = kmb_metadata_try_fmt_cap,
> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> + .vidioc_querybuf = vb2_ioctl_querybuf,
> + .vidioc_qbuf = vb2_ioctl_qbuf,
> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> + .vidioc_streamon = vb2_ioctl_streamon,
> + .vidioc_streamoff = vb2_ioctl_streamoff,
> +};
> +
> +static int kmb_metadata_open(struct file *file)
> +{
> + struct kmb_metadata *kmb_meta = video_drvdata(file);
> + int ret;
> +
> + mutex_lock(&kmb_meta->lock);
> +
> + ret = v4l2_fh_open(file);
> + if (ret) {
> + mutex_unlock(&kmb_meta->lock);
> + return ret;
> + }
> +
> + ret = kmb_pipe_request(kmb_meta->pipe);
> + if (ret < 0)
> + goto error_fh_release;
> +
> + mutex_unlock(&kmb_meta->lock);
> +
> + return 0;
> +
> +error_fh_release:
> + _vb2_fop_release(file, NULL);
> + mutex_unlock(&kmb_meta->lock);
> + return ret;
> +}
> +
> +static int kmb_metadata_release(struct file *file)
> +{
> + struct kmb_metadata *kmb_meta = video_drvdata(file);
> + int ret;
> +
> + mutex_lock(&kmb_meta->lock);
> +
> + kmb_pipe_release(kmb_meta->pipe);
> +
> + ret = _vb2_fop_release(file, NULL);
> +
> + mutex_unlock(&kmb_meta->lock);
> +
> + return ret;
> +}
> +
> +/* V4L2 file operations */
> +static const struct v4l2_file_operations kmb_vid_output_fops = {
> + .owner = THIS_MODULE,
> + .unlocked_ioctl = video_ioctl2,
> + .open = kmb_metadata_open,
> + .release = kmb_metadata_release,
> + .poll = vb2_fop_poll,
> + .mmap = vb2_fop_mmap,
> +};
> +
> /**
> - * kmb_video_init - Initialize entity
> + * kmb_metadata_init - Initialize entity
> * @kmb_meta: pointer to kmb isp config device
> *
> * Return: 0 if successful, error code otherwise.
> */
> int kmb_metadata_init(struct kmb_metadata *kmb_meta)
> {
> + int ret;
> +
> + mutex_init(&kmb_meta->lock);
> +
> + kmb_meta->table_pools_refcnt = 0;
> + memset(kmb_meta->table_pool, 0, sizeof(kmb_meta->table_pool));
> +
> + kmb_meta->video.fops = &kmb_vid_output_fops;
> + kmb_meta->video.ioctl_ops = &kmb_vid_ioctl_ops;
> + kmb_meta->video.minor = -1;
> + kmb_meta->video.release = video_device_release;
> + kmb_meta->video.vfl_type = VFL_TYPE_VIDEO;
> + kmb_meta->video.lock = &kmb_meta->lock;
> + kmb_meta->video.queue = &kmb_meta->vb2_q;
> + video_set_drvdata(&kmb_meta->video, kmb_meta);
> +
> + kmb_meta->vb2_q.drv_priv = kmb_meta;
> + kmb_meta->vb2_q.buf_struct_size = sizeof(struct kmb_metadata_buf);
> + kmb_meta->vb2_q.io_modes = VB2_DMABUF | VB2_MMAP;
> + kmb_meta->vb2_q.timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> + kmb_meta->vb2_q.dev = kmb_meta->dma_dev;
> + kmb_meta->vb2_q.lock = &kmb_meta->lock;
> + kmb_meta->vb2_q.min_buffers_needed = 1;
> +
> + /* Initialize per type variables */
> + kmb_meta->video.device_caps = V4L2_CAP_STREAMING;
> + if (kmb_meta->type == KMB_METADATA_PARAMS) {
> + kmb_meta->video.device_caps |= V4L2_CAP_META_OUTPUT;
> + kmb_meta->video.vfl_dir = VFL_DIR_TX;
> + snprintf(kmb_meta->video.name, sizeof(kmb_meta->video.name),
> + KMB_CAM_METADATA_PARAMS_NAME);
> +
> + kmb_meta->vb2_q.ops = &kmb_meta_params_vb2_q_ops;
> + kmb_meta->vb2_q.mem_ops = &vb2_dma_contig_memops;
> + kmb_meta->vb2_q.type = V4L2_BUF_TYPE_META_OUTPUT;
> +
> + kmb_meta->pad.flags = MEDIA_PAD_FL_SOURCE;
> +
> + kmb_meta->format.dataformat = V4L2_META_FMT_KMB_PARAMS;
> + kmb_meta->format.buffersize = sizeof(struct kmb_isp_params);
> + } else {
> + kmb_meta->video.device_caps |= V4L2_CAP_META_CAPTURE;
> + kmb_meta->video.vfl_dir = VFL_DIR_RX;
> +
> + snprintf(kmb_meta->video.name, sizeof(kmb_meta->video.name),
> + KMB_CAM_METADATA_STATS_NAME);
> +
> + kmb_meta->vb2_q.ops = &kmb_meta_stats_vb2_q_ops;
> + kmb_meta->vb2_q.mem_ops = &vb2_dma_contig_memops;
> + kmb_meta->vb2_q.type = V4L2_BUF_TYPE_META_CAPTURE;
> +
> + kmb_meta->pad.flags = MEDIA_PAD_FL_SINK;
> +
> + kmb_meta->format.dataformat = V4L2_META_FMT_KMB_STATS;
> + kmb_meta->format.buffersize = sizeof(struct kmb_isp_stats);
> + }
> +
> + ret = media_entity_pads_init(&kmb_meta->video.entity,
> + 1, &kmb_meta->pad);
> + if (ret < 0)
> + goto error_mutex_destroy;
> +
> + ret = vb2_queue_init(&kmb_meta->vb2_q);
> + if (ret < 0) {
> + dev_err(&kmb_meta->video.dev, "Error vb2 queue init");
> + goto error_metadata_cleanup;
> + }
> +
> + kmb_params_get_defaults(&kmb_meta->def);
> +
> return 0;
> +
> +error_metadata_cleanup:
> + kmb_metadata_cleanup(kmb_meta);
> +error_mutex_destroy:
> + mutex_destroy(&kmb_meta->lock);
> +
> + return ret;
> }
>
> /**
> @@ -22,7 +1823,10 @@ int kmb_metadata_init(struct kmb_metadata *kmb_meta)
> * @kmb_meta: pointer to kmb isp config device
> */
> void kmb_metadata_cleanup(struct kmb_metadata *kmb_meta)
> -{ }
> +{
> + media_entity_cleanup(&kmb_meta->video.entity);
> + mutex_destroy(&kmb_meta->lock);
> +}
>
> /**
> * kmb_metadata_register - Register V4L2 device
> @@ -34,7 +1838,15 @@ void kmb_metadata_cleanup(struct kmb_metadata *kmb_meta)
> int kmb_metadata_register(struct kmb_metadata *kmb_meta,
> struct v4l2_device *v4l2_dev)
> {
> - return 0;
> + int ret;
> +
> + kmb_meta->video.v4l2_dev = v4l2_dev;
> +
> + ret = video_register_device(&kmb_meta->video, VFL_TYPE_VIDEO, -1);
> + if (ret < 0)
> + dev_err(&kmb_meta->video.dev, "Failed to register video device");
> +
> + return ret;
> }
>
> /**
> @@ -42,4 +1854,7 @@ int kmb_metadata_register(struct kmb_metadata *kmb_meta,
> * @kmb_meta: pointer to kmb isp config device
> */
> void kmb_metadata_unregister(struct kmb_metadata *kmb_meta)
> -{ }
> +{
> + mutex_destroy(&kmb_meta->lock);
> + video_unregister_device(&kmb_meta->video);
> +}
> diff --git a/drivers/media/platform/keembay-camera/keembay-metadata.h b/drivers/media/platform/keembay-camera/keembay-metadata.h
> index 88e85d3caba0..ab77ed11bd15 100644
> --- a/drivers/media/platform/keembay-camera/keembay-metadata.h
> +++ b/drivers/media/platform/keembay-camera/keembay-metadata.h
> @@ -12,6 +12,7 @@
> #include <media/videobuf2-v4l2.h>
>
> #include "keembay-vpu-isp.h"
> +#include "keembay-params-defaults.h"
>
> /**
> * enum kmb_metadata_table_type - Keembay metadata table type
> @@ -68,12 +69,12 @@ struct kmb_metadata_table {
> * @vb: Video buffer for v4l2
> * @type: Metadata type
> * @stats: Statistics physical addresses
> - * @raw: VPU raw statistics physical addresses
> - * @dehaze_stats_addr: VPU dehaze statistics physical address
> + * @stats.raw: VPU raw statistics physical addresses
> + * @stats.dehaze_stats_addr: VPU dehaze statistics physical address
> * @params: VPU ISP parameters
> - * @isp: VPU ISP parameters virtual address
> - * @dma_addr_isp: VPU ISP parameters physical address
> - * @tab: Metadata tables
> + * @params.isp: VPU ISP parameters virtual address
> + * @params.dma_addr_isp: VPU ISP parameters physical address
> + * @params.tab: Metadata tables
> * @list: List for buffer queue
> */
> struct kmb_metadata_buf {
> @@ -118,6 +119,7 @@ struct kmb_metabuf_queue_ops {
> * @table_pool: ISP tables dma pool
> * @last_buf: Pointer to last enqueued buffer
> * @format: Active format
> + * @def: Default ISP params
> */
> struct kmb_metadata {
> struct mutex lock;
> @@ -138,6 +140,8 @@ struct kmb_metadata {
> struct kmb_metadata_buf *last_buf;
>
> struct v4l2_meta_format format;
> +
> + struct kmb_vpu_isp_params_defaults def;
> };
>
> int kmb_metadata_init(struct kmb_metadata *kmb_meta);
> diff --git a/drivers/media/platform/keembay-camera/keembay-params-defaults.c b/drivers/media/platform/keembay-camera/keembay-params-defaults.c
> new file mode 100644
> index 000000000000..a2dd7888375e
> --- /dev/null
> +++ b/drivers/media/platform/keembay-camera/keembay-params-defaults.c
> @@ -0,0 +1,326 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * Intel Keem Bay camera ISP parameter defaults.
> + *
> + * Copyright (C) 2021 Intel Corporation
> + */
> +#include <linux/stddef.h>
> +#include <linux/types.h>
> +
> +#include "keembay-params-defaults.h"
> +
> +static const struct kmb_vpu_blc_params blc_default[KMB_VPU_MAX_EXPOSURES] = {
> + {
> + .coeff1 = 800,
> + .coeff2 = 800,
> + .coeff3 = 800,
> + .coeff4 = 800,
> + },
> + {
> + .coeff1 = 800,
> + .coeff2 = 800,
> + .coeff3 = 800,
> + .coeff4 = 800,
> + },
> + {
> + .coeff1 = 800,
> + .coeff2 = 800,
> + .coeff3 = 800,
> + .coeff4 = 800,
> + }
> +
> +};
> +
> +static const struct kmb_vpu_sigma_dns_params
> + sigma_dns_default[KMB_VPU_MAX_EXPOSURES] = { 0 };
> +
> +static const struct kmb_vpu_lsc_params lsc_default = {
> + .threshold = 2048,
> + .width = 64,
> + .height = 44,
> + .reserved = { 0 },
> +};
> +
> +static const struct kmb_vpu_raw_params raw_default = {
> + .awb_stats_en = 0,
> + .awb_rgb_hist_en = 0,
> + .af_stats_en = 0,
> + .luma_hist_en = 0,
> + .flicker_accum_en = 0,
> + .bad_pixel_fix_en = 0,
> + .grgb_imb_en = 1,
> + .mono_imbalance_en = 0,
> + .gain1 = 269,
> + .gain2 = 452,
> + .gain3 = 634,
> + .gain4 = 269,
> + .stop1 = 400,
> + .stop2 = 450,
> + .stop3 = 700,
> + .stop4 = 800,
> + .threshold1 = 128,
> + .alpha1 = 12,
> + .alpha2 = 12,
> + .alpha3 = 12,
> + .alpha4 = 12,
> + .threshold2 = 53,
> + .static_defect_size = 1,
> + .reserved = { 0 },
> + .flicker_first_row_acc = 0,
> + .flicker_last_row_acc = 0,
> +};
> +
> +static const struct kmb_vpu_ae_awb_params ae_awb_default = {
> + .start_x = 0,
> + .start_y = 0,
> + .width = 100,
> + .height = 98,
> + .skip_x = 100,
> + .skip_y = 98,
> + .patches_x = 38,
> + .patches_y = 22,
> + .threshold1 = 0,
> + .threshold2 = 4095,
> +};
> +
> +static const struct kmb_vpu_af_params af_default = {
> + .start_x = 0,
> + .start_y = 0,
> + .width = 192,
> + .height = 144,
> + .patches_x = 20,
> + .patches_y = 15,
> + .coeff = 0,
> + .threshold1 = 0,
> + .threshold2 = 0,
> + .coeffs1 = {31, 19, -32, 31, 63, 31, -50, -35, 35, -70, 35},
> + .coeffs2 = {35, 11, -29, 8, 17, 8, 78, -39, 119, -238, 119},
> +};
> +
> +static const struct kmb_vpu_hist_params histogram_default = {
> + .start_x = 0,
> + .start_y = 0,
> + .end_x = 3839,
> + .end_y = 2156,
> + .matrix = {1719, 0, 0, 0, 1024, 0, 0, 0, 2414},
> + .weight = {64, 128, 64},
> +};
> +
> +// only address - nothing to init...
/* a comment */
> +static const struct kmb_vpu_lca_params lca_default = { 0 };
> +
> +static const struct kmb_vpu_debayer_params debayer_default = {
> + .coeff1 = 51,
> + .multiplier1 = 13107,
> + .multiplier2 = 13107,
> + .coeff2 = 77,
> + .coeff3 = 150,
> + .coeff4 = 29,
> +};
> +
> +static const struct kmb_vpu_dog_dns_params dog_dns_default = {
> + .threshold = 0,
> + .strength = 0,
> + .coeffs11 = {0, 0, 0, 0, 0, 255},
> + .coeffs15 = {0, 0, 0, 0, 0, 0, 0, 255},
> + .reserved = { 0 },
> +};
> +
> +static const struct kmb_vpu_luma_dns_params luma_dns_default = {
> + .threshold = 13094,
> + .slope = 967,
> + .shift = 7,
> + .alpha = 50,
> + .weight = 0,
> + .per_pixel_alpha_en = 0,
> + .gain_bypass_en = 0,
> + .reserved = { 0 },
> +};
> +
> +static const struct kmb_vpu_sharpen_params sharpen_default = {
> + .coeffs1 = {0, 0, 0, 4, 182, 396},
> + .coeffs2 = {0, 0, 0, 1, 141, 740},
> + .coeffs3 = {0, 0, 2, 42, 246, 444},
> + .shift = 15,
> + .gain1 = 3396,
> + .gain2 = 3378,
> + .gain3 = 3270,
> + .gain4 = 3400,
> + .gain5 = 207,
> + .stops1 = {20, 40, 605},
> + .gains = {10, 120, 60},
> + .stops2 = {11, 100, 2500, 4000},
> + .overshoot = 359,
> + .undershoot = 146,
> + .alpha = 36,
> + .gain6 = 128,
> + .offset = 637,
> +};
> +
> +static const struct kmb_vpu_chroma_gen_params chroma_gen_default = {
> + .epsilon = 2,
> + .coeff1 = 426,
> + .coeff2 = 767,
> + .coeff3 = 597,
> + .coeff4 = 77,
> + .coeff5 = 150,
> + .coeff6 = 29,
> + .strength1 = 0,
> + .strength2 = 32,
> + .coeffs = {33, 59, 71},
> + .offset1 = 2,
> + .slope1 = 230,
> + .slope2 = 256,
> + .offset2 = 0,
> + .limit = 767,
> +};
> +
> +static const struct kmb_vpu_median_params median_default = {
> + .size = 7,
> + .slope = 32,
> + .offset = -19,
> +};
> +
> +static const struct kmb_vpu_chroma_dns_params chroma_dns_default = {
> + .limit = 255,
> + .enable = 0,
> + .threshold1 = 30,
> + .threshold2 = 30,
> + .threshold3 = 30,
> + .threshold4 = 30,
> + .threshold5 = 45,
> + .threshold6 = 45,
> + .threshold7 = 45,
> + .threshold8 = 45,
> + .slope1 = 77,
> + .offset1 = -15,
> + .slope2 = 255,
> + .offset2 = 127,
> + .grey1 = 421,
> + .grey2 = 758,
> + .grey3 = 590,
> + .coeff1 = 52,
> + .coeff2 = 32,
> + .coeff3 = 19,
> +};
> +
> +static const struct kmb_vpu_color_comb_params color_comb_default = {
> + .matrix = {1303, 65427, 65367, 65172, 1463, 65462, 55, 65034, 1459},
> + .offsets = { 0 },
> + .coeff1 = 615,
> + .coeff2 = 342,
> + .coeff3 = 439,
> + .reserved = { 0 },
> + .enable = 0,
> + .weight1 = 85,
> + .weight2 = 86,
> + .weight3 = 85,
> + .limit1 = 512,
> + .limit2 = -8192,
> + .offset1 = 0,
> + .offset2 = 0,
> +};
> +
> +static const struct kmb_vpu_hdr_params hdr_default = {
> + .ratio = {256, 256},
> + .scale = {262143, 262143, 262143},
> + .offset1 = -3275,
> + .slope1 = 320,
> + .offset2 = -3685,
> + .slope2 = 641,
> + .offset3 = -4054,
> + .slope3 = 4095,
> + .offset4 = 3686,
> + .gain1 = 16,
> + .blur1 = {0, 0, 255},
> + .blur2 = {0, 0, 0, 0, 255},
> + .contrast1 = 20,
> + .contrast2 = 16,
> + .enable1 = 0,
> + .enable2 = 0,
> + .offset5 = 0,
> + .offset6 = 0,
> + .strength = 0,
> + .reserved1 = { 0 },
> + .offset7 = 15,
> + .shift = 1702133760,
> + .field1 = 16,
> + .field2 = 123,
> + .gain3 = 0,
> + .min = 0,
> + .reserved2 = { 0 },
> +};
> +
> +static const struct kmb_vpu_lut_params lut_default = {
> + .size = 512,
> + .reserved = { 0 },
> + .matrix = {262, 516, 100, 3945, 3799, 449, 449, 3720, 4023},
> + .offsets = {256, 2048, 2048},
> +};
> +
> +static const struct kmb_vpu_tnf_params tnf_default = {
> + .factor = 179,
> + .gain = 0,
> + .offset1 = 217,
> + .slope1 = 162,
> + .offset2 = 299,
> + .slope2 = 121,
> + .min1 = 0,
> + .min2 = 40,
> + .value = 128,
> + .enable = 0,
> +};
> +
> +static const struct kmb_vpu_dehaze_params dehaze_default = {
> + .gain1 = 512,
> + .min = 70,
> + .strength1 = 0,
> + .strength2 = 0,
> + .gain2 = 128,
> + .saturation = 127,
> + .value1 = 2048,
> + .value2 = 2048,
> + .value3 = 2048,
> + .filter = {0, 0, 255},
> +};
> +
> +static const struct kmb_vpu_warp_params warp_default = {
> + .type = 0,
> + .relative = 0,
> + .format = 0,
> + .position = 0,
> + .reserved = { 0 },
> + .width = 8,
> + .height = 4,
> + .stride = 128,
> + .enable = 0,
> + .matrix = {1, 0, 0, 0, 1, 0, 0, 0, 1},
> + .mode = 1,
> + .values = {0, 128, 128},
These seem to be default values for the parameters.
Are any of the values above dependent on the image sizes, for instance?
> +};
> +
> +void kmb_params_get_defaults(struct kmb_vpu_isp_params_defaults *defaults)
> +{
> + defaults->blc = blc_default;
> + defaults->sigma_dns = sigma_dns_default;
> + defaults->lsc = &lsc_default;
> + defaults->raw = &raw_default;
> + defaults->ae_awb = &ae_awb_default;
> + defaults->af = &af_default;
> + defaults->histogram = &histogram_default;
> + defaults->lca = &lca_default;
> + defaults->debayer = &debayer_default;
> + defaults->dog_dns = &dog_dns_default;
> + defaults->luma_dns = &luma_dns_default;
> + defaults->sharpen = &sharpen_default;
> + defaults->chroma_gen = &chroma_gen_default;
> + defaults->median = &median_default;
> + defaults->chroma_dns = &chroma_dns_default;
> + defaults->color_comb = &color_comb_default;
> + defaults->hdr = &hdr_default;
> + defaults->lut = &lut_default;
> + defaults->tnf = &tnf_default;
> + defaults->dehaze = &dehaze_default;
> + defaults->warp = &warp_default;
> +}
> +
> diff --git a/drivers/media/platform/keembay-camera/keembay-params-defaults.h b/drivers/media/platform/keembay-camera/keembay-params-defaults.h
> new file mode 100644
> index 000000000000..d6134d64be7c
> --- /dev/null
> +++ b/drivers/media/platform/keembay-camera/keembay-params-defaults.h
> @@ -0,0 +1,38 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Intel Keem Bay camera ISP parameter defaults.
> + *
> + * Copyright (C) 2021 Intel Corporation
> + */
> +#ifndef KEEMBAY_DEFAULTS_H
> +#define KEEMBAY_DEFAULTS_H
> +
> +#include "keembay-vpu-isp.h"
> +
> +struct kmb_vpu_isp_params_defaults {
> + const struct kmb_vpu_blc_params *blc;
> + const struct kmb_vpu_sigma_dns_params *sigma_dns;
> + const struct kmb_vpu_lsc_params *lsc;
> + const struct kmb_vpu_raw_params *raw;
> + const struct kmb_vpu_ae_awb_params *ae_awb;
> + const struct kmb_vpu_af_params *af;
> + const struct kmb_vpu_hist_params *histogram;
> + const struct kmb_vpu_lca_params *lca;
> + const struct kmb_vpu_debayer_params *debayer;
> + const struct kmb_vpu_dog_dns_params *dog_dns;
> + const struct kmb_vpu_luma_dns_params *luma_dns;
> + const struct kmb_vpu_sharpen_params *sharpen;
> + const struct kmb_vpu_chroma_gen_params *chroma_gen;
> + const struct kmb_vpu_median_params *median;
> + const struct kmb_vpu_chroma_dns_params *chroma_dns;
> + const struct kmb_vpu_color_comb_params *color_comb;
> + const struct kmb_vpu_hdr_params *hdr;
> + const struct kmb_vpu_lut_params *lut;
> + const struct kmb_vpu_tnf_params *tnf;
> + const struct kmb_vpu_dehaze_params *dehaze;
> + const struct kmb_vpu_warp_params *warp;
> +};
> +
> +void kmb_params_get_defaults(struct kmb_vpu_isp_params_defaults *defaults);
> +
> +#endif /* KEEMBAY_DEFAULTS_H */
--
Kind regards,
Sakari Ailus
^ permalink raw reply [flat|nested] 28+ messages in thread* RE: [PATCH 09/10] media: Keem Bay Camera: Add metadata video node
2021-04-09 10:24 ` Sakari Ailus
@ 2021-04-09 14:19 ` Martina Krasteva
2021-04-09 14:36 ` 'Sakari Ailus'
0 siblings, 1 reply; 28+ messages in thread
From: Martina Krasteva @ 2021-04-09 14:19 UTC (permalink / raw)
To: 'Sakari Ailus'
Cc: linux-media, mchehab, robh+dt, devicetree, daniele.alessandrelli,
paul.j.murphy, gjorgjix.rosikopulos
Hi Sakari,
Thank you for the review :)
>
> Hi Martina,
>
> On Fri, Mar 19, 2021 at 06:06:31PM +0000, Martina Krasteva wrote:
> > From: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
> >
> > Metadata video node implements output and capture meta type
> > interface.
> >
> > - Output video node is used to provide isp parameters for processing.
> >
> > Each buffer internally has real vpu isp params structure
> > allocated. User space params are copied on every qbuf based on
> > update flags. Since vpu need every time all parameters to be provided,
> > params are copied on every qbuf. Based on update flags they are copied
> > from userspace buffer or last buffer processed.
> > To reduce coping of the tables, they are allocated separately
> > in table buffer pool.
> > The tables are copied only when there is update from the userspace,
> > otherwise they are only reference from last processed frame.
> > This is possible because vpu interface has separate address for each table.
> >
> > - Capture video node is used to provide statistics to userspace.
> > Capture video node statistics memory addresses are copied to isp
> > params before processing, and corresponding update flags are set
> > based on statistics availability.
> >
> > Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
> > Co-developed-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
> > Signed-off-by: Ivan Dimitrov <ivanx.dimitrov@intel.com>
> > Co-developed-by: Martina Krasteva <martinax.krasteva@intel.com>
> > Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
> > Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
> > Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
> > ---
> > drivers/media/platform/keembay-camera/Makefile | 4 +-
> > .../platform/keembay-camera/keembay-metadata.c | 1823 +++++++++++++++++++-
> > .../platform/keembay-camera/keembay-metadata.h | 14 +-
> > .../keembay-camera/keembay-params-defaults.c | 326 ++++
> > .../keembay-camera/keembay-params-defaults.h | 38 +
> > 5 files changed, 2194 insertions(+), 11 deletions(-)
> > create mode 100644 drivers/media/platform/keembay-camera/keembay-params-defaults.c
> > create mode 100644 drivers/media/platform/keembay-camera/keembay-params-defaults.h
> >
> > diff --git a/drivers/media/platform/keembay-camera/Makefile b/drivers/media/platform/keembay-camera/Makefile
> > index 8b3ad715c5c4..1b949cf009ef 100644
> > --- a/drivers/media/platform/keembay-camera/Makefile
> > +++ b/drivers/media/platform/keembay-camera/Makefile
> > @@ -1,5 +1,5 @@
> > keembay-cam-objs = keembay-camera.o keembay-pipeline.o \
> > - keembay-cam-xlink.o keembay-isp.o \
> > - keembay-metadata.o keembay-video.o
> > + keembay-cam-xlink.o keembay-params-defaults.o \
> > + keembay-isp.o keembay-metadata.o keembay-video.o
> >
> > obj-$(CONFIG_VIDEO_INTEL_KEEMBAY_CAMERA) += keembay-cam.o
> > diff --git a/drivers/media/platform/keembay-camera/keembay-metadata.c b/drivers/media/platform/keembay-camera/keembay-
> metadata.c
> > index a1df746d9582..8807e3f322c5 100644
> > --- a/drivers/media/platform/keembay-camera/keembay-metadata.c
> > +++ b/drivers/media/platform/keembay-camera/keembay-metadata.c
> > @@ -4,17 +4,1818 @@
> > *
> > * Copyright (C) 2021 Intel Corporation
> > */
> > +
> > +#include <linux/keembay-isp-ctl.h>
> > +#include <linux/dmapool.h>
> > +
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +#include <media/videobuf2-vmalloc.h>
> > +
> > +#include "keembay-pipeline.h"
> > #include "keembay-metadata.h"
> >
> > +#define KMB_CAM_METADATA_STATS_NAME "keembay-metadata-stats"
> > +#define KMB_CAM_METADATA_PARAMS_NAME "keembay-metadata-params"
> > +
> > +#define KMB_TABLE_ALIGN 64
> > +
> > +/* Table names map */
> > +static const char *table_name[KMB_METADATA_TABLE_MAX] = {
> > + "LSC",
> > + "StaticDefect",
> > + "LCA",
> > + "HDR",
> > + "Sharpness",
> > + "Color cumb",
> > + "LUT",
> > + "TNF1",
> > + "TNF2",
> > + "Dehaze",
> > + "Warp",
> > +};
> > +
> > +static void
> > +kmb_metadata_copy_blc(struct kmb_vpu_blc_params *dst,
> > + struct kmb_blc_params *src)
> > +{
> > + int i;
>
> unsigned int would be preferred. The same for functions with similar loops
> below.
>
Thanks, will be fixed in next version
> > +
> > + for (i = 0; i < KMB_CAM_MAX_EXPOSURES; i++) {
> > + dst[i].coeff1 = src[i].coeff1;
> > + dst[i].coeff2 = src[i].coeff2;
> > + dst[i].coeff3 = src[i].coeff3;
> > + dst[i].coeff4 = src[i].coeff4;
> > + }
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_sigma_dns(struct kmb_vpu_sigma_dns_params *dst,
> > + struct kmb_sigma_dns_params *src)
> > +{
> > + int i;
> > +
> > + for (i = 0; i < KMB_CAM_MAX_EXPOSURES; i++) {
> > + dst[i].noise = src[i].noise;
> > + dst[i].threshold1 = src[i].threshold1;
> > + dst[i].threshold2 = src[i].threshold2;
> > + dst[i].threshold3 = src[i].threshold3;
> > + dst[i].threshold4 = src[i].threshold4;
> > + dst[i].threshold5 = src[i].threshold5;
> > + dst[i].threshold6 = src[i].threshold6;
> > + dst[i].threshold7 = src[i].threshold7;
> > + dst[i].threshold8 = src[i].threshold8;
> > + }
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_lsc(struct kmb_vpu_lsc_params *dst,
> > + struct kmb_lsc_params *src)
> > +{
> > + dst->threshold = src->threshold;
> > + dst->width = src->width;
> > + dst->height = src->height;
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_raw(struct kmb_vpu_raw_params *dst,
> > + struct kmb_raw_params *src)
> > +{
> > + dst->awb_stats_en = src->awb_stats_en;
> > + dst->awb_rgb_hist_en = src->awb_rgb_hist_en;
> > + dst->af_stats_en = src->af_stats_en;
> > + dst->luma_hist_en = src->luma_hist_en;
> > + dst->flicker_accum_en = src->flicker_accum_en;
> > + dst->bad_pixel_fix_en = src->bad_pixel_fix_en;
> > + dst->grgb_imb_en = src->grgb_imb_en;
> > + dst->mono_imbalance_en = src->mono_imbalance_en;
> > + dst->gain1 = src->gain1;
> > + dst->gain2 = src->gain2;
> > + dst->gain3 = src->gain3;
> > + dst->gain4 = src->gain4;
> > + dst->stop1 = src->stop1;
> > + dst->stop2 = src->stop2;
> > + dst->stop3 = src->stop3;
> > + dst->stop4 = src->stop4;
> > + dst->threshold1 = src->threshold1;
> > + dst->alpha1 = src->alpha1;
> > + dst->alpha2 = src->alpha2;
> > + dst->alpha3 = src->alpha3;
> > + dst->alpha4 = src->alpha4;
> > + dst->threshold2 = src->threshold2;
> > + dst->static_defect_size = src->static_defect_size;
> > + dst->flicker_first_row_acc = src->start_row;
> > + dst->flicker_last_row_acc = src->end_row;
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_ae_awb(struct kmb_vpu_ae_awb_params *dst,
> > + struct kmb_ae_awb_params *src)
> > +{
> > + dst->start_x = src->start_x;
> > + dst->start_y = src->start_y;
> > + dst->width = src->width;
> > + dst->height = src->height;
> > + dst->skip_x = src->skip_x;
> > + dst->skip_y = src->skip_y;
> > + dst->patches_x = src->patches_x;
> > + dst->patches_y = src->patches_y;
> > + dst->threshold1 = src->threshold1;
> > + dst->threshold2 = src->threshold2;
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_af(struct kmb_vpu_af_params *dst,
> > + struct kmb_af_params *src)
> > +{
> > + int i;
> > +
> > + dst->start_x = src->start_x;
> > + dst->start_y = src->start_y;
> > + dst->width = src->width;
> > + dst->height = src->height;
> > + dst->patches_x = src->patches_x;
> > + dst->patches_y = src->patches_y;
> > + dst->coeff = src->coeff;
> > + dst->threshold1 = src->threshold1;
> > + dst->threshold2 = src->threshold2;
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->coeffs1); i++) {
> > + dst->coeffs1[i] = src->coeffs1[i];
> > + dst->coeffs2[i] = src->coeffs2[i];
> > + }
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_histogram(struct kmb_vpu_hist_params *dst,
> > + struct kmb_hist_params *src)
> > +{
> > + int i;
> > +
> > + dst->start_x = src->start_x;
> > + dst->start_y = src->start_y;
> > + dst->end_x = src->end_x;
> > + dst->end_y = src->end_y;
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->matrix); i++)
> > + dst->matrix[i] = src->matrix[i];
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->weight); i++)
> > + dst->weight[i] = src->weight[i];
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_debayer(struct kmb_vpu_debayer_params *dst,
> > + struct kmb_debayer_params *src)
> > +{
> > + dst->coeff1 = src->coeff1;
> > + dst->multiplier1 = src->multiplier1;
> > + dst->multiplier2 = src->multiplier2;
> > + dst->coeff2 = src->coeff2;
> > + dst->coeff3 = src->coeff3;
> > + dst->coeff4 = src->coeff4;
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_dog_dns(struct kmb_vpu_dog_dns_params *dst,
> > + struct kmb_dog_dns_params *src)
> > +{
> > + int i;
> > +
> > + dst->threshold = src->threshold;
> > + dst->strength = src->strength;
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->coeffs11); i++)
> > + dst->coeffs11[i] = src->coeffs11[i];
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->coeffs15); i++)
> > + dst->coeffs15[i] = src->coeffs15[i];
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_luma_dns(struct kmb_vpu_luma_dns_params *dst,
> > + struct kmb_luma_dns_params *src)
> > +{
> > + dst->threshold = src->threshold;
> > + dst->slope = src->slope;
> > + dst->shift = src->shift;
> > + dst->alpha = src->alpha;
> > + dst->weight = src->weight;
> > + dst->per_pixel_alpha_en = src->per_pixel_alpha_en;
> > + dst->gain_bypass_en = src->gain_bypass_en;
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_sharpen(struct kmb_vpu_sharpen_params *dst,
> > + struct kmb_sharpen_params *src)
> > +{
> > + int i;
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->coeffs1); i++) {
> > + dst->coeffs1[i] = src->coeffs1[i];
> > + dst->coeffs2[i] = src->coeffs2[i];
> > + dst->coeffs3[i] = src->coeffs3[i];
> > + }
> > +
> > + dst->shift = src->shift;
> > + dst->gain1 = src->gain1;
> > + dst->gain2 = src->gain2;
> > + dst->gain3 = src->gain3;
> > + dst->gain4 = src->gain4;
> > + dst->gain5 = src->gain5;
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->stops1); i++) {
> > + dst->stops1[i] = src->stops1[i];
> > + dst->gains[i] = src->gains[i];
> > + }
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->stops2); i++)
> > + dst->stops2[i] = src->stops2[i];
> > +
> > + dst->overshoot = src->overshoot;
> > + dst->undershoot = src->undershoot;
> > + dst->alpha = src->alpha;
> > + dst->gain6 = src->gain6;
> > + dst->offset = src->offset;
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_chroma_gen(struct kmb_vpu_chroma_gen_params *dst,
> > + struct kmb_chroma_gen_params *src)
> > +{
> > + int i;
> > +
> > + dst->epsilon = src->epsilon;
> > + dst->coeff1 = src->coeff1;
> > + dst->coeff2 = src->coeff2;
> > + dst->coeff3 = src->coeff3;
> > + dst->coeff4 = src->coeff4;
> > + dst->coeff5 = src->coeff5;
> > + dst->coeff6 = src->coeff6;
> > + dst->strength1 = src->strength1;
> > + dst->strength2 = src->strength2;
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->coeffs); i++)
> > + dst->coeffs[i] = src->coeffs[i];
> > +
> > + dst->offset1 = src->offset1;
> > + dst->slope1 = src->slope1;
> > + dst->slope2 = src->slope2;
> > + dst->offset2 = src->offset2;
> > + dst->limit = src->limit;
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_median(struct kmb_vpu_median_params *dst,
> > + struct kmb_median_params *src)
> > +{
> > + dst->size = src->size;
> > + dst->slope = src->slope;
> > + dst->offset = src->offset;
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_chroma_dns(struct kmb_vpu_chroma_dns_params *dst,
> > + struct kmb_chroma_dns_params *src)
> > +{
> > + dst->limit = src->limit;
> > + dst->enable = src->enable;
> > + dst->threshold1 = src->threshold1;
> > + dst->threshold2 = src->threshold2;
> > + dst->threshold3 = src->threshold3;
> > + dst->threshold4 = src->threshold4;
> > + dst->threshold5 = src->threshold5;
> > + dst->threshold6 = src->threshold6;
> > + dst->threshold7 = src->threshold7;
> > + dst->threshold8 = src->threshold8;
> > + dst->slope1 = src->slope1;
> > + dst->offset1 = src->offset1;
> > + dst->slope2 = src->slope2;
> > + dst->offset2 = src->offset2;
> > + dst->grey1 = src->grey1;
> > + dst->grey2 = src->grey2;
> > + dst->grey3 = src->grey3;
> > + dst->coeff1 = src->coeff1;
> > + dst->coeff2 = src->coeff2;
> > + dst->coeff3 = src->coeff3;
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_color_comb(struct kmb_vpu_color_comb_params *dst,
> > + struct kmb_color_comb_params *src)
> > +{
> > + int i;
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->matrix); i++)
> > + dst->matrix[i] = src->matrix[i];
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->offsets); i++)
> > + dst->offsets[i] = src->offsets[i];
> > +
> > + dst->coeff1 = src->coeff1;
> > + dst->coeff2 = src->coeff2;
> > + dst->coeff3 = src->coeff3;
> > + dst->enable = src->enable;
> > + dst->weight1 = src->weight1;
> > + dst->weight2 = src->weight2;
> > + dst->weight3 = src->weight3;
> > + dst->limit1 = src->limit1;
> > + dst->limit2 = src->limit2;
> > + dst->offset1 = src->offset1;
> > + dst->offset2 = src->offset2;
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_hdr(struct kmb_vpu_hdr_params *dst,
> > + struct kmb_hdr_params *src)
> > +{
> > + int i;
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->ratio); i++)
> > + dst->ratio[i] = src->ratio[i];
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->scale); i++)
> > + dst->scale[i] = src->scale[i];
> > +
> > + dst->offset1 = src->offset1;
> > + dst->slope1 = src->slope1;
> > + dst->offset2 = src->offset2;
> > + dst->slope2 = src->slope2;
> > + dst->offset3 = src->offset3;
> > + dst->slope3 = src->slope3;
> > + dst->offset4 = src->offset4;
> > + dst->gain1 = src->gain1;
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->blur1); i++)
> > + dst->blur1[i] = src->blur1[i];
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->blur2); i++)
> > + dst->blur2[i] = src->blur2[i];
> > +
> > + dst->contrast1 = src->contrast1;
> > + dst->contrast2 = src->contrast2;
> > + dst->enable1 = src->enable1;
> > + dst->enable2 = src->enable2;
> > + dst->offset5 = src->offset5;
> > + dst->gain2 = src->gain2;
> > + dst->offset6 = src->offset6;
> > + dst->strength = src->strength;
> > + dst->offset7 = src->offset7;
> > + dst->shift = src->shift;
> > + dst->field1 = src->field1;
> > + dst->field2 = src->field2;
> > + dst->gain3 = src->gain3;
> > + dst->min = src->min;
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_lut(struct kmb_vpu_lut_params *dst,
> > + struct kmb_lut_params *src)
> > +{
> > + int i;
> > +
> > + dst->size = src->size;
> > + for (i = 0; i < ARRAY_SIZE(dst->matrix); i++)
> > + dst->matrix[i] = src->matrix[i];
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->offsets); i++)
> > + dst->offsets[i] = src->offsets[i];
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_tnf(struct kmb_vpu_tnf_params *dst,
> > + struct kmb_tnf_params *src)
> > +{
> > + dst->factor = src->factor;
> > + dst->gain = src->gain;
> > + dst->offset1 = src->offset1;
> > + dst->slope1 = src->slope1;
> > + dst->offset2 = src->offset2;
> > + dst->slope2 = src->slope2;
> > + dst->min1 = src->min1;
> > + dst->min2 = src->min2;
> > + dst->value = src->value;
> > + dst->enable = src->enable;
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_dehaze(struct kmb_vpu_dehaze_params *dst,
> > + struct kmb_dehaze_params *src)
> > +{
> > + int i;
> > +
> > + dst->gain1 = src->gain1;
> > + dst->min = src->min;
> > + dst->strength1 = src->strength1;
> > + dst->strength2 = src->strength2;
> > + dst->gain2 = src->gain2;
> > + dst->saturation = src->saturation;
> > + dst->value1 = src->value1;
> > + dst->value2 = src->value2;
> > + dst->value3 = src->value3;
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->filter); i++)
> > + dst->filter[i] = src->filter[i];
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_warp(struct kmb_vpu_warp_params *dst,
> > + struct kmb_warp_params *src)
> > +{
> > + int i;
> > +
> > + dst->type = src->type;
> > + dst->relative = src->relative;
> > + dst->format = src->format;
> > + dst->position = src->position;
> > + dst->width = src->width;
> > + dst->height = src->height;
> > + dst->stride = src->stride;
> > + dst->enable = src->enable;
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->matrix); i++)
> > + dst->matrix[i] = src->matrix[i];
> > +
> > + dst->mode = src->mode;
> > +
> > + for (i = 0; i < ARRAY_SIZE(dst->values); i++)
> > + dst->values[i] = src->values[i];
> > +}
> > +
> > +/* VPU Params tables */
> > +static struct kmb_metadata_table *
> > +kmb_metadata_cpalloc_table(struct kmb_metadata *kmb_meta,
> > + enum kmb_metadata_table_type type,
> > + size_t src_table_size)
> > +{
> > + struct kmb_metadata_table *table;
> > +
> > + lockdep_assert_held(&kmb_meta->lock);
> > +
> > + /* First create pool if needed */
> > + if (!kmb_meta->table_pool[type]) {
> > + kmb_meta->table_pool[type] =
> > + dma_pool_create(table_name[type],
> > + kmb_meta->dma_dev,
> > + src_table_size + sizeof(*table),
> > + KMB_TABLE_ALIGN, 0);
> > + if (!kmb_meta->table_pool[type]) {
> > + dev_err(kmb_meta->dma_dev,
> > + "Fail to create %s pool", table_name[type]);
> > + return NULL;
> > + }
> > + }
> > +
> > + table = kmalloc(sizeof(*table), GFP_KERNEL);
> > + if (!table)
> > + return NULL;
> > +
> > + kref_init(&table->refcount);
> > + table->pool = kmb_meta->table_pool[type];
> > +
> > + table->cpu_addr = dma_pool_alloc(kmb_meta->table_pool[type],
> > + GFP_KERNEL,
> > + &table->dma_addr);
> > + if (!table->cpu_addr) {
> > + kfree(table);
> > + return NULL;
> > + }
> > +
> > + return table;
> > +}
> > +
> > +static void kmb_metadata_free_table(struct kref *ref)
> > +{
> > + struct kmb_metadata_table *table =
> > + container_of(ref, struct kmb_metadata_table, refcount);
> > +
> > + dma_pool_free(table->pool, table->cpu_addr, table->dma_addr);
> > + kfree(table);
> > +}
> > +
> > +static void
> > +kmb_metadata_release_tables(struct kmb_metadata_buf *meta_buf)
>
> How about calling this kmb_metadata_put_tables()? It puts the tables that
> may result in releasing them.
>
> > +{
> > + int i;
> > +
> > + for (i = 0; i < KMB_METADATA_TABLE_MAX; i++) {
> > + if (meta_buf->params.tab[i]) {
> > + kref_put(&meta_buf->params.tab[i]->refcount,
> > + kmb_metadata_free_table);
> > + meta_buf->params.tab[i] = NULL;
> > + }
> > + }
> > +}
> > +
> > +static void
> > +kmb_metadata_destroy_table_pools(struct kmb_metadata *kmb_meta)
> > +{
> > + int i;
> > +
> > + /* Release allocated pools during streaming */
> > + for (i = 0; i < KMB_METADATA_TABLE_MAX; i++) {
> > + dma_pool_destroy(kmb_meta->table_pool[i]);
> > + kmb_meta->table_pool[i] = NULL;
> > + }
> > +}
> > +
> > +static dma_addr_t
> > +kmb_metadata_get_table_addr(struct kmb_metadata_buf *meta_buf,
> > + enum kmb_metadata_table_type type)
> > +{
> > + struct kmb_metadata_table *table = meta_buf->params.tab[type];
> > +
> > + if (!table)
> > + return 0;
> > +
> > + return table->dma_addr;
> > +}
> > +
> > +static struct kmb_metadata_table *
> > +kmb_metadata_create_table(struct kmb_metadata *kmb_meta,
> > + struct kmb_metadata_buf *meta_buf,
> > + enum kmb_metadata_table_type type,
> > + size_t user_table_size)
> > +{
> > + struct kmb_metadata_table *table;
> > +
> > + lockdep_assert_held(&kmb_meta->lock);
> > +
> > + table = kmb_metadata_cpalloc_table(kmb_meta,
> > + type,
> > + user_table_size);
>
> Fits on fewer lines.
>
> > + if (!table)
> > + return NULL;
> > +
> > + if (meta_buf->params.tab[type])
> > + kref_put(&meta_buf->params.tab[type]->refcount,
> > + kmb_metadata_free_table);
> > +
> > + meta_buf->params.tab[type] = table;
> > +
> > + return table;
> > +}
> > +
> > +static int
> > +kmb_metadata_copy_table_usr(struct kmb_metadata *kmb_meta,
> > + struct kmb_metadata_buf *meta_buf,
> > + enum kmb_metadata_table_type type,
> > + u8 *user_table, size_t user_table_size)
> > +{
> > + struct kmb_metadata_table *table;
> > +
> > + table = kmb_metadata_create_table(kmb_meta, meta_buf,
> > + type, user_table_size);
> > + if (!table)
> > + return -ENOMEM;
> > +
> > + memcpy(table->cpu_addr, user_table, user_table_size);
> > +
> > + return 0;
> > +}
> > +
> > +static int kmb_metadata_create_default_table(struct kmb_metadata *kmb_meta,
> > + struct kmb_metadata_buf *meta_buf,
> > + enum kmb_metadata_table_type type,
> > + u8 *user_table,
> > + size_t user_table_size)
> > +{
> > + struct kmb_metadata_table *table;
> > +
> > + table = kmb_metadata_create_table(kmb_meta, meta_buf,
> > + type, user_table_size);
> > + if (!table)
> > + return -ENOMEM;
> > +
> > + memset(table->cpu_addr, 0, user_table_size);
> > +
> > + return 0;
> > +}
> > +
> > +static void
> > +kmb_metadata_copy_table_vpu(struct kmb_metadata_buf *meta_buf,
> > + struct kmb_metadata_buf *last_meta_buf,
> > + enum kmb_metadata_table_type type)
> > +{
> > + /* Do nothing if params are the same */
> > + if (WARN_ON(meta_buf->params.isp == last_meta_buf->params.isp))
> > + return;
> > +
> > + meta_buf->params.tab[type] = last_meta_buf->params.tab[type];
> > + if (meta_buf->params.tab[type])
> > + kref_get(&meta_buf->params.tab[type]->refcount);
> > +}
> > +
> > +static void
> > +kmb_metadata_fill_blc(struct kmb_vpu_isp_params *params,
> > + struct kmb_isp_params *user_params,
> > + struct kmb_vpu_isp_params *last_params,
> > + struct kmb_vpu_isp_params_defaults *def_params)
> > +{
> > + if (user_params->update.blc) {
> > + kmb_metadata_copy_blc(params->blc, user_params->blc);
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(params->blc, last_params->blc,
> > + sizeof(params->blc));
> > + } else {
> > + memcpy(params->blc, def_params->blc, sizeof(params->blc));
> > + }
> > +}
> > +
> > +static void
> > +kmb_metadata_fill_signma_dns(struct kmb_vpu_isp_params *params,
> > + struct kmb_isp_params *user_params,
> > + struct kmb_vpu_isp_params *last_params,
> > + struct kmb_vpu_isp_params_defaults *def_params)
> > +{
> > + if (user_params->update.sigma_dns) {
> > + kmb_metadata_copy_sigma_dns(params->sigma_dns,
> > + user_params->sigma_dns);
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(params->sigma_dns, last_params->sigma_dns,
> > + sizeof(params->sigma_dns));
> > + } else {
> > + memcpy(params->sigma_dns, def_params->sigma_dns,
> > + sizeof(params->sigma_dns));
> > + }
> > +}
> > +
> > +static void
> > +kmb_metadata_fill_ae_awb(struct kmb_vpu_isp_params *params,
> > + struct kmb_isp_params *user_params,
> > + struct kmb_vpu_isp_params *last_params,
> > + struct kmb_vpu_isp_params_defaults *def_params)
> > +{
> > + if (user_params->update.ae_awb) {
> > + kmb_metadata_copy_ae_awb(¶ms->ae_awb,
> > + &user_params->ae_awb);
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(¶ms->ae_awb, &last_params->ae_awb,
> > + sizeof(params->ae_awb));
> > + } else {
> > + memcpy(¶ms->ae_awb, def_params->ae_awb,
> > + sizeof(params->ae_awb));
> > + }
> > +}
> > +
> > +static void
> > +kmb_metadata_fill_af(struct kmb_vpu_isp_params *params,
> > + struct kmb_isp_params *user_params,
> > + struct kmb_vpu_isp_params *last_params,
> > + struct kmb_vpu_isp_params_defaults *def_params)
> > +{
> > + if (user_params->update.af) {
> > + kmb_metadata_copy_af(¶ms->af, &user_params->af);
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(¶ms->af, &last_params->af,
> > + sizeof(params->af));
> > + } else {
> > + memcpy(¶ms->af, def_params->af, sizeof(params->af));
> > + }
> > +}
> > +
> > +static void
> > +kmb_metadata_fill_histogram(struct kmb_vpu_isp_params *params,
> > + struct kmb_isp_params *user_params,
> > + struct kmb_vpu_isp_params *last_params,
> > + struct kmb_vpu_isp_params_defaults *def_params)
> > +{
> > + if (user_params->update.histogram) {
> > + kmb_metadata_copy_histogram(¶ms->histogram,
> > + &user_params->histogram);
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(¶ms->histogram, &last_params->histogram,
> > + sizeof(params->histogram));
> > + } else {
> > + memcpy(¶ms->histogram, def_params->histogram,
> > + sizeof(params->histogram));
> > + }
> > +}
> > +
> > +static void
> > +kmb_metadata_fill_debayer(struct kmb_vpu_isp_params *params,
> > + struct kmb_isp_params *user_params,
> > + struct kmb_vpu_isp_params *last_params,
> > + struct kmb_vpu_isp_params_defaults *def_params)
> > +{
> > + if (user_params->update.debayer) {
> > + kmb_metadata_copy_debayer(¶ms->debayer,
> > + &user_params->debayer);
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(¶ms->debayer, &last_params->debayer,
> > + sizeof(params->debayer));
> > + } else {
> > + memcpy(¶ms->debayer, def_params->debayer,
> > + sizeof(params->debayer));
> > + }
> > +}
> > +
> > +static void
> > +kmb_metadata_fill_dog_dns(struct kmb_vpu_isp_params *params,
> > + struct kmb_isp_params *user_params,
> > + struct kmb_vpu_isp_params *last_params,
> > + struct kmb_vpu_isp_params_defaults *def_params)
> > +{
> > + if (user_params->update.dog_dns) {
> > + kmb_metadata_copy_dog_dns(¶ms->dog_dns,
> > + &user_params->dog_dns);
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(¶ms->dog_dns, &last_params->dog_dns,
> > + sizeof(params->dog_dns));
> > + } else {
> > + memcpy(¶ms->dog_dns, def_params->dog_dns,
> > + sizeof(params->dog_dns));
> > + }
> > +}
> > +
> > +static void
> > +kmb_metadata_fill_luma_dns(struct kmb_vpu_isp_params *params,
> > + struct kmb_isp_params *user_params,
> > + struct kmb_vpu_isp_params *last_params,
> > + struct kmb_vpu_isp_params_defaults *def_params)
> > +{
> > + if (user_params->update.luma_dns) {
> > + kmb_metadata_copy_luma_dns(¶ms->luma_dns,
> > + &user_params->luma_dns);
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(¶ms->luma_dns, &last_params->luma_dns,
> > + sizeof(params->luma_dns));
> > + } else {
> > + memcpy(¶ms->luma_dns, def_params->luma_dns,
> > + sizeof(params->luma_dns));
> > + }
> > +}
> > +
> > +static void
> > +kmb_metadata_fill_chroma_gen(struct kmb_vpu_isp_params *params,
> > + struct kmb_isp_params *user_params,
> > + struct kmb_vpu_isp_params *last_params,
> > + struct kmb_vpu_isp_params_defaults *def_params)
> > +{
> > + if (user_params->update.chroma_gen) {
> > + kmb_metadata_copy_chroma_gen(¶ms->chroma_gen,
> > + &user_params->chroma_gen);
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(¶ms->chroma_gen, &last_params->chroma_gen,
> > + sizeof(params->chroma_gen));
> > + } else {
> > + memcpy(¶ms->chroma_gen, def_params->chroma_gen,
> > + sizeof(params->chroma_gen));
> > + }
> > +}
> > +
> > +static void
> > +kmb_metadata_fill_median(struct kmb_vpu_isp_params *params,
> > + struct kmb_isp_params *user_params,
> > + struct kmb_vpu_isp_params *last_params,
> > + struct kmb_vpu_isp_params_defaults *def_params)
> > +{
> > + if (user_params->update.median) {
> > + kmb_metadata_copy_median(¶ms->median,
> > + &user_params->median);
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(¶ms->median, &last_params->median,
> > + sizeof(params->median));
> > + } else {
> > + memcpy(¶ms->median, def_params->median,
> > + sizeof(params->median));
> > + }
> > +}
> > +
> > +static void
> > +kmb_metadata_fill_chroma_dns(struct kmb_vpu_isp_params *params,
> > + struct kmb_isp_params *user_params,
> > + struct kmb_vpu_isp_params *last_params,
> > + struct kmb_vpu_isp_params_defaults *def_params)
> > +{
> > + if (user_params->update.chroma_dns) {
> > + kmb_metadata_copy_chroma_dns(¶ms->chroma_dns,
> > + &user_params->chroma_dns);
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(¶ms->chroma_dns, &last_params->chroma_dns,
> > + sizeof(params->chroma_dns));
> > + } else {
> > + memcpy(¶ms->chroma_dns, def_params->chroma_dns,
> > + sizeof(params->chroma_dns));
> > + }
> > +}
> > +
> > +static void
> > +kmb_metadata_fill_dehaze(struct kmb_vpu_isp_params *params,
> > + struct kmb_isp_params *user_params,
> > + struct kmb_vpu_isp_params *last_params,
> > + struct kmb_vpu_isp_params_defaults *def_params)
> > +{
> > + if (user_params->update.dehaze) {
> > + kmb_metadata_copy_dehaze(¶ms->dehaze,
> > + &user_params->dehaze);
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(¶ms->dehaze, &last_params->dehaze,
> > + sizeof(params->dehaze));
> > + } else {
> > + memcpy(¶ms->dehaze, def_params->dehaze,
> > + sizeof(params->dehaze));
> > + }
> > +}
> > +
> > +static int
> > +kmb_metadata_fill_lsc(struct kmb_metadata *kmb_meta,
> > + struct kmb_metadata_buf *meta_buf,
> > + struct kmb_isp_params *user_params)
> > +{
> > + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
> > + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> > + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> > + struct kmb_vpu_isp_params *last_params = NULL;
> > + int ret = 0;
> > +
> > + if (last_buf)
> > + last_params = last_buf->params.isp;
> > +
> > + if (user_params->update.lsc) {
> > + kmb_metadata_copy_lsc(¶ms->lsc,
> > + &user_params->lsc);
> > + if (params->lsc.width && params->lsc.height) {
> > + ret = kmb_metadata_copy_table_usr(kmb_meta,
> > + meta_buf,
> > + KMB_METADATA_TABLE_LSC,
> > + user_params->lsc.gain_mesh,
> > + params->lsc.width *
> > + params->lsc.height);
> > + if (ret < 0)
> > + return ret;
> > + }
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(¶ms->lsc, &last_params->lsc,
> > + sizeof(params->lsc));
> > +
> > + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
> > + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> > + KMB_METADATA_TABLE_LSC);
> > + } else {
> > + memcpy(¶ms->lsc, def_params->lsc, sizeof(params->lsc));
> > + kmb_metadata_create_default_table(kmb_meta,
> > + meta_buf,
> > + KMB_METADATA_TABLE_LSC,
> > + user_params->lsc.gain_mesh,
> > + ARRAY_SIZE(user_params->lsc.gain_mesh));
> > + }
> > +
> > + if (params->lsc.width && params->lsc.height) {
> > + params->lsc.addr =
> > + kmb_metadata_get_table_addr(meta_buf,
> > + KMB_METADATA_TABLE_LSC);
> > + if (!params->lsc.addr)
> > + ret = -EINVAL;
> > + }
> > +
> > + return ret;
> > +}
>
> There seems to be a few groups of functions that look very similar to each
> other, with the difference that they just operate on different struct
> fields. I wonder if these could be refactored into one (or a few) function
> per group that would just operate on different data. If you need to
> differentiate more, you can use smaller functions to do a particular part
> of the job that is different between these functions (within a group) and
> use the offsetof() and sizeof() macros.
>
Thanks for the suggestion. We will try to refactor the code to optimize it.
> > +
> > +static int
> > +kmb_metadata_fill_raw(struct kmb_metadata *kmb_meta,
> > + struct kmb_metadata_buf *meta_buf,
> > + struct kmb_isp_params *user_params)
> > +{
> > + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
> > + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> > + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> > + struct kmb_vpu_isp_params *last_params = NULL;
> > + int ret = 0;
> > +
> > + if (last_buf)
> > + last_params = last_buf->params.isp;
> > +
> > + if (user_params->update.raw) {
> > + kmb_metadata_copy_raw(¶ms->raw,
> > + &user_params->raw);
> > + if (params->raw.static_defect_size) {
> > + ret = kmb_metadata_copy_table_usr(kmb_meta,
> > + meta_buf,
> > + KMB_METADATA_TABLE_SDEFECT,
> > + user_params->raw.static_defect_map,
> > + params->raw.static_defect_size);
> > + if (ret < 0)
> > + return ret;
> > + }
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(¶ms->raw, &last_params->raw,
> > + sizeof(params->raw));
> > +
> > + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
> > + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> > + KMB_METADATA_TABLE_SDEFECT);
> > + } else {
> > + memcpy(¶ms->raw, def_params->raw, sizeof(params->raw));
> > + kmb_metadata_create_default_table(kmb_meta,
> > + meta_buf,
> > + KMB_METADATA_TABLE_SDEFECT,
> > + user_params->raw.static_defect_map,
> > + ARRAY_SIZE(user_params->raw.static_defect_map));
> > + }
> > +
> > + if (params->raw.static_defect_size) {
> > + params->raw.static_defect_addr =
> > + kmb_metadata_get_table_addr(meta_buf,
> > + KMB_METADATA_TABLE_SDEFECT);
> > + if (!params->raw.static_defect_addr)
> > + ret = -EINVAL;
> > + }
> > +
> > + return ret;
> > +}
> > +
> > +static int
> > +kmb_metadata_fill_lca(struct kmb_metadata *kmb_meta,
> > + struct kmb_metadata_buf *meta_buf,
> > + struct kmb_isp_params *user_params)
> > +{
> > + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> > + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> > + struct kmb_vpu_isp_params *last_params = NULL;
> > + int ret = 0;
> > +
> > + if (last_buf)
> > + last_params = last_buf->params.isp;
> > +
> > + if (user_params->update.lca) {
> > + ret = kmb_metadata_copy_table_usr(kmb_meta,
> > + meta_buf,
> > + KMB_METADATA_TABLE_LCA,
> > + user_params->lca.coeff,
> > + ARRAY_SIZE(user_params->lca.coeff));
> > + if (ret < 0)
> > + return ret;
> > + } else if (last_params) {
> > + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
> > + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> > + KMB_METADATA_TABLE_LCA);
> > + } else {
> > + kmb_metadata_create_default_table(kmb_meta,
> > + meta_buf,
> > + KMB_METADATA_TABLE_LCA,
> > + user_params->lca.coeff,
> > + ARRAY_SIZE(user_params->lca.coeff));
> > + }
> > +
> > + params->lca.addr = kmb_metadata_get_table_addr(meta_buf,
> > + KMB_METADATA_TABLE_LCA);
> > + if (!params->lca.addr)
> > + ret = -EINVAL;
> > +
> > + return ret;
> > +}
> > +
> > +static int
> > +kmb_metadata_fill_sharpen(struct kmb_metadata *kmb_meta,
> > + struct kmb_metadata_buf *meta_buf,
> > + struct kmb_isp_params *user_params)
> > +{
> > + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
> > + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> > + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> > + struct kmb_vpu_isp_params *last_params = NULL;
> > + int ret = 0;
> > +
> > + if (last_buf)
> > + last_params = last_buf->params.isp;
> > +
> > + if (user_params->update.sharpen) {
> > + kmb_metadata_copy_sharpen(¶ms->sharpen,
> > + &user_params->sharpen);
> > + ret = kmb_metadata_copy_table_usr(kmb_meta,
> > + meta_buf,
> > + KMB_METADATA_TABLE_SHARP,
> > + user_params->sharpen.radial_lut,
> > + ARRAY_SIZE(user_params->sharpen.radial_lut));
> > + if (ret < 0)
> > + return ret;
> > +
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(¶ms->sharpen, &last_params->sharpen,
> > + sizeof(params->sharpen));
> > +
> > + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
> > + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> > + KMB_METADATA_TABLE_SHARP);
> > + } else {
> > + memcpy(¶ms->sharpen, def_params->sharpen,
> > + sizeof(params->sharpen));
> > +
> > + kmb_metadata_create_default_table(kmb_meta,
> > + meta_buf,
> > + KMB_METADATA_TABLE_SHARP,
> > + user_params->sharpen.radial_lut,
> > + ARRAY_SIZE(user_params->sharpen.radial_lut));
> > + }
> > +
> > + params->sharpen.addr =
> > + kmb_metadata_get_table_addr(meta_buf,
> > + KMB_METADATA_TABLE_SHARP);
> > + if (!params->sharpen.addr)
> > + ret = -EINVAL;
> > +
> > + return ret;
> > +}
> > +
> > +static int
> > +kmb_metadata_fill_color_comb(struct kmb_metadata *kmb_meta,
> > + struct kmb_metadata_buf *meta_buf,
> > + struct kmb_isp_params *user_params)
> > +{
> > + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
> > + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> > + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> > + struct kmb_vpu_isp_params *last_params = NULL;
> > + struct kmb_color_comb_params *col = NULL;
> > + int ret = 0;
> > +
> > + if (last_buf)
> > + last_params = last_buf->params.isp;
> > +
> > + if (user_params->update.color_comb) {
> > + col = &user_params->color_comb;
> > + kmb_metadata_copy_color_comb(¶ms->color_comb,
> > + &user_params->color_comb);
> > + if (params->color_comb.enable) {
> > + ret = kmb_metadata_copy_table_usr(kmb_meta,
> > + meta_buf,
> > + KMB_METADATA_TABLE_COLOR_CUMB,
> > + col->lut_3d,
> > + ARRAY_SIZE(col->lut_3d));
> > + if (ret < 0)
> > + return ret;
> > + }
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(¶ms->color_comb, &last_params->color_comb,
> > + sizeof(params->color_comb));
> > +
> > + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
> > + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> > + KMB_METADATA_TABLE_COLOR_CUMB);
> > + } else {
> > + memcpy(¶ms->color_comb, def_params->color_comb,
> > + sizeof(params->color_comb));
> > + }
> > +
> > + if (params->color_comb.enable) {
> > + params->color_comb.addr =
> > + kmb_metadata_get_table_addr(meta_buf,
> > + KMB_METADATA_TABLE_COLOR_CUMB);
> > + if (!params->color_comb.addr)
> > + ret = -EINVAL;
> > + }
> > +
> > + return ret;
> > +}
> > +
> > +static int
> > +kmb_metadata_fill_hdr(struct kmb_metadata *kmb_meta,
> > + struct kmb_metadata_buf *meta_buf,
> > + struct kmb_isp_params *user_params)
> > +{
> > + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
> > + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> > + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> > + struct kmb_vpu_isp_params *last_params = NULL;
> > + int ret = 0;
> > +
> > + if (last_buf)
> > + last_params = last_buf->params.isp;
> > +
> > + if (user_params->update.hdr) {
> > + kmb_metadata_copy_hdr(¶ms->hdr,
> > + &user_params->hdr);
> > + if (params->hdr.enable1 || params->hdr.enable2) {
> > + ret = kmb_metadata_copy_table_usr(kmb_meta,
> > + meta_buf,
> > + KMB_METADATA_TABLE_HDR,
> > + user_params->hdr.tm_lut,
> > + ARRAY_SIZE(user_params->hdr.tm_lut));
> > + if (ret < 0)
> > + return ret;
> > + }
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(¶ms->hdr, &last_params->hdr,
> > + sizeof(params->hdr));
> > +
> > + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
> > + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> > + KMB_METADATA_TABLE_HDR);
> > + } else {
> > + memcpy(¶ms->hdr, def_params->hdr, sizeof(params->hdr));
> > + }
> > +
> > + if (params->hdr.enable1 || params->hdr.enable2) {
> > + params->hdr.luts_addr =
> > + kmb_metadata_get_table_addr(meta_buf,
> > + KMB_METADATA_TABLE_HDR);
> > + if (!params->hdr.luts_addr)
> > + ret = -EINVAL;
> > + }
> > +
> > + return ret;
> > +}
> > +
> > +static int
> > +kmb_metadata_fill_lut(struct kmb_metadata *kmb_meta,
> > + struct kmb_metadata_buf *meta_buf,
> > + struct kmb_isp_params *user_params)
> > +{
> > + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
> > + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> > + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> > + struct kmb_vpu_isp_params *last_params = NULL;
> > + int ret = 0;
> > +
> > + if (last_buf)
> > + last_params = last_buf->params.isp;
> > +
> > + if (user_params->update.lut) {
> > + kmb_metadata_copy_lut(¶ms->lut, &user_params->lut);
> > + if (params->lut.size) {
> > + ret = kmb_metadata_copy_table_usr(kmb_meta,
> > + meta_buf,
> > + KMB_METADATA_TABLE_LUT,
> > + user_params->lut.table,
> > + ARRAY_SIZE(user_params->lut.table));
> > + if (ret < 0)
> > + return ret;
> > + }
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(¶ms->lut, &last_params->lut,
> > + sizeof(params->lut));
> > +
> > + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
> > + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> > + KMB_METADATA_TABLE_LUT);
> > + } else {
> > + memcpy(¶ms->lut, def_params->lut, sizeof(params->lut));
> > + kmb_metadata_create_default_table(kmb_meta,
> > + meta_buf,
> > + KMB_METADATA_TABLE_LUT,
> > + user_params->lut.table,
> > + ARRAY_SIZE(user_params->lut.table));
> > + }
> > +
> > + if (params->lut.size) {
> > + params->lut.addr =
> > + kmb_metadata_get_table_addr(meta_buf,
> > + KMB_METADATA_TABLE_LUT);
> > + if (!params->lut.size)
> > + ret = -EINVAL;
> > + }
> > +
> > + return ret;
> > +}
> > +
> > +static int
> > +kmb_metadata_fill_warp(struct kmb_metadata *kmb_meta,
> > + struct kmb_metadata_buf *meta_buf,
> > + struct kmb_isp_params *user_params)
> > +{
> > + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
> > + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> > + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> > + struct kmb_vpu_isp_params *last_params = NULL;
> > + int ret = 0;
> > +
> > + if (last_buf)
> > + last_params = last_buf->params.isp;
> > +
> > + if (user_params->update.warp) {
> > + kmb_metadata_copy_warp(¶ms->warp, &user_params->warp);
> > + if (params->warp.enable) {
> > + ret = kmb_metadata_copy_table_usr(kmb_meta,
> > + meta_buf,
> > + KMB_METADATA_TABLE_WARP,
> > + user_params->warp.mesh_grid,
> > + ARRAY_SIZE(user_params->warp.mesh_grid));
> > + if (ret < 0)
> > + return ret;
> > + }
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(¶ms->warp, &last_params->warp,
> > + sizeof(params->warp));
> > +
> > + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf)
> > + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> > + KMB_METADATA_TABLE_WARP);
> > + } else {
> > + memcpy(¶ms->warp, def_params->warp, sizeof(params->warp));
> > + }
> > +
> > + if (params->warp.enable) {
> > + params->warp.addr =
> > + kmb_metadata_get_table_addr(meta_buf,
> > + KMB_METADATA_TABLE_WARP);
> > + if (!params->warp.addr)
> > + ret = -EINVAL;
> > + }
> > +
> > + return ret;
> > +}
> > +
> > +static int
> > +kmb_metadata_fill_tnf(struct kmb_metadata *kmb_meta,
> > + struct kmb_metadata_buf *meta_buf,
> > + struct kmb_isp_params *user_params)
> > +{
> > + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
> > + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> > + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> > + struct kmb_vpu_isp_params *last_params = NULL;
> > + struct kmb_tnf_params *tnf = NULL;
> > + int ret = 0;
> > +
> > + if (last_buf)
> > + last_params = last_buf->params.isp;
> > +
> > + if (user_params->update.tnf) {
> > + kmb_metadata_copy_tnf(¶ms->tnf, &user_params->tnf);
> > + if (params->tnf.enable) {
> > + tnf = &user_params->tnf;
> > + ret = kmb_metadata_copy_table_usr(kmb_meta,
> > + meta_buf,
> > + KMB_METADATA_TABLE_TNF0,
> > + tnf->chroma_lut0,
> > + ARRAY_SIZE(tnf->chroma_lut0));
> > + if (ret < 0)
> > + return ret;
> > +
> > + ret = kmb_metadata_copy_table_usr(kmb_meta,
> > + meta_buf,
> > + KMB_METADATA_TABLE_TNF1,
> > + tnf->chroma_lut1,
> > + ARRAY_SIZE(tnf->chroma_lut1));
> > + if (ret < 0)
> > + return ret;
> > + }
> > + } else if (last_params) {
> > + if (last_params != params)
> > + memcpy(¶ms->tnf, &last_params->tnf,
> > + sizeof(params->tnf));
> > +
> > + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf) {
> > + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> > + KMB_METADATA_TABLE_TNF0);
> > + kmb_metadata_copy_table_vpu(meta_buf, last_buf,
> > + KMB_METADATA_TABLE_TNF1);
> > + }
> > + } else {
> > + memcpy(¶ms->tnf, def_params->tnf, sizeof(params->tnf));
> > + }
> > +
> > + if (params->tnf.enable) {
> > + params->tnf.lut0_addr =
> > + kmb_metadata_get_table_addr(meta_buf,
> > + KMB_METADATA_TABLE_TNF0);
> > + if (!params->tnf.lut0_addr)
> > + return -EINVAL;
> > +
> > + params->tnf.lut1_addr =
> > + kmb_metadata_get_table_addr(meta_buf,
> > + KMB_METADATA_TABLE_TNF1);
> > + if (!params->tnf.lut1_addr)
> > + return -EINVAL;
> > + }
> > +
> > + return ret;
>
> This is a successful case, so you could return 0 and omit assigning ret in
> declaration.
>
> You could do the same in the functions above this one, too (and change one
> ret assignment to return).
Will be fixed
>
> > +}
> > +
> > +/* Fill static functions for conversions here */
> > +static int kmb_metadata_fill_isp_params(struct kmb_metadata *kmb_meta,
> > + struct kmb_metadata_buf *meta_buf,
> > + struct kmb_isp_params *user_params)
> > +{
> > + struct kmb_vpu_isp_params *params = meta_buf->params.isp;
> > + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf;
> > + struct kmb_vpu_isp_params *last_params = NULL;
> > + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def;
> > + int ret;
> > +
> > + if (last_buf)
> > + last_params = last_buf->params.isp;
> > +
> > + kmb_metadata_fill_blc(params, user_params, last_params, def_params);
> > +
> > + kmb_metadata_fill_signma_dns(params, user_params, last_params,
> > + def_params);
> > +
> > + kmb_metadata_fill_ae_awb(params, user_params, last_params, def_params);
> > +
> > + kmb_metadata_fill_af(params, user_params, last_params, def_params);
> > +
> > + kmb_metadata_fill_histogram(params, user_params, last_params,
> > + def_params);
> > +
> > + kmb_metadata_fill_debayer(params, user_params, last_params,
> > + def_params);
> > +
> > + kmb_metadata_fill_dog_dns(params, user_params, last_params,
> > + def_params);
> > +
> > + kmb_metadata_fill_luma_dns(params, user_params, last_params,
> > + def_params);
> > +
> > + kmb_metadata_fill_chroma_gen(params, user_params, last_params,
> > + def_params);
> > +
> > + kmb_metadata_fill_median(params, user_params, last_params, def_params);
> > +
> > + kmb_metadata_fill_chroma_dns(params, user_params, last_params,
> > + def_params);
> > +
> > + kmb_metadata_fill_dehaze(params, user_params, last_params, def_params);
> > +
> > + /* Copy params with tables */
> > + ret = kmb_metadata_fill_lsc(kmb_meta, meta_buf, user_params);
> > + if (ret < 0)
> > + goto error_release_tables;
> > +
> > + ret = kmb_metadata_fill_raw(kmb_meta, meta_buf, user_params);
> > + if (ret < 0)
> > + goto error_release_tables;
> > +
> > + ret = kmb_metadata_fill_lca(kmb_meta, meta_buf, user_params);
> > + if (ret < 0)
> > + goto error_release_tables;
> > +
> > + ret = kmb_metadata_fill_sharpen(kmb_meta, meta_buf, user_params);
> > + if (ret < 0)
> > + goto error_release_tables;
> > +
> > + ret = kmb_metadata_fill_color_comb(kmb_meta, meta_buf, user_params);
> > + if (ret < 0)
> > + goto error_release_tables;
> > +
> > + ret = kmb_metadata_fill_hdr(kmb_meta, meta_buf, user_params);
> > + if (ret < 0)
> > + goto error_release_tables;
> > +
> > + ret = kmb_metadata_fill_lut(kmb_meta, meta_buf, user_params);
> > + if (ret < 0)
> > + goto error_release_tables;
> > +
> > + ret = kmb_metadata_fill_warp(kmb_meta, meta_buf, user_params);
> > + if (ret < 0)
> > + goto error_release_tables;
> > +
> > + ret = kmb_metadata_fill_tnf(kmb_meta, meta_buf, user_params);
> > + if (ret < 0)
> > + goto error_release_tables;
> > +
> > + /* Store last buffer */
> > + kmb_meta->last_buf = meta_buf;
> > +
> > + return 0;
> > +
> > +error_release_tables:
> > + kmb_metadata_release_tables(meta_buf);
> > + return ret;
> > +}
> > +
> > +static int kmb_metadata_queue_setup(struct vb2_queue *q,
> > + unsigned int *num_buffers,
> > + unsigned int *num_planes,
> > + unsigned int sizes[],
> > + struct device *alloc_devs[])
> > +{
> > + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(q);
> > +
> > + *num_planes = 1;
> > + sizes[0] = kmb_meta->format.buffersize;
> > +
> > + return 0;
> > +}
> > +
> > +#define to_kmb_meta_buf(vbuf) container_of(vbuf, struct kmb_metadata_buf, vb)
> > +
> > +static int kmb_metadata_buf_params_init(struct vb2_buffer *vb)
> > +{
> > + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue);
> > + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > + struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf);
> > +
> > + buf->type = KMB_METADATA_PARAMS;
> > + buf->params.isp = dma_alloc_coherent(kmb_meta->dma_dev,
> > + sizeof(*buf->params.isp),
> > + &buf->params.dma_addr_isp, 0);
> > + if (!buf->params.isp)
> > + return -ENOMEM;
> > +
> > + memset(buf->params.isp, 0, sizeof(*buf->params.isp));
> > + /*
> > + * Table pools will be allocated per need.
> > + * The pools need to be released when last buffer is finished.
> > + * Use table reference count for that purpose
> > + */
> > + kmb_meta->table_pools_refcnt++;
> > +
> > + return 0;
> > +}
> > +
> > +static int kmb_metadata_buf_params_prepare(struct vb2_buffer *vb)
> > +{
> > + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue);
> > + struct kmb_isp_params *user_params = vb2_plane_vaddr(vb, 0);
> > + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > + struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf);
> > +
> > + vb2_set_plane_payload(vb, 0, kmb_meta->format.buffersize);
> > + return kmb_metadata_fill_isp_params(kmb_meta, buf, user_params);
> > +}
> > +
> > +static void kmb_metadata_buf_params_cleanup(struct vb2_buffer *vb)
> > +{
> > + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue);
> > + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > + struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf);
> > +
> > + if (buf == kmb_meta->last_buf)
> > + kmb_meta->last_buf = NULL;
> > +
> > + kmb_metadata_release_tables(buf);
> > + dma_free_coherent(kmb_meta->dma_dev, sizeof(*buf->params.isp),
> > + buf->params.isp, buf->params.dma_addr_isp);
> > +
> > + /* Destroy allocated table pools on last finish */
> > + if (kmb_meta->table_pools_refcnt-- == 1)
> > + kmb_metadata_destroy_table_pools(kmb_meta);
> > +}
> > +
> > +static int kmb_metadata_buf_stats_init(struct vb2_buffer *vb)
> > +{
> > + dma_addr_t stats_addr = vb2_dma_contig_plane_dma_addr(vb, 0);
> > + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > + struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf);
> > + int i;
> > +
> > + buf->type = KMB_METADATA_STATS;
> > + memset(&buf->stats.raw, 0, sizeof(buf->stats.raw));
> > + buf->stats.dehaze_stats_addr = 0;
> > +
> > + /* Fill statistics addresses */
> > + for (i = 0; i < KMB_CAM_MAX_EXPOSURES; i++) {
> > + buf->stats.raw[i].ae_awb_stats_addr = stats_addr +
> > + offsetof(struct kmb_isp_stats,
> > + exposure[i].ae_awb_stats[0]);
> > +
> > + buf->stats.raw[i].af_stats_addr = stats_addr +
> > + offsetof(struct kmb_isp_stats,
> > + exposure[i].af_stats[0]);
> > +
> > + buf->stats.raw[i].hist_luma_addr = stats_addr +
> > + offsetof(struct kmb_isp_stats,
> > + exposure[i].hist_luma[0]);
> > +
> > + buf->stats.raw[i].hist_rgb_addr = stats_addr +
> > + offsetof(struct kmb_isp_stats,
> > + exposure[i].hist_rgb[0]);
> > +
> > + buf->stats.raw[i].flicker_rows_addr = stats_addr +
> > + offsetof(struct kmb_isp_stats,
> > + exposure[i].flicker_rows[0]);
> > + }
> > +
> > + buf->stats.dehaze_stats_addr = stats_addr +
> > + offsetof(struct kmb_isp_stats, dehaze);
> > +
> > + return 0;
> > +}
> > +
> > +static int kmb_metadata_buf_stats_prepare(struct vb2_buffer *vb)
> > +{
> > + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue);
> > +
> > + vb2_set_plane_payload(vb, 0, kmb_meta->format.buffersize);
> > +
> > + return 0;
> > +}
> > +
> > +static void kmb_metadata_buf_queue(struct vb2_buffer *vb)
> > +{
> > + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue);
> > + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
> > + struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf);
> > + int ret;
> > +
> > + ret = kmb_meta->queue_ops->queue(kmb_meta->priv, buf);
> > + if (ret)
> > + dev_err(&kmb_meta->video.dev, "Fail metadata queue %d", ret);
> > +}
> > +
> > +static int kmb_metadata_start_streaming(struct vb2_queue *q,
> > + unsigned int count)
> > +{
> > + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(q);
> > + int ret;
> > +
> > + ret = kmb_pipe_prepare(kmb_meta->pipe);
> > + if (ret < 0)
> > + goto error_discard_all_bufs;
> > +
> > + ret = kmb_pipe_run(kmb_meta->pipe, &kmb_meta->video.entity);
> > + if (ret < 0)
> > + goto error_pipeline_stop;
> > +
> > + return 0;
> > +
> > +error_pipeline_stop:
> > + kmb_pipe_stop(kmb_meta->pipe, &kmb_meta->video.entity);
> > +error_discard_all_bufs:
> > + kmb_meta->queue_ops->flush(kmb_meta->priv);
> > + return 0;
> > +}
> > +
> > +static void kmb_metadata_stop_streaming(struct vb2_queue *q)
> > +{
> > + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(q);
> > +
> > + kmb_pipe_stop(kmb_meta->pipe, &kmb_meta->video.entity);
> > +
> > + kmb_meta->queue_ops->flush(kmb_meta->priv);
> > +}
> > +
> > +/* driver-specific operations */
> > +static struct vb2_ops kmb_meta_params_vb2_q_ops = {
> > + .queue_setup = kmb_metadata_queue_setup,
> > + .buf_init = kmb_metadata_buf_params_init,
> > + .buf_prepare = kmb_metadata_buf_params_prepare,
> > + .buf_cleanup = kmb_metadata_buf_params_cleanup,
> > + .start_streaming = kmb_metadata_start_streaming,
> > + .stop_streaming = kmb_metadata_stop_streaming,
> > + .buf_queue = kmb_metadata_buf_queue,
> > +};
> > +
> > +static struct vb2_ops kmb_meta_stats_vb2_q_ops = {
> > + .queue_setup = kmb_metadata_queue_setup,
> > + .buf_init = kmb_metadata_buf_stats_init,
> > + .buf_prepare = kmb_metadata_buf_stats_prepare,
> > + .start_streaming = kmb_metadata_start_streaming,
> > + .stop_streaming = kmb_metadata_stop_streaming,
> > + .buf_queue = kmb_metadata_buf_queue,
> > +};
> > +
> > +#define to_kmb_meta_dev(vdev) container_of(vdev, struct kmb_metadata, video)
> > +
> > +static int kmb_metadata_querycap(struct file *file, void *fh,
> > + struct v4l2_capability *cap)
> > +{
> > + struct v4l2_fh *vfh = file->private_data;
> > + struct kmb_metadata *kmb_meta =
> > + to_kmb_meta_dev(vfh->vdev);
> > +
> > + cap->bus_info[0] = 0;
> > + strscpy(cap->driver, kmb_meta->video.name, sizeof(cap->driver));
> > + snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> > + kmb_meta->video.name);
> > +
> > + return 0;
> > +}
> > +
> > +static int kmb_metadata_get_fmt(struct file *file, void *fh,
> > + struct v4l2_format *f)
> > +{
> > + struct v4l2_fh *vfh = file->private_data;
> > + struct kmb_metadata *kmb_meta =
> > + to_kmb_meta_dev(vfh->vdev);
> > +
> > + f->fmt.meta = kmb_meta->format;
> > +
> > + return 0;
> > +}
> > +
> > +static int kmb_metadata_try_fmt_cap(struct file *file, void *fh,
> > + struct v4l2_format *f)
> > +{
> > + f->fmt.meta.dataformat = V4L2_META_FMT_KMB_STATS;
> > + if (f->fmt.meta.buffersize < sizeof(struct kmb_isp_stats))
> > + f->fmt.meta.buffersize = sizeof(struct kmb_isp_stats);
> > +
> > + return 0;
> > +}
> > +
> > +static int kmb_metadata_set_fmt_cap(struct file *file, void *fh,
> > + struct v4l2_format *f)
> > +{
> > + struct v4l2_fh *vfh = file->private_data;
> > + struct kmb_metadata *kmb_meta =
> > + to_kmb_meta_dev(vfh->vdev);
> > + int ret;
> > +
> > + ret = kmb_metadata_try_fmt_cap(file, fh, f);
> > + if (ret < 0)
> > + return ret;
> > +
> > + kmb_meta->format = f->fmt.meta;
> > +
> > + return 0;
> > +}
> > +
> > +static int kmb_metadata_try_fmt_out(struct file *file, void *fh,
> > + struct v4l2_format *f)
> > +{
> > + f->fmt.meta.dataformat = V4L2_META_FMT_KMB_PARAMS;
> > + if (f->fmt.meta.buffersize < sizeof(struct kmb_isp_params))
> > + f->fmt.meta.buffersize = sizeof(struct kmb_isp_params);
> > +
> > + return 0;
> > +}
> > +
> > +static int kmb_metadata_set_fmt_out(struct file *file, void *fh,
> > + struct v4l2_format *f)
> > +{
> > + struct v4l2_fh *vfh = file->private_data;
> > + struct kmb_metadata *kmb_meta =
> > + to_kmb_meta_dev(vfh->vdev);
> > + int ret;
> > +
> > + ret = kmb_metadata_try_fmt_out(file, fh, f);
> > + if (ret < 0)
> > + return ret;
> > +
> > + kmb_meta->format = f->fmt.meta;
> > +
> > + return 0;
> > +}
> > +
> > +/* V4L2 ioctl operations */
> > +static const struct v4l2_ioctl_ops kmb_vid_ioctl_ops = {
> > + .vidioc_querycap = kmb_metadata_querycap,
> > + .vidioc_g_fmt_meta_out = kmb_metadata_get_fmt,
> > + .vidioc_s_fmt_meta_out = kmb_metadata_set_fmt_out,
> > + .vidioc_try_fmt_meta_out = kmb_metadata_try_fmt_out,
> > + .vidioc_g_fmt_meta_cap = kmb_metadata_get_fmt,
> > + .vidioc_s_fmt_meta_cap = kmb_metadata_set_fmt_cap,
> > + .vidioc_try_fmt_meta_cap = kmb_metadata_try_fmt_cap,
> > + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> > + .vidioc_querybuf = vb2_ioctl_querybuf,
> > + .vidioc_qbuf = vb2_ioctl_qbuf,
> > + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> > + .vidioc_streamon = vb2_ioctl_streamon,
> > + .vidioc_streamoff = vb2_ioctl_streamoff,
> > +};
> > +
> > +static int kmb_metadata_open(struct file *file)
> > +{
> > + struct kmb_metadata *kmb_meta = video_drvdata(file);
> > + int ret;
> > +
> > + mutex_lock(&kmb_meta->lock);
> > +
> > + ret = v4l2_fh_open(file);
> > + if (ret) {
> > + mutex_unlock(&kmb_meta->lock);
> > + return ret;
> > + }
> > +
> > + ret = kmb_pipe_request(kmb_meta->pipe);
> > + if (ret < 0)
> > + goto error_fh_release;
> > +
> > + mutex_unlock(&kmb_meta->lock);
> > +
> > + return 0;
> > +
> > +error_fh_release:
> > + _vb2_fop_release(file, NULL);
> > + mutex_unlock(&kmb_meta->lock);
> > + return ret;
> > +}
> > +
> > +static int kmb_metadata_release(struct file *file)
> > +{
> > + struct kmb_metadata *kmb_meta = video_drvdata(file);
> > + int ret;
> > +
> > + mutex_lock(&kmb_meta->lock);
> > +
> > + kmb_pipe_release(kmb_meta->pipe);
> > +
> > + ret = _vb2_fop_release(file, NULL);
> > +
> > + mutex_unlock(&kmb_meta->lock);
> > +
> > + return ret;
> > +}
> > +
> > +/* V4L2 file operations */
> > +static const struct v4l2_file_operations kmb_vid_output_fops = {
> > + .owner = THIS_MODULE,
> > + .unlocked_ioctl = video_ioctl2,
> > + .open = kmb_metadata_open,
> > + .release = kmb_metadata_release,
> > + .poll = vb2_fop_poll,
> > + .mmap = vb2_fop_mmap,
> > +};
> > +
> > /**
> > - * kmb_video_init - Initialize entity
> > + * kmb_metadata_init - Initialize entity
> > * @kmb_meta: pointer to kmb isp config device
> > *
> > * Return: 0 if successful, error code otherwise.
> > */
> > int kmb_metadata_init(struct kmb_metadata *kmb_meta)
> > {
> > + int ret;
> > +
> > + mutex_init(&kmb_meta->lock);
> > +
> > + kmb_meta->table_pools_refcnt = 0;
> > + memset(kmb_meta->table_pool, 0, sizeof(kmb_meta->table_pool));
> > +
> > + kmb_meta->video.fops = &kmb_vid_output_fops;
> > + kmb_meta->video.ioctl_ops = &kmb_vid_ioctl_ops;
> > + kmb_meta->video.minor = -1;
> > + kmb_meta->video.release = video_device_release;
> > + kmb_meta->video.vfl_type = VFL_TYPE_VIDEO;
> > + kmb_meta->video.lock = &kmb_meta->lock;
> > + kmb_meta->video.queue = &kmb_meta->vb2_q;
> > + video_set_drvdata(&kmb_meta->video, kmb_meta);
> > +
> > + kmb_meta->vb2_q.drv_priv = kmb_meta;
> > + kmb_meta->vb2_q.buf_struct_size = sizeof(struct kmb_metadata_buf);
> > + kmb_meta->vb2_q.io_modes = VB2_DMABUF | VB2_MMAP;
> > + kmb_meta->vb2_q.timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> > + kmb_meta->vb2_q.dev = kmb_meta->dma_dev;
> > + kmb_meta->vb2_q.lock = &kmb_meta->lock;
> > + kmb_meta->vb2_q.min_buffers_needed = 1;
> > +
> > + /* Initialize per type variables */
> > + kmb_meta->video.device_caps = V4L2_CAP_STREAMING;
> > + if (kmb_meta->type == KMB_METADATA_PARAMS) {
> > + kmb_meta->video.device_caps |= V4L2_CAP_META_OUTPUT;
> > + kmb_meta->video.vfl_dir = VFL_DIR_TX;
> > + snprintf(kmb_meta->video.name, sizeof(kmb_meta->video.name),
> > + KMB_CAM_METADATA_PARAMS_NAME);
> > +
> > + kmb_meta->vb2_q.ops = &kmb_meta_params_vb2_q_ops;
> > + kmb_meta->vb2_q.mem_ops = &vb2_dma_contig_memops;
> > + kmb_meta->vb2_q.type = V4L2_BUF_TYPE_META_OUTPUT;
> > +
> > + kmb_meta->pad.flags = MEDIA_PAD_FL_SOURCE;
> > +
> > + kmb_meta->format.dataformat = V4L2_META_FMT_KMB_PARAMS;
> > + kmb_meta->format.buffersize = sizeof(struct kmb_isp_params);
> > + } else {
> > + kmb_meta->video.device_caps |= V4L2_CAP_META_CAPTURE;
> > + kmb_meta->video.vfl_dir = VFL_DIR_RX;
> > +
> > + snprintf(kmb_meta->video.name, sizeof(kmb_meta->video.name),
> > + KMB_CAM_METADATA_STATS_NAME);
> > +
> > + kmb_meta->vb2_q.ops = &kmb_meta_stats_vb2_q_ops;
> > + kmb_meta->vb2_q.mem_ops = &vb2_dma_contig_memops;
> > + kmb_meta->vb2_q.type = V4L2_BUF_TYPE_META_CAPTURE;
> > +
> > + kmb_meta->pad.flags = MEDIA_PAD_FL_SINK;
> > +
> > + kmb_meta->format.dataformat = V4L2_META_FMT_KMB_STATS;
> > + kmb_meta->format.buffersize = sizeof(struct kmb_isp_stats);
> > + }
> > +
> > + ret = media_entity_pads_init(&kmb_meta->video.entity,
> > + 1, &kmb_meta->pad);
> > + if (ret < 0)
> > + goto error_mutex_destroy;
> > +
> > + ret = vb2_queue_init(&kmb_meta->vb2_q);
> > + if (ret < 0) {
> > + dev_err(&kmb_meta->video.dev, "Error vb2 queue init");
> > + goto error_metadata_cleanup;
> > + }
> > +
> > + kmb_params_get_defaults(&kmb_meta->def);
> > +
> > return 0;
> > +
> > +error_metadata_cleanup:
> > + kmb_metadata_cleanup(kmb_meta);
> > +error_mutex_destroy:
> > + mutex_destroy(&kmb_meta->lock);
> > +
> > + return ret;
> > }
> >
> > /**
> > @@ -22,7 +1823,10 @@ int kmb_metadata_init(struct kmb_metadata *kmb_meta)
> > * @kmb_meta: pointer to kmb isp config device
> > */
> > void kmb_metadata_cleanup(struct kmb_metadata *kmb_meta)
> > -{ }
> > +{
> > + media_entity_cleanup(&kmb_meta->video.entity);
> > + mutex_destroy(&kmb_meta->lock);
> > +}
> >
> > /**
> > * kmb_metadata_register - Register V4L2 device
> > @@ -34,7 +1838,15 @@ void kmb_metadata_cleanup(struct kmb_metadata *kmb_meta)
> > int kmb_metadata_register(struct kmb_metadata *kmb_meta,
> > struct v4l2_device *v4l2_dev)
> > {
> > - return 0;
> > + int ret;
> > +
> > + kmb_meta->video.v4l2_dev = v4l2_dev;
> > +
> > + ret = video_register_device(&kmb_meta->video, VFL_TYPE_VIDEO, -1);
> > + if (ret < 0)
> > + dev_err(&kmb_meta->video.dev, "Failed to register video device");
> > +
> > + return ret;
> > }
> >
> > /**
> > @@ -42,4 +1854,7 @@ int kmb_metadata_register(struct kmb_metadata *kmb_meta,
> > * @kmb_meta: pointer to kmb isp config device
> > */
> > void kmb_metadata_unregister(struct kmb_metadata *kmb_meta)
> > -{ }
> > +{
> > + mutex_destroy(&kmb_meta->lock);
> > + video_unregister_device(&kmb_meta->video);
> > +}
> > diff --git a/drivers/media/platform/keembay-camera/keembay-metadata.h b/drivers/media/platform/keembay-camera/keembay-
> metadata.h
> > index 88e85d3caba0..ab77ed11bd15 100644
> > --- a/drivers/media/platform/keembay-camera/keembay-metadata.h
> > +++ b/drivers/media/platform/keembay-camera/keembay-metadata.h
> > @@ -12,6 +12,7 @@
> > #include <media/videobuf2-v4l2.h>
> >
> > #include "keembay-vpu-isp.h"
> > +#include "keembay-params-defaults.h"
> >
> > /**
> > * enum kmb_metadata_table_type - Keembay metadata table type
> > @@ -68,12 +69,12 @@ struct kmb_metadata_table {
> > * @vb: Video buffer for v4l2
> > * @type: Metadata type
> > * @stats: Statistics physical addresses
> > - * @raw: VPU raw statistics physical addresses
> > - * @dehaze_stats_addr: VPU dehaze statistics physical address
> > + * @stats.raw: VPU raw statistics physical addresses
> > + * @stats.dehaze_stats_addr: VPU dehaze statistics physical address
> > * @params: VPU ISP parameters
> > - * @isp: VPU ISP parameters virtual address
> > - * @dma_addr_isp: VPU ISP parameters physical address
> > - * @tab: Metadata tables
> > + * @params.isp: VPU ISP parameters virtual address
> > + * @params.dma_addr_isp: VPU ISP parameters physical address
> > + * @params.tab: Metadata tables
> > * @list: List for buffer queue
> > */
> > struct kmb_metadata_buf {
> > @@ -118,6 +119,7 @@ struct kmb_metabuf_queue_ops {
> > * @table_pool: ISP tables dma pool
> > * @last_buf: Pointer to last enqueued buffer
> > * @format: Active format
> > + * @def: Default ISP params
> > */
> > struct kmb_metadata {
> > struct mutex lock;
> > @@ -138,6 +140,8 @@ struct kmb_metadata {
> > struct kmb_metadata_buf *last_buf;
> >
> > struct v4l2_meta_format format;
> > +
> > + struct kmb_vpu_isp_params_defaults def;
> > };
> >
> > int kmb_metadata_init(struct kmb_metadata *kmb_meta);
> > diff --git a/drivers/media/platform/keembay-camera/keembay-params-defaults.c b/drivers/media/platform/keembay-camera/keembay-
> params-defaults.c
> > new file mode 100644
> > index 000000000000..a2dd7888375e
> > --- /dev/null
> > +++ b/drivers/media/platform/keembay-camera/keembay-params-defaults.c
> > @@ -0,0 +1,326 @@
> > +// SPDX-License-Identifier: GPL-2.0-only
> > +/*
> > + * Intel Keem Bay camera ISP parameter defaults.
> > + *
> > + * Copyright (C) 2021 Intel Corporation
> > + */
> > +#include <linux/stddef.h>
> > +#include <linux/types.h>
> > +
> > +#include "keembay-params-defaults.h"
> > +
> > +static const struct kmb_vpu_blc_params blc_default[KMB_VPU_MAX_EXPOSURES] = {
> > + {
> > + .coeff1 = 800,
> > + .coeff2 = 800,
> > + .coeff3 = 800,
> > + .coeff4 = 800,
> > + },
> > + {
> > + .coeff1 = 800,
> > + .coeff2 = 800,
> > + .coeff3 = 800,
> > + .coeff4 = 800,
> > + },
> > + {
> > + .coeff1 = 800,
> > + .coeff2 = 800,
> > + .coeff3 = 800,
> > + .coeff4 = 800,
> > + }
> > +
> > +};
> > +
> > +static const struct kmb_vpu_sigma_dns_params
> > + sigma_dns_default[KMB_VPU_MAX_EXPOSURES] = { 0 };
> > +
> > +static const struct kmb_vpu_lsc_params lsc_default = {
> > + .threshold = 2048,
> > + .width = 64,
> > + .height = 44,
> > + .reserved = { 0 },
> > +};
> > +
> > +static const struct kmb_vpu_raw_params raw_default = {
> > + .awb_stats_en = 0,
> > + .awb_rgb_hist_en = 0,
> > + .af_stats_en = 0,
> > + .luma_hist_en = 0,
> > + .flicker_accum_en = 0,
> > + .bad_pixel_fix_en = 0,
> > + .grgb_imb_en = 1,
> > + .mono_imbalance_en = 0,
> > + .gain1 = 269,
> > + .gain2 = 452,
> > + .gain3 = 634,
> > + .gain4 = 269,
> > + .stop1 = 400,
> > + .stop2 = 450,
> > + .stop3 = 700,
> > + .stop4 = 800,
> > + .threshold1 = 128,
> > + .alpha1 = 12,
> > + .alpha2 = 12,
> > + .alpha3 = 12,
> > + .alpha4 = 12,
> > + .threshold2 = 53,
> > + .static_defect_size = 1,
> > + .reserved = { 0 },
> > + .flicker_first_row_acc = 0,
> > + .flicker_last_row_acc = 0,
> > +};
> > +
> > +static const struct kmb_vpu_ae_awb_params ae_awb_default = {
> > + .start_x = 0,
> > + .start_y = 0,
> > + .width = 100,
> > + .height = 98,
> > + .skip_x = 100,
> > + .skip_y = 98,
> > + .patches_x = 38,
> > + .patches_y = 22,
> > + .threshold1 = 0,
> > + .threshold2 = 4095,
> > +};
> > +
> > +static const struct kmb_vpu_af_params af_default = {
> > + .start_x = 0,
> > + .start_y = 0,
> > + .width = 192,
> > + .height = 144,
> > + .patches_x = 20,
> > + .patches_y = 15,
> > + .coeff = 0,
> > + .threshold1 = 0,
> > + .threshold2 = 0,
> > + .coeffs1 = {31, 19, -32, 31, 63, 31, -50, -35, 35, -70, 35},
> > + .coeffs2 = {35, 11, -29, 8, 17, 8, 78, -39, 119, -238, 119},
> > +};
> > +
> > +static const struct kmb_vpu_hist_params histogram_default = {
> > + .start_x = 0,
> > + .start_y = 0,
> > + .end_x = 3839,
> > + .end_y = 2156,
> > + .matrix = {1719, 0, 0, 0, 1024, 0, 0, 0, 2414},
> > + .weight = {64, 128, 64},
> > +};
> > +
> > +// only address - nothing to init...
>
> /* a comment */
>
Oops, will be fixed
> > +static const struct kmb_vpu_lca_params lca_default = { 0 };
> > +
> > +static const struct kmb_vpu_debayer_params debayer_default = {
> > + .coeff1 = 51,
> > + .multiplier1 = 13107,
> > + .multiplier2 = 13107,
> > + .coeff2 = 77,
> > + .coeff3 = 150,
> > + .coeff4 = 29,
> > +};
> > +
> > +static const struct kmb_vpu_dog_dns_params dog_dns_default = {
> > + .threshold = 0,
> > + .strength = 0,
> > + .coeffs11 = {0, 0, 0, 0, 0, 255},
> > + .coeffs15 = {0, 0, 0, 0, 0, 0, 0, 255},
> > + .reserved = { 0 },
> > +};
> > +
> > +static const struct kmb_vpu_luma_dns_params luma_dns_default = {
> > + .threshold = 13094,
> > + .slope = 967,
> > + .shift = 7,
> > + .alpha = 50,
> > + .weight = 0,
> > + .per_pixel_alpha_en = 0,
> > + .gain_bypass_en = 0,
> > + .reserved = { 0 },
> > +};
> > +
> > +static const struct kmb_vpu_sharpen_params sharpen_default = {
> > + .coeffs1 = {0, 0, 0, 4, 182, 396},
> > + .coeffs2 = {0, 0, 0, 1, 141, 740},
> > + .coeffs3 = {0, 0, 2, 42, 246, 444},
> > + .shift = 15,
> > + .gain1 = 3396,
> > + .gain2 = 3378,
> > + .gain3 = 3270,
> > + .gain4 = 3400,
> > + .gain5 = 207,
> > + .stops1 = {20, 40, 605},
> > + .gains = {10, 120, 60},
> > + .stops2 = {11, 100, 2500, 4000},
> > + .overshoot = 359,
> > + .undershoot = 146,
> > + .alpha = 36,
> > + .gain6 = 128,
> > + .offset = 637,
> > +};
> > +
> > +static const struct kmb_vpu_chroma_gen_params chroma_gen_default = {
> > + .epsilon = 2,
> > + .coeff1 = 426,
> > + .coeff2 = 767,
> > + .coeff3 = 597,
> > + .coeff4 = 77,
> > + .coeff5 = 150,
> > + .coeff6 = 29,
> > + .strength1 = 0,
> > + .strength2 = 32,
> > + .coeffs = {33, 59, 71},
> > + .offset1 = 2,
> > + .slope1 = 230,
> > + .slope2 = 256,
> > + .offset2 = 0,
> > + .limit = 767,
> > +};
> > +
> > +static const struct kmb_vpu_median_params median_default = {
> > + .size = 7,
> > + .slope = 32,
> > + .offset = -19,
> > +};
> > +
> > +static const struct kmb_vpu_chroma_dns_params chroma_dns_default = {
> > + .limit = 255,
> > + .enable = 0,
> > + .threshold1 = 30,
> > + .threshold2 = 30,
> > + .threshold3 = 30,
> > + .threshold4 = 30,
> > + .threshold5 = 45,
> > + .threshold6 = 45,
> > + .threshold7 = 45,
> > + .threshold8 = 45,
> > + .slope1 = 77,
> > + .offset1 = -15,
> > + .slope2 = 255,
> > + .offset2 = 127,
> > + .grey1 = 421,
> > + .grey2 = 758,
> > + .grey3 = 590,
> > + .coeff1 = 52,
> > + .coeff2 = 32,
> > + .coeff3 = 19,
> > +};
> > +
> > +static const struct kmb_vpu_color_comb_params color_comb_default = {
> > + .matrix = {1303, 65427, 65367, 65172, 1463, 65462, 55, 65034, 1459},
> > + .offsets = { 0 },
> > + .coeff1 = 615,
> > + .coeff2 = 342,
> > + .coeff3 = 439,
> > + .reserved = { 0 },
> > + .enable = 0,
> > + .weight1 = 85,
> > + .weight2 = 86,
> > + .weight3 = 85,
> > + .limit1 = 512,
> > + .limit2 = -8192,
> > + .offset1 = 0,
> > + .offset2 = 0,
> > +};
> > +
> > +static const struct kmb_vpu_hdr_params hdr_default = {
> > + .ratio = {256, 256},
> > + .scale = {262143, 262143, 262143},
> > + .offset1 = -3275,
> > + .slope1 = 320,
> > + .offset2 = -3685,
> > + .slope2 = 641,
> > + .offset3 = -4054,
> > + .slope3 = 4095,
> > + .offset4 = 3686,
> > + .gain1 = 16,
> > + .blur1 = {0, 0, 255},
> > + .blur2 = {0, 0, 0, 0, 255},
> > + .contrast1 = 20,
> > + .contrast2 = 16,
> > + .enable1 = 0,
> > + .enable2 = 0,
> > + .offset5 = 0,
> > + .offset6 = 0,
> > + .strength = 0,
> > + .reserved1 = { 0 },
> > + .offset7 = 15,
> > + .shift = 1702133760,
> > + .field1 = 16,
> > + .field2 = 123,
> > + .gain3 = 0,
> > + .min = 0,
> > + .reserved2 = { 0 },
> > +};
> > +
> > +static const struct kmb_vpu_lut_params lut_default = {
> > + .size = 512,
> > + .reserved = { 0 },
> > + .matrix = {262, 516, 100, 3945, 3799, 449, 449, 3720, 4023},
> > + .offsets = {256, 2048, 2048},
> > +};
> > +
> > +static const struct kmb_vpu_tnf_params tnf_default = {
> > + .factor = 179,
> > + .gain = 0,
> > + .offset1 = 217,
> > + .slope1 = 162,
> > + .offset2 = 299,
> > + .slope2 = 121,
> > + .min1 = 0,
> > + .min2 = 40,
> > + .value = 128,
> > + .enable = 0,
> > +};
> > +
> > +static const struct kmb_vpu_dehaze_params dehaze_default = {
> > + .gain1 = 512,
> > + .min = 70,
> > + .strength1 = 0,
> > + .strength2 = 0,
> > + .gain2 = 128,
> > + .saturation = 127,
> > + .value1 = 2048,
> > + .value2 = 2048,
> > + .value3 = 2048,
> > + .filter = {0, 0, 255},
> > +};
> > +
> > +static const struct kmb_vpu_warp_params warp_default = {
> > + .type = 0,
> > + .relative = 0,
> > + .format = 0,
> > + .position = 0,
> > + .reserved = { 0 },
> > + .width = 8,
> > + .height = 4,
> > + .stride = 128,
> > + .enable = 0,
> > + .matrix = {1, 0, 0, 0, 1, 0, 0, 0, 1},
> > + .mode = 1,
> > + .values = {0, 128, 128},
>
> These seem to be default values for the parameters.
>
> Are any of the values above dependent on the image sizes, for instance?
>
Some of them might be resolution dependent, I need to double check. How should we handle
the defaults that are dependent? Should we disable them or initialize with 0
> > +};
> > +
> > +void kmb_params_get_defaults(struct kmb_vpu_isp_params_defaults *defaults)
> > +{
> > + defaults->blc = blc_default;
> > + defaults->sigma_dns = sigma_dns_default;
> > + defaults->lsc = &lsc_default;
> > + defaults->raw = &raw_default;
> > + defaults->ae_awb = &ae_awb_default;
> > + defaults->af = &af_default;
> > + defaults->histogram = &histogram_default;
> > + defaults->lca = &lca_default;
> > + defaults->debayer = &debayer_default;
> > + defaults->dog_dns = &dog_dns_default;
> > + defaults->luma_dns = &luma_dns_default;
> > + defaults->sharpen = &sharpen_default;
> > + defaults->chroma_gen = &chroma_gen_default;
> > + defaults->median = &median_default;
> > + defaults->chroma_dns = &chroma_dns_default;
> > + defaults->color_comb = &color_comb_default;
> > + defaults->hdr = &hdr_default;
> > + defaults->lut = &lut_default;
> > + defaults->tnf = &tnf_default;
> > + defaults->dehaze = &dehaze_default;
> > + defaults->warp = &warp_default;
> > +}
> > +
> > diff --git a/drivers/media/platform/keembay-camera/keembay-params-defaults.h b/drivers/media/platform/keembay-camera/keembay-
> params-defaults.h
> > new file mode 100644
> > index 000000000000..d6134d64be7c
> > --- /dev/null
> > +++ b/drivers/media/platform/keembay-camera/keembay-params-defaults.h
> > @@ -0,0 +1,38 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +/*
> > + * Intel Keem Bay camera ISP parameter defaults.
> > + *
> > + * Copyright (C) 2021 Intel Corporation
> > + */
> > +#ifndef KEEMBAY_DEFAULTS_H
> > +#define KEEMBAY_DEFAULTS_H
> > +
> > +#include "keembay-vpu-isp.h"
> > +
> > +struct kmb_vpu_isp_params_defaults {
> > + const struct kmb_vpu_blc_params *blc;
> > + const struct kmb_vpu_sigma_dns_params *sigma_dns;
> > + const struct kmb_vpu_lsc_params *lsc;
> > + const struct kmb_vpu_raw_params *raw;
> > + const struct kmb_vpu_ae_awb_params *ae_awb;
> > + const struct kmb_vpu_af_params *af;
> > + const struct kmb_vpu_hist_params *histogram;
> > + const struct kmb_vpu_lca_params *lca;
> > + const struct kmb_vpu_debayer_params *debayer;
> > + const struct kmb_vpu_dog_dns_params *dog_dns;
> > + const struct kmb_vpu_luma_dns_params *luma_dns;
> > + const struct kmb_vpu_sharpen_params *sharpen;
> > + const struct kmb_vpu_chroma_gen_params *chroma_gen;
> > + const struct kmb_vpu_median_params *median;
> > + const struct kmb_vpu_chroma_dns_params *chroma_dns;
> > + const struct kmb_vpu_color_comb_params *color_comb;
> > + const struct kmb_vpu_hdr_params *hdr;
> > + const struct kmb_vpu_lut_params *lut;
> > + const struct kmb_vpu_tnf_params *tnf;
> > + const struct kmb_vpu_dehaze_params *dehaze;
> > + const struct kmb_vpu_warp_params *warp;
> > +};
> > +
> > +void kmb_params_get_defaults(struct kmb_vpu_isp_params_defaults *defaults);
> > +
> > +#endif /* KEEMBAY_DEFAULTS_H */
>
> --
> Kind regards,
>
> Sakari Ailus
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH 09/10] media: Keem Bay Camera: Add metadata video node
2021-04-09 14:19 ` Martina Krasteva
@ 2021-04-09 14:36 ` 'Sakari Ailus'
0 siblings, 0 replies; 28+ messages in thread
From: 'Sakari Ailus' @ 2021-04-09 14:36 UTC (permalink / raw)
To: Martina Krasteva
Cc: linux-media, mchehab, robh+dt, devicetree, daniele.alessandrelli,
paul.j.murphy, gjorgjix.rosikopulos
Hi Martina,
On Fri, Apr 09, 2021 at 03:19:20PM +0100, Martina Krasteva wrote:
> Hi Sakari,
>
> Thank you for the review :)
You're welcome!
...
> > > +static const struct kmb_vpu_warp_params warp_default = {
> > > + .type = 0,
> > > + .relative = 0,
> > > + .format = 0,
> > > + .position = 0,
> > > + .reserved = { 0 },
> > > + .width = 8,
> > > + .height = 4,
> > > + .stride = 128,
> > > + .enable = 0,
> > > + .matrix = {1, 0, 0, 0, 1, 0, 0, 0, 1},
> > > + .mode = 1,
> > > + .values = {0, 128, 128},
> >
> > These seem to be default values for the parameters.
> >
> > Are any of the values above dependent on the image sizes, for instance?
> >
> Some of them might be resolution dependent, I need to double check. How
> should we handle the defaults that are dependent? Should we disable them
> or initialize with 0
Good question. I hope there aren't any, but if there are, the values could
be need to be either derived from the V4L2 mbus or pixel format
configuration somehow, i.e. they'd be zeros here.
--
Kind regards,
Sakari Ailus
^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 10/10] media: admin-guide: Add documentation for Keem Bay Camera
2021-03-19 18:06 [PATCH 00/10] Keem Bay Camera Subsystem Martina Krasteva
` (8 preceding siblings ...)
2021-03-19 18:06 ` [PATCH 09/10] media: Keem Bay Camera: Add metadata " Martina Krasteva
@ 2021-03-19 18:06 ` Martina Krasteva
2021-04-16 9:37 ` [PATCH 00/10] Keem Bay Camera Subsystem Laurent Pinchart
10 siblings, 0 replies; 28+ messages in thread
From: Martina Krasteva @ 2021-03-19 18:06 UTC (permalink / raw)
To: linux-media
Cc: mchehab, robh+dt, devicetree, sakari.ailus,
daniele.alessandrelli, paul.j.murphy, gjorgjix.rosikopulos,
martinax.krasteva
From: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
Add Keem Bay Camera sub-system documentation
Signed-off-by: Gjorgji Rosikopulos <gjorgjix.rosikopulos@intel.com>
Signed-off-by: Martina Krasteva <martinax.krasteva@intel.com>
Acked-by: Paul J. Murphy <paul.j.murphy@intel.com>
Acked-by: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
---
Documentation/admin-guide/media/keembay-camera.dot | 12 ++
Documentation/admin-guide/media/keembay-camera.rst | 174 +++++++++++++++++++++
Documentation/admin-guide/media/v4l-drivers.rst | 1 +
MAINTAINERS | 6 +-
4 files changed, 191 insertions(+), 2 deletions(-)
create mode 100644 Documentation/admin-guide/media/keembay-camera.dot
create mode 100644 Documentation/admin-guide/media/keembay-camera.rst
diff --git a/Documentation/admin-guide/media/keembay-camera.dot b/Documentation/admin-guide/media/keembay-camera.dot
new file mode 100644
index 000000000000..60a7a940b11e
--- /dev/null
+++ b/Documentation/admin-guide/media/keembay-camera.dot
@@ -0,0 +1,12 @@
+digraph board {
+ rankdir=TB
+ n00000001 [label="{{} | imx334 1-001a\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000001:port0 -> n00000003:port0 [style=bold]
+ n00000003 [label="{{<port0> 0 | <port1> 1} | keembay-camera-isp\n/dev/v4l-subdev1 | {<port2> 2 | <port3> 3}}", shape=Mrecord, style=filled, fillcolor=green]
+ n00000003:port2 -> n0000000e [style=bold]
+ n00000003:port3 -> n00000014 [style=bold]
+ n00000008 [label="keembay-metadata-params\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
+ n00000008 -> n00000003:port1 [style=bold]
+ n0000000e [label="keembay-metadata-stats\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
+ n00000014 [label="kmb_video kmb-video-capture\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
+}
diff --git a/Documentation/admin-guide/media/keembay-camera.rst b/Documentation/admin-guide/media/keembay-camera.rst
new file mode 100644
index 000000000000..43bffc6c5969
--- /dev/null
+++ b/Documentation/admin-guide/media/keembay-camera.rst
@@ -0,0 +1,174 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+.. include:: <isonum.txt>
+
+===========================
+Intel Keembay camera driver
+===========================
+
+Copyright |copy| 2021 Intel Corporation
+
+Introduction
+============
+
+This file documents the Intel Keem Bay camera driver located under
+drivers/media/platform/keembay-camera.
+
+The current version of the driver supports Intel Keem Bay VPU Camera Subsystem
+found in Intel Keem Bay platform.
+
+The Keem Bay VPU camera receives the raw Bayer data from the sensors
+and outputs the frames in a YUV format, it operates in per-frame mode
+and processing parameters are required for each processed video output.
+
+The Keem Bay Camera driver uses Xlink for communication with remote Keem Bay
+VPU Camera subsystem.
+
+The driver implements V4L2, Media controller and V4L2 subdev interfaces.
+Camera sensor using V4L2 subdev interface in the kernel is supported.
+
+Topology
+========
+.. _keembay_camera_topology_graph:
+
+.. kernel-figure:: keembay_camera.dot
+ :alt: Diagram of the Keem Bay Camera media pipeline topology
+ :align: center
+
+
+The driver has 1 subdevice:
+
+- keembay-camera-isp: ISP subdevice responsible for all ISP operatios.
+ The subdevice supports V4L2_EVENT_FRAME_SYNC event.
+
+The driver has 3 video devices:
+
+- kmb-video-capture: capture device for retrieving processed YUV output.
+- keembay-metadata-stats: metadata capture device for retrieving statistics.
+- keembay-metadata-params: metadata output device that receives processing
+ parameters from userspace.
+
+Device operation
+----------------
+
+The Keem Bay Camera driver is represented as a media device with single
+V4L2 ISP subdev, which provides a V4L2 subdev interface to the user space.
+
+The V4L2 ISP subdev represents a pipe, which can support a maximum of one stream.
+
+The pipe has two source pads and two sink pads for the following purpose:
+
+.. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}|
+
+.. flat-table::
+
+ * - pad
+ - direction
+ - purpose
+
+ * - 0
+ - sink
+ - MIPI CSI-2 input, connected to the sensor subdev
+
+ * - 1
+ - sink
+ - Processing parameters
+
+ * - 2
+ - source
+ - Output processed video stream
+
+ * - 3
+ - source
+ - 3A statistics
+
+Pad 0 is connected to sensor subdev and should receive data in raw Bayer
+format over MIPI CSI-2 receiver.
+
+Pads 1, 2 and 3 are connected to a corresponding V4L2 video interface,
+exposed to userspace as a V4L2 video device node.
+
+With ISP subdev once the input video node keembay-metadata-params
+connected to pad 1 is queued with ISP processing parameters buffer,
+ISP subdev starts processing and produces the video output in
+YUV format and statistics output on respective output node.
+
+At a minimum, all of the video nodes should be enabled and have buffers queued
+to start the processing.
+
+The Keem Bay ISP V4L2 subdev has the following set of video nodes:
+
+Capture video node
+------------------
+
+The frames received by the sensor over MIPI CSI-2 input are processed by the
+Keem Bay ISP and are output to one single video node in YUV format.
+
+Only the multi-planar API is supported. More details can be found at
+:ref:`planar-apis`.
+
+Parameters video node
+---------------------
+
+The parameters video node receives the Keem Bay ISP algorithm parameters [#f1]_
+that are used to configure how the Keem Bay ISP algorithms process the image.
+
+Details on processing parameters specific to the Keem Bay ISP can be found in
+:ref:`v4l2-meta-fmt-params`.
+
+Statistics video node
+---------------------
+
+3A statistics video node is used by the Keem Bay ISP driver to output the
+statistics for the frames that are being processed by the Keem Bay ISP to
+user space applications. User space applications can use this statistics
+data to compute the desired algorithm parameters for the Keem Bay ISP.
+
+
+Configuring the Keem Bay Camera driver
+======================================
+
+The Keem Bay Camera pipeline can be configured using the Media Controller,
+defined at
+:ref:`media_controller`.
+
+Configuring Keem Bay ISP subdev for frame processing
+----------------------------------------------------
+
+The Keem Bay ISP V4L2 subdev has to be configured with media controller APIs
+to have all the video nodes setup correctly.
+
+Let us take "keembay-camera-isp" subdev as an example. We will use
+media-ctl [#f3]_ and yavta [#f2]_ tools for our example.
+Lets assume that we have sensor subdev connected which produces Raw bayer to
+
+./media-ctl -d $MDEV -V "'keembay-camera-isp':0[fmt:SRGGB12_1X12/3840x2160]"
+
+./media-ctl -d $MDEV -V "'keembay-camera-isp':3[fmt:YUYV8_1_5X8/3840x2160]"
+
+
+Now the pipeline is configured and ready to stream. Keem Bay ISP need buffers
+to be queued on the all of the video nodes to start the stream.
+For that purpose we can use multiple instancies of the yavta tool:
+
+yavta --data-prefix -Bmeta-output -c10 -n5 \
+--file=isp-config.bin /dev/video0 &
+
+yavta --data-prefix -Bmeta-capture -c10 -n5 -I \
+--file=frame-#.stat /dev/video1 &
+
+yavta --data-prefix -Bcapture-mplane -c10 -n5 -I -s3840x2160 \
+--file=frame-#.out -f NV12 /dev/video2 &
+
+
+The captured frames will be stored to frame-#.out files and statistics for
+corresponding frames in frame-#.stat files.
+
+References
+==========
+
+.. [#f1] include/uapi/linux/keembay-isp-ctl.h
+
+.. [#f2] http://git.ideasonboard.org/yavta.git
+
+.. [#f3] http://git.ideasonboard.org/?p=media-ctl.git;a=summary
diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst
index 9c7ebe2ca3bd..9b173d5aebfc 100644
--- a/Documentation/admin-guide/media/v4l-drivers.rst
+++ b/Documentation/admin-guide/media/v4l-drivers.rst
@@ -19,6 +19,7 @@ Video4Linux (V4L) driver-specific documentation
imx7
ipu3
ivtv
+ keembay-camera
meye
omap3isp
omap4_camera
diff --git a/MAINTAINERS b/MAINTAINERS
index d90eaf453012..ff30f157b41a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1971,9 +1971,11 @@ M: Daniele Alessandrelli <daniele.alessandrelli@intel.com>
L: linux-media@vger.kernel.org
S: Maintained
T: git git://linuxtv.org/media_tree.git
+F: Documentation/admin-guide/media/keembay-camera.dot
+F: Documentation/admin-guide/media/keembay-camera.rst
F: Documentation/devicetree/bindings/media/intel,keembay-camera.yaml
-F: Documentation/media/uapi/v4l/meta-formats.rst
-F: Documentation/media/uapi/v4l/pixfmt-meta-intel-kmb.rst
+F: Documentation/userspace-api/media/v4l/meta-formats.rst
+F: Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst
F: drivers/media/platform/keembay-camera/
F: include/uapi/linux/keembay-isp-ctl.h
--
2.11.0
^ permalink raw reply related [flat|nested] 28+ messages in thread* Re: [PATCH 00/10] Keem Bay Camera Subsystem
2021-03-19 18:06 [PATCH 00/10] Keem Bay Camera Subsystem Martina Krasteva
` (9 preceding siblings ...)
2021-03-19 18:06 ` [PATCH 10/10] media: admin-guide: Add documentation for Keem Bay Camera Martina Krasteva
@ 2021-04-16 9:37 ` Laurent Pinchart
2021-04-16 11:20 ` Rosikopulos, GjorgjiX
10 siblings, 1 reply; 28+ messages in thread
From: Laurent Pinchart @ 2021-04-16 9:37 UTC (permalink / raw)
To: Martina Krasteva
Cc: linux-media, mchehab, robh+dt, devicetree, sakari.ailus,
daniele.alessandrelli, paul.j.murphy, gjorgjix.rosikopulos
Hi Martina and Gjorgji,
Nice to see a new ISP driver :-)
Before reviewing patches in details, I have a few high-level questions:
- The driver seems to proxy access to the ISP through the VPU firmware.
I assume the VPU is a separate CPU core that controls the hardware
directly. Is that correct ?
- Does this driver support all the features of the ISP, or only the
subset that a particular VPU firmware exposes ? In particular, the ISP
is exposed as an inline block, which no memory buffer between the
CSI-2 receiver and the ISP, and no ability to capture raw frames. How
is one supposed to tune cameras ?
- More documentation is needed for both the device architecture (in
particular a block diagram of the processing pipeline), and the
configuration parameters. Is there ongoing work in this area ?
- Last but not least, we need a reference userspace implementation to
test this driver. I recommend implementing support in libcamera :-)
On Fri, Mar 19, 2021 at 06:06:22PM +0000, Martina Krasteva wrote:
> From: Martina Krasteva <martinax.krasteva@intel.com>
>
> Patch series contains Keem Bay Camera Subsystem driver implementation,
> documentation and devicetree binding document.
>
> Gjorgji Rosikopulos (7):
> media: Keem Bay Camera: Keem Bay camera driver
> media: Keem Bay Camera: Add VPU camera interface
> uapi: Keem Bay ISP Parameters data types
> media: v4l: Add Keem Bay Camera meta buffer formats
> media: Keem Bay Camera: Add ISP sub-device
> media: Keem Bay Camera: Add metadata video node
> media: admin-guide: Add documentation for Keem Bay Camera
>
> Martina Krasteva (3):
> dt-bindings: media: Add bindings for Keem Bay Camera
> media: Keem Bay Camera: Add pipeline support
> media: Keem Bay Camera: Add capture video node
>
> Documentation/admin-guide/media/keembay-camera.dot | 12 +
> Documentation/admin-guide/media/keembay-camera.rst | 174 ++
> Documentation/admin-guide/media/v4l-drivers.rst | 1 +
> .../bindings/media/intel,keembay-camera.yaml | 98 ++
> .../userspace-api/media/v4l/meta-formats.rst | 1 +
> .../media/v4l/pixfmt-meta-intel-kmb.rst | 98 ++
> MAINTAINERS | 14 +
> drivers/media/platform/Kconfig | 1 +
> drivers/media/platform/Makefile | 2 +
> drivers/media/platform/keembay-camera/Kconfig | 11 +
> drivers/media/platform/keembay-camera/Makefile | 5 +
> .../platform/keembay-camera/keembay-cam-xlink.c | 327 ++++
> .../platform/keembay-camera/keembay-cam-xlink.h | 49 +
> .../media/platform/keembay-camera/keembay-camera.c | 287 +++
> .../media/platform/keembay-camera/keembay-camera.h | 43 +
> .../media/platform/keembay-camera/keembay-isp.c | 1397 +++++++++++++++
> .../media/platform/keembay-camera/keembay-isp.h | 136 ++
> .../platform/keembay-camera/keembay-metadata.c | 1860 ++++++++++++++++++++
> .../platform/keembay-camera/keembay-metadata.h | 154 ++
> .../keembay-camera/keembay-params-defaults.c | 326 ++++
> .../keembay-camera/keembay-params-defaults.h | 38 +
> .../platform/keembay-camera/keembay-pipeline.c | 401 +++++
> .../platform/keembay-camera/keembay-pipeline.h | 75 +
> .../media/platform/keembay-camera/keembay-video.c | 922 ++++++++++
> .../media/platform/keembay-camera/keembay-video.h | 74 +
> .../platform/keembay-camera/keembay-vpu-cmd.h | 110 ++
> .../platform/keembay-camera/keembay-vpu-frame.h | 102 ++
> .../platform/keembay-camera/keembay-vpu-isp.h | 724 ++++++++
> .../platform/keembay-camera/keembay-vpu-pipe.h | 110 ++
> .../platform/keembay-camera/keembay-vpu-src.h | 193 ++
> include/uapi/linux/keembay-isp-ctl.h | 796 +++++++++
> include/uapi/linux/videodev2.h | 4 +
> 32 files changed, 8545 insertions(+)
> create mode 100644 Documentation/admin-guide/media/keembay-camera.dot
> create mode 100644 Documentation/admin-guide/media/keembay-camera.rst
> create mode 100644 Documentation/devicetree/bindings/media/intel,keembay-camera.yaml
> create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst
> create mode 100644 drivers/media/platform/keembay-camera/Kconfig
> create mode 100644 drivers/media/platform/keembay-camera/Makefile
> create mode 100644 drivers/media/platform/keembay-camera/keembay-cam-xlink.c
> create mode 100644 drivers/media/platform/keembay-camera/keembay-cam-xlink.h
> create mode 100644 drivers/media/platform/keembay-camera/keembay-camera.c
> create mode 100644 drivers/media/platform/keembay-camera/keembay-camera.h
> create mode 100644 drivers/media/platform/keembay-camera/keembay-isp.c
> create mode 100644 drivers/media/platform/keembay-camera/keembay-isp.h
> create mode 100644 drivers/media/platform/keembay-camera/keembay-metadata.c
> create mode 100644 drivers/media/platform/keembay-camera/keembay-metadata.h
> create mode 100644 drivers/media/platform/keembay-camera/keembay-params-defaults.c
> create mode 100644 drivers/media/platform/keembay-camera/keembay-params-defaults.h
> create mode 100644 drivers/media/platform/keembay-camera/keembay-pipeline.c
> create mode 100644 drivers/media/platform/keembay-camera/keembay-pipeline.h
> create mode 100644 drivers/media/platform/keembay-camera/keembay-video.c
> create mode 100644 drivers/media/platform/keembay-camera/keembay-video.h
> create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-cmd.h
> create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-frame.h
> create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-isp.h
> create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-pipe.h
> create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-src.h
> create mode 100644 include/uapi/linux/keembay-isp-ctl.h
>
>
> base-commit: f00397ee41c79b6155b9b44abd0055b2c0621349
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 28+ messages in thread* Re: [PATCH 00/10] Keem Bay Camera Subsystem
2021-04-16 9:37 ` [PATCH 00/10] Keem Bay Camera Subsystem Laurent Pinchart
@ 2021-04-16 11:20 ` Rosikopulos, GjorgjiX
0 siblings, 0 replies; 28+ messages in thread
From: Rosikopulos, GjorgjiX @ 2021-04-16 11:20 UTC (permalink / raw)
To: Laurent Pinchart, Martina Krasteva
Cc: linux-media, mchehab, robh+dt, devicetree, sakari.ailus,
daniele.alessandrelli, paul.j.murphy
Hi Laurent,
On 16/04/2021 09:37, Laurent Pinchart wrote:
> Hi Martina and Gjorgji,
>
> Nice to see a new ISP driver :-)
Pleasure for us submit new ISP driver :-).
>
> Before reviewing patches in details, I have a few high-level questions:
>
> - The driver seems to proxy access to the ISP through the VPU firmware.
> I assume the VPU is a separate CPU core that controls the hardware
> directly. Is that correct ?
Yes that is correct.
>
> - Does this driver support all the features of the ISP, or only the
> subset that a particular VPU firmware exposes ? In particular, the ISP
> is exposed as an inline block, which no memory buffer between the
> CSI-2 receiver and the ISP, and no ability to capture raw frames. How
> is one supposed to tune cameras ?
The driver exposes all the features supported by ISP the firmware as
high level API.
However this patch-set is not exposing all the features of the VPU API
in userspace. Some of them will came in next patch-set some of them
are part of the separate patch-set (HDR 2DOL and 3DOL support).
The changes which will be posted in next patch-set are:
1. Raw capture support (link can be activated per need).
2. Support for 2 additional scaled outputs which again their links can
be activated per need.
3. The full size output which is included in this patch-set will remain
immutable active.
The additional features which will be added as separate patch-set and
RFC are:
4. 2DOL and 3DOL support. We dont have yet interface for multiple
streams over one link. We will use current RFC for that. (That why this
is not included in first patch-set).
Also we need to discuss how to enable raw capture for 2DOL and 3DOL
usecases....
Regarding the VPU operation. The VPU captures frames from CSI2 in to
memory and then uses ISP mem2mem processing, there is internal memory
pool used by the firmware.
When raw capture is enabled the raw buffer pool is allocated in the
linux driver side and used by VPU for storing the frames to memory, then
they are processed
by ISP and returned to linux driver side.
>
> - More documentation is needed for both the device architecture (in
> particular a block diagram of the processing pipeline), and the
> configuration parameters. Is there ongoing work in this area ?
Yes this is painful part. We are working to get approval for providing
the needed documentation...
>
> - Last but not least, we need a reference userspace implementation to
> test this driver. I recommend implementing support in libcamera :-)
So i am not sure if this is a planned effort. Currently we are using
yavta for doing tests same as IPU3 example,
with prepared configurations and read them from file. I agree that it
will be great to have libcamera support but
for now is not part of our up-streaming effort :/
Regards,
~Gjorgji
>
> On Fri, Mar 19, 2021 at 06:06:22PM +0000, Martina Krasteva wrote:
>> From: Martina Krasteva <martinax.krasteva@intel.com>
>>
>> Patch series contains Keem Bay Camera Subsystem driver implementation,
>> documentation and devicetree binding document.
>>
>> Gjorgji Rosikopulos (7):
>> media: Keem Bay Camera: Keem Bay camera driver
>> media: Keem Bay Camera: Add VPU camera interface
>> uapi: Keem Bay ISP Parameters data types
>> media: v4l: Add Keem Bay Camera meta buffer formats
>> media: Keem Bay Camera: Add ISP sub-device
>> media: Keem Bay Camera: Add metadata video node
>> media: admin-guide: Add documentation for Keem Bay Camera
>>
>> Martina Krasteva (3):
>> dt-bindings: media: Add bindings for Keem Bay Camera
>> media: Keem Bay Camera: Add pipeline support
>> media: Keem Bay Camera: Add capture video node
>>
>> Documentation/admin-guide/media/keembay-camera.dot | 12 +
>> Documentation/admin-guide/media/keembay-camera.rst | 174 ++
>> Documentation/admin-guide/media/v4l-drivers.rst | 1 +
>> .../bindings/media/intel,keembay-camera.yaml | 98 ++
>> .../userspace-api/media/v4l/meta-formats.rst | 1 +
>> .../media/v4l/pixfmt-meta-intel-kmb.rst | 98 ++
>> MAINTAINERS | 14 +
>> drivers/media/platform/Kconfig | 1 +
>> drivers/media/platform/Makefile | 2 +
>> drivers/media/platform/keembay-camera/Kconfig | 11 +
>> drivers/media/platform/keembay-camera/Makefile | 5 +
>> .../platform/keembay-camera/keembay-cam-xlink.c | 327 ++++
>> .../platform/keembay-camera/keembay-cam-xlink.h | 49 +
>> .../media/platform/keembay-camera/keembay-camera.c | 287 +++
>> .../media/platform/keembay-camera/keembay-camera.h | 43 +
>> .../media/platform/keembay-camera/keembay-isp.c | 1397 +++++++++++++++
>> .../media/platform/keembay-camera/keembay-isp.h | 136 ++
>> .../platform/keembay-camera/keembay-metadata.c | 1860 ++++++++++++++++++++
>> .../platform/keembay-camera/keembay-metadata.h | 154 ++
>> .../keembay-camera/keembay-params-defaults.c | 326 ++++
>> .../keembay-camera/keembay-params-defaults.h | 38 +
>> .../platform/keembay-camera/keembay-pipeline.c | 401 +++++
>> .../platform/keembay-camera/keembay-pipeline.h | 75 +
>> .../media/platform/keembay-camera/keembay-video.c | 922 ++++++++++
>> .../media/platform/keembay-camera/keembay-video.h | 74 +
>> .../platform/keembay-camera/keembay-vpu-cmd.h | 110 ++
>> .../platform/keembay-camera/keembay-vpu-frame.h | 102 ++
>> .../platform/keembay-camera/keembay-vpu-isp.h | 724 ++++++++
>> .../platform/keembay-camera/keembay-vpu-pipe.h | 110 ++
>> .../platform/keembay-camera/keembay-vpu-src.h | 193 ++
>> include/uapi/linux/keembay-isp-ctl.h | 796 +++++++++
>> include/uapi/linux/videodev2.h | 4 +
>> 32 files changed, 8545 insertions(+)
>> create mode 100644 Documentation/admin-guide/media/keembay-camera.dot
>> create mode 100644 Documentation/admin-guide/media/keembay-camera.rst
>> create mode 100644 Documentation/devicetree/bindings/media/intel,keembay-camera.yaml
>> create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-meta-intel-kmb.rst
>> create mode 100644 drivers/media/platform/keembay-camera/Kconfig
>> create mode 100644 drivers/media/platform/keembay-camera/Makefile
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-cam-xlink.c
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-cam-xlink.h
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-camera.c
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-camera.h
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-isp.c
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-isp.h
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-metadata.c
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-metadata.h
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-params-defaults.c
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-params-defaults.h
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-pipeline.c
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-pipeline.h
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-video.c
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-video.h
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-cmd.h
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-frame.h
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-isp.h
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-pipe.h
>> create mode 100644 drivers/media/platform/keembay-camera/keembay-vpu-src.h
>> create mode 100644 include/uapi/linux/keembay-isp-ctl.h
>>
>>
>> base-commit: f00397ee41c79b6155b9b44abd0055b2c0621349
^ permalink raw reply [flat|nested] 28+ messages in thread