[PATCH v2 00/10] Sunxi-Cedrus driver for the Allwinner Video Engine, using media requests

[PATCH v2 00/10] Sunxi-Cedrus driver for the Allwinner Video Engine, using media requests

[Paul Kocialkowski]

This presents a second iteration of the updated Sunxi-Cedrus driver,
that supports the Video Engine found in most Allwinner SoCs, starting
with the A10. It was tested on both the A20 and the A33.

The initial version of this driver[0] was originally written and
submitted by Florent Revest using a previous version of the request API
that is necessary to provide coherency between controls and the buffers
they apply to.

The driver was adapted to use the latest version of the media request
API[1], as submitted by Hand Verkuil. Media request API support is a
hard requirement for the Sunxi-Cedrus driver.

This series also contains fixes for issues encountered with the current
version of the request API. If accepted, these should eventually be
squashed into the request API series.

The driver itself currently only supports MPEG2 and more codecs will be
added to the driver eventually. The output frames provided by the
Video Engine are in a multi-planar 32x32-tiled YUV format, with a plane
for luminance (Y) and a plane for chrominance (UV). A specific format is
introduced in the V4L2 API to describe it.

This implementation is based on the significant work that was conducted
by various members of the linux-sunxi community for understanding and
documenting the Video Engine's innards.

Changes since v1:
* use the latest version of the request API for Hans Verkuil;
* added media controller support and dependency
* renamed v4l2 format to the more explicit V4L2_PIX_FMT_MB32_NV12;
* reworked bindings documentation;
* moved driver to drivers/media/platforms/sunxi/cedrus to pair with
  incoming CSI support ;
* added a workqueue and lists to schedule buffer completion, since it
  cannot be done in interrupt context;
* split mpeg2 support into a setup and a trigger function to avoid race
  condition;
* split video-related ops to a dedicated sunxi_cedrus_video file;
* cleaned up the included headers for each file;
* used device PFN offset instead of subtracting PHYS_BASE;
* used reset_control_reset instead of assert+deassert;
* put the device in reset when removing driver;
* changed dt bindings to use the last 96 Mib of the first 256 MiB of
  DRAM;
* made it clear in the mpeg frame header structure that forward and
  backward indexes are used as reference frames for motion vectors;
* lots of small cosmetic and consistency changes, including naming
  harmonization and headers text rework.

Remaining tasks:
* using the dedicated SRAM controller driver;
* cleaning up registers description and documenting the fields used;
* removing the assigned-clocks property and setting the clock rate
  in the driver directly;
* testing on more platforms.

Cheers!

Paul Kocialkowski (10):
  media: v4l2-ctrls: Add missing v4l2 ctrl unlock
  media-request: Add a request complete operation to allow m2m
    scheduling
  videobuf2-core: Add helper to get buffer private data from media
    request
  media: vim2m: Implement media request complete op to schedule m2m run
  media: v4l: Add definitions for MPEG2 frame format and header metadata
  media: v4l: Add definition for Allwinner's MB32-tiled NV12 format
  media: platform: Add Sunxi-Cedrus VPU decoder driver
  dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver
  ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes
  ARM: dts: sun8i-a33: Add Video Engine and reserved memory nodes

 .../devicetree/bindings/media/sunxi-cedrus.txt     |  50 +++
 arch/arm/boot/dts/sun7i-a20.dtsi                   |  31 ++
 arch/arm/boot/dts/sun8i-a33.dtsi                   |  38 ++
 drivers/media/common/videobuf2/videobuf2-core.c    |  15 +
 drivers/media/media-request.c                      |   3 +
 drivers/media/platform/Kconfig                     |  15 +
 drivers/media/platform/Makefile                    |   1 +
 drivers/media/platform/sunxi/cedrus/Makefile       |   4 +
 drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c | 292 ++++++++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_common.h    | 105 +++++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.c | 228 ++++++++++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.h |  36 ++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.c  | 201 +++++++++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.h  |  29 ++
 .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c     | 157 +++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h     |  33 ++
 .../platform/sunxi/cedrus/sunxi_cedrus_regs.h      | 172 +++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_video.c     | 497 +++++++++++++++++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_video.h     |  31 ++
 drivers/media/platform/vim2m.c                     |  12 +
 drivers/media/v4l2-core/v4l2-ctrls.c               |  17 +-
 drivers/media/v4l2-core/v4l2-ioctl.c               |   1 +
 include/media/media-device.h                       |   2 +
 include/media/videobuf2-core.h                     |   1 +
 include/uapi/linux/v4l2-controls.h                 |  26 ++
 include/uapi/linux/videodev2.h                     |   4 +
 26 files changed, 2000 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/media/sunxi-cedrus.txt
 create mode 100644 drivers/media/platform/sunxi/cedrus/Makefile
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.h

-- 
2.16.3

[PATCH v2 00/10] Sunxi-Cedrus driver for the Allwinner Video Engine, using media requests

[Paul Kocialkowski]

This presents a second iteration of the updated Sunxi-Cedrus driver,
that supports the Video Engine found in most Allwinner SoCs, starting
with the A10. It was tested on both the A20 and the A33.

The initial version of this driver[0] was originally written and
submitted by Florent Revest using a previous version of the request API
that is necessary to provide coherency between controls and the buffers
they apply to.

The driver was adapted to use the latest version of the media request
API[1], as submitted by Hand Verkuil. Media request API support is a
hard requirement for the Sunxi-Cedrus driver.

This series also contains fixes for issues encountered with the current
version of the request API. If accepted, these should eventually be
squashed into the request API series.

The driver itself currently only supports MPEG2 and more codecs will be
added to the driver eventually. The output frames provided by the
Video Engine are in a multi-planar 32x32-tiled YUV format, with a plane
for luminance (Y) and a plane for chrominance (UV). A specific format is
introduced in the V4L2 API to describe it.

This implementation is based on the significant work that was conducted
by various members of the linux-sunxi community for understanding and
documenting the Video Engine's innards.

Changes since v1:
* use the latest version of the request API for Hans Verkuil;
* added media controller support and dependency
* renamed v4l2 format to the more explicit V4L2_PIX_FMT_MB32_NV12;
* reworked bindings documentation;
* moved driver to drivers/media/platforms/sunxi/cedrus to pair with
  incoming CSI support ;
* added a workqueue and lists to schedule buffer completion, since it
  cannot be done in interrupt context;
* split mpeg2 support into a setup and a trigger function to avoid race
  condition;
* split video-related ops to a dedicated sunxi_cedrus_video file;
* cleaned up the included headers for each file;
* used device PFN offset instead of subtracting PHYS_BASE;
* used reset_control_reset instead of assert+deassert;
* put the device in reset when removing driver;
* changed dt bindings to use the last 96 Mib of the first 256 MiB of
  DRAM;
* made it clear in the mpeg frame header structure that forward and
  backward indexes are used as reference frames for motion vectors;
* lots of small cosmetic and consistency changes, including naming
  harmonization and headers text rework.

Remaining tasks:
* using the dedicated SRAM controller driver;
* cleaning up registers description and documenting the fields used;
* removing the assigned-clocks property and setting the clock rate
  in the driver directly;
* testing on more platforms.

Cheers!

Paul Kocialkowski (10):
  media: v4l2-ctrls: Add missing v4l2 ctrl unlock
  media-request: Add a request complete operation to allow m2m
    scheduling
  videobuf2-core: Add helper to get buffer private data from media
    request
  media: vim2m: Implement media request complete op to schedule m2m run
  media: v4l: Add definitions for MPEG2 frame format and header metadata
  media: v4l: Add definition for Allwinner's MB32-tiled NV12 format
  media: platform: Add Sunxi-Cedrus VPU decoder driver
  dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver
  ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes
  ARM: dts: sun8i-a33: Add Video Engine and reserved memory nodes

 .../devicetree/bindings/media/sunxi-cedrus.txt     |  50 +++
 arch/arm/boot/dts/sun7i-a20.dtsi                   |  31 ++
 arch/arm/boot/dts/sun8i-a33.dtsi                   |  38 ++
 drivers/media/common/videobuf2/videobuf2-core.c    |  15 +
 drivers/media/media-request.c                      |   3 +
 drivers/media/platform/Kconfig                     |  15 +
 drivers/media/platform/Makefile                    |   1 +
 drivers/media/platform/sunxi/cedrus/Makefile       |   4 +
 drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c | 292 ++++++++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_common.h    | 105 +++++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.c | 228 ++++++++++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.h |  36 ++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.c  | 201 +++++++++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.h  |  29 ++
 .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c     | 157 +++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h     |  33 ++
 .../platform/sunxi/cedrus/sunxi_cedrus_regs.h      | 172 +++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_video.c     | 497 +++++++++++++++++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_video.h     |  31 ++
 drivers/media/platform/vim2m.c                     |  12 +
 drivers/media/v4l2-core/v4l2-ctrls.c               |  17 +-
 drivers/media/v4l2-core/v4l2-ioctl.c               |   1 +
 include/media/media-device.h                       |   2 +
 include/media/videobuf2-core.h                     |   1 +
 include/uapi/linux/v4l2-controls.h                 |  26 ++
 include/uapi/linux/videodev2.h                     |   4 +
 26 files changed, 2000 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/media/sunxi-cedrus.txt
 create mode 100644 drivers/media/platform/sunxi/cedrus/Makefile
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.h

-- 
2.16.3

[PATCH v2 00/10] Sunxi-Cedrus driver for the Allwinner Video Engine, using media requests

[Paul Kocialkowski]

This presents a second iteration of the updated Sunxi-Cedrus driver,
that supports the Video Engine found in most Allwinner SoCs, starting
with the A10. It was tested on both the A20 and the A33.

The initial version of this driver[0] was originally written and
submitted by Florent Revest using a previous version of the request API
that is necessary to provide coherency between controls and the buffers
they apply to.

The driver was adapted to use the latest version of the media request
API[1], as submitted by Hand Verkuil. Media request API support is a
hard requirement for the Sunxi-Cedrus driver.

This series also contains fixes for issues encountered with the current
version of the request API. If accepted, these should eventually be
squashed into the request API series.

The driver itself currently only supports MPEG2 and more codecs will be
added to the driver eventually. The output frames provided by the
Video Engine are in a multi-planar 32x32-tiled YUV format, with a plane
for luminance (Y) and a plane for chrominance (UV). A specific format is
introduced in the V4L2 API to describe it.

This implementation is based on the significant work that was conducted
by various members of the linux-sunxi community for understanding and
documenting the Video Engine's innards.

Changes since v1:
* use the latest version of the request API for Hans Verkuil;
* added media controller support and dependency
* renamed v4l2 format to the more explicit V4L2_PIX_FMT_MB32_NV12;
* reworked bindings documentation;
* moved driver to drivers/media/platforms/sunxi/cedrus to pair with
  incoming CSI support ;
* added a workqueue and lists to schedule buffer completion, since it
  cannot be done in interrupt context;
* split mpeg2 support into a setup and a trigger function to avoid race
  condition;
* split video-related ops to a dedicated sunxi_cedrus_video file;
* cleaned up the included headers for each file;
* used device PFN offset instead of subtracting PHYS_BASE;
* used reset_control_reset instead of assert+deassert;
* put the device in reset when removing driver;
* changed dt bindings to use the last 96 Mib of the first 256 MiB of
  DRAM;
* made it clear in the mpeg frame header structure that forward and
  backward indexes are used as reference frames for motion vectors;
* lots of small cosmetic and consistency changes, including naming
  harmonization and headers text rework.

Remaining tasks:
* using the dedicated SRAM controller driver;
* cleaning up registers description and documenting the fields used;
* removing the assigned-clocks property and setting the clock rate
  in the driver directly;
* testing on more platforms.

Cheers!

Paul Kocialkowski (10):
  media: v4l2-ctrls: Add missing v4l2 ctrl unlock
  media-request: Add a request complete operation to allow m2m
    scheduling
  videobuf2-core: Add helper to get buffer private data from media
    request
  media: vim2m: Implement media request complete op to schedule m2m run
  media: v4l: Add definitions for MPEG2 frame format and header metadata
  media: v4l: Add definition for Allwinner's MB32-tiled NV12 format
  media: platform: Add Sunxi-Cedrus VPU decoder driver
  dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver
  ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes
  ARM: dts: sun8i-a33: Add Video Engine and reserved memory nodes

 .../devicetree/bindings/media/sunxi-cedrus.txt     |  50 +++
 arch/arm/boot/dts/sun7i-a20.dtsi                   |  31 ++
 arch/arm/boot/dts/sun8i-a33.dtsi                   |  38 ++
 drivers/media/common/videobuf2/videobuf2-core.c    |  15 +
 drivers/media/media-request.c                      |   3 +
 drivers/media/platform/Kconfig                     |  15 +
 drivers/media/platform/Makefile                    |   1 +
 drivers/media/platform/sunxi/cedrus/Makefile       |   4 +
 drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c | 292 ++++++++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_common.h    | 105 +++++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.c | 228 ++++++++++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.h |  36 ++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.c  | 201 +++++++++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.h  |  29 ++
 .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c     | 157 +++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h     |  33 ++
 .../platform/sunxi/cedrus/sunxi_cedrus_regs.h      | 172 +++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_video.c     | 497 +++++++++++++++++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_video.h     |  31 ++
 drivers/media/platform/vim2m.c                     |  12 +
 drivers/media/v4l2-core/v4l2-ctrls.c               |  17 +-
 drivers/media/v4l2-core/v4l2-ioctl.c               |   1 +
 include/media/media-device.h                       |   2 +
 include/media/videobuf2-core.h                     |   1 +
 include/uapi/linux/v4l2-controls.h                 |  26 ++
 include/uapi/linux/videodev2.h                     |   4 +
 26 files changed, 2000 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/media/sunxi-cedrus.txt
 create mode 100644 drivers/media/platform/sunxi/cedrus/Makefile
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.h

-- 
2.16.3

[PATCH v2 01/10] media: v4l2-ctrls: Add missing v4l2 ctrl unlock

[Paul Kocialkowski]

This adds a missing v4l2_ctrl_unlock call that is required to avoid
deadlocks.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
---
 drivers/media/v4l2-core/v4l2-ctrls.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index f67e9f5531fa..ba05a8b9a095 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -3614,10 +3614,12 @@ void v4l2_ctrl_request_complete(struct media_request *req,
 			continue;
 
 		v4l2_ctrl_lock(ctrl);
+
 		if (ref->req)
 			ptr_to_ptr(ctrl, ref->req->p_req, ref->p_req);
 		else
 			ptr_to_ptr(ctrl, ctrl->p_cur, ref->p_req);
+
 		v4l2_ctrl_unlock(ctrl);
 	}
 
@@ -3677,8 +3679,11 @@ void v4l2_ctrl_request_setup(struct media_request *req,
 				}
 			}
 		}
-		if (!have_new_data)
+
+		if (!have_new_data) {
+			v4l2_ctrl_unlock(master);
 			continue;
+		}
 
 		for (i = 0; i < master->ncontrols; i++) {
 			if (master->cluster[i]) {
-- 
2.16.3

[PATCH v2 01/10] media: v4l2-ctrls: Add missing v4l2 ctrl unlock

[Paul Kocialkowski]

This adds a missing v4l2_ctrl_unlock call that is required to avoid
deadlocks.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
---
 drivers/media/v4l2-core/v4l2-ctrls.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index f67e9f5531fa..ba05a8b9a095 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -3614,10 +3614,12 @@ void v4l2_ctrl_request_complete(struct media_request *req,
 			continue;
 
 		v4l2_ctrl_lock(ctrl);
+
 		if (ref->req)
 			ptr_to_ptr(ctrl, ref->req->p_req, ref->p_req);
 		else
 			ptr_to_ptr(ctrl, ctrl->p_cur, ref->p_req);
+
 		v4l2_ctrl_unlock(ctrl);
 	}
 
@@ -3677,8 +3679,11 @@ void v4l2_ctrl_request_setup(struct media_request *req,
 				}
 			}
 		}
-		if (!have_new_data)
+
+		if (!have_new_data) {
+			v4l2_ctrl_unlock(master);
 			continue;
+		}
 
 		for (i = 0; i < master->ncontrols; i++) {
 			if (master->cluster[i]) {
-- 
2.16.3

[PATCH v2 01/10] media: v4l2-ctrls: Add missing v4l2 ctrl unlock

[Paul Kocialkowski]

This adds a missing v4l2_ctrl_unlock call that is required to avoid
deadlocks.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
---
 drivers/media/v4l2-core/v4l2-ctrls.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index f67e9f5531fa..ba05a8b9a095 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -3614,10 +3614,12 @@ void v4l2_ctrl_request_complete(struct media_request *req,
 			continue;
 
 		v4l2_ctrl_lock(ctrl);
+
 		if (ref->req)
 			ptr_to_ptr(ctrl, ref->req->p_req, ref->p_req);
 		else
 			ptr_to_ptr(ctrl, ctrl->p_cur, ref->p_req);
+
 		v4l2_ctrl_unlock(ctrl);
 	}
 
@@ -3677,8 +3679,11 @@ void v4l2_ctrl_request_setup(struct media_request *req,
 				}
 			}
 		}
-		if (!have_new_data)
+
+		if (!have_new_data) {
+			v4l2_ctrl_unlock(master);
 			continue;
+		}
 
 		for (i = 0; i < master->ncontrols; i++) {
 			if (master->cluster[i]) {
-- 
2.16.3

[PATCH v2 02/10] media-request: Add a request complete operation to allow m2m scheduling

[Paul Kocialkowski]

When using the request API in the context of a m2m driver, the
operations that come with a m2m run scheduling call in their
(m2m-specific) ioctl handler are delayed until the request is queued
(for instance, this includes queuing buffers and streamon).

Thus, the m2m run scheduling calls are not called in due time since the
request AP's internal plumbing will (rightfully) use the relevant core
functions directly instead of the ioctl handler.

This ends up in a situation where nothing happens if there is no
run-scheduling ioctl called after queuing the request.

In order to circumvent the issue, a new media operation is introduced,
called at the time of handling the media request queue ioctl. It gives
m2m drivers a chance to schedule a m2m device run at that time.

The existing req_queue operation cannot be used for this purpose, since
it is called with the request queue mutex held, that is eventually needed
in the device_run call to apply relevant controls.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
---
 drivers/media/media-request.c | 3 +++
 include/media/media-device.h  | 2 ++
 2 files changed, 5 insertions(+)

diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
index 415f7e31019d..28ac5ccfe6a2 100644
--- a/drivers/media/media-request.c
+++ b/drivers/media/media-request.c
@@ -157,6 +157,9 @@ static long media_request_ioctl_queue(struct media_request *req)
 		media_request_get(req);
 	}
 
+	if (mdev->ops->req_complete)
+		mdev->ops->req_complete(req);
+
 	return ret;
 }
 
diff --git a/include/media/media-device.h b/include/media/media-device.h
index 07e323c57202..c7dcf2079cc9 100644
--- a/include/media/media-device.h
+++ b/include/media/media-device.h
@@ -55,6 +55,7 @@ struct media_entity_notify {
  * @req_alloc: Allocate a request
  * @req_free: Free a request
  * @req_queue: Queue a request
+ * @req_complete: Complete a request
  */
 struct media_device_ops {
 	int (*link_notify)(struct media_link *link, u32 flags,
@@ -62,6 +63,7 @@ struct media_device_ops {
 	struct media_request *(*req_alloc)(struct media_device *mdev);
 	void (*req_free)(struct media_request *req);
 	int (*req_queue)(struct media_request *req);
+	void (*req_complete)(struct media_request *req);
 };
 
 /**
-- 
2.16.3

[PATCH v2 02/10] media-request: Add a request complete operation to allow m2m scheduling

[Paul Kocialkowski]

When using the request API in the context of a m2m driver, the
operations that come with a m2m run scheduling call in their
(m2m-specific) ioctl handler are delayed until the request is queued
(for instance, this includes queuing buffers and streamon).

Thus, the m2m run scheduling calls are not called in due time since the
request AP's internal plumbing will (rightfully) use the relevant core
functions directly instead of the ioctl handler.

This ends up in a situation where nothing happens if there is no
run-scheduling ioctl called after queuing the request.

In order to circumvent the issue, a new media operation is introduced,
called at the time of handling the media request queue ioctl. It gives
m2m drivers a chance to schedule a m2m device run at that time.

The existing req_queue operation cannot be used for this purpose, since
it is called with the request queue mutex held, that is eventually needed
in the device_run call to apply relevant controls.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
---
 drivers/media/media-request.c | 3 +++
 include/media/media-device.h  | 2 ++
 2 files changed, 5 insertions(+)

diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
index 415f7e31019d..28ac5ccfe6a2 100644
--- a/drivers/media/media-request.c
+++ b/drivers/media/media-request.c
@@ -157,6 +157,9 @@ static long media_request_ioctl_queue(struct media_request *req)
 		media_request_get(req);
 	}
 
+	if (mdev->ops->req_complete)
+		mdev->ops->req_complete(req);
+
 	return ret;
 }
 
diff --git a/include/media/media-device.h b/include/media/media-device.h
index 07e323c57202..c7dcf2079cc9 100644
--- a/include/media/media-device.h
+++ b/include/media/media-device.h
@@ -55,6 +55,7 @@ struct media_entity_notify {
  * @req_alloc: Allocate a request
  * @req_free: Free a request
  * @req_queue: Queue a request
+ * @req_complete: Complete a request
  */
 struct media_device_ops {
 	int (*link_notify)(struct media_link *link, u32 flags,
@@ -62,6 +63,7 @@ struct media_device_ops {
 	struct media_request *(*req_alloc)(struct media_device *mdev);
 	void (*req_free)(struct media_request *req);
 	int (*req_queue)(struct media_request *req);
+	void (*req_complete)(struct media_request *req);
 };
 
 /**
-- 
2.16.3

[PATCH v2 02/10] media-request: Add a request complete operation to allow m2m scheduling

[Paul Kocialkowski]

When using the request API in the context of a m2m driver, the
operations that come with a m2m run scheduling call in their
(m2m-specific) ioctl handler are delayed until the request is queued
(for instance, this includes queuing buffers and streamon).

Thus, the m2m run scheduling calls are not called in due time since the
request AP's internal plumbing will (rightfully) use the relevant core
functions directly instead of the ioctl handler.

This ends up in a situation where nothing happens if there is no
run-scheduling ioctl called after queuing the request.

In order to circumvent the issue, a new media operation is introduced,
called at the time of handling the media request queue ioctl. It gives
m2m drivers a chance to schedule a m2m device run at that time.

The existing req_queue operation cannot be used for this purpose, since
it is called with the request queue mutex held, that is eventually needed
in the device_run call to apply relevant controls.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
---
 drivers/media/media-request.c | 3 +++
 include/media/media-device.h  | 2 ++
 2 files changed, 5 insertions(+)

diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
index 415f7e31019d..28ac5ccfe6a2 100644
--- a/drivers/media/media-request.c
+++ b/drivers/media/media-request.c
@@ -157,6 +157,9 @@ static long media_request_ioctl_queue(struct media_request *req)
 		media_request_get(req);
 	}
 
+	if (mdev->ops->req_complete)
+		mdev->ops->req_complete(req);
+
 	return ret;
 }
 
diff --git a/include/media/media-device.h b/include/media/media-device.h
index 07e323c57202..c7dcf2079cc9 100644
--- a/include/media/media-device.h
+++ b/include/media/media-device.h
@@ -55,6 +55,7 @@ struct media_entity_notify {
  * @req_alloc: Allocate a request
  * @req_free: Free a request
  * @req_queue: Queue a request
+ * @req_complete: Complete a request
  */
 struct media_device_ops {
 	int (*link_notify)(struct media_link *link, u32 flags,
@@ -62,6 +63,7 @@ struct media_device_ops {
 	struct media_request *(*req_alloc)(struct media_device *mdev);
 	void (*req_free)(struct media_request *req);
 	int (*req_queue)(struct media_request *req);
+	void (*req_complete)(struct media_request *req);
 };
 
 /**
-- 
2.16.3

[PATCH v2 03/10] videobuf2-core: Add helper to get buffer private data from media request

[Paul Kocialkowski]

When calling media operation driver callbacks related to media requests,
only a pointer to the request itself is provided, which is insufficient
to retrieve the driver's context. Since the driver context is usually
set as vb2 queue private data and given that the core can determine
which objects attached to the request are buffers, it is possible to
extract the associated private data for the first buffer found.

This is required in order to access the current m2m context from m2m
drivers' private data in the context of media request operation
callbacks. More specifically, this allows scheduling m2m device runs
from the newly-introduced request complete operation.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
---
 drivers/media/common/videobuf2/videobuf2-core.c | 15 +++++++++++++++
 include/media/videobuf2-core.h                  |  1 +
 2 files changed, 16 insertions(+)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 13c9d9e243dd..6fa46bfc620f 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -1351,6 +1351,21 @@ bool vb2_core_request_has_buffers(struct media_request *req)
 }
 EXPORT_SYMBOL_GPL(vb2_core_request_has_buffers);
 
+void *vb2_core_request_find_buffer_priv(struct media_request *req)
+{
+	struct media_request_object *obj;
+	struct vb2_buffer *vb;
+
+	obj = media_request_object_find(req, &vb2_core_req_ops, NULL);
+	if (!obj)
+		return NULL;
+
+	vb = container_of(obj, struct vb2_buffer, req_obj);
+
+	return vb2_get_drv_priv(vb->vb2_queue);
+}
+EXPORT_SYMBOL_GPL(vb2_core_request_find_buffer_priv);
+
 int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb,
 			 struct media_request *req)
 {
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 032bd1bec555..65c0cf6afb55 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -1153,4 +1153,5 @@ int vb2_verify_memory_type(struct vb2_queue *q,
 		enum vb2_memory memory, unsigned int type);
 
 bool vb2_core_request_has_buffers(struct media_request *req);
+void *vb2_core_request_find_buffer_priv(struct media_request *req);
 #endif /* _MEDIA_VIDEOBUF2_CORE_H */
-- 
2.16.3

[PATCH v2 03/10] videobuf2-core: Add helper to get buffer private data from media request

[Paul Kocialkowski]

When calling media operation driver callbacks related to media requests,
only a pointer to the request itself is provided, which is insufficient
to retrieve the driver's context. Since the driver context is usually
set as vb2 queue private data and given that the core can determine
which objects attached to the request are buffers, it is possible to
extract the associated private data for the first buffer found.

This is required in order to access the current m2m context from m2m
drivers' private data in the context of media request operation
callbacks. More specifically, this allows scheduling m2m device runs
from the newly-introduced request complete operation.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
---
 drivers/media/common/videobuf2/videobuf2-core.c | 15 +++++++++++++++
 include/media/videobuf2-core.h                  |  1 +
 2 files changed, 16 insertions(+)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 13c9d9e243dd..6fa46bfc620f 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -1351,6 +1351,21 @@ bool vb2_core_request_has_buffers(struct media_request *req)
 }
 EXPORT_SYMBOL_GPL(vb2_core_request_has_buffers);
 
+void *vb2_core_request_find_buffer_priv(struct media_request *req)
+{
+	struct media_request_object *obj;
+	struct vb2_buffer *vb;
+
+	obj = media_request_object_find(req, &vb2_core_req_ops, NULL);
+	if (!obj)
+		return NULL;
+
+	vb = container_of(obj, struct vb2_buffer, req_obj);
+
+	return vb2_get_drv_priv(vb->vb2_queue);
+}
+EXPORT_SYMBOL_GPL(vb2_core_request_find_buffer_priv);
+
 int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb,
 			 struct media_request *req)
 {
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 032bd1bec555..65c0cf6afb55 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -1153,4 +1153,5 @@ int vb2_verify_memory_type(struct vb2_queue *q,
 		enum vb2_memory memory, unsigned int type);
 
 bool vb2_core_request_has_buffers(struct media_request *req);
+void *vb2_core_request_find_buffer_priv(struct media_request *req);
 #endif /* _MEDIA_VIDEOBUF2_CORE_H */
-- 
2.16.3

[PATCH v2 03/10] videobuf2-core: Add helper to get buffer private data from media request

[Paul Kocialkowski]

When calling media operation driver callbacks related to media requests,
only a pointer to the request itself is provided, which is insufficient
to retrieve the driver's context. Since the driver context is usually
set as vb2 queue private data and given that the core can determine
which objects attached to the request are buffers, it is possible to
extract the associated private data for the first buffer found.

This is required in order to access the current m2m context from m2m
drivers' private data in the context of media request operation
callbacks. More specifically, this allows scheduling m2m device runs
from the newly-introduced request complete operation.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
---
 drivers/media/common/videobuf2/videobuf2-core.c | 15 +++++++++++++++
 include/media/videobuf2-core.h                  |  1 +
 2 files changed, 16 insertions(+)

diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 13c9d9e243dd..6fa46bfc620f 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -1351,6 +1351,21 @@ bool vb2_core_request_has_buffers(struct media_request *req)
 }
 EXPORT_SYMBOL_GPL(vb2_core_request_has_buffers);
 
+void *vb2_core_request_find_buffer_priv(struct media_request *req)
+{
+	struct media_request_object *obj;
+	struct vb2_buffer *vb;
+
+	obj = media_request_object_find(req, &vb2_core_req_ops, NULL);
+	if (!obj)
+		return NULL;
+
+	vb = container_of(obj, struct vb2_buffer, req_obj);
+
+	return vb2_get_drv_priv(vb->vb2_queue);
+}
+EXPORT_SYMBOL_GPL(vb2_core_request_find_buffer_priv);
+
 int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb,
 			 struct media_request *req)
 {
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index 032bd1bec555..65c0cf6afb55 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -1153,4 +1153,5 @@ int vb2_verify_memory_type(struct vb2_queue *q,
 		enum vb2_memory memory, unsigned int type);
 
 bool vb2_core_request_has_buffers(struct media_request *req);
+void *vb2_core_request_find_buffer_priv(struct media_request *req);
 #endif /* _MEDIA_VIDEOBUF2_CORE_H */
-- 
2.16.3

[PATCH v2 04/10] media: vim2m: Implement media request complete op to schedule m2m run

[Paul Kocialkowski]

This adds an implementation of the media request complete operation for
the vim2m driver, that ensures that the driver will try to schedule a
m2m run whenever a request was completed. Without this operation, no m2m
device run will be scheduled in many scenarios.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
---
 drivers/media/platform/vim2m.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/media/platform/vim2m.c b/drivers/media/platform/vim2m.c
index 2dcf0ea85705..08c4c5566614 100644
--- a/drivers/media/platform/vim2m.c
+++ b/drivers/media/platform/vim2m.c
@@ -453,6 +453,17 @@ static void device_isr(struct timer_list *t)
 	schedule_work(&vim2m_dev->work_run);
 }
 
+static void request_complete(struct media_request *req)
+{
+	struct vim2m_ctx *curr_ctx;
+
+	curr_ctx = (struct vim2m_ctx *) vb2_core_request_find_buffer_priv(req);
+	if (curr_ctx == NULL)
+		return;
+
+	v4l2_m2m_try_schedule(curr_ctx->fh.m2m_ctx);
+}
+
 /*
  * video ioctls
  */
@@ -1025,6 +1036,7 @@ static const struct v4l2_m2m_ops m2m_ops = {
 
 static const struct media_device_ops m2m_media_ops = {
 	.req_queue = vb2_request_queue,
+	.req_complete = request_complete,
 };
 
 static int vim2m_probe(struct platform_device *pdev)
-- 
2.16.3

[PATCH v2 04/10] media: vim2m: Implement media request complete op to schedule m2m run

[Paul Kocialkowski]

This adds an implementation of the media request complete operation for
the vim2m driver, that ensures that the driver will try to schedule a
m2m run whenever a request was completed. Without this operation, no m2m
device run will be scheduled in many scenarios.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
---
 drivers/media/platform/vim2m.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/media/platform/vim2m.c b/drivers/media/platform/vim2m.c
index 2dcf0ea85705..08c4c5566614 100644
--- a/drivers/media/platform/vim2m.c
+++ b/drivers/media/platform/vim2m.c
@@ -453,6 +453,17 @@ static void device_isr(struct timer_list *t)
 	schedule_work(&vim2m_dev->work_run);
 }
 
+static void request_complete(struct media_request *req)
+{
+	struct vim2m_ctx *curr_ctx;
+
+	curr_ctx = (struct vim2m_ctx *) vb2_core_request_find_buffer_priv(req);
+	if (curr_ctx == NULL)
+		return;
+
+	v4l2_m2m_try_schedule(curr_ctx->fh.m2m_ctx);
+}
+
 /*
  * video ioctls
  */
@@ -1025,6 +1036,7 @@ static const struct v4l2_m2m_ops m2m_ops = {
 
 static const struct media_device_ops m2m_media_ops = {
 	.req_queue = vb2_request_queue,
+	.req_complete = request_complete,
 };
 
 static int vim2m_probe(struct platform_device *pdev)
-- 
2.16.3

[PATCH v2 04/10] media: vim2m: Implement media request complete op to schedule m2m run

[Paul Kocialkowski]

This adds an implementation of the media request complete operation for
the vim2m driver, that ensures that the driver will try to schedule a
m2m run whenever a request was completed. Without this operation, no m2m
device run will be scheduled in many scenarios.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
---
 drivers/media/platform/vim2m.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/media/platform/vim2m.c b/drivers/media/platform/vim2m.c
index 2dcf0ea85705..08c4c5566614 100644
--- a/drivers/media/platform/vim2m.c
+++ b/drivers/media/platform/vim2m.c
@@ -453,6 +453,17 @@ static void device_isr(struct timer_list *t)
 	schedule_work(&vim2m_dev->work_run);
 }
 
+static void request_complete(struct media_request *req)
+{
+	struct vim2m_ctx *curr_ctx;
+
+	curr_ctx = (struct vim2m_ctx *) vb2_core_request_find_buffer_priv(req);
+	if (curr_ctx == NULL)
+		return;
+
+	v4l2_m2m_try_schedule(curr_ctx->fh.m2m_ctx);
+}
+
 /*
  * video ioctls
  */
@@ -1025,6 +1036,7 @@ static const struct v4l2_m2m_ops m2m_ops = {
 
 static const struct media_device_ops m2m_media_ops = {
 	.req_queue = vb2_request_queue,
+	.req_complete = request_complete,
 };
 
 static int vim2m_probe(struct platform_device *pdev)
-- 
2.16.3

[PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Paul Kocialkowski]

Stateless video decoding engines require both the MPEG slices and
associated metadata from the video stream in order to decode frames.

This introduces definitions for a new pixel format, describing buffers
with MPEG2 slice data, as well as a control structure for passing the
frame header (metadata) to drivers.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Signed-off-by: Florent Revest <florent.revest@free-electrons.com>
---
 drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
 include/uapi/linux/v4l2-controls.h   | 26 ++++++++++++++++++++++++++
 include/uapi/linux/videodev2.h       |  3 +++
 4 files changed, 40 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index ba05a8b9a095..fcdc12b9a9e0 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		return "Vertical MV Search Range";
 	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		return "Repeat Sequence Header";
 	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		return "Force Key Frame";
+	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		return "MPEG2 Frame Header";
 
 	/* VPX controls */
 	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		return "VPX Number of Partitions";
@@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
 	case V4L2_CID_RDS_TX_ALT_FREQS:
 		*type = V4L2_CTRL_TYPE_U32;
 		break;
+	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
+		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
+		break;
 	default:
 		*type = V4L2_CTRL_TYPE_INTEGER;
 		break;
@@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
 			return -ERANGE;
 		return 0;
 
+	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
+		return 0;
+
 	default:
 		return -EINVAL;
 	}
@@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
 	case V4L2_CTRL_TYPE_U32:
 		elem_size = sizeof(u32);
 		break;
+	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
+		elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
+		break;
 	default:
 		if (type < V4L2_CTRL_COMPOUND_TYPES)
 			elem_size = sizeof(s32);
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 468c3c65362d..8070203da5d2 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1 (SMPTE 412M Annex L)"; break;
 		case V4L2_PIX_FMT_VP8:		descr = "VP8"; break;
 		case V4L2_PIX_FMT_VP9:		descr = "VP9"; break;
+		case V4L2_PIX_FMT_MPEG2_FRAME:	descr = "MPEG2 Frame"; break;
 		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA YUV"; break;
 		case V4L2_PIX_FMT_WNVA:		descr = "WNVA"; break;
 		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA SN9C10X"; break;
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index cbbb750d87d1..8431b2a540c7 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
 };
 #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPEG_BASE+407)
 
+#define V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
+
 /*  Control IDs for VP8 streams
  *  Although VP8 is not part of MPEG we add these controls to the MPEG class
  *  as that class is already handling other video compression standards
@@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
 #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_CLASS_BASE + 3)
 #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DETECT_CLASS_BASE + 4)
 
+struct v4l2_ctrl_mpeg2_frame_hdr {
+	__u32 slice_len;
+	__u32 slice_pos;
+	enum { MPEG1, MPEG2 } type;
+
+	__u16 width;
+	__u16 height;
+
+	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;
+	__u8 f_code[2][2];
+
+	__u8 intra_dc_precision;
+	__u8 picture_structure;
+	__u8 top_field_first;
+	__u8 frame_pred_frame_dct;
+	__u8 concealment_motion_vectors;
+	__u8 q_scale_type;
+	__u8 intra_vlc_format;
+	__u8 alternate_scan;
+
+	__u8 backward_ref_index;
+	__u8 forward_ref_index;
+};
+
 #endif
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 31b5728b56e9..4b8336f7bcf0 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -635,6 +635,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /* SMPTE 421M Annex L compliant stream */
 #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
 #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
+#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /* MPEG2 frame */
 
 /*  Vendor-specific formats   */
 #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
@@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
 		__u8 __user *p_u8;
 		__u16 __user *p_u16;
 		__u32 __user *p_u32;
+		struct v4l2_ctrl_mpeg2_frame_hdr __user *p_mpeg2_frame_hdr;
 		void __user *ptr;
 	};
 } __attribute__ ((packed));
@@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
 	V4L2_CTRL_TYPE_U8	     = 0x0100,
 	V4L2_CTRL_TYPE_U16	     = 0x0101,
 	V4L2_CTRL_TYPE_U32	     = 0x0102,
+	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
 };
 
 /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
-- 
2.16.3

[PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Paul Kocialkowski]

Stateless video decoding engines require both the MPEG slices and
associated metadata from the video stream in order to decode frames.

This introduces definitions for a new pixel format, describing buffers
with MPEG2 slice data, as well as a control structure for passing the
frame header (metadata) to drivers.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
---
 drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
 include/uapi/linux/v4l2-controls.h   | 26 ++++++++++++++++++++++++++
 include/uapi/linux/videodev2.h       |  3 +++
 4 files changed, 40 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index ba05a8b9a095..fcdc12b9a9e0 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		return "Vertical MV Search Range";
 	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		return "Repeat Sequence Header";
 	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		return "Force Key Frame";
+	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		return "MPEG2 Frame Header";
 
 	/* VPX controls */
 	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		return "VPX Number of Partitions";
@@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
 	case V4L2_CID_RDS_TX_ALT_FREQS:
 		*type = V4L2_CTRL_TYPE_U32;
 		break;
+	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
+		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
+		break;
 	default:
 		*type = V4L2_CTRL_TYPE_INTEGER;
 		break;
@@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
 			return -ERANGE;
 		return 0;
 
+	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
+		return 0;
+
 	default:
 		return -EINVAL;
 	}
@@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
 	case V4L2_CTRL_TYPE_U32:
 		elem_size = sizeof(u32);
 		break;
+	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
+		elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
+		break;
 	default:
 		if (type < V4L2_CTRL_COMPOUND_TYPES)
 			elem_size = sizeof(s32);
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 468c3c65362d..8070203da5d2 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1 (SMPTE 412M Annex L)"; break;
 		case V4L2_PIX_FMT_VP8:		descr = "VP8"; break;
 		case V4L2_PIX_FMT_VP9:		descr = "VP9"; break;
+		case V4L2_PIX_FMT_MPEG2_FRAME:	descr = "MPEG2 Frame"; break;
 		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA YUV"; break;
 		case V4L2_PIX_FMT_WNVA:		descr = "WNVA"; break;
 		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA SN9C10X"; break;
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index cbbb750d87d1..8431b2a540c7 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
 };
 #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPEG_BASE+407)
 
+#define V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
+
 /*  Control IDs for VP8 streams
  *  Although VP8 is not part of MPEG we add these controls to the MPEG class
  *  as that class is already handling other video compression standards
@@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
 #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_CLASS_BASE + 3)
 #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DETECT_CLASS_BASE + 4)
 
+struct v4l2_ctrl_mpeg2_frame_hdr {
+	__u32 slice_len;
+	__u32 slice_pos;
+	enum { MPEG1, MPEG2 } type;
+
+	__u16 width;
+	__u16 height;
+
+	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;
+	__u8 f_code[2][2];
+
+	__u8 intra_dc_precision;
+	__u8 picture_structure;
+	__u8 top_field_first;
+	__u8 frame_pred_frame_dct;
+	__u8 concealment_motion_vectors;
+	__u8 q_scale_type;
+	__u8 intra_vlc_format;
+	__u8 alternate_scan;
+
+	__u8 backward_ref_index;
+	__u8 forward_ref_index;
+};
+
 #endif
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 31b5728b56e9..4b8336f7bcf0 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -635,6 +635,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /* SMPTE 421M Annex L compliant stream */
 #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
 #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
+#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /* MPEG2 frame */
 
 /*  Vendor-specific formats   */
 #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
@@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
 		__u8 __user *p_u8;
 		__u16 __user *p_u16;
 		__u32 __user *p_u32;
+		struct v4l2_ctrl_mpeg2_frame_hdr __user *p_mpeg2_frame_hdr;
 		void __user *ptr;
 	};
 } __attribute__ ((packed));
@@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
 	V4L2_CTRL_TYPE_U8	     = 0x0100,
 	V4L2_CTRL_TYPE_U16	     = 0x0101,
 	V4L2_CTRL_TYPE_U32	     = 0x0102,
+	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
 };
 
 /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
-- 
2.16.3

[PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Paul Kocialkowski]

Stateless video decoding engines require both the MPEG slices and
associated metadata from the video stream in order to decode frames.

This introduces definitions for a new pixel format, describing buffers
with MPEG2 slice data, as well as a control structure for passing the
frame header (metadata) to drivers.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Signed-off-by: Florent Revest <florent.revest@free-electrons.com>
---
 drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
 include/uapi/linux/v4l2-controls.h   | 26 ++++++++++++++++++++++++++
 include/uapi/linux/videodev2.h       |  3 +++
 4 files changed, 40 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
index ba05a8b9a095..fcdc12b9a9e0 100644
--- a/drivers/media/v4l2-core/v4l2-ctrls.c
+++ b/drivers/media/v4l2-core/v4l2-ctrls.c
@@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
 	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		return "Vertical MV Search Range";
 	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		return "Repeat Sequence Header";
 	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		return "Force Key Frame";
+	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		return "MPEG2 Frame Header";
 
 	/* VPX controls */
 	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		return "VPX Number of Partitions";
@@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
 	case V4L2_CID_RDS_TX_ALT_FREQS:
 		*type = V4L2_CTRL_TYPE_U32;
 		break;
+	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
+		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
+		break;
 	default:
 		*type = V4L2_CTRL_TYPE_INTEGER;
 		break;
@@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
 			return -ERANGE;
 		return 0;
 
+	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
+		return 0;
+
 	default:
 		return -EINVAL;
 	}
@@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
 	case V4L2_CTRL_TYPE_U32:
 		elem_size = sizeof(u32);
 		break;
+	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
+		elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
+		break;
 	default:
 		if (type < V4L2_CTRL_COMPOUND_TYPES)
 			elem_size = sizeof(s32);
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 468c3c65362d..8070203da5d2 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1 (SMPTE 412M Annex L)"; break;
 		case V4L2_PIX_FMT_VP8:		descr = "VP8"; break;
 		case V4L2_PIX_FMT_VP9:		descr = "VP9"; break;
+		case V4L2_PIX_FMT_MPEG2_FRAME:	descr = "MPEG2 Frame"; break;
 		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA YUV"; break;
 		case V4L2_PIX_FMT_WNVA:		descr = "WNVA"; break;
 		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA SN9C10X"; break;
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index cbbb750d87d1..8431b2a540c7 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
 };
 #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPEG_BASE+407)
 
+#define V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
+
 /*  Control IDs for VP8 streams
  *  Although VP8 is not part of MPEG we add these controls to the MPEG class
  *  as that class is already handling other video compression standards
@@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
 #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_CLASS_BASE + 3)
 #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DETECT_CLASS_BASE + 4)
 
+struct v4l2_ctrl_mpeg2_frame_hdr {
+	__u32 slice_len;
+	__u32 slice_pos;
+	enum { MPEG1, MPEG2 } type;
+
+	__u16 width;
+	__u16 height;
+
+	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;
+	__u8 f_code[2][2];
+
+	__u8 intra_dc_precision;
+	__u8 picture_structure;
+	__u8 top_field_first;
+	__u8 frame_pred_frame_dct;
+	__u8 concealment_motion_vectors;
+	__u8 q_scale_type;
+	__u8 intra_vlc_format;
+	__u8 alternate_scan;
+
+	__u8 backward_ref_index;
+	__u8 forward_ref_index;
+};
+
 #endif
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 31b5728b56e9..4b8336f7bcf0 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -635,6 +635,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /* SMPTE 421M Annex L compliant stream */
 #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
 #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
+#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /* MPEG2 frame */
 
 /*  Vendor-specific formats   */
 #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
@@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
 		__u8 __user *p_u8;
 		__u16 __user *p_u16;
 		__u32 __user *p_u32;
+		struct v4l2_ctrl_mpeg2_frame_hdr __user *p_mpeg2_frame_hdr;
 		void __user *ptr;
 	};
 } __attribute__ ((packed));
@@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
 	V4L2_CTRL_TYPE_U8	     = 0x0100,
 	V4L2_CTRL_TYPE_U16	     = 0x0101,
 	V4L2_CTRL_TYPE_U32	     = 0x0102,
+	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
 };
 
 /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
-- 
2.16.3

[PATCH v2 06/10] media: v4l: Add definition for Allwinner's MB32-tiled NV12 format

[Paul Kocialkowski]

This introduces support for Allwinner's MB32-tiled NV12 format, where
each plane is divided into macroblocks of 32x32 pixels. Hence, the size
of each plane has to be aligned to 32 bytes. The pixels inside each
macroblock are coded as they would be if the macroblock was a single
plane, line after line.

The MB32-tiled NV12 format is used by the video engine on Allwinner
platforms: it is the default format for decoded frames (and the only one
available in the oldest supported platforms).

Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
---
 include/uapi/linux/videodev2.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 4b8336f7bcf0..43993a116e2b 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -669,6 +669,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_Z16      v4l2_fourcc('Z', '1', '6', ' ') /* Depth data 16-bit */
 #define V4L2_PIX_FMT_MT21C    v4l2_fourcc('M', 'T', '2', '1') /* Mediatek compressed block mode  */
 #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
+#define V4L2_PIX_FMT_MB32_NV12 v4l2_fourcc('M', 'N', '1', '2') /* Allwinner NV12 in 32x32 macroblocks */
 
 /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
 #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
-- 
2.16.3

[PATCH v2 06/10] media: v4l: Add definition for Allwinner's MB32-tiled NV12 format

[Paul Kocialkowski]

This introduces support for Allwinner's MB32-tiled NV12 format, where
each plane is divided into macroblocks of 32x32 pixels. Hence, the size
of each plane has to be aligned to 32 bytes. The pixels inside each
macroblock are coded as they would be if the macroblock was a single
plane, line after line.

The MB32-tiled NV12 format is used by the video engine on Allwinner
platforms: it is the default format for decoded frames (and the only one
available in the oldest supported platforms).

Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
---
 include/uapi/linux/videodev2.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 4b8336f7bcf0..43993a116e2b 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -669,6 +669,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_Z16      v4l2_fourcc('Z', '1', '6', ' ') /* Depth data 16-bit */
 #define V4L2_PIX_FMT_MT21C    v4l2_fourcc('M', 'T', '2', '1') /* Mediatek compressed block mode  */
 #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
+#define V4L2_PIX_FMT_MB32_NV12 v4l2_fourcc('M', 'N', '1', '2') /* Allwinner NV12 in 32x32 macroblocks */
 
 /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
 #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
-- 
2.16.3

[PATCH v2 06/10] media: v4l: Add definition for Allwinner's MB32-tiled NV12 format

[Paul Kocialkowski]

This introduces support for Allwinner's MB32-tiled NV12 format, where
each plane is divided into macroblocks of 32x32 pixels. Hence, the size
of each plane has to be aligned to 32 bytes. The pixels inside each
macroblock are coded as they would be if the macroblock was a single
plane, line after line.

The MB32-tiled NV12 format is used by the video engine on Allwinner
platforms: it is the default format for decoded frames (and the only one
available in the oldest supported platforms).

Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
---
 include/uapi/linux/videodev2.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 4b8336f7bcf0..43993a116e2b 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -669,6 +669,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_Z16      v4l2_fourcc('Z', '1', '6', ' ') /* Depth data 16-bit */
 #define V4L2_PIX_FMT_MT21C    v4l2_fourcc('M', 'T', '2', '1') /* Mediatek compressed block mode  */
 #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
+#define V4L2_PIX_FMT_MB32_NV12 v4l2_fourcc('M', 'N', '1', '2') /* Allwinner NV12 in 32x32 macroblocks */
 
 /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
 #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
-- 
2.16.3

[PATCH v2 07/10] media: platform: Add Sunxi-Cedrus VPU decoder driver

[Paul Kocialkowski]

This introduces the Sunxi-Cedrus VPU driver that supports the VPU found
in Allwinner SoCs, also known as Video Engine. It is implemented through
a v4l2 m2m decoder device and a media device (used for media requests).
So far, it only supports MPEG2 decoding.

Since this VPU is stateless, synchronization with media requests is
required in order to ensure consistency between frame headers that
contain metadata about the frame to process and the raw slice data that
is used to generate the frame.

This driver was made possible thanks to the long-standing effort
carried out by the linux-sunxi community in the interest of reverse
engineering, documenting and implementing support for Allwinner VPU.

Signed-off-by: Florent Revest <florent.revest@free-electrons.com>
Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
---
 drivers/media/platform/Kconfig                     |  15 +
 drivers/media/platform/Makefile                    |   1 +
 drivers/media/platform/sunxi/cedrus/Makefile       |   4 +
 drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c | 292 ++++++++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_common.h    | 105 +++++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.c | 228 ++++++++++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.h |  36 ++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.c  | 201 +++++++++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.h  |  29 ++
 .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c     | 157 +++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h     |  33 ++
 .../platform/sunxi/cedrus/sunxi_cedrus_regs.h      | 172 +++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_video.c     | 497 +++++++++++++++++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_video.h     |  31 ++
 14 files changed, 1801 insertions(+)
 create mode 100644 drivers/media/platform/sunxi/cedrus/Makefile
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 614fbef08ddc..77d89e84ed3b 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -488,6 +488,21 @@ config VIDEO_TI_VPE
 	  Support for the TI VPE(Video Processing Engine) block
 	  found on DRA7XX SoC.
 
+config VIDEO_SUNXI_CEDRUS
+	tristate "Sunxi-Cedrus VPU driver"
+	depends on VIDEO_DEV && VIDEO_V4L2 && MEDIA_CONTROLLER
+	depends on ARCH_SUNXI
+	depends on HAS_DMA
+	select VIDEOBUF2_DMA_CONTIG
+	select MEDIA_REQUEST_API
+	select V4L2_MEM2MEM_DEV
+	---help---
+	  Support for the VPU found in Allwinner SoCs, also known as the Cedar
+	  video engine.
+
+	  To compile this driver as a module, choose M here: the module
+	  will be called sunxi-cedrus.
+
 config VIDEO_TI_VPE_DEBUG
 	bool "VPE debug messages"
 	depends on VIDEO_TI_VPE
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 7f3080437be6..a2be170f6dff 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -72,6 +72,7 @@ obj-$(CONFIG_VIDEO_ROCKCHIP_RGA)	+= rockchip/rga/
 obj-y	+= omap/
 
 obj-$(CONFIG_VIDEO_AM437X_VPFE)		+= am437x/
+obj-$(CONFIG_VIDEO_SUNXI_CEDRUS)	+= sunxi/cedrus/
 
 obj-$(CONFIG_VIDEO_XILINX)		+= xilinx/
 
diff --git a/drivers/media/platform/sunxi/cedrus/Makefile b/drivers/media/platform/sunxi/cedrus/Makefile
new file mode 100644
index 000000000000..98f30df626a9
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/Makefile
@@ -0,0 +1,4 @@
+obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += sunxi-cedrus.o
+
+sunxi-cedrus-y = sunxi_cedrus.o sunxi_cedrus_video.o sunxi_cedrus_hw.o \
+		 sunxi_cedrus_dec.o sunxi_cedrus_mpeg2.o
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
new file mode 100644
index 000000000000..364799c00703
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
@@ -0,0 +1,292 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/platform_device.h>
+#include <linux/module.h>
+#include <linux/of.h>
+
+#include <media/videobuf2-dma-contig.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "sunxi_cedrus_common.h"
+#include "sunxi_cedrus_video.h"
+#include "sunxi_cedrus_dec.h"
+#include "sunxi_cedrus_hw.h"
+
+static int sunxi_cedrus_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct sunxi_cedrus_ctx *ctx =
+		container_of(ctrl->handler, struct sunxi_cedrus_ctx, hdl);
+
+	switch (ctrl->id) {
+	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
+		/* This is kept in memory and used directly. */
+		break;
+	default:
+		v4l2_err(&ctx->dev->v4l2_dev, "Invalid control\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static const struct v4l2_ctrl_ops sunxi_cedrus_ctrl_ops = {
+	.s_ctrl = sunxi_cedrus_s_ctrl,
+};
+
+static const struct v4l2_ctrl_config sunxi_cedrus_ctrl_mpeg2_frame_hdr = {
+	.ops = &sunxi_cedrus_ctrl_ops,
+	.id = V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR,
+	.elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr),
+};
+
+static int sunxi_cedrus_open(struct file *file)
+{
+	struct sunxi_cedrus_dev *dev = video_drvdata(file);
+	struct sunxi_cedrus_ctx *ctx = NULL;
+	struct v4l2_ctrl_handler *hdl;
+	int rc = 0;
+
+	if (mutex_lock_interruptible(&dev->dev_mutex))
+		return -ERESTARTSYS;
+
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	if (!ctx) {
+		rc = -ENOMEM;
+		goto open_unlock;
+	}
+
+	INIT_WORK(&ctx->run_work, sunxi_cedrus_device_work);
+
+	INIT_LIST_HEAD(&ctx->src_list);
+	INIT_LIST_HEAD(&ctx->dst_list);
+
+	v4l2_fh_init(&ctx->fh, video_devdata(file));
+	file->private_data = &ctx->fh;
+	ctx->dev = dev;
+	hdl = &ctx->hdl;
+	v4l2_ctrl_handler_init(hdl, 2);
+
+	ctx->mpeg2_frame_hdr_ctrl = v4l2_ctrl_new_custom(hdl,
+			&sunxi_cedrus_ctrl_mpeg2_frame_hdr, NULL);
+	if (hdl->error) {
+		rc = hdl->error;
+		v4l2_ctrl_handler_free(hdl);
+		goto open_unlock;
+	}
+
+	ctx->fh.ctrl_handler = hdl;
+	v4l2_ctrl_handler_setup(hdl);
+
+	ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev, ctx,
+					    &sunxi_cedrus_queue_init);
+	if (IS_ERR(ctx->fh.m2m_ctx)) {
+		rc = PTR_ERR(ctx->fh.m2m_ctx);
+
+		v4l2_ctrl_handler_free(hdl);
+		kfree(ctx);
+		goto open_unlock;
+	}
+
+	v4l2_fh_add(&ctx->fh);
+
+	dev_dbg(dev->dev, "Created instance: %p, m2m_ctx: %p\n",
+		ctx, ctx->fh.m2m_ctx);
+
+open_unlock:
+	mutex_unlock(&dev->dev_mutex);
+	return rc;
+}
+
+static int sunxi_cedrus_release(struct file *file)
+{
+	struct sunxi_cedrus_dev *dev = video_drvdata(file);
+	struct sunxi_cedrus_ctx *ctx = container_of(file->private_data,
+			struct sunxi_cedrus_ctx, fh);
+
+	dev_dbg(dev->dev, "Releasing instance %p\n", ctx);
+
+	v4l2_fh_del(&ctx->fh);
+	v4l2_fh_exit(&ctx->fh);
+	v4l2_ctrl_handler_free(&ctx->hdl);
+	ctx->mpeg2_frame_hdr_ctrl = NULL;
+	mutex_lock(&dev->dev_mutex);
+	v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+	mutex_unlock(&dev->dev_mutex);
+	kfree(ctx);
+
+	return 0;
+}
+
+static const struct v4l2_file_operations sunxi_cedrus_fops = {
+	.owner		= THIS_MODULE,
+	.open		= sunxi_cedrus_open,
+	.release	= sunxi_cedrus_release,
+	.poll		= v4l2_m2m_fop_poll,
+	.unlocked_ioctl	= video_ioctl2,
+	.mmap		= v4l2_m2m_fop_mmap,
+};
+
+static const struct video_device sunxi_cedrus_video_device = {
+	.name		= SUNXI_CEDRUS_NAME,
+	.vfl_dir	= VFL_DIR_M2M,
+	.fops		= &sunxi_cedrus_fops,
+	.ioctl_ops	= &sunxi_cedrus_ioctl_ops,
+	.minor		= -1,
+	.release	= video_device_release_empty,
+};
+
+static const struct v4l2_m2m_ops sunxi_cedrus_m2m_ops = {
+	.device_run	= sunxi_cedrus_device_run,
+	.job_abort	= sunxi_cedrus_job_abort,
+};
+
+static const struct media_device_ops sunxi_cedrus_m2m_media_ops = {
+	.req_queue = vb2_request_queue,
+	.req_complete = sunxi_cedrus_request_complete,
+};
+
+static int sunxi_cedrus_probe(struct platform_device *pdev)
+{
+	struct sunxi_cedrus_dev *dev;
+	struct video_device *vfd;
+	int ret;
+
+	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
+	if (!dev)
+		return -ENOMEM;
+
+	dev->dev = &pdev->dev;
+	dev->pdev = pdev;
+
+	ret = sunxi_cedrus_hw_probe(dev);
+	if (ret) {
+		dev_err(&pdev->dev, "Failed to probe hardware\n");
+		return ret;
+	}
+
+	mutex_init(&dev->dev_mutex);
+	spin_lock_init(&dev->irq_lock);
+
+	dev->vfd = sunxi_cedrus_video_device;
+	vfd = &dev->vfd;
+	vfd->lock = &dev->dev_mutex;
+	vfd->v4l2_dev = &dev->v4l2_dev;
+
+	dev->mdev.dev = &pdev->dev;
+	strlcpy(dev->mdev.model, SUNXI_CEDRUS_NAME, sizeof(dev->mdev.model));
+	media_device_init(&dev->mdev);
+	dev->mdev.ops = &sunxi_cedrus_m2m_media_ops;
+	dev->v4l2_dev.mdev = &dev->mdev;
+	dev->pad[0].flags = MEDIA_PAD_FL_SINK;
+	dev->pad[1].flags = MEDIA_PAD_FL_SOURCE;
+	ret = media_entity_pads_init(&vfd->entity, 2, dev->pad);
+	if (ret)
+		return ret;
+
+	ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
+	if (ret)
+		goto unreg_media;
+
+	ret = video_register_device(vfd, VFL_TYPE_GRABBER, 0);
+	if (ret) {
+		v4l2_err(&dev->v4l2_dev, "Failed to register video device\n");
+		goto unreg_dev;
+	}
+
+	video_set_drvdata(vfd, dev);
+	snprintf(vfd->name, sizeof(vfd->name), "%s",
+		 sunxi_cedrus_video_device.name);
+	v4l2_info(&dev->v4l2_dev,
+		  "Device registered as /dev/video%d\n", vfd->num);
+
+	platform_set_drvdata(pdev, dev);
+
+	dev->m2m_dev = v4l2_m2m_init(&sunxi_cedrus_m2m_ops);
+	if (IS_ERR(dev->m2m_dev)) {
+		v4l2_err(&dev->v4l2_dev, "Failed to init mem2mem device\n");
+		ret = PTR_ERR(dev->m2m_dev);
+		goto err_m2m;
+	}
+
+	/* Register the media device node */
+	ret = media_device_register(&dev->mdev);
+	if (ret)
+		goto err_m2m;
+
+	return 0;
+
+err_m2m:
+	v4l2_m2m_release(dev->m2m_dev);
+	video_unregister_device(&dev->vfd);
+unreg_media:
+	media_device_unregister(&dev->mdev);
+unreg_dev:
+	v4l2_device_unregister(&dev->v4l2_dev);
+
+	return ret;
+}
+
+static int sunxi_cedrus_remove(struct platform_device *pdev)
+{
+	struct sunxi_cedrus_dev *dev = platform_get_drvdata(pdev);
+
+	v4l2_info(&dev->v4l2_dev, "Removing " SUNXI_CEDRUS_NAME);
+
+	if (media_devnode_is_registered(dev->mdev.devnode)) {
+		media_device_unregister(&dev->mdev);
+		media_device_cleanup(&dev->mdev);
+	}
+
+	v4l2_m2m_release(dev->m2m_dev);
+	video_unregister_device(&dev->vfd);
+	v4l2_device_unregister(&dev->v4l2_dev);
+	sunxi_cedrus_hw_remove(dev);
+
+	return 0;
+}
+
+#ifdef CONFIG_OF
+static const struct of_device_id of_sunxi_cedrus_match[] = {
+	{ .compatible = "allwinner,sun4i-a10-video-engine" },
+	{ /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, of_sunxi_cedrus_match);
+#endif
+
+static struct platform_driver sunxi_cedrus_driver = {
+	.probe		= sunxi_cedrus_probe,
+	.remove		= sunxi_cedrus_remove,
+	.driver		= {
+		.name	= SUNXI_CEDRUS_NAME,
+		.owner = THIS_MODULE,
+		.of_match_table = of_match_ptr(of_sunxi_cedrus_match),
+	},
+};
+module_platform_driver(sunxi_cedrus_driver);
+
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("Florent Revest <florent.revest@free-electrons.com>");
+MODULE_DESCRIPTION("Sunxi-Cedrus VPU driver");
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
new file mode 100644
index 000000000000..bab806c7c28d
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
@@ -0,0 +1,105 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _SUNXI_CEDRUS_COMMON_H_
+#define _SUNXI_CEDRUS_COMMON_H_
+
+#include <linux/platform_device.h>
+
+#include <media/videobuf2-v4l2.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+
+#define SUNXI_CEDRUS_NAME	"sunxi-cedrus"
+
+struct sunxi_cedrus_dev {
+	struct v4l2_device v4l2_dev;
+	struct video_device vfd;
+	struct media_device mdev;
+	struct media_pad pad[2];
+	struct platform_device *pdev;
+	struct device *dev;
+	struct v4l2_m2m_dev *m2m_dev;
+
+	/* Mutex for device file */
+	struct mutex dev_mutex;
+	/* Spinlock for interrupt */
+	spinlock_t irq_lock;
+
+	struct clk *mod_clk;
+	struct clk *ahb_clk;
+	struct clk *ram_clk;
+
+	struct reset_control *rstc;
+
+	struct regmap *syscon;
+
+	char *base;
+};
+
+struct sunxi_cedrus_fmt {
+	u32 fourcc;
+	int depth;
+	u32 types;
+	unsigned int num_planes;
+};
+
+struct sunxi_cedrus_ctx {
+	struct v4l2_fh fh;
+	struct sunxi_cedrus_dev	*dev;
+
+	struct sunxi_cedrus_fmt *vpu_src_fmt;
+	struct v4l2_pix_format_mplane src_fmt;
+	struct sunxi_cedrus_fmt *vpu_dst_fmt;
+	struct v4l2_pix_format_mplane dst_fmt;
+
+	struct v4l2_ctrl_handler hdl;
+
+	struct vb2_buffer *dst_bufs[VIDEO_MAX_FRAME];
+
+	struct v4l2_ctrl *mpeg2_frame_hdr_ctrl;
+	int job_abort;
+
+	struct work_struct try_schedule_work;
+	struct work_struct run_work;
+	struct list_head src_list;
+	struct list_head dst_list;
+};
+
+struct sunxi_cedrus_buffer {
+	struct vb2_v4l2_buffer vb;
+	enum vb2_buffer_state state;
+	struct list_head list;
+};
+
+static inline void sunxi_cedrus_write(struct sunxi_cedrus_dev *dev,
+				      u32 val, u32 reg)
+{
+	writel(val, dev->base + reg);
+}
+
+static inline u32 sunxi_cedrus_read(struct sunxi_cedrus_dev *dev, u32 reg)
+{
+	return readl(dev->base + reg);
+}
+
+#endif
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
new file mode 100644
index 000000000000..64ec412c68f2
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
@@ -0,0 +1,228 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <media/videobuf2-dma-contig.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "sunxi_cedrus_common.h"
+#include "sunxi_cedrus_mpeg2.h"
+#include "sunxi_cedrus_dec.h"
+#include "sunxi_cedrus_hw.h"
+
+void sunxi_cedrus_device_work(struct work_struct *work)
+{
+	struct sunxi_cedrus_ctx *ctx = container_of(work,
+			struct sunxi_cedrus_ctx, run_work);
+	struct sunxi_cedrus_buffer *buffer_entry;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	unsigned long flags;
+
+	spin_lock_irqsave(&ctx->dev->irq_lock, flags);
+
+	if (list_empty(&ctx->src_list) ||
+	    list_empty(&ctx->dst_list)) {
+		pr_err("Empty source and/or destination buffers lists\n");
+		spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
+		return;
+	}
+
+	buffer_entry = list_last_entry(&ctx->src_list, struct sunxi_cedrus_buffer, list);
+	list_del(ctx->src_list.prev);
+
+	src_buf = &buffer_entry->vb;
+	v4l2_m2m_buf_done(src_buf, buffer_entry->state);
+
+	buffer_entry = list_last_entry(&ctx->dst_list, struct sunxi_cedrus_buffer, list);
+	list_del(ctx->dst_list.prev);
+
+	dst_buf = &buffer_entry->vb;
+	v4l2_m2m_buf_done(dst_buf, buffer_entry->state);
+
+	spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
+
+	v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx);
+}
+
+void sunxi_cedrus_device_run(void *priv)
+{
+	struct sunxi_cedrus_ctx *ctx = priv;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	struct media_request *src_req, *dst_req;
+	dma_addr_t src_buf_addr, dst_luma_addr, dst_chroma_addr;
+	unsigned long flags;
+	struct v4l2_ctrl_mpeg2_frame_hdr *mpeg2_frame_hdr;
+	bool mpeg1 = false;
+
+	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+	if (!src_buf) {
+		v4l2_err(&ctx->dev->v4l2_dev,
+			 "No source buffer to prepare\n");
+		return;
+	}
+
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+	if (!dst_buf) {
+		v4l2_err(&ctx->dev->v4l2_dev,
+			 "No destination buffer to prepare\n");
+		return;
+	}
+
+	src_buf_addr = vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf, 0);
+	dst_luma_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
+	dst_chroma_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 1);
+	if (!src_buf || !dst_luma_addr || !dst_chroma_addr) {
+		v4l2_err(&ctx->dev->v4l2_dev,
+			 "Acquiring kernel pointers to buffers failed\n");
+		return;
+	}
+
+	/* Apply request(s) controls if needed. */
+
+	src_req = src_buf->vb2_buf.req_obj.req;
+	dst_req = dst_buf->vb2_buf.req_obj.req;
+
+	if (src_req) {
+		if (src_req->state != MEDIA_REQUEST_STATE_QUEUED) {
+			v4l2_err(&ctx->dev->v4l2_dev,
+				 "Unexpected state for request %s\n",
+				 src_req->debug_str);
+			return;
+		}
+
+		v4l2_ctrl_request_setup(src_req, &ctx->hdl);
+	}
+
+	if (dst_req && dst_req != src_req) {
+		if (dst_req->state != MEDIA_REQUEST_STATE_QUEUED) {
+			v4l2_err(&ctx->dev->v4l2_dev,
+				 "Unexpected state for request %s\n",
+				 dst_req->debug_str);
+			return;
+		}
+
+		v4l2_ctrl_request_setup(dst_req, &ctx->hdl);
+	}
+
+	dst_buf->vb2_buf.timestamp = src_buf->vb2_buf.timestamp;
+	if (src_buf->flags & V4L2_BUF_FLAG_TIMECODE)
+		dst_buf->timecode = src_buf->timecode;
+	dst_buf->field = src_buf->field;
+	dst_buf->flags = src_buf->flags & (V4L2_BUF_FLAG_TIMECODE |
+		 V4L2_BUF_FLAG_KEYFRAME | V4L2_BUF_FLAG_PFRAME |
+		 V4L2_BUF_FLAG_BFRAME   | V4L2_BUF_FLAG_TSTAMP_SRC_MASK);
+
+	ctx->job_abort = 0;
+
+	spin_lock_irqsave(&ctx->dev->irq_lock, flags);
+
+	if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_MPEG2_FRAME) {
+		if (!ctx->mpeg2_frame_hdr_ctrl) {
+			v4l2_err(&ctx->dev->v4l2_dev,
+				 "Invalid MPEG2 frame header control\n");
+			ctx->job_abort = 1;
+			goto unlock_complete;
+		}
+
+		mpeg2_frame_hdr = (struct v4l2_ctrl_mpeg2_frame_hdr *)
+				ctx->mpeg2_frame_hdr_ctrl->p_new.p;
+
+		sunxi_cedrus_mpeg2_setup(ctx, src_buf_addr, dst_luma_addr,
+					 dst_chroma_addr, mpeg2_frame_hdr);
+
+		mpeg1 = mpeg2_frame_hdr->type == MPEG1;
+	} else {
+		ctx->job_abort = 1;
+	}
+
+unlock_complete:
+	spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
+
+	/* Complete request(s) controls if needed. */
+
+	if (src_req)
+		v4l2_ctrl_request_complete(src_req, &ctx->hdl);
+
+	if (dst_req && dst_req != src_req)
+		v4l2_ctrl_request_complete(dst_req, &ctx->hdl);
+
+	spin_lock_irqsave(&ctx->dev->irq_lock, flags);
+
+	if (!ctx->job_abort) {
+		if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_MPEG2_FRAME)
+			sunxi_cedrus_mpeg2_trigger(ctx, mpeg1);
+	} else {
+		v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
+		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
+	}
+
+	spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
+
+	if (ctx->job_abort)
+		v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx);
+}
+
+void sunxi_cedrus_job_abort(void *priv)
+{
+	struct sunxi_cedrus_ctx *ctx = priv;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	unsigned long flags;
+
+	ctx->job_abort = 1;
+
+	/*
+	 * V4L2 m2m and request API cleanup is done here while hardware state
+	 * cleanup is done in the interrupt context. Doing all the cleanup in
+	 * the interrupt context is a bit risky, since the job_abort call might
+	 * originate from the release hook, where interrupts have already been
+	 * disabled.
+	 */
+
+	spin_lock_irqsave(&ctx->dev->irq_lock, flags);
+
+	src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	if (src_buf)
+		v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
+
+	dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+	if (dst_buf)
+		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
+
+	spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
+
+	v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx);
+}
+
+void sunxi_cedrus_request_complete(struct media_request *req)
+{
+	struct sunxi_cedrus_ctx *ctx;
+
+	ctx = (struct sunxi_cedrus_ctx *)
+			vb2_core_request_find_buffer_priv(req);
+	if (ctx == NULL)
+		return;
+
+	v4l2_m2m_try_schedule(ctx->fh.m2m_ctx);
+}
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.h
new file mode 100644
index 000000000000..a1a9e4c98850
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.h
@@ -0,0 +1,36 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _SUNXI_CEDRUS_DEC_H_
+#define _SUNXI_CEDRUS_DEC_H_
+
+extern const struct v4l2_ioctl_ops sunxi_cedrus_ioctl_ops;
+
+void sunxi_cedrus_device_work(struct work_struct *work);
+void sunxi_cedrus_device_run(void *priv);
+void sunxi_cedrus_job_abort(void *priv);
+void sunxi_cedrus_request_complete(struct media_request *req);
+
+int sunxi_cedrus_queue_init(void *priv, struct vb2_queue *src_vq,
+			    struct vb2_queue *dst_vq);
+
+#endif
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
new file mode 100644
index 000000000000..1d5d1d3e8531
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
@@ -0,0 +1,201 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/platform_device.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/dma-mapping.h>
+#include <linux/mfd/syscon.h>
+#include <linux/interrupt.h>
+#include <linux/clk.h>
+#include <linux/regmap.h>
+#include <linux/reset.h>
+
+#include <media/videobuf2-core.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "sunxi_cedrus_common.h"
+#include "sunxi_cedrus_regs.h"
+
+#define SYSCON_SRAM_CTRL_REG0	0x0
+#define SYSCON_SRAM_C1_MAP_VE	0x7fffffff
+
+static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
+{
+	struct sunxi_cedrus_dev *dev = dev_id;
+	struct sunxi_cedrus_ctx *ctx;
+	struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
+	struct vb2_v4l2_buffer *src_vb, *dst_vb;
+	unsigned long flags;
+	unsigned int value, status;
+
+	spin_lock_irqsave(&dev->irq_lock, flags);
+
+	/* Disable MPEG interrupts and stop the MPEG engine */
+	value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
+	sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
+
+	status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
+	sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
+
+	sunxi_cedrus_write(dev, VE_CTRL_REINIT, VE_CTRL);
+
+	ctx = v4l2_m2m_get_curr_priv(dev->m2m_dev);
+	if (!ctx) {
+		pr_err("Instance released before the end of transaction\n");
+		spin_unlock_irqrestore(&dev->irq_lock, flags);
+
+		return IRQ_HANDLED;
+	}
+
+	src_vb = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	dst_vb = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+
+	if (!src_vb || !dst_vb) {
+		pr_err("Unable to get source and/or destination buffers\n");
+		spin_unlock_irqrestore(&dev->irq_lock, flags);
+
+		return IRQ_HANDLED;
+	}
+
+	src_buffer = container_of(src_vb, struct sunxi_cedrus_buffer, vb);
+	dst_buffer = container_of(dst_vb, struct sunxi_cedrus_buffer, vb);
+
+	/* First bit of MPEG_STATUS indicates success. */
+	if (ctx->job_abort || !(status & 0x01))
+		src_buffer->state = dst_buffer->state = VB2_BUF_STATE_ERROR;
+	else
+		src_buffer->state = dst_buffer->state = VB2_BUF_STATE_DONE;
+
+	list_add_tail(&src_buffer->list, &ctx->src_list);
+	list_add_tail(&dst_buffer->list, &ctx->dst_list);
+
+	spin_unlock_irqrestore(&dev->irq_lock, flags);
+
+	schedule_work(&ctx->run_work);
+
+	return IRQ_HANDLED;
+}
+
+int sunxi_cedrus_hw_probe(struct sunxi_cedrus_dev *dev)
+{
+	struct resource *res;
+	int irq_dec;
+	int ret;
+
+	irq_dec = platform_get_irq(dev->pdev, 0);
+	if (irq_dec <= 0) {
+		dev_err(dev->dev, "could not get ve IRQ\n");
+		return -ENXIO;
+	}
+	ret = devm_request_irq(dev->dev, irq_dec, sunxi_cedrus_ve_irq, 0,
+			       dev_name(dev->dev), dev);
+	if (ret) {
+		dev_err(dev->dev, "could not request ve IRQ\n");
+		return -ENXIO;
+	}
+
+	/*
+	 * The VPU is only able to handle bus addresses so we have to subtract
+	 * the RAM offset to the physcal addresses.
+	 */
+	dev->dev->dma_pfn_offset = PHYS_PFN_OFFSET;
+
+	ret = of_reserved_mem_device_init(dev->dev);
+	if (ret) {
+		dev_err(dev->dev, "could not reserve memory\n");
+		return -ENODEV;
+	}
+
+	dev->ahb_clk = devm_clk_get(dev->dev, "ahb");
+	if (IS_ERR(dev->ahb_clk)) {
+		dev_err(dev->dev, "failed to get ahb clock\n");
+		return PTR_ERR(dev->ahb_clk);
+	}
+	dev->mod_clk = devm_clk_get(dev->dev, "mod");
+	if (IS_ERR(dev->mod_clk)) {
+		dev_err(dev->dev, "failed to get mod clock\n");
+		return PTR_ERR(dev->mod_clk);
+	}
+	dev->ram_clk = devm_clk_get(dev->dev, "ram");
+	if (IS_ERR(dev->ram_clk)) {
+		dev_err(dev->dev, "failed to get ram clock\n");
+		return PTR_ERR(dev->ram_clk);
+	}
+
+	dev->rstc = devm_reset_control_get(dev->dev, NULL);
+
+	res = platform_get_resource(dev->pdev, IORESOURCE_MEM, 0);
+	dev->base = devm_ioremap_resource(dev->dev, res);
+	if (!dev->base)
+		dev_err(dev->dev, "could not maps MACC registers\n");
+
+	dev->syscon = syscon_regmap_lookup_by_phandle(dev->dev->of_node,
+						      "syscon");
+	if (IS_ERR(dev->syscon)) {
+		dev->syscon = NULL;
+	} else {
+		regmap_write_bits(dev->syscon, SYSCON_SRAM_CTRL_REG0,
+				  SYSCON_SRAM_C1_MAP_VE,
+				  SYSCON_SRAM_C1_MAP_VE);
+	}
+
+	ret = clk_prepare_enable(dev->ahb_clk);
+	if (ret) {
+		dev_err(dev->dev, "could not enable ahb clock\n");
+		return -EFAULT;
+	}
+	ret = clk_prepare_enable(dev->mod_clk);
+	if (ret) {
+		clk_disable_unprepare(dev->ahb_clk);
+		dev_err(dev->dev, "could not enable mod clock\n");
+		return -EFAULT;
+	}
+	ret = clk_prepare_enable(dev->ram_clk);
+	if (ret) {
+		clk_disable_unprepare(dev->mod_clk);
+		clk_disable_unprepare(dev->ahb_clk);
+		dev_err(dev->dev, "could not enable ram clock\n");
+		return -EFAULT;
+	}
+
+	ret = reset_control_reset(dev->rstc);
+	if (ret) {
+		clk_disable_unprepare(dev->ram_clk);
+		clk_disable_unprepare(dev->mod_clk);
+		clk_disable_unprepare(dev->ahb_clk);
+		dev_err(dev->dev, "could not reset device\n");
+		return -EFAULT;
+	}
+
+	return 0;
+}
+
+void sunxi_cedrus_hw_remove(struct sunxi_cedrus_dev *dev)
+{
+	reset_control_assert(dev->rstc);
+
+	clk_disable_unprepare(dev->ram_clk);
+	clk_disable_unprepare(dev->mod_clk);
+	clk_disable_unprepare(dev->ahb_clk);
+
+	of_reserved_mem_device_release(dev->dev);
+}
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
new file mode 100644
index 000000000000..8184d7591363
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
@@ -0,0 +1,29 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _SUNXI_CEDRUS_HW_H_
+#define _SUNXI_CEDRUS_HW_H_
+
+int sunxi_cedrus_hw_probe(struct sunxi_cedrus_dev *dev);
+void sunxi_cedrus_hw_remove(struct sunxi_cedrus_dev *dev);
+
+#endif
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
new file mode 100644
index 000000000000..aa86c2f9c49a
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
@@ -0,0 +1,157 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <media/videobuf2-dma-contig.h>
+
+#include "sunxi_cedrus_common.h"
+#include "sunxi_cedrus_regs.h"
+
+static const u8 mpeg_default_intra_quant[64] = {
+	 8, 16, 16, 19, 16, 19, 22, 22,
+	22, 22, 22, 22, 26, 24, 26, 27,
+	27, 27, 26, 26, 26, 26, 27, 27,
+	27, 29, 29, 29, 34, 34, 34, 29,
+	29, 29, 27, 27, 29, 29, 32, 32,
+	34, 34, 37, 38, 37, 35, 35, 34,
+	35, 38, 38, 40, 40, 40, 48, 48,
+	46, 46, 56, 56, 58, 69, 69, 83
+};
+
+#define m_iq(i) (((64 + i) << 8) | mpeg_default_intra_quant[i])
+
+static const u8 mpeg_default_non_intra_quant[64] = {
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16
+};
+
+#define m_niq(i) ((i << 8) | mpeg_default_non_intra_quant[i])
+
+void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
+			      dma_addr_t src_buf_addr,
+			      dma_addr_t dst_luma_addr,
+			      dma_addr_t dst_chroma_addr,
+			      struct v4l2_ctrl_mpeg2_frame_hdr *frame_hdr)
+{
+	struct sunxi_cedrus_dev *dev = ctx->dev;
+
+	u16 width = DIV_ROUND_UP(frame_hdr->width, 16);
+	u16 height = DIV_ROUND_UP(frame_hdr->height, 16);
+
+	u32 pic_header = 0;
+	u32 vld_len = frame_hdr->slice_len - frame_hdr->slice_pos;
+	int i;
+
+	struct vb2_buffer *fwd_vb2_buf, *bwd_vb2_buf;
+	dma_addr_t fwd_luma = 0, fwd_chroma = 0, bwd_luma = 0, bwd_chroma = 0;
+
+
+	fwd_vb2_buf = ctx->dst_bufs[frame_hdr->forward_ref_index];
+	if (fwd_vb2_buf) {
+		fwd_luma = vb2_dma_contig_plane_dma_addr(fwd_vb2_buf, 0);
+		fwd_chroma = vb2_dma_contig_plane_dma_addr(fwd_vb2_buf, 1);
+	}
+
+	bwd_vb2_buf = ctx->dst_bufs[frame_hdr->backward_ref_index];
+	if (bwd_vb2_buf) {
+		bwd_luma = vb2_dma_contig_plane_dma_addr(bwd_vb2_buf, 0);
+		bwd_chroma = vb2_dma_contig_plane_dma_addr(bwd_vb2_buf, 1);
+	}
+
+	/* Activate MPEG engine. */
+	sunxi_cedrus_write(dev, VE_CTRL_MPEG, VE_CTRL);
+
+	/* Set quantization matrices. */
+	for (i = 0; i < 64; i++) {
+		sunxi_cedrus_write(dev, m_iq(i), VE_MPEG_IQ_MIN_INPUT);
+		sunxi_cedrus_write(dev, m_niq(i), VE_MPEG_IQ_MIN_INPUT);
+	}
+
+	/* Set frame dimensions. */
+	sunxi_cedrus_write(dev, width << 8 | height, VE_MPEG_SIZE);
+	sunxi_cedrus_write(dev, width << 20 | height << 4, VE_MPEG_FRAME_SIZE);
+
+	/* Set MPEG picture header. */
+	pic_header |= (frame_hdr->picture_coding_type & 0xf) << 28;
+	pic_header |= (frame_hdr->f_code[0][0] & 0xf) << 24;
+	pic_header |= (frame_hdr->f_code[0][1] & 0xf) << 20;
+	pic_header |= (frame_hdr->f_code[1][0] & 0xf) << 16;
+	pic_header |= (frame_hdr->f_code[1][1] & 0xf) << 12;
+	pic_header |= (frame_hdr->intra_dc_precision & 0x3) << 10;
+	pic_header |= (frame_hdr->picture_structure & 0x3) << 8;
+	pic_header |= (frame_hdr->top_field_first & 0x1) << 7;
+	pic_header |= (frame_hdr->frame_pred_frame_dct & 0x1) << 6;
+	pic_header |= (frame_hdr->concealment_motion_vectors & 0x1) << 5;
+	pic_header |= (frame_hdr->q_scale_type & 0x1) << 4;
+	pic_header |= (frame_hdr->intra_vlc_format & 0x1) << 3;
+	pic_header |= (frame_hdr->alternate_scan & 0x1) << 2;
+	sunxi_cedrus_write(dev, pic_header, VE_MPEG_PIC_HDR);
+
+	/* Enable interrupt and an unknown control flag. */
+	sunxi_cedrus_write(dev, VE_MPEG_CTRL_MPEG2, VE_MPEG_CTRL);
+
+	/* Macroblock address. */
+	sunxi_cedrus_write(dev, 0, VE_MPEG_MBA);
+
+	/* Clear previous errors. */
+	sunxi_cedrus_write(dev, 0, VE_MPEG_ERROR);
+
+	/* Clear correct macroblocks register. */
+	sunxi_cedrus_write(dev, 0, VE_MPEG_CTR_MB);
+
+	/* Forward and backward prediction reference buffers. */
+	sunxi_cedrus_write(dev, fwd_luma, VE_MPEG_FWD_LUMA);
+	sunxi_cedrus_write(dev, fwd_chroma, VE_MPEG_FWD_CHROMA);
+	sunxi_cedrus_write(dev, bwd_luma, VE_MPEG_BACK_LUMA);
+	sunxi_cedrus_write(dev, bwd_chroma, VE_MPEG_BACK_CHROMA);
+
+	/* Desination luma and chroma buffers. */
+	sunxi_cedrus_write(dev, dst_luma_addr, VE_MPEG_REC_LUMA);
+	sunxi_cedrus_write(dev, dst_chroma_addr, VE_MPEG_REC_CHROMA);
+	sunxi_cedrus_write(dev, dst_luma_addr, VE_MPEG_ROT_LUMA);
+	sunxi_cedrus_write(dev, dst_chroma_addr, VE_MPEG_ROT_CHROMA);
+
+	/* Source offset and length in bits. */
+	sunxi_cedrus_write(dev, frame_hdr->slice_pos, VE_MPEG_VLD_OFFSET);
+	sunxi_cedrus_write(dev, vld_len, VE_MPEG_VLD_LEN);
+
+	/* Source beginning and end addresses. */
+	sunxi_cedrus_write(dev, VE_MPEG_VLD_ADDR_VAL(src_buf_addr),
+			   VE_MPEG_VLD_ADDR);
+	sunxi_cedrus_write(dev, src_buf_addr + VBV_SIZE - 1, VE_MPEG_VLD_END);
+}
+
+void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx, bool mpeg1)
+{
+	struct sunxi_cedrus_dev *dev = ctx->dev;
+
+	/* Trigger MPEG engine. */
+	if (mpeg1)
+		sunxi_cedrus_write(dev, VE_TRIG_MPEG1, VE_MPEG_TRIGGER);
+	else
+		sunxi_cedrus_write(dev, VE_TRIG_MPEG2, VE_MPEG_TRIGGER);
+}
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
new file mode 100644
index 000000000000..4ab532ff173c
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
@@ -0,0 +1,33 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _SUNXI_CEDRUS_MPEG2_H_
+#define _SUNXI_CEDRUS_MPEG2_H_
+
+void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
+			      dma_addr_t src_buf_addr,
+			      dma_addr_t dst_luma_addr,
+			      dma_addr_t dst_chroma_addr,
+			      struct v4l2_ctrl_mpeg2_frame_hdr *frame_hdr);
+void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx, bool mpeg1);
+
+#endif
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h
new file mode 100644
index 000000000000..ff5f79878ada
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h
@@ -0,0 +1,172 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _SUNXI_CEDRUS_REGS_H_
+#define _SUNXI_CEDRUS_REGS_H_
+
+/*
+ * For more information, consult http://linux-sunxi.org/VE_Register_guide
+ */
+
+/* Special registers values */
+
+/* VE_CTRL:
+ * The first 3 bits indicate the engine (0 for MPEG, 1 for H264, b for AVC...)
+ * The 16th and 17th bits indicate the memory type (3 for DDR3 32 bits)
+ * The 20th bit is unknown but needed
+ */
+#define VE_CTRL_MPEG		0x130000
+#define VE_CTRL_H264		0x130001
+#define VE_CTRL_AVC		0x13000b
+#define VE_CTRL_REINIT		0x130007
+
+/* VE_MPEG_CTRL:
+ * The bit 3 (0x8) is used to enable IRQs
+ * The other bits are unknown but needed
+ */
+#define VE_MPEG_CTRL_MPEG2	0x800001b8
+#define VE_MPEG_CTRL_MPEG4	(0x80084118 | BIT(7))
+#define VE_MPEG_CTRL_MPEG4_P	(VE_MPEG_CTRL_MPEG4 | BIT(12))
+
+/* VE_MPEG_VLD_ADDR:
+ * The bits 27 to 4 are used for the address
+ * The bits 31 to 28 (0x7) are used to select the MPEG or JPEG engine
+ */
+#define VE_MPEG_VLD_ADDR_VAL(x)	((x & 0x0ffffff0) | (x >> 28) | (0x7 << 28))
+
+/* VE_MPEG_TRIGGER:
+ * The first three bits are used to trigger the engine
+ * The bits 24 to 26 are used to select the input format (1 for MPEG1, 2 for
+ *                           MPEG2, 4 for MPEG4)
+ * The bit 21 (0x8) is used to disable bitstream error handling
+ *
+ * In MPEG4 the w*h value is somehow used for an offset, unknown but needed
+ */
+#define VE_TRIG_MPEG1		0x8100000f
+#define VE_TRIG_MPEG2		0x8200000f
+#define VE_TRIG_MPEG4(w, h)	(0x8400000d | ((w * h) << 8))
+
+/* VE_MPEG_SDROT_CTRL:
+ * The bit 8 at zero is used to disable x downscaling
+ * The bit 10 at 0 is used to disable y downscaling
+ * The other bits are unknown but needed
+ */
+#define VE_NO_SDROT_CTRL	0x40620000
+
+/* Decent size fo video buffering verifier */
+#define VBV_SIZE		(1024 * 1024)
+
+/* Registers addresses */
+#define VE_CTRL				0x000
+#define VE_VERSION			0x0f0
+
+#define VE_MPEG_PIC_HDR			0x100
+#define VE_MPEG_VOP_HDR			0x104
+#define VE_MPEG_SIZE			0x108
+#define VE_MPEG_FRAME_SIZE		0x10c
+#define VE_MPEG_MBA			0x110
+#define VE_MPEG_CTRL			0x114
+#define VE_MPEG_TRIGGER			0x118
+#define VE_MPEG_STATUS			0x11c
+#define VE_MPEG_TRBTRD_FIELD		0x120
+#define VE_MPEG_TRBTRD_FRAME		0x124
+#define VE_MPEG_VLD_ADDR		0x128
+#define VE_MPEG_VLD_OFFSET		0x12c
+#define VE_MPEG_VLD_LEN			0x130
+#define VE_MPEG_VLD_END			0x134
+#define VE_MPEG_MBH_ADDR		0x138
+#define VE_MPEG_DCAC_ADDR		0x13c
+#define VE_MPEG_NCF_ADDR		0x144
+#define VE_MPEG_REC_LUMA		0x148
+#define VE_MPEG_REC_CHROMA		0x14c
+#define VE_MPEG_FWD_LUMA		0x150
+#define VE_MPEG_FWD_CHROMA		0x154
+#define VE_MPEG_BACK_LUMA		0x158
+#define VE_MPEG_BACK_CHROMA		0x15c
+#define VE_MPEG_IQ_MIN_INPUT		0x180
+#define VE_MPEG_QP_INPUT		0x184
+#define VE_MPEG_JPEG_SIZE		0x1b8
+#define VE_MPEG_JPEG_RES_INT		0x1c0
+#define VE_MPEG_ERROR			0x1c4
+#define VE_MPEG_CTR_MB			0x1c8
+#define VE_MPEG_ROT_LUMA		0x1cc
+#define VE_MPEG_ROT_CHROMA		0x1d0
+#define VE_MPEG_SDROT_CTRL		0x1d4
+#define VE_MPEG_RAM_WRITE_PTR		0x1e0
+#define VE_MPEG_RAM_WRITE_DATA		0x1e4
+
+#define VE_H264_FRAME_SIZE		0x200
+#define VE_H264_PIC_HDR			0x204
+#define VE_H264_SLICE_HDR		0x208
+#define VE_H264_SLICE_HDR2		0x20c
+#define VE_H264_PRED_WEIGHT		0x210
+#define VE_H264_QP_PARAM		0x21c
+#define VE_H264_CTRL			0x220
+#define VE_H264_TRIGGER			0x224
+#define VE_H264_STATUS			0x228
+#define VE_H264_CUR_MB_NUM		0x22c
+#define VE_H264_VLD_ADDR		0x230
+#define VE_H264_VLD_OFFSET		0x234
+#define VE_H264_VLD_LEN			0x238
+#define VE_H264_VLD_END			0x23c
+#define VE_H264_SDROT_CTRL		0x240
+#define VE_H264_OUTPUT_FRAME_IDX	0x24c
+#define VE_H264_EXTRA_BUFFER1		0x250
+#define VE_H264_EXTRA_BUFFER2		0x254
+#define VE_H264_BASIC_BITS		0x2dc
+#define VE_H264_RAM_WRITE_PTR		0x2e0
+#define VE_H264_RAM_WRITE_DATA		0x2e4
+
+#define VE_SRAM_H264_PRED_WEIGHT_TABLE	0x000
+#define VE_SRAM_H264_FRAMEBUFFER_LIST	0x400
+#define VE_SRAM_H264_REF_LIST0		0x640
+#define VE_SRAM_H264_REF_LIST1		0x664
+#define VE_SRAM_H264_SCALING_LISTS	0x800
+
+#define VE_ISP_INPUT_SIZE		0xa00
+#define VE_ISP_INPUT_STRIDE		0xa04
+#define VE_ISP_CTRL			0xa08
+#define VE_ISP_INPUT_LUMA		0xa78
+#define VE_ISP_INPUT_CHROMA		0xa7c
+
+#define VE_AVC_PARAM			0xb04
+#define VE_AVC_QP			0xb08
+#define VE_AVC_MOTION_EST		0xb10
+#define VE_AVC_CTRL			0xb14
+#define VE_AVC_TRIGGER			0xb18
+#define VE_AVC_STATUS			0xb1c
+#define VE_AVC_BASIC_BITS		0xb20
+#define VE_AVC_UNK_BUF			0xb60
+#define VE_AVC_VLE_ADDR			0xb80
+#define VE_AVC_VLE_END			0xb84
+#define VE_AVC_VLE_OFFSET		0xb88
+#define VE_AVC_VLE_MAX			0xb8c
+#define VE_AVC_VLE_LENGTH		0xb90
+#define VE_AVC_REF_LUMA			0xba0
+#define VE_AVC_REF_CHROMA		0xba4
+#define VE_AVC_REC_LUMA			0xbb0
+#define VE_AVC_REC_CHROMA		0xbb4
+#define VE_AVC_REF_SLUMA		0xbb8
+#define VE_AVC_REC_SLUMA		0xbbc
+#define VE_AVC_MB_INFO			0xbc0
+
+#endif
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
new file mode 100644
index 000000000000..8bbec294bc91
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
@@ -0,0 +1,497 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <media/videobuf2-dma-contig.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "sunxi_cedrus_common.h"
+#include "sunxi_cedrus_mpeg2.h"
+#include "sunxi_cedrus_dec.h"
+#include "sunxi_cedrus_hw.h"
+
+/* Flags that indicate a format can be used for capture/output. */
+#define SUNXI_CEDRUS_CAPTURE	BIT(0)
+#define SUNXI_CEDRUS_OUTPUT	BIT(1)
+
+#define SUNXI_CEDRUS_MIN_WIDTH	16U
+#define SUNXI_CEDRUS_MIN_HEIGHT	16U
+#define SUNXI_CEDRUS_MAX_WIDTH	3840U
+#define SUNXI_CEDRUS_MAX_HEIGHT	2160U
+
+static struct sunxi_cedrus_fmt formats[] = {
+	{
+		.fourcc = V4L2_PIX_FMT_MB32_NV12,
+		.types	= SUNXI_CEDRUS_CAPTURE,
+		.depth = 2,
+		.num_planes = 2,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_MPEG2_FRAME,
+		.types	= SUNXI_CEDRUS_OUTPUT,
+		.num_planes = 1,
+	},
+};
+
+#define NUM_FORMATS ARRAY_SIZE(formats)
+
+static struct sunxi_cedrus_fmt *find_format(struct v4l2_format *f)
+{
+	struct sunxi_cedrus_fmt *fmt;
+	unsigned int k;
+
+	for (k = 0; k < NUM_FORMATS; k++) {
+		fmt = &formats[k];
+		if (fmt->fourcc == f->fmt.pix_mp.pixelformat)
+			break;
+	}
+
+	if (k == NUM_FORMATS)
+		return NULL;
+
+	return &formats[k];
+}
+
+static inline struct sunxi_cedrus_ctx *file2ctx(struct file *file)
+{
+	return container_of(file->private_data, struct sunxi_cedrus_ctx, fh);
+}
+
+static int vidioc_querycap(struct file *file, void *priv,
+			   struct v4l2_capability *cap)
+{
+	strncpy(cap->driver, SUNXI_CEDRUS_NAME, sizeof(cap->driver) - 1);
+	strncpy(cap->card, SUNXI_CEDRUS_NAME, sizeof(cap->card) - 1);
+	snprintf(cap->bus_info, sizeof(cap->bus_info),
+		 "platform:%s", SUNXI_CEDRUS_NAME);
+	cap->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
+	cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
+	return 0;
+}
+
+static int enum_fmt(struct v4l2_fmtdesc *f, u32 type)
+{
+	struct sunxi_cedrus_fmt *fmt;
+	int i, num = 0;
+
+	for (i = 0; i < NUM_FORMATS; ++i) {
+		if (formats[i].types & type) {
+			/* index-th format of type type found ? */
+			if (num == f->index)
+				break;
+			/*
+			 * Correct type but haven't reached our index yet,
+			 * just increment per-type index
+			 */
+			++num;
+		}
+	}
+
+	if (i < NUM_FORMATS) {
+		fmt = &formats[i];
+		f->pixelformat = fmt->fourcc;
+		return 0;
+	}
+
+	return -EINVAL;
+}
+
+static int vidioc_enum_fmt_vid_cap(struct file *file, void *priv,
+				   struct v4l2_fmtdesc *f)
+{
+	return enum_fmt(f, SUNXI_CEDRUS_CAPTURE);
+}
+
+static int vidioc_enum_fmt_vid_out(struct file *file, void *priv,
+				   struct v4l2_fmtdesc *f)
+{
+	return enum_fmt(f, SUNXI_CEDRUS_OUTPUT);
+}
+
+static int vidioc_g_fmt(struct sunxi_cedrus_ctx *ctx, struct v4l2_format *f)
+{
+	switch (f->type) {
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		f->fmt.pix_mp = ctx->dst_fmt;
+		break;
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		f->fmt.pix_mp = ctx->src_fmt;
+		break;
+	default:
+		dev_dbg(ctx->dev->dev, "invalid buf type\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int vidioc_g_fmt_vid_out(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	return vidioc_g_fmt(file2ctx(file), f);
+}
+
+static int vidioc_g_fmt_vid_cap(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	return vidioc_g_fmt(file2ctx(file), f);
+}
+
+static int vidioc_try_fmt(struct v4l2_format *f, struct sunxi_cedrus_fmt *fmt)
+{
+	int i;
+	__u32 bpl;
+
+	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+	f->fmt.pix_mp.num_planes = fmt->num_planes;
+
+	switch (f->type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		if (f->fmt.pix_mp.plane_fmt[0].sizeimage == 0)
+			return -EINVAL;
+
+		f->fmt.pix_mp.plane_fmt[0].bytesperline = 0;
+		break;
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		/* Limit to hardware min/max. */
+		f->fmt.pix_mp.width = clamp(f->fmt.pix_mp.width,
+			SUNXI_CEDRUS_MIN_WIDTH, SUNXI_CEDRUS_MAX_WIDTH);
+		f->fmt.pix_mp.height = clamp(f->fmt.pix_mp.height,
+			SUNXI_CEDRUS_MIN_HEIGHT, SUNXI_CEDRUS_MAX_HEIGHT);
+
+		for (i = 0; i < f->fmt.pix_mp.num_planes; ++i) {
+			bpl = (f->fmt.pix_mp.width * fmt->depth) >> 3;
+			f->fmt.pix_mp.plane_fmt[i].bytesperline = bpl;
+			f->fmt.pix_mp.plane_fmt[i].sizeimage =
+				f->fmt.pix_mp.height * bpl;
+		}
+		break;
+	}
+	return 0;
+}
+
+static int vidioc_try_fmt_vid_cap(struct file *file, void *priv,
+				  struct v4l2_format *f)
+{
+	struct sunxi_cedrus_fmt *fmt;
+	struct sunxi_cedrus_ctx *ctx = file2ctx(file);
+
+	fmt = find_format(f);
+	if (!fmt) {
+		f->fmt.pix_mp.pixelformat = formats[0].fourcc;
+		fmt = find_format(f);
+	}
+	if (!(fmt->types & SUNXI_CEDRUS_CAPTURE)) {
+		v4l2_err(&ctx->dev->v4l2_dev,
+			 "Fourcc format (0x%08x) invalid.\n",
+			 f->fmt.pix_mp.pixelformat);
+		return -EINVAL;
+	}
+
+	return vidioc_try_fmt(f, fmt);
+}
+
+static int vidioc_try_fmt_vid_out(struct file *file, void *priv,
+				  struct v4l2_format *f)
+{
+	struct sunxi_cedrus_fmt *fmt;
+	struct sunxi_cedrus_ctx *ctx = file2ctx(file);
+
+	fmt = find_format(f);
+	if (!fmt) {
+		f->fmt.pix_mp.pixelformat = formats[0].fourcc;
+		fmt = find_format(f);
+	}
+	if (!(fmt->types & SUNXI_CEDRUS_OUTPUT)) {
+		v4l2_err(&ctx->dev->v4l2_dev,
+			 "Fourcc format (0x%08x) invalid.\n",
+			 f->fmt.pix_mp.pixelformat);
+		return -EINVAL;
+	}
+
+	return vidioc_try_fmt(f, fmt);
+}
+
+static int vidioc_s_fmt(struct sunxi_cedrus_ctx *ctx, struct v4l2_format *f)
+{
+	struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
+	struct sunxi_cedrus_fmt *fmt;
+	int i, ret = 0;
+
+	switch (f->type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		ctx->vpu_src_fmt = find_format(f);
+		ctx->src_fmt = *pix_fmt_mp;
+		break;
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		fmt = find_format(f);
+		ctx->vpu_dst_fmt = fmt;
+
+		for (i = 0; i < fmt->num_planes; ++i) {
+			pix_fmt_mp->plane_fmt[i].bytesperline =
+				pix_fmt_mp->width * fmt->depth;
+			pix_fmt_mp->plane_fmt[i].sizeimage =
+				pix_fmt_mp->plane_fmt[i].bytesperline
+				* pix_fmt_mp->height;
+		}
+		ctx->dst_fmt = *pix_fmt_mp;
+		break;
+	default:
+		dev_dbg(ctx->dev->dev, "invalid buf type\n");
+		return -EINVAL;
+	}
+
+	return ret;
+}
+
+static int vidioc_s_fmt_vid_cap(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	int ret;
+
+	ret = vidioc_try_fmt_vid_cap(file, priv, f);
+	if (ret)
+		return ret;
+
+	return vidioc_s_fmt(file2ctx(file), f);
+}
+
+static int vidioc_s_fmt_vid_out(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	int ret;
+
+	ret = vidioc_try_fmt_vid_out(file, priv, f);
+	if (ret)
+		return ret;
+
+	ret = vidioc_s_fmt(file2ctx(file), f);
+	return ret;
+}
+
+const struct v4l2_ioctl_ops sunxi_cedrus_ioctl_ops = {
+	.vidioc_querycap		= vidioc_querycap,
+
+	.vidioc_enum_fmt_vid_cap	= vidioc_enum_fmt_vid_cap,
+	.vidioc_g_fmt_vid_cap_mplane	= vidioc_g_fmt_vid_cap,
+	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap,
+	.vidioc_s_fmt_vid_cap_mplane	= vidioc_s_fmt_vid_cap,
+
+	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out,
+	.vidioc_g_fmt_vid_out_mplane	= vidioc_g_fmt_vid_out,
+	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out,
+	.vidioc_s_fmt_vid_out_mplane	= vidioc_s_fmt_vid_out,
+
+	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
+	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
+	.vidioc_qbuf			= v4l2_m2m_ioctl_qbuf,
+	.vidioc_dqbuf			= v4l2_m2m_ioctl_dqbuf,
+	.vidioc_prepare_buf		= v4l2_m2m_ioctl_prepare_buf,
+	.vidioc_create_bufs		= v4l2_m2m_ioctl_create_bufs,
+	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
+
+	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
+	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
+
+	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
+};
+
+static int sunxi_cedrus_queue_setup(struct vb2_queue *vq, unsigned int *nbufs,
+				    unsigned int *nplanes, unsigned int sizes[],
+				    struct device *alloc_devs[])
+{
+	struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(vq);
+
+	if (*nbufs < 1)
+		*nbufs = 1;
+
+	if (*nbufs > VIDEO_MAX_FRAME)
+		*nbufs = VIDEO_MAX_FRAME;
+
+	switch (vq->type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		*nplanes = ctx->vpu_src_fmt->num_planes;
+
+		sizes[0] = ctx->src_fmt.plane_fmt[0].sizeimage;
+		break;
+
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		*nplanes = ctx->vpu_dst_fmt->num_planes;
+
+		sizes[0] = round_up(ctx->dst_fmt.plane_fmt[0].sizeimage, 8);
+		sizes[1] = sizes[0];
+		break;
+
+	default:
+		dev_dbg(ctx->dev->dev, "invalid queue type: %d\n", vq->type);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int sunxi_cedrus_buf_init(struct vb2_buffer *vb)
+{
+	struct vb2_queue *vq = vb->vb2_queue;
+	struct sunxi_cedrus_ctx *ctx = container_of(vq->drv_priv,
+			struct sunxi_cedrus_ctx, fh);
+
+	if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+		ctx->dst_bufs[vb->index] = vb;
+
+	return 0;
+}
+
+static void sunxi_cedrus_buf_cleanup(struct vb2_buffer *vb)
+{
+	struct vb2_queue *vq = vb->vb2_queue;
+	struct sunxi_cedrus_ctx *ctx = container_of(vq->drv_priv,
+			struct sunxi_cedrus_ctx, fh);
+
+	if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+		ctx->dst_bufs[vb->index] = NULL;
+}
+
+static int sunxi_cedrus_buf_prepare(struct vb2_buffer *vb)
+{
+	struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	struct vb2_queue *vq = vb->vb2_queue;
+	int i;
+
+	dev_dbg(ctx->dev->dev, "type: %d\n", vb->vb2_queue->type);
+
+	switch (vq->type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		if (vb2_plane_size(vb, 0)
+		    < ctx->src_fmt.plane_fmt[0].sizeimage) {
+			dev_dbg(ctx->dev->dev, "plane too small for output\n");
+			return -EINVAL;
+		}
+		break;
+
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		for (i = 0; i < ctx->vpu_dst_fmt->num_planes; ++i) {
+			if (vb2_plane_size(vb, i)
+			    < ctx->dst_fmt.plane_fmt[i].sizeimage) {
+				dev_dbg(ctx->dev->dev,
+					"plane %d too small for capture\n", i);
+				break;
+			}
+		}
+
+		if (i != ctx->vpu_dst_fmt->num_planes)
+			return -EINVAL;
+		break;
+
+	default:
+		dev_dbg(ctx->dev->dev, "invalid queue type: %d\n", vq->type);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static void sunxi_cedrus_stop_streaming(struct vb2_queue *q)
+{
+	struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
+	struct vb2_v4l2_buffer *vbuf;
+	unsigned long flags;
+
+	flush_scheduled_work();
+	for (;;) {
+		spin_lock_irqsave(&ctx->dev->irq_lock, flags);
+
+		if (V4L2_TYPE_IS_OUTPUT(q->type))
+			vbuf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+		else
+			vbuf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+
+		spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
+
+		if (vbuf == NULL)
+			return;
+
+		v4l2_ctrl_request_complete(vbuf->vb2_buf.req_obj.req,
+					   &ctx->hdl);
+		v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
+	}
+}
+
+static void sunxi_cedrus_buf_queue(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+	struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+
+	v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, vbuf);
+}
+
+static struct vb2_ops sunxi_cedrus_qops = {
+	.queue_setup		= sunxi_cedrus_queue_setup,
+	.buf_prepare		= sunxi_cedrus_buf_prepare,
+	.buf_init		= sunxi_cedrus_buf_init,
+	.buf_cleanup		= sunxi_cedrus_buf_cleanup,
+	.buf_queue		= sunxi_cedrus_buf_queue,
+	.stop_streaming		= sunxi_cedrus_stop_streaming,
+	.wait_prepare		= vb2_ops_wait_prepare,
+	.wait_finish		= vb2_ops_wait_finish,
+};
+
+int sunxi_cedrus_queue_init(void *priv, struct vb2_queue *src_vq,
+			    struct vb2_queue *dst_vq)
+{
+	struct sunxi_cedrus_ctx *ctx = priv;
+	int ret;
+
+	src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+	src_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+	src_vq->drv_priv = ctx;
+	src_vq->buf_struct_size = sizeof(struct sunxi_cedrus_buffer);
+	src_vq->allow_zero_bytesused = 1;
+	src_vq->min_buffers_needed = 1;
+	src_vq->ops = &sunxi_cedrus_qops;
+	src_vq->mem_ops = &vb2_dma_contig_memops;
+	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	src_vq->lock = &ctx->dev->dev_mutex;
+	src_vq->dev = ctx->dev->dev;
+
+	ret = vb2_queue_init(src_vq);
+	if (ret)
+		return ret;
+
+	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+	dst_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+	dst_vq->drv_priv = ctx;
+	dst_vq->buf_struct_size = sizeof(struct sunxi_cedrus_buffer);
+	dst_vq->allow_zero_bytesused = 1;
+	dst_vq->min_buffers_needed = 1;
+	dst_vq->ops = &sunxi_cedrus_qops;
+	dst_vq->mem_ops = &vb2_dma_contig_memops;
+	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	dst_vq->lock = &ctx->dev->dev_mutex;
+	dst_vq->dev = ctx->dev->dev;
+
+	return vb2_queue_init(dst_vq);
+}
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.h
new file mode 100644
index 000000000000..d5b7f881c386
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.h
@@ -0,0 +1,31 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _SUNXI_CEDRUS_VIDEO_H_
+#define _SUNXI_CEDRUS_VIDEO_H_
+
+extern const struct v4l2_ioctl_ops sunxi_cedrus_ioctl_ops;
+
+int sunxi_cedrus_queue_init(void *priv, struct vb2_queue *src_vq,
+			    struct vb2_queue *dst_vq);
+
+#endif
-- 
2.16.3

[PATCH v2 07/10] media: platform: Add Sunxi-Cedrus VPU decoder driver

[Paul Kocialkowski]

This introduces the Sunxi-Cedrus VPU driver that supports the VPU found
in Allwinner SoCs, also known as Video Engine. It is implemented through
a v4l2 m2m decoder device and a media device (used for media requests).
So far, it only supports MPEG2 decoding.

Since this VPU is stateless, synchronization with media requests is
required in order to ensure consistency between frame headers that
contain metadata about the frame to process and the raw slice data that
is used to generate the frame.

This driver was made possible thanks to the long-standing effort
carried out by the linux-sunxi community in the interest of reverse
engineering, documenting and implementing support for Allwinner VPU.

Signed-off-by: Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
---
 drivers/media/platform/Kconfig                     |  15 +
 drivers/media/platform/Makefile                    |   1 +
 drivers/media/platform/sunxi/cedrus/Makefile       |   4 +
 drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c | 292 ++++++++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_common.h    | 105 +++++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.c | 228 ++++++++++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.h |  36 ++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.c  | 201 +++++++++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.h  |  29 ++
 .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c     | 157 +++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h     |  33 ++
 .../platform/sunxi/cedrus/sunxi_cedrus_regs.h      | 172 +++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_video.c     | 497 +++++++++++++++++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_video.h     |  31 ++
 14 files changed, 1801 insertions(+)
 create mode 100644 drivers/media/platform/sunxi/cedrus/Makefile
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 614fbef08ddc..77d89e84ed3b 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -488,6 +488,21 @@ config VIDEO_TI_VPE
 	  Support for the TI VPE(Video Processing Engine) block
 	  found on DRA7XX SoC.
 
+config VIDEO_SUNXI_CEDRUS
+	tristate "Sunxi-Cedrus VPU driver"
+	depends on VIDEO_DEV && VIDEO_V4L2 && MEDIA_CONTROLLER
+	depends on ARCH_SUNXI
+	depends on HAS_DMA
+	select VIDEOBUF2_DMA_CONTIG
+	select MEDIA_REQUEST_API
+	select V4L2_MEM2MEM_DEV
+	---help---
+	  Support for the VPU found in Allwinner SoCs, also known as the Cedar
+	  video engine.
+
+	  To compile this driver as a module, choose M here: the module
+	  will be called sunxi-cedrus.
+
 config VIDEO_TI_VPE_DEBUG
 	bool "VPE debug messages"
 	depends on VIDEO_TI_VPE
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 7f3080437be6..a2be170f6dff 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -72,6 +72,7 @@ obj-$(CONFIG_VIDEO_ROCKCHIP_RGA)	+= rockchip/rga/
 obj-y	+= omap/
 
 obj-$(CONFIG_VIDEO_AM437X_VPFE)		+= am437x/
+obj-$(CONFIG_VIDEO_SUNXI_CEDRUS)	+= sunxi/cedrus/
 
 obj-$(CONFIG_VIDEO_XILINX)		+= xilinx/
 
diff --git a/drivers/media/platform/sunxi/cedrus/Makefile b/drivers/media/platform/sunxi/cedrus/Makefile
new file mode 100644
index 000000000000..98f30df626a9
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/Makefile
@@ -0,0 +1,4 @@
+obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += sunxi-cedrus.o
+
+sunxi-cedrus-y = sunxi_cedrus.o sunxi_cedrus_video.o sunxi_cedrus_hw.o \
+		 sunxi_cedrus_dec.o sunxi_cedrus_mpeg2.o
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
new file mode 100644
index 000000000000..364799c00703
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
@@ -0,0 +1,292 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
+ * Copyright (C) 2016 Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel-FA/gS7QP4orQT0dZR+AlfA@public.gmane.org>
+ * Marek Szyprowski, <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/platform_device.h>
+#include <linux/module.h>
+#include <linux/of.h>
+
+#include <media/videobuf2-dma-contig.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "sunxi_cedrus_common.h"
+#include "sunxi_cedrus_video.h"
+#include "sunxi_cedrus_dec.h"
+#include "sunxi_cedrus_hw.h"
+
+static int sunxi_cedrus_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct sunxi_cedrus_ctx *ctx =
+		container_of(ctrl->handler, struct sunxi_cedrus_ctx, hdl);
+
+	switch (ctrl->id) {
+	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
+		/* This is kept in memory and used directly. */
+		break;
+	default:
+		v4l2_err(&ctx->dev->v4l2_dev, "Invalid control\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static const struct v4l2_ctrl_ops sunxi_cedrus_ctrl_ops = {
+	.s_ctrl = sunxi_cedrus_s_ctrl,
+};
+
+static const struct v4l2_ctrl_config sunxi_cedrus_ctrl_mpeg2_frame_hdr = {
+	.ops = &sunxi_cedrus_ctrl_ops,
+	.id = V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR,
+	.elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr),
+};
+
+static int sunxi_cedrus_open(struct file *file)
+{
+	struct sunxi_cedrus_dev *dev = video_drvdata(file);
+	struct sunxi_cedrus_ctx *ctx = NULL;
+	struct v4l2_ctrl_handler *hdl;
+	int rc = 0;
+
+	if (mutex_lock_interruptible(&dev->dev_mutex))
+		return -ERESTARTSYS;
+
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	if (!ctx) {
+		rc = -ENOMEM;
+		goto open_unlock;
+	}
+
+	INIT_WORK(&ctx->run_work, sunxi_cedrus_device_work);
+
+	INIT_LIST_HEAD(&ctx->src_list);
+	INIT_LIST_HEAD(&ctx->dst_list);
+
+	v4l2_fh_init(&ctx->fh, video_devdata(file));
+	file->private_data = &ctx->fh;
+	ctx->dev = dev;
+	hdl = &ctx->hdl;
+	v4l2_ctrl_handler_init(hdl, 2);
+
+	ctx->mpeg2_frame_hdr_ctrl = v4l2_ctrl_new_custom(hdl,
+			&sunxi_cedrus_ctrl_mpeg2_frame_hdr, NULL);
+	if (hdl->error) {
+		rc = hdl->error;
+		v4l2_ctrl_handler_free(hdl);
+		goto open_unlock;
+	}
+
+	ctx->fh.ctrl_handler = hdl;
+	v4l2_ctrl_handler_setup(hdl);
+
+	ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev, ctx,
+					    &sunxi_cedrus_queue_init);
+	if (IS_ERR(ctx->fh.m2m_ctx)) {
+		rc = PTR_ERR(ctx->fh.m2m_ctx);
+
+		v4l2_ctrl_handler_free(hdl);
+		kfree(ctx);
+		goto open_unlock;
+	}
+
+	v4l2_fh_add(&ctx->fh);
+
+	dev_dbg(dev->dev, "Created instance: %p, m2m_ctx: %p\n",
+		ctx, ctx->fh.m2m_ctx);
+
+open_unlock:
+	mutex_unlock(&dev->dev_mutex);
+	return rc;
+}
+
+static int sunxi_cedrus_release(struct file *file)
+{
+	struct sunxi_cedrus_dev *dev = video_drvdata(file);
+	struct sunxi_cedrus_ctx *ctx = container_of(file->private_data,
+			struct sunxi_cedrus_ctx, fh);
+
+	dev_dbg(dev->dev, "Releasing instance %p\n", ctx);
+
+	v4l2_fh_del(&ctx->fh);
+	v4l2_fh_exit(&ctx->fh);
+	v4l2_ctrl_handler_free(&ctx->hdl);
+	ctx->mpeg2_frame_hdr_ctrl = NULL;
+	mutex_lock(&dev->dev_mutex);
+	v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+	mutex_unlock(&dev->dev_mutex);
+	kfree(ctx);
+
+	return 0;
+}
+
+static const struct v4l2_file_operations sunxi_cedrus_fops = {
+	.owner		= THIS_MODULE,
+	.open		= sunxi_cedrus_open,
+	.release	= sunxi_cedrus_release,
+	.poll		= v4l2_m2m_fop_poll,
+	.unlocked_ioctl	= video_ioctl2,
+	.mmap		= v4l2_m2m_fop_mmap,
+};
+
+static const struct video_device sunxi_cedrus_video_device = {
+	.name		= SUNXI_CEDRUS_NAME,
+	.vfl_dir	= VFL_DIR_M2M,
+	.fops		= &sunxi_cedrus_fops,
+	.ioctl_ops	= &sunxi_cedrus_ioctl_ops,
+	.minor		= -1,
+	.release	= video_device_release_empty,
+};
+
+static const struct v4l2_m2m_ops sunxi_cedrus_m2m_ops = {
+	.device_run	= sunxi_cedrus_device_run,
+	.job_abort	= sunxi_cedrus_job_abort,
+};
+
+static const struct media_device_ops sunxi_cedrus_m2m_media_ops = {
+	.req_queue = vb2_request_queue,
+	.req_complete = sunxi_cedrus_request_complete,
+};
+
+static int sunxi_cedrus_probe(struct platform_device *pdev)
+{
+	struct sunxi_cedrus_dev *dev;
+	struct video_device *vfd;
+	int ret;
+
+	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
+	if (!dev)
+		return -ENOMEM;
+
+	dev->dev = &pdev->dev;
+	dev->pdev = pdev;
+
+	ret = sunxi_cedrus_hw_probe(dev);
+	if (ret) {
+		dev_err(&pdev->dev, "Failed to probe hardware\n");
+		return ret;
+	}
+
+	mutex_init(&dev->dev_mutex);
+	spin_lock_init(&dev->irq_lock);
+
+	dev->vfd = sunxi_cedrus_video_device;
+	vfd = &dev->vfd;
+	vfd->lock = &dev->dev_mutex;
+	vfd->v4l2_dev = &dev->v4l2_dev;
+
+	dev->mdev.dev = &pdev->dev;
+	strlcpy(dev->mdev.model, SUNXI_CEDRUS_NAME, sizeof(dev->mdev.model));
+	media_device_init(&dev->mdev);
+	dev->mdev.ops = &sunxi_cedrus_m2m_media_ops;
+	dev->v4l2_dev.mdev = &dev->mdev;
+	dev->pad[0].flags = MEDIA_PAD_FL_SINK;
+	dev->pad[1].flags = MEDIA_PAD_FL_SOURCE;
+	ret = media_entity_pads_init(&vfd->entity, 2, dev->pad);
+	if (ret)
+		return ret;
+
+	ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
+	if (ret)
+		goto unreg_media;
+
+	ret = video_register_device(vfd, VFL_TYPE_GRABBER, 0);
+	if (ret) {
+		v4l2_err(&dev->v4l2_dev, "Failed to register video device\n");
+		goto unreg_dev;
+	}
+
+	video_set_drvdata(vfd, dev);
+	snprintf(vfd->name, sizeof(vfd->name), "%s",
+		 sunxi_cedrus_video_device.name);
+	v4l2_info(&dev->v4l2_dev,
+		  "Device registered as /dev/video%d\n", vfd->num);
+
+	platform_set_drvdata(pdev, dev);
+
+	dev->m2m_dev = v4l2_m2m_init(&sunxi_cedrus_m2m_ops);
+	if (IS_ERR(dev->m2m_dev)) {
+		v4l2_err(&dev->v4l2_dev, "Failed to init mem2mem device\n");
+		ret = PTR_ERR(dev->m2m_dev);
+		goto err_m2m;
+	}
+
+	/* Register the media device node */
+	ret = media_device_register(&dev->mdev);
+	if (ret)
+		goto err_m2m;
+
+	return 0;
+
+err_m2m:
+	v4l2_m2m_release(dev->m2m_dev);
+	video_unregister_device(&dev->vfd);
+unreg_media:
+	media_device_unregister(&dev->mdev);
+unreg_dev:
+	v4l2_device_unregister(&dev->v4l2_dev);
+
+	return ret;
+}
+
+static int sunxi_cedrus_remove(struct platform_device *pdev)
+{
+	struct sunxi_cedrus_dev *dev = platform_get_drvdata(pdev);
+
+	v4l2_info(&dev->v4l2_dev, "Removing " SUNXI_CEDRUS_NAME);
+
+	if (media_devnode_is_registered(dev->mdev.devnode)) {
+		media_device_unregister(&dev->mdev);
+		media_device_cleanup(&dev->mdev);
+	}
+
+	v4l2_m2m_release(dev->m2m_dev);
+	video_unregister_device(&dev->vfd);
+	v4l2_device_unregister(&dev->v4l2_dev);
+	sunxi_cedrus_hw_remove(dev);
+
+	return 0;
+}
+
+#ifdef CONFIG_OF
+static const struct of_device_id of_sunxi_cedrus_match[] = {
+	{ .compatible = "allwinner,sun4i-a10-video-engine" },
+	{ /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, of_sunxi_cedrus_match);
+#endif
+
+static struct platform_driver sunxi_cedrus_driver = {
+	.probe		= sunxi_cedrus_probe,
+	.remove		= sunxi_cedrus_remove,
+	.driver		= {
+		.name	= SUNXI_CEDRUS_NAME,
+		.owner = THIS_MODULE,
+		.of_match_table = of_match_ptr(of_sunxi_cedrus_match),
+	},
+};
+module_platform_driver(sunxi_cedrus_driver);
+
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>");
+MODULE_DESCRIPTION("Sunxi-Cedrus VPU driver");
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
new file mode 100644
index 000000000000..bab806c7c28d
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
@@ -0,0 +1,105 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
+ * Copyright (C) 2016 Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel-FA/gS7QP4orQT0dZR+AlfA@public.gmane.org>
+ * Marek Szyprowski, <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _SUNXI_CEDRUS_COMMON_H_
+#define _SUNXI_CEDRUS_COMMON_H_
+
+#include <linux/platform_device.h>
+
+#include <media/videobuf2-v4l2.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+
+#define SUNXI_CEDRUS_NAME	"sunxi-cedrus"
+
+struct sunxi_cedrus_dev {
+	struct v4l2_device v4l2_dev;
+	struct video_device vfd;
+	struct media_device mdev;
+	struct media_pad pad[2];
+	struct platform_device *pdev;
+	struct device *dev;
+	struct v4l2_m2m_dev *m2m_dev;
+
+	/* Mutex for device file */
+	struct mutex dev_mutex;
+	/* Spinlock for interrupt */
+	spinlock_t irq_lock;
+
+	struct clk *mod_clk;
+	struct clk *ahb_clk;
+	struct clk *ram_clk;
+
+	struct reset_control *rstc;
+
+	struct regmap *syscon;
+
+	char *base;
+};
+
+struct sunxi_cedrus_fmt {
+	u32 fourcc;
+	int depth;
+	u32 types;
+	unsigned int num_planes;
+};
+
+struct sunxi_cedrus_ctx {
+	struct v4l2_fh fh;
+	struct sunxi_cedrus_dev	*dev;
+
+	struct sunxi_cedrus_fmt *vpu_src_fmt;
+	struct v4l2_pix_format_mplane src_fmt;
+	struct sunxi_cedrus_fmt *vpu_dst_fmt;
+	struct v4l2_pix_format_mplane dst_fmt;
+
+	struct v4l2_ctrl_handler hdl;
+
+	struct vb2_buffer *dst_bufs[VIDEO_MAX_FRAME];
+
+	struct v4l2_ctrl *mpeg2_frame_hdr_ctrl;
+	int job_abort;
+
+	struct work_struct try_schedule_work;
+	struct work_struct run_work;
+	struct list_head src_list;
+	struct list_head dst_list;
+};
+
+struct sunxi_cedrus_buffer {
+	struct vb2_v4l2_buffer vb;
+	enum vb2_buffer_state state;
+	struct list_head list;
+};
+
+static inline void sunxi_cedrus_write(struct sunxi_cedrus_dev *dev,
+				      u32 val, u32 reg)
+{
+	writel(val, dev->base + reg);
+}
+
+static inline u32 sunxi_cedrus_read(struct sunxi_cedrus_dev *dev, u32 reg)
+{
+	return readl(dev->base + reg);
+}
+
+#endif
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
new file mode 100644
index 000000000000..64ec412c68f2
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
@@ -0,0 +1,228 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
+ * Copyright (C) 2016 Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel-FA/gS7QP4orQT0dZR+AlfA@public.gmane.org>
+ * Marek Szyprowski, <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <media/videobuf2-dma-contig.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "sunxi_cedrus_common.h"
+#include "sunxi_cedrus_mpeg2.h"
+#include "sunxi_cedrus_dec.h"
+#include "sunxi_cedrus_hw.h"
+
+void sunxi_cedrus_device_work(struct work_struct *work)
+{
+	struct sunxi_cedrus_ctx *ctx = container_of(work,
+			struct sunxi_cedrus_ctx, run_work);
+	struct sunxi_cedrus_buffer *buffer_entry;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	unsigned long flags;
+
+	spin_lock_irqsave(&ctx->dev->irq_lock, flags);
+
+	if (list_empty(&ctx->src_list) ||
+	    list_empty(&ctx->dst_list)) {
+		pr_err("Empty source and/or destination buffers lists\n");
+		spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
+		return;
+	}
+
+	buffer_entry = list_last_entry(&ctx->src_list, struct sunxi_cedrus_buffer, list);
+	list_del(ctx->src_list.prev);
+
+	src_buf = &buffer_entry->vb;
+	v4l2_m2m_buf_done(src_buf, buffer_entry->state);
+
+	buffer_entry = list_last_entry(&ctx->dst_list, struct sunxi_cedrus_buffer, list);
+	list_del(ctx->dst_list.prev);
+
+	dst_buf = &buffer_entry->vb;
+	v4l2_m2m_buf_done(dst_buf, buffer_entry->state);
+
+	spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
+
+	v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx);
+}
+
+void sunxi_cedrus_device_run(void *priv)
+{
+	struct sunxi_cedrus_ctx *ctx = priv;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	struct media_request *src_req, *dst_req;
+	dma_addr_t src_buf_addr, dst_luma_addr, dst_chroma_addr;
+	unsigned long flags;
+	struct v4l2_ctrl_mpeg2_frame_hdr *mpeg2_frame_hdr;
+	bool mpeg1 = false;
+
+	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+	if (!src_buf) {
+		v4l2_err(&ctx->dev->v4l2_dev,
+			 "No source buffer to prepare\n");
+		return;
+	}
+
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+	if (!dst_buf) {
+		v4l2_err(&ctx->dev->v4l2_dev,
+			 "No destination buffer to prepare\n");
+		return;
+	}
+
+	src_buf_addr = vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf, 0);
+	dst_luma_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
+	dst_chroma_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 1);
+	if (!src_buf || !dst_luma_addr || !dst_chroma_addr) {
+		v4l2_err(&ctx->dev->v4l2_dev,
+			 "Acquiring kernel pointers to buffers failed\n");
+		return;
+	}
+
+	/* Apply request(s) controls if needed. */
+
+	src_req = src_buf->vb2_buf.req_obj.req;
+	dst_req = dst_buf->vb2_buf.req_obj.req;
+
+	if (src_req) {
+		if (src_req->state != MEDIA_REQUEST_STATE_QUEUED) {
+			v4l2_err(&ctx->dev->v4l2_dev,
+				 "Unexpected state for request %s\n",
+				 src_req->debug_str);
+			return;
+		}
+
+		v4l2_ctrl_request_setup(src_req, &ctx->hdl);
+	}
+
+	if (dst_req && dst_req != src_req) {
+		if (dst_req->state != MEDIA_REQUEST_STATE_QUEUED) {
+			v4l2_err(&ctx->dev->v4l2_dev,
+				 "Unexpected state for request %s\n",
+				 dst_req->debug_str);
+			return;
+		}
+
+		v4l2_ctrl_request_setup(dst_req, &ctx->hdl);
+	}
+
+	dst_buf->vb2_buf.timestamp = src_buf->vb2_buf.timestamp;
+	if (src_buf->flags & V4L2_BUF_FLAG_TIMECODE)
+		dst_buf->timecode = src_buf->timecode;
+	dst_buf->field = src_buf->field;
+	dst_buf->flags = src_buf->flags & (V4L2_BUF_FLAG_TIMECODE |
+		 V4L2_BUF_FLAG_KEYFRAME | V4L2_BUF_FLAG_PFRAME |
+		 V4L2_BUF_FLAG_BFRAME   | V4L2_BUF_FLAG_TSTAMP_SRC_MASK);
+
+	ctx->job_abort = 0;
+
+	spin_lock_irqsave(&ctx->dev->irq_lock, flags);
+
+	if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_MPEG2_FRAME) {
+		if (!ctx->mpeg2_frame_hdr_ctrl) {
+			v4l2_err(&ctx->dev->v4l2_dev,
+				 "Invalid MPEG2 frame header control\n");
+			ctx->job_abort = 1;
+			goto unlock_complete;
+		}
+
+		mpeg2_frame_hdr = (struct v4l2_ctrl_mpeg2_frame_hdr *)
+				ctx->mpeg2_frame_hdr_ctrl->p_new.p;
+
+		sunxi_cedrus_mpeg2_setup(ctx, src_buf_addr, dst_luma_addr,
+					 dst_chroma_addr, mpeg2_frame_hdr);
+
+		mpeg1 = mpeg2_frame_hdr->type == MPEG1;
+	} else {
+		ctx->job_abort = 1;
+	}
+
+unlock_complete:
+	spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
+
+	/* Complete request(s) controls if needed. */
+
+	if (src_req)
+		v4l2_ctrl_request_complete(src_req, &ctx->hdl);
+
+	if (dst_req && dst_req != src_req)
+		v4l2_ctrl_request_complete(dst_req, &ctx->hdl);
+
+	spin_lock_irqsave(&ctx->dev->irq_lock, flags);
+
+	if (!ctx->job_abort) {
+		if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_MPEG2_FRAME)
+			sunxi_cedrus_mpeg2_trigger(ctx, mpeg1);
+	} else {
+		v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
+		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
+	}
+
+	spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
+
+	if (ctx->job_abort)
+		v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx);
+}
+
+void sunxi_cedrus_job_abort(void *priv)
+{
+	struct sunxi_cedrus_ctx *ctx = priv;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	unsigned long flags;
+
+	ctx->job_abort = 1;
+
+	/*
+	 * V4L2 m2m and request API cleanup is done here while hardware state
+	 * cleanup is done in the interrupt context. Doing all the cleanup in
+	 * the interrupt context is a bit risky, since the job_abort call might
+	 * originate from the release hook, where interrupts have already been
+	 * disabled.
+	 */
+
+	spin_lock_irqsave(&ctx->dev->irq_lock, flags);
+
+	src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	if (src_buf)
+		v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
+
+	dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+	if (dst_buf)
+		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
+
+	spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
+
+	v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx);
+}
+
+void sunxi_cedrus_request_complete(struct media_request *req)
+{
+	struct sunxi_cedrus_ctx *ctx;
+
+	ctx = (struct sunxi_cedrus_ctx *)
+			vb2_core_request_find_buffer_priv(req);
+	if (ctx == NULL)
+		return;
+
+	v4l2_m2m_try_schedule(ctx->fh.m2m_ctx);
+}
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.h
new file mode 100644
index 000000000000..a1a9e4c98850
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.h
@@ -0,0 +1,36 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
+ * Copyright (C) 2016 Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel-FA/gS7QP4orQT0dZR+AlfA@public.gmane.org>
+ * Marek Szyprowski, <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _SUNXI_CEDRUS_DEC_H_
+#define _SUNXI_CEDRUS_DEC_H_
+
+extern const struct v4l2_ioctl_ops sunxi_cedrus_ioctl_ops;
+
+void sunxi_cedrus_device_work(struct work_struct *work);
+void sunxi_cedrus_device_run(void *priv);
+void sunxi_cedrus_job_abort(void *priv);
+void sunxi_cedrus_request_complete(struct media_request *req);
+
+int sunxi_cedrus_queue_init(void *priv, struct vb2_queue *src_vq,
+			    struct vb2_queue *dst_vq);
+
+#endif
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
new file mode 100644
index 000000000000..1d5d1d3e8531
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
@@ -0,0 +1,201 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
+ * Copyright (C) 2016 Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel-FA/gS7QP4orQT0dZR+AlfA@public.gmane.org>
+ * Marek Szyprowski, <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/platform_device.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/dma-mapping.h>
+#include <linux/mfd/syscon.h>
+#include <linux/interrupt.h>
+#include <linux/clk.h>
+#include <linux/regmap.h>
+#include <linux/reset.h>
+
+#include <media/videobuf2-core.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "sunxi_cedrus_common.h"
+#include "sunxi_cedrus_regs.h"
+
+#define SYSCON_SRAM_CTRL_REG0	0x0
+#define SYSCON_SRAM_C1_MAP_VE	0x7fffffff
+
+static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
+{
+	struct sunxi_cedrus_dev *dev = dev_id;
+	struct sunxi_cedrus_ctx *ctx;
+	struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
+	struct vb2_v4l2_buffer *src_vb, *dst_vb;
+	unsigned long flags;
+	unsigned int value, status;
+
+	spin_lock_irqsave(&dev->irq_lock, flags);
+
+	/* Disable MPEG interrupts and stop the MPEG engine */
+	value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
+	sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
+
+	status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
+	sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
+
+	sunxi_cedrus_write(dev, VE_CTRL_REINIT, VE_CTRL);
+
+	ctx = v4l2_m2m_get_curr_priv(dev->m2m_dev);
+	if (!ctx) {
+		pr_err("Instance released before the end of transaction\n");
+		spin_unlock_irqrestore(&dev->irq_lock, flags);
+
+		return IRQ_HANDLED;
+	}
+
+	src_vb = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	dst_vb = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+
+	if (!src_vb || !dst_vb) {
+		pr_err("Unable to get source and/or destination buffers\n");
+		spin_unlock_irqrestore(&dev->irq_lock, flags);
+
+		return IRQ_HANDLED;
+	}
+
+	src_buffer = container_of(src_vb, struct sunxi_cedrus_buffer, vb);
+	dst_buffer = container_of(dst_vb, struct sunxi_cedrus_buffer, vb);
+
+	/* First bit of MPEG_STATUS indicates success. */
+	if (ctx->job_abort || !(status & 0x01))
+		src_buffer->state = dst_buffer->state = VB2_BUF_STATE_ERROR;
+	else
+		src_buffer->state = dst_buffer->state = VB2_BUF_STATE_DONE;
+
+	list_add_tail(&src_buffer->list, &ctx->src_list);
+	list_add_tail(&dst_buffer->list, &ctx->dst_list);
+
+	spin_unlock_irqrestore(&dev->irq_lock, flags);
+
+	schedule_work(&ctx->run_work);
+
+	return IRQ_HANDLED;
+}
+
+int sunxi_cedrus_hw_probe(struct sunxi_cedrus_dev *dev)
+{
+	struct resource *res;
+	int irq_dec;
+	int ret;
+
+	irq_dec = platform_get_irq(dev->pdev, 0);
+	if (irq_dec <= 0) {
+		dev_err(dev->dev, "could not get ve IRQ\n");
+		return -ENXIO;
+	}
+	ret = devm_request_irq(dev->dev, irq_dec, sunxi_cedrus_ve_irq, 0,
+			       dev_name(dev->dev), dev);
+	if (ret) {
+		dev_err(dev->dev, "could not request ve IRQ\n");
+		return -ENXIO;
+	}
+
+	/*
+	 * The VPU is only able to handle bus addresses so we have to subtract
+	 * the RAM offset to the physcal addresses.
+	 */
+	dev->dev->dma_pfn_offset = PHYS_PFN_OFFSET;
+
+	ret = of_reserved_mem_device_init(dev->dev);
+	if (ret) {
+		dev_err(dev->dev, "could not reserve memory\n");
+		return -ENODEV;
+	}
+
+	dev->ahb_clk = devm_clk_get(dev->dev, "ahb");
+	if (IS_ERR(dev->ahb_clk)) {
+		dev_err(dev->dev, "failed to get ahb clock\n");
+		return PTR_ERR(dev->ahb_clk);
+	}
+	dev->mod_clk = devm_clk_get(dev->dev, "mod");
+	if (IS_ERR(dev->mod_clk)) {
+		dev_err(dev->dev, "failed to get mod clock\n");
+		return PTR_ERR(dev->mod_clk);
+	}
+	dev->ram_clk = devm_clk_get(dev->dev, "ram");
+	if (IS_ERR(dev->ram_clk)) {
+		dev_err(dev->dev, "failed to get ram clock\n");
+		return PTR_ERR(dev->ram_clk);
+	}
+
+	dev->rstc = devm_reset_control_get(dev->dev, NULL);
+
+	res = platform_get_resource(dev->pdev, IORESOURCE_MEM, 0);
+	dev->base = devm_ioremap_resource(dev->dev, res);
+	if (!dev->base)
+		dev_err(dev->dev, "could not maps MACC registers\n");
+
+	dev->syscon = syscon_regmap_lookup_by_phandle(dev->dev->of_node,
+						      "syscon");
+	if (IS_ERR(dev->syscon)) {
+		dev->syscon = NULL;
+	} else {
+		regmap_write_bits(dev->syscon, SYSCON_SRAM_CTRL_REG0,
+				  SYSCON_SRAM_C1_MAP_VE,
+				  SYSCON_SRAM_C1_MAP_VE);
+	}
+
+	ret = clk_prepare_enable(dev->ahb_clk);
+	if (ret) {
+		dev_err(dev->dev, "could not enable ahb clock\n");
+		return -EFAULT;
+	}
+	ret = clk_prepare_enable(dev->mod_clk);
+	if (ret) {
+		clk_disable_unprepare(dev->ahb_clk);
+		dev_err(dev->dev, "could not enable mod clock\n");
+		return -EFAULT;
+	}
+	ret = clk_prepare_enable(dev->ram_clk);
+	if (ret) {
+		clk_disable_unprepare(dev->mod_clk);
+		clk_disable_unprepare(dev->ahb_clk);
+		dev_err(dev->dev, "could not enable ram clock\n");
+		return -EFAULT;
+	}
+
+	ret = reset_control_reset(dev->rstc);
+	if (ret) {
+		clk_disable_unprepare(dev->ram_clk);
+		clk_disable_unprepare(dev->mod_clk);
+		clk_disable_unprepare(dev->ahb_clk);
+		dev_err(dev->dev, "could not reset device\n");
+		return -EFAULT;
+	}
+
+	return 0;
+}
+
+void sunxi_cedrus_hw_remove(struct sunxi_cedrus_dev *dev)
+{
+	reset_control_assert(dev->rstc);
+
+	clk_disable_unprepare(dev->ram_clk);
+	clk_disable_unprepare(dev->mod_clk);
+	clk_disable_unprepare(dev->ahb_clk);
+
+	of_reserved_mem_device_release(dev->dev);
+}
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
new file mode 100644
index 000000000000..8184d7591363
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
@@ -0,0 +1,29 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
+ * Copyright (C) 2016 Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel-FA/gS7QP4orQT0dZR+AlfA@public.gmane.org>
+ * Marek Szyprowski, <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _SUNXI_CEDRUS_HW_H_
+#define _SUNXI_CEDRUS_HW_H_
+
+int sunxi_cedrus_hw_probe(struct sunxi_cedrus_dev *dev);
+void sunxi_cedrus_hw_remove(struct sunxi_cedrus_dev *dev);
+
+#endif
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
new file mode 100644
index 000000000000..aa86c2f9c49a
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
@@ -0,0 +1,157 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
+ * Copyright (C) 2016 Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel-FA/gS7QP4orQT0dZR+AlfA@public.gmane.org>
+ * Marek Szyprowski, <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <media/videobuf2-dma-contig.h>
+
+#include "sunxi_cedrus_common.h"
+#include "sunxi_cedrus_regs.h"
+
+static const u8 mpeg_default_intra_quant[64] = {
+	 8, 16, 16, 19, 16, 19, 22, 22,
+	22, 22, 22, 22, 26, 24, 26, 27,
+	27, 27, 26, 26, 26, 26, 27, 27,
+	27, 29, 29, 29, 34, 34, 34, 29,
+	29, 29, 27, 27, 29, 29, 32, 32,
+	34, 34, 37, 38, 37, 35, 35, 34,
+	35, 38, 38, 40, 40, 40, 48, 48,
+	46, 46, 56, 56, 58, 69, 69, 83
+};
+
+#define m_iq(i) (((64 + i) << 8) | mpeg_default_intra_quant[i])
+
+static const u8 mpeg_default_non_intra_quant[64] = {
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16
+};
+
+#define m_niq(i) ((i << 8) | mpeg_default_non_intra_quant[i])
+
+void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
+			      dma_addr_t src_buf_addr,
+			      dma_addr_t dst_luma_addr,
+			      dma_addr_t dst_chroma_addr,
+			      struct v4l2_ctrl_mpeg2_frame_hdr *frame_hdr)
+{
+	struct sunxi_cedrus_dev *dev = ctx->dev;
+
+	u16 width = DIV_ROUND_UP(frame_hdr->width, 16);
+	u16 height = DIV_ROUND_UP(frame_hdr->height, 16);
+
+	u32 pic_header = 0;
+	u32 vld_len = frame_hdr->slice_len - frame_hdr->slice_pos;
+	int i;
+
+	struct vb2_buffer *fwd_vb2_buf, *bwd_vb2_buf;
+	dma_addr_t fwd_luma = 0, fwd_chroma = 0, bwd_luma = 0, bwd_chroma = 0;
+
+
+	fwd_vb2_buf = ctx->dst_bufs[frame_hdr->forward_ref_index];
+	if (fwd_vb2_buf) {
+		fwd_luma = vb2_dma_contig_plane_dma_addr(fwd_vb2_buf, 0);
+		fwd_chroma = vb2_dma_contig_plane_dma_addr(fwd_vb2_buf, 1);
+	}
+
+	bwd_vb2_buf = ctx->dst_bufs[frame_hdr->backward_ref_index];
+	if (bwd_vb2_buf) {
+		bwd_luma = vb2_dma_contig_plane_dma_addr(bwd_vb2_buf, 0);
+		bwd_chroma = vb2_dma_contig_plane_dma_addr(bwd_vb2_buf, 1);
+	}
+
+	/* Activate MPEG engine. */
+	sunxi_cedrus_write(dev, VE_CTRL_MPEG, VE_CTRL);
+
+	/* Set quantization matrices. */
+	for (i = 0; i < 64; i++) {
+		sunxi_cedrus_write(dev, m_iq(i), VE_MPEG_IQ_MIN_INPUT);
+		sunxi_cedrus_write(dev, m_niq(i), VE_MPEG_IQ_MIN_INPUT);
+	}
+
+	/* Set frame dimensions. */
+	sunxi_cedrus_write(dev, width << 8 | height, VE_MPEG_SIZE);
+	sunxi_cedrus_write(dev, width << 20 | height << 4, VE_MPEG_FRAME_SIZE);
+
+	/* Set MPEG picture header. */
+	pic_header |= (frame_hdr->picture_coding_type & 0xf) << 28;
+	pic_header |= (frame_hdr->f_code[0][0] & 0xf) << 24;
+	pic_header |= (frame_hdr->f_code[0][1] & 0xf) << 20;
+	pic_header |= (frame_hdr->f_code[1][0] & 0xf) << 16;
+	pic_header |= (frame_hdr->f_code[1][1] & 0xf) << 12;
+	pic_header |= (frame_hdr->intra_dc_precision & 0x3) << 10;
+	pic_header |= (frame_hdr->picture_structure & 0x3) << 8;
+	pic_header |= (frame_hdr->top_field_first & 0x1) << 7;
+	pic_header |= (frame_hdr->frame_pred_frame_dct & 0x1) << 6;
+	pic_header |= (frame_hdr->concealment_motion_vectors & 0x1) << 5;
+	pic_header |= (frame_hdr->q_scale_type & 0x1) << 4;
+	pic_header |= (frame_hdr->intra_vlc_format & 0x1) << 3;
+	pic_header |= (frame_hdr->alternate_scan & 0x1) << 2;
+	sunxi_cedrus_write(dev, pic_header, VE_MPEG_PIC_HDR);
+
+	/* Enable interrupt and an unknown control flag. */
+	sunxi_cedrus_write(dev, VE_MPEG_CTRL_MPEG2, VE_MPEG_CTRL);
+
+	/* Macroblock address. */
+	sunxi_cedrus_write(dev, 0, VE_MPEG_MBA);
+
+	/* Clear previous errors. */
+	sunxi_cedrus_write(dev, 0, VE_MPEG_ERROR);
+
+	/* Clear correct macroblocks register. */
+	sunxi_cedrus_write(dev, 0, VE_MPEG_CTR_MB);
+
+	/* Forward and backward prediction reference buffers. */
+	sunxi_cedrus_write(dev, fwd_luma, VE_MPEG_FWD_LUMA);
+	sunxi_cedrus_write(dev, fwd_chroma, VE_MPEG_FWD_CHROMA);
+	sunxi_cedrus_write(dev, bwd_luma, VE_MPEG_BACK_LUMA);
+	sunxi_cedrus_write(dev, bwd_chroma, VE_MPEG_BACK_CHROMA);
+
+	/* Desination luma and chroma buffers. */
+	sunxi_cedrus_write(dev, dst_luma_addr, VE_MPEG_REC_LUMA);
+	sunxi_cedrus_write(dev, dst_chroma_addr, VE_MPEG_REC_CHROMA);
+	sunxi_cedrus_write(dev, dst_luma_addr, VE_MPEG_ROT_LUMA);
+	sunxi_cedrus_write(dev, dst_chroma_addr, VE_MPEG_ROT_CHROMA);
+
+	/* Source offset and length in bits. */
+	sunxi_cedrus_write(dev, frame_hdr->slice_pos, VE_MPEG_VLD_OFFSET);
+	sunxi_cedrus_write(dev, vld_len, VE_MPEG_VLD_LEN);
+
+	/* Source beginning and end addresses. */
+	sunxi_cedrus_write(dev, VE_MPEG_VLD_ADDR_VAL(src_buf_addr),
+			   VE_MPEG_VLD_ADDR);
+	sunxi_cedrus_write(dev, src_buf_addr + VBV_SIZE - 1, VE_MPEG_VLD_END);
+}
+
+void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx, bool mpeg1)
+{
+	struct sunxi_cedrus_dev *dev = ctx->dev;
+
+	/* Trigger MPEG engine. */
+	if (mpeg1)
+		sunxi_cedrus_write(dev, VE_TRIG_MPEG1, VE_MPEG_TRIGGER);
+	else
+		sunxi_cedrus_write(dev, VE_TRIG_MPEG2, VE_MPEG_TRIGGER);
+}
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
new file mode 100644
index 000000000000..4ab532ff173c
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
@@ -0,0 +1,33 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
+ * Copyright (C) 2016 Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel-FA/gS7QP4orQT0dZR+AlfA@public.gmane.org>
+ * Marek Szyprowski, <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _SUNXI_CEDRUS_MPEG2_H_
+#define _SUNXI_CEDRUS_MPEG2_H_
+
+void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
+			      dma_addr_t src_buf_addr,
+			      dma_addr_t dst_luma_addr,
+			      dma_addr_t dst_chroma_addr,
+			      struct v4l2_ctrl_mpeg2_frame_hdr *frame_hdr);
+void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx, bool mpeg1);
+
+#endif
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h
new file mode 100644
index 000000000000..ff5f79878ada
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h
@@ -0,0 +1,172 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
+ * Copyright (C) 2016 Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel-FA/gS7QP4orQT0dZR+AlfA@public.gmane.org>
+ * Marek Szyprowski, <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _SUNXI_CEDRUS_REGS_H_
+#define _SUNXI_CEDRUS_REGS_H_
+
+/*
+ * For more information, consult http://linux-sunxi.org/VE_Register_guide
+ */
+
+/* Special registers values */
+
+/* VE_CTRL:
+ * The first 3 bits indicate the engine (0 for MPEG, 1 for H264, b for AVC...)
+ * The 16th and 17th bits indicate the memory type (3 for DDR3 32 bits)
+ * The 20th bit is unknown but needed
+ */
+#define VE_CTRL_MPEG		0x130000
+#define VE_CTRL_H264		0x130001
+#define VE_CTRL_AVC		0x13000b
+#define VE_CTRL_REINIT		0x130007
+
+/* VE_MPEG_CTRL:
+ * The bit 3 (0x8) is used to enable IRQs
+ * The other bits are unknown but needed
+ */
+#define VE_MPEG_CTRL_MPEG2	0x800001b8
+#define VE_MPEG_CTRL_MPEG4	(0x80084118 | BIT(7))
+#define VE_MPEG_CTRL_MPEG4_P	(VE_MPEG_CTRL_MPEG4 | BIT(12))
+
+/* VE_MPEG_VLD_ADDR:
+ * The bits 27 to 4 are used for the address
+ * The bits 31 to 28 (0x7) are used to select the MPEG or JPEG engine
+ */
+#define VE_MPEG_VLD_ADDR_VAL(x)	((x & 0x0ffffff0) | (x >> 28) | (0x7 << 28))
+
+/* VE_MPEG_TRIGGER:
+ * The first three bits are used to trigger the engine
+ * The bits 24 to 26 are used to select the input format (1 for MPEG1, 2 for
+ *                           MPEG2, 4 for MPEG4)
+ * The bit 21 (0x8) is used to disable bitstream error handling
+ *
+ * In MPEG4 the w*h value is somehow used for an offset, unknown but needed
+ */
+#define VE_TRIG_MPEG1		0x8100000f
+#define VE_TRIG_MPEG2		0x8200000f
+#define VE_TRIG_MPEG4(w, h)	(0x8400000d | ((w * h) << 8))
+
+/* VE_MPEG_SDROT_CTRL:
+ * The bit 8 at zero is used to disable x downscaling
+ * The bit 10 at 0 is used to disable y downscaling
+ * The other bits are unknown but needed
+ */
+#define VE_NO_SDROT_CTRL	0x40620000
+
+/* Decent size fo video buffering verifier */
+#define VBV_SIZE		(1024 * 1024)
+
+/* Registers addresses */
+#define VE_CTRL				0x000
+#define VE_VERSION			0x0f0
+
+#define VE_MPEG_PIC_HDR			0x100
+#define VE_MPEG_VOP_HDR			0x104
+#define VE_MPEG_SIZE			0x108
+#define VE_MPEG_FRAME_SIZE		0x10c
+#define VE_MPEG_MBA			0x110
+#define VE_MPEG_CTRL			0x114
+#define VE_MPEG_TRIGGER			0x118
+#define VE_MPEG_STATUS			0x11c
+#define VE_MPEG_TRBTRD_FIELD		0x120
+#define VE_MPEG_TRBTRD_FRAME		0x124
+#define VE_MPEG_VLD_ADDR		0x128
+#define VE_MPEG_VLD_OFFSET		0x12c
+#define VE_MPEG_VLD_LEN			0x130
+#define VE_MPEG_VLD_END			0x134
+#define VE_MPEG_MBH_ADDR		0x138
+#define VE_MPEG_DCAC_ADDR		0x13c
+#define VE_MPEG_NCF_ADDR		0x144
+#define VE_MPEG_REC_LUMA		0x148
+#define VE_MPEG_REC_CHROMA		0x14c
+#define VE_MPEG_FWD_LUMA		0x150
+#define VE_MPEG_FWD_CHROMA		0x154
+#define VE_MPEG_BACK_LUMA		0x158
+#define VE_MPEG_BACK_CHROMA		0x15c
+#define VE_MPEG_IQ_MIN_INPUT		0x180
+#define VE_MPEG_QP_INPUT		0x184
+#define VE_MPEG_JPEG_SIZE		0x1b8
+#define VE_MPEG_JPEG_RES_INT		0x1c0
+#define VE_MPEG_ERROR			0x1c4
+#define VE_MPEG_CTR_MB			0x1c8
+#define VE_MPEG_ROT_LUMA		0x1cc
+#define VE_MPEG_ROT_CHROMA		0x1d0
+#define VE_MPEG_SDROT_CTRL		0x1d4
+#define VE_MPEG_RAM_WRITE_PTR		0x1e0
+#define VE_MPEG_RAM_WRITE_DATA		0x1e4
+
+#define VE_H264_FRAME_SIZE		0x200
+#define VE_H264_PIC_HDR			0x204
+#define VE_H264_SLICE_HDR		0x208
+#define VE_H264_SLICE_HDR2		0x20c
+#define VE_H264_PRED_WEIGHT		0x210
+#define VE_H264_QP_PARAM		0x21c
+#define VE_H264_CTRL			0x220
+#define VE_H264_TRIGGER			0x224
+#define VE_H264_STATUS			0x228
+#define VE_H264_CUR_MB_NUM		0x22c
+#define VE_H264_VLD_ADDR		0x230
+#define VE_H264_VLD_OFFSET		0x234
+#define VE_H264_VLD_LEN			0x238
+#define VE_H264_VLD_END			0x23c
+#define VE_H264_SDROT_CTRL		0x240
+#define VE_H264_OUTPUT_FRAME_IDX	0x24c
+#define VE_H264_EXTRA_BUFFER1		0x250
+#define VE_H264_EXTRA_BUFFER2		0x254
+#define VE_H264_BASIC_BITS		0x2dc
+#define VE_H264_RAM_WRITE_PTR		0x2e0
+#define VE_H264_RAM_WRITE_DATA		0x2e4
+
+#define VE_SRAM_H264_PRED_WEIGHT_TABLE	0x000
+#define VE_SRAM_H264_FRAMEBUFFER_LIST	0x400
+#define VE_SRAM_H264_REF_LIST0		0x640
+#define VE_SRAM_H264_REF_LIST1		0x664
+#define VE_SRAM_H264_SCALING_LISTS	0x800
+
+#define VE_ISP_INPUT_SIZE		0xa00
+#define VE_ISP_INPUT_STRIDE		0xa04
+#define VE_ISP_CTRL			0xa08
+#define VE_ISP_INPUT_LUMA		0xa78
+#define VE_ISP_INPUT_CHROMA		0xa7c
+
+#define VE_AVC_PARAM			0xb04
+#define VE_AVC_QP			0xb08
+#define VE_AVC_MOTION_EST		0xb10
+#define VE_AVC_CTRL			0xb14
+#define VE_AVC_TRIGGER			0xb18
+#define VE_AVC_STATUS			0xb1c
+#define VE_AVC_BASIC_BITS		0xb20
+#define VE_AVC_UNK_BUF			0xb60
+#define VE_AVC_VLE_ADDR			0xb80
+#define VE_AVC_VLE_END			0xb84
+#define VE_AVC_VLE_OFFSET		0xb88
+#define VE_AVC_VLE_MAX			0xb8c
+#define VE_AVC_VLE_LENGTH		0xb90
+#define VE_AVC_REF_LUMA			0xba0
+#define VE_AVC_REF_CHROMA		0xba4
+#define VE_AVC_REC_LUMA			0xbb0
+#define VE_AVC_REC_CHROMA		0xbb4
+#define VE_AVC_REF_SLUMA		0xbb8
+#define VE_AVC_REC_SLUMA		0xbbc
+#define VE_AVC_MB_INFO			0xbc0
+
+#endif
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
new file mode 100644
index 000000000000..8bbec294bc91
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
@@ -0,0 +1,497 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
+ * Copyright (C) 2016 Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel-FA/gS7QP4orQT0dZR+AlfA@public.gmane.org>
+ * Marek Szyprowski, <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <media/videobuf2-dma-contig.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "sunxi_cedrus_common.h"
+#include "sunxi_cedrus_mpeg2.h"
+#include "sunxi_cedrus_dec.h"
+#include "sunxi_cedrus_hw.h"
+
+/* Flags that indicate a format can be used for capture/output. */
+#define SUNXI_CEDRUS_CAPTURE	BIT(0)
+#define SUNXI_CEDRUS_OUTPUT	BIT(1)
+
+#define SUNXI_CEDRUS_MIN_WIDTH	16U
+#define SUNXI_CEDRUS_MIN_HEIGHT	16U
+#define SUNXI_CEDRUS_MAX_WIDTH	3840U
+#define SUNXI_CEDRUS_MAX_HEIGHT	2160U
+
+static struct sunxi_cedrus_fmt formats[] = {
+	{
+		.fourcc = V4L2_PIX_FMT_MB32_NV12,
+		.types	= SUNXI_CEDRUS_CAPTURE,
+		.depth = 2,
+		.num_planes = 2,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_MPEG2_FRAME,
+		.types	= SUNXI_CEDRUS_OUTPUT,
+		.num_planes = 1,
+	},
+};
+
+#define NUM_FORMATS ARRAY_SIZE(formats)
+
+static struct sunxi_cedrus_fmt *find_format(struct v4l2_format *f)
+{
+	struct sunxi_cedrus_fmt *fmt;
+	unsigned int k;
+
+	for (k = 0; k < NUM_FORMATS; k++) {
+		fmt = &formats[k];
+		if (fmt->fourcc == f->fmt.pix_mp.pixelformat)
+			break;
+	}
+
+	if (k == NUM_FORMATS)
+		return NULL;
+
+	return &formats[k];
+}
+
+static inline struct sunxi_cedrus_ctx *file2ctx(struct file *file)
+{
+	return container_of(file->private_data, struct sunxi_cedrus_ctx, fh);
+}
+
+static int vidioc_querycap(struct file *file, void *priv,
+			   struct v4l2_capability *cap)
+{
+	strncpy(cap->driver, SUNXI_CEDRUS_NAME, sizeof(cap->driver) - 1);
+	strncpy(cap->card, SUNXI_CEDRUS_NAME, sizeof(cap->card) - 1);
+	snprintf(cap->bus_info, sizeof(cap->bus_info),
+		 "platform:%s", SUNXI_CEDRUS_NAME);
+	cap->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
+	cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
+	return 0;
+}
+
+static int enum_fmt(struct v4l2_fmtdesc *f, u32 type)
+{
+	struct sunxi_cedrus_fmt *fmt;
+	int i, num = 0;
+
+	for (i = 0; i < NUM_FORMATS; ++i) {
+		if (formats[i].types & type) {
+			/* index-th format of type type found ? */
+			if (num == f->index)
+				break;
+			/*
+			 * Correct type but haven't reached our index yet,
+			 * just increment per-type index
+			 */
+			++num;
+		}
+	}
+
+	if (i < NUM_FORMATS) {
+		fmt = &formats[i];
+		f->pixelformat = fmt->fourcc;
+		return 0;
+	}
+
+	return -EINVAL;
+}
+
+static int vidioc_enum_fmt_vid_cap(struct file *file, void *priv,
+				   struct v4l2_fmtdesc *f)
+{
+	return enum_fmt(f, SUNXI_CEDRUS_CAPTURE);
+}
+
+static int vidioc_enum_fmt_vid_out(struct file *file, void *priv,
+				   struct v4l2_fmtdesc *f)
+{
+	return enum_fmt(f, SUNXI_CEDRUS_OUTPUT);
+}
+
+static int vidioc_g_fmt(struct sunxi_cedrus_ctx *ctx, struct v4l2_format *f)
+{
+	switch (f->type) {
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		f->fmt.pix_mp = ctx->dst_fmt;
+		break;
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		f->fmt.pix_mp = ctx->src_fmt;
+		break;
+	default:
+		dev_dbg(ctx->dev->dev, "invalid buf type\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int vidioc_g_fmt_vid_out(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	return vidioc_g_fmt(file2ctx(file), f);
+}
+
+static int vidioc_g_fmt_vid_cap(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	return vidioc_g_fmt(file2ctx(file), f);
+}
+
+static int vidioc_try_fmt(struct v4l2_format *f, struct sunxi_cedrus_fmt *fmt)
+{
+	int i;
+	__u32 bpl;
+
+	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+	f->fmt.pix_mp.num_planes = fmt->num_planes;
+
+	switch (f->type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		if (f->fmt.pix_mp.plane_fmt[0].sizeimage == 0)
+			return -EINVAL;
+
+		f->fmt.pix_mp.plane_fmt[0].bytesperline = 0;
+		break;
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		/* Limit to hardware min/max. */
+		f->fmt.pix_mp.width = clamp(f->fmt.pix_mp.width,
+			SUNXI_CEDRUS_MIN_WIDTH, SUNXI_CEDRUS_MAX_WIDTH);
+		f->fmt.pix_mp.height = clamp(f->fmt.pix_mp.height,
+			SUNXI_CEDRUS_MIN_HEIGHT, SUNXI_CEDRUS_MAX_HEIGHT);
+
+		for (i = 0; i < f->fmt.pix_mp.num_planes; ++i) {
+			bpl = (f->fmt.pix_mp.width * fmt->depth) >> 3;
+			f->fmt.pix_mp.plane_fmt[i].bytesperline = bpl;
+			f->fmt.pix_mp.plane_fmt[i].sizeimage =
+				f->fmt.pix_mp.height * bpl;
+		}
+		break;
+	}
+	return 0;
+}
+
+static int vidioc_try_fmt_vid_cap(struct file *file, void *priv,
+				  struct v4l2_format *f)
+{
+	struct sunxi_cedrus_fmt *fmt;
+	struct sunxi_cedrus_ctx *ctx = file2ctx(file);
+
+	fmt = find_format(f);
+	if (!fmt) {
+		f->fmt.pix_mp.pixelformat = formats[0].fourcc;
+		fmt = find_format(f);
+	}
+	if (!(fmt->types & SUNXI_CEDRUS_CAPTURE)) {
+		v4l2_err(&ctx->dev->v4l2_dev,
+			 "Fourcc format (0x%08x) invalid.\n",
+			 f->fmt.pix_mp.pixelformat);
+		return -EINVAL;
+	}
+
+	return vidioc_try_fmt(f, fmt);
+}
+
+static int vidioc_try_fmt_vid_out(struct file *file, void *priv,
+				  struct v4l2_format *f)
+{
+	struct sunxi_cedrus_fmt *fmt;
+	struct sunxi_cedrus_ctx *ctx = file2ctx(file);
+
+	fmt = find_format(f);
+	if (!fmt) {
+		f->fmt.pix_mp.pixelformat = formats[0].fourcc;
+		fmt = find_format(f);
+	}
+	if (!(fmt->types & SUNXI_CEDRUS_OUTPUT)) {
+		v4l2_err(&ctx->dev->v4l2_dev,
+			 "Fourcc format (0x%08x) invalid.\n",
+			 f->fmt.pix_mp.pixelformat);
+		return -EINVAL;
+	}
+
+	return vidioc_try_fmt(f, fmt);
+}
+
+static int vidioc_s_fmt(struct sunxi_cedrus_ctx *ctx, struct v4l2_format *f)
+{
+	struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
+	struct sunxi_cedrus_fmt *fmt;
+	int i, ret = 0;
+
+	switch (f->type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		ctx->vpu_src_fmt = find_format(f);
+		ctx->src_fmt = *pix_fmt_mp;
+		break;
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		fmt = find_format(f);
+		ctx->vpu_dst_fmt = fmt;
+
+		for (i = 0; i < fmt->num_planes; ++i) {
+			pix_fmt_mp->plane_fmt[i].bytesperline =
+				pix_fmt_mp->width * fmt->depth;
+			pix_fmt_mp->plane_fmt[i].sizeimage =
+				pix_fmt_mp->plane_fmt[i].bytesperline
+				* pix_fmt_mp->height;
+		}
+		ctx->dst_fmt = *pix_fmt_mp;
+		break;
+	default:
+		dev_dbg(ctx->dev->dev, "invalid buf type\n");
+		return -EINVAL;
+	}
+
+	return ret;
+}
+
+static int vidioc_s_fmt_vid_cap(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	int ret;
+
+	ret = vidioc_try_fmt_vid_cap(file, priv, f);
+	if (ret)
+		return ret;
+
+	return vidioc_s_fmt(file2ctx(file), f);
+}
+
+static int vidioc_s_fmt_vid_out(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	int ret;
+
+	ret = vidioc_try_fmt_vid_out(file, priv, f);
+	if (ret)
+		return ret;
+
+	ret = vidioc_s_fmt(file2ctx(file), f);
+	return ret;
+}
+
+const struct v4l2_ioctl_ops sunxi_cedrus_ioctl_ops = {
+	.vidioc_querycap		= vidioc_querycap,
+
+	.vidioc_enum_fmt_vid_cap	= vidioc_enum_fmt_vid_cap,
+	.vidioc_g_fmt_vid_cap_mplane	= vidioc_g_fmt_vid_cap,
+	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap,
+	.vidioc_s_fmt_vid_cap_mplane	= vidioc_s_fmt_vid_cap,
+
+	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out,
+	.vidioc_g_fmt_vid_out_mplane	= vidioc_g_fmt_vid_out,
+	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out,
+	.vidioc_s_fmt_vid_out_mplane	= vidioc_s_fmt_vid_out,
+
+	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
+	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
+	.vidioc_qbuf			= v4l2_m2m_ioctl_qbuf,
+	.vidioc_dqbuf			= v4l2_m2m_ioctl_dqbuf,
+	.vidioc_prepare_buf		= v4l2_m2m_ioctl_prepare_buf,
+	.vidioc_create_bufs		= v4l2_m2m_ioctl_create_bufs,
+	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
+
+	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
+	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
+
+	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
+};
+
+static int sunxi_cedrus_queue_setup(struct vb2_queue *vq, unsigned int *nbufs,
+				    unsigned int *nplanes, unsigned int sizes[],
+				    struct device *alloc_devs[])
+{
+	struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(vq);
+
+	if (*nbufs < 1)
+		*nbufs = 1;
+
+	if (*nbufs > VIDEO_MAX_FRAME)
+		*nbufs = VIDEO_MAX_FRAME;
+
+	switch (vq->type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		*nplanes = ctx->vpu_src_fmt->num_planes;
+
+		sizes[0] = ctx->src_fmt.plane_fmt[0].sizeimage;
+		break;
+
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		*nplanes = ctx->vpu_dst_fmt->num_planes;
+
+		sizes[0] = round_up(ctx->dst_fmt.plane_fmt[0].sizeimage, 8);
+		sizes[1] = sizes[0];
+		break;
+
+	default:
+		dev_dbg(ctx->dev->dev, "invalid queue type: %d\n", vq->type);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int sunxi_cedrus_buf_init(struct vb2_buffer *vb)
+{
+	struct vb2_queue *vq = vb->vb2_queue;
+	struct sunxi_cedrus_ctx *ctx = container_of(vq->drv_priv,
+			struct sunxi_cedrus_ctx, fh);
+
+	if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+		ctx->dst_bufs[vb->index] = vb;
+
+	return 0;
+}
+
+static void sunxi_cedrus_buf_cleanup(struct vb2_buffer *vb)
+{
+	struct vb2_queue *vq = vb->vb2_queue;
+	struct sunxi_cedrus_ctx *ctx = container_of(vq->drv_priv,
+			struct sunxi_cedrus_ctx, fh);
+
+	if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+		ctx->dst_bufs[vb->index] = NULL;
+}
+
+static int sunxi_cedrus_buf_prepare(struct vb2_buffer *vb)
+{
+	struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	struct vb2_queue *vq = vb->vb2_queue;
+	int i;
+
+	dev_dbg(ctx->dev->dev, "type: %d\n", vb->vb2_queue->type);
+
+	switch (vq->type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		if (vb2_plane_size(vb, 0)
+		    < ctx->src_fmt.plane_fmt[0].sizeimage) {
+			dev_dbg(ctx->dev->dev, "plane too small for output\n");
+			return -EINVAL;
+		}
+		break;
+
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		for (i = 0; i < ctx->vpu_dst_fmt->num_planes; ++i) {
+			if (vb2_plane_size(vb, i)
+			    < ctx->dst_fmt.plane_fmt[i].sizeimage) {
+				dev_dbg(ctx->dev->dev,
+					"plane %d too small for capture\n", i);
+				break;
+			}
+		}
+
+		if (i != ctx->vpu_dst_fmt->num_planes)
+			return -EINVAL;
+		break;
+
+	default:
+		dev_dbg(ctx->dev->dev, "invalid queue type: %d\n", vq->type);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static void sunxi_cedrus_stop_streaming(struct vb2_queue *q)
+{
+	struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
+	struct vb2_v4l2_buffer *vbuf;
+	unsigned long flags;
+
+	flush_scheduled_work();
+	for (;;) {
+		spin_lock_irqsave(&ctx->dev->irq_lock, flags);
+
+		if (V4L2_TYPE_IS_OUTPUT(q->type))
+			vbuf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+		else
+			vbuf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+
+		spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
+
+		if (vbuf == NULL)
+			return;
+
+		v4l2_ctrl_request_complete(vbuf->vb2_buf.req_obj.req,
+					   &ctx->hdl);
+		v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
+	}
+}
+
+static void sunxi_cedrus_buf_queue(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+	struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+
+	v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, vbuf);
+}
+
+static struct vb2_ops sunxi_cedrus_qops = {
+	.queue_setup		= sunxi_cedrus_queue_setup,
+	.buf_prepare		= sunxi_cedrus_buf_prepare,
+	.buf_init		= sunxi_cedrus_buf_init,
+	.buf_cleanup		= sunxi_cedrus_buf_cleanup,
+	.buf_queue		= sunxi_cedrus_buf_queue,
+	.stop_streaming		= sunxi_cedrus_stop_streaming,
+	.wait_prepare		= vb2_ops_wait_prepare,
+	.wait_finish		= vb2_ops_wait_finish,
+};
+
+int sunxi_cedrus_queue_init(void *priv, struct vb2_queue *src_vq,
+			    struct vb2_queue *dst_vq)
+{
+	struct sunxi_cedrus_ctx *ctx = priv;
+	int ret;
+
+	src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+	src_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+	src_vq->drv_priv = ctx;
+	src_vq->buf_struct_size = sizeof(struct sunxi_cedrus_buffer);
+	src_vq->allow_zero_bytesused = 1;
+	src_vq->min_buffers_needed = 1;
+	src_vq->ops = &sunxi_cedrus_qops;
+	src_vq->mem_ops = &vb2_dma_contig_memops;
+	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	src_vq->lock = &ctx->dev->dev_mutex;
+	src_vq->dev = ctx->dev->dev;
+
+	ret = vb2_queue_init(src_vq);
+	if (ret)
+		return ret;
+
+	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+	dst_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+	dst_vq->drv_priv = ctx;
+	dst_vq->buf_struct_size = sizeof(struct sunxi_cedrus_buffer);
+	dst_vq->allow_zero_bytesused = 1;
+	dst_vq->min_buffers_needed = 1;
+	dst_vq->ops = &sunxi_cedrus_qops;
+	dst_vq->mem_ops = &vb2_dma_contig_memops;
+	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	dst_vq->lock = &ctx->dev->dev_mutex;
+	dst_vq->dev = ctx->dev->dev;
+
+	return vb2_queue_init(dst_vq);
+}
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.h
new file mode 100644
index 000000000000..d5b7f881c386
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.h
@@ -0,0 +1,31 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
+ * Copyright (C) 2016 Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel-FA/gS7QP4orQT0dZR+AlfA@public.gmane.org>
+ * Marek Szyprowski, <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _SUNXI_CEDRUS_VIDEO_H_
+#define _SUNXI_CEDRUS_VIDEO_H_
+
+extern const struct v4l2_ioctl_ops sunxi_cedrus_ioctl_ops;
+
+int sunxi_cedrus_queue_init(void *priv, struct vb2_queue *src_vq,
+			    struct vb2_queue *dst_vq);
+
+#endif
-- 
2.16.3

[PATCH v2 07/10] media: platform: Add Sunxi-Cedrus VPU decoder driver

[Paul Kocialkowski]

This introduces the Sunxi-Cedrus VPU driver that supports the VPU found
in Allwinner SoCs, also known as Video Engine. It is implemented through
a v4l2 m2m decoder device and a media device (used for media requests).
So far, it only supports MPEG2 decoding.

Since this VPU is stateless, synchronization with media requests is
required in order to ensure consistency between frame headers that
contain metadata about the frame to process and the raw slice data that
is used to generate the frame.

This driver was made possible thanks to the long-standing effort
carried out by the linux-sunxi community in the interest of reverse
engineering, documenting and implementing support for Allwinner VPU.

Signed-off-by: Florent Revest <florent.revest@free-electrons.com>
Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
---
 drivers/media/platform/Kconfig                     |  15 +
 drivers/media/platform/Makefile                    |   1 +
 drivers/media/platform/sunxi/cedrus/Makefile       |   4 +
 drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c | 292 ++++++++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_common.h    | 105 +++++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.c | 228 ++++++++++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_dec.h |  36 ++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.c  | 201 +++++++++
 .../media/platform/sunxi/cedrus/sunxi_cedrus_hw.h  |  29 ++
 .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c     | 157 +++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h     |  33 ++
 .../platform/sunxi/cedrus/sunxi_cedrus_regs.h      | 172 +++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_video.c     | 497 +++++++++++++++++++++
 .../platform/sunxi/cedrus/sunxi_cedrus_video.h     |  31 ++
 14 files changed, 1801 insertions(+)
 create mode 100644 drivers/media/platform/sunxi/cedrus/Makefile
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
 create mode 100644 drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 614fbef08ddc..77d89e84ed3b 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -488,6 +488,21 @@ config VIDEO_TI_VPE
 	  Support for the TI VPE(Video Processing Engine) block
 	  found on DRA7XX SoC.
 
+config VIDEO_SUNXI_CEDRUS
+	tristate "Sunxi-Cedrus VPU driver"
+	depends on VIDEO_DEV && VIDEO_V4L2 && MEDIA_CONTROLLER
+	depends on ARCH_SUNXI
+	depends on HAS_DMA
+	select VIDEOBUF2_DMA_CONTIG
+	select MEDIA_REQUEST_API
+	select V4L2_MEM2MEM_DEV
+	---help---
+	  Support for the VPU found in Allwinner SoCs, also known as the Cedar
+	  video engine.
+
+	  To compile this driver as a module, choose M here: the module
+	  will be called sunxi-cedrus.
+
 config VIDEO_TI_VPE_DEBUG
 	bool "VPE debug messages"
 	depends on VIDEO_TI_VPE
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 7f3080437be6..a2be170f6dff 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -72,6 +72,7 @@ obj-$(CONFIG_VIDEO_ROCKCHIP_RGA)	+= rockchip/rga/
 obj-y	+= omap/
 
 obj-$(CONFIG_VIDEO_AM437X_VPFE)		+= am437x/
+obj-$(CONFIG_VIDEO_SUNXI_CEDRUS)	+= sunxi/cedrus/
 
 obj-$(CONFIG_VIDEO_XILINX)		+= xilinx/
 
diff --git a/drivers/media/platform/sunxi/cedrus/Makefile b/drivers/media/platform/sunxi/cedrus/Makefile
new file mode 100644
index 000000000000..98f30df626a9
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/Makefile
@@ -0,0 +1,4 @@
+obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += sunxi-cedrus.o
+
+sunxi-cedrus-y = sunxi_cedrus.o sunxi_cedrus_video.o sunxi_cedrus_hw.o \
+		 sunxi_cedrus_dec.o sunxi_cedrus_mpeg2.o
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
new file mode 100644
index 000000000000..364799c00703
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus.c
@@ -0,0 +1,292 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/platform_device.h>
+#include <linux/module.h>
+#include <linux/of.h>
+
+#include <media/videobuf2-dma-contig.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "sunxi_cedrus_common.h"
+#include "sunxi_cedrus_video.h"
+#include "sunxi_cedrus_dec.h"
+#include "sunxi_cedrus_hw.h"
+
+static int sunxi_cedrus_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct sunxi_cedrus_ctx *ctx =
+		container_of(ctrl->handler, struct sunxi_cedrus_ctx, hdl);
+
+	switch (ctrl->id) {
+	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
+		/* This is kept in memory and used directly. */
+		break;
+	default:
+		v4l2_err(&ctx->dev->v4l2_dev, "Invalid control\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static const struct v4l2_ctrl_ops sunxi_cedrus_ctrl_ops = {
+	.s_ctrl = sunxi_cedrus_s_ctrl,
+};
+
+static const struct v4l2_ctrl_config sunxi_cedrus_ctrl_mpeg2_frame_hdr = {
+	.ops = &sunxi_cedrus_ctrl_ops,
+	.id = V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR,
+	.elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr),
+};
+
+static int sunxi_cedrus_open(struct file *file)
+{
+	struct sunxi_cedrus_dev *dev = video_drvdata(file);
+	struct sunxi_cedrus_ctx *ctx = NULL;
+	struct v4l2_ctrl_handler *hdl;
+	int rc = 0;
+
+	if (mutex_lock_interruptible(&dev->dev_mutex))
+		return -ERESTARTSYS;
+
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	if (!ctx) {
+		rc = -ENOMEM;
+		goto open_unlock;
+	}
+
+	INIT_WORK(&ctx->run_work, sunxi_cedrus_device_work);
+
+	INIT_LIST_HEAD(&ctx->src_list);
+	INIT_LIST_HEAD(&ctx->dst_list);
+
+	v4l2_fh_init(&ctx->fh, video_devdata(file));
+	file->private_data = &ctx->fh;
+	ctx->dev = dev;
+	hdl = &ctx->hdl;
+	v4l2_ctrl_handler_init(hdl, 2);
+
+	ctx->mpeg2_frame_hdr_ctrl = v4l2_ctrl_new_custom(hdl,
+			&sunxi_cedrus_ctrl_mpeg2_frame_hdr, NULL);
+	if (hdl->error) {
+		rc = hdl->error;
+		v4l2_ctrl_handler_free(hdl);
+		goto open_unlock;
+	}
+
+	ctx->fh.ctrl_handler = hdl;
+	v4l2_ctrl_handler_setup(hdl);
+
+	ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(dev->m2m_dev, ctx,
+					    &sunxi_cedrus_queue_init);
+	if (IS_ERR(ctx->fh.m2m_ctx)) {
+		rc = PTR_ERR(ctx->fh.m2m_ctx);
+
+		v4l2_ctrl_handler_free(hdl);
+		kfree(ctx);
+		goto open_unlock;
+	}
+
+	v4l2_fh_add(&ctx->fh);
+
+	dev_dbg(dev->dev, "Created instance: %p, m2m_ctx: %p\n",
+		ctx, ctx->fh.m2m_ctx);
+
+open_unlock:
+	mutex_unlock(&dev->dev_mutex);
+	return rc;
+}
+
+static int sunxi_cedrus_release(struct file *file)
+{
+	struct sunxi_cedrus_dev *dev = video_drvdata(file);
+	struct sunxi_cedrus_ctx *ctx = container_of(file->private_data,
+			struct sunxi_cedrus_ctx, fh);
+
+	dev_dbg(dev->dev, "Releasing instance %p\n", ctx);
+
+	v4l2_fh_del(&ctx->fh);
+	v4l2_fh_exit(&ctx->fh);
+	v4l2_ctrl_handler_free(&ctx->hdl);
+	ctx->mpeg2_frame_hdr_ctrl = NULL;
+	mutex_lock(&dev->dev_mutex);
+	v4l2_m2m_ctx_release(ctx->fh.m2m_ctx);
+	mutex_unlock(&dev->dev_mutex);
+	kfree(ctx);
+
+	return 0;
+}
+
+static const struct v4l2_file_operations sunxi_cedrus_fops = {
+	.owner		= THIS_MODULE,
+	.open		= sunxi_cedrus_open,
+	.release	= sunxi_cedrus_release,
+	.poll		= v4l2_m2m_fop_poll,
+	.unlocked_ioctl	= video_ioctl2,
+	.mmap		= v4l2_m2m_fop_mmap,
+};
+
+static const struct video_device sunxi_cedrus_video_device = {
+	.name		= SUNXI_CEDRUS_NAME,
+	.vfl_dir	= VFL_DIR_M2M,
+	.fops		= &sunxi_cedrus_fops,
+	.ioctl_ops	= &sunxi_cedrus_ioctl_ops,
+	.minor		= -1,
+	.release	= video_device_release_empty,
+};
+
+static const struct v4l2_m2m_ops sunxi_cedrus_m2m_ops = {
+	.device_run	= sunxi_cedrus_device_run,
+	.job_abort	= sunxi_cedrus_job_abort,
+};
+
+static const struct media_device_ops sunxi_cedrus_m2m_media_ops = {
+	.req_queue = vb2_request_queue,
+	.req_complete = sunxi_cedrus_request_complete,
+};
+
+static int sunxi_cedrus_probe(struct platform_device *pdev)
+{
+	struct sunxi_cedrus_dev *dev;
+	struct video_device *vfd;
+	int ret;
+
+	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
+	if (!dev)
+		return -ENOMEM;
+
+	dev->dev = &pdev->dev;
+	dev->pdev = pdev;
+
+	ret = sunxi_cedrus_hw_probe(dev);
+	if (ret) {
+		dev_err(&pdev->dev, "Failed to probe hardware\n");
+		return ret;
+	}
+
+	mutex_init(&dev->dev_mutex);
+	spin_lock_init(&dev->irq_lock);
+
+	dev->vfd = sunxi_cedrus_video_device;
+	vfd = &dev->vfd;
+	vfd->lock = &dev->dev_mutex;
+	vfd->v4l2_dev = &dev->v4l2_dev;
+
+	dev->mdev.dev = &pdev->dev;
+	strlcpy(dev->mdev.model, SUNXI_CEDRUS_NAME, sizeof(dev->mdev.model));
+	media_device_init(&dev->mdev);
+	dev->mdev.ops = &sunxi_cedrus_m2m_media_ops;
+	dev->v4l2_dev.mdev = &dev->mdev;
+	dev->pad[0].flags = MEDIA_PAD_FL_SINK;
+	dev->pad[1].flags = MEDIA_PAD_FL_SOURCE;
+	ret = media_entity_pads_init(&vfd->entity, 2, dev->pad);
+	if (ret)
+		return ret;
+
+	ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
+	if (ret)
+		goto unreg_media;
+
+	ret = video_register_device(vfd, VFL_TYPE_GRABBER, 0);
+	if (ret) {
+		v4l2_err(&dev->v4l2_dev, "Failed to register video device\n");
+		goto unreg_dev;
+	}
+
+	video_set_drvdata(vfd, dev);
+	snprintf(vfd->name, sizeof(vfd->name), "%s",
+		 sunxi_cedrus_video_device.name);
+	v4l2_info(&dev->v4l2_dev,
+		  "Device registered as /dev/video%d\n", vfd->num);
+
+	platform_set_drvdata(pdev, dev);
+
+	dev->m2m_dev = v4l2_m2m_init(&sunxi_cedrus_m2m_ops);
+	if (IS_ERR(dev->m2m_dev)) {
+		v4l2_err(&dev->v4l2_dev, "Failed to init mem2mem device\n");
+		ret = PTR_ERR(dev->m2m_dev);
+		goto err_m2m;
+	}
+
+	/* Register the media device node */
+	ret = media_device_register(&dev->mdev);
+	if (ret)
+		goto err_m2m;
+
+	return 0;
+
+err_m2m:
+	v4l2_m2m_release(dev->m2m_dev);
+	video_unregister_device(&dev->vfd);
+unreg_media:
+	media_device_unregister(&dev->mdev);
+unreg_dev:
+	v4l2_device_unregister(&dev->v4l2_dev);
+
+	return ret;
+}
+
+static int sunxi_cedrus_remove(struct platform_device *pdev)
+{
+	struct sunxi_cedrus_dev *dev = platform_get_drvdata(pdev);
+
+	v4l2_info(&dev->v4l2_dev, "Removing " SUNXI_CEDRUS_NAME);
+
+	if (media_devnode_is_registered(dev->mdev.devnode)) {
+		media_device_unregister(&dev->mdev);
+		media_device_cleanup(&dev->mdev);
+	}
+
+	v4l2_m2m_release(dev->m2m_dev);
+	video_unregister_device(&dev->vfd);
+	v4l2_device_unregister(&dev->v4l2_dev);
+	sunxi_cedrus_hw_remove(dev);
+
+	return 0;
+}
+
+#ifdef CONFIG_OF
+static const struct of_device_id of_sunxi_cedrus_match[] = {
+	{ .compatible = "allwinner,sun4i-a10-video-engine" },
+	{ /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, of_sunxi_cedrus_match);
+#endif
+
+static struct platform_driver sunxi_cedrus_driver = {
+	.probe		= sunxi_cedrus_probe,
+	.remove		= sunxi_cedrus_remove,
+	.driver		= {
+		.name	= SUNXI_CEDRUS_NAME,
+		.owner = THIS_MODULE,
+		.of_match_table = of_match_ptr(of_sunxi_cedrus_match),
+	},
+};
+module_platform_driver(sunxi_cedrus_driver);
+
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("Florent Revest <florent.revest@free-electrons.com>");
+MODULE_DESCRIPTION("Sunxi-Cedrus VPU driver");
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
new file mode 100644
index 000000000000..bab806c7c28d
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_common.h
@@ -0,0 +1,105 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _SUNXI_CEDRUS_COMMON_H_
+#define _SUNXI_CEDRUS_COMMON_H_
+
+#include <linux/platform_device.h>
+
+#include <media/videobuf2-v4l2.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ctrls.h>
+
+#define SUNXI_CEDRUS_NAME	"sunxi-cedrus"
+
+struct sunxi_cedrus_dev {
+	struct v4l2_device v4l2_dev;
+	struct video_device vfd;
+	struct media_device mdev;
+	struct media_pad pad[2];
+	struct platform_device *pdev;
+	struct device *dev;
+	struct v4l2_m2m_dev *m2m_dev;
+
+	/* Mutex for device file */
+	struct mutex dev_mutex;
+	/* Spinlock for interrupt */
+	spinlock_t irq_lock;
+
+	struct clk *mod_clk;
+	struct clk *ahb_clk;
+	struct clk *ram_clk;
+
+	struct reset_control *rstc;
+
+	struct regmap *syscon;
+
+	char *base;
+};
+
+struct sunxi_cedrus_fmt {
+	u32 fourcc;
+	int depth;
+	u32 types;
+	unsigned int num_planes;
+};
+
+struct sunxi_cedrus_ctx {
+	struct v4l2_fh fh;
+	struct sunxi_cedrus_dev	*dev;
+
+	struct sunxi_cedrus_fmt *vpu_src_fmt;
+	struct v4l2_pix_format_mplane src_fmt;
+	struct sunxi_cedrus_fmt *vpu_dst_fmt;
+	struct v4l2_pix_format_mplane dst_fmt;
+
+	struct v4l2_ctrl_handler hdl;
+
+	struct vb2_buffer *dst_bufs[VIDEO_MAX_FRAME];
+
+	struct v4l2_ctrl *mpeg2_frame_hdr_ctrl;
+	int job_abort;
+
+	struct work_struct try_schedule_work;
+	struct work_struct run_work;
+	struct list_head src_list;
+	struct list_head dst_list;
+};
+
+struct sunxi_cedrus_buffer {
+	struct vb2_v4l2_buffer vb;
+	enum vb2_buffer_state state;
+	struct list_head list;
+};
+
+static inline void sunxi_cedrus_write(struct sunxi_cedrus_dev *dev,
+				      u32 val, u32 reg)
+{
+	writel(val, dev->base + reg);
+}
+
+static inline u32 sunxi_cedrus_read(struct sunxi_cedrus_dev *dev, u32 reg)
+{
+	return readl(dev->base + reg);
+}
+
+#endif
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
new file mode 100644
index 000000000000..64ec412c68f2
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.c
@@ -0,0 +1,228 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <media/videobuf2-dma-contig.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "sunxi_cedrus_common.h"
+#include "sunxi_cedrus_mpeg2.h"
+#include "sunxi_cedrus_dec.h"
+#include "sunxi_cedrus_hw.h"
+
+void sunxi_cedrus_device_work(struct work_struct *work)
+{
+	struct sunxi_cedrus_ctx *ctx = container_of(work,
+			struct sunxi_cedrus_ctx, run_work);
+	struct sunxi_cedrus_buffer *buffer_entry;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	unsigned long flags;
+
+	spin_lock_irqsave(&ctx->dev->irq_lock, flags);
+
+	if (list_empty(&ctx->src_list) ||
+	    list_empty(&ctx->dst_list)) {
+		pr_err("Empty source and/or destination buffers lists\n");
+		spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
+		return;
+	}
+
+	buffer_entry = list_last_entry(&ctx->src_list, struct sunxi_cedrus_buffer, list);
+	list_del(ctx->src_list.prev);
+
+	src_buf = &buffer_entry->vb;
+	v4l2_m2m_buf_done(src_buf, buffer_entry->state);
+
+	buffer_entry = list_last_entry(&ctx->dst_list, struct sunxi_cedrus_buffer, list);
+	list_del(ctx->dst_list.prev);
+
+	dst_buf = &buffer_entry->vb;
+	v4l2_m2m_buf_done(dst_buf, buffer_entry->state);
+
+	spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
+
+	v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx);
+}
+
+void sunxi_cedrus_device_run(void *priv)
+{
+	struct sunxi_cedrus_ctx *ctx = priv;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	struct media_request *src_req, *dst_req;
+	dma_addr_t src_buf_addr, dst_luma_addr, dst_chroma_addr;
+	unsigned long flags;
+	struct v4l2_ctrl_mpeg2_frame_hdr *mpeg2_frame_hdr;
+	bool mpeg1 = false;
+
+	src_buf = v4l2_m2m_next_src_buf(ctx->fh.m2m_ctx);
+	if (!src_buf) {
+		v4l2_err(&ctx->dev->v4l2_dev,
+			 "No source buffer to prepare\n");
+		return;
+	}
+
+	dst_buf = v4l2_m2m_next_dst_buf(ctx->fh.m2m_ctx);
+	if (!dst_buf) {
+		v4l2_err(&ctx->dev->v4l2_dev,
+			 "No destination buffer to prepare\n");
+		return;
+	}
+
+	src_buf_addr = vb2_dma_contig_plane_dma_addr(&src_buf->vb2_buf, 0);
+	dst_luma_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0);
+	dst_chroma_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 1);
+	if (!src_buf || !dst_luma_addr || !dst_chroma_addr) {
+		v4l2_err(&ctx->dev->v4l2_dev,
+			 "Acquiring kernel pointers to buffers failed\n");
+		return;
+	}
+
+	/* Apply request(s) controls if needed. */
+
+	src_req = src_buf->vb2_buf.req_obj.req;
+	dst_req = dst_buf->vb2_buf.req_obj.req;
+
+	if (src_req) {
+		if (src_req->state != MEDIA_REQUEST_STATE_QUEUED) {
+			v4l2_err(&ctx->dev->v4l2_dev,
+				 "Unexpected state for request %s\n",
+				 src_req->debug_str);
+			return;
+		}
+
+		v4l2_ctrl_request_setup(src_req, &ctx->hdl);
+	}
+
+	if (dst_req && dst_req != src_req) {
+		if (dst_req->state != MEDIA_REQUEST_STATE_QUEUED) {
+			v4l2_err(&ctx->dev->v4l2_dev,
+				 "Unexpected state for request %s\n",
+				 dst_req->debug_str);
+			return;
+		}
+
+		v4l2_ctrl_request_setup(dst_req, &ctx->hdl);
+	}
+
+	dst_buf->vb2_buf.timestamp = src_buf->vb2_buf.timestamp;
+	if (src_buf->flags & V4L2_BUF_FLAG_TIMECODE)
+		dst_buf->timecode = src_buf->timecode;
+	dst_buf->field = src_buf->field;
+	dst_buf->flags = src_buf->flags & (V4L2_BUF_FLAG_TIMECODE |
+		 V4L2_BUF_FLAG_KEYFRAME | V4L2_BUF_FLAG_PFRAME |
+		 V4L2_BUF_FLAG_BFRAME   | V4L2_BUF_FLAG_TSTAMP_SRC_MASK);
+
+	ctx->job_abort = 0;
+
+	spin_lock_irqsave(&ctx->dev->irq_lock, flags);
+
+	if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_MPEG2_FRAME) {
+		if (!ctx->mpeg2_frame_hdr_ctrl) {
+			v4l2_err(&ctx->dev->v4l2_dev,
+				 "Invalid MPEG2 frame header control\n");
+			ctx->job_abort = 1;
+			goto unlock_complete;
+		}
+
+		mpeg2_frame_hdr = (struct v4l2_ctrl_mpeg2_frame_hdr *)
+				ctx->mpeg2_frame_hdr_ctrl->p_new.p;
+
+		sunxi_cedrus_mpeg2_setup(ctx, src_buf_addr, dst_luma_addr,
+					 dst_chroma_addr, mpeg2_frame_hdr);
+
+		mpeg1 = mpeg2_frame_hdr->type == MPEG1;
+	} else {
+		ctx->job_abort = 1;
+	}
+
+unlock_complete:
+	spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
+
+	/* Complete request(s) controls if needed. */
+
+	if (src_req)
+		v4l2_ctrl_request_complete(src_req, &ctx->hdl);
+
+	if (dst_req && dst_req != src_req)
+		v4l2_ctrl_request_complete(dst_req, &ctx->hdl);
+
+	spin_lock_irqsave(&ctx->dev->irq_lock, flags);
+
+	if (!ctx->job_abort) {
+		if (ctx->vpu_src_fmt->fourcc == V4L2_PIX_FMT_MPEG2_FRAME)
+			sunxi_cedrus_mpeg2_trigger(ctx, mpeg1);
+	} else {
+		v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
+		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
+	}
+
+	spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
+
+	if (ctx->job_abort)
+		v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx);
+}
+
+void sunxi_cedrus_job_abort(void *priv)
+{
+	struct sunxi_cedrus_ctx *ctx = priv;
+	struct vb2_v4l2_buffer *src_buf, *dst_buf;
+	unsigned long flags;
+
+	ctx->job_abort = 1;
+
+	/*
+	 * V4L2 m2m and request API cleanup is done here while hardware state
+	 * cleanup is done in the interrupt context. Doing all the cleanup in
+	 * the interrupt context is a bit risky, since the job_abort call might
+	 * originate from the release hook, where interrupts have already been
+	 * disabled.
+	 */
+
+	spin_lock_irqsave(&ctx->dev->irq_lock, flags);
+
+	src_buf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	if (src_buf)
+		v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR);
+
+	dst_buf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+	if (dst_buf)
+		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
+
+	spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
+
+	v4l2_m2m_job_finish(ctx->dev->m2m_dev, ctx->fh.m2m_ctx);
+}
+
+void sunxi_cedrus_request_complete(struct media_request *req)
+{
+	struct sunxi_cedrus_ctx *ctx;
+
+	ctx = (struct sunxi_cedrus_ctx *)
+			vb2_core_request_find_buffer_priv(req);
+	if (ctx == NULL)
+		return;
+
+	v4l2_m2m_try_schedule(ctx->fh.m2m_ctx);
+}
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.h
new file mode 100644
index 000000000000..a1a9e4c98850
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_dec.h
@@ -0,0 +1,36 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _SUNXI_CEDRUS_DEC_H_
+#define _SUNXI_CEDRUS_DEC_H_
+
+extern const struct v4l2_ioctl_ops sunxi_cedrus_ioctl_ops;
+
+void sunxi_cedrus_device_work(struct work_struct *work);
+void sunxi_cedrus_device_run(void *priv);
+void sunxi_cedrus_job_abort(void *priv);
+void sunxi_cedrus_request_complete(struct media_request *req);
+
+int sunxi_cedrus_queue_init(void *priv, struct vb2_queue *src_vq,
+			    struct vb2_queue *dst_vq);
+
+#endif
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
new file mode 100644
index 000000000000..1d5d1d3e8531
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.c
@@ -0,0 +1,201 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/platform_device.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/dma-mapping.h>
+#include <linux/mfd/syscon.h>
+#include <linux/interrupt.h>
+#include <linux/clk.h>
+#include <linux/regmap.h>
+#include <linux/reset.h>
+
+#include <media/videobuf2-core.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "sunxi_cedrus_common.h"
+#include "sunxi_cedrus_regs.h"
+
+#define SYSCON_SRAM_CTRL_REG0	0x0
+#define SYSCON_SRAM_C1_MAP_VE	0x7fffffff
+
+static irqreturn_t sunxi_cedrus_ve_irq(int irq, void *dev_id)
+{
+	struct sunxi_cedrus_dev *dev = dev_id;
+	struct sunxi_cedrus_ctx *ctx;
+	struct sunxi_cedrus_buffer *src_buffer, *dst_buffer;
+	struct vb2_v4l2_buffer *src_vb, *dst_vb;
+	unsigned long flags;
+	unsigned int value, status;
+
+	spin_lock_irqsave(&dev->irq_lock, flags);
+
+	/* Disable MPEG interrupts and stop the MPEG engine */
+	value = sunxi_cedrus_read(dev, VE_MPEG_CTRL);
+	sunxi_cedrus_write(dev, value & (~0xf), VE_MPEG_CTRL);
+
+	status = sunxi_cedrus_read(dev, VE_MPEG_STATUS);
+	sunxi_cedrus_write(dev, 0x0000c00f, VE_MPEG_STATUS);
+
+	sunxi_cedrus_write(dev, VE_CTRL_REINIT, VE_CTRL);
+
+	ctx = v4l2_m2m_get_curr_priv(dev->m2m_dev);
+	if (!ctx) {
+		pr_err("Instance released before the end of transaction\n");
+		spin_unlock_irqrestore(&dev->irq_lock, flags);
+
+		return IRQ_HANDLED;
+	}
+
+	src_vb = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+	dst_vb = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+
+	if (!src_vb || !dst_vb) {
+		pr_err("Unable to get source and/or destination buffers\n");
+		spin_unlock_irqrestore(&dev->irq_lock, flags);
+
+		return IRQ_HANDLED;
+	}
+
+	src_buffer = container_of(src_vb, struct sunxi_cedrus_buffer, vb);
+	dst_buffer = container_of(dst_vb, struct sunxi_cedrus_buffer, vb);
+
+	/* First bit of MPEG_STATUS indicates success. */
+	if (ctx->job_abort || !(status & 0x01))
+		src_buffer->state = dst_buffer->state = VB2_BUF_STATE_ERROR;
+	else
+		src_buffer->state = dst_buffer->state = VB2_BUF_STATE_DONE;
+
+	list_add_tail(&src_buffer->list, &ctx->src_list);
+	list_add_tail(&dst_buffer->list, &ctx->dst_list);
+
+	spin_unlock_irqrestore(&dev->irq_lock, flags);
+
+	schedule_work(&ctx->run_work);
+
+	return IRQ_HANDLED;
+}
+
+int sunxi_cedrus_hw_probe(struct sunxi_cedrus_dev *dev)
+{
+	struct resource *res;
+	int irq_dec;
+	int ret;
+
+	irq_dec = platform_get_irq(dev->pdev, 0);
+	if (irq_dec <= 0) {
+		dev_err(dev->dev, "could not get ve IRQ\n");
+		return -ENXIO;
+	}
+	ret = devm_request_irq(dev->dev, irq_dec, sunxi_cedrus_ve_irq, 0,
+			       dev_name(dev->dev), dev);
+	if (ret) {
+		dev_err(dev->dev, "could not request ve IRQ\n");
+		return -ENXIO;
+	}
+
+	/*
+	 * The VPU is only able to handle bus addresses so we have to subtract
+	 * the RAM offset to the physcal addresses.
+	 */
+	dev->dev->dma_pfn_offset = PHYS_PFN_OFFSET;
+
+	ret = of_reserved_mem_device_init(dev->dev);
+	if (ret) {
+		dev_err(dev->dev, "could not reserve memory\n");
+		return -ENODEV;
+	}
+
+	dev->ahb_clk = devm_clk_get(dev->dev, "ahb");
+	if (IS_ERR(dev->ahb_clk)) {
+		dev_err(dev->dev, "failed to get ahb clock\n");
+		return PTR_ERR(dev->ahb_clk);
+	}
+	dev->mod_clk = devm_clk_get(dev->dev, "mod");
+	if (IS_ERR(dev->mod_clk)) {
+		dev_err(dev->dev, "failed to get mod clock\n");
+		return PTR_ERR(dev->mod_clk);
+	}
+	dev->ram_clk = devm_clk_get(dev->dev, "ram");
+	if (IS_ERR(dev->ram_clk)) {
+		dev_err(dev->dev, "failed to get ram clock\n");
+		return PTR_ERR(dev->ram_clk);
+	}
+
+	dev->rstc = devm_reset_control_get(dev->dev, NULL);
+
+	res = platform_get_resource(dev->pdev, IORESOURCE_MEM, 0);
+	dev->base = devm_ioremap_resource(dev->dev, res);
+	if (!dev->base)
+		dev_err(dev->dev, "could not maps MACC registers\n");
+
+	dev->syscon = syscon_regmap_lookup_by_phandle(dev->dev->of_node,
+						      "syscon");
+	if (IS_ERR(dev->syscon)) {
+		dev->syscon = NULL;
+	} else {
+		regmap_write_bits(dev->syscon, SYSCON_SRAM_CTRL_REG0,
+				  SYSCON_SRAM_C1_MAP_VE,
+				  SYSCON_SRAM_C1_MAP_VE);
+	}
+
+	ret = clk_prepare_enable(dev->ahb_clk);
+	if (ret) {
+		dev_err(dev->dev, "could not enable ahb clock\n");
+		return -EFAULT;
+	}
+	ret = clk_prepare_enable(dev->mod_clk);
+	if (ret) {
+		clk_disable_unprepare(dev->ahb_clk);
+		dev_err(dev->dev, "could not enable mod clock\n");
+		return -EFAULT;
+	}
+	ret = clk_prepare_enable(dev->ram_clk);
+	if (ret) {
+		clk_disable_unprepare(dev->mod_clk);
+		clk_disable_unprepare(dev->ahb_clk);
+		dev_err(dev->dev, "could not enable ram clock\n");
+		return -EFAULT;
+	}
+
+	ret = reset_control_reset(dev->rstc);
+	if (ret) {
+		clk_disable_unprepare(dev->ram_clk);
+		clk_disable_unprepare(dev->mod_clk);
+		clk_disable_unprepare(dev->ahb_clk);
+		dev_err(dev->dev, "could not reset device\n");
+		return -EFAULT;
+	}
+
+	return 0;
+}
+
+void sunxi_cedrus_hw_remove(struct sunxi_cedrus_dev *dev)
+{
+	reset_control_assert(dev->rstc);
+
+	clk_disable_unprepare(dev->ram_clk);
+	clk_disable_unprepare(dev->mod_clk);
+	clk_disable_unprepare(dev->ahb_clk);
+
+	of_reserved_mem_device_release(dev->dev);
+}
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
new file mode 100644
index 000000000000..8184d7591363
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_hw.h
@@ -0,0 +1,29 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _SUNXI_CEDRUS_HW_H_
+#define _SUNXI_CEDRUS_HW_H_
+
+int sunxi_cedrus_hw_probe(struct sunxi_cedrus_dev *dev);
+void sunxi_cedrus_hw_remove(struct sunxi_cedrus_dev *dev);
+
+#endif
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
new file mode 100644
index 000000000000..aa86c2f9c49a
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.c
@@ -0,0 +1,157 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <media/videobuf2-dma-contig.h>
+
+#include "sunxi_cedrus_common.h"
+#include "sunxi_cedrus_regs.h"
+
+static const u8 mpeg_default_intra_quant[64] = {
+	 8, 16, 16, 19, 16, 19, 22, 22,
+	22, 22, 22, 22, 26, 24, 26, 27,
+	27, 27, 26, 26, 26, 26, 27, 27,
+	27, 29, 29, 29, 34, 34, 34, 29,
+	29, 29, 27, 27, 29, 29, 32, 32,
+	34, 34, 37, 38, 37, 35, 35, 34,
+	35, 38, 38, 40, 40, 40, 48, 48,
+	46, 46, 56, 56, 58, 69, 69, 83
+};
+
+#define m_iq(i) (((64 + i) << 8) | mpeg_default_intra_quant[i])
+
+static const u8 mpeg_default_non_intra_quant[64] = {
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16,
+	16, 16, 16, 16, 16, 16, 16, 16
+};
+
+#define m_niq(i) ((i << 8) | mpeg_default_non_intra_quant[i])
+
+void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
+			      dma_addr_t src_buf_addr,
+			      dma_addr_t dst_luma_addr,
+			      dma_addr_t dst_chroma_addr,
+			      struct v4l2_ctrl_mpeg2_frame_hdr *frame_hdr)
+{
+	struct sunxi_cedrus_dev *dev = ctx->dev;
+
+	u16 width = DIV_ROUND_UP(frame_hdr->width, 16);
+	u16 height = DIV_ROUND_UP(frame_hdr->height, 16);
+
+	u32 pic_header = 0;
+	u32 vld_len = frame_hdr->slice_len - frame_hdr->slice_pos;
+	int i;
+
+	struct vb2_buffer *fwd_vb2_buf, *bwd_vb2_buf;
+	dma_addr_t fwd_luma = 0, fwd_chroma = 0, bwd_luma = 0, bwd_chroma = 0;
+
+
+	fwd_vb2_buf = ctx->dst_bufs[frame_hdr->forward_ref_index];
+	if (fwd_vb2_buf) {
+		fwd_luma = vb2_dma_contig_plane_dma_addr(fwd_vb2_buf, 0);
+		fwd_chroma = vb2_dma_contig_plane_dma_addr(fwd_vb2_buf, 1);
+	}
+
+	bwd_vb2_buf = ctx->dst_bufs[frame_hdr->backward_ref_index];
+	if (bwd_vb2_buf) {
+		bwd_luma = vb2_dma_contig_plane_dma_addr(bwd_vb2_buf, 0);
+		bwd_chroma = vb2_dma_contig_plane_dma_addr(bwd_vb2_buf, 1);
+	}
+
+	/* Activate MPEG engine. */
+	sunxi_cedrus_write(dev, VE_CTRL_MPEG, VE_CTRL);
+
+	/* Set quantization matrices. */
+	for (i = 0; i < 64; i++) {
+		sunxi_cedrus_write(dev, m_iq(i), VE_MPEG_IQ_MIN_INPUT);
+		sunxi_cedrus_write(dev, m_niq(i), VE_MPEG_IQ_MIN_INPUT);
+	}
+
+	/* Set frame dimensions. */
+	sunxi_cedrus_write(dev, width << 8 | height, VE_MPEG_SIZE);
+	sunxi_cedrus_write(dev, width << 20 | height << 4, VE_MPEG_FRAME_SIZE);
+
+	/* Set MPEG picture header. */
+	pic_header |= (frame_hdr->picture_coding_type & 0xf) << 28;
+	pic_header |= (frame_hdr->f_code[0][0] & 0xf) << 24;
+	pic_header |= (frame_hdr->f_code[0][1] & 0xf) << 20;
+	pic_header |= (frame_hdr->f_code[1][0] & 0xf) << 16;
+	pic_header |= (frame_hdr->f_code[1][1] & 0xf) << 12;
+	pic_header |= (frame_hdr->intra_dc_precision & 0x3) << 10;
+	pic_header |= (frame_hdr->picture_structure & 0x3) << 8;
+	pic_header |= (frame_hdr->top_field_first & 0x1) << 7;
+	pic_header |= (frame_hdr->frame_pred_frame_dct & 0x1) << 6;
+	pic_header |= (frame_hdr->concealment_motion_vectors & 0x1) << 5;
+	pic_header |= (frame_hdr->q_scale_type & 0x1) << 4;
+	pic_header |= (frame_hdr->intra_vlc_format & 0x1) << 3;
+	pic_header |= (frame_hdr->alternate_scan & 0x1) << 2;
+	sunxi_cedrus_write(dev, pic_header, VE_MPEG_PIC_HDR);
+
+	/* Enable interrupt and an unknown control flag. */
+	sunxi_cedrus_write(dev, VE_MPEG_CTRL_MPEG2, VE_MPEG_CTRL);
+
+	/* Macroblock address. */
+	sunxi_cedrus_write(dev, 0, VE_MPEG_MBA);
+
+	/* Clear previous errors. */
+	sunxi_cedrus_write(dev, 0, VE_MPEG_ERROR);
+
+	/* Clear correct macroblocks register. */
+	sunxi_cedrus_write(dev, 0, VE_MPEG_CTR_MB);
+
+	/* Forward and backward prediction reference buffers. */
+	sunxi_cedrus_write(dev, fwd_luma, VE_MPEG_FWD_LUMA);
+	sunxi_cedrus_write(dev, fwd_chroma, VE_MPEG_FWD_CHROMA);
+	sunxi_cedrus_write(dev, bwd_luma, VE_MPEG_BACK_LUMA);
+	sunxi_cedrus_write(dev, bwd_chroma, VE_MPEG_BACK_CHROMA);
+
+	/* Desination luma and chroma buffers. */
+	sunxi_cedrus_write(dev, dst_luma_addr, VE_MPEG_REC_LUMA);
+	sunxi_cedrus_write(dev, dst_chroma_addr, VE_MPEG_REC_CHROMA);
+	sunxi_cedrus_write(dev, dst_luma_addr, VE_MPEG_ROT_LUMA);
+	sunxi_cedrus_write(dev, dst_chroma_addr, VE_MPEG_ROT_CHROMA);
+
+	/* Source offset and length in bits. */
+	sunxi_cedrus_write(dev, frame_hdr->slice_pos, VE_MPEG_VLD_OFFSET);
+	sunxi_cedrus_write(dev, vld_len, VE_MPEG_VLD_LEN);
+
+	/* Source beginning and end addresses. */
+	sunxi_cedrus_write(dev, VE_MPEG_VLD_ADDR_VAL(src_buf_addr),
+			   VE_MPEG_VLD_ADDR);
+	sunxi_cedrus_write(dev, src_buf_addr + VBV_SIZE - 1, VE_MPEG_VLD_END);
+}
+
+void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx, bool mpeg1)
+{
+	struct sunxi_cedrus_dev *dev = ctx->dev;
+
+	/* Trigger MPEG engine. */
+	if (mpeg1)
+		sunxi_cedrus_write(dev, VE_TRIG_MPEG1, VE_MPEG_TRIGGER);
+	else
+		sunxi_cedrus_write(dev, VE_TRIG_MPEG2, VE_MPEG_TRIGGER);
+}
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
new file mode 100644
index 000000000000..4ab532ff173c
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_mpeg2.h
@@ -0,0 +1,33 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _SUNXI_CEDRUS_MPEG2_H_
+#define _SUNXI_CEDRUS_MPEG2_H_
+
+void sunxi_cedrus_mpeg2_setup(struct sunxi_cedrus_ctx *ctx,
+			      dma_addr_t src_buf_addr,
+			      dma_addr_t dst_luma_addr,
+			      dma_addr_t dst_chroma_addr,
+			      struct v4l2_ctrl_mpeg2_frame_hdr *frame_hdr);
+void sunxi_cedrus_mpeg2_trigger(struct sunxi_cedrus_ctx *ctx, bool mpeg1);
+
+#endif
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h
new file mode 100644
index 000000000000..ff5f79878ada
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_regs.h
@@ -0,0 +1,172 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _SUNXI_CEDRUS_REGS_H_
+#define _SUNXI_CEDRUS_REGS_H_
+
+/*
+ * For more information, consult http://linux-sunxi.org/VE_Register_guide
+ */
+
+/* Special registers values */
+
+/* VE_CTRL:
+ * The first 3 bits indicate the engine (0 for MPEG, 1 for H264, b for AVC...)
+ * The 16th and 17th bits indicate the memory type (3 for DDR3 32 bits)
+ * The 20th bit is unknown but needed
+ */
+#define VE_CTRL_MPEG		0x130000
+#define VE_CTRL_H264		0x130001
+#define VE_CTRL_AVC		0x13000b
+#define VE_CTRL_REINIT		0x130007
+
+/* VE_MPEG_CTRL:
+ * The bit 3 (0x8) is used to enable IRQs
+ * The other bits are unknown but needed
+ */
+#define VE_MPEG_CTRL_MPEG2	0x800001b8
+#define VE_MPEG_CTRL_MPEG4	(0x80084118 | BIT(7))
+#define VE_MPEG_CTRL_MPEG4_P	(VE_MPEG_CTRL_MPEG4 | BIT(12))
+
+/* VE_MPEG_VLD_ADDR:
+ * The bits 27 to 4 are used for the address
+ * The bits 31 to 28 (0x7) are used to select the MPEG or JPEG engine
+ */
+#define VE_MPEG_VLD_ADDR_VAL(x)	((x & 0x0ffffff0) | (x >> 28) | (0x7 << 28))
+
+/* VE_MPEG_TRIGGER:
+ * The first three bits are used to trigger the engine
+ * The bits 24 to 26 are used to select the input format (1 for MPEG1, 2 for
+ *                           MPEG2, 4 for MPEG4)
+ * The bit 21 (0x8) is used to disable bitstream error handling
+ *
+ * In MPEG4 the w*h value is somehow used for an offset, unknown but needed
+ */
+#define VE_TRIG_MPEG1		0x8100000f
+#define VE_TRIG_MPEG2		0x8200000f
+#define VE_TRIG_MPEG4(w, h)	(0x8400000d | ((w * h) << 8))
+
+/* VE_MPEG_SDROT_CTRL:
+ * The bit 8 at zero is used to disable x downscaling
+ * The bit 10 at 0 is used to disable y downscaling
+ * The other bits are unknown but needed
+ */
+#define VE_NO_SDROT_CTRL	0x40620000
+
+/* Decent size fo video buffering verifier */
+#define VBV_SIZE		(1024 * 1024)
+
+/* Registers addresses */
+#define VE_CTRL				0x000
+#define VE_VERSION			0x0f0
+
+#define VE_MPEG_PIC_HDR			0x100
+#define VE_MPEG_VOP_HDR			0x104
+#define VE_MPEG_SIZE			0x108
+#define VE_MPEG_FRAME_SIZE		0x10c
+#define VE_MPEG_MBA			0x110
+#define VE_MPEG_CTRL			0x114
+#define VE_MPEG_TRIGGER			0x118
+#define VE_MPEG_STATUS			0x11c
+#define VE_MPEG_TRBTRD_FIELD		0x120
+#define VE_MPEG_TRBTRD_FRAME		0x124
+#define VE_MPEG_VLD_ADDR		0x128
+#define VE_MPEG_VLD_OFFSET		0x12c
+#define VE_MPEG_VLD_LEN			0x130
+#define VE_MPEG_VLD_END			0x134
+#define VE_MPEG_MBH_ADDR		0x138
+#define VE_MPEG_DCAC_ADDR		0x13c
+#define VE_MPEG_NCF_ADDR		0x144
+#define VE_MPEG_REC_LUMA		0x148
+#define VE_MPEG_REC_CHROMA		0x14c
+#define VE_MPEG_FWD_LUMA		0x150
+#define VE_MPEG_FWD_CHROMA		0x154
+#define VE_MPEG_BACK_LUMA		0x158
+#define VE_MPEG_BACK_CHROMA		0x15c
+#define VE_MPEG_IQ_MIN_INPUT		0x180
+#define VE_MPEG_QP_INPUT		0x184
+#define VE_MPEG_JPEG_SIZE		0x1b8
+#define VE_MPEG_JPEG_RES_INT		0x1c0
+#define VE_MPEG_ERROR			0x1c4
+#define VE_MPEG_CTR_MB			0x1c8
+#define VE_MPEG_ROT_LUMA		0x1cc
+#define VE_MPEG_ROT_CHROMA		0x1d0
+#define VE_MPEG_SDROT_CTRL		0x1d4
+#define VE_MPEG_RAM_WRITE_PTR		0x1e0
+#define VE_MPEG_RAM_WRITE_DATA		0x1e4
+
+#define VE_H264_FRAME_SIZE		0x200
+#define VE_H264_PIC_HDR			0x204
+#define VE_H264_SLICE_HDR		0x208
+#define VE_H264_SLICE_HDR2		0x20c
+#define VE_H264_PRED_WEIGHT		0x210
+#define VE_H264_QP_PARAM		0x21c
+#define VE_H264_CTRL			0x220
+#define VE_H264_TRIGGER			0x224
+#define VE_H264_STATUS			0x228
+#define VE_H264_CUR_MB_NUM		0x22c
+#define VE_H264_VLD_ADDR		0x230
+#define VE_H264_VLD_OFFSET		0x234
+#define VE_H264_VLD_LEN			0x238
+#define VE_H264_VLD_END			0x23c
+#define VE_H264_SDROT_CTRL		0x240
+#define VE_H264_OUTPUT_FRAME_IDX	0x24c
+#define VE_H264_EXTRA_BUFFER1		0x250
+#define VE_H264_EXTRA_BUFFER2		0x254
+#define VE_H264_BASIC_BITS		0x2dc
+#define VE_H264_RAM_WRITE_PTR		0x2e0
+#define VE_H264_RAM_WRITE_DATA		0x2e4
+
+#define VE_SRAM_H264_PRED_WEIGHT_TABLE	0x000
+#define VE_SRAM_H264_FRAMEBUFFER_LIST	0x400
+#define VE_SRAM_H264_REF_LIST0		0x640
+#define VE_SRAM_H264_REF_LIST1		0x664
+#define VE_SRAM_H264_SCALING_LISTS	0x800
+
+#define VE_ISP_INPUT_SIZE		0xa00
+#define VE_ISP_INPUT_STRIDE		0xa04
+#define VE_ISP_CTRL			0xa08
+#define VE_ISP_INPUT_LUMA		0xa78
+#define VE_ISP_INPUT_CHROMA		0xa7c
+
+#define VE_AVC_PARAM			0xb04
+#define VE_AVC_QP			0xb08
+#define VE_AVC_MOTION_EST		0xb10
+#define VE_AVC_CTRL			0xb14
+#define VE_AVC_TRIGGER			0xb18
+#define VE_AVC_STATUS			0xb1c
+#define VE_AVC_BASIC_BITS		0xb20
+#define VE_AVC_UNK_BUF			0xb60
+#define VE_AVC_VLE_ADDR			0xb80
+#define VE_AVC_VLE_END			0xb84
+#define VE_AVC_VLE_OFFSET		0xb88
+#define VE_AVC_VLE_MAX			0xb8c
+#define VE_AVC_VLE_LENGTH		0xb90
+#define VE_AVC_REF_LUMA			0xba0
+#define VE_AVC_REF_CHROMA		0xba4
+#define VE_AVC_REC_LUMA			0xbb0
+#define VE_AVC_REC_CHROMA		0xbb4
+#define VE_AVC_REF_SLUMA		0xbb8
+#define VE_AVC_REC_SLUMA		0xbbc
+#define VE_AVC_MB_INFO			0xbc0
+
+#endif
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
new file mode 100644
index 000000000000..8bbec294bc91
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.c
@@ -0,0 +1,497 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include <media/videobuf2-dma-contig.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-mem2mem.h>
+
+#include "sunxi_cedrus_common.h"
+#include "sunxi_cedrus_mpeg2.h"
+#include "sunxi_cedrus_dec.h"
+#include "sunxi_cedrus_hw.h"
+
+/* Flags that indicate a format can be used for capture/output. */
+#define SUNXI_CEDRUS_CAPTURE	BIT(0)
+#define SUNXI_CEDRUS_OUTPUT	BIT(1)
+
+#define SUNXI_CEDRUS_MIN_WIDTH	16U
+#define SUNXI_CEDRUS_MIN_HEIGHT	16U
+#define SUNXI_CEDRUS_MAX_WIDTH	3840U
+#define SUNXI_CEDRUS_MAX_HEIGHT	2160U
+
+static struct sunxi_cedrus_fmt formats[] = {
+	{
+		.fourcc = V4L2_PIX_FMT_MB32_NV12,
+		.types	= SUNXI_CEDRUS_CAPTURE,
+		.depth = 2,
+		.num_planes = 2,
+	},
+	{
+		.fourcc = V4L2_PIX_FMT_MPEG2_FRAME,
+		.types	= SUNXI_CEDRUS_OUTPUT,
+		.num_planes = 1,
+	},
+};
+
+#define NUM_FORMATS ARRAY_SIZE(formats)
+
+static struct sunxi_cedrus_fmt *find_format(struct v4l2_format *f)
+{
+	struct sunxi_cedrus_fmt *fmt;
+	unsigned int k;
+
+	for (k = 0; k < NUM_FORMATS; k++) {
+		fmt = &formats[k];
+		if (fmt->fourcc == f->fmt.pix_mp.pixelformat)
+			break;
+	}
+
+	if (k == NUM_FORMATS)
+		return NULL;
+
+	return &formats[k];
+}
+
+static inline struct sunxi_cedrus_ctx *file2ctx(struct file *file)
+{
+	return container_of(file->private_data, struct sunxi_cedrus_ctx, fh);
+}
+
+static int vidioc_querycap(struct file *file, void *priv,
+			   struct v4l2_capability *cap)
+{
+	strncpy(cap->driver, SUNXI_CEDRUS_NAME, sizeof(cap->driver) - 1);
+	strncpy(cap->card, SUNXI_CEDRUS_NAME, sizeof(cap->card) - 1);
+	snprintf(cap->bus_info, sizeof(cap->bus_info),
+		 "platform:%s", SUNXI_CEDRUS_NAME);
+	cap->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING;
+	cap->capabilities = cap->device_caps | V4L2_CAP_DEVICE_CAPS;
+	return 0;
+}
+
+static int enum_fmt(struct v4l2_fmtdesc *f, u32 type)
+{
+	struct sunxi_cedrus_fmt *fmt;
+	int i, num = 0;
+
+	for (i = 0; i < NUM_FORMATS; ++i) {
+		if (formats[i].types & type) {
+			/* index-th format of type type found ? */
+			if (num == f->index)
+				break;
+			/*
+			 * Correct type but haven't reached our index yet,
+			 * just increment per-type index
+			 */
+			++num;
+		}
+	}
+
+	if (i < NUM_FORMATS) {
+		fmt = &formats[i];
+		f->pixelformat = fmt->fourcc;
+		return 0;
+	}
+
+	return -EINVAL;
+}
+
+static int vidioc_enum_fmt_vid_cap(struct file *file, void *priv,
+				   struct v4l2_fmtdesc *f)
+{
+	return enum_fmt(f, SUNXI_CEDRUS_CAPTURE);
+}
+
+static int vidioc_enum_fmt_vid_out(struct file *file, void *priv,
+				   struct v4l2_fmtdesc *f)
+{
+	return enum_fmt(f, SUNXI_CEDRUS_OUTPUT);
+}
+
+static int vidioc_g_fmt(struct sunxi_cedrus_ctx *ctx, struct v4l2_format *f)
+{
+	switch (f->type) {
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		f->fmt.pix_mp = ctx->dst_fmt;
+		break;
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		f->fmt.pix_mp = ctx->src_fmt;
+		break;
+	default:
+		dev_dbg(ctx->dev->dev, "invalid buf type\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int vidioc_g_fmt_vid_out(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	return vidioc_g_fmt(file2ctx(file), f);
+}
+
+static int vidioc_g_fmt_vid_cap(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	return vidioc_g_fmt(file2ctx(file), f);
+}
+
+static int vidioc_try_fmt(struct v4l2_format *f, struct sunxi_cedrus_fmt *fmt)
+{
+	int i;
+	__u32 bpl;
+
+	f->fmt.pix_mp.field = V4L2_FIELD_NONE;
+	f->fmt.pix_mp.num_planes = fmt->num_planes;
+
+	switch (f->type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		if (f->fmt.pix_mp.plane_fmt[0].sizeimage == 0)
+			return -EINVAL;
+
+		f->fmt.pix_mp.plane_fmt[0].bytesperline = 0;
+		break;
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		/* Limit to hardware min/max. */
+		f->fmt.pix_mp.width = clamp(f->fmt.pix_mp.width,
+			SUNXI_CEDRUS_MIN_WIDTH, SUNXI_CEDRUS_MAX_WIDTH);
+		f->fmt.pix_mp.height = clamp(f->fmt.pix_mp.height,
+			SUNXI_CEDRUS_MIN_HEIGHT, SUNXI_CEDRUS_MAX_HEIGHT);
+
+		for (i = 0; i < f->fmt.pix_mp.num_planes; ++i) {
+			bpl = (f->fmt.pix_mp.width * fmt->depth) >> 3;
+			f->fmt.pix_mp.plane_fmt[i].bytesperline = bpl;
+			f->fmt.pix_mp.plane_fmt[i].sizeimage =
+				f->fmt.pix_mp.height * bpl;
+		}
+		break;
+	}
+	return 0;
+}
+
+static int vidioc_try_fmt_vid_cap(struct file *file, void *priv,
+				  struct v4l2_format *f)
+{
+	struct sunxi_cedrus_fmt *fmt;
+	struct sunxi_cedrus_ctx *ctx = file2ctx(file);
+
+	fmt = find_format(f);
+	if (!fmt) {
+		f->fmt.pix_mp.pixelformat = formats[0].fourcc;
+		fmt = find_format(f);
+	}
+	if (!(fmt->types & SUNXI_CEDRUS_CAPTURE)) {
+		v4l2_err(&ctx->dev->v4l2_dev,
+			 "Fourcc format (0x%08x) invalid.\n",
+			 f->fmt.pix_mp.pixelformat);
+		return -EINVAL;
+	}
+
+	return vidioc_try_fmt(f, fmt);
+}
+
+static int vidioc_try_fmt_vid_out(struct file *file, void *priv,
+				  struct v4l2_format *f)
+{
+	struct sunxi_cedrus_fmt *fmt;
+	struct sunxi_cedrus_ctx *ctx = file2ctx(file);
+
+	fmt = find_format(f);
+	if (!fmt) {
+		f->fmt.pix_mp.pixelformat = formats[0].fourcc;
+		fmt = find_format(f);
+	}
+	if (!(fmt->types & SUNXI_CEDRUS_OUTPUT)) {
+		v4l2_err(&ctx->dev->v4l2_dev,
+			 "Fourcc format (0x%08x) invalid.\n",
+			 f->fmt.pix_mp.pixelformat);
+		return -EINVAL;
+	}
+
+	return vidioc_try_fmt(f, fmt);
+}
+
+static int vidioc_s_fmt(struct sunxi_cedrus_ctx *ctx, struct v4l2_format *f)
+{
+	struct v4l2_pix_format_mplane *pix_fmt_mp = &f->fmt.pix_mp;
+	struct sunxi_cedrus_fmt *fmt;
+	int i, ret = 0;
+
+	switch (f->type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		ctx->vpu_src_fmt = find_format(f);
+		ctx->src_fmt = *pix_fmt_mp;
+		break;
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		fmt = find_format(f);
+		ctx->vpu_dst_fmt = fmt;
+
+		for (i = 0; i < fmt->num_planes; ++i) {
+			pix_fmt_mp->plane_fmt[i].bytesperline =
+				pix_fmt_mp->width * fmt->depth;
+			pix_fmt_mp->plane_fmt[i].sizeimage =
+				pix_fmt_mp->plane_fmt[i].bytesperline
+				* pix_fmt_mp->height;
+		}
+		ctx->dst_fmt = *pix_fmt_mp;
+		break;
+	default:
+		dev_dbg(ctx->dev->dev, "invalid buf type\n");
+		return -EINVAL;
+	}
+
+	return ret;
+}
+
+static int vidioc_s_fmt_vid_cap(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	int ret;
+
+	ret = vidioc_try_fmt_vid_cap(file, priv, f);
+	if (ret)
+		return ret;
+
+	return vidioc_s_fmt(file2ctx(file), f);
+}
+
+static int vidioc_s_fmt_vid_out(struct file *file, void *priv,
+				struct v4l2_format *f)
+{
+	int ret;
+
+	ret = vidioc_try_fmt_vid_out(file, priv, f);
+	if (ret)
+		return ret;
+
+	ret = vidioc_s_fmt(file2ctx(file), f);
+	return ret;
+}
+
+const struct v4l2_ioctl_ops sunxi_cedrus_ioctl_ops = {
+	.vidioc_querycap		= vidioc_querycap,
+
+	.vidioc_enum_fmt_vid_cap	= vidioc_enum_fmt_vid_cap,
+	.vidioc_g_fmt_vid_cap_mplane	= vidioc_g_fmt_vid_cap,
+	.vidioc_try_fmt_vid_cap_mplane	= vidioc_try_fmt_vid_cap,
+	.vidioc_s_fmt_vid_cap_mplane	= vidioc_s_fmt_vid_cap,
+
+	.vidioc_enum_fmt_vid_out_mplane = vidioc_enum_fmt_vid_out,
+	.vidioc_g_fmt_vid_out_mplane	= vidioc_g_fmt_vid_out,
+	.vidioc_try_fmt_vid_out_mplane	= vidioc_try_fmt_vid_out,
+	.vidioc_s_fmt_vid_out_mplane	= vidioc_s_fmt_vid_out,
+
+	.vidioc_reqbufs			= v4l2_m2m_ioctl_reqbufs,
+	.vidioc_querybuf		= v4l2_m2m_ioctl_querybuf,
+	.vidioc_qbuf			= v4l2_m2m_ioctl_qbuf,
+	.vidioc_dqbuf			= v4l2_m2m_ioctl_dqbuf,
+	.vidioc_prepare_buf		= v4l2_m2m_ioctl_prepare_buf,
+	.vidioc_create_bufs		= v4l2_m2m_ioctl_create_bufs,
+	.vidioc_expbuf			= v4l2_m2m_ioctl_expbuf,
+
+	.vidioc_streamon		= v4l2_m2m_ioctl_streamon,
+	.vidioc_streamoff		= v4l2_m2m_ioctl_streamoff,
+
+	.vidioc_subscribe_event		= v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event	= v4l2_event_unsubscribe,
+};
+
+static int sunxi_cedrus_queue_setup(struct vb2_queue *vq, unsigned int *nbufs,
+				    unsigned int *nplanes, unsigned int sizes[],
+				    struct device *alloc_devs[])
+{
+	struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(vq);
+
+	if (*nbufs < 1)
+		*nbufs = 1;
+
+	if (*nbufs > VIDEO_MAX_FRAME)
+		*nbufs = VIDEO_MAX_FRAME;
+
+	switch (vq->type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		*nplanes = ctx->vpu_src_fmt->num_planes;
+
+		sizes[0] = ctx->src_fmt.plane_fmt[0].sizeimage;
+		break;
+
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		*nplanes = ctx->vpu_dst_fmt->num_planes;
+
+		sizes[0] = round_up(ctx->dst_fmt.plane_fmt[0].sizeimage, 8);
+		sizes[1] = sizes[0];
+		break;
+
+	default:
+		dev_dbg(ctx->dev->dev, "invalid queue type: %d\n", vq->type);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int sunxi_cedrus_buf_init(struct vb2_buffer *vb)
+{
+	struct vb2_queue *vq = vb->vb2_queue;
+	struct sunxi_cedrus_ctx *ctx = container_of(vq->drv_priv,
+			struct sunxi_cedrus_ctx, fh);
+
+	if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+		ctx->dst_bufs[vb->index] = vb;
+
+	return 0;
+}
+
+static void sunxi_cedrus_buf_cleanup(struct vb2_buffer *vb)
+{
+	struct vb2_queue *vq = vb->vb2_queue;
+	struct sunxi_cedrus_ctx *ctx = container_of(vq->drv_priv,
+			struct sunxi_cedrus_ctx, fh);
+
+	if (vq->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)
+		ctx->dst_bufs[vb->index] = NULL;
+}
+
+static int sunxi_cedrus_buf_prepare(struct vb2_buffer *vb)
+{
+	struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+	struct vb2_queue *vq = vb->vb2_queue;
+	int i;
+
+	dev_dbg(ctx->dev->dev, "type: %d\n", vb->vb2_queue->type);
+
+	switch (vq->type) {
+	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
+		if (vb2_plane_size(vb, 0)
+		    < ctx->src_fmt.plane_fmt[0].sizeimage) {
+			dev_dbg(ctx->dev->dev, "plane too small for output\n");
+			return -EINVAL;
+		}
+		break;
+
+	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
+		for (i = 0; i < ctx->vpu_dst_fmt->num_planes; ++i) {
+			if (vb2_plane_size(vb, i)
+			    < ctx->dst_fmt.plane_fmt[i].sizeimage) {
+				dev_dbg(ctx->dev->dev,
+					"plane %d too small for capture\n", i);
+				break;
+			}
+		}
+
+		if (i != ctx->vpu_dst_fmt->num_planes)
+			return -EINVAL;
+		break;
+
+	default:
+		dev_dbg(ctx->dev->dev, "invalid queue type: %d\n", vq->type);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static void sunxi_cedrus_stop_streaming(struct vb2_queue *q)
+{
+	struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(q);
+	struct vb2_v4l2_buffer *vbuf;
+	unsigned long flags;
+
+	flush_scheduled_work();
+	for (;;) {
+		spin_lock_irqsave(&ctx->dev->irq_lock, flags);
+
+		if (V4L2_TYPE_IS_OUTPUT(q->type))
+			vbuf = v4l2_m2m_src_buf_remove(ctx->fh.m2m_ctx);
+		else
+			vbuf = v4l2_m2m_dst_buf_remove(ctx->fh.m2m_ctx);
+
+		spin_unlock_irqrestore(&ctx->dev->irq_lock, flags);
+
+		if (vbuf == NULL)
+			return;
+
+		v4l2_ctrl_request_complete(vbuf->vb2_buf.req_obj.req,
+					   &ctx->hdl);
+		v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR);
+	}
+}
+
+static void sunxi_cedrus_buf_queue(struct vb2_buffer *vb)
+{
+	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
+	struct sunxi_cedrus_ctx *ctx = vb2_get_drv_priv(vb->vb2_queue);
+
+	v4l2_m2m_buf_queue(ctx->fh.m2m_ctx, vbuf);
+}
+
+static struct vb2_ops sunxi_cedrus_qops = {
+	.queue_setup		= sunxi_cedrus_queue_setup,
+	.buf_prepare		= sunxi_cedrus_buf_prepare,
+	.buf_init		= sunxi_cedrus_buf_init,
+	.buf_cleanup		= sunxi_cedrus_buf_cleanup,
+	.buf_queue		= sunxi_cedrus_buf_queue,
+	.stop_streaming		= sunxi_cedrus_stop_streaming,
+	.wait_prepare		= vb2_ops_wait_prepare,
+	.wait_finish		= vb2_ops_wait_finish,
+};
+
+int sunxi_cedrus_queue_init(void *priv, struct vb2_queue *src_vq,
+			    struct vb2_queue *dst_vq)
+{
+	struct sunxi_cedrus_ctx *ctx = priv;
+	int ret;
+
+	src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
+	src_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+	src_vq->drv_priv = ctx;
+	src_vq->buf_struct_size = sizeof(struct sunxi_cedrus_buffer);
+	src_vq->allow_zero_bytesused = 1;
+	src_vq->min_buffers_needed = 1;
+	src_vq->ops = &sunxi_cedrus_qops;
+	src_vq->mem_ops = &vb2_dma_contig_memops;
+	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	src_vq->lock = &ctx->dev->dev_mutex;
+	src_vq->dev = ctx->dev->dev;
+
+	ret = vb2_queue_init(src_vq);
+	if (ret)
+		return ret;
+
+	dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
+	dst_vq->io_modes = VB2_MMAP | VB2_DMABUF;
+	dst_vq->drv_priv = ctx;
+	dst_vq->buf_struct_size = sizeof(struct sunxi_cedrus_buffer);
+	dst_vq->allow_zero_bytesused = 1;
+	dst_vq->min_buffers_needed = 1;
+	dst_vq->ops = &sunxi_cedrus_qops;
+	dst_vq->mem_ops = &vb2_dma_contig_memops;
+	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+	dst_vq->lock = &ctx->dev->dev_mutex;
+	dst_vq->dev = ctx->dev->dev;
+
+	return vb2_queue_init(dst_vq);
+}
diff --git a/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.h b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.h
new file mode 100644
index 000000000000..d5b7f881c386
--- /dev/null
+++ b/drivers/media/platform/sunxi/cedrus/sunxi_cedrus_video.h
@@ -0,0 +1,31 @@
+/*
+ * Sunxi-Cedrus VPU driver
+ *
+ * Copyright (C) 2018 Paul Kocialkowski <paul.kocialkowski@bootlin.com>
+ * Copyright (C) 2016 Florent Revest <florent.revest@free-electrons.com>
+ *
+ * Based on the vim2m driver, that is:
+ *
+ * Copyright (c) 2009-2010 Samsung Electronics Co., Ltd.
+ * Pawel Osciak, <pawel@osciak.com>
+ * Marek Szyprowski, <m.szyprowski@samsung.com>
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _SUNXI_CEDRUS_VIDEO_H_
+#define _SUNXI_CEDRUS_VIDEO_H_
+
+extern const struct v4l2_ioctl_ops sunxi_cedrus_ioctl_ops;
+
+int sunxi_cedrus_queue_init(void *priv, struct vb2_queue *src_vq,
+			    struct vb2_queue *dst_vq);
+
+#endif
-- 
2.16.3

[PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Paul Kocialkowski]

This adds a device-tree binding document that specifies the properties
used by the Sunxi-Cedurs VPU driver, as well as examples.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
---
 .../devicetree/bindings/media/sunxi-cedrus.txt     | 50 ++++++++++++++++++++++
 1 file changed, 50 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/sunxi-cedrus.txt

diff --git a/Documentation/devicetree/bindings/media/sunxi-cedrus.txt b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
new file mode 100644
index 000000000000..71ad3f9c3352
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
@@ -0,0 +1,50 @@
+Device-tree bindings for the VPU found in Allwinner SoCs, referred to as the
+Video Engine (VE) in Allwinner literature.
+
+The VPU can only access the first 256 MiB of DRAM, that are DMA-mapped starting
+from the DRAM base. This requires specific memory allocation and handling.
+
+Required properties:
+- compatible	        : "allwinner,sun4i-a10-video-engine";
+- memory-region         : DMA pool for buffers allocation;
+- clocks	        : list of clock specifiers, corresponding to entries in
+                          the clock-names property;
+- clock-names	        : should contain "ahb", "mod" and "ram" entries;
+- assigned-clocks       : list of clocks assigned to the VE;
+- assigned-clocks-rates : list of clock rates for the clocks assigned to the VE;
+- resets	        : phandle for reset;
+- interrupts	        : should contain VE interrupt number;
+- reg		        : should contain register base and length of VE.
+
+Example:
+
+reserved-memory {
+	#address-cells = <1>;
+	#size-cells = <1>;
+	ranges;
+
+	/* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+	ve_memory: cma@4a000000 {
+		compatible = "shared-dma-pool";
+		reg = <0x4a000000 0x6000000>;
+		no-map;
+		linux,cma-default;
+	};
+};
+
+video-engine@1c0e000 {
+	compatible = "allwinner,sun4i-a10-video-engine";
+	reg = <0x01c0e000 0x1000>;
+	memory-region = <&ve_memory>;
+
+	clocks = <&ccu CLK_AHB_VE>, <&ccu CLK_VE>,
+		 <&ccu CLK_DRAM_VE>;
+	clock-names = "ahb", "mod", "ram";
+
+	assigned-clocks = <&ccu CLK_VE>;
+	assigned-clock-rates = <320000000>;
+
+	resets = <&ccu RST_VE>;
+
+	interrupts = <GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>;
+};
-- 
2.16.3

[PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Paul Kocialkowski]

This adds a device-tree binding document that specifies the properties
used by the Sunxi-Cedurs VPU driver, as well as examples.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
---
 .../devicetree/bindings/media/sunxi-cedrus.txt     | 50 ++++++++++++++++++++++
 1 file changed, 50 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/sunxi-cedrus.txt

diff --git a/Documentation/devicetree/bindings/media/sunxi-cedrus.txt b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
new file mode 100644
index 000000000000..71ad3f9c3352
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
@@ -0,0 +1,50 @@
+Device-tree bindings for the VPU found in Allwinner SoCs, referred to as the
+Video Engine (VE) in Allwinner literature.
+
+The VPU can only access the first 256 MiB of DRAM, that are DMA-mapped starting
+from the DRAM base. This requires specific memory allocation and handling.
+
+Required properties:
+- compatible	        : "allwinner,sun4i-a10-video-engine";
+- memory-region         : DMA pool for buffers allocation;
+- clocks	        : list of clock specifiers, corresponding to entries in
+                          the clock-names property;
+- clock-names	        : should contain "ahb", "mod" and "ram" entries;
+- assigned-clocks       : list of clocks assigned to the VE;
+- assigned-clocks-rates : list of clock rates for the clocks assigned to the VE;
+- resets	        : phandle for reset;
+- interrupts	        : should contain VE interrupt number;
+- reg		        : should contain register base and length of VE.
+
+Example:
+
+reserved-memory {
+	#address-cells = <1>;
+	#size-cells = <1>;
+	ranges;
+
+	/* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+	ve_memory: cma@4a000000 {
+		compatible = "shared-dma-pool";
+		reg = <0x4a000000 0x6000000>;
+		no-map;
+		linux,cma-default;
+	};
+};
+
+video-engine@1c0e000 {
+	compatible = "allwinner,sun4i-a10-video-engine";
+	reg = <0x01c0e000 0x1000>;
+	memory-region = <&ve_memory>;
+
+	clocks = <&ccu CLK_AHB_VE>, <&ccu CLK_VE>,
+		 <&ccu CLK_DRAM_VE>;
+	clock-names = "ahb", "mod", "ram";
+
+	assigned-clocks = <&ccu CLK_VE>;
+	assigned-clock-rates = <320000000>;
+
+	resets = <&ccu RST_VE>;
+
+	interrupts = <GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>;
+};
-- 
2.16.3

[PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Paul Kocialkowski]

This adds a device-tree binding document that specifies the properties
used by the Sunxi-Cedurs VPU driver, as well as examples.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
---
 .../devicetree/bindings/media/sunxi-cedrus.txt     | 50 ++++++++++++++++++++++
 1 file changed, 50 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/sunxi-cedrus.txt

diff --git a/Documentation/devicetree/bindings/media/sunxi-cedrus.txt b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
new file mode 100644
index 000000000000..71ad3f9c3352
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
@@ -0,0 +1,50 @@
+Device-tree bindings for the VPU found in Allwinner SoCs, referred to as the
+Video Engine (VE) in Allwinner literature.
+
+The VPU can only access the first 256 MiB of DRAM, that are DMA-mapped starting
+from the DRAM base. This requires specific memory allocation and handling.
+
+Required properties:
+- compatible	        : "allwinner,sun4i-a10-video-engine";
+- memory-region         : DMA pool for buffers allocation;
+- clocks	        : list of clock specifiers, corresponding to entries in
+                          the clock-names property;
+- clock-names	        : should contain "ahb", "mod" and "ram" entries;
+- assigned-clocks       : list of clocks assigned to the VE;
+- assigned-clocks-rates : list of clock rates for the clocks assigned to the VE;
+- resets	        : phandle for reset;
+- interrupts	        : should contain VE interrupt number;
+- reg		        : should contain register base and length of VE.
+
+Example:
+
+reserved-memory {
+	#address-cells = <1>;
+	#size-cells = <1>;
+	ranges;
+
+	/* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+	ve_memory: cma at 4a000000 {
+		compatible = "shared-dma-pool";
+		reg = <0x4a000000 0x6000000>;
+		no-map;
+		linux,cma-default;
+	};
+};
+
+video-engine at 1c0e000 {
+	compatible = "allwinner,sun4i-a10-video-engine";
+	reg = <0x01c0e000 0x1000>;
+	memory-region = <&ve_memory>;
+
+	clocks = <&ccu CLK_AHB_VE>, <&ccu CLK_VE>,
+		 <&ccu CLK_DRAM_VE>;
+	clock-names = "ahb", "mod", "ram";
+
+	assigned-clocks = <&ccu CLK_VE>;
+	assigned-clock-rates = <320000000>;
+
+	resets = <&ccu RST_VE>;
+
+	interrupts = <GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>;
+};
-- 
2.16.3

[PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

This adds nodes for the Video Engine and the associated reserved memory
for the Allwinner A20. Up to 96 MiB of memory are dedicated to the VPU.

The VPU can only map the first 256 MiB of DRAM, so the reserved memory
pool has to be located in that area. Following Allwinner's decision in
downstream software, the last 96 MiB of the first 256 MiB of RAM are
reserved for this purpose.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
---
 arch/arm/boot/dts/sun7i-a20.dtsi | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/arch/arm/boot/dts/sun7i-a20.dtsi b/arch/arm/boot/dts/sun7i-a20.dtsi
index bd0cd3204273..cb6d82065dcf 100644
--- a/arch/arm/boot/dts/sun7i-a20.dtsi
+++ b/arch/arm/boot/dts/sun7i-a20.dtsi
@@ -163,6 +163,20 @@
 		reg = <0x40000000 0x80000000>;
 	};
 
+	reserved-memory {
+		#address-cells = <1>;
+		#size-cells = <1>;
+		ranges;
+
+		/* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+		ve_memory: cma@4a000000 {
+			compatible = "shared-dma-pool";
+			reg = <0x4a000000 0x6000000>;
+			no-map;
+			linux,cma-default;
+		};
+	};
+
 	timer {
 		compatible = "arm,armv7-timer";
 		interrupts = <GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_LEVEL_LOW)>,
@@ -451,6 +465,23 @@
 			};
 		};
 
+		ve: video-engine@1c0e000 {
+			compatible = "allwinner,sun4i-a10-video-engine";
+			reg = <0x01c0e000 0x1000>;
+			memory-region = <&ve_memory>;
+
+			clocks = <&ccu CLK_AHB_VE>, <&ccu CLK_VE>,
+				 <&ccu CLK_DRAM_VE>;
+			clock-names = "ahb", "mod", "ram";
+
+			assigned-clocks = <&ccu CLK_VE>;
+			assigned-clock-rates = <320000000>;
+
+			resets = <&ccu RST_VE>;
+
+			interrupts = <GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>;
+		};
+
 		mmc0: mmc@1c0f000 {
 			compatible = "allwinner,sun7i-a20-mmc";
 			reg = <0x01c0f000 0x1000>;
-- 
2.16.3

[PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

This adds nodes for the Video Engine and the associated reserved memory
for the Allwinner A20. Up to 96 MiB of memory are dedicated to the VPU.

The VPU can only map the first 256 MiB of DRAM, so the reserved memory
pool has to be located in that area. Following Allwinner's decision in
downstream software, the last 96 MiB of the first 256 MiB of RAM are
reserved for this purpose.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
---
 arch/arm/boot/dts/sun7i-a20.dtsi | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/arch/arm/boot/dts/sun7i-a20.dtsi b/arch/arm/boot/dts/sun7i-a20.dtsi
index bd0cd3204273..cb6d82065dcf 100644
--- a/arch/arm/boot/dts/sun7i-a20.dtsi
+++ b/arch/arm/boot/dts/sun7i-a20.dtsi
@@ -163,6 +163,20 @@
 		reg = <0x40000000 0x80000000>;
 	};
 
+	reserved-memory {
+		#address-cells = <1>;
+		#size-cells = <1>;
+		ranges;
+
+		/* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+		ve_memory: cma@4a000000 {
+			compatible = "shared-dma-pool";
+			reg = <0x4a000000 0x6000000>;
+			no-map;
+			linux,cma-default;
+		};
+	};
+
 	timer {
 		compatible = "arm,armv7-timer";
 		interrupts = <GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_LEVEL_LOW)>,
@@ -451,6 +465,23 @@
 			};
 		};
 
+		ve: video-engine@1c0e000 {
+			compatible = "allwinner,sun4i-a10-video-engine";
+			reg = <0x01c0e000 0x1000>;
+			memory-region = <&ve_memory>;
+
+			clocks = <&ccu CLK_AHB_VE>, <&ccu CLK_VE>,
+				 <&ccu CLK_DRAM_VE>;
+			clock-names = "ahb", "mod", "ram";
+
+			assigned-clocks = <&ccu CLK_VE>;
+			assigned-clock-rates = <320000000>;
+
+			resets = <&ccu RST_VE>;
+
+			interrupts = <GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>;
+		};
+
 		mmc0: mmc@1c0f000 {
 			compatible = "allwinner,sun7i-a20-mmc";
 			reg = <0x01c0f000 0x1000>;
-- 
2.16.3

[PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

This adds nodes for the Video Engine and the associated reserved memory
for the Allwinner A20. Up to 96 MiB of memory are dedicated to the VPU.

The VPU can only map the first 256 MiB of DRAM, so the reserved memory
pool has to be located in that area. Following Allwinner's decision in
downstream software, the last 96 MiB of the first 256 MiB of RAM are
reserved for this purpose.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
---
 arch/arm/boot/dts/sun7i-a20.dtsi | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/arch/arm/boot/dts/sun7i-a20.dtsi b/arch/arm/boot/dts/sun7i-a20.dtsi
index bd0cd3204273..cb6d82065dcf 100644
--- a/arch/arm/boot/dts/sun7i-a20.dtsi
+++ b/arch/arm/boot/dts/sun7i-a20.dtsi
@@ -163,6 +163,20 @@
 		reg = <0x40000000 0x80000000>;
 	};
 
+	reserved-memory {
+		#address-cells = <1>;
+		#size-cells = <1>;
+		ranges;
+
+		/* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+		ve_memory: cma at 4a000000 {
+			compatible = "shared-dma-pool";
+			reg = <0x4a000000 0x6000000>;
+			no-map;
+			linux,cma-default;
+		};
+	};
+
 	timer {
 		compatible = "arm,armv7-timer";
 		interrupts = <GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_LEVEL_LOW)>,
@@ -451,6 +465,23 @@
 			};
 		};
 
+		ve: video-engine at 1c0e000 {
+			compatible = "allwinner,sun4i-a10-video-engine";
+			reg = <0x01c0e000 0x1000>;
+			memory-region = <&ve_memory>;
+
+			clocks = <&ccu CLK_AHB_VE>, <&ccu CLK_VE>,
+				 <&ccu CLK_DRAM_VE>;
+			clock-names = "ahb", "mod", "ram";
+
+			assigned-clocks = <&ccu CLK_VE>;
+			assigned-clock-rates = <320000000>;
+
+			resets = <&ccu RST_VE>;
+
+			interrupts = <GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>;
+		};
+
 		mmc0: mmc at 1c0f000 {
 			compatible = "allwinner,sun7i-a20-mmc";
 			reg = <0x01c0f000 0x1000>;
-- 
2.16.3

[PATCH v2 10/10] ARM: dts: sun8i-a33: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

This adds nodes for the Video Engine and the associated reserved memory
for the Allwinner A33. Up to 96 MiB of memory are dedicated to the VPU.

The VPU can only map the first 256 MiB of DRAM, so the reserved memory
pool has to be located in that area. Following Allwinner's decision in
downstream software, the last 96 MiB of the first 256 MiB of RAM are
reserved for this purpose.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
---
 arch/arm/boot/dts/sun8i-a33.dtsi | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/arch/arm/boot/dts/sun8i-a33.dtsi b/arch/arm/boot/dts/sun8i-a33.dtsi
index a21f2ed07a52..308b532aee1d 100644
--- a/arch/arm/boot/dts/sun8i-a33.dtsi
+++ b/arch/arm/boot/dts/sun8i-a33.dtsi
@@ -181,6 +181,20 @@
 		reg = <0x40000000 0x80000000>;
 	};
 
+	reserved-memory {
+		#address-cells = <1>;
+		#size-cells = <1>;
+		ranges;
+
+		/* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+		ve_memory: cma@4a000000 {
+			compatible = "shared-dma-pool";
+			reg = <0x4a000000 0x6000000>;
+			no-map;
+			linux,cma-default;
+		};
+	};
+
 	sound: sound {
 		compatible = "simple-audio-card";
 		simple-audio-card,name = "sun8i-a33-audio";
@@ -204,6 +218,11 @@
 	};
 
 	soc@1c00000 {
+		syscon: system-controller@01c00000 {
+			compatible = "allwinner,sun8i-a33-syscon", "syscon";
+			reg = <0x01c00000 0x1000>;
+		};
+
 		tcon0: lcd-controller@1c0c000 {
 			compatible = "allwinner,sun8i-a33-tcon";
 			reg = <0x01c0c000 0x1000>;
@@ -240,6 +259,25 @@
 			};
 		};
 
+		ve: video-engine@01c0e000 {
+			compatible = "allwinner,sun4i-a10-video-engine";
+			memory-region = <&ve_memory>;
+			syscon = <&syscon>;
+
+			clocks = <&ccu CLK_BUS_VE>, <&ccu CLK_VE>,
+				 <&ccu CLK_DRAM_VE>;
+			clock-names = "ahb", "mod", "ram";
+
+			assigned-clocks = <&ccu CLK_VE>;
+			assigned-clock-rates = <320000000>;
+
+			resets = <&ccu RST_BUS_VE>;
+
+			interrupts = <GIC_SPI 58 IRQ_TYPE_LEVEL_HIGH>;
+
+			reg = <0x01c0e000 0x1000>;
+		};
+
 		crypto: crypto-engine@1c15000 {
 			compatible = "allwinner,sun4i-a10-crypto";
 			reg = <0x01c15000 0x1000>;
-- 
2.16.3

[PATCH v2 10/10] ARM: dts: sun8i-a33: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

This adds nodes for the Video Engine and the associated reserved memory
for the Allwinner A33. Up to 96 MiB of memory are dedicated to the VPU.

The VPU can only map the first 256 MiB of DRAM, so the reserved memory
pool has to be located in that area. Following Allwinner's decision in
downstream software, the last 96 MiB of the first 256 MiB of RAM are
reserved for this purpose.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
---
 arch/arm/boot/dts/sun8i-a33.dtsi | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/arch/arm/boot/dts/sun8i-a33.dtsi b/arch/arm/boot/dts/sun8i-a33.dtsi
index a21f2ed07a52..308b532aee1d 100644
--- a/arch/arm/boot/dts/sun8i-a33.dtsi
+++ b/arch/arm/boot/dts/sun8i-a33.dtsi
@@ -181,6 +181,20 @@
 		reg = <0x40000000 0x80000000>;
 	};
 
+	reserved-memory {
+		#address-cells = <1>;
+		#size-cells = <1>;
+		ranges;
+
+		/* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+		ve_memory: cma@4a000000 {
+			compatible = "shared-dma-pool";
+			reg = <0x4a000000 0x6000000>;
+			no-map;
+			linux,cma-default;
+		};
+	};
+
 	sound: sound {
 		compatible = "simple-audio-card";
 		simple-audio-card,name = "sun8i-a33-audio";
@@ -204,6 +218,11 @@
 	};
 
 	soc@1c00000 {
+		syscon: system-controller@01c00000 {
+			compatible = "allwinner,sun8i-a33-syscon", "syscon";
+			reg = <0x01c00000 0x1000>;
+		};
+
 		tcon0: lcd-controller@1c0c000 {
 			compatible = "allwinner,sun8i-a33-tcon";
 			reg = <0x01c0c000 0x1000>;
@@ -240,6 +259,25 @@
 			};
 		};
 
+		ve: video-engine@01c0e000 {
+			compatible = "allwinner,sun4i-a10-video-engine";
+			memory-region = <&ve_memory>;
+			syscon = <&syscon>;
+
+			clocks = <&ccu CLK_BUS_VE>, <&ccu CLK_VE>,
+				 <&ccu CLK_DRAM_VE>;
+			clock-names = "ahb", "mod", "ram";
+
+			assigned-clocks = <&ccu CLK_VE>;
+			assigned-clock-rates = <320000000>;
+
+			resets = <&ccu RST_BUS_VE>;
+
+			interrupts = <GIC_SPI 58 IRQ_TYPE_LEVEL_HIGH>;
+
+			reg = <0x01c0e000 0x1000>;
+		};
+
 		crypto: crypto-engine@1c15000 {
 			compatible = "allwinner,sun4i-a10-crypto";
 			reg = <0x01c15000 0x1000>;
-- 
2.16.3

[PATCH v2 10/10] ARM: dts: sun8i-a33: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

This adds nodes for the Video Engine and the associated reserved memory
for the Allwinner A33. Up to 96 MiB of memory are dedicated to the VPU.

The VPU can only map the first 256 MiB of DRAM, so the reserved memory
pool has to be located in that area. Following Allwinner's decision in
downstream software, the last 96 MiB of the first 256 MiB of RAM are
reserved for this purpose.

Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
---
 arch/arm/boot/dts/sun8i-a33.dtsi | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/arch/arm/boot/dts/sun8i-a33.dtsi b/arch/arm/boot/dts/sun8i-a33.dtsi
index a21f2ed07a52..308b532aee1d 100644
--- a/arch/arm/boot/dts/sun8i-a33.dtsi
+++ b/arch/arm/boot/dts/sun8i-a33.dtsi
@@ -181,6 +181,20 @@
 		reg = <0x40000000 0x80000000>;
 	};
 
+	reserved-memory {
+		#address-cells = <1>;
+		#size-cells = <1>;
+		ranges;
+
+		/* Address must be kept in the lower 256 MiBs of DRAM for VE. */
+		ve_memory: cma at 4a000000 {
+			compatible = "shared-dma-pool";
+			reg = <0x4a000000 0x6000000>;
+			no-map;
+			linux,cma-default;
+		};
+	};
+
 	sound: sound {
 		compatible = "simple-audio-card";
 		simple-audio-card,name = "sun8i-a33-audio";
@@ -204,6 +218,11 @@
 	};
 
 	soc at 1c00000 {
+		syscon: system-controller at 01c00000 {
+			compatible = "allwinner,sun8i-a33-syscon", "syscon";
+			reg = <0x01c00000 0x1000>;
+		};
+
 		tcon0: lcd-controller at 1c0c000 {
 			compatible = "allwinner,sun8i-a33-tcon";
 			reg = <0x01c0c000 0x1000>;
@@ -240,6 +259,25 @@
 			};
 		};
 
+		ve: video-engine at 01c0e000 {
+			compatible = "allwinner,sun4i-a10-video-engine";
+			memory-region = <&ve_memory>;
+			syscon = <&syscon>;
+
+			clocks = <&ccu CLK_BUS_VE>, <&ccu CLK_VE>,
+				 <&ccu CLK_DRAM_VE>;
+			clock-names = "ahb", "mod", "ram";
+
+			assigned-clocks = <&ccu CLK_VE>;
+			assigned-clock-rates = <320000000>;
+
+			resets = <&ccu RST_BUS_VE>;
+
+			interrupts = <GIC_SPI 58 IRQ_TYPE_LEVEL_HIGH>;
+
+			reg = <0x01c0e000 0x1000>;
+		};
+
 		crypto: crypto-engine at 1c15000 {
 			compatible = "allwinner,sun4i-a10-crypto";
 			reg = <0x01c15000 0x1000>;
-- 
2.16.3

Re: [PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Philipp Zabel]

Hi Paul,

On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> This adds a device-tree binding document that specifies the properties
> used by the Sunxi-Cedurs VPU driver, as well as examples.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  .../devicetree/bindings/media/sunxi-cedrus.txt     | 50 ++++++++++++++++++++++
>  1 file changed, 50 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> 
> diff --git a/Documentation/devicetree/bindings/media/sunxi-cedrus.txt b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> new file mode 100644
> index 000000000000..71ad3f9c3352
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> @@ -0,0 +1,50 @@
> +Device-tree bindings for the VPU found in Allwinner SoCs, referred to as the
> +Video Engine (VE) in Allwinner literature.
> +
> +The VPU can only access the first 256 MiB of DRAM, that are DMA-mapped starting
> +from the DRAM base. This requires specific memory allocation and handling.
> +
> +Required properties:
> +- compatible	        : "allwinner,sun4i-a10-video-engine";
> +- memory-region         : DMA pool for buffers allocation;
> +- clocks	        : list of clock specifiers, corresponding to entries in
> +                          the clock-names property;
> +- clock-names	        : should contain "ahb", "mod" and "ram" entries;
> +- assigned-clocks       : list of clocks assigned to the VE;
> +- assigned-clocks-rates : list of clock rates for the clocks assigned to the VE;
> +- resets	        : phandle for reset;
> +- interrupts	        : should contain VE interrupt number;
> +- reg		        : should contain register base and length of VE.
> +
> +Example:
> +
> +reserved-memory {
> +	#address-cells = <1>;
> +	#size-cells = <1>;
> +	ranges;
> +
> +	/* Address must be kept in the lower 256 MiBs of DRAM for VE. */
> +	ve_memory: cma@4a000000 {
> +		compatible = "shared-dma-pool";
> +		reg = <0x4a000000 0x6000000>;
> +		no-map;
> +		linux,cma-default;
> +	};
> +};
> +
> +video-engine@1c0e000 {

This is not really required by any specification, and not as common as
gpu@..., but could this reasonably be called "vpu@1c0e000" to follow
somewhat-common practice?

> +	compatible = "allwinner,sun4i-a10-video-engine";
> +	reg = <0x01c0e000 0x1000>;
> +	memory-region = <&ve_memory>;
> +
> +	clocks = <&ccu CLK_AHB_VE>, <&ccu CLK_VE>,
> +		 <&ccu CLK_DRAM_VE>;
> +	clock-names = "ahb", "mod", "ram";
> +
> +	assigned-clocks = <&ccu CLK_VE>;
> +	assigned-clock-rates = <320000000>;
> +
> +	resets = <&ccu RST_VE>;
> +
> +	interrupts = <GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>;
> +};

regards
Philipp

Re: [PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Philipp Zabel]

Hi Paul,

On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> This adds a device-tree binding document that specifies the properties
> used by the Sunxi-Cedurs VPU driver, as well as examples.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> ---
>  .../devicetree/bindings/media/sunxi-cedrus.txt     | 50 ++++++++++++++++++++++
>  1 file changed, 50 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> 
> diff --git a/Documentation/devicetree/bindings/media/sunxi-cedrus.txt b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> new file mode 100644
> index 000000000000..71ad3f9c3352
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> @@ -0,0 +1,50 @@
> +Device-tree bindings for the VPU found in Allwinner SoCs, referred to as the
> +Video Engine (VE) in Allwinner literature.
> +
> +The VPU can only access the first 256 MiB of DRAM, that are DMA-mapped starting
> +from the DRAM base. This requires specific memory allocation and handling.
> +
> +Required properties:
> +- compatible	        : "allwinner,sun4i-a10-video-engine";
> +- memory-region         : DMA pool for buffers allocation;
> +- clocks	        : list of clock specifiers, corresponding to entries in
> +                          the clock-names property;
> +- clock-names	        : should contain "ahb", "mod" and "ram" entries;
> +- assigned-clocks       : list of clocks assigned to the VE;
> +- assigned-clocks-rates : list of clock rates for the clocks assigned to the VE;
> +- resets	        : phandle for reset;
> +- interrupts	        : should contain VE interrupt number;
> +- reg		        : should contain register base and length of VE.
> +
> +Example:
> +
> +reserved-memory {
> +	#address-cells = <1>;
> +	#size-cells = <1>;
> +	ranges;
> +
> +	/* Address must be kept in the lower 256 MiBs of DRAM for VE. */
> +	ve_memory: cma@4a000000 {
> +		compatible = "shared-dma-pool";
> +		reg = <0x4a000000 0x6000000>;
> +		no-map;
> +		linux,cma-default;
> +	};
> +};
> +
> +video-engine@1c0e000 {

This is not really required by any specification, and not as common as
gpu@..., but could this reasonably be called "vpu@1c0e000" to follow
somewhat-common practice?

> +	compatible = "allwinner,sun4i-a10-video-engine";
> +	reg = <0x01c0e000 0x1000>;
> +	memory-region = <&ve_memory>;
> +
> +	clocks = <&ccu CLK_AHB_VE>, <&ccu CLK_VE>,
> +		 <&ccu CLK_DRAM_VE>;
> +	clock-names = "ahb", "mod", "ram";
> +
> +	assigned-clocks = <&ccu CLK_VE>;
> +	assigned-clock-rates = <320000000>;
> +
> +	resets = <&ccu RST_VE>;
> +
> +	interrupts = <GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>;
> +};

regards
Philipp

[PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Philipp Zabel]

Hi Paul,

On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> This adds a device-tree binding document that specifies the properties
> used by the Sunxi-Cedurs VPU driver, as well as examples.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  .../devicetree/bindings/media/sunxi-cedrus.txt     | 50 ++++++++++++++++++++++
>  1 file changed, 50 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> 
> diff --git a/Documentation/devicetree/bindings/media/sunxi-cedrus.txt b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> new file mode 100644
> index 000000000000..71ad3f9c3352
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> @@ -0,0 +1,50 @@
> +Device-tree bindings for the VPU found in Allwinner SoCs, referred to as the
> +Video Engine (VE) in Allwinner literature.
> +
> +The VPU can only access the first 256 MiB of DRAM, that are DMA-mapped starting
> +from the DRAM base. This requires specific memory allocation and handling.
> +
> +Required properties:
> +- compatible	        : "allwinner,sun4i-a10-video-engine";
> +- memory-region         : DMA pool for buffers allocation;
> +- clocks	        : list of clock specifiers, corresponding to entries in
> +                          the clock-names property;
> +- clock-names	        : should contain "ahb", "mod" and "ram" entries;
> +- assigned-clocks       : list of clocks assigned to the VE;
> +- assigned-clocks-rates : list of clock rates for the clocks assigned to the VE;
> +- resets	        : phandle for reset;
> +- interrupts	        : should contain VE interrupt number;
> +- reg		        : should contain register base and length of VE.
> +
> +Example:
> +
> +reserved-memory {
> +	#address-cells = <1>;
> +	#size-cells = <1>;
> +	ranges;
> +
> +	/* Address must be kept in the lower 256 MiBs of DRAM for VE. */
> +	ve_memory: cma at 4a000000 {
> +		compatible = "shared-dma-pool";
> +		reg = <0x4a000000 0x6000000>;
> +		no-map;
> +		linux,cma-default;
> +	};
> +};
> +
> +video-engine at 1c0e000 {

This is not really required by any specification, and not as common as
gpu at ..., but could this reasonably be called "vpu at 1c0e000" to follow
somewhat-common practice?

> +	compatible = "allwinner,sun4i-a10-video-engine";
> +	reg = <0x01c0e000 0x1000>;
> +	memory-region = <&ve_memory>;
> +
> +	clocks = <&ccu CLK_AHB_VE>, <&ccu CLK_VE>,
> +		 <&ccu CLK_DRAM_VE>;
> +	clock-names = "ahb", "mod", "ram";
> +
> +	assigned-clocks = <&ccu CLK_VE>;
> +	assigned-clock-rates = <320000000>;
> +
> +	resets = <&ccu RST_VE>;
> +
> +	interrupts = <GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>;
> +};

regards
Philipp

Re: [PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Tomasz Figa]

Hi Paul, Philipp,

On Fri, Apr 20, 2018 at 1:04 AM Philipp Zabel <p.zabel@pengutronix.de>
wrote:

> Hi Paul,

> On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> > This adds a device-tree binding document that specifies the properties
> > used by the Sunxi-Cedurs VPU driver, as well as examples.
> >
> > Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> > ---
> >  .../devicetree/bindings/media/sunxi-cedrus.txt     | 50
++++++++++++++++++++++
> >  1 file changed, 50 insertions(+)
> >  create mode 100644
Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> >
> > diff --git a/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > new file mode 100644
> > index 000000000000..71ad3f9c3352
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > @@ -0,0 +1,50 @@
> > +Device-tree bindings for the VPU found in Allwinner SoCs, referred to
as the
> > +Video Engine (VE) in Allwinner literature.
> > +
> > +The VPU can only access the first 256 MiB of DRAM, that are DMA-mapped
starting
> > +from the DRAM base. This requires specific memory allocation and
handling.

And no IOMMU? Brings back memories.

> > +
> > +Required properties:
> > +- compatible         : "allwinner,sun4i-a10-video-engine";
> > +- memory-region         : DMA pool for buffers allocation;
> > +- clocks             : list of clock specifiers, corresponding to
entries in
> > +                          the clock-names property;
> > +- clock-names                : should contain "ahb", "mod" and "ram"
entries;
> > +- assigned-clocks       : list of clocks assigned to the VE;
> > +- assigned-clocks-rates : list of clock rates for the clocks assigned
to the VE;
> > +- resets             : phandle for reset;
> > +- interrupts         : should contain VE interrupt number;
> > +- reg                        : should contain register base and length
of VE.
> > +
> > +Example:
> > +
> > +reserved-memory {
> > +     #address-cells = <1>;
> > +     #size-cells = <1>;
> > +     ranges;
> > +
> > +     /* Address must be kept in the lower 256 MiBs of DRAM for VE. */
> > +     ve_memory: cma@4a000000 {
> > +             compatible = "shared-dma-pool";
> > +             reg = <0x4a000000 0x6000000>;
> > +             no-map;
> > +             linux,cma-default;
> > +     };
> > +};
> > +
> > +video-engine@1c0e000 {

> This is not really required by any specification, and not as common as
> gpu@..., but could this reasonably be called "vpu@1c0e000" to follow
> somewhat-common practice?

AFAIR the name is supposed to be somewhat readable for someone that doesn't
know the hardware. To me, "video-engine" sounds more obvious than "vpu",
but we actually use "codec" already, in case of MFC and JPEG codec on
Exynos. If encode/decode is the only functionality of this block, I'd
personally go with "codec". If it can do other things, e.g.
scaling/rotation without encode/decode, I'd probably call it
"video-processor".

Best regards,
Tomasz

Re: [PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Tomasz Figa]

Hi Paul, Philipp,

On Fri, Apr 20, 2018 at 1:04 AM Philipp Zabel <p.zabel@pengutronix.de>
wrote:

> Hi Paul,

> On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> > This adds a device-tree binding document that specifies the properties
> > used by the Sunxi-Cedurs VPU driver, as well as examples.
> >
> > Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> > ---
> >  .../devicetree/bindings/media/sunxi-cedrus.txt     | 50
++++++++++++++++++++++
> >  1 file changed, 50 insertions(+)
> >  create mode 100644
Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> >
> > diff --git a/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > new file mode 100644
> > index 000000000000..71ad3f9c3352
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > @@ -0,0 +1,50 @@
> > +Device-tree bindings for the VPU found in Allwinner SoCs, referred to
as the
> > +Video Engine (VE) in Allwinner literature.
> > +
> > +The VPU can only access the first 256 MiB of DRAM, that are DMA-mapped
starting
> > +from the DRAM base. This requires specific memory allocation and
handling.

And no IOMMU? Brings back memories.

> > +
> > +Required properties:
> > +- compatible         : "allwinner,sun4i-a10-video-engine";
> > +- memory-region         : DMA pool for buffers allocation;
> > +- clocks             : list of clock specifiers, corresponding to
entries in
> > +                          the clock-names property;
> > +- clock-names                : should contain "ahb", "mod" and "ram"
entries;
> > +- assigned-clocks       : list of clocks assigned to the VE;
> > +- assigned-clocks-rates : list of clock rates for the clocks assigned
to the VE;
> > +- resets             : phandle for reset;
> > +- interrupts         : should contain VE interrupt number;
> > +- reg                        : should contain register base and length
of VE.
> > +
> > +Example:
> > +
> > +reserved-memory {
> > +     #address-cells = <1>;
> > +     #size-cells = <1>;
> > +     ranges;
> > +
> > +     /* Address must be kept in the lower 256 MiBs of DRAM for VE. */
> > +     ve_memory: cma@4a000000 {
> > +             compatible = "shared-dma-pool";
> > +             reg = <0x4a000000 0x6000000>;
> > +             no-map;
> > +             linux,cma-default;
> > +     };
> > +};
> > +
> > +video-engine@1c0e000 {

> This is not really required by any specification, and not as common as
> gpu@..., but could this reasonably be called "vpu@1c0e000" to follow
> somewhat-common practice?

AFAIR the name is supposed to be somewhat readable for someone that doesn't
know the hardware. To me, "video-engine" sounds more obvious than "vpu",
but we actually use "codec" already, in case of MFC and JPEG codec on
Exynos. If encode/decode is the only functionality of this block, I'd
personally go with "codec". If it can do other things, e.g.
scaling/rotation without encode/decode, I'd probably call it
"video-processor".

Best regards,
Tomasz

[PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Tomasz Figa]

Hi Paul, Philipp,

On Fri, Apr 20, 2018 at 1:04 AM Philipp Zabel <p.zabel@pengutronix.de>
wrote:

> Hi Paul,

> On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> > This adds a device-tree binding document that specifies the properties
> > used by the Sunxi-Cedurs VPU driver, as well as examples.
> >
> > Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> > ---
> >  .../devicetree/bindings/media/sunxi-cedrus.txt     | 50
++++++++++++++++++++++
> >  1 file changed, 50 insertions(+)
> >  create mode 100644
Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> >
> > diff --git a/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > new file mode 100644
> > index 000000000000..71ad3f9c3352
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > @@ -0,0 +1,50 @@
> > +Device-tree bindings for the VPU found in Allwinner SoCs, referred to
as the
> > +Video Engine (VE) in Allwinner literature.
> > +
> > +The VPU can only access the first 256 MiB of DRAM, that are DMA-mapped
starting
> > +from the DRAM base. This requires specific memory allocation and
handling.

And no IOMMU? Brings back memories.

> > +
> > +Required properties:
> > +- compatible         : "allwinner,sun4i-a10-video-engine";
> > +- memory-region         : DMA pool for buffers allocation;
> > +- clocks             : list of clock specifiers, corresponding to
entries in
> > +                          the clock-names property;
> > +- clock-names                : should contain "ahb", "mod" and "ram"
entries;
> > +- assigned-clocks       : list of clocks assigned to the VE;
> > +- assigned-clocks-rates : list of clock rates for the clocks assigned
to the VE;
> > +- resets             : phandle for reset;
> > +- interrupts         : should contain VE interrupt number;
> > +- reg                        : should contain register base and length
of VE.
> > +
> > +Example:
> > +
> > +reserved-memory {
> > +     #address-cells = <1>;
> > +     #size-cells = <1>;
> > +     ranges;
> > +
> > +     /* Address must be kept in the lower 256 MiBs of DRAM for VE. */
> > +     ve_memory: cma at 4a000000 {
> > +             compatible = "shared-dma-pool";
> > +             reg = <0x4a000000 0x6000000>;
> > +             no-map;
> > +             linux,cma-default;
> > +     };
> > +};
> > +
> > +video-engine at 1c0e000 {

> This is not really required by any specification, and not as common as
> gpu at ..., but could this reasonably be called "vpu at 1c0e000" to follow
> somewhat-common practice?

AFAIR the name is supposed to be somewhat readable for someone that doesn't
know the hardware. To me, "video-engine" sounds more obvious than "vpu",
but we actually use "codec" already, in case of MFC and JPEG codec on
Exynos. If encode/decode is the only functionality of this block, I'd
personally go with "codec". If it can do other things, e.g.
scaling/rotation without encode/decode, I'd probably call it
"video-processor".

Best regards,
Tomasz

Re: [PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 4174 bytes --]

Hi and thanks for the review,

On Fri, 2018-04-20 at 01:31 +0000, Tomasz Figa wrote:
> Hi Paul, Philipp,
> 
> On Fri, Apr 20, 2018 at 1:04 AM Philipp Zabel <p.zabel@pengutronix.de>
> wrote:
> 
> > Hi Paul,
> > On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> > > This adds a device-tree binding document that specifies the
> > > properties
> > > used by the Sunxi-Cedurs VPU driver, as well as examples.
> > > 
> > > Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> > > ---
> > >  .../devicetree/bindings/media/sunxi-cedrus.txt     | 50
> 
> ++++++++++++++++++++++
> > >  1 file changed, 50 insertions(+)
> > >  create mode 100644
> 
> Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > 
> > > diff --git a/Documentation/devicetree/bindings/media/sunxi-
> > > cedrus.txt
> 
> b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > new file mode 100644
> > > index 000000000000..71ad3f9c3352
> > > --- /dev/null
> > > +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > @@ -0,0 +1,50 @@
> > > +Device-tree bindings for the VPU found in Allwinner SoCs,
> > > referred to
> 
> as the
> > > +Video Engine (VE) in Allwinner literature.
> > > +
> > > +The VPU can only access the first 256 MiB of DRAM, that are DMA-
> > > mapped
> 
> starting
> > > +from the DRAM base. This requires specific memory allocation and
> 
> handling.
> 
> And no IOMMU? Brings back memories.

Exactly, no IOMMU so we don't have much choice but cope with that
hardware limitation...

> > > +
> > > +Required properties:
> > > +- compatible         : "allwinner,sun4i-a10-video-engine";
> > > +- memory-region         : DMA pool for buffers allocation;
> > > +- clocks             : list of clock specifiers, corresponding to
> 
> entries in
> > > +                          the clock-names property;
> > > +- clock-names                : should contain "ahb", "mod" and
> > > "ram"
> 
> entries;
> > > +- assigned-clocks       : list of clocks assigned to the VE;
> > > +- assigned-clocks-rates : list of clock rates for the clocks
> > > assigned
> 
> to the VE;
> > > +- resets             : phandle for reset;
> > > +- interrupts         : should contain VE interrupt number;
> > > +- reg                        : should contain register base and
> > > length
> 
> of VE.
> > > +
> > > +Example:
> > > +
> > > +reserved-memory {
> > > +     #address-cells = <1>;
> > > +     #size-cells = <1>;
> > > +     ranges;
> > > +
> > > +     /* Address must be kept in the lower 256 MiBs of DRAM for
> > > VE. */
> > > +     ve_memory: cma@4a000000 {
> > > +             compatible = "shared-dma-pool";
> > > +             reg = <0x4a000000 0x6000000>;
> > > +             no-map;
> > > +             linux,cma-default;
> > > +     };
> > > +};
> > > +
> > > +video-engine@1c0e000 {
> > This is not really required by any specification, and not as common
> > as
> > gpu@..., but could this reasonably be called "vpu@1c0e000" to follow
> > somewhat-common practice?
> 
> AFAIR the name is supposed to be somewhat readable for someone that
> doesn't know the hardware. To me, "video-engine" sounds more obvious
> than "vpu", but we actually use "codec" already, in case of MFC and
> JPEG codec on Exynos. If encode/decode is the only functionality of
> this block, I'd personally go with "codec". If it can do other things,
> e.g. scaling/rotation without encode/decode, I'd probably call it
> "video-processor".

I agree that the term VPU is more commonly associated with video
decoding, while video engine could mean a number of things.

The reason I went with "video-engine" here (while still presenting the
driver as a VPU driver) is that Video Engine is the term used in
Allwinner's litterature. Other nodes in Allwinner device-trees generally
stick to these terms (for instance, we have "display-engine" nodes).
This also makes it easier to find the matching parts in the
documentation.

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 4105 bytes --]

Hi and thanks for the review,

On Fri, 2018-04-20 at 01:31 +0000, Tomasz Figa wrote:
> Hi Paul, Philipp,
> 
> On Fri, Apr 20, 2018 at 1:04 AM Philipp Zabel <p.zabel-bIcnvbaLZ9MEGnE8C9+IrQ@public.gmane.org>
> wrote:
> 
> > Hi Paul,
> > On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> > > This adds a device-tree binding document that specifies the
> > > properties
> > > used by the Sunxi-Cedurs VPU driver, as well as examples.
> > > 
> > > Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> > > ---
> > >  .../devicetree/bindings/media/sunxi-cedrus.txt     | 50
> 
> ++++++++++++++++++++++
> > >  1 file changed, 50 insertions(+)
> > >  create mode 100644
> 
> Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > 
> > > diff --git a/Documentation/devicetree/bindings/media/sunxi-
> > > cedrus.txt
> 
> b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > new file mode 100644
> > > index 000000000000..71ad3f9c3352
> > > --- /dev/null
> > > +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > @@ -0,0 +1,50 @@
> > > +Device-tree bindings for the VPU found in Allwinner SoCs,
> > > referred to
> 
> as the
> > > +Video Engine (VE) in Allwinner literature.
> > > +
> > > +The VPU can only access the first 256 MiB of DRAM, that are DMA-
> > > mapped
> 
> starting
> > > +from the DRAM base. This requires specific memory allocation and
> 
> handling.
> 
> And no IOMMU? Brings back memories.

Exactly, no IOMMU so we don't have much choice but cope with that
hardware limitation...

> > > +
> > > +Required properties:
> > > +- compatible         : "allwinner,sun4i-a10-video-engine";
> > > +- memory-region         : DMA pool for buffers allocation;
> > > +- clocks             : list of clock specifiers, corresponding to
> 
> entries in
> > > +                          the clock-names property;
> > > +- clock-names                : should contain "ahb", "mod" and
> > > "ram"
> 
> entries;
> > > +- assigned-clocks       : list of clocks assigned to the VE;
> > > +- assigned-clocks-rates : list of clock rates for the clocks
> > > assigned
> 
> to the VE;
> > > +- resets             : phandle for reset;
> > > +- interrupts         : should contain VE interrupt number;
> > > +- reg                        : should contain register base and
> > > length
> 
> of VE.
> > > +
> > > +Example:
> > > +
> > > +reserved-memory {
> > > +     #address-cells = <1>;
> > > +     #size-cells = <1>;
> > > +     ranges;
> > > +
> > > +     /* Address must be kept in the lower 256 MiBs of DRAM for
> > > VE. */
> > > +     ve_memory: cma@4a000000 {
> > > +             compatible = "shared-dma-pool";
> > > +             reg = <0x4a000000 0x6000000>;
> > > +             no-map;
> > > +             linux,cma-default;
> > > +     };
> > > +};
> > > +
> > > +video-engine@1c0e000 {
> > This is not really required by any specification, and not as common
> > as
> > gpu@..., but could this reasonably be called "vpu@1c0e000" to follow
> > somewhat-common practice?
> 
> AFAIR the name is supposed to be somewhat readable for someone that
> doesn't know the hardware. To me, "video-engine" sounds more obvious
> than "vpu", but we actually use "codec" already, in case of MFC and
> JPEG codec on Exynos. If encode/decode is the only functionality of
> this block, I'd personally go with "codec". If it can do other things,
> e.g. scaling/rotation without encode/decode, I'd probably call it
> "video-processor".

I agree that the term VPU is more commonly associated with video
decoding, while video engine could mean a number of things.

The reason I went with "video-engine" here (while still presenting the
driver as a VPU driver) is that Video Engine is the term used in
Allwinner's litterature. Other nodes in Allwinner device-trees generally
stick to these terms (for instance, we have "display-engine" nodes).
This also makes it easier to find the matching parts in the
documentation.

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Paul Kocialkowski]

Hi and thanks for the review,

On Fri, 2018-04-20 at 01:31 +0000, Tomasz Figa wrote:
> Hi Paul, Philipp,
> 
> On Fri, Apr 20, 2018 at 1:04 AM Philipp Zabel <p.zabel@pengutronix.de>
> wrote:
> 
> > Hi Paul,
> > On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> > > This adds a device-tree binding document that specifies the
> > > properties
> > > used by the Sunxi-Cedurs VPU driver, as well as examples.
> > > 
> > > Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> > > ---
> > >  .../devicetree/bindings/media/sunxi-cedrus.txt     | 50
> 
> ++++++++++++++++++++++
> > >  1 file changed, 50 insertions(+)
> > >  create mode 100644
> 
> Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > 
> > > diff --git a/Documentation/devicetree/bindings/media/sunxi-
> > > cedrus.txt
> 
> b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > new file mode 100644
> > > index 000000000000..71ad3f9c3352
> > > --- /dev/null
> > > +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > @@ -0,0 +1,50 @@
> > > +Device-tree bindings for the VPU found in Allwinner SoCs,
> > > referred to
> 
> as the
> > > +Video Engine (VE) in Allwinner literature.
> > > +
> > > +The VPU can only access the first 256 MiB of DRAM, that are DMA-
> > > mapped
> 
> starting
> > > +from the DRAM base. This requires specific memory allocation and
> 
> handling.
> 
> And no IOMMU? Brings back memories.

Exactly, no IOMMU so we don't have much choice but cope with that
hardware limitation...

> > > +
> > > +Required properties:
> > > +- compatible         : "allwinner,sun4i-a10-video-engine";
> > > +- memory-region         : DMA pool for buffers allocation;
> > > +- clocks             : list of clock specifiers, corresponding to
> 
> entries in
> > > +                          the clock-names property;
> > > +- clock-names                : should contain "ahb", "mod" and
> > > "ram"
> 
> entries;
> > > +- assigned-clocks       : list of clocks assigned to the VE;
> > > +- assigned-clocks-rates : list of clock rates for the clocks
> > > assigned
> 
> to the VE;
> > > +- resets             : phandle for reset;
> > > +- interrupts         : should contain VE interrupt number;
> > > +- reg                        : should contain register base and
> > > length
> 
> of VE.
> > > +
> > > +Example:
> > > +
> > > +reserved-memory {
> > > +     #address-cells = <1>;
> > > +     #size-cells = <1>;
> > > +     ranges;
> > > +
> > > +     /* Address must be kept in the lower 256 MiBs of DRAM for
> > > VE. */
> > > +     ve_memory: cma at 4a000000 {
> > > +             compatible = "shared-dma-pool";
> > > +             reg = <0x4a000000 0x6000000>;
> > > +             no-map;
> > > +             linux,cma-default;
> > > +     };
> > > +};
> > > +
> > > +video-engine at 1c0e000 {
> > This is not really required by any specification, and not as common
> > as
> > gpu at ..., but could this reasonably be called "vpu at 1c0e000" to follow
> > somewhat-common practice?
> 
> AFAIR the name is supposed to be somewhat readable for someone that
> doesn't know the hardware. To me, "video-engine" sounds more obvious
> than "vpu", but we actually use "codec" already, in case of MFC and
> JPEG codec on Exynos. If encode/decode is the only functionality of
> this block, I'd personally go with "codec". If it can do other things,
> e.g. scaling/rotation without encode/decode, I'd probably call it
> "video-processor".

I agree that the term VPU is more commonly associated with video
decoding, while video engine could mean a number of things.

The reason I went with "video-engine" here (while still presenting the
driver as a VPU driver) is that Video Engine is the term used in
Allwinner's litterature. Other nodes in Allwinner device-trees generally
stick to these terms (for instance, we have "display-engine" nodes).
This also makes it easier to find the matching parts in the
documentation.

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180420/18984815/attachment.sig>

Re: [PATCH v2 01/10] media: v4l2-ctrls: Add missing v4l2 ctrl unlock

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 1082 bytes --]

On Thu, Apr 19, 2018 at 05:41:15PM +0200, Paul Kocialkowski wrote:
> This adds a missing v4l2_ctrl_unlock call that is required to avoid
> deadlocks.

Maybe you can explain what the deadlock scenario is?

> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index f67e9f5531fa..ba05a8b9a095 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -3614,10 +3614,12 @@ void v4l2_ctrl_request_complete(struct media_request *req,
>  			continue;
>  
>  		v4l2_ctrl_lock(ctrl);
> +
>  		if (ref->req)
>  			ptr_to_ptr(ctrl, ref->req->p_req, ref->p_req);
>  		else
>  			ptr_to_ptr(ctrl, ctrl->p_cur, ref->p_req);
> +

I'm not sure that this is relevant in this patch.

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 01/10] media: v4l2-ctrls: Add missing v4l2 ctrl unlock

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 1075 bytes --]

On Thu, Apr 19, 2018 at 05:41:15PM +0200, Paul Kocialkowski wrote:
> This adds a missing v4l2_ctrl_unlock call that is required to avoid
> deadlocks.

Maybe you can explain what the deadlock scenario is?

> Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index f67e9f5531fa..ba05a8b9a095 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -3614,10 +3614,12 @@ void v4l2_ctrl_request_complete(struct media_request *req,
>  			continue;
>  
>  		v4l2_ctrl_lock(ctrl);
> +
>  		if (ref->req)
>  			ptr_to_ptr(ctrl, ref->req->p_req, ref->p_req);
>  		else
>  			ptr_to_ptr(ctrl, ctrl->p_cur, ref->p_req);
> +

I'm not sure that this is relevant in this patch.

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[PATCH v2 01/10] media: v4l2-ctrls: Add missing v4l2 ctrl unlock

[Maxime Ripard]

On Thu, Apr 19, 2018 at 05:41:15PM +0200, Paul Kocialkowski wrote:
> This adds a missing v4l2_ctrl_unlock call that is required to avoid
> deadlocks.

Maybe you can explain what the deadlock scenario is?

> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index f67e9f5531fa..ba05a8b9a095 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -3614,10 +3614,12 @@ void v4l2_ctrl_request_complete(struct media_request *req,
>  			continue;
>  
>  		v4l2_ctrl_lock(ctrl);
> +
>  		if (ref->req)
>  			ptr_to_ptr(ctrl, ref->req->p_req, ref->p_req);
>  		else
>  			ptr_to_ptr(ctrl, ctrl->p_cur, ref->p_req);
> +

I'm not sure that this is relevant in this patch.

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180420/6b564f27/attachment.sig>

Re: [PATCH v2 02/10] media-request: Add a request complete operation to allow m2m scheduling

[Alexandre Courbot]

On Fri, Apr 20, 2018 at 12:43 AM Paul Kocialkowski <
paul.kocialkowski@bootlin.com> wrote:

> When using the request API in the context of a m2m driver, the
> operations that come with a m2m run scheduling call in their
> (m2m-specific) ioctl handler are delayed until the request is queued
> (for instance, this includes queuing buffers and streamon).

> Thus, the m2m run scheduling calls are not called in due time since the
> request AP's internal plumbing will (rightfully) use the relevant core
> functions directly instead of the ioctl handler.

> This ends up in a situation where nothing happens if there is no
> run-scheduling ioctl called after queuing the request.

> In order to circumvent the issue, a new media operation is introduced,
> called at the time of handling the media request queue ioctl. It gives
> m2m drivers a chance to schedule a m2m device run at that time.

> The existing req_queue operation cannot be used for this purpose, since
> it is called with the request queue mutex held, that is eventually needed
> in the device_run call to apply relevant controls.

> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>   drivers/media/media-request.c | 3 +++
>   include/media/media-device.h  | 2 ++
>   2 files changed, 5 insertions(+)

> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> index 415f7e31019d..28ac5ccfe6a2 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -157,6 +157,9 @@ static long media_request_ioctl_queue(struct
media_request *req)
>                  media_request_get(req);
>          }

> +       if (mdev->ops->req_complete)
> +               mdev->ops->req_complete(req);
> +
>          return ret;
>   }

> diff --git a/include/media/media-device.h b/include/media/media-device.h
> index 07e323c57202..c7dcf2079cc9 100644
> --- a/include/media/media-device.h
> +++ b/include/media/media-device.h
> @@ -55,6 +55,7 @@ struct media_entity_notify {
>    * @req_alloc: Allocate a request
>    * @req_free: Free a request
>    * @req_queue: Queue a request
> + * @req_complete: Complete a request
>    */
>   struct media_device_ops {
>          int (*link_notify)(struct media_link *link, u32 flags,
> @@ -62,6 +63,7 @@ struct media_device_ops {
>          struct media_request *(*req_alloc)(struct media_device *mdev);
>          void (*req_free)(struct media_request *req);
>          int (*req_queue)(struct media_request *req);
> +       void (*req_complete)(struct media_request *req);

This is called *before* the request is actually run, isn't it? In that
case, wouldn't something like "req_schedule" be less confusing?
req_complete implies that the request is already completed.

[PATCH v2 02/10] media-request: Add a request complete operation to allow m2m scheduling

[Alexandre Courbot]

On Fri, Apr 20, 2018 at 12:43 AM Paul Kocialkowski <
paul.kocialkowski@bootlin.com> wrote:

> When using the request API in the context of a m2m driver, the
> operations that come with a m2m run scheduling call in their
> (m2m-specific) ioctl handler are delayed until the request is queued
> (for instance, this includes queuing buffers and streamon).

> Thus, the m2m run scheduling calls are not called in due time since the
> request AP's internal plumbing will (rightfully) use the relevant core
> functions directly instead of the ioctl handler.

> This ends up in a situation where nothing happens if there is no
> run-scheduling ioctl called after queuing the request.

> In order to circumvent the issue, a new media operation is introduced,
> called at the time of handling the media request queue ioctl. It gives
> m2m drivers a chance to schedule a m2m device run at that time.

> The existing req_queue operation cannot be used for this purpose, since
> it is called with the request queue mutex held, that is eventually needed
> in the device_run call to apply relevant controls.

> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>   drivers/media/media-request.c | 3 +++
>   include/media/media-device.h  | 2 ++
>   2 files changed, 5 insertions(+)

> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> index 415f7e31019d..28ac5ccfe6a2 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -157,6 +157,9 @@ static long media_request_ioctl_queue(struct
media_request *req)
>                  media_request_get(req);
>          }

> +       if (mdev->ops->req_complete)
> +               mdev->ops->req_complete(req);
> +
>          return ret;
>   }

> diff --git a/include/media/media-device.h b/include/media/media-device.h
> index 07e323c57202..c7dcf2079cc9 100644
> --- a/include/media/media-device.h
> +++ b/include/media/media-device.h
> @@ -55,6 +55,7 @@ struct media_entity_notify {
>    * @req_alloc: Allocate a request
>    * @req_free: Free a request
>    * @req_queue: Queue a request
> + * @req_complete: Complete a request
>    */
>   struct media_device_ops {
>          int (*link_notify)(struct media_link *link, u32 flags,
> @@ -62,6 +63,7 @@ struct media_device_ops {
>          struct media_request *(*req_alloc)(struct media_device *mdev);
>          void (*req_free)(struct media_request *req);
>          int (*req_queue)(struct media_request *req);
> +       void (*req_complete)(struct media_request *req);

This is called *before* the request is actually run, isn't it? In that
case, wouldn't something like "req_schedule" be less confusing?
req_complete implies that the request is already completed.

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 2867 bytes --]

On Thu, Apr 19, 2018 at 05:45:35PM +0200, Paul Kocialkowski wrote:
> This adds nodes for the Video Engine and the associated reserved memory
> for the Allwinner A20. Up to 96 MiB of memory are dedicated to the VPU.
> 
> The VPU can only map the first 256 MiB of DRAM, so the reserved memory
> pool has to be located in that area. Following Allwinner's decision in
> downstream software, the last 96 MiB of the first 256 MiB of RAM are
> reserved for this purpose.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  arch/arm/boot/dts/sun7i-a20.dtsi | 31 +++++++++++++++++++++++++++++++
>  1 file changed, 31 insertions(+)
> 
> diff --git a/arch/arm/boot/dts/sun7i-a20.dtsi b/arch/arm/boot/dts/sun7i-a20.dtsi
> index bd0cd3204273..cb6d82065dcf 100644
> --- a/arch/arm/boot/dts/sun7i-a20.dtsi
> +++ b/arch/arm/boot/dts/sun7i-a20.dtsi
> @@ -163,6 +163,20 @@
>  		reg = <0x40000000 0x80000000>;
>  	};
>  
> +	reserved-memory {
> +		#address-cells = <1>;
> +		#size-cells = <1>;
> +		ranges;
> +
> +		/* Address must be kept in the lower 256 MiBs of DRAM for VE. */
> +		ve_memory: cma@4a000000 {
> +			compatible = "shared-dma-pool";
> +			reg = <0x4a000000 0x6000000>;
> +			no-map;

I'm not sure why no-map is needed.

And I guess we could use alloc-ranges to make sure the region is in
the proper memory range, instead of hardcoding it.

> +			linux,cma-default;
> +		};
> +	};
> +
>  	timer {
>  		compatible = "arm,armv7-timer";
>  		interrupts = <GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_LEVEL_LOW)>,
> @@ -451,6 +465,23 @@
>  			};
>  		};
>  
> +		ve: video-engine@1c0e000 {
> +			compatible = "allwinner,sun4i-a10-video-engine";

We should have an A20-specific compatible here, at least, so that we
can deal with any quirk we might find down the road.

> +			reg = <0x01c0e000 0x1000>;
> +			memory-region = <&ve_memory>;

Since you made the CMA region the default one, you don't need to tie
it to that device in particular (and you can drop it being mandatory
from your binding as well).

> +
> +			clocks = <&ccu CLK_AHB_VE>, <&ccu CLK_VE>,
> +				 <&ccu CLK_DRAM_VE>;
> +			clock-names = "ahb", "mod", "ram";
> +
> +			assigned-clocks = <&ccu CLK_VE>;
> +			assigned-clock-rates = <320000000>;

This should be set from within the driver. If it's something that you
absolutely needed for the device to operate, you have no guarantee
that the clock rate won't change at any point in time after the device
probe, so that's not a proper solution.

And if it's not needed and can be adjusted depending on the
framerate/codec/resolution, then it shouldn't be in the DT either.

Don't you also need to map the SRAM on the A20?

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 2808 bytes --]

On Thu, Apr 19, 2018 at 05:45:35PM +0200, Paul Kocialkowski wrote:
> This adds nodes for the Video Engine and the associated reserved memory
> for the Allwinner A20. Up to 96 MiB of memory are dedicated to the VPU.
> 
> The VPU can only map the first 256 MiB of DRAM, so the reserved memory
> pool has to be located in that area. Following Allwinner's decision in
> downstream software, the last 96 MiB of the first 256 MiB of RAM are
> reserved for this purpose.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> ---
>  arch/arm/boot/dts/sun7i-a20.dtsi | 31 +++++++++++++++++++++++++++++++
>  1 file changed, 31 insertions(+)
> 
> diff --git a/arch/arm/boot/dts/sun7i-a20.dtsi b/arch/arm/boot/dts/sun7i-a20.dtsi
> index bd0cd3204273..cb6d82065dcf 100644
> --- a/arch/arm/boot/dts/sun7i-a20.dtsi
> +++ b/arch/arm/boot/dts/sun7i-a20.dtsi
> @@ -163,6 +163,20 @@
>  		reg = <0x40000000 0x80000000>;
>  	};
>  
> +	reserved-memory {
> +		#address-cells = <1>;
> +		#size-cells = <1>;
> +		ranges;
> +
> +		/* Address must be kept in the lower 256 MiBs of DRAM for VE. */
> +		ve_memory: cma@4a000000 {
> +			compatible = "shared-dma-pool";
> +			reg = <0x4a000000 0x6000000>;
> +			no-map;

I'm not sure why no-map is needed.

And I guess we could use alloc-ranges to make sure the region is in
the proper memory range, instead of hardcoding it.

> +			linux,cma-default;
> +		};
> +	};
> +
>  	timer {
>  		compatible = "arm,armv7-timer";
>  		interrupts = <GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_LEVEL_LOW)>,
> @@ -451,6 +465,23 @@
>  			};
>  		};
>  
> +		ve: video-engine@1c0e000 {
> +			compatible = "allwinner,sun4i-a10-video-engine";

We should have an A20-specific compatible here, at least, so that we
can deal with any quirk we might find down the road.

> +			reg = <0x01c0e000 0x1000>;
> +			memory-region = <&ve_memory>;

Since you made the CMA region the default one, you don't need to tie
it to that device in particular (and you can drop it being mandatory
from your binding as well).

> +
> +			clocks = <&ccu CLK_AHB_VE>, <&ccu CLK_VE>,
> +				 <&ccu CLK_DRAM_VE>;
> +			clock-names = "ahb", "mod", "ram";
> +
> +			assigned-clocks = <&ccu CLK_VE>;
> +			assigned-clock-rates = <320000000>;

This should be set from within the driver. If it's something that you
absolutely needed for the device to operate, you have no guarantee
that the clock rate won't change at any point in time after the device
probe, so that's not a proper solution.

And if it's not needed and can be adjusted depending on the
framerate/codec/resolution, then it shouldn't be in the DT either.

Don't you also need to map the SRAM on the A20?

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Maxime Ripard]

On Thu, Apr 19, 2018 at 05:45:35PM +0200, Paul Kocialkowski wrote:
> This adds nodes for the Video Engine and the associated reserved memory
> for the Allwinner A20. Up to 96 MiB of memory are dedicated to the VPU.
> 
> The VPU can only map the first 256 MiB of DRAM, so the reserved memory
> pool has to be located in that area. Following Allwinner's decision in
> downstream software, the last 96 MiB of the first 256 MiB of RAM are
> reserved for this purpose.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  arch/arm/boot/dts/sun7i-a20.dtsi | 31 +++++++++++++++++++++++++++++++
>  1 file changed, 31 insertions(+)
> 
> diff --git a/arch/arm/boot/dts/sun7i-a20.dtsi b/arch/arm/boot/dts/sun7i-a20.dtsi
> index bd0cd3204273..cb6d82065dcf 100644
> --- a/arch/arm/boot/dts/sun7i-a20.dtsi
> +++ b/arch/arm/boot/dts/sun7i-a20.dtsi
> @@ -163,6 +163,20 @@
>  		reg = <0x40000000 0x80000000>;
>  	};
>  
> +	reserved-memory {
> +		#address-cells = <1>;
> +		#size-cells = <1>;
> +		ranges;
> +
> +		/* Address must be kept in the lower 256 MiBs of DRAM for VE. */
> +		ve_memory: cma at 4a000000 {
> +			compatible = "shared-dma-pool";
> +			reg = <0x4a000000 0x6000000>;
> +			no-map;

I'm not sure why no-map is needed.

And I guess we could use alloc-ranges to make sure the region is in
the proper memory range, instead of hardcoding it.

> +			linux,cma-default;
> +		};
> +	};
> +
>  	timer {
>  		compatible = "arm,armv7-timer";
>  		interrupts = <GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_LEVEL_LOW)>,
> @@ -451,6 +465,23 @@
>  			};
>  		};
>  
> +		ve: video-engine at 1c0e000 {
> +			compatible = "allwinner,sun4i-a10-video-engine";

We should have an A20-specific compatible here, at least, so that we
can deal with any quirk we might find down the road.

> +			reg = <0x01c0e000 0x1000>;
> +			memory-region = <&ve_memory>;

Since you made the CMA region the default one, you don't need to tie
it to that device in particular (and you can drop it being mandatory
from your binding as well).

> +
> +			clocks = <&ccu CLK_AHB_VE>, <&ccu CLK_VE>,
> +				 <&ccu CLK_DRAM_VE>;
> +			clock-names = "ahb", "mod", "ram";
> +
> +			assigned-clocks = <&ccu CLK_VE>;
> +			assigned-clock-rates = <320000000>;

This should be set from within the driver. If it's something that you
absolutely needed for the device to operate, you have no guarantee
that the clock rate won't change at any point in time after the device
probe, so that's not a proper solution.

And if it's not needed and can be adjusted depending on the
framerate/codec/resolution, then it shouldn't be in the DT either.

Don't you also need to map the SRAM on the A20?

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180420/709dd34d/attachment-0001.sig>

Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Tomasz Figa]

Hi Paul,

On Fri, Apr 20, 2018 at 12:46 AM Paul Kocialkowski <
paul.kocialkowski@bootlin.com> wrote:
[snip]
> +struct v4l2_ctrl_mpeg2_frame_hdr {
> +       __u32 slice_len;
> +       __u32 slice_pos;
> +       enum { MPEG1, MPEG2 } type;

Is enum suitable for UAPI?

> +
> +       __u16 width;
> +       __u16 height;
> +
> +       enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;

Ditto.

> +       __u8 f_code[2][2];
> +
> +       __u8 intra_dc_precision;
> +       __u8 picture_structure;
> +       __u8 top_field_first;
> +       __u8 frame_pred_frame_dct;
> +       __u8 concealment_motion_vectors;
> +       __u8 q_scale_type;
> +       __u8 intra_vlc_format;
> +       __u8 alternate_scan;
> +
> +       __u8 backward_ref_index;
> +       __u8 forward_ref_index;
> +};
> +
>   #endif
> diff --git a/include/uapi/linux/videodev2.h
b/include/uapi/linux/videodev2.h
> index 31b5728b56e9..4b8336f7bcf0 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -635,6 +635,7 @@ struct v4l2_pix_format {
>   #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /*
SMPTE 421M Annex L compliant stream */
>   #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
>   #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
> +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /*
MPEG2 frame */

>   /*  Vendor-specific formats   */
>   #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1
YUV */
> @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
>                  __u8 __user *p_u8;
>                  __u16 __user *p_u16;
>                  __u32 __user *p_u32;
> +               struct v4l2_ctrl_mpeg2_frame_hdr __user
*p_mpeg2_frame_hdr;
>                  void __user *ptr;
>          };
>   } __attribute__ ((packed));
> @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
>          V4L2_CTRL_TYPE_U8            = 0x0100,
>          V4L2_CTRL_TYPE_U16           = 0x0101,
>          V4L2_CTRL_TYPE_U32           = 0x0102,
> +       V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,

Why 0x0109?

Best regards,
Tomasz

Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Tomasz Figa]

Hi Paul,

On Fri, Apr 20, 2018 at 12:46 AM Paul Kocialkowski <
paul.kocialkowski@bootlin.com> wrote:
[snip]
> +struct v4l2_ctrl_mpeg2_frame_hdr {
> +       __u32 slice_len;
> +       __u32 slice_pos;
> +       enum { MPEG1, MPEG2 } type;

Is enum suitable for UAPI?

> +
> +       __u16 width;
> +       __u16 height;
> +
> +       enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;

Ditto.

> +       __u8 f_code[2][2];
> +
> +       __u8 intra_dc_precision;
> +       __u8 picture_structure;
> +       __u8 top_field_first;
> +       __u8 frame_pred_frame_dct;
> +       __u8 concealment_motion_vectors;
> +       __u8 q_scale_type;
> +       __u8 intra_vlc_format;
> +       __u8 alternate_scan;
> +
> +       __u8 backward_ref_index;
> +       __u8 forward_ref_index;
> +};
> +
>   #endif
> diff --git a/include/uapi/linux/videodev2.h
b/include/uapi/linux/videodev2.h
> index 31b5728b56e9..4b8336f7bcf0 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -635,6 +635,7 @@ struct v4l2_pix_format {
>   #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /*
SMPTE 421M Annex L compliant stream */
>   #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
>   #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
> +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /*
MPEG2 frame */

>   /*  Vendor-specific formats   */
>   #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1
YUV */
> @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
>                  __u8 __user *p_u8;
>                  __u16 __user *p_u16;
>                  __u32 __user *p_u32;
> +               struct v4l2_ctrl_mpeg2_frame_hdr __user
*p_mpeg2_frame_hdr;
>                  void __user *ptr;
>          };
>   } __attribute__ ((packed));
> @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
>          V4L2_CTRL_TYPE_U8            = 0x0100,
>          V4L2_CTRL_TYPE_U16           = 0x0101,
>          V4L2_CTRL_TYPE_U32           = 0x0102,
> +       V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,

Why 0x0109?

Best regards,
Tomasz

[PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Tomasz Figa]

Hi Paul,

On Fri, Apr 20, 2018 at 12:46 AM Paul Kocialkowski <
paul.kocialkowski@bootlin.com> wrote:
[snip]
> +struct v4l2_ctrl_mpeg2_frame_hdr {
> +       __u32 slice_len;
> +       __u32 slice_pos;
> +       enum { MPEG1, MPEG2 } type;

Is enum suitable for UAPI?

> +
> +       __u16 width;
> +       __u16 height;
> +
> +       enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;

Ditto.

> +       __u8 f_code[2][2];
> +
> +       __u8 intra_dc_precision;
> +       __u8 picture_structure;
> +       __u8 top_field_first;
> +       __u8 frame_pred_frame_dct;
> +       __u8 concealment_motion_vectors;
> +       __u8 q_scale_type;
> +       __u8 intra_vlc_format;
> +       __u8 alternate_scan;
> +
> +       __u8 backward_ref_index;
> +       __u8 forward_ref_index;
> +};
> +
>   #endif
> diff --git a/include/uapi/linux/videodev2.h
b/include/uapi/linux/videodev2.h
> index 31b5728b56e9..4b8336f7bcf0 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -635,6 +635,7 @@ struct v4l2_pix_format {
>   #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /*
SMPTE 421M Annex L compliant stream */
>   #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
>   #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
> +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /*
MPEG2 frame */

>   /*  Vendor-specific formats   */
>   #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1
YUV */
> @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
>                  __u8 __user *p_u8;
>                  __u16 __user *p_u16;
>                  __u32 __user *p_u32;
> +               struct v4l2_ctrl_mpeg2_frame_hdr __user
*p_mpeg2_frame_hdr;
>                  void __user *ptr;
>          };
>   } __attribute__ ((packed));
> @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
>          V4L2_CTRL_TYPE_U8            = 0x0100,
>          V4L2_CTRL_TYPE_U16           = 0x0101,
>          V4L2_CTRL_TYPE_U32           = 0x0102,
> +       V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,

Why 0x0109?

Best regards,
Tomasz

Re: [PATCH v2 01/10] media: v4l2-ctrls: Add missing v4l2 ctrl unlock

[Hans Verkuil]

On 04/19/18 17:41, Paul Kocialkowski wrote:
> This adds a missing v4l2_ctrl_unlock call that is required to avoid
> deadlocks.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index f67e9f5531fa..ba05a8b9a095 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -3614,10 +3614,12 @@ void v4l2_ctrl_request_complete(struct media_request *req,
>  			continue;
>  
>  		v4l2_ctrl_lock(ctrl);
> +
>  		if (ref->req)
>  			ptr_to_ptr(ctrl, ref->req->p_req, ref->p_req);
>  		else
>  			ptr_to_ptr(ctrl, ctrl->p_cur, ref->p_req);
> +
>  		v4l2_ctrl_unlock(ctrl);
>  	}
>  
> @@ -3677,8 +3679,11 @@ void v4l2_ctrl_request_setup(struct media_request *req,
>  				}
>  			}
>  		}
> -		if (!have_new_data)
> +
> +		if (!have_new_data) {
> +			v4l2_ctrl_unlock(master);
>  			continue;
> +		}

Oops! Thanks for finding this. I'll fold this into the relevant patch in my tree.

>  
>  		for (i = 0; i < master->ncontrols; i++) {
>  			if (master->cluster[i]) {
> 

Regards,

	Hans

Re: [PATCH v2 01/10] media: v4l2-ctrls: Add missing v4l2 ctrl unlock

[Hans Verkuil]

On 04/19/18 17:41, Paul Kocialkowski wrote:
> This adds a missing v4l2_ctrl_unlock call that is required to avoid
> deadlocks.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index f67e9f5531fa..ba05a8b9a095 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -3614,10 +3614,12 @@ void v4l2_ctrl_request_complete(struct media_request *req,
>  			continue;
>  
>  		v4l2_ctrl_lock(ctrl);
> +
>  		if (ref->req)
>  			ptr_to_ptr(ctrl, ref->req->p_req, ref->p_req);
>  		else
>  			ptr_to_ptr(ctrl, ctrl->p_cur, ref->p_req);
> +
>  		v4l2_ctrl_unlock(ctrl);
>  	}
>  
> @@ -3677,8 +3679,11 @@ void v4l2_ctrl_request_setup(struct media_request *req,
>  				}
>  			}
>  		}
> -		if (!have_new_data)
> +
> +		if (!have_new_data) {
> +			v4l2_ctrl_unlock(master);
>  			continue;
> +		}

Oops! Thanks for finding this. I'll fold this into the relevant patch in my tree.

>  
>  		for (i = 0; i < master->ncontrols; i++) {
>  			if (master->cluster[i]) {
> 

Regards,

	Hans

[PATCH v2 01/10] media: v4l2-ctrls: Add missing v4l2 ctrl unlock

[Hans Verkuil]

On 04/19/18 17:41, Paul Kocialkowski wrote:
> This adds a missing v4l2_ctrl_unlock call that is required to avoid
> deadlocks.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index f67e9f5531fa..ba05a8b9a095 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -3614,10 +3614,12 @@ void v4l2_ctrl_request_complete(struct media_request *req,
>  			continue;
>  
>  		v4l2_ctrl_lock(ctrl);
> +
>  		if (ref->req)
>  			ptr_to_ptr(ctrl, ref->req->p_req, ref->p_req);
>  		else
>  			ptr_to_ptr(ctrl, ctrl->p_cur, ref->p_req);
> +
>  		v4l2_ctrl_unlock(ctrl);
>  	}
>  
> @@ -3677,8 +3679,11 @@ void v4l2_ctrl_request_setup(struct media_request *req,
>  				}
>  			}
>  		}
> -		if (!have_new_data)
> +
> +		if (!have_new_data) {
> +			v4l2_ctrl_unlock(master);
>  			continue;
> +		}

Oops! Thanks for finding this. I'll fold this into the relevant patch in my tree.

>  
>  		for (i = 0; i < master->ncontrols; i++) {
>  			if (master->cluster[i]) {
> 

Regards,

	Hans

Re: [PATCH v2 03/10] videobuf2-core: Add helper to get buffer private data from media request

[Hans Verkuil]

On 04/19/18 17:41, Paul Kocialkowski wrote:
> When calling media operation driver callbacks related to media requests,
> only a pointer to the request itself is provided, which is insufficient
> to retrieve the driver's context. Since the driver context is usually
> set as vb2 queue private data and given that the core can determine
> which objects attached to the request are buffers, it is possible to
> extract the associated private data for the first buffer found.
> 
> This is required in order to access the current m2m context from m2m
> drivers' private data in the context of media request operation
> callbacks. More specifically, this allows scheduling m2m device runs
> from the newly-introduced request complete operation.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  drivers/media/common/videobuf2/videobuf2-core.c | 15 +++++++++++++++
>  include/media/videobuf2-core.h                  |  1 +
>  2 files changed, 16 insertions(+)
> 
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index 13c9d9e243dd..6fa46bfc620f 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -1351,6 +1351,21 @@ bool vb2_core_request_has_buffers(struct media_request *req)
>  }
>  EXPORT_SYMBOL_GPL(vb2_core_request_has_buffers);
>  
> +void *vb2_core_request_find_buffer_priv(struct media_request *req)
> +{
> +	struct media_request_object *obj;
> +	struct vb2_buffer *vb;
> +
> +	obj = media_request_object_find(req, &vb2_core_req_ops, NULL);

This increases the object refcount but it is never decreased here.

> +	if (!obj)
> +		return NULL;
> +
> +	vb = container_of(obj, struct vb2_buffer, req_obj);
> +
> +	return vb2_get_drv_priv(vb->vb2_queue);

You need to add a media_request_object_put(obj); before returning here.

Regards,

	Hans

> +}
> +EXPORT_SYMBOL_GPL(vb2_core_request_find_buffer_priv);
> +
>  int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb,
>  			 struct media_request *req)
>  {
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index 032bd1bec555..65c0cf6afb55 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -1153,4 +1153,5 @@ int vb2_verify_memory_type(struct vb2_queue *q,
>  		enum vb2_memory memory, unsigned int type);
>  
>  bool vb2_core_request_has_buffers(struct media_request *req);
> +void *vb2_core_request_find_buffer_priv(struct media_request *req);
>  #endif /* _MEDIA_VIDEOBUF2_CORE_H */
> 

Re: [PATCH v2 03/10] videobuf2-core: Add helper to get buffer private data from media request

[Hans Verkuil]

On 04/19/18 17:41, Paul Kocialkowski wrote:
> When calling media operation driver callbacks related to media requests,
> only a pointer to the request itself is provided, which is insufficient
> to retrieve the driver's context. Since the driver context is usually
> set as vb2 queue private data and given that the core can determine
> which objects attached to the request are buffers, it is possible to
> extract the associated private data for the first buffer found.
> 
> This is required in order to access the current m2m context from m2m
> drivers' private data in the context of media request operation
> callbacks. More specifically, this allows scheduling m2m device runs
> from the newly-introduced request complete operation.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> ---
>  drivers/media/common/videobuf2/videobuf2-core.c | 15 +++++++++++++++
>  include/media/videobuf2-core.h                  |  1 +
>  2 files changed, 16 insertions(+)
> 
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index 13c9d9e243dd..6fa46bfc620f 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -1351,6 +1351,21 @@ bool vb2_core_request_has_buffers(struct media_request *req)
>  }
>  EXPORT_SYMBOL_GPL(vb2_core_request_has_buffers);
>  
> +void *vb2_core_request_find_buffer_priv(struct media_request *req)
> +{
> +	struct media_request_object *obj;
> +	struct vb2_buffer *vb;
> +
> +	obj = media_request_object_find(req, &vb2_core_req_ops, NULL);

This increases the object refcount but it is never decreased here.

> +	if (!obj)
> +		return NULL;
> +
> +	vb = container_of(obj, struct vb2_buffer, req_obj);
> +
> +	return vb2_get_drv_priv(vb->vb2_queue);

You need to add a media_request_object_put(obj); before returning here.

Regards,

	Hans

> +}
> +EXPORT_SYMBOL_GPL(vb2_core_request_find_buffer_priv);
> +
>  int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb,
>  			 struct media_request *req)
>  {
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index 032bd1bec555..65c0cf6afb55 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -1153,4 +1153,5 @@ int vb2_verify_memory_type(struct vb2_queue *q,
>  		enum vb2_memory memory, unsigned int type);
>  
>  bool vb2_core_request_has_buffers(struct media_request *req);
> +void *vb2_core_request_find_buffer_priv(struct media_request *req);
>  #endif /* _MEDIA_VIDEOBUF2_CORE_H */
> 

[PATCH v2 03/10] videobuf2-core: Add helper to get buffer private data from media request

[Hans Verkuil]

On 04/19/18 17:41, Paul Kocialkowski wrote:
> When calling media operation driver callbacks related to media requests,
> only a pointer to the request itself is provided, which is insufficient
> to retrieve the driver's context. Since the driver context is usually
> set as vb2 queue private data and given that the core can determine
> which objects attached to the request are buffers, it is possible to
> extract the associated private data for the first buffer found.
> 
> This is required in order to access the current m2m context from m2m
> drivers' private data in the context of media request operation
> callbacks. More specifically, this allows scheduling m2m device runs
> from the newly-introduced request complete operation.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  drivers/media/common/videobuf2/videobuf2-core.c | 15 +++++++++++++++
>  include/media/videobuf2-core.h                  |  1 +
>  2 files changed, 16 insertions(+)
> 
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index 13c9d9e243dd..6fa46bfc620f 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -1351,6 +1351,21 @@ bool vb2_core_request_has_buffers(struct media_request *req)
>  }
>  EXPORT_SYMBOL_GPL(vb2_core_request_has_buffers);
>  
> +void *vb2_core_request_find_buffer_priv(struct media_request *req)
> +{
> +	struct media_request_object *obj;
> +	struct vb2_buffer *vb;
> +
> +	obj = media_request_object_find(req, &vb2_core_req_ops, NULL);

This increases the object refcount but it is never decreased here.

> +	if (!obj)
> +		return NULL;
> +
> +	vb = container_of(obj, struct vb2_buffer, req_obj);
> +
> +	return vb2_get_drv_priv(vb->vb2_queue);

You need to add a media_request_object_put(obj); before returning here.

Regards,

	Hans

> +}
> +EXPORT_SYMBOL_GPL(vb2_core_request_find_buffer_priv);
> +
>  int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb,
>  			 struct media_request *req)
>  {
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index 032bd1bec555..65c0cf6afb55 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -1153,4 +1153,5 @@ int vb2_verify_memory_type(struct vb2_queue *q,
>  		enum vb2_memory memory, unsigned int type);
>  
>  bool vb2_core_request_has_buffers(struct media_request *req);
> +void *vb2_core_request_find_buffer_priv(struct media_request *req);
>  #endif /* _MEDIA_VIDEOBUF2_CORE_H */
> 

Re: [PATCH v2 02/10] media-request: Add a request complete operation to allow m2m scheduling

[Hans Verkuil]

On 04/19/18 17:41, Paul Kocialkowski wrote:
> When using the request API in the context of a m2m driver, the
> operations that come with a m2m run scheduling call in their
> (m2m-specific) ioctl handler are delayed until the request is queued
> (for instance, this includes queuing buffers and streamon).
> 
> Thus, the m2m run scheduling calls are not called in due time since the
> request AP's internal plumbing will (rightfully) use the relevant core
> functions directly instead of the ioctl handler.
> 
> This ends up in a situation where nothing happens if there is no
> run-scheduling ioctl called after queuing the request.
> 
> In order to circumvent the issue, a new media operation is introduced,
> called at the time of handling the media request queue ioctl. It gives
> m2m drivers a chance to schedule a m2m device run at that time.
> 
> The existing req_queue operation cannot be used for this purpose, since
> it is called with the request queue mutex held, that is eventually needed
> in the device_run call to apply relevant controls.

I'll need to think about this a bit more. I understand the problem so something
needs to be done. I would like to avoid adding yet another op, though.

Regards,

	Hans

> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  drivers/media/media-request.c | 3 +++
>  include/media/media-device.h  | 2 ++
>  2 files changed, 5 insertions(+)
> 
> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> index 415f7e31019d..28ac5ccfe6a2 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -157,6 +157,9 @@ static long media_request_ioctl_queue(struct media_request *req)
>  		media_request_get(req);
>  	}
>  
> +	if (mdev->ops->req_complete)
> +		mdev->ops->req_complete(req);
> +
>  	return ret;
>  }
>  
> diff --git a/include/media/media-device.h b/include/media/media-device.h
> index 07e323c57202..c7dcf2079cc9 100644
> --- a/include/media/media-device.h
> +++ b/include/media/media-device.h
> @@ -55,6 +55,7 @@ struct media_entity_notify {
>   * @req_alloc: Allocate a request
>   * @req_free: Free a request
>   * @req_queue: Queue a request
> + * @req_complete: Complete a request
>   */
>  struct media_device_ops {
>  	int (*link_notify)(struct media_link *link, u32 flags,
> @@ -62,6 +63,7 @@ struct media_device_ops {
>  	struct media_request *(*req_alloc)(struct media_device *mdev);
>  	void (*req_free)(struct media_request *req);
>  	int (*req_queue)(struct media_request *req);
> +	void (*req_complete)(struct media_request *req);
>  };
>  
>  /**
> 

Re: [PATCH v2 02/10] media-request: Add a request complete operation to allow m2m scheduling

[Hans Verkuil]

On 04/19/18 17:41, Paul Kocialkowski wrote:
> When using the request API in the context of a m2m driver, the
> operations that come with a m2m run scheduling call in their
> (m2m-specific) ioctl handler are delayed until the request is queued
> (for instance, this includes queuing buffers and streamon).
> 
> Thus, the m2m run scheduling calls are not called in due time since the
> request AP's internal plumbing will (rightfully) use the relevant core
> functions directly instead of the ioctl handler.
> 
> This ends up in a situation where nothing happens if there is no
> run-scheduling ioctl called after queuing the request.
> 
> In order to circumvent the issue, a new media operation is introduced,
> called at the time of handling the media request queue ioctl. It gives
> m2m drivers a chance to schedule a m2m device run at that time.
> 
> The existing req_queue operation cannot be used for this purpose, since
> it is called with the request queue mutex held, that is eventually needed
> in the device_run call to apply relevant controls.

I'll need to think about this a bit more. I understand the problem so something
needs to be done. I would like to avoid adding yet another op, though.

Regards,

	Hans

> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> ---
>  drivers/media/media-request.c | 3 +++
>  include/media/media-device.h  | 2 ++
>  2 files changed, 5 insertions(+)
> 
> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> index 415f7e31019d..28ac5ccfe6a2 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -157,6 +157,9 @@ static long media_request_ioctl_queue(struct media_request *req)
>  		media_request_get(req);
>  	}
>  
> +	if (mdev->ops->req_complete)
> +		mdev->ops->req_complete(req);
> +
>  	return ret;
>  }
>  
> diff --git a/include/media/media-device.h b/include/media/media-device.h
> index 07e323c57202..c7dcf2079cc9 100644
> --- a/include/media/media-device.h
> +++ b/include/media/media-device.h
> @@ -55,6 +55,7 @@ struct media_entity_notify {
>   * @req_alloc: Allocate a request
>   * @req_free: Free a request
>   * @req_queue: Queue a request
> + * @req_complete: Complete a request
>   */
>  struct media_device_ops {
>  	int (*link_notify)(struct media_link *link, u32 flags,
> @@ -62,6 +63,7 @@ struct media_device_ops {
>  	struct media_request *(*req_alloc)(struct media_device *mdev);
>  	void (*req_free)(struct media_request *req);
>  	int (*req_queue)(struct media_request *req);
> +	void (*req_complete)(struct media_request *req);
>  };
>  
>  /**
> 

[PATCH v2 02/10] media-request: Add a request complete operation to allow m2m scheduling

[Hans Verkuil]

On 04/19/18 17:41, Paul Kocialkowski wrote:
> When using the request API in the context of a m2m driver, the
> operations that come with a m2m run scheduling call in their
> (m2m-specific) ioctl handler are delayed until the request is queued
> (for instance, this includes queuing buffers and streamon).
> 
> Thus, the m2m run scheduling calls are not called in due time since the
> request AP's internal plumbing will (rightfully) use the relevant core
> functions directly instead of the ioctl handler.
> 
> This ends up in a situation where nothing happens if there is no
> run-scheduling ioctl called after queuing the request.
> 
> In order to circumvent the issue, a new media operation is introduced,
> called at the time of handling the media request queue ioctl. It gives
> m2m drivers a chance to schedule a m2m device run at that time.
> 
> The existing req_queue operation cannot be used for this purpose, since
> it is called with the request queue mutex held, that is eventually needed
> in the device_run call to apply relevant controls.

I'll need to think about this a bit more. I understand the problem so something
needs to be done. I would like to avoid adding yet another op, though.

Regards,

	Hans

> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  drivers/media/media-request.c | 3 +++
>  include/media/media-device.h  | 2 ++
>  2 files changed, 5 insertions(+)
> 
> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> index 415f7e31019d..28ac5ccfe6a2 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -157,6 +157,9 @@ static long media_request_ioctl_queue(struct media_request *req)
>  		media_request_get(req);
>  	}
>  
> +	if (mdev->ops->req_complete)
> +		mdev->ops->req_complete(req);
> +
>  	return ret;
>  }
>  
> diff --git a/include/media/media-device.h b/include/media/media-device.h
> index 07e323c57202..c7dcf2079cc9 100644
> --- a/include/media/media-device.h
> +++ b/include/media/media-device.h
> @@ -55,6 +55,7 @@ struct media_entity_notify {
>   * @req_alloc: Allocate a request
>   * @req_free: Free a request
>   * @req_queue: Queue a request
> + * @req_complete: Complete a request
>   */
>  struct media_device_ops {
>  	int (*link_notify)(struct media_link *link, u32 flags,
> @@ -62,6 +63,7 @@ struct media_device_ops {
>  	struct media_request *(*req_alloc)(struct media_device *mdev);
>  	void (*req_free)(struct media_request *req);
>  	int (*req_queue)(struct media_request *req);
> +	void (*req_complete)(struct media_request *req);
>  };
>  
>  /**
> 

Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Hans Verkuil]

On 04/19/18 17:45, Paul Kocialkowski wrote:
> Stateless video decoding engines require both the MPEG slices and
> associated metadata from the video stream in order to decode frames.
> 
> This introduces definitions for a new pixel format, describing buffers
> with MPEG2 slice data, as well as a control structure for passing the
> frame header (metadata) to drivers.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> Signed-off-by: Florent Revest <florent.revest@free-electrons.com>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
>  include/uapi/linux/v4l2-controls.h   | 26 ++++++++++++++++++++++++++
>  include/uapi/linux/videodev2.h       |  3 +++
>  4 files changed, 40 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index ba05a8b9a095..fcdc12b9a9e0 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		return "Vertical MV Search Range";
>  	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		return "Repeat Sequence Header";
>  	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		return "Force Key Frame";
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		return "MPEG2 Frame Header";
>  
>  	/* VPX controls */
>  	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		return "VPX Number of Partitions";
> @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>  	case V4L2_CID_RDS_TX_ALT_FREQS:
>  		*type = V4L2_CTRL_TYPE_U32;
>  		break;
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> +		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> +		break;
>  	default:
>  		*type = V4L2_CTRL_TYPE_INTEGER;
>  		break;
> @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
>  			return -ERANGE;
>  		return 0;
>  
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		return 0;
> +
>  	default:
>  		return -EINVAL;
>  	}
> @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
>  	case V4L2_CTRL_TYPE_U32:
>  		elem_size = sizeof(u32);
>  		break;
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
> +		break;
>  	default:
>  		if (type < V4L2_CTRL_COMPOUND_TYPES)
>  			elem_size = sizeof(s32);
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 468c3c65362d..8070203da5d2 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1 (SMPTE 412M Annex L)"; break;
>  		case V4L2_PIX_FMT_VP8:		descr = "VP8"; break;
>  		case V4L2_PIX_FMT_VP9:		descr = "VP9"; break;
> +		case V4L2_PIX_FMT_MPEG2_FRAME:	descr = "MPEG2 Frame"; break;
>  		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA YUV"; break;
>  		case V4L2_PIX_FMT_WNVA:		descr = "WNVA"; break;
>  		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA SN9C10X"; break;
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index cbbb750d87d1..8431b2a540c7 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
>  };
>  #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPEG_BASE+407)
>  
> +#define V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
> +
>  /*  Control IDs for VP8 streams
>   *  Although VP8 is not part of MPEG we add these controls to the MPEG class
>   *  as that class is already handling other video compression standards
> @@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
>  #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_CLASS_BASE + 3)
>  #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DETECT_CLASS_BASE + 4)
>  
> +struct v4l2_ctrl_mpeg2_frame_hdr {
> +	__u32 slice_len;
> +	__u32 slice_pos;
> +	enum { MPEG1, MPEG2 } type;
> +
> +	__u16 width;
> +	__u16 height;
> +
> +	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;

As someone else already mentioned (I believe): avoid enums. Use __u16
instead.

> +	__u8 f_code[2][2];
> +
> +	__u8 intra_dc_precision;
> +	__u8 picture_structure;
> +	__u8 top_field_first;
> +	__u8 frame_pred_frame_dct;
> +	__u8 concealment_motion_vectors;
> +	__u8 q_scale_type;
> +	__u8 intra_vlc_format;
> +	__u8 alternate_scan;
> +
> +	__u8 backward_ref_index;
> +	__u8 forward_ref_index;
> +};

Please test that the layout and size of this structure is identical for 32 and 64 bit
architectures, both on Intel and ARM.

It looks good, but it doesn't hurt to check.

Regards,

	Hans

> +
>  #endif
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 31b5728b56e9..4b8336f7bcf0 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -635,6 +635,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /* SMPTE 421M Annex L compliant stream */
>  #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
>  #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
> +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /* MPEG2 frame */
>  
>  /*  Vendor-specific formats   */
>  #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
> @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
>  		__u8 __user *p_u8;
>  		__u16 __user *p_u16;
>  		__u32 __user *p_u32;
> +		struct v4l2_ctrl_mpeg2_frame_hdr __user *p_mpeg2_frame_hdr;
>  		void __user *ptr;
>  	};
>  } __attribute__ ((packed));
> @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
>  	V4L2_CTRL_TYPE_U8	     = 0x0100,
>  	V4L2_CTRL_TYPE_U16	     = 0x0101,
>  	V4L2_CTRL_TYPE_U32	     = 0x0102,
> +	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
>  };
>  
>  /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
> 

Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Hans Verkuil]

On 04/19/18 17:45, Paul Kocialkowski wrote:
> Stateless video decoding engines require both the MPEG slices and
> associated metadata from the video stream in order to decode frames.
> 
> This introduces definitions for a new pixel format, describing buffers
> with MPEG2 slice data, as well as a control structure for passing the
> frame header (metadata) to drivers.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> Signed-off-by: Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
>  include/uapi/linux/v4l2-controls.h   | 26 ++++++++++++++++++++++++++
>  include/uapi/linux/videodev2.h       |  3 +++
>  4 files changed, 40 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index ba05a8b9a095..fcdc12b9a9e0 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		return "Vertical MV Search Range";
>  	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		return "Repeat Sequence Header";
>  	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		return "Force Key Frame";
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		return "MPEG2 Frame Header";
>  
>  	/* VPX controls */
>  	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		return "VPX Number of Partitions";
> @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>  	case V4L2_CID_RDS_TX_ALT_FREQS:
>  		*type = V4L2_CTRL_TYPE_U32;
>  		break;
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> +		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> +		break;
>  	default:
>  		*type = V4L2_CTRL_TYPE_INTEGER;
>  		break;
> @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
>  			return -ERANGE;
>  		return 0;
>  
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		return 0;
> +
>  	default:
>  		return -EINVAL;
>  	}
> @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
>  	case V4L2_CTRL_TYPE_U32:
>  		elem_size = sizeof(u32);
>  		break;
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
> +		break;
>  	default:
>  		if (type < V4L2_CTRL_COMPOUND_TYPES)
>  			elem_size = sizeof(s32);
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 468c3c65362d..8070203da5d2 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1 (SMPTE 412M Annex L)"; break;
>  		case V4L2_PIX_FMT_VP8:		descr = "VP8"; break;
>  		case V4L2_PIX_FMT_VP9:		descr = "VP9"; break;
> +		case V4L2_PIX_FMT_MPEG2_FRAME:	descr = "MPEG2 Frame"; break;
>  		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA YUV"; break;
>  		case V4L2_PIX_FMT_WNVA:		descr = "WNVA"; break;
>  		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA SN9C10X"; break;
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index cbbb750d87d1..8431b2a540c7 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
>  };
>  #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPEG_BASE+407)
>  
> +#define V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
> +
>  /*  Control IDs for VP8 streams
>   *  Although VP8 is not part of MPEG we add these controls to the MPEG class
>   *  as that class is already handling other video compression standards
> @@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
>  #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_CLASS_BASE + 3)
>  #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DETECT_CLASS_BASE + 4)
>  
> +struct v4l2_ctrl_mpeg2_frame_hdr {
> +	__u32 slice_len;
> +	__u32 slice_pos;
> +	enum { MPEG1, MPEG2 } type;
> +
> +	__u16 width;
> +	__u16 height;
> +
> +	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;

As someone else already mentioned (I believe): avoid enums. Use __u16
instead.

> +	__u8 f_code[2][2];
> +
> +	__u8 intra_dc_precision;
> +	__u8 picture_structure;
> +	__u8 top_field_first;
> +	__u8 frame_pred_frame_dct;
> +	__u8 concealment_motion_vectors;
> +	__u8 q_scale_type;
> +	__u8 intra_vlc_format;
> +	__u8 alternate_scan;
> +
> +	__u8 backward_ref_index;
> +	__u8 forward_ref_index;
> +};

Please test that the layout and size of this structure is identical for 32 and 64 bit
architectures, both on Intel and ARM.

It looks good, but it doesn't hurt to check.

Regards,

	Hans

> +
>  #endif
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 31b5728b56e9..4b8336f7bcf0 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -635,6 +635,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /* SMPTE 421M Annex L compliant stream */
>  #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
>  #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
> +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /* MPEG2 frame */
>  
>  /*  Vendor-specific formats   */
>  #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
> @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
>  		__u8 __user *p_u8;
>  		__u16 __user *p_u16;
>  		__u32 __user *p_u32;
> +		struct v4l2_ctrl_mpeg2_frame_hdr __user *p_mpeg2_frame_hdr;
>  		void __user *ptr;
>  	};
>  } __attribute__ ((packed));
> @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
>  	V4L2_CTRL_TYPE_U8	     = 0x0100,
>  	V4L2_CTRL_TYPE_U16	     = 0x0101,
>  	V4L2_CTRL_TYPE_U32	     = 0x0102,
> +	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
>  };
>  
>  /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
> 

[PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Hans Verkuil]

On 04/19/18 17:45, Paul Kocialkowski wrote:
> Stateless video decoding engines require both the MPEG slices and
> associated metadata from the video stream in order to decode frames.
> 
> This introduces definitions for a new pixel format, describing buffers
> with MPEG2 slice data, as well as a control structure for passing the
> frame header (metadata) to drivers.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> Signed-off-by: Florent Revest <florent.revest@free-electrons.com>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
>  include/uapi/linux/v4l2-controls.h   | 26 ++++++++++++++++++++++++++
>  include/uapi/linux/videodev2.h       |  3 +++
>  4 files changed, 40 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index ba05a8b9a095..fcdc12b9a9e0 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		return "Vertical MV Search Range";
>  	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		return "Repeat Sequence Header";
>  	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		return "Force Key Frame";
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		return "MPEG2 Frame Header";
>  
>  	/* VPX controls */
>  	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		return "VPX Number of Partitions";
> @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>  	case V4L2_CID_RDS_TX_ALT_FREQS:
>  		*type = V4L2_CTRL_TYPE_U32;
>  		break;
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> +		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> +		break;
>  	default:
>  		*type = V4L2_CTRL_TYPE_INTEGER;
>  		break;
> @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
>  			return -ERANGE;
>  		return 0;
>  
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		return 0;
> +
>  	default:
>  		return -EINVAL;
>  	}
> @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
>  	case V4L2_CTRL_TYPE_U32:
>  		elem_size = sizeof(u32);
>  		break;
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
> +		break;
>  	default:
>  		if (type < V4L2_CTRL_COMPOUND_TYPES)
>  			elem_size = sizeof(s32);
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 468c3c65362d..8070203da5d2 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1 (SMPTE 412M Annex L)"; break;
>  		case V4L2_PIX_FMT_VP8:		descr = "VP8"; break;
>  		case V4L2_PIX_FMT_VP9:		descr = "VP9"; break;
> +		case V4L2_PIX_FMT_MPEG2_FRAME:	descr = "MPEG2 Frame"; break;
>  		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA YUV"; break;
>  		case V4L2_PIX_FMT_WNVA:		descr = "WNVA"; break;
>  		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA SN9C10X"; break;
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index cbbb750d87d1..8431b2a540c7 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
>  };
>  #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPEG_BASE+407)
>  
> +#define V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
> +
>  /*  Control IDs for VP8 streams
>   *  Although VP8 is not part of MPEG we add these controls to the MPEG class
>   *  as that class is already handling other video compression standards
> @@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
>  #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_CLASS_BASE + 3)
>  #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DETECT_CLASS_BASE + 4)
>  
> +struct v4l2_ctrl_mpeg2_frame_hdr {
> +	__u32 slice_len;
> +	__u32 slice_pos;
> +	enum { MPEG1, MPEG2 } type;
> +
> +	__u16 width;
> +	__u16 height;
> +
> +	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;

As someone else already mentioned (I believe): avoid enums. Use __u16
instead.

> +	__u8 f_code[2][2];
> +
> +	__u8 intra_dc_precision;
> +	__u8 picture_structure;
> +	__u8 top_field_first;
> +	__u8 frame_pred_frame_dct;
> +	__u8 concealment_motion_vectors;
> +	__u8 q_scale_type;
> +	__u8 intra_vlc_format;
> +	__u8 alternate_scan;
> +
> +	__u8 backward_ref_index;
> +	__u8 forward_ref_index;
> +};

Please test that the layout and size of this structure is identical for 32 and 64 bit
architectures, both on Intel and ARM.

It looks good, but it doesn't hurt to check.

Regards,

	Hans

> +
>  #endif
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 31b5728b56e9..4b8336f7bcf0 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -635,6 +635,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /* SMPTE 421M Annex L compliant stream */
>  #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
>  #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
> +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /* MPEG2 frame */
>  
>  /*  Vendor-specific formats   */
>  #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
> @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
>  		__u8 __user *p_u8;
>  		__u16 __user *p_u16;
>  		__u32 __user *p_u32;
> +		struct v4l2_ctrl_mpeg2_frame_hdr __user *p_mpeg2_frame_hdr;
>  		void __user *ptr;
>  	};
>  } __attribute__ ((packed));
> @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
>  	V4L2_CTRL_TYPE_U8	     = 0x0100,
>  	V4L2_CTRL_TYPE_U16	     = 0x0101,
>  	V4L2_CTRL_TYPE_U32	     = 0x0102,
> +	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
>  };
>  
>  /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
> 

Re: [PATCH v2 06/10] media: v4l: Add definition for Allwinner's MB32-tiled NV12 format

[Hans Verkuil]

On 04/19/18 17:45, Paul Kocialkowski wrote:
> This introduces support for Allwinner's MB32-tiled NV12 format, where
> each plane is divided into macroblocks of 32x32 pixels. Hence, the size
> of each plane has to be aligned to 32 bytes. The pixels inside each
> macroblock are coded as they would be if the macroblock was a single
> plane, line after line.
> 
> The MB32-tiled NV12 format is used by the video engine on Allwinner
> platforms: it is the default format for decoded frames (and the only one
> available in the oldest supported platforms).
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  include/uapi/linux/videodev2.h | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 4b8336f7bcf0..43993a116e2b 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -669,6 +669,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_Z16      v4l2_fourcc('Z', '1', '6', ' ') /* Depth data 16-bit */
>  #define V4L2_PIX_FMT_MT21C    v4l2_fourcc('M', 'T', '2', '1') /* Mediatek compressed block mode  */
>  #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
> +#define V4L2_PIX_FMT_MB32_NV12 v4l2_fourcc('M', 'N', '1', '2') /* Allwinner NV12 in 32x32 macroblocks */
>  
>  /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
>  #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
> 

Add an entry for this to v4l_fill_fmtdesc() in v4l2-ioctl.c.

It also needs to be documented in the spec.

Regards,

	Hans

Re: [PATCH v2 06/10] media: v4l: Add definition for Allwinner's MB32-tiled NV12 format

[Hans Verkuil]

On 04/19/18 17:45, Paul Kocialkowski wrote:
> This introduces support for Allwinner's MB32-tiled NV12 format, where
> each plane is divided into macroblocks of 32x32 pixels. Hence, the size
> of each plane has to be aligned to 32 bytes. The pixels inside each
> macroblock are coded as they would be if the macroblock was a single
> plane, line after line.
> 
> The MB32-tiled NV12 format is used by the video engine on Allwinner
> platforms: it is the default format for decoded frames (and the only one
> available in the oldest supported platforms).
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> ---
>  include/uapi/linux/videodev2.h | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 4b8336f7bcf0..43993a116e2b 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -669,6 +669,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_Z16      v4l2_fourcc('Z', '1', '6', ' ') /* Depth data 16-bit */
>  #define V4L2_PIX_FMT_MT21C    v4l2_fourcc('M', 'T', '2', '1') /* Mediatek compressed block mode  */
>  #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
> +#define V4L2_PIX_FMT_MB32_NV12 v4l2_fourcc('M', 'N', '1', '2') /* Allwinner NV12 in 32x32 macroblocks */
>  
>  /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
>  #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
> 

Add an entry for this to v4l_fill_fmtdesc() in v4l2-ioctl.c.

It also needs to be documented in the spec.

Regards,

	Hans

[PATCH v2 06/10] media: v4l: Add definition for Allwinner's MB32-tiled NV12 format

[Hans Verkuil]

On 04/19/18 17:45, Paul Kocialkowski wrote:
> This introduces support for Allwinner's MB32-tiled NV12 format, where
> each plane is divided into macroblocks of 32x32 pixels. Hence, the size
> of each plane has to be aligned to 32 bytes. The pixels inside each
> macroblock are coded as they would be if the macroblock was a single
> plane, line after line.
> 
> The MB32-tiled NV12 format is used by the video engine on Allwinner
> platforms: it is the default format for decoded frames (and the only one
> available in the oldest supported platforms).
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  include/uapi/linux/videodev2.h | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 4b8336f7bcf0..43993a116e2b 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -669,6 +669,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_Z16      v4l2_fourcc('Z', '1', '6', ' ') /* Depth data 16-bit */
>  #define V4L2_PIX_FMT_MT21C    v4l2_fourcc('M', 'T', '2', '1') /* Mediatek compressed block mode  */
>  #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
> +#define V4L2_PIX_FMT_MB32_NV12 v4l2_fourcc('M', 'N', '1', '2') /* Allwinner NV12 in 32x32 macroblocks */
>  
>  /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
>  #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
> 

Add an entry for this to v4l_fill_fmtdesc() in v4l2-ioctl.c.

It also needs to be documented in the spec.

Regards,

	Hans

Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Hans Verkuil]

On 04/19/18 17:45, Paul Kocialkowski wrote:
> Stateless video decoding engines require both the MPEG slices and
> associated metadata from the video stream in order to decode frames.
> 
> This introduces definitions for a new pixel format, describing buffers
> with MPEG2 slice data, as well as a control structure for passing the
> frame header (metadata) to drivers.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> Signed-off-by: Florent Revest <florent.revest@free-electrons.com>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
>  include/uapi/linux/v4l2-controls.h   | 26 ++++++++++++++++++++++++++
>  include/uapi/linux/videodev2.h       |  3 +++
>  4 files changed, 40 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index ba05a8b9a095..fcdc12b9a9e0 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		return "Vertical MV Search Range";
>  	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		return "Repeat Sequence Header";
>  	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		return "Force Key Frame";
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		return "MPEG2 Frame Header";
>  
>  	/* VPX controls */
>  	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		return "VPX Number of Partitions";
> @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>  	case V4L2_CID_RDS_TX_ALT_FREQS:
>  		*type = V4L2_CTRL_TYPE_U32;
>  		break;
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> +		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> +		break;
>  	default:
>  		*type = V4L2_CTRL_TYPE_INTEGER;
>  		break;
> @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
>  			return -ERANGE;
>  		return 0;
>  
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		return 0;
> +
>  	default:
>  		return -EINVAL;
>  	}
> @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
>  	case V4L2_CTRL_TYPE_U32:
>  		elem_size = sizeof(u32);
>  		break;
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
> +		break;
>  	default:
>  		if (type < V4L2_CTRL_COMPOUND_TYPES)
>  			elem_size = sizeof(s32);
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 468c3c65362d..8070203da5d2 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1 (SMPTE 412M Annex L)"; break;
>  		case V4L2_PIX_FMT_VP8:		descr = "VP8"; break;
>  		case V4L2_PIX_FMT_VP9:		descr = "VP9"; break;
> +		case V4L2_PIX_FMT_MPEG2_FRAME:	descr = "MPEG2 Frame"; break;
>  		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA YUV"; break;
>  		case V4L2_PIX_FMT_WNVA:		descr = "WNVA"; break;
>  		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA SN9C10X"; break;
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index cbbb750d87d1..8431b2a540c7 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
>  };
>  #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPEG_BASE+407)
>  
> +#define V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
> +
>  /*  Control IDs for VP8 streams
>   *  Although VP8 is not part of MPEG we add these controls to the MPEG class
>   *  as that class is already handling other video compression standards
> @@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
>  #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_CLASS_BASE + 3)
>  #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DETECT_CLASS_BASE + 4)
>  
> +struct v4l2_ctrl_mpeg2_frame_hdr {
> +	__u32 slice_len;
> +	__u32 slice_pos;
> +	enum { MPEG1, MPEG2 } type;
> +
> +	__u16 width;
> +	__u16 height;
> +
> +	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;
> +	__u8 f_code[2][2];
> +
> +	__u8 intra_dc_precision;
> +	__u8 picture_structure;
> +	__u8 top_field_first;
> +	__u8 frame_pred_frame_dct;
> +	__u8 concealment_motion_vectors;
> +	__u8 q_scale_type;
> +	__u8 intra_vlc_format;
> +	__u8 alternate_scan;
> +
> +	__u8 backward_ref_index;
> +	__u8 forward_ref_index;
> +};
> +
>  #endif
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 31b5728b56e9..4b8336f7bcf0 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -635,6 +635,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /* SMPTE 421M Annex L compliant stream */
>  #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
>  #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
> +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /* MPEG2 frame */

Needs to be added to v4l_fill_fmtdesc in v4l2-ioctl.c.

Regards,

	Hans

>  
>  /*  Vendor-specific formats   */
>  #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
> @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
>  		__u8 __user *p_u8;
>  		__u16 __user *p_u16;
>  		__u32 __user *p_u32;
> +		struct v4l2_ctrl_mpeg2_frame_hdr __user *p_mpeg2_frame_hdr;
>  		void __user *ptr;
>  	};
>  } __attribute__ ((packed));
> @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
>  	V4L2_CTRL_TYPE_U8	     = 0x0100,
>  	V4L2_CTRL_TYPE_U16	     = 0x0101,
>  	V4L2_CTRL_TYPE_U32	     = 0x0102,
> +	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
>  };
>  
>  /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
> 

Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Hans Verkuil]

On 04/19/18 17:45, Paul Kocialkowski wrote:
> Stateless video decoding engines require both the MPEG slices and
> associated metadata from the video stream in order to decode frames.
> 
> This introduces definitions for a new pixel format, describing buffers
> with MPEG2 slice data, as well as a control structure for passing the
> frame header (metadata) to drivers.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> Signed-off-by: Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
>  include/uapi/linux/v4l2-controls.h   | 26 ++++++++++++++++++++++++++
>  include/uapi/linux/videodev2.h       |  3 +++
>  4 files changed, 40 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index ba05a8b9a095..fcdc12b9a9e0 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		return "Vertical MV Search Range";
>  	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		return "Repeat Sequence Header";
>  	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		return "Force Key Frame";
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		return "MPEG2 Frame Header";
>  
>  	/* VPX controls */
>  	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		return "VPX Number of Partitions";
> @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>  	case V4L2_CID_RDS_TX_ALT_FREQS:
>  		*type = V4L2_CTRL_TYPE_U32;
>  		break;
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> +		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> +		break;
>  	default:
>  		*type = V4L2_CTRL_TYPE_INTEGER;
>  		break;
> @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
>  			return -ERANGE;
>  		return 0;
>  
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		return 0;
> +
>  	default:
>  		return -EINVAL;
>  	}
> @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
>  	case V4L2_CTRL_TYPE_U32:
>  		elem_size = sizeof(u32);
>  		break;
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
> +		break;
>  	default:
>  		if (type < V4L2_CTRL_COMPOUND_TYPES)
>  			elem_size = sizeof(s32);
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 468c3c65362d..8070203da5d2 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1 (SMPTE 412M Annex L)"; break;
>  		case V4L2_PIX_FMT_VP8:		descr = "VP8"; break;
>  		case V4L2_PIX_FMT_VP9:		descr = "VP9"; break;
> +		case V4L2_PIX_FMT_MPEG2_FRAME:	descr = "MPEG2 Frame"; break;
>  		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA YUV"; break;
>  		case V4L2_PIX_FMT_WNVA:		descr = "WNVA"; break;
>  		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA SN9C10X"; break;
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index cbbb750d87d1..8431b2a540c7 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
>  };
>  #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPEG_BASE+407)
>  
> +#define V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
> +
>  /*  Control IDs for VP8 streams
>   *  Although VP8 is not part of MPEG we add these controls to the MPEG class
>   *  as that class is already handling other video compression standards
> @@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
>  #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_CLASS_BASE + 3)
>  #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DETECT_CLASS_BASE + 4)
>  
> +struct v4l2_ctrl_mpeg2_frame_hdr {
> +	__u32 slice_len;
> +	__u32 slice_pos;
> +	enum { MPEG1, MPEG2 } type;
> +
> +	__u16 width;
> +	__u16 height;
> +
> +	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;
> +	__u8 f_code[2][2];
> +
> +	__u8 intra_dc_precision;
> +	__u8 picture_structure;
> +	__u8 top_field_first;
> +	__u8 frame_pred_frame_dct;
> +	__u8 concealment_motion_vectors;
> +	__u8 q_scale_type;
> +	__u8 intra_vlc_format;
> +	__u8 alternate_scan;
> +
> +	__u8 backward_ref_index;
> +	__u8 forward_ref_index;
> +};
> +
>  #endif
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 31b5728b56e9..4b8336f7bcf0 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -635,6 +635,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /* SMPTE 421M Annex L compliant stream */
>  #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
>  #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
> +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /* MPEG2 frame */

Needs to be added to v4l_fill_fmtdesc in v4l2-ioctl.c.

Regards,

	Hans

>  
>  /*  Vendor-specific formats   */
>  #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
> @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
>  		__u8 __user *p_u8;
>  		__u16 __user *p_u16;
>  		__u32 __user *p_u32;
> +		struct v4l2_ctrl_mpeg2_frame_hdr __user *p_mpeg2_frame_hdr;
>  		void __user *ptr;
>  	};
>  } __attribute__ ((packed));
> @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
>  	V4L2_CTRL_TYPE_U8	     = 0x0100,
>  	V4L2_CTRL_TYPE_U16	     = 0x0101,
>  	V4L2_CTRL_TYPE_U32	     = 0x0102,
> +	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
>  };
>  
>  /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
> 

[PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Hans Verkuil]

On 04/19/18 17:45, Paul Kocialkowski wrote:
> Stateless video decoding engines require both the MPEG slices and
> associated metadata from the video stream in order to decode frames.
> 
> This introduces definitions for a new pixel format, describing buffers
> with MPEG2 slice data, as well as a control structure for passing the
> frame header (metadata) to drivers.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> Signed-off-by: Florent Revest <florent.revest@free-electrons.com>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
>  include/uapi/linux/v4l2-controls.h   | 26 ++++++++++++++++++++++++++
>  include/uapi/linux/videodev2.h       |  3 +++
>  4 files changed, 40 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index ba05a8b9a095..fcdc12b9a9e0 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		return "Vertical MV Search Range";
>  	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		return "Repeat Sequence Header";
>  	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		return "Force Key Frame";
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		return "MPEG2 Frame Header";
>  
>  	/* VPX controls */
>  	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		return "VPX Number of Partitions";
> @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>  	case V4L2_CID_RDS_TX_ALT_FREQS:
>  		*type = V4L2_CTRL_TYPE_U32;
>  		break;
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> +		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> +		break;
>  	default:
>  		*type = V4L2_CTRL_TYPE_INTEGER;
>  		break;
> @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
>  			return -ERANGE;
>  		return 0;
>  
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		return 0;
> +
>  	default:
>  		return -EINVAL;
>  	}
> @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
>  	case V4L2_CTRL_TYPE_U32:
>  		elem_size = sizeof(u32);
>  		break;
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
> +		break;
>  	default:
>  		if (type < V4L2_CTRL_COMPOUND_TYPES)
>  			elem_size = sizeof(s32);
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 468c3c65362d..8070203da5d2 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1 (SMPTE 412M Annex L)"; break;
>  		case V4L2_PIX_FMT_VP8:		descr = "VP8"; break;
>  		case V4L2_PIX_FMT_VP9:		descr = "VP9"; break;
> +		case V4L2_PIX_FMT_MPEG2_FRAME:	descr = "MPEG2 Frame"; break;
>  		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA YUV"; break;
>  		case V4L2_PIX_FMT_WNVA:		descr = "WNVA"; break;
>  		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA SN9C10X"; break;
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index cbbb750d87d1..8431b2a540c7 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
>  };
>  #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPEG_BASE+407)
>  
> +#define V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
> +
>  /*  Control IDs for VP8 streams
>   *  Although VP8 is not part of MPEG we add these controls to the MPEG class
>   *  as that class is already handling other video compression standards
> @@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
>  #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_CLASS_BASE + 3)
>  #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DETECT_CLASS_BASE + 4)
>  
> +struct v4l2_ctrl_mpeg2_frame_hdr {
> +	__u32 slice_len;
> +	__u32 slice_pos;
> +	enum { MPEG1, MPEG2 } type;
> +
> +	__u16 width;
> +	__u16 height;
> +
> +	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;
> +	__u8 f_code[2][2];
> +
> +	__u8 intra_dc_precision;
> +	__u8 picture_structure;
> +	__u8 top_field_first;
> +	__u8 frame_pred_frame_dct;
> +	__u8 concealment_motion_vectors;
> +	__u8 q_scale_type;
> +	__u8 intra_vlc_format;
> +	__u8 alternate_scan;
> +
> +	__u8 backward_ref_index;
> +	__u8 forward_ref_index;
> +};
> +
>  #endif
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 31b5728b56e9..4b8336f7bcf0 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -635,6 +635,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /* SMPTE 421M Annex L compliant stream */
>  #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
>  #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
> +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /* MPEG2 frame */

Needs to be added to v4l_fill_fmtdesc in v4l2-ioctl.c.

Regards,

	Hans

>  
>  /*  Vendor-specific formats   */
>  #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
> @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
>  		__u8 __user *p_u8;
>  		__u16 __user *p_u16;
>  		__u32 __user *p_u32;
> +		struct v4l2_ctrl_mpeg2_frame_hdr __user *p_mpeg2_frame_hdr;
>  		void __user *ptr;
>  	};
>  } __attribute__ ((packed));
> @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
>  	V4L2_CTRL_TYPE_U8	     = 0x0100,
>  	V4L2_CTRL_TYPE_U16	     = 0x0101,
>  	V4L2_CTRL_TYPE_U32	     = 0x0102,
> +	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
>  };
>  
>  /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
> 

Re: [PATCH v2 02/10] media-request: Add a request complete operation to allow m2m scheduling

[Sakari Ailus]

Hi Paul,

On Thu, Apr 19, 2018 at 05:41:16PM +0200, Paul Kocialkowski wrote:
> When using the request API in the context of a m2m driver, the
> operations that come with a m2m run scheduling call in their
> (m2m-specific) ioctl handler are delayed until the request is queued
> (for instance, this includes queuing buffers and streamon).
> 
> Thus, the m2m run scheduling calls are not called in due time since the
> request AP's internal plumbing will (rightfully) use the relevant core
> functions directly instead of the ioctl handler.
> 
> This ends up in a situation where nothing happens if there is no
> run-scheduling ioctl called after queuing the request.
> 
> In order to circumvent the issue, a new media operation is introduced,
> called at the time of handling the media request queue ioctl. It gives
> m2m drivers a chance to schedule a m2m device run at that time.
> 
> The existing req_queue operation cannot be used for this purpose, since
> it is called with the request queue mutex held, that is eventually needed
> in the device_run call to apply relevant controls.

Based on the description I'm not entirely sure what's the intent here.

What would be the purpose for acquiring the request queue mutex during
the execution of the request? It is only intended for serialising access to
the request objects while the request is being validated and queued.

If a driver does a significant amount of processing that is not related to
managing the request as such, then one option could be to add a work queue
for it.

> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  drivers/media/media-request.c | 3 +++
>  include/media/media-device.h  | 2 ++
>  2 files changed, 5 insertions(+)
> 
> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> index 415f7e31019d..28ac5ccfe6a2 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -157,6 +157,9 @@ static long media_request_ioctl_queue(struct media_request *req)
>  		media_request_get(req);
>  	}
>  
> +	if (mdev->ops->req_complete)
> +		mdev->ops->req_complete(req);
> +
>  	return ret;
>  }
>  
> diff --git a/include/media/media-device.h b/include/media/media-device.h
> index 07e323c57202..c7dcf2079cc9 100644
> --- a/include/media/media-device.h
> +++ b/include/media/media-device.h
> @@ -55,6 +55,7 @@ struct media_entity_notify {
>   * @req_alloc: Allocate a request
>   * @req_free: Free a request
>   * @req_queue: Queue a request
> + * @req_complete: Complete a request
>   */
>  struct media_device_ops {
>  	int (*link_notify)(struct media_link *link, u32 flags,
> @@ -62,6 +63,7 @@ struct media_device_ops {
>  	struct media_request *(*req_alloc)(struct media_device *mdev);
>  	void (*req_free)(struct media_request *req);
>  	int (*req_queue)(struct media_request *req);
> +	void (*req_complete)(struct media_request *req);
>  };
>  
>  /**
> -- 
> 2.16.3
> 

-- 
Sakari Ailus
sakari.ailus@linux.intel.com

[PATCH v2 02/10] media-request: Add a request complete operation to allow m2m scheduling

[Sakari Ailus]

Hi Paul,

On Thu, Apr 19, 2018 at 05:41:16PM +0200, Paul Kocialkowski wrote:
> When using the request API in the context of a m2m driver, the
> operations that come with a m2m run scheduling call in their
> (m2m-specific) ioctl handler are delayed until the request is queued
> (for instance, this includes queuing buffers and streamon).
> 
> Thus, the m2m run scheduling calls are not called in due time since the
> request AP's internal plumbing will (rightfully) use the relevant core
> functions directly instead of the ioctl handler.
> 
> This ends up in a situation where nothing happens if there is no
> run-scheduling ioctl called after queuing the request.
> 
> In order to circumvent the issue, a new media operation is introduced,
> called at the time of handling the media request queue ioctl. It gives
> m2m drivers a chance to schedule a m2m device run at that time.
> 
> The existing req_queue operation cannot be used for this purpose, since
> it is called with the request queue mutex held, that is eventually needed
> in the device_run call to apply relevant controls.

Based on the description I'm not entirely sure what's the intent here.

What would be the purpose for acquiring the request queue mutex during
the execution of the request? It is only intended for serialising access to
the request objects while the request is being validated and queued.

If a driver does a significant amount of processing that is not related to
managing the request as such, then one option could be to add a work queue
for it.

> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  drivers/media/media-request.c | 3 +++
>  include/media/media-device.h  | 2 ++
>  2 files changed, 5 insertions(+)
> 
> diff --git a/drivers/media/media-request.c b/drivers/media/media-request.c
> index 415f7e31019d..28ac5ccfe6a2 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -157,6 +157,9 @@ static long media_request_ioctl_queue(struct media_request *req)
>  		media_request_get(req);
>  	}
>  
> +	if (mdev->ops->req_complete)
> +		mdev->ops->req_complete(req);
> +
>  	return ret;
>  }
>  
> diff --git a/include/media/media-device.h b/include/media/media-device.h
> index 07e323c57202..c7dcf2079cc9 100644
> --- a/include/media/media-device.h
> +++ b/include/media/media-device.h
> @@ -55,6 +55,7 @@ struct media_entity_notify {
>   * @req_alloc: Allocate a request
>   * @req_free: Free a request
>   * @req_queue: Queue a request
> + * @req_complete: Complete a request
>   */
>  struct media_device_ops {
>  	int (*link_notify)(struct media_link *link, u32 flags,
> @@ -62,6 +63,7 @@ struct media_device_ops {
>  	struct media_request *(*req_alloc)(struct media_device *mdev);
>  	void (*req_free)(struct media_request *req);
>  	int (*req_queue)(struct media_request *req);
> +	void (*req_complete)(struct media_request *req);
>  };
>  
>  /**
> -- 
> 2.16.3
> 

-- 
Sakari Ailus
sakari.ailus at linux.intel.com

Re: [PATCH v2 03/10] videobuf2-core: Add helper to get buffer private data from media request

[Sakari Ailus]

Hi Paul,

On Thu, Apr 19, 2018 at 05:41:17PM +0200, Paul Kocialkowski wrote:
> When calling media operation driver callbacks related to media requests,
> only a pointer to the request itself is provided, which is insufficient
> to retrieve the driver's context. Since the driver context is usually
> set as vb2 queue private data and given that the core can determine
> which objects attached to the request are buffers, it is possible to
> extract the associated private data for the first buffer found.
> 
> This is required in order to access the current m2m context from m2m
> drivers' private data in the context of media request operation
> callbacks. More specifically, this allows scheduling m2m device runs
> from the newly-introduced request complete operation.

Rather than fetching a random buffer from the request objects, I'd suggest
to allocate a struct that contains the media request and place the required
information there.

Such as was in my older patchset. The patch won't apply but the approach
would be useful I think.

<URL:https://git.linuxtv.org/sailus/media_tree.git/commit/?h=request&id=f5f6e5925223b445e49279b9fdf070976fd844b9>

> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  drivers/media/common/videobuf2/videobuf2-core.c | 15 +++++++++++++++
>  include/media/videobuf2-core.h                  |  1 +
>  2 files changed, 16 insertions(+)
> 
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index 13c9d9e243dd..6fa46bfc620f 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -1351,6 +1351,21 @@ bool vb2_core_request_has_buffers(struct media_request *req)
>  }
>  EXPORT_SYMBOL_GPL(vb2_core_request_has_buffers);
>  
> +void *vb2_core_request_find_buffer_priv(struct media_request *req)
> +{
> +	struct media_request_object *obj;
> +	struct vb2_buffer *vb;
> +
> +	obj = media_request_object_find(req, &vb2_core_req_ops, NULL);
> +	if (!obj)
> +		return NULL;
> +
> +	vb = container_of(obj, struct vb2_buffer, req_obj);
> +
> +	return vb2_get_drv_priv(vb->vb2_queue);
> +}
> +EXPORT_SYMBOL_GPL(vb2_core_request_find_buffer_priv);
> +
>  int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb,
>  			 struct media_request *req)
>  {
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index 032bd1bec555..65c0cf6afb55 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -1153,4 +1153,5 @@ int vb2_verify_memory_type(struct vb2_queue *q,
>  		enum vb2_memory memory, unsigned int type);
>  
>  bool vb2_core_request_has_buffers(struct media_request *req);
> +void *vb2_core_request_find_buffer_priv(struct media_request *req);
>  #endif /* _MEDIA_VIDEOBUF2_CORE_H */
> -- 
> 2.16.3
> 

-- 
Sakari Ailus
sakari.ailus@linux.intel.com

[PATCH v2 03/10] videobuf2-core: Add helper to get buffer private data from media request

[Sakari Ailus]

Hi Paul,

On Thu, Apr 19, 2018 at 05:41:17PM +0200, Paul Kocialkowski wrote:
> When calling media operation driver callbacks related to media requests,
> only a pointer to the request itself is provided, which is insufficient
> to retrieve the driver's context. Since the driver context is usually
> set as vb2 queue private data and given that the core can determine
> which objects attached to the request are buffers, it is possible to
> extract the associated private data for the first buffer found.
> 
> This is required in order to access the current m2m context from m2m
> drivers' private data in the context of media request operation
> callbacks. More specifically, this allows scheduling m2m device runs
> from the newly-introduced request complete operation.

Rather than fetching a random buffer from the request objects, I'd suggest
to allocate a struct that contains the media request and place the required
information there.

Such as was in my older patchset. The patch won't apply but the approach
would be useful I think.

<URL:https://git.linuxtv.org/sailus/media_tree.git/commit/?h=request&id=f5f6e5925223b445e49279b9fdf070976fd844b9>

> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  drivers/media/common/videobuf2/videobuf2-core.c | 15 +++++++++++++++
>  include/media/videobuf2-core.h                  |  1 +
>  2 files changed, 16 insertions(+)
> 
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index 13c9d9e243dd..6fa46bfc620f 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -1351,6 +1351,21 @@ bool vb2_core_request_has_buffers(struct media_request *req)
>  }
>  EXPORT_SYMBOL_GPL(vb2_core_request_has_buffers);
>  
> +void *vb2_core_request_find_buffer_priv(struct media_request *req)
> +{
> +	struct media_request_object *obj;
> +	struct vb2_buffer *vb;
> +
> +	obj = media_request_object_find(req, &vb2_core_req_ops, NULL);
> +	if (!obj)
> +		return NULL;
> +
> +	vb = container_of(obj, struct vb2_buffer, req_obj);
> +
> +	return vb2_get_drv_priv(vb->vb2_queue);
> +}
> +EXPORT_SYMBOL_GPL(vb2_core_request_find_buffer_priv);
> +
>  int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index, void *pb,
>  			 struct media_request *req)
>  {
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index 032bd1bec555..65c0cf6afb55 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -1153,4 +1153,5 @@ int vb2_verify_memory_type(struct vb2_queue *q,
>  		enum vb2_memory memory, unsigned int type);
>  
>  bool vb2_core_request_has_buffers(struct media_request *req);
> +void *vb2_core_request_find_buffer_priv(struct media_request *req);
>  #endif /* _MEDIA_VIDEOBUF2_CORE_H */
> -- 
> 2.16.3
> 

-- 
Sakari Ailus
sakari.ailus at linux.intel.com

Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Sakari Ailus]

Hi Paul,

On Thu, Apr 19, 2018 at 05:45:31PM +0200, Paul Kocialkowski wrote:
> Stateless video decoding engines require both the MPEG slices and
> associated metadata from the video stream in order to decode frames.
> 
> This introduces definitions for a new pixel format, describing buffers
> with MPEG2 slice data, as well as a control structure for passing the
> frame header (metadata) to drivers.

What's the typical relationship between MPEG2 slice data and the header?

Are the two always used together, do they originate from the same source,
for instance? I have to admit I'm not very familiar with MPEG...

> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> Signed-off-by: Florent Revest <florent.revest@free-electrons.com>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
>  include/uapi/linux/v4l2-controls.h   | 26 ++++++++++++++++++++++++++
>  include/uapi/linux/videodev2.h       |  3 +++
>  4 files changed, 40 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index ba05a8b9a095..fcdc12b9a9e0 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		return "Vertical MV Search Range";
>  	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		return "Repeat Sequence Header";
>  	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		return "Force Key Frame";
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		return "MPEG2 Frame Header";
>  
>  	/* VPX controls */
>  	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		return "VPX Number of Partitions";
> @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>  	case V4L2_CID_RDS_TX_ALT_FREQS:
>  		*type = V4L2_CTRL_TYPE_U32;
>  		break;
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> +		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> +		break;
>  	default:
>  		*type = V4L2_CTRL_TYPE_INTEGER;
>  		break;
> @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
>  			return -ERANGE;
>  		return 0;
>  
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		return 0;
> +
>  	default:
>  		return -EINVAL;
>  	}
> @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
>  	case V4L2_CTRL_TYPE_U32:
>  		elem_size = sizeof(u32);
>  		break;
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
> +		break;
>  	default:
>  		if (type < V4L2_CTRL_COMPOUND_TYPES)
>  			elem_size = sizeof(s32);
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 468c3c65362d..8070203da5d2 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1 (SMPTE 412M Annex L)"; break;
>  		case V4L2_PIX_FMT_VP8:		descr = "VP8"; break;
>  		case V4L2_PIX_FMT_VP9:		descr = "VP9"; break;
> +		case V4L2_PIX_FMT_MPEG2_FRAME:	descr = "MPEG2 Frame"; break;
>  		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA YUV"; break;
>  		case V4L2_PIX_FMT_WNVA:		descr = "WNVA"; break;
>  		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA SN9C10X"; break;
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index cbbb750d87d1..8431b2a540c7 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
>  };
>  #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPEG_BASE+407)
>  
> +#define V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
> +
>  /*  Control IDs for VP8 streams
>   *  Although VP8 is not part of MPEG we add these controls to the MPEG class
>   *  as that class is already handling other video compression standards
> @@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
>  #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_CLASS_BASE + 3)
>  #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DETECT_CLASS_BASE + 4)
>  
> +struct v4l2_ctrl_mpeg2_frame_hdr {
> +	__u32 slice_len;
> +	__u32 slice_pos;
> +	enum { MPEG1, MPEG2 } type;
> +
> +	__u16 width;
> +	__u16 height;
> +
> +	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;
> +	__u8 f_code[2][2];
> +
> +	__u8 intra_dc_precision;
> +	__u8 picture_structure;
> +	__u8 top_field_first;
> +	__u8 frame_pred_frame_dct;
> +	__u8 concealment_motion_vectors;
> +	__u8 q_scale_type;
> +	__u8 intra_vlc_format;
> +	__u8 alternate_scan;
> +
> +	__u8 backward_ref_index;
> +	__u8 forward_ref_index;
> +};

Besides the comments that have been given, please add
__attribute__((packed)) to the struct definition.

> +
>  #endif
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 31b5728b56e9..4b8336f7bcf0 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -635,6 +635,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /* SMPTE 421M Annex L compliant stream */
>  #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
>  #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
> +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /* MPEG2 frame */

This format needs documentation.

>  
>  /*  Vendor-specific formats   */
>  #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
> @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
>  		__u8 __user *p_u8;
>  		__u16 __user *p_u16;
>  		__u32 __user *p_u32;
> +		struct v4l2_ctrl_mpeg2_frame_hdr __user *p_mpeg2_frame_hdr;
>  		void __user *ptr;
>  	};
>  } __attribute__ ((packed));
> @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
>  	V4L2_CTRL_TYPE_U8	     = 0x0100,
>  	V4L2_CTRL_TYPE_U16	     = 0x0101,
>  	V4L2_CTRL_TYPE_U32	     = 0x0102,
> +	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
>  };
>  
>  /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
> -- 
> 2.16.3
> 

-- 
Sakari Ailus
sakari.ailus@linux.intel.com

[PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Sakari Ailus]

Hi Paul,

On Thu, Apr 19, 2018 at 05:45:31PM +0200, Paul Kocialkowski wrote:
> Stateless video decoding engines require both the MPEG slices and
> associated metadata from the video stream in order to decode frames.
> 
> This introduces definitions for a new pixel format, describing buffers
> with MPEG2 slice data, as well as a control structure for passing the
> frame header (metadata) to drivers.

What's the typical relationship between MPEG2 slice data and the header?

Are the two always used together, do they originate from the same source,
for instance? I have to admit I'm not very familiar with MPEG...

> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> Signed-off-by: Florent Revest <florent.revest@free-electrons.com>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
>  include/uapi/linux/v4l2-controls.h   | 26 ++++++++++++++++++++++++++
>  include/uapi/linux/videodev2.h       |  3 +++
>  4 files changed, 40 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c
> index ba05a8b9a095..fcdc12b9a9e0 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		return "Vertical MV Search Range";
>  	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		return "Repeat Sequence Header";
>  	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		return "Force Key Frame";
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		return "MPEG2 Frame Header";
>  
>  	/* VPX controls */
>  	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		return "VPX Number of Partitions";
> @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
>  	case V4L2_CID_RDS_TX_ALT_FREQS:
>  		*type = V4L2_CTRL_TYPE_U32;
>  		break;
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> +		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> +		break;
>  	default:
>  		*type = V4L2_CTRL_TYPE_INTEGER;
>  		break;
> @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
>  			return -ERANGE;
>  		return 0;
>  
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		return 0;
> +
>  	default:
>  		return -EINVAL;
>  	}
> @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
>  	case V4L2_CTRL_TYPE_U32:
>  		elem_size = sizeof(u32);
>  		break;
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
> +		break;
>  	default:
>  		if (type < V4L2_CTRL_COMPOUND_TYPES)
>  			elem_size = sizeof(s32);
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 468c3c65362d..8070203da5d2 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1 (SMPTE 412M Annex L)"; break;
>  		case V4L2_PIX_FMT_VP8:		descr = "VP8"; break;
>  		case V4L2_PIX_FMT_VP9:		descr = "VP9"; break;
> +		case V4L2_PIX_FMT_MPEG2_FRAME:	descr = "MPEG2 Frame"; break;
>  		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA YUV"; break;
>  		case V4L2_PIX_FMT_WNVA:		descr = "WNVA"; break;
>  		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA SN9C10X"; break;
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index cbbb750d87d1..8431b2a540c7 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
>  };
>  #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPEG_BASE+407)
>  
> +#define V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
> +
>  /*  Control IDs for VP8 streams
>   *  Although VP8 is not part of MPEG we add these controls to the MPEG class
>   *  as that class is already handling other video compression standards
> @@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
>  #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_CLASS_BASE + 3)
>  #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DETECT_CLASS_BASE + 4)
>  
> +struct v4l2_ctrl_mpeg2_frame_hdr {
> +	__u32 slice_len;
> +	__u32 slice_pos;
> +	enum { MPEG1, MPEG2 } type;
> +
> +	__u16 width;
> +	__u16 height;
> +
> +	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;
> +	__u8 f_code[2][2];
> +
> +	__u8 intra_dc_precision;
> +	__u8 picture_structure;
> +	__u8 top_field_first;
> +	__u8 frame_pred_frame_dct;
> +	__u8 concealment_motion_vectors;
> +	__u8 q_scale_type;
> +	__u8 intra_vlc_format;
> +	__u8 alternate_scan;
> +
> +	__u8 backward_ref_index;
> +	__u8 forward_ref_index;
> +};

Besides the comments that have been given, please add
__attribute__((packed)) to the struct definition.

> +
>  #endif
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 31b5728b56e9..4b8336f7bcf0 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -635,6 +635,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /* SMPTE 421M Annex L compliant stream */
>  #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8 */
>  #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /* VP9 */
> +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /* MPEG2 frame */

This format needs documentation.

>  
>  /*  Vendor-specific formats   */
>  #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */
> @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
>  		__u8 __user *p_u8;
>  		__u16 __user *p_u16;
>  		__u32 __user *p_u32;
> +		struct v4l2_ctrl_mpeg2_frame_hdr __user *p_mpeg2_frame_hdr;
>  		void __user *ptr;
>  	};
>  } __attribute__ ((packed));
> @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
>  	V4L2_CTRL_TYPE_U8	     = 0x0100,
>  	V4L2_CTRL_TYPE_U16	     = 0x0101,
>  	V4L2_CTRL_TYPE_U32	     = 0x0102,
> +	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
>  };
>  
>  /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
> -- 
> 2.16.3
> 

-- 
Sakari Ailus
sakari.ailus at linux.intel.com

Re: [PATCH v2 07/10] media: platform: Add Sunxi-Cedrus VPU decoder driver

[Sakari Ailus]

Hi Paul

On Thu, Apr 19, 2018 at 05:45:33PM +0200, Paul Kocialkowski wrote:
> This introduces the Sunxi-Cedrus VPU driver that supports the VPU found
> in Allwinner SoCs, also known as Video Engine. It is implemented through
> a v4l2 m2m decoder device and a media device (used for media requests).
> So far, it only supports MPEG2 decoding.
> 
> Since this VPU is stateless, synchronization with media requests is
> required in order to ensure consistency between frame headers that
> contain metadata about the frame to process and the raw slice data that
> is used to generate the frame.
> 
> This driver was made possible thanks to the long-standing effort
> carried out by the linux-sunxi community in the interest of reverse
> engineering, documenting and implementing support for Allwinner VPU.

No code review yet, but DT bindings precede the driver. Please also add
the appropriate MAINTAINERS entries.

-- 
Sakari Ailus
sakari.ailus@linux.intel.com

[PATCH v2 07/10] media: platform: Add Sunxi-Cedrus VPU decoder driver

[Sakari Ailus]

Hi Paul

On Thu, Apr 19, 2018 at 05:45:33PM +0200, Paul Kocialkowski wrote:
> This introduces the Sunxi-Cedrus VPU driver that supports the VPU found
> in Allwinner SoCs, also known as Video Engine. It is implemented through
> a v4l2 m2m decoder device and a media device (used for media requests).
> So far, it only supports MPEG2 decoding.
> 
> Since this VPU is stateless, synchronization with media requests is
> required in order to ensure consistency between frame headers that
> contain metadata about the frame to process and the raw slice data that
> is used to generate the frame.
> 
> This driver was made possible thanks to the long-standing effort
> carried out by the linux-sunxi community in the interest of reverse
> engineering, documenting and implementing support for Allwinner VPU.

No code review yet, but DT bindings precede the driver. Please also add
the appropriate MAINTAINERS entries.

-- 
Sakari Ailus
sakari.ailus at linux.intel.com

Re: [PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Rob Herring]

On Fri, Apr 20, 2018 at 09:22:20AM +0200, Paul Kocialkowski wrote:
> Hi and thanks for the review,
> 
> On Fri, 2018-04-20 at 01:31 +0000, Tomasz Figa wrote:
> > Hi Paul, Philipp,
> > 
> > On Fri, Apr 20, 2018 at 1:04 AM Philipp Zabel <p.zabel@pengutronix.de>
> > wrote:
> > 
> > > Hi Paul,
> > > On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> > > > This adds a device-tree binding document that specifies the
> > > > properties
> > > > used by the Sunxi-Cedurs VPU driver, as well as examples.
> > > > 
> > > > Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> > > > ---
> > > >  .../devicetree/bindings/media/sunxi-cedrus.txt     | 50
> > 
> > ++++++++++++++++++++++
> > > >  1 file changed, 50 insertions(+)
> > > >  create mode 100644
> > 
> > Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > 
> > > > diff --git a/Documentation/devicetree/bindings/media/sunxi-
> > > > cedrus.txt
> > 
> > b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > new file mode 100644
> > > > index 000000000000..71ad3f9c3352
> > > > --- /dev/null
> > > > +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > @@ -0,0 +1,50 @@
> > > > +Device-tree bindings for the VPU found in Allwinner SoCs,
> > > > referred to
> > 
> > as the
> > > > +Video Engine (VE) in Allwinner literature.
> > > > +
> > > > +The VPU can only access the first 256 MiB of DRAM, that are DMA-
> > > > mapped
> > 
> > starting
> > > > +from the DRAM base. This requires specific memory allocation and
> > 
> > handling.
> > 
> > And no IOMMU? Brings back memories.
> 
> Exactly, no IOMMU so we don't have much choice but cope with that
> hardware limitation...
> 
> > > > +
> > > > +Required properties:
> > > > +- compatible         : "allwinner,sun4i-a10-video-engine";
> > > > +- memory-region         : DMA pool for buffers allocation;
> > > > +- clocks             : list of clock specifiers, corresponding to
> > 
> > entries in
> > > > +                          the clock-names property;
> > > > +- clock-names                : should contain "ahb", "mod" and
> > > > "ram"
> > 
> > entries;
> > > > +- assigned-clocks       : list of clocks assigned to the VE;
> > > > +- assigned-clocks-rates : list of clock rates for the clocks
> > > > assigned
> > 
> > to the VE;
> > > > +- resets             : phandle for reset;
> > > > +- interrupts         : should contain VE interrupt number;
> > > > +- reg                        : should contain register base and
> > > > length
> > 
> > of VE.
> > > > +
> > > > +Example:
> > > > +
> > > > +reserved-memory {
> > > > +     #address-cells = <1>;
> > > > +     #size-cells = <1>;
> > > > +     ranges;
> > > > +
> > > > +     /* Address must be kept in the lower 256 MiBs of DRAM for
> > > > VE. */
> > > > +     ve_memory: cma@4a000000 {
> > > > +             compatible = "shared-dma-pool";
> > > > +             reg = <0x4a000000 0x6000000>;
> > > > +             no-map;
> > > > +             linux,cma-default;
> > > > +     };
> > > > +};
> > > > +
> > > > +video-engine@1c0e000 {
> > > This is not really required by any specification, and not as common
> > > as
> > > gpu@..., but could this reasonably be called "vpu@1c0e000" to follow
> > > somewhat-common practice?
> > 
> > AFAIR the name is supposed to be somewhat readable for someone that
> > doesn't know the hardware. To me, "video-engine" sounds more obvious
> > than "vpu", but we actually use "codec" already, in case of MFC and
> > JPEG codec on Exynos. If encode/decode is the only functionality of
> > this block, I'd personally go with "codec". If it can do other things,
> > e.g. scaling/rotation without encode/decode, I'd probably call it
> > "video-processor".
> 
> I agree that the term VPU is more commonly associated with video
> decoding, while video engine could mean a number of things.
> 
> The reason I went with "video-engine" here (while still presenting the
> driver as a VPU driver) is that Video Engine is the term used in
> Allwinner's litterature. Other nodes in Allwinner device-trees generally
> stick to these terms (for instance, we have "display-engine" nodes).
> This also makes it easier to find the matching parts in the
> documentation.

'video-codec' is what is defined in the DT spec.

Rob

Re: [PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Rob Herring]

On Fri, Apr 20, 2018 at 09:22:20AM +0200, Paul Kocialkowski wrote:
> Hi and thanks for the review,
> 
> On Fri, 2018-04-20 at 01:31 +0000, Tomasz Figa wrote:
> > Hi Paul, Philipp,
> > 
> > On Fri, Apr 20, 2018 at 1:04 AM Philipp Zabel <p.zabel-bIcnvbaLZ9MEGnE8C9+IrQ@public.gmane.org>
> > wrote:
> > 
> > > Hi Paul,
> > > On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> > > > This adds a device-tree binding document that specifies the
> > > > properties
> > > > used by the Sunxi-Cedurs VPU driver, as well as examples.
> > > > 
> > > > Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> > > > ---
> > > >  .../devicetree/bindings/media/sunxi-cedrus.txt     | 50
> > 
> > ++++++++++++++++++++++
> > > >  1 file changed, 50 insertions(+)
> > > >  create mode 100644
> > 
> > Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > 
> > > > diff --git a/Documentation/devicetree/bindings/media/sunxi-
> > > > cedrus.txt
> > 
> > b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > new file mode 100644
> > > > index 000000000000..71ad3f9c3352
> > > > --- /dev/null
> > > > +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > @@ -0,0 +1,50 @@
> > > > +Device-tree bindings for the VPU found in Allwinner SoCs,
> > > > referred to
> > 
> > as the
> > > > +Video Engine (VE) in Allwinner literature.
> > > > +
> > > > +The VPU can only access the first 256 MiB of DRAM, that are DMA-
> > > > mapped
> > 
> > starting
> > > > +from the DRAM base. This requires specific memory allocation and
> > 
> > handling.
> > 
> > And no IOMMU? Brings back memories.
> 
> Exactly, no IOMMU so we don't have much choice but cope with that
> hardware limitation...
> 
> > > > +
> > > > +Required properties:
> > > > +- compatible         : "allwinner,sun4i-a10-video-engine";
> > > > +- memory-region         : DMA pool for buffers allocation;
> > > > +- clocks             : list of clock specifiers, corresponding to
> > 
> > entries in
> > > > +                          the clock-names property;
> > > > +- clock-names                : should contain "ahb", "mod" and
> > > > "ram"
> > 
> > entries;
> > > > +- assigned-clocks       : list of clocks assigned to the VE;
> > > > +- assigned-clocks-rates : list of clock rates for the clocks
> > > > assigned
> > 
> > to the VE;
> > > > +- resets             : phandle for reset;
> > > > +- interrupts         : should contain VE interrupt number;
> > > > +- reg                        : should contain register base and
> > > > length
> > 
> > of VE.
> > > > +
> > > > +Example:
> > > > +
> > > > +reserved-memory {
> > > > +     #address-cells = <1>;
> > > > +     #size-cells = <1>;
> > > > +     ranges;
> > > > +
> > > > +     /* Address must be kept in the lower 256 MiBs of DRAM for
> > > > VE. */
> > > > +     ve_memory: cma@4a000000 {
> > > > +             compatible = "shared-dma-pool";
> > > > +             reg = <0x4a000000 0x6000000>;
> > > > +             no-map;
> > > > +             linux,cma-default;
> > > > +     };
> > > > +};
> > > > +
> > > > +video-engine@1c0e000 {
> > > This is not really required by any specification, and not as common
> > > as
> > > gpu@..., but could this reasonably be called "vpu@1c0e000" to follow
> > > somewhat-common practice?
> > 
> > AFAIR the name is supposed to be somewhat readable for someone that
> > doesn't know the hardware. To me, "video-engine" sounds more obvious
> > than "vpu", but we actually use "codec" already, in case of MFC and
> > JPEG codec on Exynos. If encode/decode is the only functionality of
> > this block, I'd personally go with "codec". If it can do other things,
> > e.g. scaling/rotation without encode/decode, I'd probably call it
> > "video-processor".
> 
> I agree that the term VPU is more commonly associated with video
> decoding, while video engine could mean a number of things.
> 
> The reason I went with "video-engine" here (while still presenting the
> driver as a VPU driver) is that Video Engine is the term used in
> Allwinner's litterature. Other nodes in Allwinner device-trees generally
> stick to these terms (for instance, we have "display-engine" nodes).
> This also makes it easier to find the matching parts in the
> documentation.

'video-codec' is what is defined in the DT spec.

Rob

[PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Rob Herring]

On Fri, Apr 20, 2018 at 09:22:20AM +0200, Paul Kocialkowski wrote:
> Hi and thanks for the review,
> 
> On Fri, 2018-04-20 at 01:31 +0000, Tomasz Figa wrote:
> > Hi Paul, Philipp,
> > 
> > On Fri, Apr 20, 2018 at 1:04 AM Philipp Zabel <p.zabel@pengutronix.de>
> > wrote:
> > 
> > > Hi Paul,
> > > On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> > > > This adds a device-tree binding document that specifies the
> > > > properties
> > > > used by the Sunxi-Cedurs VPU driver, as well as examples.
> > > > 
> > > > Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> > > > ---
> > > >  .../devicetree/bindings/media/sunxi-cedrus.txt     | 50
> > 
> > ++++++++++++++++++++++
> > > >  1 file changed, 50 insertions(+)
> > > >  create mode 100644
> > 
> > Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > 
> > > > diff --git a/Documentation/devicetree/bindings/media/sunxi-
> > > > cedrus.txt
> > 
> > b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > new file mode 100644
> > > > index 000000000000..71ad3f9c3352
> > > > --- /dev/null
> > > > +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > @@ -0,0 +1,50 @@
> > > > +Device-tree bindings for the VPU found in Allwinner SoCs,
> > > > referred to
> > 
> > as the
> > > > +Video Engine (VE) in Allwinner literature.
> > > > +
> > > > +The VPU can only access the first 256 MiB of DRAM, that are DMA-
> > > > mapped
> > 
> > starting
> > > > +from the DRAM base. This requires specific memory allocation and
> > 
> > handling.
> > 
> > And no IOMMU? Brings back memories.
> 
> Exactly, no IOMMU so we don't have much choice but cope with that
> hardware limitation...
> 
> > > > +
> > > > +Required properties:
> > > > +- compatible         : "allwinner,sun4i-a10-video-engine";
> > > > +- memory-region         : DMA pool for buffers allocation;
> > > > +- clocks             : list of clock specifiers, corresponding to
> > 
> > entries in
> > > > +                          the clock-names property;
> > > > +- clock-names                : should contain "ahb", "mod" and
> > > > "ram"
> > 
> > entries;
> > > > +- assigned-clocks       : list of clocks assigned to the VE;
> > > > +- assigned-clocks-rates : list of clock rates for the clocks
> > > > assigned
> > 
> > to the VE;
> > > > +- resets             : phandle for reset;
> > > > +- interrupts         : should contain VE interrupt number;
> > > > +- reg                        : should contain register base and
> > > > length
> > 
> > of VE.
> > > > +
> > > > +Example:
> > > > +
> > > > +reserved-memory {
> > > > +     #address-cells = <1>;
> > > > +     #size-cells = <1>;
> > > > +     ranges;
> > > > +
> > > > +     /* Address must be kept in the lower 256 MiBs of DRAM for
> > > > VE. */
> > > > +     ve_memory: cma at 4a000000 {
> > > > +             compatible = "shared-dma-pool";
> > > > +             reg = <0x4a000000 0x6000000>;
> > > > +             no-map;
> > > > +             linux,cma-default;
> > > > +     };
> > > > +};
> > > > +
> > > > +video-engine at 1c0e000 {
> > > This is not really required by any specification, and not as common
> > > as
> > > gpu at ..., but could this reasonably be called "vpu at 1c0e000" to follow
> > > somewhat-common practice?
> > 
> > AFAIR the name is supposed to be somewhat readable for someone that
> > doesn't know the hardware. To me, "video-engine" sounds more obvious
> > than "vpu", but we actually use "codec" already, in case of MFC and
> > JPEG codec on Exynos. If encode/decode is the only functionality of
> > this block, I'd personally go with "codec". If it can do other things,
> > e.g. scaling/rotation without encode/decode, I'd probably call it
> > "video-processor".
> 
> I agree that the term VPU is more commonly associated with video
> decoding, while video engine could mean a number of things.
> 
> The reason I went with "video-engine" here (while still presenting the
> driver as a VPU driver) is that Video Engine is the term used in
> Allwinner's litterature. Other nodes in Allwinner device-trees generally
> stick to these terms (for instance, we have "display-engine" nodes).
> This also makes it easier to find the matching parts in the
> documentation.

'video-codec' is what is defined in the DT spec.

Rob

Re: [PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Rob Herring]

On Thu, Apr 19, 2018 at 05:45:34PM +0200, Paul Kocialkowski wrote:
> This adds a device-tree binding document that specifies the properties
> used by the Sunxi-Cedurs VPU driver, as well as examples.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  .../devicetree/bindings/media/sunxi-cedrus.txt     | 50 ++++++++++++++++++++++
>  1 file changed, 50 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/sunxi-cedrus.txt

Other than the one nit about the node name,

Reviewed-by: Rob Herring <robh@kernel.org>

Re: [PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Rob Herring]

On Thu, Apr 19, 2018 at 05:45:34PM +0200, Paul Kocialkowski wrote:
> This adds a device-tree binding document that specifies the properties
> used by the Sunxi-Cedurs VPU driver, as well as examples.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> ---
>  .../devicetree/bindings/media/sunxi-cedrus.txt     | 50 ++++++++++++++++++++++
>  1 file changed, 50 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/sunxi-cedrus.txt

Other than the one nit about the node name,

Reviewed-by: Rob Herring <robh-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>

[PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Rob Herring]

On Thu, Apr 19, 2018 at 05:45:34PM +0200, Paul Kocialkowski wrote:
> This adds a device-tree binding document that specifies the properties
> used by the Sunxi-Cedurs VPU driver, as well as examples.
> 
> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  .../devicetree/bindings/media/sunxi-cedrus.txt     | 50 ++++++++++++++++++++++
>  1 file changed, 50 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/sunxi-cedrus.txt

Other than the one nit about the node name,

Reviewed-by: Rob Herring <robh@kernel.org>

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 4427 bytes --]

Hi,

On Fri, 2018-04-20 at 09:39 +0200, Maxime Ripard wrote:
> On Thu, Apr 19, 2018 at 05:45:35PM +0200, Paul Kocialkowski wrote:
> > This adds nodes for the Video Engine and the associated reserved
> > memory
> > for the Allwinner A20. Up to 96 MiB of memory are dedicated to the
> > VPU.
> > 
> > The VPU can only map the first 256 MiB of DRAM, so the reserved
> > memory
> > pool has to be located in that area. Following Allwinner's decision
> > in
> > downstream software, the last 96 MiB of the first 256 MiB of RAM are
> > reserved for this purpose.
> > 
> > Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> > ---
> >  arch/arm/boot/dts/sun7i-a20.dtsi | 31
> > +++++++++++++++++++++++++++++++
> >  1 file changed, 31 insertions(+)
> > 
> > diff --git a/arch/arm/boot/dts/sun7i-a20.dtsi
> > b/arch/arm/boot/dts/sun7i-a20.dtsi
> > index bd0cd3204273..cb6d82065dcf 100644
> > --- a/arch/arm/boot/dts/sun7i-a20.dtsi
> > +++ b/arch/arm/boot/dts/sun7i-a20.dtsi
> > @@ -163,6 +163,20 @@
> >  		reg = <0x40000000 0x80000000>;
> >  	};
> >  
> > +	reserved-memory {
> > +		#address-cells = <1>;
> > +		#size-cells = <1>;
> > +		ranges;
> > +
> > +		/* Address must be kept in the lower 256 MiBs of
> > DRAM for VE. */
> > +		ve_memory: cma@4a000000 {
> > +			compatible = "shared-dma-pool";
> > +			reg = <0x4a000000 0x6000000>;
> > +			no-map;
> 
> I'm not sure why no-map is needed.

In fact, having no-map here would lead to reserving the area as cache-
coherent instead of contiguous and thus prevented dmabuf support.
Replacing it by "resuable" allows proper CMA reservation.

> And I guess we could use alloc-ranges to make sure the region is in
> the proper memory range, instead of hardcoding it.

As far as I could understand from the documentation, "alloc-ranges" is
used for dynamic allocation while only "reg" is used for static
allocation. We are currently going with static allocation and thus
reserve the whole 96 MiB. Is using dynamic allocation instead desirable
here?

> > +			linux,cma-default;
> > +		};
> > +	};
> > +
> >  	timer {
> >  		compatible = "arm,armv7-timer";
> >  		interrupts = <GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(4) |
> > IRQ_TYPE_LEVEL_LOW)>,
> > @@ -451,6 +465,23 @@
> >  			};
> >  		};
> >  
> > +		ve: video-engine@1c0e000 {
> > +			compatible = "allwinner,sun4i-a10-video-
> > engine";
> 
> We should have an A20-specific compatible here, at least, so that we
> can deal with any quirk we might find down the road.

Yes that's a very good point.

> > +			reg = <0x01c0e000 0x1000>;
> > +			memory-region = <&ve_memory>;
> 
> Since you made the CMA region the default one, you don't need to tie
> it to that device in particular (and you can drop it being mandatory
> from your binding as well).

What if another driver (or the system) claims memory from that zone and
that the reserved memory ends up not being available for the VPU
anymore?

Acccording to the reserved-memory documentation, the reusable property
(that we need for dmabuf) puts a limitation that the device driver
owning the region must be able to reclaim it back.

How does that work out if the CMA region is not tied to a driver in
particular?

> > +
> > +			clocks = <&ccu CLK_AHB_VE>, <&ccu CLK_VE>,
> > +				 <&ccu CLK_DRAM_VE>;
> > +			clock-names = "ahb", "mod", "ram";
> > +
> > +			assigned-clocks = <&ccu CLK_VE>;
> > +			assigned-clock-rates = <320000000>;
> 
> This should be set from within the driver. If it's something that you
> absolutely needed for the device to operate, you have no guarantee
> that the clock rate won't change at any point in time after the device
> probe, so that's not a proper solution.
> 
> And if it's not needed and can be adjusted depending on the
> framerate/codec/resolution, then it shouldn't be in the DT either.

Yes, that makes sense.

> Don't you also need to map the SRAM on the A20?

That's a good point, there is currently no syscon handle for A20 (and
also A13). Maybe SRAM is muxed to the VE by default so it "just works"? 

I'll investigate on this side, also keeping in mind that the actual
solution is to use the SRAM controller driver (but that won't make it to
v3).

Cheers and thanks for the review!

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 4326 bytes --]

Hi,

On Fri, 2018-04-20 at 09:39 +0200, Maxime Ripard wrote:
> On Thu, Apr 19, 2018 at 05:45:35PM +0200, Paul Kocialkowski wrote:
> > This adds nodes for the Video Engine and the associated reserved
> > memory
> > for the Allwinner A20. Up to 96 MiB of memory are dedicated to the
> > VPU.
> > 
> > The VPU can only map the first 256 MiB of DRAM, so the reserved
> > memory
> > pool has to be located in that area. Following Allwinner's decision
> > in
> > downstream software, the last 96 MiB of the first 256 MiB of RAM are
> > reserved for this purpose.
> > 
> > Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> > ---
> >  arch/arm/boot/dts/sun7i-a20.dtsi | 31
> > +++++++++++++++++++++++++++++++
> >  1 file changed, 31 insertions(+)
> > 
> > diff --git a/arch/arm/boot/dts/sun7i-a20.dtsi
> > b/arch/arm/boot/dts/sun7i-a20.dtsi
> > index bd0cd3204273..cb6d82065dcf 100644
> > --- a/arch/arm/boot/dts/sun7i-a20.dtsi
> > +++ b/arch/arm/boot/dts/sun7i-a20.dtsi
> > @@ -163,6 +163,20 @@
> >  		reg = <0x40000000 0x80000000>;
> >  	};
> >  
> > +	reserved-memory {
> > +		#address-cells = <1>;
> > +		#size-cells = <1>;
> > +		ranges;
> > +
> > +		/* Address must be kept in the lower 256 MiBs of
> > DRAM for VE. */
> > +		ve_memory: cma@4a000000 {
> > +			compatible = "shared-dma-pool";
> > +			reg = <0x4a000000 0x6000000>;
> > +			no-map;
> 
> I'm not sure why no-map is needed.

In fact, having no-map here would lead to reserving the area as cache-
coherent instead of contiguous and thus prevented dmabuf support.
Replacing it by "resuable" allows proper CMA reservation.

> And I guess we could use alloc-ranges to make sure the region is in
> the proper memory range, instead of hardcoding it.

As far as I could understand from the documentation, "alloc-ranges" is
used for dynamic allocation while only "reg" is used for static
allocation. We are currently going with static allocation and thus
reserve the whole 96 MiB. Is using dynamic allocation instead desirable
here?

> > +			linux,cma-default;
> > +		};
> > +	};
> > +
> >  	timer {
> >  		compatible = "arm,armv7-timer";
> >  		interrupts = <GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(4) |
> > IRQ_TYPE_LEVEL_LOW)>,
> > @@ -451,6 +465,23 @@
> >  			};
> >  		};
> >  
> > +		ve: video-engine@1c0e000 {
> > +			compatible = "allwinner,sun4i-a10-video-
> > engine";
> 
> We should have an A20-specific compatible here, at least, so that we
> can deal with any quirk we might find down the road.

Yes that's a very good point.

> > +			reg = <0x01c0e000 0x1000>;
> > +			memory-region = <&ve_memory>;
> 
> Since you made the CMA region the default one, you don't need to tie
> it to that device in particular (and you can drop it being mandatory
> from your binding as well).

What if another driver (or the system) claims memory from that zone and
that the reserved memory ends up not being available for the VPU
anymore?

Acccording to the reserved-memory documentation, the reusable property
(that we need for dmabuf) puts a limitation that the device driver
owning the region must be able to reclaim it back.

How does that work out if the CMA region is not tied to a driver in
particular?

> > +
> > +			clocks = <&ccu CLK_AHB_VE>, <&ccu CLK_VE>,
> > +				 <&ccu CLK_DRAM_VE>;
> > +			clock-names = "ahb", "mod", "ram";
> > +
> > +			assigned-clocks = <&ccu CLK_VE>;
> > +			assigned-clock-rates = <320000000>;
> 
> This should be set from within the driver. If it's something that you
> absolutely needed for the device to operate, you have no guarantee
> that the clock rate won't change at any point in time after the device
> probe, so that's not a proper solution.
> 
> And if it's not needed and can be adjusted depending on the
> framerate/codec/resolution, then it shouldn't be in the DT either.

Yes, that makes sense.

> Don't you also need to map the SRAM on the A20?

That's a good point, there is currently no syscon handle for A20 (and
also A13). Maybe SRAM is muxed to the VE by default so it "just works"? 

I'll investigate on this side, also keeping in mind that the actual
solution is to use the SRAM controller driver (but that won't make it to
v3).

Cheers and thanks for the review!

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

Hi,

On Fri, 2018-04-20 at 09:39 +0200, Maxime Ripard wrote:
> On Thu, Apr 19, 2018 at 05:45:35PM +0200, Paul Kocialkowski wrote:
> > This adds nodes for the Video Engine and the associated reserved
> > memory
> > for the Allwinner A20. Up to 96 MiB of memory are dedicated to the
> > VPU.
> > 
> > The VPU can only map the first 256 MiB of DRAM, so the reserved
> > memory
> > pool has to be located in that area. Following Allwinner's decision
> > in
> > downstream software, the last 96 MiB of the first 256 MiB of RAM are
> > reserved for this purpose.
> > 
> > Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> > ---
> >  arch/arm/boot/dts/sun7i-a20.dtsi | 31
> > +++++++++++++++++++++++++++++++
> >  1 file changed, 31 insertions(+)
> > 
> > diff --git a/arch/arm/boot/dts/sun7i-a20.dtsi
> > b/arch/arm/boot/dts/sun7i-a20.dtsi
> > index bd0cd3204273..cb6d82065dcf 100644
> > --- a/arch/arm/boot/dts/sun7i-a20.dtsi
> > +++ b/arch/arm/boot/dts/sun7i-a20.dtsi
> > @@ -163,6 +163,20 @@
> >  		reg = <0x40000000 0x80000000>;
> >  	};
> >  
> > +	reserved-memory {
> > +		#address-cells = <1>;
> > +		#size-cells = <1>;
> > +		ranges;
> > +
> > +		/* Address must be kept in the lower 256 MiBs of
> > DRAM for VE. */
> > +		ve_memory: cma at 4a000000 {
> > +			compatible = "shared-dma-pool";
> > +			reg = <0x4a000000 0x6000000>;
> > +			no-map;
> 
> I'm not sure why no-map is needed.

In fact, having no-map here would lead to reserving the area as cache-
coherent instead of contiguous and thus prevented dmabuf support.
Replacing it by "resuable" allows proper CMA reservation.

> And I guess we could use alloc-ranges to make sure the region is in
> the proper memory range, instead of hardcoding it.

As far as I could understand from the documentation, "alloc-ranges" is
used for dynamic allocation while only "reg" is used for static
allocation. We are currently going with static allocation and thus
reserve the whole 96 MiB. Is using dynamic allocation instead desirable
here?

> > +			linux,cma-default;
> > +		};
> > +	};
> > +
> >  	timer {
> >  		compatible = "arm,armv7-timer";
> >  		interrupts = <GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(4) |
> > IRQ_TYPE_LEVEL_LOW)>,
> > @@ -451,6 +465,23 @@
> >  			};
> >  		};
> >  
> > +		ve: video-engine at 1c0e000 {
> > +			compatible = "allwinner,sun4i-a10-video-
> > engine";
> 
> We should have an A20-specific compatible here, at least, so that we
> can deal with any quirk we might find down the road.

Yes that's a very good point.

> > +			reg = <0x01c0e000 0x1000>;
> > +			memory-region = <&ve_memory>;
> 
> Since you made the CMA region the default one, you don't need to tie
> it to that device in particular (and you can drop it being mandatory
> from your binding as well).

What if another driver (or the system) claims memory from that zone and
that the reserved memory ends up not being available for the VPU
anymore?

Acccording to the reserved-memory documentation, the reusable property
(that we need for dmabuf) puts a limitation that the device driver
owning the region must be able to reclaim it back.

How does that work out if the CMA region is not tied to a driver in
particular?

> > +
> > +			clocks = <&ccu CLK_AHB_VE>, <&ccu CLK_VE>,
> > +				 <&ccu CLK_DRAM_VE>;
> > +			clock-names = "ahb", "mod", "ram";
> > +
> > +			assigned-clocks = <&ccu CLK_VE>;
> > +			assigned-clock-rates = <320000000>;
> 
> This should be set from within the driver. If it's something that you
> absolutely needed for the device to operate, you have no guarantee
> that the clock rate won't change at any point in time after the device
> probe, so that's not a proper solution.
> 
> And if it's not needed and can be adjusted depending on the
> framerate/codec/resolution, then it shouldn't be in the DT either.

Yes, that makes sense.

> Don't you also need to map the SRAM on the A20?

That's a good point, there is currently no syscon handle for A20 (and
also A13). Maybe SRAM is muxed to the VE by default so it "just works"? 

I'll investigate on this side, also keeping in mind that the actual
solution is to use the SRAM controller driver (but that won't make it to
v3).

Cheers and thanks for the review!

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/62afecee/attachment.sig>

Re: [PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 5333 bytes --]

Hi,

On Thu, 2018-04-26 at 22:04 -0500, Rob Herring wrote:
> On Fri, Apr 20, 2018 at 09:22:20AM +0200, Paul Kocialkowski wrote:
> > Hi and thanks for the review,
> > 
> > On Fri, 2018-04-20 at 01:31 +0000, Tomasz Figa wrote:
> > > Hi Paul, Philipp,
> > > 
> > > On Fri, Apr 20, 2018 at 1:04 AM Philipp Zabel <p.zabel@pengutronix
> > > .de>
> > > wrote:
> > > 
> > > > Hi Paul,
> > > > On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> > > > > This adds a device-tree binding document that specifies the
> > > > > properties
> > > > > used by the Sunxi-Cedurs VPU driver, as well as examples.
> > > > > 
> > > > > Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.co
> > > > > m>
> > > > > ---
> > > > >  .../devicetree/bindings/media/sunxi-cedrus.txt     | 50
> > > 
> > > ++++++++++++++++++++++
> > > > >  1 file changed, 50 insertions(+)
> > > > >  create mode 100644
> > > 
> > > Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > > 
> > > > > diff --git a/Documentation/devicetree/bindings/media/sunxi-
> > > > > cedrus.txt
> > > 
> > > b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > > new file mode 100644
> > > > > index 000000000000..71ad3f9c3352
> > > > > --- /dev/null
> > > > > +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > > @@ -0,0 +1,50 @@
> > > > > +Device-tree bindings for the VPU found in Allwinner SoCs,
> > > > > referred to
> > > 
> > > as the
> > > > > +Video Engine (VE) in Allwinner literature.
> > > > > +
> > > > > +The VPU can only access the first 256 MiB of DRAM, that are
> > > > > DMA-
> > > > > mapped
> > > 
> > > starting
> > > > > +from the DRAM base. This requires specific memory allocation
> > > > > and
> > > 
> > > handling.
> > > 
> > > And no IOMMU? Brings back memories.
> > 
> > Exactly, no IOMMU so we don't have much choice but cope with that
> > hardware limitation...
> > 
> > > > > +
> > > > > +Required properties:
> > > > > +- compatible         : "allwinner,sun4i-a10-video-engine";
> > > > > +- memory-region         : DMA pool for buffers allocation;
> > > > > +- clocks             : list of clock specifiers,
> > > > > corresponding to
> > > 
> > > entries in
> > > > > +                          the clock-names property;
> > > > > +- clock-names                : should contain "ahb", "mod"
> > > > > and
> > > > > "ram"
> > > 
> > > entries;
> > > > > +- assigned-clocks       : list of clocks assigned to the VE;
> > > > > +- assigned-clocks-rates : list of clock rates for the clocks
> > > > > assigned
> > > 
> > > to the VE;
> > > > > +- resets             : phandle for reset;
> > > > > +- interrupts         : should contain VE interrupt number;
> > > > > +- reg                        : should contain register base
> > > > > and
> > > > > length
> > > 
> > > of VE.
> > > > > +
> > > > > +Example:
> > > > > +
> > > > > +reserved-memory {
> > > > > +     #address-cells = <1>;
> > > > > +     #size-cells = <1>;
> > > > > +     ranges;
> > > > > +
> > > > > +     /* Address must be kept in the lower 256 MiBs of DRAM
> > > > > for
> > > > > VE. */
> > > > > +     ve_memory: cma@4a000000 {
> > > > > +             compatible = "shared-dma-pool";
> > > > > +             reg = <0x4a000000 0x6000000>;
> > > > > +             no-map;
> > > > > +             linux,cma-default;
> > > > > +     };
> > > > > +};
> > > > > +
> > > > > +video-engine@1c0e000 {
> > > > 
> > > > This is not really required by any specification, and not as
> > > > common
> > > > as
> > > > gpu@..., but could this reasonably be called "vpu@1c0e000" to
> > > > follow
> > > > somewhat-common practice?
> > > 
> > > AFAIR the name is supposed to be somewhat readable for someone
> > > that
> > > doesn't know the hardware. To me, "video-engine" sounds more
> > > obvious
> > > than "vpu", but we actually use "codec" already, in case of MFC
> > > and
> > > JPEG codec on Exynos. If encode/decode is the only functionality
> > > of
> > > this block, I'd personally go with "codec". If it can do other
> > > things,
> > > e.g. scaling/rotation without encode/decode, I'd probably call it
> > > "video-processor".
> > 
> > I agree that the term VPU is more commonly associated with video
> > decoding, while video engine could mean a number of things.
> > 
> > The reason I went with "video-engine" here (while still presenting
> > the
> > driver as a VPU driver) is that Video Engine is the term used in
> > Allwinner's litterature. Other nodes in Allwinner device-trees
> > generally
> > stick to these terms (for instance, we have "display-engine" nodes).
> > This also makes it easier to find the matching parts in the
> > documentation.
> 
> 'video-codec' is what is defined in the DT spec.

Is that an actively-enforced guideline or a suggestion? I'd like to keep
video-engine just to stick with the technical documentation wording and
my personal taste is also to prefer vpu over video-codec (in terms of
clarity/straightforwardness) as a second choice.

Still, if the choice isn't up to me, we can go with video-codec (or
vpu).

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 5208 bytes --]

Hi,

On Thu, 2018-04-26 at 22:04 -0500, Rob Herring wrote:
> On Fri, Apr 20, 2018 at 09:22:20AM +0200, Paul Kocialkowski wrote:
> > Hi and thanks for the review,
> > 
> > On Fri, 2018-04-20 at 01:31 +0000, Tomasz Figa wrote:
> > > Hi Paul, Philipp,
> > > 
> > > On Fri, Apr 20, 2018 at 1:04 AM Philipp Zabel <p.zabel@pengutronix
> > > .de>
> > > wrote:
> > > 
> > > > Hi Paul,
> > > > On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> > > > > This adds a device-tree binding document that specifies the
> > > > > properties
> > > > > used by the Sunxi-Cedurs VPU driver, as well as examples.
> > > > > 
> > > > > Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcKIwRZHo2/mJg@public.gmane.org
> > > > > m>
> > > > > ---
> > > > >  .../devicetree/bindings/media/sunxi-cedrus.txt     | 50
> > > 
> > > ++++++++++++++++++++++
> > > > >  1 file changed, 50 insertions(+)
> > > > >  create mode 100644
> > > 
> > > Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > > 
> > > > > diff --git a/Documentation/devicetree/bindings/media/sunxi-
> > > > > cedrus.txt
> > > 
> > > b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > > new file mode 100644
> > > > > index 000000000000..71ad3f9c3352
> > > > > --- /dev/null
> > > > > +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > > @@ -0,0 +1,50 @@
> > > > > +Device-tree bindings for the VPU found in Allwinner SoCs,
> > > > > referred to
> > > 
> > > as the
> > > > > +Video Engine (VE) in Allwinner literature.
> > > > > +
> > > > > +The VPU can only access the first 256 MiB of DRAM, that are
> > > > > DMA-
> > > > > mapped
> > > 
> > > starting
> > > > > +from the DRAM base. This requires specific memory allocation
> > > > > and
> > > 
> > > handling.
> > > 
> > > And no IOMMU? Brings back memories.
> > 
> > Exactly, no IOMMU so we don't have much choice but cope with that
> > hardware limitation...
> > 
> > > > > +
> > > > > +Required properties:
> > > > > +- compatible         : "allwinner,sun4i-a10-video-engine";
> > > > > +- memory-region         : DMA pool for buffers allocation;
> > > > > +- clocks             : list of clock specifiers,
> > > > > corresponding to
> > > 
> > > entries in
> > > > > +                          the clock-names property;
> > > > > +- clock-names                : should contain "ahb", "mod"
> > > > > and
> > > > > "ram"
> > > 
> > > entries;
> > > > > +- assigned-clocks       : list of clocks assigned to the VE;
> > > > > +- assigned-clocks-rates : list of clock rates for the clocks
> > > > > assigned
> > > 
> > > to the VE;
> > > > > +- resets             : phandle for reset;
> > > > > +- interrupts         : should contain VE interrupt number;
> > > > > +- reg                        : should contain register base
> > > > > and
> > > > > length
> > > 
> > > of VE.
> > > > > +
> > > > > +Example:
> > > > > +
> > > > > +reserved-memory {
> > > > > +     #address-cells = <1>;
> > > > > +     #size-cells = <1>;
> > > > > +     ranges;
> > > > > +
> > > > > +     /* Address must be kept in the lower 256 MiBs of DRAM
> > > > > for
> > > > > VE. */
> > > > > +     ve_memory: cma@4a000000 {
> > > > > +             compatible = "shared-dma-pool";
> > > > > +             reg = <0x4a000000 0x6000000>;
> > > > > +             no-map;
> > > > > +             linux,cma-default;
> > > > > +     };
> > > > > +};
> > > > > +
> > > > > +video-engine@1c0e000 {
> > > > 
> > > > This is not really required by any specification, and not as
> > > > common
> > > > as
> > > > gpu@..., but could this reasonably be called "vpu@1c0e000" to
> > > > follow
> > > > somewhat-common practice?
> > > 
> > > AFAIR the name is supposed to be somewhat readable for someone
> > > that
> > > doesn't know the hardware. To me, "video-engine" sounds more
> > > obvious
> > > than "vpu", but we actually use "codec" already, in case of MFC
> > > and
> > > JPEG codec on Exynos. If encode/decode is the only functionality
> > > of
> > > this block, I'd personally go with "codec". If it can do other
> > > things,
> > > e.g. scaling/rotation without encode/decode, I'd probably call it
> > > "video-processor".
> > 
> > I agree that the term VPU is more commonly associated with video
> > decoding, while video engine could mean a number of things.
> > 
> > The reason I went with "video-engine" here (while still presenting
> > the
> > driver as a VPU driver) is that Video Engine is the term used in
> > Allwinner's litterature. Other nodes in Allwinner device-trees
> > generally
> > stick to these terms (for instance, we have "display-engine" nodes).
> > This also makes it easier to find the matching parts in the
> > documentation.
> 
> 'video-codec' is what is defined in the DT spec.

Is that an actively-enforced guideline or a suggestion? I'd like to keep
video-engine just to stick with the technical documentation wording and
my personal taste is also to prefer vpu over video-codec (in terms of
clarity/straightforwardness) as a second choice.

Still, if the choice isn't up to me, we can go with video-codec (or
vpu).

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Paul Kocialkowski]

Hi,

On Thu, 2018-04-26 at 22:04 -0500, Rob Herring wrote:
> On Fri, Apr 20, 2018 at 09:22:20AM +0200, Paul Kocialkowski wrote:
> > Hi and thanks for the review,
> > 
> > On Fri, 2018-04-20 at 01:31 +0000, Tomasz Figa wrote:
> > > Hi Paul, Philipp,
> > > 
> > > On Fri, Apr 20, 2018 at 1:04 AM Philipp Zabel <p.zabel@pengutronix
> > > .de>
> > > wrote:
> > > 
> > > > Hi Paul,
> > > > On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> > > > > This adds a device-tree binding document that specifies the
> > > > > properties
> > > > > used by the Sunxi-Cedurs VPU driver, as well as examples.
> > > > > 
> > > > > Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.co
> > > > > m>
> > > > > ---
> > > > >  .../devicetree/bindings/media/sunxi-cedrus.txt     | 50
> > > 
> > > ++++++++++++++++++++++
> > > > >  1 file changed, 50 insertions(+)
> > > > >  create mode 100644
> > > 
> > > Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > > 
> > > > > diff --git a/Documentation/devicetree/bindings/media/sunxi-
> > > > > cedrus.txt
> > > 
> > > b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > > new file mode 100644
> > > > > index 000000000000..71ad3f9c3352
> > > > > --- /dev/null
> > > > > +++ b/Documentation/devicetree/bindings/media/sunxi-cedrus.txt
> > > > > @@ -0,0 +1,50 @@
> > > > > +Device-tree bindings for the VPU found in Allwinner SoCs,
> > > > > referred to
> > > 
> > > as the
> > > > > +Video Engine (VE) in Allwinner literature.
> > > > > +
> > > > > +The VPU can only access the first 256 MiB of DRAM, that are
> > > > > DMA-
> > > > > mapped
> > > 
> > > starting
> > > > > +from the DRAM base. This requires specific memory allocation
> > > > > and
> > > 
> > > handling.
> > > 
> > > And no IOMMU? Brings back memories.
> > 
> > Exactly, no IOMMU so we don't have much choice but cope with that
> > hardware limitation...
> > 
> > > > > +
> > > > > +Required properties:
> > > > > +- compatible         : "allwinner,sun4i-a10-video-engine";
> > > > > +- memory-region         : DMA pool for buffers allocation;
> > > > > +- clocks             : list of clock specifiers,
> > > > > corresponding to
> > > 
> > > entries in
> > > > > +                          the clock-names property;
> > > > > +- clock-names                : should contain "ahb", "mod"
> > > > > and
> > > > > "ram"
> > > 
> > > entries;
> > > > > +- assigned-clocks       : list of clocks assigned to the VE;
> > > > > +- assigned-clocks-rates : list of clock rates for the clocks
> > > > > assigned
> > > 
> > > to the VE;
> > > > > +- resets             : phandle for reset;
> > > > > +- interrupts         : should contain VE interrupt number;
> > > > > +- reg                        : should contain register base
> > > > > and
> > > > > length
> > > 
> > > of VE.
> > > > > +
> > > > > +Example:
> > > > > +
> > > > > +reserved-memory {
> > > > > +     #address-cells = <1>;
> > > > > +     #size-cells = <1>;
> > > > > +     ranges;
> > > > > +
> > > > > +     /* Address must be kept in the lower 256 MiBs of DRAM
> > > > > for
> > > > > VE. */
> > > > > +     ve_memory: cma at 4a000000 {
> > > > > +             compatible = "shared-dma-pool";
> > > > > +             reg = <0x4a000000 0x6000000>;
> > > > > +             no-map;
> > > > > +             linux,cma-default;
> > > > > +     };
> > > > > +};
> > > > > +
> > > > > +video-engine at 1c0e000 {
> > > > 
> > > > This is not really required by any specification, and not as
> > > > common
> > > > as
> > > > gpu at ..., but could this reasonably be called "vpu at 1c0e000" to
> > > > follow
> > > > somewhat-common practice?
> > > 
> > > AFAIR the name is supposed to be somewhat readable for someone
> > > that
> > > doesn't know the hardware. To me, "video-engine" sounds more
> > > obvious
> > > than "vpu", but we actually use "codec" already, in case of MFC
> > > and
> > > JPEG codec on Exynos. If encode/decode is the only functionality
> > > of
> > > this block, I'd personally go with "codec". If it can do other
> > > things,
> > > e.g. scaling/rotation without encode/decode, I'd probably call it
> > > "video-processor".
> > 
> > I agree that the term VPU is more commonly associated with video
> > decoding, while video engine could mean a number of things.
> > 
> > The reason I went with "video-engine" here (while still presenting
> > the
> > driver as a VPU driver) is that Video Engine is the term used in
> > Allwinner's litterature. Other nodes in Allwinner device-trees
> > generally
> > stick to these terms (for instance, we have "display-engine" nodes).
> > This also makes it easier to find the matching parts in the
> > documentation.
> 
> 'video-codec' is what is defined in the DT spec.

Is that an actively-enforced guideline or a suggestion? I'd like to keep
video-engine just to stick with the technical documentation wording and
my personal taste is also to prefer vpu over video-codec (in terms of
clarity/straightforwardness) as a second choice.

Still, if the choice isn't up to me, we can go with video-codec (or
vpu).

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/5bbb8dfd/attachment.sig>

Re: [PATCH v2 07/10] media: platform: Add Sunxi-Cedrus VPU decoder driver

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 1244 bytes --]

Hi,

On Tue, 2018-04-24 at 12:13 +0300, Sakari Ailus wrote:
> Hi Paul
> 
> On Thu, Apr 19, 2018 at 05:45:33PM +0200, Paul Kocialkowski wrote:
> > This introduces the Sunxi-Cedrus VPU driver that supports the VPU
> > found
> > in Allwinner SoCs, also known as Video Engine. It is implemented
> > through
> > a v4l2 m2m decoder device and a media device (used for media
> > requests).
> > So far, it only supports MPEG2 decoding.
> > 
> > Since this VPU is stateless, synchronization with media requests is
> > required in order to ensure consistency between frame headers that
> > contain metadata about the frame to process and the raw slice data
> > that
> > is used to generate the frame.
> > 
> > This driver was made possible thanks to the long-standing effort
> > carried out by the linux-sunxi community in the interest of reverse
> > engineering, documenting and implementing support for Allwinner VPU.
> 
> No code review yet, but DT bindings precede the driver. Please also
> add the appropriate MAINTAINERS entries.

Thanks for the indication, will do in the next version(s).

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 07/10] media: platform: Add Sunxi-Cedrus VPU decoder driver

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 1210 bytes --]

Hi,

On Tue, 2018-04-24 at 12:13 +0300, Sakari Ailus wrote:
> Hi Paul
> 
> On Thu, Apr 19, 2018 at 05:45:33PM +0200, Paul Kocialkowski wrote:
> > This introduces the Sunxi-Cedrus VPU driver that supports the VPU
> > found
> > in Allwinner SoCs, also known as Video Engine. It is implemented
> > through
> > a v4l2 m2m decoder device and a media device (used for media
> > requests).
> > So far, it only supports MPEG2 decoding.
> > 
> > Since this VPU is stateless, synchronization with media requests is
> > required in order to ensure consistency between frame headers that
> > contain metadata about the frame to process and the raw slice data
> > that
> > is used to generate the frame.
> > 
> > This driver was made possible thanks to the long-standing effort
> > carried out by the linux-sunxi community in the interest of reverse
> > engineering, documenting and implementing support for Allwinner VPU.
> 
> No code review yet, but DT bindings precede the driver. Please also
> add the appropriate MAINTAINERS entries.

Thanks for the indication, will do in the next version(s).

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[PATCH v2 07/10] media: platform: Add Sunxi-Cedrus VPU decoder driver

[Paul Kocialkowski]

Hi,

On Tue, 2018-04-24 at 12:13 +0300, Sakari Ailus wrote:
> Hi Paul
> 
> On Thu, Apr 19, 2018 at 05:45:33PM +0200, Paul Kocialkowski wrote:
> > This introduces the Sunxi-Cedrus VPU driver that supports the VPU
> > found
> > in Allwinner SoCs, also known as Video Engine. It is implemented
> > through
> > a v4l2 m2m decoder device and a media device (used for media
> > requests).
> > So far, it only supports MPEG2 decoding.
> > 
> > Since this VPU is stateless, synchronization with media requests is
> > required in order to ensure consistency between frame headers that
> > contain metadata about the frame to process and the raw slice data
> > that
> > is used to generate the frame.
> > 
> > This driver was made possible thanks to the long-standing effort
> > carried out by the linux-sunxi community in the interest of reverse
> > engineering, documenting and implementing support for Allwinner VPU.
> 
> No code review yet, but DT bindings precede the driver. Please also
> add the appropriate MAINTAINERS entries.

Thanks for the indication, will do in the next version(s).

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/acc41e91/attachment.sig>

Re: [PATCH v2 06/10] media: v4l: Add definition for Allwinner's MB32-tiled NV12 format

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 2082 bytes --]

Hi,

On Fri, 2018-04-20 at 15:59 +0200, Hans Verkuil wrote:
> On 04/19/18 17:45, Paul Kocialkowski wrote:
> > This introduces support for Allwinner's MB32-tiled NV12 format,
> > where
> > each plane is divided into macroblocks of 32x32 pixels. Hence, the
> > size
> > of each plane has to be aligned to 32 bytes. The pixels inside each
> > macroblock are coded as they would be if the macroblock was a single
> > plane, line after line.
> > 
> > The MB32-tiled NV12 format is used by the video engine on Allwinner
> > platforms: it is the default format for decoded frames (and the only
> > one
> > available in the oldest supported platforms).
> > 
> > Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> > ---
> >  include/uapi/linux/videodev2.h | 1 +
> >  1 file changed, 1 insertion(+)
> > 
> > diff --git a/include/uapi/linux/videodev2.h
> > b/include/uapi/linux/videodev2.h
> > index 4b8336f7bcf0..43993a116e2b 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -669,6 +669,7 @@ struct v4l2_pix_format {
> >  #define V4L2_PIX_FMT_Z16      v4l2_fourcc('Z', '1', '6', ' ') /*
> > Depth data 16-bit */
> >  #define V4L2_PIX_FMT_MT21C    v4l2_fourcc('M', 'T', '2', '1') /*
> > Mediatek compressed block mode  */
> >  #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /*
> > Intel Planar Greyscale 10-bit and Depth 16-bit */
> > +#define V4L2_PIX_FMT_MB32_NV12 v4l2_fourcc('M', 'N', '1', '2') /*
> > Allwinner NV12 in 32x32 macroblocks */
> >  
> >  /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6
> > bits unused */
> >  #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3',
> > 'b') /* IPU3 packed 10-bit BGGR bayer */
> > 
> 
> Add an entry for this to v4l_fill_fmtdesc() in v4l2-ioctl.c.
> 
> It also needs to be documented in the spec.

Noted, I will look in that direction for the future versions.

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 06/10] media: v4l: Add definition for Allwinner's MB32-tiled NV12 format

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 2056 bytes --]

Hi,

On Fri, 2018-04-20 at 15:59 +0200, Hans Verkuil wrote:
> On 04/19/18 17:45, Paul Kocialkowski wrote:
> > This introduces support for Allwinner's MB32-tiled NV12 format,
> > where
> > each plane is divided into macroblocks of 32x32 pixels. Hence, the
> > size
> > of each plane has to be aligned to 32 bytes. The pixels inside each
> > macroblock are coded as they would be if the macroblock was a single
> > plane, line after line.
> > 
> > The MB32-tiled NV12 format is used by the video engine on Allwinner
> > platforms: it is the default format for decoded frames (and the only
> > one
> > available in the oldest supported platforms).
> > 
> > Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> > ---
> >  include/uapi/linux/videodev2.h | 1 +
> >  1 file changed, 1 insertion(+)
> > 
> > diff --git a/include/uapi/linux/videodev2.h
> > b/include/uapi/linux/videodev2.h
> > index 4b8336f7bcf0..43993a116e2b 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -669,6 +669,7 @@ struct v4l2_pix_format {
> >  #define V4L2_PIX_FMT_Z16      v4l2_fourcc('Z', '1', '6', ' ') /*
> > Depth data 16-bit */
> >  #define V4L2_PIX_FMT_MT21C    v4l2_fourcc('M', 'T', '2', '1') /*
> > Mediatek compressed block mode  */
> >  #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /*
> > Intel Planar Greyscale 10-bit and Depth 16-bit */
> > +#define V4L2_PIX_FMT_MB32_NV12 v4l2_fourcc('M', 'N', '1', '2') /*
> > Allwinner NV12 in 32x32 macroblocks */
> >  
> >  /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6
> > bits unused */
> >  #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3',
> > 'b') /* IPU3 packed 10-bit BGGR bayer */
> > 
> 
> Add an entry for this to v4l_fill_fmtdesc() in v4l2-ioctl.c.
> 
> It also needs to be documented in the spec.

Noted, I will look in that direction for the future versions.

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[PATCH v2 06/10] media: v4l: Add definition for Allwinner's MB32-tiled NV12 format

[Paul Kocialkowski]

Hi,

On Fri, 2018-04-20 at 15:59 +0200, Hans Verkuil wrote:
> On 04/19/18 17:45, Paul Kocialkowski wrote:
> > This introduces support for Allwinner's MB32-tiled NV12 format,
> > where
> > each plane is divided into macroblocks of 32x32 pixels. Hence, the
> > size
> > of each plane has to be aligned to 32 bytes. The pixels inside each
> > macroblock are coded as they would be if the macroblock was a single
> > plane, line after line.
> > 
> > The MB32-tiled NV12 format is used by the video engine on Allwinner
> > platforms: it is the default format for decoded frames (and the only
> > one
> > available in the oldest supported platforms).
> > 
> > Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> > ---
> >  include/uapi/linux/videodev2.h | 1 +
> >  1 file changed, 1 insertion(+)
> > 
> > diff --git a/include/uapi/linux/videodev2.h
> > b/include/uapi/linux/videodev2.h
> > index 4b8336f7bcf0..43993a116e2b 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -669,6 +669,7 @@ struct v4l2_pix_format {
> >  #define V4L2_PIX_FMT_Z16      v4l2_fourcc('Z', '1', '6', ' ') /*
> > Depth data 16-bit */
> >  #define V4L2_PIX_FMT_MT21C    v4l2_fourcc('M', 'T', '2', '1') /*
> > Mediatek compressed block mode  */
> >  #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /*
> > Intel Planar Greyscale 10-bit and Depth 16-bit */
> > +#define V4L2_PIX_FMT_MB32_NV12 v4l2_fourcc('M', 'N', '1', '2') /*
> > Allwinner NV12 in 32x32 macroblocks */
> >  
> >  /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6
> > bits unused */
> >  #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3',
> > 'b') /* IPU3 packed 10-bit BGGR bayer */
> > 
> 
> Add an entry for this to v4l_fill_fmtdesc() in v4l2-ioctl.c.
> 
> It also needs to be documented in the spec.

Noted, I will look in that direction for the future versions.

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/89c68886/attachment-0001.sig>

Re: [PATCH v2 02/10] media-request: Add a request complete operation to allow m2m scheduling

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 2752 bytes --]

Hi,

On Thu, 2018-04-19 at 17:41 +0200, Paul Kocialkowski wrote:
> When using the request API in the context of a m2m driver, the
> operations that come with a m2m run scheduling call in their
> (m2m-specific) ioctl handler are delayed until the request is queued
> (for instance, this includes queuing buffers and streamon).
> 
> Thus, the m2m run scheduling calls are not called in due time since
> the
> request AP's internal plumbing will (rightfully) use the relevant core
> functions directly instead of the ioctl handler.
> 
> This ends up in a situation where nothing happens if there is no
> run-scheduling ioctl called after queuing the request.
> 
> In order to circumvent the issue, a new media operation is introduced,
> called at the time of handling the media request queue ioctl. It gives
> m2m drivers a chance to schedule a m2m device run at that time.
> 
> The existing req_queue operation cannot be used for this purpose,
> since
> it is called with the request queue mutex held, that is eventually
> needed
> in the device_run call to apply relevant controls.

This patch will be dropped since it's no longer useful with the latest
version of the request API.

> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  drivers/media/media-request.c | 3 +++
>  include/media/media-device.h  | 2 ++
>  2 files changed, 5 insertions(+)
> 
> diff --git a/drivers/media/media-request.c b/drivers/media/media-
> request.c
> index 415f7e31019d..28ac5ccfe6a2 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -157,6 +157,9 @@ static long media_request_ioctl_queue(struct
> media_request *req)
>  		media_request_get(req);
>  	}
>  
> +	if (mdev->ops->req_complete)
> +		mdev->ops->req_complete(req);
> +
>  	return ret;
>  }
>  
> diff --git a/include/media/media-device.h b/include/media/media-
> device.h
> index 07e323c57202..c7dcf2079cc9 100644
> --- a/include/media/media-device.h
> +++ b/include/media/media-device.h
> @@ -55,6 +55,7 @@ struct media_entity_notify {
>   * @req_alloc: Allocate a request
>   * @req_free: Free a request
>   * @req_queue: Queue a request
> + * @req_complete: Complete a request
>   */
>  struct media_device_ops {
>  	int (*link_notify)(struct media_link *link, u32 flags,
> @@ -62,6 +63,7 @@ struct media_device_ops {
>  	struct media_request *(*req_alloc)(struct media_device
> *mdev);
>  	void (*req_free)(struct media_request *req);
>  	int (*req_queue)(struct media_request *req);
> +	void (*req_complete)(struct media_request *req);
>  };
>  
>  /**
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 02/10] media-request: Add a request complete operation to allow m2m scheduling

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 2704 bytes --]

Hi,

On Thu, 2018-04-19 at 17:41 +0200, Paul Kocialkowski wrote:
> When using the request API in the context of a m2m driver, the
> operations that come with a m2m run scheduling call in their
> (m2m-specific) ioctl handler are delayed until the request is queued
> (for instance, this includes queuing buffers and streamon).
> 
> Thus, the m2m run scheduling calls are not called in due time since
> the
> request AP's internal plumbing will (rightfully) use the relevant core
> functions directly instead of the ioctl handler.
> 
> This ends up in a situation where nothing happens if there is no
> run-scheduling ioctl called after queuing the request.
> 
> In order to circumvent the issue, a new media operation is introduced,
> called at the time of handling the media request queue ioctl. It gives
> m2m drivers a chance to schedule a m2m device run at that time.
> 
> The existing req_queue operation cannot be used for this purpose,
> since
> it is called with the request queue mutex held, that is eventually
> needed
> in the device_run call to apply relevant controls.

This patch will be dropped since it's no longer useful with the latest
version of the request API.

> Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> ---
>  drivers/media/media-request.c | 3 +++
>  include/media/media-device.h  | 2 ++
>  2 files changed, 5 insertions(+)
> 
> diff --git a/drivers/media/media-request.c b/drivers/media/media-
> request.c
> index 415f7e31019d..28ac5ccfe6a2 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -157,6 +157,9 @@ static long media_request_ioctl_queue(struct
> media_request *req)
>  		media_request_get(req);
>  	}
>  
> +	if (mdev->ops->req_complete)
> +		mdev->ops->req_complete(req);
> +
>  	return ret;
>  }
>  
> diff --git a/include/media/media-device.h b/include/media/media-
> device.h
> index 07e323c57202..c7dcf2079cc9 100644
> --- a/include/media/media-device.h
> +++ b/include/media/media-device.h
> @@ -55,6 +55,7 @@ struct media_entity_notify {
>   * @req_alloc: Allocate a request
>   * @req_free: Free a request
>   * @req_queue: Queue a request
> + * @req_complete: Complete a request
>   */
>  struct media_device_ops {
>  	int (*link_notify)(struct media_link *link, u32 flags,
> @@ -62,6 +63,7 @@ struct media_device_ops {
>  	struct media_request *(*req_alloc)(struct media_device
> *mdev);
>  	void (*req_free)(struct media_request *req);
>  	int (*req_queue)(struct media_request *req);
> +	void (*req_complete)(struct media_request *req);
>  };
>  
>  /**
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[PATCH v2 02/10] media-request: Add a request complete operation to allow m2m scheduling

[Paul Kocialkowski]

Hi,

On Thu, 2018-04-19 at 17:41 +0200, Paul Kocialkowski wrote:
> When using the request API in the context of a m2m driver, the
> operations that come with a m2m run scheduling call in their
> (m2m-specific) ioctl handler are delayed until the request is queued
> (for instance, this includes queuing buffers and streamon).
> 
> Thus, the m2m run scheduling calls are not called in due time since
> the
> request AP's internal plumbing will (rightfully) use the relevant core
> functions directly instead of the ioctl handler.
> 
> This ends up in a situation where nothing happens if there is no
> run-scheduling ioctl called after queuing the request.
> 
> In order to circumvent the issue, a new media operation is introduced,
> called at the time of handling the media request queue ioctl. It gives
> m2m drivers a chance to schedule a m2m device run at that time.
> 
> The existing req_queue operation cannot be used for this purpose,
> since
> it is called with the request queue mutex held, that is eventually
> needed
> in the device_run call to apply relevant controls.

This patch will be dropped since it's no longer useful with the latest
version of the request API.

> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  drivers/media/media-request.c | 3 +++
>  include/media/media-device.h  | 2 ++
>  2 files changed, 5 insertions(+)
> 
> diff --git a/drivers/media/media-request.c b/drivers/media/media-
> request.c
> index 415f7e31019d..28ac5ccfe6a2 100644
> --- a/drivers/media/media-request.c
> +++ b/drivers/media/media-request.c
> @@ -157,6 +157,9 @@ static long media_request_ioctl_queue(struct
> media_request *req)
>  		media_request_get(req);
>  	}
>  
> +	if (mdev->ops->req_complete)
> +		mdev->ops->req_complete(req);
> +
>  	return ret;
>  }
>  
> diff --git a/include/media/media-device.h b/include/media/media-
> device.h
> index 07e323c57202..c7dcf2079cc9 100644
> --- a/include/media/media-device.h
> +++ b/include/media/media-device.h
> @@ -55,6 +55,7 @@ struct media_entity_notify {
>   * @req_alloc: Allocate a request
>   * @req_free: Free a request
>   * @req_queue: Queue a request
> + * @req_complete: Complete a request
>   */
>  struct media_device_ops {
>  	int (*link_notify)(struct media_link *link, u32 flags,
> @@ -62,6 +63,7 @@ struct media_device_ops {
>  	struct media_request *(*req_alloc)(struct media_device
> *mdev);
>  	void (*req_free)(struct media_request *req);
>  	int (*req_queue)(struct media_request *req);
> +	void (*req_complete)(struct media_request *req);
>  };
>  
>  /**
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/d4d89b15/attachment.sig>

Re: [PATCH v2 03/10] videobuf2-core: Add helper to get buffer private data from media request

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 2756 bytes --]

Hi,

On Thu, 2018-04-19 at 17:41 +0200, Paul Kocialkowski wrote:
> When calling media operation driver callbacks related to media
> requests,
> only a pointer to the request itself is provided, which is
> insufficient
> to retrieve the driver's context. Since the driver context is usually
> set as vb2 queue private data and given that the core can determine
> which objects attached to the request are buffers, it is possible to
> extract the associated private data for the first buffer found.
> 
> This is required in order to access the current m2m context from m2m
> drivers' private data in the context of media request operation
> callbacks. More specifically, this allows scheduling m2m device runs
> from the newly-introduced request complete operation.

This patch too will be dropped since it's no longer useful with the
latest version of the request API.

> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  drivers/media/common/videobuf2/videobuf2-core.c | 15 +++++++++++++++
>  include/media/videobuf2-core.h                  |  1 +
>  2 files changed, 16 insertions(+)
> 
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> b/drivers/media/common/videobuf2/videobuf2-core.c
> index 13c9d9e243dd..6fa46bfc620f 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -1351,6 +1351,21 @@ bool vb2_core_request_has_buffers(struct
> media_request *req)
>  }
>  EXPORT_SYMBOL_GPL(vb2_core_request_has_buffers);
>  
> +void *vb2_core_request_find_buffer_priv(struct media_request *req)
> +{
> +	struct media_request_object *obj;
> +	struct vb2_buffer *vb;
> +
> +	obj = media_request_object_find(req, &vb2_core_req_ops,
> NULL);
> +	if (!obj)
> +		return NULL;
> +
> +	vb = container_of(obj, struct vb2_buffer, req_obj);
> +
> +	return vb2_get_drv_priv(vb->vb2_queue);
> +}
> +EXPORT_SYMBOL_GPL(vb2_core_request_find_buffer_priv);
> +
>  int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
> void *pb,
>  			 struct media_request *req)
>  {
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-
> core.h
> index 032bd1bec555..65c0cf6afb55 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -1153,4 +1153,5 @@ int vb2_verify_memory_type(struct vb2_queue *q,
>  		enum vb2_memory memory, unsigned int type);
>  
>  bool vb2_core_request_has_buffers(struct media_request *req);
> +void *vb2_core_request_find_buffer_priv(struct media_request *req);
>  #endif /* _MEDIA_VIDEOBUF2_CORE_H */
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 03/10] videobuf2-core: Add helper to get buffer private data from media request

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 2714 bytes --]

Hi,

On Thu, 2018-04-19 at 17:41 +0200, Paul Kocialkowski wrote:
> When calling media operation driver callbacks related to media
> requests,
> only a pointer to the request itself is provided, which is
> insufficient
> to retrieve the driver's context. Since the driver context is usually
> set as vb2 queue private data and given that the core can determine
> which objects attached to the request are buffers, it is possible to
> extract the associated private data for the first buffer found.
> 
> This is required in order to access the current m2m context from m2m
> drivers' private data in the context of media request operation
> callbacks. More specifically, this allows scheduling m2m device runs
> from the newly-introduced request complete operation.

This patch too will be dropped since it's no longer useful with the
latest version of the request API.

> Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> ---
>  drivers/media/common/videobuf2/videobuf2-core.c | 15 +++++++++++++++
>  include/media/videobuf2-core.h                  |  1 +
>  2 files changed, 16 insertions(+)
> 
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> b/drivers/media/common/videobuf2/videobuf2-core.c
> index 13c9d9e243dd..6fa46bfc620f 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -1351,6 +1351,21 @@ bool vb2_core_request_has_buffers(struct
> media_request *req)
>  }
>  EXPORT_SYMBOL_GPL(vb2_core_request_has_buffers);
>  
> +void *vb2_core_request_find_buffer_priv(struct media_request *req)
> +{
> +	struct media_request_object *obj;
> +	struct vb2_buffer *vb;
> +
> +	obj = media_request_object_find(req, &vb2_core_req_ops,
> NULL);
> +	if (!obj)
> +		return NULL;
> +
> +	vb = container_of(obj, struct vb2_buffer, req_obj);
> +
> +	return vb2_get_drv_priv(vb->vb2_queue);
> +}
> +EXPORT_SYMBOL_GPL(vb2_core_request_find_buffer_priv);
> +
>  int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
> void *pb,
>  			 struct media_request *req)
>  {
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-
> core.h
> index 032bd1bec555..65c0cf6afb55 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -1153,4 +1153,5 @@ int vb2_verify_memory_type(struct vb2_queue *q,
>  		enum vb2_memory memory, unsigned int type);
>  
>  bool vb2_core_request_has_buffers(struct media_request *req);
> +void *vb2_core_request_find_buffer_priv(struct media_request *req);
>  #endif /* _MEDIA_VIDEOBUF2_CORE_H */
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[PATCH v2 03/10] videobuf2-core: Add helper to get buffer private data from media request

[Paul Kocialkowski]

Hi,

On Thu, 2018-04-19 at 17:41 +0200, Paul Kocialkowski wrote:
> When calling media operation driver callbacks related to media
> requests,
> only a pointer to the request itself is provided, which is
> insufficient
> to retrieve the driver's context. Since the driver context is usually
> set as vb2 queue private data and given that the core can determine
> which objects attached to the request are buffers, it is possible to
> extract the associated private data for the first buffer found.
> 
> This is required in order to access the current m2m context from m2m
> drivers' private data in the context of media request operation
> callbacks. More specifically, this allows scheduling m2m device runs
> from the newly-introduced request complete operation.

This patch too will be dropped since it's no longer useful with the
latest version of the request API.

> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> ---
>  drivers/media/common/videobuf2/videobuf2-core.c | 15 +++++++++++++++
>  include/media/videobuf2-core.h                  |  1 +
>  2 files changed, 16 insertions(+)
> 
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c
> b/drivers/media/common/videobuf2/videobuf2-core.c
> index 13c9d9e243dd..6fa46bfc620f 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -1351,6 +1351,21 @@ bool vb2_core_request_has_buffers(struct
> media_request *req)
>  }
>  EXPORT_SYMBOL_GPL(vb2_core_request_has_buffers);
>  
> +void *vb2_core_request_find_buffer_priv(struct media_request *req)
> +{
> +	struct media_request_object *obj;
> +	struct vb2_buffer *vb;
> +
> +	obj = media_request_object_find(req, &vb2_core_req_ops,
> NULL);
> +	if (!obj)
> +		return NULL;
> +
> +	vb = container_of(obj, struct vb2_buffer, req_obj);
> +
> +	return vb2_get_drv_priv(vb->vb2_queue);
> +}
> +EXPORT_SYMBOL_GPL(vb2_core_request_find_buffer_priv);
> +
>  int vb2_core_prepare_buf(struct vb2_queue *q, unsigned int index,
> void *pb,
>  			 struct media_request *req)
>  {
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-
> core.h
> index 032bd1bec555..65c0cf6afb55 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -1153,4 +1153,5 @@ int vb2_verify_memory_type(struct vb2_queue *q,
>  		enum vb2_memory memory, unsigned int type);
>  
>  bool vb2_core_request_has_buffers(struct media_request *req);
> +void *vb2_core_request_find_buffer_priv(struct media_request *req);
>  #endif /* _MEDIA_VIDEOBUF2_CORE_H */
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/0d885d22/attachment.sig>

Re: [PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 1391 bytes --]

On Fri, May 04, 2018 at 09:56:20AM +0200, Paul Kocialkowski wrote:
> > > I agree that the term VPU is more commonly associated with video
> > > decoding, while video engine could mean a number of things.
> > > 
> > > The reason I went with "video-engine" here (while still presenting
> > > the
> > > driver as a VPU driver) is that Video Engine is the term used in
> > > Allwinner's litterature. Other nodes in Allwinner device-trees
> > > generally
> > > stick to these terms (for instance, we have "display-engine" nodes).
> > > This also makes it easier to find the matching parts in the
> > > documentation.
> > 
> > 'video-codec' is what is defined in the DT spec.
> 
> Is that an actively-enforced guideline or a suggestion? I'd like to keep
> video-engine just to stick with the technical documentation wording and
> my personal taste is also to prefer vpu over video-codec (in terms of
> clarity/straightforwardness) as a second choice.
> 
> Still, if the choice isn't up to me, we can go with video-codec (or
> vpu).

The unit-name is supposed to reflect the class of the device, and
nothing else. If there's already a pre-existing class name defined for
these kind of devices, then there's no point in choosing something
else.

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 1356 bytes --]

On Fri, May 04, 2018 at 09:56:20AM +0200, Paul Kocialkowski wrote:
> > > I agree that the term VPU is more commonly associated with video
> > > decoding, while video engine could mean a number of things.
> > > 
> > > The reason I went with "video-engine" here (while still presenting
> > > the
> > > driver as a VPU driver) is that Video Engine is the term used in
> > > Allwinner's litterature. Other nodes in Allwinner device-trees
> > > generally
> > > stick to these terms (for instance, we have "display-engine" nodes).
> > > This also makes it easier to find the matching parts in the
> > > documentation.
> > 
> > 'video-codec' is what is defined in the DT spec.
> 
> Is that an actively-enforced guideline or a suggestion? I'd like to keep
> video-engine just to stick with the technical documentation wording and
> my personal taste is also to prefer vpu over video-codec (in terms of
> clarity/straightforwardness) as a second choice.
> 
> Still, if the choice isn't up to me, we can go with video-codec (or
> vpu).

The unit-name is supposed to reflect the class of the device, and
nothing else. If there's already a pre-existing class name defined for
these kind of devices, then there's no point in choosing something
else.

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[PATCH v2 08/10] dt-bindings: media: Document bindings for the Sunxi-Cedrus VPU driver

[Maxime Ripard]

On Fri, May 04, 2018 at 09:56:20AM +0200, Paul Kocialkowski wrote:
> > > I agree that the term VPU is more commonly associated with video
> > > decoding, while video engine could mean a number of things.
> > > 
> > > The reason I went with "video-engine" here (while still presenting
> > > the
> > > driver as a VPU driver) is that Video Engine is the term used in
> > > Allwinner's litterature. Other nodes in Allwinner device-trees
> > > generally
> > > stick to these terms (for instance, we have "display-engine" nodes).
> > > This also makes it easier to find the matching parts in the
> > > documentation.
> > 
> > 'video-codec' is what is defined in the DT spec.
> 
> Is that an actively-enforced guideline or a suggestion? I'd like to keep
> video-engine just to stick with the technical documentation wording and
> my personal taste is also to prefer vpu over video-codec (in terms of
> clarity/straightforwardness) as a second choice.
> 
> Still, if the choice isn't up to me, we can go with video-codec (or
> vpu).

The unit-name is supposed to reflect the class of the device, and
nothing else. If there's already a pre-existing class name defined for
these kind of devices, then there's no point in choosing something
else.

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/edc240ab/attachment.sig>

Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 7363 bytes --]

Hi,

On Tue, 2018-04-24 at 12:01 +0300, Sakari Ailus wrote:
> Hi Paul,
> 
> On Thu, Apr 19, 2018 at 05:45:31PM +0200, Paul Kocialkowski wrote:
> > Stateless video decoding engines require both the MPEG slices and
> > associated metadata from the video stream in order to decode frames.
> > 
> > This introduces definitions for a new pixel format, describing
> > buffers
> > with MPEG2 slice data, as well as a control structure for passing
> > the
> > frame header (metadata) to drivers.
> 
> What's the typical relationship between MPEG2 slice data and the
> header?
> 
> Are the two always used together, do they originate from the same
> source, for instance? I have to admit I'm not very familiar with
> MPEG...

Yes, the header is closely related to the slice data, as it expresses
the metadata required for properly decoding the slice data. Both are
extracted from the MPEG bitstream and there is dedicated metadata for
each new set of slices that is decoded to a single frame.

> > Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> > Signed-off-by: Florent Revest <florent.revest@free-electrons.com>
> > ---
> >  drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
> >  drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
> >  include/uapi/linux/v4l2-controls.h   | 26
> > ++++++++++++++++++++++++++
> >  include/uapi/linux/videodev2.h       |  3 +++
> >  4 files changed, 40 insertions(+)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> > b/drivers/media/v4l2-core/v4l2-ctrls.c
> > index ba05a8b9a095..fcdc12b9a9e0 100644
> > --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> > +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> > @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
> >  	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		
> > return "Vertical MV Search Range";
> >  	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		
> > return "Repeat Sequence Header";
> >  	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		re
> > turn "Force Key Frame";
> > +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		re
> > turn "MPEG2 Frame Header";
> >  
> >  	/* VPX controls */
> >  	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		
> > return "VPX Number of Partitions";
> > @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name,
> > enum v4l2_ctrl_type *type,
> >  	case V4L2_CID_RDS_TX_ALT_FREQS:
> >  		*type = V4L2_CTRL_TYPE_U32;
> >  		break;
> > +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> > +		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> > +		break;
> >  	default:
> >  		*type = V4L2_CTRL_TYPE_INTEGER;
> >  		break;
> > @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl
> > *ctrl, u32 idx,
> >  			return -ERANGE;
> >  		return 0;
> >  
> > +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> > +		return 0;
> > +
> >  	default:
> >  		return -EINVAL;
> >  	}
> > @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct
> > v4l2_ctrl_handler *hdl,
> >  	case V4L2_CTRL_TYPE_U32:
> >  		elem_size = sizeof(u32);
> >  		break;
> > +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> > +		elem_size = sizeof(struct
> > v4l2_ctrl_mpeg2_frame_hdr);
> > +		break;
> >  	default:
> >  		if (type < V4L2_CTRL_COMPOUND_TYPES)
> >  			elem_size = sizeof(s32);
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> > b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index 468c3c65362d..8070203da5d2 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct
> > v4l2_fmtdesc *fmt)
> >  		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1 
> > (SMPTE 412M Annex L)"; break;
> >  		case V4L2_PIX_FMT_VP8:		descr =
> > "VP8"; break;
> >  		case V4L2_PIX_FMT_VP9:		descr =
> > "VP9"; break;
> > +		case V4L2_PIX_FMT_MPEG2_FRAME:	descr =
> > "MPEG2 Frame"; break;
> >  		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA
> > YUV"; break;
> >  		case V4L2_PIX_FMT_WNVA:		descr =
> > "WNVA"; break;
> >  		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA
> > SN9C10X"; break;
> > diff --git a/include/uapi/linux/v4l2-controls.h
> > b/include/uapi/linux/v4l2-controls.h
> > index cbbb750d87d1..8431b2a540c7 100644
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
> >  };
> >  #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPE
> > G_BASE+407)
> >  
> > +#define
> > V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
> > +
> >  /*  Control IDs for VP8 streams
> >   *  Although VP8 is not part of MPEG we add these controls to the
> > MPEG class
> >   *  as that class is already handling other video compression
> > standards
> > @@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
> >  #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_C
> > LASS_BASE + 3)
> >  #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DET
> > ECT_CLASS_BASE + 4)
> >  
> > +struct v4l2_ctrl_mpeg2_frame_hdr {
> > +	__u32 slice_len;
> > +	__u32 slice_pos;
> > +	enum { MPEG1, MPEG2 } type;
> > +
> > +	__u16 width;
> > +	__u16 height;
> > +
> > +	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D }
> > picture_coding_type;
> > +	__u8 f_code[2][2];
> > +
> > +	__u8 intra_dc_precision;
> > +	__u8 picture_structure;
> > +	__u8 top_field_first;
> > +	__u8 frame_pred_frame_dct;
> > +	__u8 concealment_motion_vectors;
> > +	__u8 q_scale_type;
> > +	__u8 intra_vlc_format;
> > +	__u8 alternate_scan;
> > +
> > +	__u8 backward_ref_index;
> > +	__u8 forward_ref_index;
> > +};
> 
> Besides the comments that have been given, please add
> __attribute__((packed)) to the struct definition.

Noted.

> > +
> >  #endif
> > diff --git a/include/uapi/linux/videodev2.h
> > b/include/uapi/linux/videodev2.h
> > index 31b5728b56e9..4b8336f7bcf0 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -635,6 +635,7 @@ struct v4l2_pix_format {
> >  #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /*
> > SMPTE 421M Annex L compliant stream */
> >  #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /*
> > VP8 */
> >  #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /*
> > VP9 */
> > +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /*
> > MPEG2 frame */
> 
> This format needs documentation.

Noted too. Thanks for the review!

> >  
> >  /*  Vendor-specific formats   */
> >  #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /*
> > cpia1 YUV */
> > @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
> >  		__u8 __user *p_u8;
> >  		__u16 __user *p_u16;
> >  		__u32 __user *p_u32;
> > +		struct v4l2_ctrl_mpeg2_frame_hdr __user
> > *p_mpeg2_frame_hdr;
> >  		void __user *ptr;
> >  	};
> >  } __attribute__ ((packed));
> > @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
> >  	V4L2_CTRL_TYPE_U8	     = 0x0100,
> >  	V4L2_CTRL_TYPE_U16	     = 0x0101,
> >  	V4L2_CTRL_TYPE_U32	     = 0x0102,
> > +	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
> >  };
> >  
> >  /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
> > -- 
> > 2.16.3
> > 
> 
> 
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 7208 bytes --]

Hi,

On Tue, 2018-04-24 at 12:01 +0300, Sakari Ailus wrote:
> Hi Paul,
> 
> On Thu, Apr 19, 2018 at 05:45:31PM +0200, Paul Kocialkowski wrote:
> > Stateless video decoding engines require both the MPEG slices and
> > associated metadata from the video stream in order to decode frames.
> > 
> > This introduces definitions for a new pixel format, describing
> > buffers
> > with MPEG2 slice data, as well as a control structure for passing
> > the
> > frame header (metadata) to drivers.
> 
> What's the typical relationship between MPEG2 slice data and the
> header?
> 
> Are the two always used together, do they originate from the same
> source, for instance? I have to admit I'm not very familiar with
> MPEG...

Yes, the header is closely related to the slice data, as it expresses
the metadata required for properly decoding the slice data. Both are
extracted from the MPEG bitstream and there is dedicated metadata for
each new set of slices that is decoded to a single frame.

> > Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> > Signed-off-by: Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
> > ---
> >  drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
> >  drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
> >  include/uapi/linux/v4l2-controls.h   | 26
> > ++++++++++++++++++++++++++
> >  include/uapi/linux/videodev2.h       |  3 +++
> >  4 files changed, 40 insertions(+)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> > b/drivers/media/v4l2-core/v4l2-ctrls.c
> > index ba05a8b9a095..fcdc12b9a9e0 100644
> > --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> > +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> > @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
> >  	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		
> > return "Vertical MV Search Range";
> >  	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		
> > return "Repeat Sequence Header";
> >  	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		re
> > turn "Force Key Frame";
> > +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		re
> > turn "MPEG2 Frame Header";
> >  
> >  	/* VPX controls */
> >  	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		
> > return "VPX Number of Partitions";
> > @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name,
> > enum v4l2_ctrl_type *type,
> >  	case V4L2_CID_RDS_TX_ALT_FREQS:
> >  		*type = V4L2_CTRL_TYPE_U32;
> >  		break;
> > +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> > +		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> > +		break;
> >  	default:
> >  		*type = V4L2_CTRL_TYPE_INTEGER;
> >  		break;
> > @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl
> > *ctrl, u32 idx,
> >  			return -ERANGE;
> >  		return 0;
> >  
> > +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> > +		return 0;
> > +
> >  	default:
> >  		return -EINVAL;
> >  	}
> > @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct
> > v4l2_ctrl_handler *hdl,
> >  	case V4L2_CTRL_TYPE_U32:
> >  		elem_size = sizeof(u32);
> >  		break;
> > +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> > +		elem_size = sizeof(struct
> > v4l2_ctrl_mpeg2_frame_hdr);
> > +		break;
> >  	default:
> >  		if (type < V4L2_CTRL_COMPOUND_TYPES)
> >  			elem_size = sizeof(s32);
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> > b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index 468c3c65362d..8070203da5d2 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct
> > v4l2_fmtdesc *fmt)
> >  		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1 
> > (SMPTE 412M Annex L)"; break;
> >  		case V4L2_PIX_FMT_VP8:		descr =
> > "VP8"; break;
> >  		case V4L2_PIX_FMT_VP9:		descr =
> > "VP9"; break;
> > +		case V4L2_PIX_FMT_MPEG2_FRAME:	descr =
> > "MPEG2 Frame"; break;
> >  		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA
> > YUV"; break;
> >  		case V4L2_PIX_FMT_WNVA:		descr =
> > "WNVA"; break;
> >  		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA
> > SN9C10X"; break;
> > diff --git a/include/uapi/linux/v4l2-controls.h
> > b/include/uapi/linux/v4l2-controls.h
> > index cbbb750d87d1..8431b2a540c7 100644
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
> >  };
> >  #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPE
> > G_BASE+407)
> >  
> > +#define
> > V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
> > +
> >  /*  Control IDs for VP8 streams
> >   *  Although VP8 is not part of MPEG we add these controls to the
> > MPEG class
> >   *  as that class is already handling other video compression
> > standards
> > @@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
> >  #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_C
> > LASS_BASE + 3)
> >  #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DET
> > ECT_CLASS_BASE + 4)
> >  
> > +struct v4l2_ctrl_mpeg2_frame_hdr {
> > +	__u32 slice_len;
> > +	__u32 slice_pos;
> > +	enum { MPEG1, MPEG2 } type;
> > +
> > +	__u16 width;
> > +	__u16 height;
> > +
> > +	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D }
> > picture_coding_type;
> > +	__u8 f_code[2][2];
> > +
> > +	__u8 intra_dc_precision;
> > +	__u8 picture_structure;
> > +	__u8 top_field_first;
> > +	__u8 frame_pred_frame_dct;
> > +	__u8 concealment_motion_vectors;
> > +	__u8 q_scale_type;
> > +	__u8 intra_vlc_format;
> > +	__u8 alternate_scan;
> > +
> > +	__u8 backward_ref_index;
> > +	__u8 forward_ref_index;
> > +};
> 
> Besides the comments that have been given, please add
> __attribute__((packed)) to the struct definition.

Noted.

> > +
> >  #endif
> > diff --git a/include/uapi/linux/videodev2.h
> > b/include/uapi/linux/videodev2.h
> > index 31b5728b56e9..4b8336f7bcf0 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -635,6 +635,7 @@ struct v4l2_pix_format {
> >  #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /*
> > SMPTE 421M Annex L compliant stream */
> >  #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /*
> > VP8 */
> >  #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /*
> > VP9 */
> > +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /*
> > MPEG2 frame */
> 
> This format needs documentation.

Noted too. Thanks for the review!

> >  
> >  /*  Vendor-specific formats   */
> >  #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /*
> > cpia1 YUV */
> > @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
> >  		__u8 __user *p_u8;
> >  		__u16 __user *p_u16;
> >  		__u32 __user *p_u32;
> > +		struct v4l2_ctrl_mpeg2_frame_hdr __user
> > *p_mpeg2_frame_hdr;
> >  		void __user *ptr;
> >  	};
> >  } __attribute__ ((packed));
> > @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
> >  	V4L2_CTRL_TYPE_U8	     = 0x0100,
> >  	V4L2_CTRL_TYPE_U16	     = 0x0101,
> >  	V4L2_CTRL_TYPE_U32	     = 0x0102,
> > +	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
> >  };
> >  
> >  /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
> > -- 
> > 2.16.3
> > 
> 
> 
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Paul Kocialkowski]

Hi,

On Tue, 2018-04-24 at 12:01 +0300, Sakari Ailus wrote:
> Hi Paul,
> 
> On Thu, Apr 19, 2018 at 05:45:31PM +0200, Paul Kocialkowski wrote:
> > Stateless video decoding engines require both the MPEG slices and
> > associated metadata from the video stream in order to decode frames.
> > 
> > This introduces definitions for a new pixel format, describing
> > buffers
> > with MPEG2 slice data, as well as a control structure for passing
> > the
> > frame header (metadata) to drivers.
> 
> What's the typical relationship between MPEG2 slice data and the
> header?
> 
> Are the two always used together, do they originate from the same
> source, for instance? I have to admit I'm not very familiar with
> MPEG...

Yes, the header is closely related to the slice data, as it expresses
the metadata required for properly decoding the slice data. Both are
extracted from the MPEG bitstream and there is dedicated metadata for
each new set of slices that is decoded to a single frame.

> > Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> > Signed-off-by: Florent Revest <florent.revest@free-electrons.com>
> > ---
> >  drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
> >  drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
> >  include/uapi/linux/v4l2-controls.h   | 26
> > ++++++++++++++++++++++++++
> >  include/uapi/linux/videodev2.h       |  3 +++
> >  4 files changed, 40 insertions(+)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> > b/drivers/media/v4l2-core/v4l2-ctrls.c
> > index ba05a8b9a095..fcdc12b9a9e0 100644
> > --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> > +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> > @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
> >  	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		
> > return "Vertical MV Search Range";
> >  	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		
> > return "Repeat Sequence Header";
> >  	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		re
> > turn "Force Key Frame";
> > +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		re
> > turn "MPEG2 Frame Header";
> >  
> >  	/* VPX controls */
> >  	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		
> > return "VPX Number of Partitions";
> > @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name,
> > enum v4l2_ctrl_type *type,
> >  	case V4L2_CID_RDS_TX_ALT_FREQS:
> >  		*type = V4L2_CTRL_TYPE_U32;
> >  		break;
> > +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> > +		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> > +		break;
> >  	default:
> >  		*type = V4L2_CTRL_TYPE_INTEGER;
> >  		break;
> > @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl
> > *ctrl, u32 idx,
> >  			return -ERANGE;
> >  		return 0;
> >  
> > +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> > +		return 0;
> > +
> >  	default:
> >  		return -EINVAL;
> >  	}
> > @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct
> > v4l2_ctrl_handler *hdl,
> >  	case V4L2_CTRL_TYPE_U32:
> >  		elem_size = sizeof(u32);
> >  		break;
> > +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> > +		elem_size = sizeof(struct
> > v4l2_ctrl_mpeg2_frame_hdr);
> > +		break;
> >  	default:
> >  		if (type < V4L2_CTRL_COMPOUND_TYPES)
> >  			elem_size = sizeof(s32);
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> > b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index 468c3c65362d..8070203da5d2 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct
> > v4l2_fmtdesc *fmt)
> >  		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1 
> > (SMPTE 412M Annex L)"; break;
> >  		case V4L2_PIX_FMT_VP8:		descr =
> > "VP8"; break;
> >  		case V4L2_PIX_FMT_VP9:		descr =
> > "VP9"; break;
> > +		case V4L2_PIX_FMT_MPEG2_FRAME:	descr =
> > "MPEG2 Frame"; break;
> >  		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA
> > YUV"; break;
> >  		case V4L2_PIX_FMT_WNVA:		descr =
> > "WNVA"; break;
> >  		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA
> > SN9C10X"; break;
> > diff --git a/include/uapi/linux/v4l2-controls.h
> > b/include/uapi/linux/v4l2-controls.h
> > index cbbb750d87d1..8431b2a540c7 100644
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
> >  };
> >  #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPE
> > G_BASE+407)
> >  
> > +#define
> > V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
> > +
> >  /*  Control IDs for VP8 streams
> >   *  Although VP8 is not part of MPEG we add these controls to the
> > MPEG class
> >   *  as that class is already handling other video compression
> > standards
> > @@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
> >  #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_C
> > LASS_BASE + 3)
> >  #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DET
> > ECT_CLASS_BASE + 4)
> >  
> > +struct v4l2_ctrl_mpeg2_frame_hdr {
> > +	__u32 slice_len;
> > +	__u32 slice_pos;
> > +	enum { MPEG1, MPEG2 } type;
> > +
> > +	__u16 width;
> > +	__u16 height;
> > +
> > +	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D }
> > picture_coding_type;
> > +	__u8 f_code[2][2];
> > +
> > +	__u8 intra_dc_precision;
> > +	__u8 picture_structure;
> > +	__u8 top_field_first;
> > +	__u8 frame_pred_frame_dct;
> > +	__u8 concealment_motion_vectors;
> > +	__u8 q_scale_type;
> > +	__u8 intra_vlc_format;
> > +	__u8 alternate_scan;
> > +
> > +	__u8 backward_ref_index;
> > +	__u8 forward_ref_index;
> > +};
> 
> Besides the comments that have been given, please add
> __attribute__((packed)) to the struct definition.

Noted.

> > +
> >  #endif
> > diff --git a/include/uapi/linux/videodev2.h
> > b/include/uapi/linux/videodev2.h
> > index 31b5728b56e9..4b8336f7bcf0 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -635,6 +635,7 @@ struct v4l2_pix_format {
> >  #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /*
> > SMPTE 421M Annex L compliant stream */
> >  #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /*
> > VP8 */
> >  #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /*
> > VP9 */
> > +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /*
> > MPEG2 frame */
> 
> This format needs documentation.

Noted too. Thanks for the review!

> >  
> >  /*  Vendor-specific formats   */
> >  #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /*
> > cpia1 YUV */
> > @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
> >  		__u8 __user *p_u8;
> >  		__u16 __user *p_u16;
> >  		__u32 __user *p_u32;
> > +		struct v4l2_ctrl_mpeg2_frame_hdr __user
> > *p_mpeg2_frame_hdr;
> >  		void __user *ptr;
> >  	};
> >  } __attribute__ ((packed));
> > @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
> >  	V4L2_CTRL_TYPE_U8	     = 0x0100,
> >  	V4L2_CTRL_TYPE_U16	     = 0x0101,
> >  	V4L2_CTRL_TYPE_U32	     = 0x0102,
> > +	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
> >  };
> >  
> >  /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
> > -- 
> > 2.16.3
> > 
> 
> 
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/618ed7c4/attachment-0001.sig>

Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 7055 bytes --]

Hi,

On Fri, 2018-04-20 at 15:57 +0200, Hans Verkuil wrote:
> On 04/19/18 17:45, Paul Kocialkowski wrote:
> > Stateless video decoding engines require both the MPEG slices and
> > associated metadata from the video stream in order to decode frames.
> > 
> > This introduces definitions for a new pixel format, describing
> > buffers
> > with MPEG2 slice data, as well as a control structure for passing
> > the
> > frame header (metadata) to drivers.
> > 
> > Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> > Signed-off-by: Florent Revest <florent.revest@free-electrons.com>
> > ---
> >  drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
> >  drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
> >  include/uapi/linux/v4l2-controls.h   | 26
> > ++++++++++++++++++++++++++
> >  include/uapi/linux/videodev2.h       |  3 +++
> >  4 files changed, 40 insertions(+)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> > b/drivers/media/v4l2-core/v4l2-ctrls.c
> > index ba05a8b9a095..fcdc12b9a9e0 100644
> > --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> > +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> > @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
> >  	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		
> > return "Vertical MV Search Range";
> >  	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		
> > return "Repeat Sequence Header";
> >  	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		re
> > turn "Force Key Frame";
> > +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		re
> > turn "MPEG2 Frame Header";
> >  
> >  	/* VPX controls */
> >  	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		
> > return "VPX Number of Partitions";
> > @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name,
> > enum v4l2_ctrl_type *type,
> >  	case V4L2_CID_RDS_TX_ALT_FREQS:
> >  		*type = V4L2_CTRL_TYPE_U32;
> >  		break;
> > +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> > +		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> > +		break;
> >  	default:
> >  		*type = V4L2_CTRL_TYPE_INTEGER;
> >  		break;
> > @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl
> > *ctrl, u32 idx,
> >  			return -ERANGE;
> >  		return 0;
> >  
> > +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> > +		return 0;
> > +
> >  	default:
> >  		return -EINVAL;
> >  	}
> > @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct
> > v4l2_ctrl_handler *hdl,
> >  	case V4L2_CTRL_TYPE_U32:
> >  		elem_size = sizeof(u32);
> >  		break;
> > +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> > +		elem_size = sizeof(struct
> > v4l2_ctrl_mpeg2_frame_hdr);
> > +		break;
> >  	default:
> >  		if (type < V4L2_CTRL_COMPOUND_TYPES)
> >  			elem_size = sizeof(s32);
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> > b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index 468c3c65362d..8070203da5d2 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct
> > v4l2_fmtdesc *fmt)
> >  		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1 
> > (SMPTE 412M Annex L)"; break;
> >  		case V4L2_PIX_FMT_VP8:		descr =
> > "VP8"; break;
> >  		case V4L2_PIX_FMT_VP9:		descr =
> > "VP9"; break;
> > +		case V4L2_PIX_FMT_MPEG2_FRAME:	descr =
> > "MPEG2 Frame"; break;
> >  		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA
> > YUV"; break;
> >  		case V4L2_PIX_FMT_WNVA:		descr =
> > "WNVA"; break;
> >  		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA
> > SN9C10X"; break;
> > diff --git a/include/uapi/linux/v4l2-controls.h
> > b/include/uapi/linux/v4l2-controls.h
> > index cbbb750d87d1..8431b2a540c7 100644
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
> >  };
> >  #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPE
> > G_BASE+407)
> >  
> > +#define
> > V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
> > +
> >  /*  Control IDs for VP8 streams
> >   *  Although VP8 is not part of MPEG we add these controls to the
> > MPEG class
> >   *  as that class is already handling other video compression
> > standards
> > @@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
> >  #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_C
> > LASS_BASE + 3)
> >  #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DET
> > ECT_CLASS_BASE + 4)
> >  
> > +struct v4l2_ctrl_mpeg2_frame_hdr {
> > +	__u32 slice_len;
> > +	__u32 slice_pos;
> > +	enum { MPEG1, MPEG2 } type;
> > +
> > +	__u16 width;
> > +	__u16 height;
> > +
> > +	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D }
> > picture_coding_type;
> 
> As someone else already mentioned (I believe): avoid enums. Use __u16
> instead.

Indeed. I have taken a note of that.

> > +	__u8 f_code[2][2];
> > +
> > +	__u8 intra_dc_precision;
> > +	__u8 picture_structure;
> > +	__u8 top_field_first;
> > +	__u8 frame_pred_frame_dct;
> > +	__u8 concealment_motion_vectors;
> > +	__u8 q_scale_type;
> > +	__u8 intra_vlc_format;
> > +	__u8 alternate_scan;
> > +
> > +	__u8 backward_ref_index;
> > +	__u8 forward_ref_index;
> > +};
> 
> Please test that the layout and size of this structure is identical
> for 32 and 64 bit
> architectures, both on Intel and ARM.
> 
> It looks good, but it doesn't hurt to check.

Right, I'll keep that in mind and make sure this is the case.

Cheers and thanks for the review!

Paul

> Regards,
> 
> 	Hans
> 
> > +
> >  #endif
> > diff --git a/include/uapi/linux/videodev2.h
> > b/include/uapi/linux/videodev2.h
> > index 31b5728b56e9..4b8336f7bcf0 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -635,6 +635,7 @@ struct v4l2_pix_format {
> >  #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /*
> > SMPTE 421M Annex L compliant stream */
> >  #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /*
> > VP8 */
> >  #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /*
> > VP9 */
> > +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /*
> > MPEG2 frame */
> >  
> >  /*  Vendor-specific formats   */
> >  #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /*
> > cpia1 YUV */
> > @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
> >  		__u8 __user *p_u8;
> >  		__u16 __user *p_u16;
> >  		__u32 __user *p_u32;
> > +		struct v4l2_ctrl_mpeg2_frame_hdr __user
> > *p_mpeg2_frame_hdr;
> >  		void __user *ptr;
> >  	};
> >  } __attribute__ ((packed));
> > @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
> >  	V4L2_CTRL_TYPE_U8	     = 0x0100,
> >  	V4L2_CTRL_TYPE_U16	     = 0x0101,
> >  	V4L2_CTRL_TYPE_U32	     = 0x0102,
> > +	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
> >  };
> >  
> >  /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
> > 
> 
> 
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 6904 bytes --]

Hi,

On Fri, 2018-04-20 at 15:57 +0200, Hans Verkuil wrote:
> On 04/19/18 17:45, Paul Kocialkowski wrote:
> > Stateless video decoding engines require both the MPEG slices and
> > associated metadata from the video stream in order to decode frames.
> > 
> > This introduces definitions for a new pixel format, describing
> > buffers
> > with MPEG2 slice data, as well as a control structure for passing
> > the
> > frame header (metadata) to drivers.
> > 
> > Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> > Signed-off-by: Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
> > ---
> >  drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
> >  drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
> >  include/uapi/linux/v4l2-controls.h   | 26
> > ++++++++++++++++++++++++++
> >  include/uapi/linux/videodev2.h       |  3 +++
> >  4 files changed, 40 insertions(+)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> > b/drivers/media/v4l2-core/v4l2-ctrls.c
> > index ba05a8b9a095..fcdc12b9a9e0 100644
> > --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> > +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> > @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
> >  	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		
> > return "Vertical MV Search Range";
> >  	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		
> > return "Repeat Sequence Header";
> >  	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		re
> > turn "Force Key Frame";
> > +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		re
> > turn "MPEG2 Frame Header";
> >  
> >  	/* VPX controls */
> >  	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		
> > return "VPX Number of Partitions";
> > @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name,
> > enum v4l2_ctrl_type *type,
> >  	case V4L2_CID_RDS_TX_ALT_FREQS:
> >  		*type = V4L2_CTRL_TYPE_U32;
> >  		break;
> > +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> > +		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> > +		break;
> >  	default:
> >  		*type = V4L2_CTRL_TYPE_INTEGER;
> >  		break;
> > @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl
> > *ctrl, u32 idx,
> >  			return -ERANGE;
> >  		return 0;
> >  
> > +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> > +		return 0;
> > +
> >  	default:
> >  		return -EINVAL;
> >  	}
> > @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct
> > v4l2_ctrl_handler *hdl,
> >  	case V4L2_CTRL_TYPE_U32:
> >  		elem_size = sizeof(u32);
> >  		break;
> > +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> > +		elem_size = sizeof(struct
> > v4l2_ctrl_mpeg2_frame_hdr);
> > +		break;
> >  	default:
> >  		if (type < V4L2_CTRL_COMPOUND_TYPES)
> >  			elem_size = sizeof(s32);
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> > b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index 468c3c65362d..8070203da5d2 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct
> > v4l2_fmtdesc *fmt)
> >  		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1 
> > (SMPTE 412M Annex L)"; break;
> >  		case V4L2_PIX_FMT_VP8:		descr =
> > "VP8"; break;
> >  		case V4L2_PIX_FMT_VP9:		descr =
> > "VP9"; break;
> > +		case V4L2_PIX_FMT_MPEG2_FRAME:	descr =
> > "MPEG2 Frame"; break;
> >  		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA
> > YUV"; break;
> >  		case V4L2_PIX_FMT_WNVA:		descr =
> > "WNVA"; break;
> >  		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA
> > SN9C10X"; break;
> > diff --git a/include/uapi/linux/v4l2-controls.h
> > b/include/uapi/linux/v4l2-controls.h
> > index cbbb750d87d1..8431b2a540c7 100644
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
> >  };
> >  #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPE
> > G_BASE+407)
> >  
> > +#define
> > V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
> > +
> >  /*  Control IDs for VP8 streams
> >   *  Although VP8 is not part of MPEG we add these controls to the
> > MPEG class
> >   *  as that class is already handling other video compression
> > standards
> > @@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
> >  #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_C
> > LASS_BASE + 3)
> >  #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DET
> > ECT_CLASS_BASE + 4)
> >  
> > +struct v4l2_ctrl_mpeg2_frame_hdr {
> > +	__u32 slice_len;
> > +	__u32 slice_pos;
> > +	enum { MPEG1, MPEG2 } type;
> > +
> > +	__u16 width;
> > +	__u16 height;
> > +
> > +	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D }
> > picture_coding_type;
> 
> As someone else already mentioned (I believe): avoid enums. Use __u16
> instead.

Indeed. I have taken a note of that.

> > +	__u8 f_code[2][2];
> > +
> > +	__u8 intra_dc_precision;
> > +	__u8 picture_structure;
> > +	__u8 top_field_first;
> > +	__u8 frame_pred_frame_dct;
> > +	__u8 concealment_motion_vectors;
> > +	__u8 q_scale_type;
> > +	__u8 intra_vlc_format;
> > +	__u8 alternate_scan;
> > +
> > +	__u8 backward_ref_index;
> > +	__u8 forward_ref_index;
> > +};
> 
> Please test that the layout and size of this structure is identical
> for 32 and 64 bit
> architectures, both on Intel and ARM.
> 
> It looks good, but it doesn't hurt to check.

Right, I'll keep that in mind and make sure this is the case.

Cheers and thanks for the review!

Paul

> Regards,
> 
> 	Hans
> 
> > +
> >  #endif
> > diff --git a/include/uapi/linux/videodev2.h
> > b/include/uapi/linux/videodev2.h
> > index 31b5728b56e9..4b8336f7bcf0 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -635,6 +635,7 @@ struct v4l2_pix_format {
> >  #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /*
> > SMPTE 421M Annex L compliant stream */
> >  #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /*
> > VP8 */
> >  #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /*
> > VP9 */
> > +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /*
> > MPEG2 frame */
> >  
> >  /*  Vendor-specific formats   */
> >  #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /*
> > cpia1 YUV */
> > @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
> >  		__u8 __user *p_u8;
> >  		__u16 __user *p_u16;
> >  		__u32 __user *p_u32;
> > +		struct v4l2_ctrl_mpeg2_frame_hdr __user
> > *p_mpeg2_frame_hdr;
> >  		void __user *ptr;
> >  	};
> >  } __attribute__ ((packed));
> > @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
> >  	V4L2_CTRL_TYPE_U8	     = 0x0100,
> >  	V4L2_CTRL_TYPE_U16	     = 0x0101,
> >  	V4L2_CTRL_TYPE_U32	     = 0x0102,
> > +	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
> >  };
> >  
> >  /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
> > 
> 
> 
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Paul Kocialkowski]

Hi,

On Fri, 2018-04-20 at 15:57 +0200, Hans Verkuil wrote:
> On 04/19/18 17:45, Paul Kocialkowski wrote:
> > Stateless video decoding engines require both the MPEG slices and
> > associated metadata from the video stream in order to decode frames.
> > 
> > This introduces definitions for a new pixel format, describing
> > buffers
> > with MPEG2 slice data, as well as a control structure for passing
> > the
> > frame header (metadata) to drivers.
> > 
> > Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> > Signed-off-by: Florent Revest <florent.revest@free-electrons.com>
> > ---
> >  drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
> >  drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
> >  include/uapi/linux/v4l2-controls.h   | 26
> > ++++++++++++++++++++++++++
> >  include/uapi/linux/videodev2.h       |  3 +++
> >  4 files changed, 40 insertions(+)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> > b/drivers/media/v4l2-core/v4l2-ctrls.c
> > index ba05a8b9a095..fcdc12b9a9e0 100644
> > --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> > +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> > @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
> >  	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		
> > return "Vertical MV Search Range";
> >  	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		
> > return "Repeat Sequence Header";
> >  	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		re
> > turn "Force Key Frame";
> > +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		re
> > turn "MPEG2 Frame Header";
> >  
> >  	/* VPX controls */
> >  	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		
> > return "VPX Number of Partitions";
> > @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name,
> > enum v4l2_ctrl_type *type,
> >  	case V4L2_CID_RDS_TX_ALT_FREQS:
> >  		*type = V4L2_CTRL_TYPE_U32;
> >  		break;
> > +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> > +		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> > +		break;
> >  	default:
> >  		*type = V4L2_CTRL_TYPE_INTEGER;
> >  		break;
> > @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl
> > *ctrl, u32 idx,
> >  			return -ERANGE;
> >  		return 0;
> >  
> > +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> > +		return 0;
> > +
> >  	default:
> >  		return -EINVAL;
> >  	}
> > @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct
> > v4l2_ctrl_handler *hdl,
> >  	case V4L2_CTRL_TYPE_U32:
> >  		elem_size = sizeof(u32);
> >  		break;
> > +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> > +		elem_size = sizeof(struct
> > v4l2_ctrl_mpeg2_frame_hdr);
> > +		break;
> >  	default:
> >  		if (type < V4L2_CTRL_COMPOUND_TYPES)
> >  			elem_size = sizeof(s32);
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> > b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index 468c3c65362d..8070203da5d2 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct
> > v4l2_fmtdesc *fmt)
> >  		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1 
> > (SMPTE 412M Annex L)"; break;
> >  		case V4L2_PIX_FMT_VP8:		descr =
> > "VP8"; break;
> >  		case V4L2_PIX_FMT_VP9:		descr =
> > "VP9"; break;
> > +		case V4L2_PIX_FMT_MPEG2_FRAME:	descr =
> > "MPEG2 Frame"; break;
> >  		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA
> > YUV"; break;
> >  		case V4L2_PIX_FMT_WNVA:		descr =
> > "WNVA"; break;
> >  		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA
> > SN9C10X"; break;
> > diff --git a/include/uapi/linux/v4l2-controls.h
> > b/include/uapi/linux/v4l2-controls.h
> > index cbbb750d87d1..8431b2a540c7 100644
> > --- a/include/uapi/linux/v4l2-controls.h
> > +++ b/include/uapi/linux/v4l2-controls.h
> > @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
> >  };
> >  #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPE
> > G_BASE+407)
> >  
> > +#define
> > V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
> > +
> >  /*  Control IDs for VP8 streams
> >   *  Although VP8 is not part of MPEG we add these controls to the
> > MPEG class
> >   *  as that class is already handling other video compression
> > standards
> > @@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
> >  #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_C
> > LASS_BASE + 3)
> >  #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DET
> > ECT_CLASS_BASE + 4)
> >  
> > +struct v4l2_ctrl_mpeg2_frame_hdr {
> > +	__u32 slice_len;
> > +	__u32 slice_pos;
> > +	enum { MPEG1, MPEG2 } type;
> > +
> > +	__u16 width;
> > +	__u16 height;
> > +
> > +	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D }
> > picture_coding_type;
> 
> As someone else already mentioned (I believe): avoid enums. Use __u16
> instead.

Indeed. I have taken a note of that.

> > +	__u8 f_code[2][2];
> > +
> > +	__u8 intra_dc_precision;
> > +	__u8 picture_structure;
> > +	__u8 top_field_first;
> > +	__u8 frame_pred_frame_dct;
> > +	__u8 concealment_motion_vectors;
> > +	__u8 q_scale_type;
> > +	__u8 intra_vlc_format;
> > +	__u8 alternate_scan;
> > +
> > +	__u8 backward_ref_index;
> > +	__u8 forward_ref_index;
> > +};
> 
> Please test that the layout and size of this structure is identical
> for 32 and 64 bit
> architectures, both on Intel and ARM.
> 
> It looks good, but it doesn't hurt to check.

Right, I'll keep that in mind and make sure this is the case.

Cheers and thanks for the review!

Paul

> Regards,
> 
> 	Hans
> 
> > +
> >  #endif
> > diff --git a/include/uapi/linux/videodev2.h
> > b/include/uapi/linux/videodev2.h
> > index 31b5728b56e9..4b8336f7bcf0 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -635,6 +635,7 @@ struct v4l2_pix_format {
> >  #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /*
> > SMPTE 421M Annex L compliant stream */
> >  #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /*
> > VP8 */
> >  #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /*
> > VP9 */
> > +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /*
> > MPEG2 frame */
> >  
> >  /*  Vendor-specific formats   */
> >  #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /*
> > cpia1 YUV */
> > @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
> >  		__u8 __user *p_u8;
> >  		__u16 __user *p_u16;
> >  		__u32 __user *p_u32;
> > +		struct v4l2_ctrl_mpeg2_frame_hdr __user
> > *p_mpeg2_frame_hdr;
> >  		void __user *ptr;
> >  	};
> >  } __attribute__ ((packed));
> > @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
> >  	V4L2_CTRL_TYPE_U8	     = 0x0100,
> >  	V4L2_CTRL_TYPE_U16	     = 0x0101,
> >  	V4L2_CTRL_TYPE_U32	     = 0x0102,
> > +	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
> >  };
> >  
> >  /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
> > 
> 
> 
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/629ee026/attachment.sig>

Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 2690 bytes --]

Hi,

On Fri, 2018-04-20 at 09:51 +0000, Tomasz Figa wrote:
> Hi Paul,
> 
> On Fri, Apr 20, 2018 at 12:46 AM Paul Kocialkowski <
> paul.kocialkowski@bootlin.com> wrote:
> [snip]
> > +struct v4l2_ctrl_mpeg2_frame_hdr {
> > +       __u32 slice_len;
> > +       __u32 slice_pos;
> > +       enum { MPEG1, MPEG2 } type;
> 
> Is enum suitable for UAPI?

As it turns out, it's not :)

> > +
> > +       __u16 width;
> > +       __u16 height;
> > +
> > +       enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;
> 
> Ditto.
> 
> > +       __u8 f_code[2][2];
> > +
> > +       __u8 intra_dc_precision;
> > +       __u8 picture_structure;
> > +       __u8 top_field_first;
> > +       __u8 frame_pred_frame_dct;
> > +       __u8 concealment_motion_vectors;
> > +       __u8 q_scale_type;
> > +       __u8 intra_vlc_format;
> > +       __u8 alternate_scan;
> > +
> > +       __u8 backward_ref_index;
> > +       __u8 forward_ref_index;
> > +};
> > +
> >   #endif
> > diff --git a/include/uapi/linux/videodev2.h
> 
> b/include/uapi/linux/videodev2.h
> > index 31b5728b56e9..4b8336f7bcf0 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -635,6 +635,7 @@ struct v4l2_pix_format {
> >   #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L')
> > /*
> 
> SMPTE 421M Annex L compliant stream */
> >   #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /*
> > VP8 */
> >   #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /*
> > VP9 */
> > +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /*
> 
> MPEG2 frame */
> 
> >   /*  Vendor-specific formats   */
> >   #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /*
> > cpia1
> 
> YUV */
> > @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
> >                  __u8 __user *p_u8;
> >                  __u16 __user *p_u16;
> >                  __u32 __user *p_u32;
> > +               struct v4l2_ctrl_mpeg2_frame_hdr __user
> 
> *p_mpeg2_frame_hdr;
> >                  void __user *ptr;
> >          };
> >   } __attribute__ ((packed));
> > @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
> >          V4L2_CTRL_TYPE_U8            = 0x0100,
> >          V4L2_CTRL_TYPE_U16           = 0x0101,
> >          V4L2_CTRL_TYPE_U32           = 0x0102,
> > +       V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
> 
> Why 0x0109?

Good catch. I see no reason in particular, so I'll probably make it
0x0103 eventually.

Cheers and thanks for the review!

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 2627 bytes --]

Hi,

On Fri, 2018-04-20 at 09:51 +0000, Tomasz Figa wrote:
> Hi Paul,
> 
> On Fri, Apr 20, 2018 at 12:46 AM Paul Kocialkowski <
> paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org> wrote:
> [snip]
> > +struct v4l2_ctrl_mpeg2_frame_hdr {
> > +       __u32 slice_len;
> > +       __u32 slice_pos;
> > +       enum { MPEG1, MPEG2 } type;
> 
> Is enum suitable for UAPI?

As it turns out, it's not :)

> > +
> > +       __u16 width;
> > +       __u16 height;
> > +
> > +       enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;
> 
> Ditto.
> 
> > +       __u8 f_code[2][2];
> > +
> > +       __u8 intra_dc_precision;
> > +       __u8 picture_structure;
> > +       __u8 top_field_first;
> > +       __u8 frame_pred_frame_dct;
> > +       __u8 concealment_motion_vectors;
> > +       __u8 q_scale_type;
> > +       __u8 intra_vlc_format;
> > +       __u8 alternate_scan;
> > +
> > +       __u8 backward_ref_index;
> > +       __u8 forward_ref_index;
> > +};
> > +
> >   #endif
> > diff --git a/include/uapi/linux/videodev2.h
> 
> b/include/uapi/linux/videodev2.h
> > index 31b5728b56e9..4b8336f7bcf0 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -635,6 +635,7 @@ struct v4l2_pix_format {
> >   #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L')
> > /*
> 
> SMPTE 421M Annex L compliant stream */
> >   #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /*
> > VP8 */
> >   #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /*
> > VP9 */
> > +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /*
> 
> MPEG2 frame */
> 
> >   /*  Vendor-specific formats   */
> >   #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /*
> > cpia1
> 
> YUV */
> > @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
> >                  __u8 __user *p_u8;
> >                  __u16 __user *p_u16;
> >                  __u32 __user *p_u32;
> > +               struct v4l2_ctrl_mpeg2_frame_hdr __user
> 
> *p_mpeg2_frame_hdr;
> >                  void __user *ptr;
> >          };
> >   } __attribute__ ((packed));
> > @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
> >          V4L2_CTRL_TYPE_U8            = 0x0100,
> >          V4L2_CTRL_TYPE_U16           = 0x0101,
> >          V4L2_CTRL_TYPE_U32           = 0x0102,
> > +       V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
> 
> Why 0x0109?

Good catch. I see no reason in particular, so I'll probably make it
0x0103 eventually.

Cheers and thanks for the review!

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Paul Kocialkowski]

Hi,

On Fri, 2018-04-20 at 09:51 +0000, Tomasz Figa wrote:
> Hi Paul,
> 
> On Fri, Apr 20, 2018 at 12:46 AM Paul Kocialkowski <
> paul.kocialkowski at bootlin.com> wrote:
> [snip]
> > +struct v4l2_ctrl_mpeg2_frame_hdr {
> > +       __u32 slice_len;
> > +       __u32 slice_pos;
> > +       enum { MPEG1, MPEG2 } type;
> 
> Is enum suitable for UAPI?

As it turns out, it's not :)

> > +
> > +       __u16 width;
> > +       __u16 height;
> > +
> > +       enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;
> 
> Ditto.
> 
> > +       __u8 f_code[2][2];
> > +
> > +       __u8 intra_dc_precision;
> > +       __u8 picture_structure;
> > +       __u8 top_field_first;
> > +       __u8 frame_pred_frame_dct;
> > +       __u8 concealment_motion_vectors;
> > +       __u8 q_scale_type;
> > +       __u8 intra_vlc_format;
> > +       __u8 alternate_scan;
> > +
> > +       __u8 backward_ref_index;
> > +       __u8 forward_ref_index;
> > +};
> > +
> >   #endif
> > diff --git a/include/uapi/linux/videodev2.h
> 
> b/include/uapi/linux/videodev2.h
> > index 31b5728b56e9..4b8336f7bcf0 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -635,6 +635,7 @@ struct v4l2_pix_format {
> >   #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L')
> > /*
> 
> SMPTE 421M Annex L compliant stream */
> >   #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /*
> > VP8 */
> >   #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /*
> > VP9 */
> > +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /*
> 
> MPEG2 frame */
> 
> >   /*  Vendor-specific formats   */
> >   #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /*
> > cpia1
> 
> YUV */
> > @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
> >                  __u8 __user *p_u8;
> >                  __u16 __user *p_u16;
> >                  __u32 __user *p_u32;
> > +               struct v4l2_ctrl_mpeg2_frame_hdr __user
> 
> *p_mpeg2_frame_hdr;
> >                  void __user *ptr;
> >          };
> >   } __attribute__ ((packed));
> > @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
> >          V4L2_CTRL_TYPE_U8            = 0x0100,
> >          V4L2_CTRL_TYPE_U16           = 0x0101,
> >          V4L2_CTRL_TYPE_U32           = 0x0102,
> > +       V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
> 
> Why 0x0109?

Good catch. I see no reason in particular, so I'll probably make it
0x0103 eventually.

Cheers and thanks for the review!

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/7bc96baf/attachment.sig>

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 3513 bytes --]

On Fri, May 04, 2018 at 09:49:16AM +0200, Paul Kocialkowski wrote:
> > > +	reserved-memory {
> > > +		#address-cells = <1>;
> > > +		#size-cells = <1>;
> > > +		ranges;
> > > +
> > > +		/* Address must be kept in the lower 256 MiBs of
> > > DRAM for VE. */
> > > +		ve_memory: cma@4a000000 {
> > > +			compatible = "shared-dma-pool";
> > > +			reg = <0x4a000000 0x6000000>;
> > > +			no-map;
> > 
> > I'm not sure why no-map is needed.
> 
> In fact, having no-map here would lead to reserving the area as cache-
> coherent instead of contiguous and thus prevented dmabuf support.
> Replacing it by "resuable" allows proper CMA reservation.
> 
> > And I guess we could use alloc-ranges to make sure the region is in
> > the proper memory range, instead of hardcoding it.
> 
> As far as I could understand from the documentation, "alloc-ranges" is
> used for dynamic allocation while only "reg" is used for static
> allocation. We are currently going with static allocation and thus
> reserve the whole 96 MiB. Is using dynamic allocation instead desirable
> here?

I guess we could turn the question backward. Why do we need a static
allocation? This isn't a buffer that is always allocated on the same
area, but rather that we have a range available. So our constraint is
on the range, nothing else.

> > > +			reg = <0x01c0e000 0x1000>;
> > > +			memory-region = <&ve_memory>;
> > 
> > Since you made the CMA region the default one, you don't need to tie
> > it to that device in particular (and you can drop it being mandatory
> > from your binding as well).
> 
> What if another driver (or the system) claims memory from that zone and
> that the reserved memory ends up not being available for the VPU
> anymore?
> 
> Acccording to the reserved-memory documentation, the reusable property
> (that we need for dmabuf) puts a limitation that the device driver
> owning the region must be able to reclaim it back.
> 
> How does that work out if the CMA region is not tied to a driver in
> particular?

I'm not sure to get what you're saying. You have the property
linux,cma-default in your reserved region, so the behaviour you
described is what you explicitly asked for.

> 
> > > +
> > > +			clocks = <&ccu CLK_AHB_VE>, <&ccu CLK_VE>,
> > > +				 <&ccu CLK_DRAM_VE>;
> > > +			clock-names = "ahb", "mod", "ram";
> > > +
> > > +			assigned-clocks = <&ccu CLK_VE>;
> > > +			assigned-clock-rates = <320000000>;
> > 
> > This should be set from within the driver. If it's something that you
> > absolutely needed for the device to operate, you have no guarantee
> > that the clock rate won't change at any point in time after the device
> > probe, so that's not a proper solution.
> > 
> > And if it's not needed and can be adjusted depending on the
> > framerate/codec/resolution, then it shouldn't be in the DT either.
> 
> Yes, that makes sense.
> 
> > Don't you also need to map the SRAM on the A20?
> 
> That's a good point, there is currently no syscon handle for A20 (and
> also A13). Maybe SRAM is muxed to the VE by default so it "just works"? 
> 
> I'll investigate on this side, also keeping in mind that the actual
> solution is to use the SRAM controller driver (but that won't make it to
> v3).

The SRAM driver is available on the A20, so you should really use that
instead of a syscon.

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 3420 bytes --]

On Fri, May 04, 2018 at 09:49:16AM +0200, Paul Kocialkowski wrote:
> > > +	reserved-memory {
> > > +		#address-cells = <1>;
> > > +		#size-cells = <1>;
> > > +		ranges;
> > > +
> > > +		/* Address must be kept in the lower 256 MiBs of
> > > DRAM for VE. */
> > > +		ve_memory: cma@4a000000 {
> > > +			compatible = "shared-dma-pool";
> > > +			reg = <0x4a000000 0x6000000>;
> > > +			no-map;
> > 
> > I'm not sure why no-map is needed.
> 
> In fact, having no-map here would lead to reserving the area as cache-
> coherent instead of contiguous and thus prevented dmabuf support.
> Replacing it by "resuable" allows proper CMA reservation.
> 
> > And I guess we could use alloc-ranges to make sure the region is in
> > the proper memory range, instead of hardcoding it.
> 
> As far as I could understand from the documentation, "alloc-ranges" is
> used for dynamic allocation while only "reg" is used for static
> allocation. We are currently going with static allocation and thus
> reserve the whole 96 MiB. Is using dynamic allocation instead desirable
> here?

I guess we could turn the question backward. Why do we need a static
allocation? This isn't a buffer that is always allocated on the same
area, but rather that we have a range available. So our constraint is
on the range, nothing else.

> > > +			reg = <0x01c0e000 0x1000>;
> > > +			memory-region = <&ve_memory>;
> > 
> > Since you made the CMA region the default one, you don't need to tie
> > it to that device in particular (and you can drop it being mandatory
> > from your binding as well).
> 
> What if another driver (or the system) claims memory from that zone and
> that the reserved memory ends up not being available for the VPU
> anymore?
> 
> Acccording to the reserved-memory documentation, the reusable property
> (that we need for dmabuf) puts a limitation that the device driver
> owning the region must be able to reclaim it back.
> 
> How does that work out if the CMA region is not tied to a driver in
> particular?

I'm not sure to get what you're saying. You have the property
linux,cma-default in your reserved region, so the behaviour you
described is what you explicitly asked for.

> 
> > > +
> > > +			clocks = <&ccu CLK_AHB_VE>, <&ccu CLK_VE>,
> > > +				 <&ccu CLK_DRAM_VE>;
> > > +			clock-names = "ahb", "mod", "ram";
> > > +
> > > +			assigned-clocks = <&ccu CLK_VE>;
> > > +			assigned-clock-rates = <320000000>;
> > 
> > This should be set from within the driver. If it's something that you
> > absolutely needed for the device to operate, you have no guarantee
> > that the clock rate won't change at any point in time after the device
> > probe, so that's not a proper solution.
> > 
> > And if it's not needed and can be adjusted depending on the
> > framerate/codec/resolution, then it shouldn't be in the DT either.
> 
> Yes, that makes sense.
> 
> > Don't you also need to map the SRAM on the A20?
> 
> That's a good point, there is currently no syscon handle for A20 (and
> also A13). Maybe SRAM is muxed to the VE by default so it "just works"? 
> 
> I'll investigate on this side, also keeping in mind that the actual
> solution is to use the SRAM controller driver (but that won't make it to
> v3).

The SRAM driver is available on the A20, so you should really use that
instead of a syscon.

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Maxime Ripard]

On Fri, May 04, 2018 at 09:49:16AM +0200, Paul Kocialkowski wrote:
> > > +	reserved-memory {
> > > +		#address-cells = <1>;
> > > +		#size-cells = <1>;
> > > +		ranges;
> > > +
> > > +		/* Address must be kept in the lower 256 MiBs of
> > > DRAM for VE. */
> > > +		ve_memory: cma at 4a000000 {
> > > +			compatible = "shared-dma-pool";
> > > +			reg = <0x4a000000 0x6000000>;
> > > +			no-map;
> > 
> > I'm not sure why no-map is needed.
> 
> In fact, having no-map here would lead to reserving the area as cache-
> coherent instead of contiguous and thus prevented dmabuf support.
> Replacing it by "resuable" allows proper CMA reservation.
> 
> > And I guess we could use alloc-ranges to make sure the region is in
> > the proper memory range, instead of hardcoding it.
> 
> As far as I could understand from the documentation, "alloc-ranges" is
> used for dynamic allocation while only "reg" is used for static
> allocation. We are currently going with static allocation and thus
> reserve the whole 96 MiB. Is using dynamic allocation instead desirable
> here?

I guess we could turn the question backward. Why do we need a static
allocation? This isn't a buffer that is always allocated on the same
area, but rather that we have a range available. So our constraint is
on the range, nothing else.

> > > +			reg = <0x01c0e000 0x1000>;
> > > +			memory-region = <&ve_memory>;
> > 
> > Since you made the CMA region the default one, you don't need to tie
> > it to that device in particular (and you can drop it being mandatory
> > from your binding as well).
> 
> What if another driver (or the system) claims memory from that zone and
> that the reserved memory ends up not being available for the VPU
> anymore?
> 
> Acccording to the reserved-memory documentation, the reusable property
> (that we need for dmabuf) puts a limitation that the device driver
> owning the region must be able to reclaim it back.
> 
> How does that work out if the CMA region is not tied to a driver in
> particular?

I'm not sure to get what you're saying. You have the property
linux,cma-default in your reserved region, so the behaviour you
described is what you explicitly asked for.

> 
> > > +
> > > +			clocks = <&ccu CLK_AHB_VE>, <&ccu CLK_VE>,
> > > +				 <&ccu CLK_DRAM_VE>;
> > > +			clock-names = "ahb", "mod", "ram";
> > > +
> > > +			assigned-clocks = <&ccu CLK_VE>;
> > > +			assigned-clock-rates = <320000000>;
> > 
> > This should be set from within the driver. If it's something that you
> > absolutely needed for the device to operate, you have no guarantee
> > that the clock rate won't change at any point in time after the device
> > probe, so that's not a proper solution.
> > 
> > And if it's not needed and can be adjusted depending on the
> > framerate/codec/resolution, then it shouldn't be in the DT either.
> 
> Yes, that makes sense.
> 
> > Don't you also need to map the SRAM on the A20?
> 
> That's a good point, there is currently no syscon handle for A20 (and
> also A13). Maybe SRAM is muxed to the VE by default so it "just works"? 
> 
> I'll investigate on this side, also keeping in mind that the actual
> solution is to use the SRAM controller driver (but that won't make it to
> v3).

The SRAM driver is available on the A20, so you should really use that
instead of a syscon.

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/5dcc102b/attachment.sig>

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 4504 bytes --]

Hi,

On Fri, 2018-05-04 at 10:40 +0200, Maxime Ripard wrote:
> On Fri, May 04, 2018 at 09:49:16AM +0200, Paul Kocialkowski wrote:
> > > > +	reserved-memory {
> > > > +		#address-cells = <1>;
> > > > +		#size-cells = <1>;
> > > > +		ranges;
> > > > +
> > > > +		/* Address must be kept in the lower 256 MiBs
> > > > of
> > > > DRAM for VE. */
> > > > +		ve_memory: cma@4a000000 {
> > > > +			compatible = "shared-dma-pool";
> > > > +			reg = <0x4a000000 0x6000000>;
> > > > +			no-map;
> > > 
> > > I'm not sure why no-map is needed.
> > 
> > In fact, having no-map here would lead to reserving the area as
> > cache-
> > coherent instead of contiguous and thus prevented dmabuf support.
> > Replacing it by "resuable" allows proper CMA reservation.
> > 
> > > And I guess we could use alloc-ranges to make sure the region is
> > > in
> > > the proper memory range, instead of hardcoding it.
> > 
> > As far as I could understand from the documentation, "alloc-ranges"
> > is
> > used for dynamic allocation while only "reg" is used for static
> > allocation. We are currently going with static allocation and thus
> > reserve the whole 96 MiB. Is using dynamic allocation instead
> > desirable
> > here?
> 
> I guess we could turn the question backward. Why do we need a static
> allocation? This isn't a buffer that is always allocated on the same
> area, but rather that we have a range available. So our constraint is
> on the range, nothing else.

That makes sense, I will give it a shot with a range then.

> > > > +			reg = <0x01c0e000 0x1000>;
> > > > +			memory-region = <&ve_memory>;
> > > 
> > > Since you made the CMA region the default one, you don't need to
> > > tie
> > > it to that device in particular (and you can drop it being
> > > mandatory
> > > from your binding as well).
> > 
> > What if another driver (or the system) claims memory from that zone
> > and
> > that the reserved memory ends up not being available for the VPU
> > anymore?
> > 
> > Acccording to the reserved-memory documentation, the reusable
> > property
> > (that we need for dmabuf) puts a limitation that the device driver
> > owning the region must be able to reclaim it back.
> > 
> > How does that work out if the CMA region is not tied to a driver in
> > particular?
> 
> I'm not sure to get what you're saying. You have the property
> linux,cma-default in your reserved region, so the behaviour you
> described is what you explicitly asked for.

My point is that I don't see how the driver can claim back (part of) the
reserved area if the area is not explicitly attached to it.

Or is that mechanism made in a way that all drivers wishing to use the
reserved memory area can claim it back from the system, but there is no
priority (other than first-come first-served) for which drivers claims
it back in case two want to use the same reserved region (in a scenario
where there isn't enough memory to allow both drivers)?

> > > > +
> > > > +			clocks = <&ccu CLK_AHB_VE>, <&ccu
> > > > CLK_VE>,
> > > > +				 <&ccu CLK_DRAM_VE>;
> > > > +			clock-names = "ahb", "mod", "ram";
> > > > +
> > > > +			assigned-clocks = <&ccu CLK_VE>;
> > > > +			assigned-clock-rates = <320000000>;
> > > 
> > > This should be set from within the driver. If it's something that
> > > you
> > > absolutely needed for the device to operate, you have no guarantee
> > > that the clock rate won't change at any point in time after the
> > > device
> > > probe, so that's not a proper solution.
> > > 
> > > And if it's not needed and can be adjusted depending on the
> > > framerate/codec/resolution, then it shouldn't be in the DT either.
> > 
> > Yes, that makes sense.
> > 
> > > Don't you also need to map the SRAM on the A20?
> > 
> > That's a good point, there is currently no syscon handle for A20
> > (and
> > also A13). Maybe SRAM is muxed to the VE by default so it "just
> > works"? 
> > 
> > I'll investigate on this side, also keeping in mind that the actual
> > solution is to use the SRAM controller driver (but that won't make
> > it to
> > v3).
> 
> The SRAM driver is available on the A20, so you should really use that
> instead of a syscon.

The SRAM driver is indeed available for the A20, but still lacks support
for the VE in particular as far as I can see.

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 4382 bytes --]

Hi,

On Fri, 2018-05-04 at 10:40 +0200, Maxime Ripard wrote:
> On Fri, May 04, 2018 at 09:49:16AM +0200, Paul Kocialkowski wrote:
> > > > +	reserved-memory {
> > > > +		#address-cells = <1>;
> > > > +		#size-cells = <1>;
> > > > +		ranges;
> > > > +
> > > > +		/* Address must be kept in the lower 256 MiBs
> > > > of
> > > > DRAM for VE. */
> > > > +		ve_memory: cma@4a000000 {
> > > > +			compatible = "shared-dma-pool";
> > > > +			reg = <0x4a000000 0x6000000>;
> > > > +			no-map;
> > > 
> > > I'm not sure why no-map is needed.
> > 
> > In fact, having no-map here would lead to reserving the area as
> > cache-
> > coherent instead of contiguous and thus prevented dmabuf support.
> > Replacing it by "resuable" allows proper CMA reservation.
> > 
> > > And I guess we could use alloc-ranges to make sure the region is
> > > in
> > > the proper memory range, instead of hardcoding it.
> > 
> > As far as I could understand from the documentation, "alloc-ranges"
> > is
> > used for dynamic allocation while only "reg" is used for static
> > allocation. We are currently going with static allocation and thus
> > reserve the whole 96 MiB. Is using dynamic allocation instead
> > desirable
> > here?
> 
> I guess we could turn the question backward. Why do we need a static
> allocation? This isn't a buffer that is always allocated on the same
> area, but rather that we have a range available. So our constraint is
> on the range, nothing else.

That makes sense, I will give it a shot with a range then.

> > > > +			reg = <0x01c0e000 0x1000>;
> > > > +			memory-region = <&ve_memory>;
> > > 
> > > Since you made the CMA region the default one, you don't need to
> > > tie
> > > it to that device in particular (and you can drop it being
> > > mandatory
> > > from your binding as well).
> > 
> > What if another driver (or the system) claims memory from that zone
> > and
> > that the reserved memory ends up not being available for the VPU
> > anymore?
> > 
> > Acccording to the reserved-memory documentation, the reusable
> > property
> > (that we need for dmabuf) puts a limitation that the device driver
> > owning the region must be able to reclaim it back.
> > 
> > How does that work out if the CMA region is not tied to a driver in
> > particular?
> 
> I'm not sure to get what you're saying. You have the property
> linux,cma-default in your reserved region, so the behaviour you
> described is what you explicitly asked for.

My point is that I don't see how the driver can claim back (part of) the
reserved area if the area is not explicitly attached to it.

Or is that mechanism made in a way that all drivers wishing to use the
reserved memory area can claim it back from the system, but there is no
priority (other than first-come first-served) for which drivers claims
it back in case two want to use the same reserved region (in a scenario
where there isn't enough memory to allow both drivers)?

> > > > +
> > > > +			clocks = <&ccu CLK_AHB_VE>, <&ccu
> > > > CLK_VE>,
> > > > +				 <&ccu CLK_DRAM_VE>;
> > > > +			clock-names = "ahb", "mod", "ram";
> > > > +
> > > > +			assigned-clocks = <&ccu CLK_VE>;
> > > > +			assigned-clock-rates = <320000000>;
> > > 
> > > This should be set from within the driver. If it's something that
> > > you
> > > absolutely needed for the device to operate, you have no guarantee
> > > that the clock rate won't change at any point in time after the
> > > device
> > > probe, so that's not a proper solution.
> > > 
> > > And if it's not needed and can be adjusted depending on the
> > > framerate/codec/resolution, then it shouldn't be in the DT either.
> > 
> > Yes, that makes sense.
> > 
> > > Don't you also need to map the SRAM on the A20?
> > 
> > That's a good point, there is currently no syscon handle for A20
> > (and
> > also A13). Maybe SRAM is muxed to the VE by default so it "just
> > works"? 
> > 
> > I'll investigate on this side, also keeping in mind that the actual
> > solution is to use the SRAM controller driver (but that won't make
> > it to
> > v3).
> 
> The SRAM driver is available on the A20, so you should really use that
> instead of a syscon.

The SRAM driver is indeed available for the A20, but still lacks support
for the VE in particular as far as I can see.

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

Hi,

On Fri, 2018-05-04 at 10:40 +0200, Maxime Ripard wrote:
> On Fri, May 04, 2018 at 09:49:16AM +0200, Paul Kocialkowski wrote:
> > > > +	reserved-memory {
> > > > +		#address-cells = <1>;
> > > > +		#size-cells = <1>;
> > > > +		ranges;
> > > > +
> > > > +		/* Address must be kept in the lower 256 MiBs
> > > > of
> > > > DRAM for VE. */
> > > > +		ve_memory: cma at 4a000000 {
> > > > +			compatible = "shared-dma-pool";
> > > > +			reg = <0x4a000000 0x6000000>;
> > > > +			no-map;
> > > 
> > > I'm not sure why no-map is needed.
> > 
> > In fact, having no-map here would lead to reserving the area as
> > cache-
> > coherent instead of contiguous and thus prevented dmabuf support.
> > Replacing it by "resuable" allows proper CMA reservation.
> > 
> > > And I guess we could use alloc-ranges to make sure the region is
> > > in
> > > the proper memory range, instead of hardcoding it.
> > 
> > As far as I could understand from the documentation, "alloc-ranges"
> > is
> > used for dynamic allocation while only "reg" is used for static
> > allocation. We are currently going with static allocation and thus
> > reserve the whole 96 MiB. Is using dynamic allocation instead
> > desirable
> > here?
> 
> I guess we could turn the question backward. Why do we need a static
> allocation? This isn't a buffer that is always allocated on the same
> area, but rather that we have a range available. So our constraint is
> on the range, nothing else.

That makes sense, I will give it a shot with a range then.

> > > > +			reg = <0x01c0e000 0x1000>;
> > > > +			memory-region = <&ve_memory>;
> > > 
> > > Since you made the CMA region the default one, you don't need to
> > > tie
> > > it to that device in particular (and you can drop it being
> > > mandatory
> > > from your binding as well).
> > 
> > What if another driver (or the system) claims memory from that zone
> > and
> > that the reserved memory ends up not being available for the VPU
> > anymore?
> > 
> > Acccording to the reserved-memory documentation, the reusable
> > property
> > (that we need for dmabuf) puts a limitation that the device driver
> > owning the region must be able to reclaim it back.
> > 
> > How does that work out if the CMA region is not tied to a driver in
> > particular?
> 
> I'm not sure to get what you're saying. You have the property
> linux,cma-default in your reserved region, so the behaviour you
> described is what you explicitly asked for.

My point is that I don't see how the driver can claim back (part of) the
reserved area if the area is not explicitly attached to it.

Or is that mechanism made in a way that all drivers wishing to use the
reserved memory area can claim it back from the system, but there is no
priority (other than first-come first-served) for which drivers claims
it back in case two want to use the same reserved region (in a scenario
where there isn't enough memory to allow both drivers)?

> > > > +
> > > > +			clocks = <&ccu CLK_AHB_VE>, <&ccu
> > > > CLK_VE>,
> > > > +				 <&ccu CLK_DRAM_VE>;
> > > > +			clock-names = "ahb", "mod", "ram";
> > > > +
> > > > +			assigned-clocks = <&ccu CLK_VE>;
> > > > +			assigned-clock-rates = <320000000>;
> > > 
> > > This should be set from within the driver. If it's something that
> > > you
> > > absolutely needed for the device to operate, you have no guarantee
> > > that the clock rate won't change at any point in time after the
> > > device
> > > probe, so that's not a proper solution.
> > > 
> > > And if it's not needed and can be adjusted depending on the
> > > framerate/codec/resolution, then it shouldn't be in the DT either.
> > 
> > Yes, that makes sense.
> > 
> > > Don't you also need to map the SRAM on the A20?
> > 
> > That's a good point, there is currently no syscon handle for A20
> > (and
> > also A13). Maybe SRAM is muxed to the VE by default so it "just
> > works"? 
> > 
> > I'll investigate on this side, also keeping in mind that the actual
> > solution is to use the SRAM controller driver (but that won't make
> > it to
> > v3).
> 
> The SRAM driver is available on the A20, so you should really use that
> instead of a syscon.

The SRAM driver is indeed available for the A20, but still lacks support
for the VE in particular as far as I can see.

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/dd75265f/attachment-0001.sig>

Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 6809 bytes --]

On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> Stateless video decoding engines require both the MPEG slices and
> associated metadata from the video stream in order to decode frames.
> 
> This introduces definitions for a new pixel format, describing buffers
> with MPEG2 slice data, as well as a control structure for passing the
> frame header (metadata) to drivers.

While working on this, I came accross Hugues Fruchet's series that also
adds similar definitions for parsed MPEG2 metadata:
https://patchwork.kernel.org/patch/9704707/

Since that version made it to a v6, I will take the time to read the
discussion and see what needs to be changed in my proposal, so that we
can avoid discussing the same points over a year later.

This will most likely not make it to the next revision of the driver
series, so I will keep the format/controls definitions in their v2 state
(despite all the useful comments received) and take the time to properly
rework things in a future revision.

Cheers,

Paul

> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> Signed-off-by: Florent Revest <florent.revest@free-electrons.com>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
>  include/uapi/linux/v4l2-controls.h   | 26 ++++++++++++++++++++++++++
>  include/uapi/linux/videodev2.h       |  3 +++
>  4 files changed, 40 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> b/drivers/media/v4l2-core/v4l2-ctrls.c
> index ba05a8b9a095..fcdc12b9a9e0 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		re
> turn "Vertical MV Search Range";
>  	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		re
> turn "Repeat Sequence Header";
>  	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		retu
> rn "Force Key Frame";
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		retu
> rn "MPEG2 Frame Header";
>  
>  	/* VPX controls */
>  	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		r
> eturn "VPX Number of Partitions";
> @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name,
> enum v4l2_ctrl_type *type,
>  	case V4L2_CID_RDS_TX_ALT_FREQS:
>  		*type = V4L2_CTRL_TYPE_U32;
>  		break;
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> +		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> +		break;
>  	default:
>  		*type = V4L2_CTRL_TYPE_INTEGER;
>  		break;
> @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl
> *ctrl, u32 idx,
>  			return -ERANGE;
>  		return 0;
>  
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		return 0;
> +
>  	default:
>  		return -EINVAL;
>  	}
> @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct
> v4l2_ctrl_handler *hdl,
>  	case V4L2_CTRL_TYPE_U32:
>  		elem_size = sizeof(u32);
>  		break;
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
> +		break;
>  	default:
>  		if (type < V4L2_CTRL_COMPOUND_TYPES)
>  			elem_size = sizeof(s32);
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 468c3c65362d..8070203da5d2 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc
> *fmt)
>  		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1
> (SMPTE 412M Annex L)"; break;
>  		case V4L2_PIX_FMT_VP8:		descr = "VP8";
> break;
>  		case V4L2_PIX_FMT_VP9:		descr = "VP9";
> break;
> +		case V4L2_PIX_FMT_MPEG2_FRAME:	descr = "MPEG2
> Frame"; break;
>  		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA
> YUV"; break;
>  		case V4L2_PIX_FMT_WNVA:		descr =
> "WNVA"; break;
>  		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA
> SN9C10X"; break;
> diff --git a/include/uapi/linux/v4l2-controls.h
> b/include/uapi/linux/v4l2-controls.h
> index cbbb750d87d1..8431b2a540c7 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
>  };
>  #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPEG_
> BASE+407)
>  
> +#define
> V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
> +
>  /*  Control IDs for VP8 streams
>   *  Although VP8 is not part of MPEG we add these controls to the
> MPEG class
>   *  as that class is already handling other video compression
> standards
> @@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
>  #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_CLA
> SS_BASE + 3)
>  #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DETEC
> T_CLASS_BASE + 4)
>  
> +struct v4l2_ctrl_mpeg2_frame_hdr {
> +	__u32 slice_len;
> +	__u32 slice_pos;
> +	enum { MPEG1, MPEG2 } type;
> +
> +	__u16 width;
> +	__u16 height;
> +
> +	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;
> +	__u8 f_code[2][2];
> +
> +	__u8 intra_dc_precision;
> +	__u8 picture_structure;
> +	__u8 top_field_first;
> +	__u8 frame_pred_frame_dct;
> +	__u8 concealment_motion_vectors;
> +	__u8 q_scale_type;
> +	__u8 intra_vlc_format;
> +	__u8 alternate_scan;
> +
> +	__u8 backward_ref_index;
> +	__u8 forward_ref_index;
> +};
> +
>  #endif
> diff --git a/include/uapi/linux/videodev2.h
> b/include/uapi/linux/videodev2.h
> index 31b5728b56e9..4b8336f7bcf0 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -635,6 +635,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /*
> SMPTE 421M Annex L compliant stream */
>  #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8
> */
>  #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /* VP9
> */
> +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /*
> MPEG2 frame */
>  
>  /*  Vendor-specific formats   */
>  #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /*
> cpia1 YUV */
> @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
>  		__u8 __user *p_u8;
>  		__u16 __user *p_u16;
>  		__u32 __user *p_u32;
> +		struct v4l2_ctrl_mpeg2_frame_hdr __user
> *p_mpeg2_frame_hdr;
>  		void __user *ptr;
>  	};
>  } __attribute__ ((packed));
> @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
>  	V4L2_CTRL_TYPE_U8	     = 0x0100,
>  	V4L2_CTRL_TYPE_U16	     = 0x0101,
>  	V4L2_CTRL_TYPE_U32	     = 0x0102,
> +	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
>  };
>  
>  /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 6675 bytes --]

On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> Stateless video decoding engines require both the MPEG slices and
> associated metadata from the video stream in order to decode frames.
> 
> This introduces definitions for a new pixel format, describing buffers
> with MPEG2 slice data, as well as a control structure for passing the
> frame header (metadata) to drivers.

While working on this, I came accross Hugues Fruchet's series that also
adds similar definitions for parsed MPEG2 metadata:
https://patchwork.kernel.org/patch/9704707/

Since that version made it to a v6, I will take the time to read the
discussion and see what needs to be changed in my proposal, so that we
can avoid discussing the same points over a year later.

This will most likely not make it to the next revision of the driver
series, so I will keep the format/controls definitions in their v2 state
(despite all the useful comments received) and take the time to properly
rework things in a future revision.

Cheers,

Paul

> Signed-off-by: Paul Kocialkowski <paul.kocialkowski-LDxbnhwyfcJBDgjK7y7TUQ@public.gmane.org>
> Signed-off-by: Florent Revest <florent.revest-wi1+55ScJUtKEb57/3fJTNBPR1lH4CV8@public.gmane.org>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
>  include/uapi/linux/v4l2-controls.h   | 26 ++++++++++++++++++++++++++
>  include/uapi/linux/videodev2.h       |  3 +++
>  4 files changed, 40 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> b/drivers/media/v4l2-core/v4l2-ctrls.c
> index ba05a8b9a095..fcdc12b9a9e0 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		re
> turn "Vertical MV Search Range";
>  	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		re
> turn "Repeat Sequence Header";
>  	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		retu
> rn "Force Key Frame";
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		retu
> rn "MPEG2 Frame Header";
>  
>  	/* VPX controls */
>  	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		r
> eturn "VPX Number of Partitions";
> @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name,
> enum v4l2_ctrl_type *type,
>  	case V4L2_CID_RDS_TX_ALT_FREQS:
>  		*type = V4L2_CTRL_TYPE_U32;
>  		break;
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> +		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> +		break;
>  	default:
>  		*type = V4L2_CTRL_TYPE_INTEGER;
>  		break;
> @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl
> *ctrl, u32 idx,
>  			return -ERANGE;
>  		return 0;
>  
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		return 0;
> +
>  	default:
>  		return -EINVAL;
>  	}
> @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct
> v4l2_ctrl_handler *hdl,
>  	case V4L2_CTRL_TYPE_U32:
>  		elem_size = sizeof(u32);
>  		break;
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
> +		break;
>  	default:
>  		if (type < V4L2_CTRL_COMPOUND_TYPES)
>  			elem_size = sizeof(s32);
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 468c3c65362d..8070203da5d2 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc
> *fmt)
>  		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1
> (SMPTE 412M Annex L)"; break;
>  		case V4L2_PIX_FMT_VP8:		descr = "VP8";
> break;
>  		case V4L2_PIX_FMT_VP9:		descr = "VP9";
> break;
> +		case V4L2_PIX_FMT_MPEG2_FRAME:	descr = "MPEG2
> Frame"; break;
>  		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA
> YUV"; break;
>  		case V4L2_PIX_FMT_WNVA:		descr =
> "WNVA"; break;
>  		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA
> SN9C10X"; break;
> diff --git a/include/uapi/linux/v4l2-controls.h
> b/include/uapi/linux/v4l2-controls.h
> index cbbb750d87d1..8431b2a540c7 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
>  };
>  #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPEG_
> BASE+407)
>  
> +#define
> V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
> +
>  /*  Control IDs for VP8 streams
>   *  Although VP8 is not part of MPEG we add these controls to the
> MPEG class
>   *  as that class is already handling other video compression
> standards
> @@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
>  #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_CLA
> SS_BASE + 3)
>  #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DETEC
> T_CLASS_BASE + 4)
>  
> +struct v4l2_ctrl_mpeg2_frame_hdr {
> +	__u32 slice_len;
> +	__u32 slice_pos;
> +	enum { MPEG1, MPEG2 } type;
> +
> +	__u16 width;
> +	__u16 height;
> +
> +	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;
> +	__u8 f_code[2][2];
> +
> +	__u8 intra_dc_precision;
> +	__u8 picture_structure;
> +	__u8 top_field_first;
> +	__u8 frame_pred_frame_dct;
> +	__u8 concealment_motion_vectors;
> +	__u8 q_scale_type;
> +	__u8 intra_vlc_format;
> +	__u8 alternate_scan;
> +
> +	__u8 backward_ref_index;
> +	__u8 forward_ref_index;
> +};
> +
>  #endif
> diff --git a/include/uapi/linux/videodev2.h
> b/include/uapi/linux/videodev2.h
> index 31b5728b56e9..4b8336f7bcf0 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -635,6 +635,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /*
> SMPTE 421M Annex L compliant stream */
>  #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8
> */
>  #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /* VP9
> */
> +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /*
> MPEG2 frame */
>  
>  /*  Vendor-specific formats   */
>  #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /*
> cpia1 YUV */
> @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
>  		__u8 __user *p_u8;
>  		__u16 __user *p_u16;
>  		__u32 __user *p_u32;
> +		struct v4l2_ctrl_mpeg2_frame_hdr __user
> *p_mpeg2_frame_hdr;
>  		void __user *ptr;
>  	};
>  } __attribute__ ((packed));
> @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
>  	V4L2_CTRL_TYPE_U8	     = 0x0100,
>  	V4L2_CTRL_TYPE_U16	     = 0x0101,
>  	V4L2_CTRL_TYPE_U32	     = 0x0102,
> +	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
>  };
>  
>  /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[PATCH v2 05/10] media: v4l: Add definitions for MPEG2 frame format and header metadata

[Paul Kocialkowski]

On Thu, 2018-04-19 at 17:45 +0200, Paul Kocialkowski wrote:
> Stateless video decoding engines require both the MPEG slices and
> associated metadata from the video stream in order to decode frames.
> 
> This introduces definitions for a new pixel format, describing buffers
> with MPEG2 slice data, as well as a control structure for passing the
> frame header (metadata) to drivers.

While working on this, I came accross Hugues Fruchet's series that also
adds similar definitions for parsed MPEG2 metadata:
https://patchwork.kernel.org/patch/9704707/

Since that version made it to a v6, I will take the time to read the
discussion and see what needs to be changed in my proposal, so that we
can avoid discussing the same points over a year later.

This will most likely not make it to the next revision of the driver
series, so I will keep the format/controls definitions in their v2 state
(despite all the useful comments received) and take the time to properly
rework things in a future revision.

Cheers,

Paul

> Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
> Signed-off-by: Florent Revest <florent.revest@free-electrons.com>
> ---
>  drivers/media/v4l2-core/v4l2-ctrls.c | 10 ++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c |  1 +
>  include/uapi/linux/v4l2-controls.h   | 26 ++++++++++++++++++++++++++
>  include/uapi/linux/videodev2.h       |  3 +++
>  4 files changed, 40 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c
> b/drivers/media/v4l2-core/v4l2-ctrls.c
> index ba05a8b9a095..fcdc12b9a9e0 100644
> --- a/drivers/media/v4l2-core/v4l2-ctrls.c
> +++ b/drivers/media/v4l2-core/v4l2-ctrls.c
> @@ -761,6 +761,7 @@ const char *v4l2_ctrl_get_name(u32 id)
>  	case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE:		re
> turn "Vertical MV Search Range";
>  	case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER:		re
> turn "Repeat Sequence Header";
>  	case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME:		retu
> rn "Force Key Frame";
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:		retu
> rn "MPEG2 Frame Header";
>  
>  	/* VPX controls */
>  	case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS:		r
> eturn "VPX Number of Partitions";
> @@ -1152,6 +1153,9 @@ void v4l2_ctrl_fill(u32 id, const char **name,
> enum v4l2_ctrl_type *type,
>  	case V4L2_CID_RDS_TX_ALT_FREQS:
>  		*type = V4L2_CTRL_TYPE_U32;
>  		break;
> +	case V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR:
> +		*type = V4L2_CTRL_TYPE_MPEG2_FRAME_HDR;
> +		break;
>  	default:
>  		*type = V4L2_CTRL_TYPE_INTEGER;
>  		break;
> @@ -1472,6 +1476,9 @@ static int std_validate(const struct v4l2_ctrl
> *ctrl, u32 idx,
>  			return -ERANGE;
>  		return 0;
>  
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		return 0;
> +
>  	default:
>  		return -EINVAL;
>  	}
> @@ -2046,6 +2053,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct
> v4l2_ctrl_handler *hdl,
>  	case V4L2_CTRL_TYPE_U32:
>  		elem_size = sizeof(u32);
>  		break;
> +	case V4L2_CTRL_TYPE_MPEG2_FRAME_HDR:
> +		elem_size = sizeof(struct v4l2_ctrl_mpeg2_frame_hdr);
> +		break;
>  	default:
>  		if (type < V4L2_CTRL_COMPOUND_TYPES)
>  			elem_size = sizeof(s32);
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 468c3c65362d..8070203da5d2 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1273,6 +1273,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc
> *fmt)
>  		case V4L2_PIX_FMT_VC1_ANNEX_L:	descr = "VC-1
> (SMPTE 412M Annex L)"; break;
>  		case V4L2_PIX_FMT_VP8:		descr = "VP8";
> break;
>  		case V4L2_PIX_FMT_VP9:		descr = "VP9";
> break;
> +		case V4L2_PIX_FMT_MPEG2_FRAME:	descr = "MPEG2
> Frame"; break;
>  		case V4L2_PIX_FMT_CPIA1:	descr = "GSPCA CPiA
> YUV"; break;
>  		case V4L2_PIX_FMT_WNVA:		descr =
> "WNVA"; break;
>  		case V4L2_PIX_FMT_SN9C10X:	descr = "GSPCA
> SN9C10X"; break;
> diff --git a/include/uapi/linux/v4l2-controls.h
> b/include/uapi/linux/v4l2-controls.h
> index cbbb750d87d1..8431b2a540c7 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -557,6 +557,8 @@ enum v4l2_mpeg_video_mpeg4_profile {
>  };
>  #define V4L2_CID_MPEG_VIDEO_MPEG4_QPEL		(V4L2_CID_MPEG_
> BASE+407)
>  
> +#define
> V4L2_CID_MPEG_VIDEO_MPEG2_FRAME_HDR     (V4L2_CID_MPEG_BASE+450)
> +
>  /*  Control IDs for VP8 streams
>   *  Although VP8 is not part of MPEG we add these controls to the
> MPEG class
>   *  as that class is already handling other video compression
> standards
> @@ -985,4 +987,28 @@ enum v4l2_detect_md_mode {
>  #define V4L2_CID_DETECT_MD_THRESHOLD_GRID	(V4L2_CID_DETECT_CLA
> SS_BASE + 3)
>  #define V4L2_CID_DETECT_MD_REGION_GRID		(V4L2_CID_DETEC
> T_CLASS_BASE + 4)
>  
> +struct v4l2_ctrl_mpeg2_frame_hdr {
> +	__u32 slice_len;
> +	__u32 slice_pos;
> +	enum { MPEG1, MPEG2 } type;
> +
> +	__u16 width;
> +	__u16 height;
> +
> +	enum { PCT_I = 1, PCT_P, PCT_B, PCT_D } picture_coding_type;
> +	__u8 f_code[2][2];
> +
> +	__u8 intra_dc_precision;
> +	__u8 picture_structure;
> +	__u8 top_field_first;
> +	__u8 frame_pred_frame_dct;
> +	__u8 concealment_motion_vectors;
> +	__u8 q_scale_type;
> +	__u8 intra_vlc_format;
> +	__u8 alternate_scan;
> +
> +	__u8 backward_ref_index;
> +	__u8 forward_ref_index;
> +};
> +
>  #endif
> diff --git a/include/uapi/linux/videodev2.h
> b/include/uapi/linux/videodev2.h
> index 31b5728b56e9..4b8336f7bcf0 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -635,6 +635,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_VC1_ANNEX_L v4l2_fourcc('V', 'C', '1', 'L') /*
> SMPTE 421M Annex L compliant stream */
>  #define V4L2_PIX_FMT_VP8      v4l2_fourcc('V', 'P', '8', '0') /* VP8
> */
>  #define V4L2_PIX_FMT_VP9      v4l2_fourcc('V', 'P', '9', '0') /* VP9
> */
> +#define V4L2_PIX_FMT_MPEG2_FRAME v4l2_fourcc('M', 'G', '2', 'F') /*
> MPEG2 frame */
>  
>  /*  Vendor-specific formats   */
>  #define V4L2_PIX_FMT_CPIA1    v4l2_fourcc('C', 'P', 'I', 'A') /*
> cpia1 YUV */
> @@ -1586,6 +1587,7 @@ struct v4l2_ext_control {
>  		__u8 __user *p_u8;
>  		__u16 __user *p_u16;
>  		__u32 __user *p_u32;
> +		struct v4l2_ctrl_mpeg2_frame_hdr __user
> *p_mpeg2_frame_hdr;
>  		void __user *ptr;
>  	};
>  } __attribute__ ((packed));
> @@ -1631,6 +1633,7 @@ enum v4l2_ctrl_type {
>  	V4L2_CTRL_TYPE_U8	     = 0x0100,
>  	V4L2_CTRL_TYPE_U16	     = 0x0101,
>  	V4L2_CTRL_TYPE_U32	     = 0x0102,
> +	V4L2_CTRL_TYPE_MPEG2_FRAME_HDR = 0x0109,
>  };
>  
>  /*  Used in the VIDIOC_QUERYCTRL ioctl for querying controls */
-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/5e7e40ea/attachment.sig>

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 629 bytes --]

On Fri, 2018-05-04 at 10:47 +0200, Paul Kocialkowski wrote:
> > > > Don't you also need to map the SRAM on the A20?
> > > 
> > > That's a good point, there is currently no syscon handle for A20
> > > (and
> > > also A13). Maybe SRAM is muxed to the VE by default so it "just
> > > works"? 

I just checked on the manual and it appears that SRAM Area C1 is muxed
to the VE at reset, so we can probably keep things as-is until the SRAM
driver is ready to handle explicitly muxing that area to the VE.

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 614 bytes --]

On Fri, 2018-05-04 at 10:47 +0200, Paul Kocialkowski wrote:
> > > > Don't you also need to map the SRAM on the A20?
> > > 
> > > That's a good point, there is currently no syscon handle for A20
> > > (and
> > > also A13). Maybe SRAM is muxed to the VE by default so it "just
> > > works"? 

I just checked on the manual and it appears that SRAM Area C1 is muxed
to the VE at reset, so we can probably keep things as-is until the SRAM
driver is ready to handle explicitly muxing that area to the VE.

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

On Fri, 2018-05-04 at 10:47 +0200, Paul Kocialkowski wrote:
> > > > Don't you also need to map the SRAM on the A20?
> > > 
> > > That's a good point, there is currently no syscon handle for A20
> > > (and
> > > also A13). Maybe SRAM is muxed to the VE by default so it "just
> > > works"? 

I just checked on the manual and it appears that SRAM Area C1 is muxed
to the VE at reset, so we can probably keep things as-is until the SRAM
driver is ready to handle explicitly muxing that area to the VE.

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/ea0e5029/attachment.sig>

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 2019 bytes --]

On Fri, May 04, 2018 at 10:47:44AM +0200, Paul Kocialkowski wrote:
> > > > > +			reg = <0x01c0e000 0x1000>;
> > > > > +			memory-region = <&ve_memory>;
> > > > 
> > > > Since you made the CMA region the default one, you don't need to
> > > > tie
> > > > it to that device in particular (and you can drop it being
> > > > mandatory
> > > > from your binding as well).
> > > 
> > > What if another driver (or the system) claims memory from that zone
> > > and
> > > that the reserved memory ends up not being available for the VPU
> > > anymore?
> > > 
> > > Acccording to the reserved-memory documentation, the reusable
> > > property
> > > (that we need for dmabuf) puts a limitation that the device driver
> > > owning the region must be able to reclaim it back.
> > > 
> > > How does that work out if the CMA region is not tied to a driver in
> > > particular?
> > 
> > I'm not sure to get what you're saying. You have the property
> > linux,cma-default in your reserved region, so the behaviour you
> > described is what you explicitly asked for.
> 
> My point is that I don't see how the driver can claim back (part of) the
> reserved area if the area is not explicitly attached to it.
> 
> Or is that mechanism made in a way that all drivers wishing to use the
> reserved memory area can claim it back from the system, but there is no
> priority (other than first-come first-served) for which drivers claims
> it back in case two want to use the same reserved region (in a scenario
> where there isn't enough memory to allow both drivers)?

This is indeed what happens. Reusable is to let the system use the
reserved memory for things like caches that can easily be dropped when
a driver wants to use the memory in that reserved area. Once that
memory has been allocated, there's no claiming back, unless that
memory segment was freed of course.

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 1970 bytes --]

On Fri, May 04, 2018 at 10:47:44AM +0200, Paul Kocialkowski wrote:
> > > > > +			reg = <0x01c0e000 0x1000>;
> > > > > +			memory-region = <&ve_memory>;
> > > > 
> > > > Since you made the CMA region the default one, you don't need to
> > > > tie
> > > > it to that device in particular (and you can drop it being
> > > > mandatory
> > > > from your binding as well).
> > > 
> > > What if another driver (or the system) claims memory from that zone
> > > and
> > > that the reserved memory ends up not being available for the VPU
> > > anymore?
> > > 
> > > Acccording to the reserved-memory documentation, the reusable
> > > property
> > > (that we need for dmabuf) puts a limitation that the device driver
> > > owning the region must be able to reclaim it back.
> > > 
> > > How does that work out if the CMA region is not tied to a driver in
> > > particular?
> > 
> > I'm not sure to get what you're saying. You have the property
> > linux,cma-default in your reserved region, so the behaviour you
> > described is what you explicitly asked for.
> 
> My point is that I don't see how the driver can claim back (part of) the
> reserved area if the area is not explicitly attached to it.
> 
> Or is that mechanism made in a way that all drivers wishing to use the
> reserved memory area can claim it back from the system, but there is no
> priority (other than first-come first-served) for which drivers claims
> it back in case two want to use the same reserved region (in a scenario
> where there isn't enough memory to allow both drivers)?

This is indeed what happens. Reusable is to let the system use the
reserved memory for things like caches that can easily be dropped when
a driver wants to use the memory in that reserved area. Once that
memory has been allocated, there's no claiming back, unless that
memory segment was freed of course.

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Maxime Ripard]

On Fri, May 04, 2018 at 10:47:44AM +0200, Paul Kocialkowski wrote:
> > > > > +			reg = <0x01c0e000 0x1000>;
> > > > > +			memory-region = <&ve_memory>;
> > > > 
> > > > Since you made the CMA region the default one, you don't need to
> > > > tie
> > > > it to that device in particular (and you can drop it being
> > > > mandatory
> > > > from your binding as well).
> > > 
> > > What if another driver (or the system) claims memory from that zone
> > > and
> > > that the reserved memory ends up not being available for the VPU
> > > anymore?
> > > 
> > > Acccording to the reserved-memory documentation, the reusable
> > > property
> > > (that we need for dmabuf) puts a limitation that the device driver
> > > owning the region must be able to reclaim it back.
> > > 
> > > How does that work out if the CMA region is not tied to a driver in
> > > particular?
> > 
> > I'm not sure to get what you're saying. You have the property
> > linux,cma-default in your reserved region, so the behaviour you
> > described is what you explicitly asked for.
> 
> My point is that I don't see how the driver can claim back (part of) the
> reserved area if the area is not explicitly attached to it.
> 
> Or is that mechanism made in a way that all drivers wishing to use the
> reserved memory area can claim it back from the system, but there is no
> priority (other than first-come first-served) for which drivers claims
> it back in case two want to use the same reserved region (in a scenario
> where there isn't enough memory to allow both drivers)?

This is indeed what happens. Reusable is to let the system use the
reserved memory for things like caches that can easily be dropped when
a driver wants to use the memory in that reserved area. Once that
memory has been allocated, there's no claiming back, unless that
memory segment was freed of course.

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/41c1fcef/attachment-0001.sig>

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 2472 bytes --]

Hi,

On Fri, 2018-05-04 at 11:15 +0200, Maxime Ripard wrote:
> On Fri, May 04, 2018 at 10:47:44AM +0200, Paul Kocialkowski wrote:
> > > > > > +			reg = <0x01c0e000 0x1000>;
> > > > > > +			memory-region = <&ve_memory>;
> > > > > 
> > > > > Since you made the CMA region the default one, you don't need
> > > > > to
> > > > > tie
> > > > > it to that device in particular (and you can drop it being
> > > > > mandatory
> > > > > from your binding as well).
> > > > 
> > > > What if another driver (or the system) claims memory from that
> > > > zone
> > > > and
> > > > that the reserved memory ends up not being available for the VPU
> > > > anymore?
> > > > 
> > > > Acccording to the reserved-memory documentation, the reusable
> > > > property
> > > > (that we need for dmabuf) puts a limitation that the device
> > > > driver
> > > > owning the region must be able to reclaim it back.
> > > > 
> > > > How does that work out if the CMA region is not tied to a driver
> > > > in
> > > > particular?
> > > 
> > > I'm not sure to get what you're saying. You have the property
> > > linux,cma-default in your reserved region, so the behaviour you
> > > described is what you explicitly asked for.
> > 
> > My point is that I don't see how the driver can claim back (part of)
> > the
> > reserved area if the area is not explicitly attached to it.
> > 
> > Or is that mechanism made in a way that all drivers wishing to use
> > the
> > reserved memory area can claim it back from the system, but there is
> > no
> > priority (other than first-come first-served) for which drivers
> > claims
> > it back in case two want to use the same reserved region (in a
> > scenario
> > where there isn't enough memory to allow both drivers)?
> 
> This is indeed what happens. Reusable is to let the system use the
> reserved memory for things like caches that can easily be dropped when
> a driver wants to use the memory in that reserved area. Once that
> memory has been allocated, there's no claiming back, unless that
> memory segment was freed of course.

Thanks for the clarification. So in our case, perhaps the best fit would
be to make that area the default CMA pool so that we can be ensured that
the whole 96 MiB is available for the VPU and that no other consumer of
CMA will use it?

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 2408 bytes --]

Hi,

On Fri, 2018-05-04 at 11:15 +0200, Maxime Ripard wrote:
> On Fri, May 04, 2018 at 10:47:44AM +0200, Paul Kocialkowski wrote:
> > > > > > +			reg = <0x01c0e000 0x1000>;
> > > > > > +			memory-region = <&ve_memory>;
> > > > > 
> > > > > Since you made the CMA region the default one, you don't need
> > > > > to
> > > > > tie
> > > > > it to that device in particular (and you can drop it being
> > > > > mandatory
> > > > > from your binding as well).
> > > > 
> > > > What if another driver (or the system) claims memory from that
> > > > zone
> > > > and
> > > > that the reserved memory ends up not being available for the VPU
> > > > anymore?
> > > > 
> > > > Acccording to the reserved-memory documentation, the reusable
> > > > property
> > > > (that we need for dmabuf) puts a limitation that the device
> > > > driver
> > > > owning the region must be able to reclaim it back.
> > > > 
> > > > How does that work out if the CMA region is not tied to a driver
> > > > in
> > > > particular?
> > > 
> > > I'm not sure to get what you're saying. You have the property
> > > linux,cma-default in your reserved region, so the behaviour you
> > > described is what you explicitly asked for.
> > 
> > My point is that I don't see how the driver can claim back (part of)
> > the
> > reserved area if the area is not explicitly attached to it.
> > 
> > Or is that mechanism made in a way that all drivers wishing to use
> > the
> > reserved memory area can claim it back from the system, but there is
> > no
> > priority (other than first-come first-served) for which drivers
> > claims
> > it back in case two want to use the same reserved region (in a
> > scenario
> > where there isn't enough memory to allow both drivers)?
> 
> This is indeed what happens. Reusable is to let the system use the
> reserved memory for things like caches that can easily be dropped when
> a driver wants to use the memory in that reserved area. Once that
> memory has been allocated, there's no claiming back, unless that
> memory segment was freed of course.

Thanks for the clarification. So in our case, perhaps the best fit would
be to make that area the default CMA pool so that we can be ensured that
the whole 96 MiB is available for the VPU and that no other consumer of
CMA will use it?

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

Hi,

On Fri, 2018-05-04 at 11:15 +0200, Maxime Ripard wrote:
> On Fri, May 04, 2018 at 10:47:44AM +0200, Paul Kocialkowski wrote:
> > > > > > +			reg = <0x01c0e000 0x1000>;
> > > > > > +			memory-region = <&ve_memory>;
> > > > > 
> > > > > Since you made the CMA region the default one, you don't need
> > > > > to
> > > > > tie
> > > > > it to that device in particular (and you can drop it being
> > > > > mandatory
> > > > > from your binding as well).
> > > > 
> > > > What if another driver (or the system) claims memory from that
> > > > zone
> > > > and
> > > > that the reserved memory ends up not being available for the VPU
> > > > anymore?
> > > > 
> > > > Acccording to the reserved-memory documentation, the reusable
> > > > property
> > > > (that we need for dmabuf) puts a limitation that the device
> > > > driver
> > > > owning the region must be able to reclaim it back.
> > > > 
> > > > How does that work out if the CMA region is not tied to a driver
> > > > in
> > > > particular?
> > > 
> > > I'm not sure to get what you're saying. You have the property
> > > linux,cma-default in your reserved region, so the behaviour you
> > > described is what you explicitly asked for.
> > 
> > My point is that I don't see how the driver can claim back (part of)
> > the
> > reserved area if the area is not explicitly attached to it.
> > 
> > Or is that mechanism made in a way that all drivers wishing to use
> > the
> > reserved memory area can claim it back from the system, but there is
> > no
> > priority (other than first-come first-served) for which drivers
> > claims
> > it back in case two want to use the same reserved region (in a
> > scenario
> > where there isn't enough memory to allow both drivers)?
> 
> This is indeed what happens. Reusable is to let the system use the
> reserved memory for things like caches that can easily be dropped when
> a driver wants to use the memory in that reserved area. Once that
> memory has been allocated, there's no claiming back, unless that
> memory segment was freed of course.

Thanks for the clarification. So in our case, perhaps the best fit would
be to make that area the default CMA pool so that we can be ensured that
the whole 96 MiB is available for the VPU and that no other consumer of
CMA will use it?

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/dc8274eb/attachment-0001.sig>

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 2960 bytes --]

On Fri, May 04, 2018 at 02:04:38PM +0200, Paul Kocialkowski wrote:
> On Fri, 2018-05-04 at 11:15 +0200, Maxime Ripard wrote:
> > On Fri, May 04, 2018 at 10:47:44AM +0200, Paul Kocialkowski wrote:
> > > > > > > +			reg = <0x01c0e000 0x1000>;
> > > > > > > +			memory-region = <&ve_memory>;
> > > > > > 
> > > > > > Since you made the CMA region the default one, you don't need
> > > > > > to
> > > > > > tie
> > > > > > it to that device in particular (and you can drop it being
> > > > > > mandatory
> > > > > > from your binding as well).
> > > > > 
> > > > > What if another driver (or the system) claims memory from that
> > > > > zone
> > > > > and
> > > > > that the reserved memory ends up not being available for the VPU
> > > > > anymore?
> > > > > 
> > > > > Acccording to the reserved-memory documentation, the reusable
> > > > > property
> > > > > (that we need for dmabuf) puts a limitation that the device
> > > > > driver
> > > > > owning the region must be able to reclaim it back.
> > > > > 
> > > > > How does that work out if the CMA region is not tied to a driver
> > > > > in
> > > > > particular?
> > > > 
> > > > I'm not sure to get what you're saying. You have the property
> > > > linux,cma-default in your reserved region, so the behaviour you
> > > > described is what you explicitly asked for.
> > > 
> > > My point is that I don't see how the driver can claim back (part of)
> > > the
> > > reserved area if the area is not explicitly attached to it.
> > > 
> > > Or is that mechanism made in a way that all drivers wishing to use
> > > the
> > > reserved memory area can claim it back from the system, but there is
> > > no
> > > priority (other than first-come first-served) for which drivers
> > > claims
> > > it back in case two want to use the same reserved region (in a
> > > scenario
> > > where there isn't enough memory to allow both drivers)?
> > 
> > This is indeed what happens. Reusable is to let the system use the
> > reserved memory for things like caches that can easily be dropped when
> > a driver wants to use the memory in that reserved area. Once that
> > memory has been allocated, there's no claiming back, unless that
> > memory segment was freed of course.
> 
> Thanks for the clarification. So in our case, perhaps the best fit would
> be to make that area the default CMA pool so that we can be ensured that
> the whole 96 MiB is available for the VPU and that no other consumer of
> CMA will use it?

The best fit for what use case ? We already discussed this, and I
don't see any point in having two separate CMA regions. If you have a
reasonably sized region that will accomodate for both the VPU and
display engine, why would we want to split them?

Or did you have any experience of running out of buffers?

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 2888 bytes --]

On Fri, May 04, 2018 at 02:04:38PM +0200, Paul Kocialkowski wrote:
> On Fri, 2018-05-04 at 11:15 +0200, Maxime Ripard wrote:
> > On Fri, May 04, 2018 at 10:47:44AM +0200, Paul Kocialkowski wrote:
> > > > > > > +			reg = <0x01c0e000 0x1000>;
> > > > > > > +			memory-region = <&ve_memory>;
> > > > > > 
> > > > > > Since you made the CMA region the default one, you don't need
> > > > > > to
> > > > > > tie
> > > > > > it to that device in particular (and you can drop it being
> > > > > > mandatory
> > > > > > from your binding as well).
> > > > > 
> > > > > What if another driver (or the system) claims memory from that
> > > > > zone
> > > > > and
> > > > > that the reserved memory ends up not being available for the VPU
> > > > > anymore?
> > > > > 
> > > > > Acccording to the reserved-memory documentation, the reusable
> > > > > property
> > > > > (that we need for dmabuf) puts a limitation that the device
> > > > > driver
> > > > > owning the region must be able to reclaim it back.
> > > > > 
> > > > > How does that work out if the CMA region is not tied to a driver
> > > > > in
> > > > > particular?
> > > > 
> > > > I'm not sure to get what you're saying. You have the property
> > > > linux,cma-default in your reserved region, so the behaviour you
> > > > described is what you explicitly asked for.
> > > 
> > > My point is that I don't see how the driver can claim back (part of)
> > > the
> > > reserved area if the area is not explicitly attached to it.
> > > 
> > > Or is that mechanism made in a way that all drivers wishing to use
> > > the
> > > reserved memory area can claim it back from the system, but there is
> > > no
> > > priority (other than first-come first-served) for which drivers
> > > claims
> > > it back in case two want to use the same reserved region (in a
> > > scenario
> > > where there isn't enough memory to allow both drivers)?
> > 
> > This is indeed what happens. Reusable is to let the system use the
> > reserved memory for things like caches that can easily be dropped when
> > a driver wants to use the memory in that reserved area. Once that
> > memory has been allocated, there's no claiming back, unless that
> > memory segment was freed of course.
> 
> Thanks for the clarification. So in our case, perhaps the best fit would
> be to make that area the default CMA pool so that we can be ensured that
> the whole 96 MiB is available for the VPU and that no other consumer of
> CMA will use it?

The best fit for what use case ? We already discussed this, and I
don't see any point in having two separate CMA regions. If you have a
reasonably sized region that will accomodate for both the VPU and
display engine, why would we want to split them?

Or did you have any experience of running out of buffers?

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Maxime Ripard]

On Fri, May 04, 2018 at 02:04:38PM +0200, Paul Kocialkowski wrote:
> On Fri, 2018-05-04 at 11:15 +0200, Maxime Ripard wrote:
> > On Fri, May 04, 2018 at 10:47:44AM +0200, Paul Kocialkowski wrote:
> > > > > > > +			reg = <0x01c0e000 0x1000>;
> > > > > > > +			memory-region = <&ve_memory>;
> > > > > > 
> > > > > > Since you made the CMA region the default one, you don't need
> > > > > > to
> > > > > > tie
> > > > > > it to that device in particular (and you can drop it being
> > > > > > mandatory
> > > > > > from your binding as well).
> > > > > 
> > > > > What if another driver (or the system) claims memory from that
> > > > > zone
> > > > > and
> > > > > that the reserved memory ends up not being available for the VPU
> > > > > anymore?
> > > > > 
> > > > > Acccording to the reserved-memory documentation, the reusable
> > > > > property
> > > > > (that we need for dmabuf) puts a limitation that the device
> > > > > driver
> > > > > owning the region must be able to reclaim it back.
> > > > > 
> > > > > How does that work out if the CMA region is not tied to a driver
> > > > > in
> > > > > particular?
> > > > 
> > > > I'm not sure to get what you're saying. You have the property
> > > > linux,cma-default in your reserved region, so the behaviour you
> > > > described is what you explicitly asked for.
> > > 
> > > My point is that I don't see how the driver can claim back (part of)
> > > the
> > > reserved area if the area is not explicitly attached to it.
> > > 
> > > Or is that mechanism made in a way that all drivers wishing to use
> > > the
> > > reserved memory area can claim it back from the system, but there is
> > > no
> > > priority (other than first-come first-served) for which drivers
> > > claims
> > > it back in case two want to use the same reserved region (in a
> > > scenario
> > > where there isn't enough memory to allow both drivers)?
> > 
> > This is indeed what happens. Reusable is to let the system use the
> > reserved memory for things like caches that can easily be dropped when
> > a driver wants to use the memory in that reserved area. Once that
> > memory has been allocated, there's no claiming back, unless that
> > memory segment was freed of course.
> 
> Thanks for the clarification. So in our case, perhaps the best fit would
> be to make that area the default CMA pool so that we can be ensured that
> the whole 96 MiB is available for the VPU and that no other consumer of
> CMA will use it?

The best fit for what use case ? We already discussed this, and I
don't see any point in having two separate CMA regions. If you have a
reasonably sized region that will accomodate for both the VPU and
display engine, why would we want to split them?

Or did you have any experience of running out of buffers?

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/d2d87aa3/attachment.sig>

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 4425 bytes --]

On Fri, 2018-05-04 at 15:40 +0200, Maxime Ripard wrote:
> On Fri, May 04, 2018 at 02:04:38PM +0200, Paul Kocialkowski wrote:
> > On Fri, 2018-05-04 at 11:15 +0200, Maxime Ripard wrote:
> > > On Fri, May 04, 2018 at 10:47:44AM +0200, Paul Kocialkowski wrote:
> > > > > > > > +			reg = <0x01c0e000 0x1000>;
> > > > > > > > +			memory-region = <&ve_memory>;
> > > > > > > 
> > > > > > > Since you made the CMA region the default one, you don't
> > > > > > > need
> > > > > > > to
> > > > > > > tie
> > > > > > > it to that device in particular (and you can drop it being
> > > > > > > mandatory
> > > > > > > from your binding as well).
> > > > > > 
> > > > > > What if another driver (or the system) claims memory from
> > > > > > that
> > > > > > zone
> > > > > > and
> > > > > > that the reserved memory ends up not being available for the
> > > > > > VPU
> > > > > > anymore?
> > > > > > 
> > > > > > Acccording to the reserved-memory documentation, the
> > > > > > reusable
> > > > > > property
> > > > > > (that we need for dmabuf) puts a limitation that the device
> > > > > > driver
> > > > > > owning the region must be able to reclaim it back.
> > > > > > 
> > > > > > How does that work out if the CMA region is not tied to a
> > > > > > driver
> > > > > > in
> > > > > > particular?
> > > > > 
> > > > > I'm not sure to get what you're saying. You have the property
> > > > > linux,cma-default in your reserved region, so the behaviour
> > > > > you
> > > > > described is what you explicitly asked for.
> > > > 
> > > > My point is that I don't see how the driver can claim back (part
> > > > of)
> > > > the
> > > > reserved area if the area is not explicitly attached to it.
> > > > 
> > > > Or is that mechanism made in a way that all drivers wishing to
> > > > use
> > > > the
> > > > reserved memory area can claim it back from the system, but
> > > > there is
> > > > no
> > > > priority (other than first-come first-served) for which drivers
> > > > claims
> > > > it back in case two want to use the same reserved region (in a
> > > > scenario
> > > > where there isn't enough memory to allow both drivers)?
> > > 
> > > This is indeed what happens. Reusable is to let the system use the
> > > reserved memory for things like caches that can easily be dropped
> > > when
> > > a driver wants to use the memory in that reserved area. Once that
> > > memory has been allocated, there's no claiming back, unless that
> > > memory segment was freed of course.
> > 
> > Thanks for the clarification. So in our case, perhaps the best fit
> > would
> > be to make that area the default CMA pool so that we can be ensured
> > that
> > the whole 96 MiB is available for the VPU and that no other consumer
> > of
> > CMA will use it?
> 
> The best fit for what use case ? We already discussed this, and I
> don't see any point in having two separate CMA regions. If you have a
> reasonably sized region that will accomodate for both the VPU and
> display engine, why would we want to split them?

The use case I have in mind is boilerplate use of the VPU with the
display engine, say with DMAbuf.

It wasn't exactly clear in my memory whether we had decided that the CMA
pool we use for the VPU should also be used for other CMA consumers (I
realize that this is what we've been doing all along by having
linux,cma-default; though).

The fact that the memory region will accomodate for both the display
engine and the VPU is not straightforward IMO and I think this has to be
an explicit choice that we take. I was under the impression that we
chose the 96 MiB because that's what the Allwinner reference code does.
Does the reference code also use this pool for display?

I liked the idea of having 96 MiB fully reserved to the VPU because it
allows us to provide a limit on the use case, such as "this guarantees N
buffers for resolution foo in format bar". If the display engine also
uses it, then the limit also depends on how many GEM buffers are
allocated (unless I'm missing something).

> Or did you have any experience of running out of buffers?

Not yet, I haven't. I have no objection with making the reserved region
the default CMA pool and have other consumers use it too.

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

[-- Attachment #1: Type: text/plain, Size: 4318 bytes --]

On Fri, 2018-05-04 at 15:40 +0200, Maxime Ripard wrote:
> On Fri, May 04, 2018 at 02:04:38PM +0200, Paul Kocialkowski wrote:
> > On Fri, 2018-05-04 at 11:15 +0200, Maxime Ripard wrote:
> > > On Fri, May 04, 2018 at 10:47:44AM +0200, Paul Kocialkowski wrote:
> > > > > > > > +			reg = <0x01c0e000 0x1000>;
> > > > > > > > +			memory-region = <&ve_memory>;
> > > > > > > 
> > > > > > > Since you made the CMA region the default one, you don't
> > > > > > > need
> > > > > > > to
> > > > > > > tie
> > > > > > > it to that device in particular (and you can drop it being
> > > > > > > mandatory
> > > > > > > from your binding as well).
> > > > > > 
> > > > > > What if another driver (or the system) claims memory from
> > > > > > that
> > > > > > zone
> > > > > > and
> > > > > > that the reserved memory ends up not being available for the
> > > > > > VPU
> > > > > > anymore?
> > > > > > 
> > > > > > Acccording to the reserved-memory documentation, the
> > > > > > reusable
> > > > > > property
> > > > > > (that we need for dmabuf) puts a limitation that the device
> > > > > > driver
> > > > > > owning the region must be able to reclaim it back.
> > > > > > 
> > > > > > How does that work out if the CMA region is not tied to a
> > > > > > driver
> > > > > > in
> > > > > > particular?
> > > > > 
> > > > > I'm not sure to get what you're saying. You have the property
> > > > > linux,cma-default in your reserved region, so the behaviour
> > > > > you
> > > > > described is what you explicitly asked for.
> > > > 
> > > > My point is that I don't see how the driver can claim back (part
> > > > of)
> > > > the
> > > > reserved area if the area is not explicitly attached to it.
> > > > 
> > > > Or is that mechanism made in a way that all drivers wishing to
> > > > use
> > > > the
> > > > reserved memory area can claim it back from the system, but
> > > > there is
> > > > no
> > > > priority (other than first-come first-served) for which drivers
> > > > claims
> > > > it back in case two want to use the same reserved region (in a
> > > > scenario
> > > > where there isn't enough memory to allow both drivers)?
> > > 
> > > This is indeed what happens. Reusable is to let the system use the
> > > reserved memory for things like caches that can easily be dropped
> > > when
> > > a driver wants to use the memory in that reserved area. Once that
> > > memory has been allocated, there's no claiming back, unless that
> > > memory segment was freed of course.
> > 
> > Thanks for the clarification. So in our case, perhaps the best fit
> > would
> > be to make that area the default CMA pool so that we can be ensured
> > that
> > the whole 96 MiB is available for the VPU and that no other consumer
> > of
> > CMA will use it?
> 
> The best fit for what use case ? We already discussed this, and I
> don't see any point in having two separate CMA regions. If you have a
> reasonably sized region that will accomodate for both the VPU and
> display engine, why would we want to split them?

The use case I have in mind is boilerplate use of the VPU with the
display engine, say with DMAbuf.

It wasn't exactly clear in my memory whether we had decided that the CMA
pool we use for the VPU should also be used for other CMA consumers (I
realize that this is what we've been doing all along by having
linux,cma-default; though).

The fact that the memory region will accomodate for both the display
engine and the VPU is not straightforward IMO and I think this has to be
an explicit choice that we take. I was under the impression that we
chose the 96 MiB because that's what the Allwinner reference code does.
Does the reference code also use this pool for display?

I liked the idea of having 96 MiB fully reserved to the VPU because it
allows us to provide a limit on the use case, such as "this guarantees N
buffers for resolution foo in format bar". If the display engine also
uses it, then the limit also depends on how many GEM buffers are
allocated (unless I'm missing something).

> Or did you have any experience of running out of buffers?

Not yet, I haven't. I have no objection with making the reserved region
the default CMA pool and have other consumers use it too.

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com

[PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Paul Kocialkowski]

On Fri, 2018-05-04 at 15:40 +0200, Maxime Ripard wrote:
> On Fri, May 04, 2018 at 02:04:38PM +0200, Paul Kocialkowski wrote:
> > On Fri, 2018-05-04 at 11:15 +0200, Maxime Ripard wrote:
> > > On Fri, May 04, 2018 at 10:47:44AM +0200, Paul Kocialkowski wrote:
> > > > > > > > +			reg = <0x01c0e000 0x1000>;
> > > > > > > > +			memory-region = <&ve_memory>;
> > > > > > > 
> > > > > > > Since you made the CMA region the default one, you don't
> > > > > > > need
> > > > > > > to
> > > > > > > tie
> > > > > > > it to that device in particular (and you can drop it being
> > > > > > > mandatory
> > > > > > > from your binding as well).
> > > > > > 
> > > > > > What if another driver (or the system) claims memory from
> > > > > > that
> > > > > > zone
> > > > > > and
> > > > > > that the reserved memory ends up not being available for the
> > > > > > VPU
> > > > > > anymore?
> > > > > > 
> > > > > > Acccording to the reserved-memory documentation, the
> > > > > > reusable
> > > > > > property
> > > > > > (that we need for dmabuf) puts a limitation that the device
> > > > > > driver
> > > > > > owning the region must be able to reclaim it back.
> > > > > > 
> > > > > > How does that work out if the CMA region is not tied to a
> > > > > > driver
> > > > > > in
> > > > > > particular?
> > > > > 
> > > > > I'm not sure to get what you're saying. You have the property
> > > > > linux,cma-default in your reserved region, so the behaviour
> > > > > you
> > > > > described is what you explicitly asked for.
> > > > 
> > > > My point is that I don't see how the driver can claim back (part
> > > > of)
> > > > the
> > > > reserved area if the area is not explicitly attached to it.
> > > > 
> > > > Or is that mechanism made in a way that all drivers wishing to
> > > > use
> > > > the
> > > > reserved memory area can claim it back from the system, but
> > > > there is
> > > > no
> > > > priority (other than first-come first-served) for which drivers
> > > > claims
> > > > it back in case two want to use the same reserved region (in a
> > > > scenario
> > > > where there isn't enough memory to allow both drivers)?
> > > 
> > > This is indeed what happens. Reusable is to let the system use the
> > > reserved memory for things like caches that can easily be dropped
> > > when
> > > a driver wants to use the memory in that reserved area. Once that
> > > memory has been allocated, there's no claiming back, unless that
> > > memory segment was freed of course.
> > 
> > Thanks for the clarification. So in our case, perhaps the best fit
> > would
> > be to make that area the default CMA pool so that we can be ensured
> > that
> > the whole 96 MiB is available for the VPU and that no other consumer
> > of
> > CMA will use it?
> 
> The best fit for what use case ? We already discussed this, and I
> don't see any point in having two separate CMA regions. If you have a
> reasonably sized region that will accomodate for both the VPU and
> display engine, why would we want to split them?

The use case I have in mind is boilerplate use of the VPU with the
display engine, say with DMAbuf.

It wasn't exactly clear in my memory whether we had decided that the CMA
pool we use for the VPU should also be used for other CMA consumers (I
realize that this is what we've been doing all along by having
linux,cma-default; though).

The fact that the memory region will accomodate for both the display
engine and the VPU is not straightforward IMO and I think this has to be
an explicit choice that we take. I was under the impression that we
chose the 96 MiB because that's what the Allwinner reference code does.
Does the reference code also use this pool for display?

I liked the idea of having 96 MiB fully reserved to the VPU because it
allows us to provide a limit on the use case, such as "this guarantees N
buffers for resolution foo in format bar". If the display engine also
uses it, then the limit also depends on how many GEM buffers are
allocated (unless I'm missing something).

> Or did you have any experience of running out of buffers?

Not yet, I haven't. I have no objection with making the reserved region
the default CMA pool and have other consumers use it too.

Cheers,

-- 
Paul Kocialkowski, Bootlin (formerly Free Electrons)
Embedded Linux and kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/84c48205/attachment.sig>

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 5142 bytes --]

On Fri, May 04, 2018 at 03:57:48PM +0200, Paul Kocialkowski wrote:
> On Fri, 2018-05-04 at 15:40 +0200, Maxime Ripard wrote:
> > On Fri, May 04, 2018 at 02:04:38PM +0200, Paul Kocialkowski wrote:
> > > On Fri, 2018-05-04 at 11:15 +0200, Maxime Ripard wrote:
> > > > On Fri, May 04, 2018 at 10:47:44AM +0200, Paul Kocialkowski wrote:
> > > > > > > > > +			reg = <0x01c0e000 0x1000>;
> > > > > > > > > +			memory-region = <&ve_memory>;
> > > > > > > > 
> > > > > > > > Since you made the CMA region the default one, you don't
> > > > > > > > need
> > > > > > > > to
> > > > > > > > tie
> > > > > > > > it to that device in particular (and you can drop it being
> > > > > > > > mandatory
> > > > > > > > from your binding as well).
> > > > > > > 
> > > > > > > What if another driver (or the system) claims memory from
> > > > > > > that
> > > > > > > zone
> > > > > > > and
> > > > > > > that the reserved memory ends up not being available for the
> > > > > > > VPU
> > > > > > > anymore?
> > > > > > > 
> > > > > > > Acccording to the reserved-memory documentation, the
> > > > > > > reusable
> > > > > > > property
> > > > > > > (that we need for dmabuf) puts a limitation that the device
> > > > > > > driver
> > > > > > > owning the region must be able to reclaim it back.
> > > > > > > 
> > > > > > > How does that work out if the CMA region is not tied to a
> > > > > > > driver
> > > > > > > in
> > > > > > > particular?
> > > > > > 
> > > > > > I'm not sure to get what you're saying. You have the property
> > > > > > linux,cma-default in your reserved region, so the behaviour
> > > > > > you
> > > > > > described is what you explicitly asked for.
> > > > > 
> > > > > My point is that I don't see how the driver can claim back (part
> > > > > of)
> > > > > the
> > > > > reserved area if the area is not explicitly attached to it.
> > > > > 
> > > > > Or is that mechanism made in a way that all drivers wishing to
> > > > > use
> > > > > the
> > > > > reserved memory area can claim it back from the system, but
> > > > > there is
> > > > > no
> > > > > priority (other than first-come first-served) for which drivers
> > > > > claims
> > > > > it back in case two want to use the same reserved region (in a
> > > > > scenario
> > > > > where there isn't enough memory to allow both drivers)?
> > > > 
> > > > This is indeed what happens. Reusable is to let the system use the
> > > > reserved memory for things like caches that can easily be dropped
> > > > when
> > > > a driver wants to use the memory in that reserved area. Once that
> > > > memory has been allocated, there's no claiming back, unless that
> > > > memory segment was freed of course.
> > > 
> > > Thanks for the clarification. So in our case, perhaps the best fit
> > > would
> > > be to make that area the default CMA pool so that we can be ensured
> > > that
> > > the whole 96 MiB is available for the VPU and that no other consumer
> > > of
> > > CMA will use it?
> > 
> > The best fit for what use case ? We already discussed this, and I
> > don't see any point in having two separate CMA regions. If you have a
> > reasonably sized region that will accomodate for both the VPU and
> > display engine, why would we want to split them?
> 
> The use case I have in mind is boilerplate use of the VPU with the
> display engine, say with DMAbuf.
> 
> It wasn't exactly clear in my memory whether we had decided that the CMA
> pool we use for the VPU should also be used for other CMA consumers (I
> realize that this is what we've been doing all along by having
> linux,cma-default; though).
> 
> The fact that the memory region will accomodate for both the display
> engine and the VPU is not straightforward IMO and I think this has to be
> an explicit choice that we take. I was under the impression that we
> chose the 96 MiB because that's what the Allwinner reference code does.
> Does the reference code also use this pool for display?

Yes

> I liked the idea of having 96 MiB fully reserved to the VPU because it
> allows us to provide a limit on the use case, such as "this guarantees N
> buffers for resolution foo in format bar". If the display engine also
> uses it, then the limit also depends on how many GEM buffers are
> allocated (unless I'm missing something).

This also guarantees that you take away a fifth of the RAM on some
boards. If we had yet another CMA pool of 64MB as is the multi_v7
defconfig, that's a third of your RAM that's gone, possibly for no
particular reason.

If we make the math, let's say that we are running a system with 4
planes used in 1080p, with 4 Bpp, in double buffering (which is
already an unlikely setup). Let's add on top of that that we're
decoding a 1080p video with 8 buffers pre-allocated with 2Bpp (in
YUV422). Which really seems extreme now :)

And we're at 80MB. My guess is that your memory bus is going to be
dead before you need to use all that memory.

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Maxime Ripard]

[-- Attachment #1: Type: text/plain, Size: 5021 bytes --]

On Fri, May 04, 2018 at 03:57:48PM +0200, Paul Kocialkowski wrote:
> On Fri, 2018-05-04 at 15:40 +0200, Maxime Ripard wrote:
> > On Fri, May 04, 2018 at 02:04:38PM +0200, Paul Kocialkowski wrote:
> > > On Fri, 2018-05-04 at 11:15 +0200, Maxime Ripard wrote:
> > > > On Fri, May 04, 2018 at 10:47:44AM +0200, Paul Kocialkowski wrote:
> > > > > > > > > +			reg = <0x01c0e000 0x1000>;
> > > > > > > > > +			memory-region = <&ve_memory>;
> > > > > > > > 
> > > > > > > > Since you made the CMA region the default one, you don't
> > > > > > > > need
> > > > > > > > to
> > > > > > > > tie
> > > > > > > > it to that device in particular (and you can drop it being
> > > > > > > > mandatory
> > > > > > > > from your binding as well).
> > > > > > > 
> > > > > > > What if another driver (or the system) claims memory from
> > > > > > > that
> > > > > > > zone
> > > > > > > and
> > > > > > > that the reserved memory ends up not being available for the
> > > > > > > VPU
> > > > > > > anymore?
> > > > > > > 
> > > > > > > Acccording to the reserved-memory documentation, the
> > > > > > > reusable
> > > > > > > property
> > > > > > > (that we need for dmabuf) puts a limitation that the device
> > > > > > > driver
> > > > > > > owning the region must be able to reclaim it back.
> > > > > > > 
> > > > > > > How does that work out if the CMA region is not tied to a
> > > > > > > driver
> > > > > > > in
> > > > > > > particular?
> > > > > > 
> > > > > > I'm not sure to get what you're saying. You have the property
> > > > > > linux,cma-default in your reserved region, so the behaviour
> > > > > > you
> > > > > > described is what you explicitly asked for.
> > > > > 
> > > > > My point is that I don't see how the driver can claim back (part
> > > > > of)
> > > > > the
> > > > > reserved area if the area is not explicitly attached to it.
> > > > > 
> > > > > Or is that mechanism made in a way that all drivers wishing to
> > > > > use
> > > > > the
> > > > > reserved memory area can claim it back from the system, but
> > > > > there is
> > > > > no
> > > > > priority (other than first-come first-served) for which drivers
> > > > > claims
> > > > > it back in case two want to use the same reserved region (in a
> > > > > scenario
> > > > > where there isn't enough memory to allow both drivers)?
> > > > 
> > > > This is indeed what happens. Reusable is to let the system use the
> > > > reserved memory for things like caches that can easily be dropped
> > > > when
> > > > a driver wants to use the memory in that reserved area. Once that
> > > > memory has been allocated, there's no claiming back, unless that
> > > > memory segment was freed of course.
> > > 
> > > Thanks for the clarification. So in our case, perhaps the best fit
> > > would
> > > be to make that area the default CMA pool so that we can be ensured
> > > that
> > > the whole 96 MiB is available for the VPU and that no other consumer
> > > of
> > > CMA will use it?
> > 
> > The best fit for what use case ? We already discussed this, and I
> > don't see any point in having two separate CMA regions. If you have a
> > reasonably sized region that will accomodate for both the VPU and
> > display engine, why would we want to split them?
> 
> The use case I have in mind is boilerplate use of the VPU with the
> display engine, say with DMAbuf.
> 
> It wasn't exactly clear in my memory whether we had decided that the CMA
> pool we use for the VPU should also be used for other CMA consumers (I
> realize that this is what we've been doing all along by having
> linux,cma-default; though).
> 
> The fact that the memory region will accomodate for both the display
> engine and the VPU is not straightforward IMO and I think this has to be
> an explicit choice that we take. I was under the impression that we
> chose the 96 MiB because that's what the Allwinner reference code does.
> Does the reference code also use this pool for display?

Yes

> I liked the idea of having 96 MiB fully reserved to the VPU because it
> allows us to provide a limit on the use case, such as "this guarantees N
> buffers for resolution foo in format bar". If the display engine also
> uses it, then the limit also depends on how many GEM buffers are
> allocated (unless I'm missing something).

This also guarantees that you take away a fifth of the RAM on some
boards. If we had yet another CMA pool of 64MB as is the multi_v7
defconfig, that's a third of your RAM that's gone, possibly for no
particular reason.

If we make the math, let's say that we are running a system with 4
planes used in 1080p, with 4 Bpp, in double buffering (which is
already an unlikely setup). Let's add on top of that that we're
decoding a 1080p video with 8 buffers pre-allocated with 2Bpp (in
YUV422). Which really seems extreme now :)

And we're at 80MB. My guess is that your memory bus is going to be
dead before you need to use all that memory.

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com

[PATCH v2 09/10] ARM: dts: sun7i-a20: Add Video Engine and reserved memory nodes

[Maxime Ripard]

On Fri, May 04, 2018 at 03:57:48PM +0200, Paul Kocialkowski wrote:
> On Fri, 2018-05-04 at 15:40 +0200, Maxime Ripard wrote:
> > On Fri, May 04, 2018 at 02:04:38PM +0200, Paul Kocialkowski wrote:
> > > On Fri, 2018-05-04 at 11:15 +0200, Maxime Ripard wrote:
> > > > On Fri, May 04, 2018 at 10:47:44AM +0200, Paul Kocialkowski wrote:
> > > > > > > > > +			reg = <0x01c0e000 0x1000>;
> > > > > > > > > +			memory-region = <&ve_memory>;
> > > > > > > > 
> > > > > > > > Since you made the CMA region the default one, you don't
> > > > > > > > need
> > > > > > > > to
> > > > > > > > tie
> > > > > > > > it to that device in particular (and you can drop it being
> > > > > > > > mandatory
> > > > > > > > from your binding as well).
> > > > > > > 
> > > > > > > What if another driver (or the system) claims memory from
> > > > > > > that
> > > > > > > zone
> > > > > > > and
> > > > > > > that the reserved memory ends up not being available for the
> > > > > > > VPU
> > > > > > > anymore?
> > > > > > > 
> > > > > > > Acccording to the reserved-memory documentation, the
> > > > > > > reusable
> > > > > > > property
> > > > > > > (that we need for dmabuf) puts a limitation that the device
> > > > > > > driver
> > > > > > > owning the region must be able to reclaim it back.
> > > > > > > 
> > > > > > > How does that work out if the CMA region is not tied to a
> > > > > > > driver
> > > > > > > in
> > > > > > > particular?
> > > > > > 
> > > > > > I'm not sure to get what you're saying. You have the property
> > > > > > linux,cma-default in your reserved region, so the behaviour
> > > > > > you
> > > > > > described is what you explicitly asked for.
> > > > > 
> > > > > My point is that I don't see how the driver can claim back (part
> > > > > of)
> > > > > the
> > > > > reserved area if the area is not explicitly attached to it.
> > > > > 
> > > > > Or is that mechanism made in a way that all drivers wishing to
> > > > > use
> > > > > the
> > > > > reserved memory area can claim it back from the system, but
> > > > > there is
> > > > > no
> > > > > priority (other than first-come first-served) for which drivers
> > > > > claims
> > > > > it back in case two want to use the same reserved region (in a
> > > > > scenario
> > > > > where there isn't enough memory to allow both drivers)?
> > > > 
> > > > This is indeed what happens. Reusable is to let the system use the
> > > > reserved memory for things like caches that can easily be dropped
> > > > when
> > > > a driver wants to use the memory in that reserved area. Once that
> > > > memory has been allocated, there's no claiming back, unless that
> > > > memory segment was freed of course.
> > > 
> > > Thanks for the clarification. So in our case, perhaps the best fit
> > > would
> > > be to make that area the default CMA pool so that we can be ensured
> > > that
> > > the whole 96 MiB is available for the VPU and that no other consumer
> > > of
> > > CMA will use it?
> > 
> > The best fit for what use case ? We already discussed this, and I
> > don't see any point in having two separate CMA regions. If you have a
> > reasonably sized region that will accomodate for both the VPU and
> > display engine, why would we want to split them?
> 
> The use case I have in mind is boilerplate use of the VPU with the
> display engine, say with DMAbuf.
> 
> It wasn't exactly clear in my memory whether we had decided that the CMA
> pool we use for the VPU should also be used for other CMA consumers (I
> realize that this is what we've been doing all along by having
> linux,cma-default; though).
> 
> The fact that the memory region will accomodate for both the display
> engine and the VPU is not straightforward IMO and I think this has to be
> an explicit choice that we take. I was under the impression that we
> chose the 96 MiB because that's what the Allwinner reference code does.
> Does the reference code also use this pool for display?

Yes

> I liked the idea of having 96 MiB fully reserved to the VPU because it
> allows us to provide a limit on the use case, such as "this guarantees N
> buffers for resolution foo in format bar". If the display engine also
> uses it, then the limit also depends on how many GEM buffers are
> allocated (unless I'm missing something).

This also guarantees that you take away a fifth of the RAM on some
boards. If we had yet another CMA pool of 64MB as is the multi_v7
defconfig, that's a third of your RAM that's gone, possibly for no
particular reason.

If we make the math, let's say that we are running a system with 4
planes used in 1080p, with 4 Bpp, in double buffering (which is
already an unlikely setup). Let's add on top of that that we're
decoding a 1080p video with 8 buffers pre-allocated with 2Bpp (in
YUV422). Which really seems extreme now :)

And we're at 80MB. My guess is that your memory bus is going to be
dead before you need to use all that memory.

Maxime

-- 
Maxime Ripard, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20180504/dafc7cb2/attachment.sig>