[PATCH 00/18] media: allegro: add HEVC support

[PATCH 00/18] media: allegro: add HEVC support

[Michael Tretter]

Hello,

The Allegro DVT video encoder IP core supports HEVC/H.265 and H.264. This
series adds the HEVC encoding support to the allegro driver.

Patches 1 - 3 add the VPS/SPS/PPS generator that is necessary, because
although being a stateful encoder, the encoder firmware does not produce these
non-VCL NALs. The generator is implemented in the same way as it has been
implemented for H.264 and both share the code to encode bits into RBSP.

Patches 4 - 12 restructure the driver to make it easier to change encoding
parameters on format changes. Patch 7 is special and fixes a bug in the
parameters that are used to configure the encoder channel. It became apparent
when synchronizing the values in the channel configuration with the SPS and,
thus, is included here.

Patches 13 - 15 add helpers for getting codec-specific values in a common
function if the messages to the firmware have only single field for both
codecs.

Patch 16 increases the reserved space at the beginning of the CAPTURE buffers.
The previously reserved space was not sufficient for the HEVC VPS/SPS/PPS
anymore.

Patch 17 adds the deactivation the H.264 specific controls if the coded format
is not H.264.

Patch 18 finally adds the format and controls for HEVC encoding and the code
to configure the encoder accordingly.

The series is based on the series that moved the driver out of staging:

https://lore.kernel.org/linux-media/20201202133040.1954837-1-m.tretter@pengutronix.de/

The v4l2-compliance result is attached.

Michael

v4l2-compliance 1.21.0-4683, 64 bits, 64-bit time_t
v4l2-compliance SHA: 0aee9991e0c0 2020-12-01 09:48:02

Compliance test for allegro device /dev/video3:

Driver Info:
	Driver name      : allegro
	Card type        : Allegro DVT Video Encoder
	Bus info         : platform:a0009000.video-codec
	Driver version   : 5.10.0
	Capabilities     : 0x84208000
		Video Memory-to-Memory
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04208000
		Video Memory-to-Memory
		Streaming
		Extended Pix Format
	Detected Stateful Encoder

Required ioctls:
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second /dev/video3 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

	test invalid ioctls: OK
Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 24 Private Controls: 0

Format ioctls:
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
	test VIDIOC_G/S_PARM: OK
	test VIDIOC_G_FBUF: OK (Not Supported)
	test VIDIOC_G_FMT: OK
	test VIDIOC_TRY_FMT: OK
	test VIDIOC_S_FMT: OK
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
	test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK (Not Supported)

Test input 0:

Streaming ioctls:
	test read/write: OK (Not Supported)
	test blocking wait: OK
	Video Capture: Captured 60 buffers
	test MMAP (select): OK
	Video Capture: Captured 60 buffers
	test MMAP (epoll): OK
	test USERPTR (select): OK (Not Supported)
	test DMABUF: Cannot test, specify --expbuf-device

Total for allegro device /dev/video3: 50, Succeeded: 50, Failed: 0, Warnings: 0

Michael Tretter (18):
  media: allegro: extract RBSP handler from H.264 NAL generator
  media: allegro: add helper to report unsupported fields
  media: allegro: add HEVC NAL unit generator
  media: allegro: implement S_FMT for CAPTURE
  media: allegro: adjust channel after format change
  media: allegro: move encoding options to channel
  media: allegro: fix log2_max_poc in firmware 2019.1
  media: allegro: use handler_setup to configure channel
  media: allegro: initialize bitrate using v4l2_ctrl
  media: allegro: implement scaling of cpb size in SPS
  media: allegro: remove cpb_size and gop_size from channel
  media: allegro: remove profile and level from channel
  media: allegro: use accessor functions for QP values
  media: allegro: add helper to get entropy mode
  media: allegro: rename codec specific functions
  media: allegro: increase offset in CAPTURE buffer
  media: allegro: activate v4l2-ctrls only for current codec
  media: allegro: add support for HEVC encoding

 drivers/media/platform/allegro-dvt/Makefile   |   3 +-
 .../media/platform/allegro-dvt/allegro-core.c | 810 +++++++++++++++--
 .../media/platform/allegro-dvt/allegro-mail.c |  13 +-
 .../media/platform/allegro-dvt/allegro-mail.h |   1 +
 drivers/media/platform/allegro-dvt/nal-h264.c | 336 +------
 drivers/media/platform/allegro-dvt/nal-hevc.c | 824 ++++++++++++++++++
 drivers/media/platform/allegro-dvt/nal-hevc.h | 350 ++++++++
 drivers/media/platform/allegro-dvt/nal-rbsp.c | 310 +++++++
 drivers/media/platform/allegro-dvt/nal-rbsp.h |  61 ++
 9 files changed, 2283 insertions(+), 425 deletions(-)
 create mode 100644 drivers/media/platform/allegro-dvt/nal-hevc.c
 create mode 100644 drivers/media/platform/allegro-dvt/nal-hevc.h
 create mode 100644 drivers/media/platform/allegro-dvt/nal-rbsp.c
 create mode 100644 drivers/media/platform/allegro-dvt/nal-rbsp.h

-- 
2.20.1

[PATCH 01/18] media: allegro: extract RBSP handler from H.264 NAL generator

[Michael Tretter]

The RBSP structure is the same for HEVC and H.264. In order to be able
to reuse the RBSP handler for generating HEVC NAL units, extract the
functions from the H.264 generator.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 drivers/media/platform/allegro-dvt/Makefile   |   3 +-
 drivers/media/platform/allegro-dvt/nal-h264.c | 336 +-----------------
 drivers/media/platform/allegro-dvt/nal-rbsp.c | 305 ++++++++++++++++
 drivers/media/platform/allegro-dvt/nal-rbsp.h |  60 ++++
 4 files changed, 374 insertions(+), 330 deletions(-)
 create mode 100644 drivers/media/platform/allegro-dvt/nal-rbsp.c
 create mode 100644 drivers/media/platform/allegro-dvt/nal-rbsp.h

diff --git a/drivers/media/platform/allegro-dvt/Makefile b/drivers/media/platform/allegro-dvt/Makefile
index 8e306dcdc55c..10119e84a60f 100644
--- a/drivers/media/platform/allegro-dvt/Makefile
+++ b/drivers/media/platform/allegro-dvt/Makefile
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
 
-allegro-objs := allegro-core.o nal-h264.o allegro-mail.o
+allegro-objs := allegro-core.o allegro-mail.o
+allegro-objs += nal-rbsp.o nal-h264.o
 
 obj-$(CONFIG_VIDEO_ALLEGRO_DVT) += allegro.o
diff --git a/drivers/media/platform/allegro-dvt/nal-h264.c b/drivers/media/platform/allegro-dvt/nal-h264.c
index bd48b8883572..94dd9266d850 100644
--- a/drivers/media/platform/allegro-dvt/nal-h264.c
+++ b/drivers/media/platform/allegro-dvt/nal-h264.c
@@ -22,6 +22,7 @@
 #include <linux/log2.h>
 
 #include "nal-h264.h"
+#include "nal-rbsp.h"
 
 /*
  * See Rec. ITU-T H.264 (04/2017) Table 7-1 – NAL unit type codes, syntax
@@ -33,54 +34,6 @@ enum nal_unit_type {
 	FILLER_DATA = 12,
 };
 
-struct rbsp;
-
-struct nal_h264_ops {
-	int (*rbsp_bit)(struct rbsp *rbsp, int *val);
-	int (*rbsp_bits)(struct rbsp *rbsp, int n, unsigned int *val);
-	int (*rbsp_uev)(struct rbsp *rbsp, unsigned int *val);
-	int (*rbsp_sev)(struct rbsp *rbsp, int *val);
-};
-
-/**
- * struct rbsp - State object for handling a raw byte sequence payload
- * @data: pointer to the data of the rbsp
- * @size: maximum size of the data of the rbsp
- * @pos: current bit position inside the rbsp
- * @num_consecutive_zeros: number of zeros before @pos
- * @ops: per datatype functions for interacting with the rbsp
- * @error: an error occurred while handling the rbsp
- *
- * This struct is passed around the various parsing functions and tracks the
- * current position within the raw byte sequence payload.
- *
- * The @ops field allows to separate the operation, i.e., reading/writing a
- * value from/to that rbsp, from the structure of the NAL unit. This allows to
- * have a single function for iterating the NAL unit, while @ops has function
- * pointers for handling each type in the rbsp.
- */
-struct rbsp {
-	u8 *data;
-	size_t size;
-	unsigned int pos;
-	unsigned int num_consecutive_zeros;
-	struct nal_h264_ops *ops;
-	int error;
-};
-
-static void rbsp_init(struct rbsp *rbsp, void *addr, size_t size,
-		      struct nal_h264_ops *ops)
-{
-	if (!rbsp)
-		return;
-
-	rbsp->data = addr;
-	rbsp->size = size;
-	rbsp->pos = 0;
-	rbsp->ops = ops;
-	rbsp->error = 0;
-}
-
 /**
  * nal_h264_profile_from_v4l2() - Get profile_idc for v4l2 h264 profile
  * @profile: the profile as &enum v4l2_mpeg_video_h264_profile
@@ -155,281 +108,6 @@ int nal_h264_level_from_v4l2(enum v4l2_mpeg_video_h264_level level)
 	}
 }
 
-static int rbsp_read_bits(struct rbsp *rbsp, int n, unsigned int *value);
-static int rbsp_write_bits(struct rbsp *rbsp, int n, unsigned int value);
-
-/*
- * When reading or writing, the emulation_prevention_three_byte is detected
- * only when the 2 one bits need to be inserted. Therefore, we are not
- * actually adding the 0x3 byte, but the 2 one bits and the six 0 bits of the
- * next byte.
- */
-#define EMULATION_PREVENTION_THREE_BYTE (0x3 << 6)
-
-static int add_emulation_prevention_three_byte(struct rbsp *rbsp)
-{
-	rbsp->num_consecutive_zeros = 0;
-	rbsp_write_bits(rbsp, 8, EMULATION_PREVENTION_THREE_BYTE);
-
-	return 0;
-}
-
-static int discard_emulation_prevention_three_byte(struct rbsp *rbsp)
-{
-	unsigned int tmp = 0;
-
-	rbsp->num_consecutive_zeros = 0;
-	rbsp_read_bits(rbsp, 8, &tmp);
-	if (tmp != EMULATION_PREVENTION_THREE_BYTE)
-		return -EINVAL;
-
-	return 0;
-}
-
-static inline int rbsp_read_bit(struct rbsp *rbsp)
-{
-	int shift;
-	int ofs;
-	int bit;
-	int err;
-
-	if (rbsp->num_consecutive_zeros == 22) {
-		err = discard_emulation_prevention_three_byte(rbsp);
-		if (err)
-			return err;
-	}
-
-	shift = 7 - (rbsp->pos % 8);
-	ofs = rbsp->pos / 8;
-	if (ofs >= rbsp->size)
-		return -EINVAL;
-
-	bit = (rbsp->data[ofs] >> shift) & 1;
-
-	rbsp->pos++;
-
-	if (bit == 1 ||
-	    (rbsp->num_consecutive_zeros < 7 && (rbsp->pos % 8 == 0)))
-		rbsp->num_consecutive_zeros = 0;
-	else
-		rbsp->num_consecutive_zeros++;
-
-	return bit;
-}
-
-static inline int rbsp_write_bit(struct rbsp *rbsp, bool value)
-{
-	int shift;
-	int ofs;
-
-	if (rbsp->num_consecutive_zeros == 22)
-		add_emulation_prevention_three_byte(rbsp);
-
-	shift = 7 - (rbsp->pos % 8);
-	ofs = rbsp->pos / 8;
-	if (ofs >= rbsp->size)
-		return -EINVAL;
-
-	rbsp->data[ofs] &= ~(1 << shift);
-	rbsp->data[ofs] |= value << shift;
-
-	rbsp->pos++;
-
-	if (value ||
-	    (rbsp->num_consecutive_zeros < 7 && (rbsp->pos % 8 == 0))) {
-		rbsp->num_consecutive_zeros = 0;
-	} else {
-		rbsp->num_consecutive_zeros++;
-	}
-
-	return 0;
-}
-
-static inline int rbsp_read_bits(struct rbsp *rbsp, int n, unsigned int *value)
-{
-	int i;
-	int bit;
-	unsigned int tmp = 0;
-
-	if (n > 8 * sizeof(*value))
-		return -EINVAL;
-
-	for (i = n; i > 0; i--) {
-		bit = rbsp_read_bit(rbsp);
-		if (bit < 0)
-			return bit;
-		tmp |= bit << (i - 1);
-	}
-
-	if (value)
-		*value = tmp;
-
-	return 0;
-}
-
-static int rbsp_write_bits(struct rbsp *rbsp, int n, unsigned int value)
-{
-	int ret;
-
-	if (n > 8 * sizeof(value))
-		return -EINVAL;
-
-	while (n--) {
-		ret = rbsp_write_bit(rbsp, (value >> n) & 1);
-		if (ret)
-			return ret;
-	}
-
-	return 0;
-}
-
-static int rbsp_read_uev(struct rbsp *rbsp, unsigned int *value)
-{
-	int leading_zero_bits = 0;
-	unsigned int tmp = 0;
-	int ret;
-
-	while ((ret = rbsp_read_bit(rbsp)) == 0)
-		leading_zero_bits++;
-	if (ret < 0)
-		return ret;
-
-	if (leading_zero_bits > 0) {
-		ret = rbsp_read_bits(rbsp, leading_zero_bits, &tmp);
-		if (ret)
-			return ret;
-	}
-
-	if (value)
-		*value = (1 << leading_zero_bits) - 1 + tmp;
-
-	return 0;
-}
-
-static int rbsp_write_uev(struct rbsp *rbsp, unsigned int *value)
-{
-	int ret;
-	int leading_zero_bits;
-
-	if (!value)
-		return -EINVAL;
-
-	leading_zero_bits = ilog2(*value + 1);
-
-	ret = rbsp_write_bits(rbsp, leading_zero_bits, 0);
-	if (ret)
-		return ret;
-
-	return rbsp_write_bits(rbsp, leading_zero_bits + 1, *value + 1);
-}
-
-static int rbsp_read_sev(struct rbsp *rbsp, int *value)
-{
-	int ret;
-	unsigned int tmp;
-
-	ret = rbsp_read_uev(rbsp, &tmp);
-	if (ret)
-		return ret;
-
-	if (value) {
-		if (tmp & 1)
-			*value = (tmp + 1) / 2;
-		else
-			*value = -(tmp / 2);
-	}
-
-	return 0;
-}
-
-static int rbsp_write_sev(struct rbsp *rbsp, int *value)
-{
-	unsigned int tmp;
-
-	if (!value)
-		return -EINVAL;
-
-	if (*value > 0)
-		tmp = (2 * (*value)) | 1;
-	else
-		tmp = -2 * (*value);
-
-	return rbsp_write_uev(rbsp, &tmp);
-}
-
-static int __rbsp_write_bit(struct rbsp *rbsp, int *value)
-{
-	return rbsp_write_bit(rbsp, *value);
-}
-
-static int __rbsp_write_bits(struct rbsp *rbsp, int n, unsigned int *value)
-{
-	return rbsp_write_bits(rbsp, n, *value);
-}
-
-static struct nal_h264_ops write = {
-	.rbsp_bit = __rbsp_write_bit,
-	.rbsp_bits = __rbsp_write_bits,
-	.rbsp_uev = rbsp_write_uev,
-	.rbsp_sev = rbsp_write_sev,
-};
-
-static int __rbsp_read_bit(struct rbsp *rbsp, int *value)
-{
-	int tmp = rbsp_read_bit(rbsp);
-
-	if (tmp < 0)
-		return tmp;
-	*value = tmp;
-
-	return 0;
-}
-
-static struct nal_h264_ops read = {
-	.rbsp_bit = __rbsp_read_bit,
-	.rbsp_bits = rbsp_read_bits,
-	.rbsp_uev = rbsp_read_uev,
-	.rbsp_sev = rbsp_read_sev,
-};
-
-static inline void rbsp_bit(struct rbsp *rbsp, int *value)
-{
-	if (rbsp->error)
-		return;
-	rbsp->error = rbsp->ops->rbsp_bit(rbsp, value);
-}
-
-static inline void rbsp_bits(struct rbsp *rbsp, int n, int *value)
-{
-	if (rbsp->error)
-		return;
-	rbsp->error = rbsp->ops->rbsp_bits(rbsp, n, value);
-}
-
-static inline void rbsp_uev(struct rbsp *rbsp, unsigned int *value)
-{
-	if (rbsp->error)
-		return;
-	rbsp->error = rbsp->ops->rbsp_uev(rbsp, value);
-}
-
-static inline void rbsp_sev(struct rbsp *rbsp, int *value)
-{
-	if (rbsp->error)
-		return;
-	rbsp->error = rbsp->ops->rbsp_sev(rbsp, value);
-}
-
-static void nal_h264_rbsp_trailing_bits(struct rbsp *rbsp)
-{
-	unsigned int rbsp_stop_one_bit = 1;
-	unsigned int rbsp_alignment_zero_bit = 0;
-
-	rbsp_bit(rbsp, &rbsp_stop_one_bit);
-	rbsp_bits(rbsp, round_up(rbsp->pos, 8) - rbsp->pos,
-		  &rbsp_alignment_zero_bit);
-}
-
 static void nal_h264_write_start_code_prefix(struct rbsp *rbsp)
 {
 	u8 *p = rbsp->data + DIV_ROUND_UP(rbsp->pos, 8);
@@ -767,7 +445,7 @@ ssize_t nal_h264_write_sps(const struct device *dev,
 
 	nal_h264_rbsp_sps(&rbsp, sps);
 
-	nal_h264_rbsp_trailing_bits(&rbsp);
+	rbsp_trailing_bits(&rbsp);
 
 	if (rbsp.error)
 		return rbsp.error;
@@ -814,7 +492,7 @@ ssize_t nal_h264_read_sps(const struct device *dev,
 
 	nal_h264_rbsp_sps(&rbsp, sps);
 
-	nal_h264_rbsp_trailing_bits(&rbsp);
+	rbsp_trailing_bits(&rbsp);
 
 	if (rbsp.error)
 		return rbsp.error;
@@ -859,7 +537,7 @@ ssize_t nal_h264_write_pps(const struct device *dev,
 
 	nal_h264_rbsp_pps(&rbsp, pps);
 
-	nal_h264_rbsp_trailing_bits(&rbsp);
+	rbsp_trailing_bits(&rbsp);
 
 	if (rbsp.error)
 		return rbsp.error;
@@ -896,7 +574,7 @@ ssize_t nal_h264_read_pps(const struct device *dev,
 
 	nal_h264_rbsp_pps(&rbsp, pps);
 
-	nal_h264_rbsp_trailing_bits(&rbsp);
+	rbsp_trailing_bits(&rbsp);
 
 	if (rbsp.error)
 		return rbsp.error;
@@ -942,7 +620,7 @@ ssize_t nal_h264_write_filler(const struct device *dev, void *dest, size_t n)
 
 	nal_h264_write_filler_data(&rbsp);
 
-	nal_h264_rbsp_trailing_bits(&rbsp);
+	rbsp_trailing_bits(&rbsp);
 
 	return DIV_ROUND_UP(rbsp.pos, 8);
 }
@@ -991,7 +669,7 @@ ssize_t nal_h264_read_filler(const struct device *dev, void *src, size_t n)
 		return -EINVAL;
 
 	nal_h264_read_filler_data(&rbsp);
-	nal_h264_rbsp_trailing_bits(&rbsp);
+	rbsp_trailing_bits(&rbsp);
 
 	if (rbsp.error)
 		return rbsp.error;
diff --git a/drivers/media/platform/allegro-dvt/nal-rbsp.c b/drivers/media/platform/allegro-dvt/nal-rbsp.c
new file mode 100644
index 000000000000..935ba23844f2
--- /dev/null
+++ b/drivers/media/platform/allegro-dvt/nal-rbsp.c
@@ -0,0 +1,305 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2019-2020 Pengutronix, Michael Tretter <kernel@pengutronix.de>
+ *
+ * Helper functions to generate a raw byte sequence payload from values.
+ */
+
+#include <linux/kernel.h>
+#include <linux/types.h>
+#include <linux/string.h>
+#include <linux/v4l2-controls.h>
+
+#include <linux/device.h>
+#include <linux/export.h>
+#include <linux/log2.h>
+
+#include "nal-rbsp.h"
+
+void rbsp_init(struct rbsp *rbsp, void *addr, size_t size,
+	       struct nal_rbsp_ops *ops)
+{
+	if (!rbsp)
+		return;
+
+	rbsp->data = addr;
+	rbsp->size = size;
+	rbsp->pos = 0;
+	rbsp->ops = ops;
+	rbsp->error = 0;
+}
+
+static int rbsp_read_bits(struct rbsp *rbsp, int n, unsigned int *value);
+static int rbsp_write_bits(struct rbsp *rbsp, int n, unsigned int value);
+
+/*
+ * When reading or writing, the emulation_prevention_three_byte is detected
+ * only when the 2 one bits need to be inserted. Therefore, we are not
+ * actually adding the 0x3 byte, but the 2 one bits and the six 0 bits of the
+ * next byte.
+ */
+#define EMULATION_PREVENTION_THREE_BYTE (0x3 << 6)
+
+static int add_emulation_prevention_three_byte(struct rbsp *rbsp)
+{
+	rbsp->num_consecutive_zeros = 0;
+	rbsp_write_bits(rbsp, 8, EMULATION_PREVENTION_THREE_BYTE);
+
+	return 0;
+}
+
+static int discard_emulation_prevention_three_byte(struct rbsp *rbsp)
+{
+	unsigned int tmp = 0;
+
+	rbsp->num_consecutive_zeros = 0;
+	rbsp_read_bits(rbsp, 8, &tmp);
+	if (tmp != EMULATION_PREVENTION_THREE_BYTE)
+		return -EINVAL;
+
+	return 0;
+}
+
+static inline int rbsp_read_bit(struct rbsp *rbsp)
+{
+	int shift;
+	int ofs;
+	int bit;
+	int err;
+
+	if (rbsp->num_consecutive_zeros == 22) {
+		err = discard_emulation_prevention_three_byte(rbsp);
+		if (err)
+			return err;
+	}
+
+	shift = 7 - (rbsp->pos % 8);
+	ofs = rbsp->pos / 8;
+	if (ofs >= rbsp->size)
+		return -EINVAL;
+
+	bit = (rbsp->data[ofs] >> shift) & 1;
+
+	rbsp->pos++;
+
+	if (bit == 1 ||
+	    (rbsp->num_consecutive_zeros < 7 && (rbsp->pos % 8 == 0)))
+		rbsp->num_consecutive_zeros = 0;
+	else
+		rbsp->num_consecutive_zeros++;
+
+	return bit;
+}
+
+static inline int rbsp_write_bit(struct rbsp *rbsp, bool value)
+{
+	int shift;
+	int ofs;
+
+	if (rbsp->num_consecutive_zeros == 22)
+		add_emulation_prevention_three_byte(rbsp);
+
+	shift = 7 - (rbsp->pos % 8);
+	ofs = rbsp->pos / 8;
+	if (ofs >= rbsp->size)
+		return -EINVAL;
+
+	rbsp->data[ofs] &= ~(1 << shift);
+	rbsp->data[ofs] |= value << shift;
+
+	rbsp->pos++;
+
+	if (value ||
+	    (rbsp->num_consecutive_zeros < 7 && (rbsp->pos % 8 == 0))) {
+		rbsp->num_consecutive_zeros = 0;
+	} else {
+		rbsp->num_consecutive_zeros++;
+	}
+
+	return 0;
+}
+
+static inline int rbsp_read_bits(struct rbsp *rbsp, int n, unsigned int *value)
+{
+	int i;
+	int bit;
+	unsigned int tmp = 0;
+
+	if (n > 8 * sizeof(*value))
+		return -EINVAL;
+
+	for (i = n; i > 0; i--) {
+		bit = rbsp_read_bit(rbsp);
+		if (bit < 0)
+			return bit;
+		tmp |= bit << (i - 1);
+	}
+
+	if (value)
+		*value = tmp;
+
+	return 0;
+}
+
+static int rbsp_write_bits(struct rbsp *rbsp, int n, unsigned int value)
+{
+	int ret;
+
+	if (n > 8 * sizeof(value))
+		return -EINVAL;
+
+	while (n--) {
+		ret = rbsp_write_bit(rbsp, (value >> n) & 1);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
+}
+
+static int rbsp_read_uev(struct rbsp *rbsp, unsigned int *value)
+{
+	int leading_zero_bits = 0;
+	unsigned int tmp = 0;
+	int ret;
+
+	while ((ret = rbsp_read_bit(rbsp)) == 0)
+		leading_zero_bits++;
+	if (ret < 0)
+		return ret;
+
+	if (leading_zero_bits > 0) {
+		ret = rbsp_read_bits(rbsp, leading_zero_bits, &tmp);
+		if (ret)
+			return ret;
+	}
+
+	if (value)
+		*value = (1 << leading_zero_bits) - 1 + tmp;
+
+	return 0;
+}
+
+static int rbsp_write_uev(struct rbsp *rbsp, unsigned int *value)
+{
+	int ret;
+	int leading_zero_bits;
+
+	if (!value)
+		return -EINVAL;
+
+	leading_zero_bits = ilog2(*value + 1);
+
+	ret = rbsp_write_bits(rbsp, leading_zero_bits, 0);
+	if (ret)
+		return ret;
+
+	return rbsp_write_bits(rbsp, leading_zero_bits + 1, *value + 1);
+}
+
+static int rbsp_read_sev(struct rbsp *rbsp, int *value)
+{
+	int ret;
+	unsigned int tmp;
+
+	ret = rbsp_read_uev(rbsp, &tmp);
+	if (ret)
+		return ret;
+
+	if (value) {
+		if (tmp & 1)
+			*value = (tmp + 1) / 2;
+		else
+			*value = -(tmp / 2);
+	}
+
+	return 0;
+}
+
+static int rbsp_write_sev(struct rbsp *rbsp, int *value)
+{
+	unsigned int tmp;
+
+	if (!value)
+		return -EINVAL;
+
+	if (*value > 0)
+		tmp = (2 * (*value)) | 1;
+	else
+		tmp = -2 * (*value);
+
+	return rbsp_write_uev(rbsp, &tmp);
+}
+
+static int __rbsp_write_bit(struct rbsp *rbsp, int *value)
+{
+	return rbsp_write_bit(rbsp, *value);
+}
+
+static int __rbsp_write_bits(struct rbsp *rbsp, int n, unsigned int *value)
+{
+	return rbsp_write_bits(rbsp, n, *value);
+}
+
+struct nal_rbsp_ops write = {
+	.rbsp_bit = __rbsp_write_bit,
+	.rbsp_bits = __rbsp_write_bits,
+	.rbsp_uev = rbsp_write_uev,
+	.rbsp_sev = rbsp_write_sev,
+};
+
+static int __rbsp_read_bit(struct rbsp *rbsp, int *value)
+{
+	int tmp = rbsp_read_bit(rbsp);
+
+	if (tmp < 0)
+		return tmp;
+	*value = tmp;
+
+	return 0;
+}
+
+struct nal_rbsp_ops read = {
+	.rbsp_bit = __rbsp_read_bit,
+	.rbsp_bits = rbsp_read_bits,
+	.rbsp_uev = rbsp_read_uev,
+	.rbsp_sev = rbsp_read_sev,
+};
+
+void rbsp_bit(struct rbsp *rbsp, int *value)
+{
+	if (rbsp->error)
+		return;
+	rbsp->error = rbsp->ops->rbsp_bit(rbsp, value);
+}
+
+void rbsp_bits(struct rbsp *rbsp, int n, int *value)
+{
+	if (rbsp->error)
+		return;
+	rbsp->error = rbsp->ops->rbsp_bits(rbsp, n, value);
+}
+
+void rbsp_uev(struct rbsp *rbsp, unsigned int *value)
+{
+	if (rbsp->error)
+		return;
+	rbsp->error = rbsp->ops->rbsp_uev(rbsp, value);
+}
+
+void rbsp_sev(struct rbsp *rbsp, int *value)
+{
+	if (rbsp->error)
+		return;
+	rbsp->error = rbsp->ops->rbsp_sev(rbsp, value);
+}
+
+void rbsp_trailing_bits(struct rbsp *rbsp)
+{
+	unsigned int rbsp_stop_one_bit = 1;
+	unsigned int rbsp_alignment_zero_bit = 0;
+
+	rbsp_bit(rbsp, &rbsp_stop_one_bit);
+	rbsp_bits(rbsp, round_up(rbsp->pos, 8) - rbsp->pos,
+		  &rbsp_alignment_zero_bit);
+}
diff --git a/drivers/media/platform/allegro-dvt/nal-rbsp.h b/drivers/media/platform/allegro-dvt/nal-rbsp.h
new file mode 100644
index 000000000000..90cc1a4f716d
--- /dev/null
+++ b/drivers/media/platform/allegro-dvt/nal-rbsp.h
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2019-2020 Pengutronix, Michael Tretter <kernel@pengutronix.de>
+ */
+
+#ifndef __NAL_RBSP_H__
+#define __NAL_RBSP_H__
+
+#include <linux/kernel.h>
+#include <linux/types.h>
+
+struct rbsp;
+
+struct nal_rbsp_ops {
+	int (*rbsp_bit)(struct rbsp *rbsp, int *val);
+	int (*rbsp_bits)(struct rbsp *rbsp, int n, unsigned int *val);
+	int (*rbsp_uev)(struct rbsp *rbsp, unsigned int *val);
+	int (*rbsp_sev)(struct rbsp *rbsp, int *val);
+};
+
+/**
+ * struct rbsp - State object for handling a raw byte sequence payload
+ * @data: pointer to the data of the rbsp
+ * @size: maximum size of the data of the rbsp
+ * @pos: current bit position inside the rbsp
+ * @num_consecutive_zeros: number of zeros before @pos
+ * @ops: per datatype functions for interacting with the rbsp
+ * @error: an error occurred while handling the rbsp
+ *
+ * This struct is passed around the various parsing functions and tracks the
+ * current position within the raw byte sequence payload.
+ *
+ * The @ops field allows to separate the operation, i.e., reading/writing a
+ * value from/to that rbsp, from the structure of the NAL unit. This allows to
+ * have a single function for iterating the NAL unit, while @ops has function
+ * pointers for handling each type in the rbsp.
+ */
+struct rbsp {
+	u8 *data;
+	size_t size;
+	unsigned int pos;
+	unsigned int num_consecutive_zeros;
+	struct nal_rbsp_ops *ops;
+	int error;
+};
+
+extern struct nal_rbsp_ops write;
+extern struct nal_rbsp_ops read;
+
+void rbsp_init(struct rbsp *rbsp, void *addr, size_t size,
+	       struct nal_rbsp_ops *ops);
+
+void rbsp_bit(struct rbsp *rbsp, int *value);
+void rbsp_bits(struct rbsp *rbsp, int n, int *value);
+void rbsp_uev(struct rbsp *rbsp, unsigned int *value);
+void rbsp_sev(struct rbsp *rbsp, int *value);
+
+void rbsp_trailing_bits(struct rbsp *rbsp);
+
+#endif /* __NAL_RBSP_H__ */
-- 
2.20.1

[PATCH 02/18] media: allegro: add helper to report unsupported fields

[Michael Tretter]

Allow generators to explicitly signal an error if the C structs contain
unsupported or invalid fields.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 drivers/media/platform/allegro-dvt/nal-rbsp.c | 5 +++++
 drivers/media/platform/allegro-dvt/nal-rbsp.h | 1 +
 2 files changed, 6 insertions(+)

diff --git a/drivers/media/platform/allegro-dvt/nal-rbsp.c b/drivers/media/platform/allegro-dvt/nal-rbsp.c
index 935ba23844f2..d911322d0efa 100644
--- a/drivers/media/platform/allegro-dvt/nal-rbsp.c
+++ b/drivers/media/platform/allegro-dvt/nal-rbsp.c
@@ -29,6 +29,11 @@ void rbsp_init(struct rbsp *rbsp, void *addr, size_t size,
 	rbsp->error = 0;
 }
 
+void rbsp_unsupported(struct rbsp *rbsp)
+{
+	rbsp->error = -EINVAL;
+}
+
 static int rbsp_read_bits(struct rbsp *rbsp, int n, unsigned int *value);
 static int rbsp_write_bits(struct rbsp *rbsp, int n, unsigned int value);
 
diff --git a/drivers/media/platform/allegro-dvt/nal-rbsp.h b/drivers/media/platform/allegro-dvt/nal-rbsp.h
index 90cc1a4f716d..c72f49fed8d3 100644
--- a/drivers/media/platform/allegro-dvt/nal-rbsp.h
+++ b/drivers/media/platform/allegro-dvt/nal-rbsp.h
@@ -49,6 +49,7 @@ extern struct nal_rbsp_ops read;
 
 void rbsp_init(struct rbsp *rbsp, void *addr, size_t size,
 	       struct nal_rbsp_ops *ops);
+void rbsp_unsupported(struct rbsp *rbsp);
 
 void rbsp_bit(struct rbsp *rbsp, int *value);
 void rbsp_bits(struct rbsp *rbsp, int n, int *value);
-- 
2.20.1

[PATCH 03/18] media: allegro: add HEVC NAL unit generator

[Michael Tretter]

When encoding a video as HEVC, the allegro driver needs to generate the
Non-VCL NAL units for HEVC. Do the same as for H.264 and add a module
that takes C structs for the VPS/SPS/PPS and encodes the fields as RBPS
as specified by "ITU-T Rec. H.265 (02/2018) high efficiency video
coding".

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 drivers/media/platform/allegro-dvt/Makefile   |   2 +-
 drivers/media/platform/allegro-dvt/nal-hevc.c | 824 ++++++++++++++++++
 drivers/media/platform/allegro-dvt/nal-hevc.h | 350 ++++++++
 3 files changed, 1175 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/platform/allegro-dvt/nal-hevc.c
 create mode 100644 drivers/media/platform/allegro-dvt/nal-hevc.h

diff --git a/drivers/media/platform/allegro-dvt/Makefile b/drivers/media/platform/allegro-dvt/Makefile
index 10119e84a60f..66108a303774 100644
--- a/drivers/media/platform/allegro-dvt/Makefile
+++ b/drivers/media/platform/allegro-dvt/Makefile
@@ -1,6 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
 
 allegro-objs := allegro-core.o allegro-mail.o
-allegro-objs += nal-rbsp.o nal-h264.o
+allegro-objs += nal-rbsp.o nal-h264.o nal-hevc.o
 
 obj-$(CONFIG_VIDEO_ALLEGRO_DVT) += allegro.o
diff --git a/drivers/media/platform/allegro-dvt/nal-hevc.c b/drivers/media/platform/allegro-dvt/nal-hevc.c
new file mode 100644
index 000000000000..2dcc516729d7
--- /dev/null
+++ b/drivers/media/platform/allegro-dvt/nal-hevc.c
@@ -0,0 +1,824 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2019-2020 Pengutronix, Michael Tretter <kernel@pengutronix.de>
+ *
+ * Convert NAL units between raw byte sequence payloads (RBSP) and C structs.
+ *
+ * The conversion is defined in "ITU-T Rec. H.265 (02/2018) high efficiency
+ * video coding". Decoder drivers may use the parser to parse RBSP from
+ * encoded streams and configure the hardware, if the hardware is not able to
+ * parse RBSP itself. Encoder drivers may use the generator to generate the
+ * RBSP for VPS/SPS/PPS nal units and add them to the encoded stream if the
+ * hardware does not generate the units.
+ */
+
+#include <linux/kernel.h>
+#include <linux/types.h>
+#include <linux/string.h>
+#include <linux/v4l2-controls.h>
+
+#include <linux/device.h>
+#include <linux/export.h>
+#include <linux/log2.h>
+
+#include "nal-hevc.h"
+#include "nal-rbsp.h"
+
+/*
+ * See Rec. ITU-T H.265 (02/2018) Table 7-1 – NAL unit type codes and NAL unit
+ * type classes
+ */
+enum nal_unit_type {
+	VPS_NUT = 32,
+	SPS_NUT = 33,
+	PPS_NUT = 34,
+	FD_NUT = 38,
+};
+
+int nal_hevc_profile_from_v4l2(enum v4l2_mpeg_video_hevc_profile profile)
+{
+	switch (profile) {
+	case V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN:
+		return 1;
+	case V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10:
+		return 2;
+	case V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE:
+		return 3;
+	default:
+		return -EINVAL;
+	}
+}
+EXPORT_SYMBOL_GPL(nal_hevc_profile_from_v4l2);
+
+int nal_hevc_tier_from_v4l2(enum v4l2_mpeg_video_hevc_tier tier)
+{
+	switch (tier) {
+	case V4L2_MPEG_VIDEO_HEVC_TIER_MAIN:
+		return 0;
+	case V4L2_MPEG_VIDEO_HEVC_TIER_HIGH:
+		return 1;
+	default:
+		return -EINVAL;
+	}
+}
+EXPORT_SYMBOL_GPL(nal_hevc_tier_from_v4l2);
+
+int nal_hevc_level_from_v4l2(enum v4l2_mpeg_video_hevc_level level)
+{
+	/*
+	 * T-Rec-H.265 p. 280: general_level_idc and sub_layer_level_idc[ i ]
+	 * shall be set equal to a value of 30 times the level number
+	 * specified in Table A.6.
+	 */
+	int factor = 30 / 10;
+
+	switch (level) {
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_1:
+		return factor * 10;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_2:
+		return factor * 20;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1:
+		return factor * 21;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_3:
+		return factor * 30;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1:
+		return factor * 31;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_4:
+		return factor * 40;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_4_1:
+		return factor * 41;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_5:
+		return factor * 50;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1:
+		return factor * 51;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_5_2:
+		return factor * 52;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_6:
+		return factor * 60;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_6_1:
+		return factor * 61;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2:
+		return factor * 62;
+	default:
+		return -EINVAL;
+	}
+}
+EXPORT_SYMBOL_GPL(nal_hevc_level_from_v4l2);
+
+void nal_hevc_write_start_code_prefix(struct rbsp *rbsp)
+{
+	u8 *p = rbsp->data + DIV_ROUND_UP(rbsp->pos, 8);
+	int i = 4;
+
+	if (DIV_ROUND_UP(rbsp->pos, 8) + i > rbsp->size) {
+		rbsp->error = -EINVAL;
+		return;
+	}
+
+	p[0] = 0x00;
+	p[1] = 0x00;
+	p[2] = 0x00;
+	p[3] = 0x01;
+
+	rbsp->pos += i * 8;
+}
+
+void nal_hevc_read_start_code_prefix(struct rbsp *rbsp)
+{
+	u8 *p = rbsp->data + DIV_ROUND_UP(rbsp->pos, 8);
+	int i = 4;
+
+	if (DIV_ROUND_UP(rbsp->pos, 8) + i > rbsp->size) {
+		rbsp->error = -EINVAL;
+		return;
+	}
+
+	if (p[0] != 0x00 || p[1] != 0x00 || p[2] != 0x00 || p[3] != 0x01) {
+		rbsp->error = -EINVAL;
+		return;
+	}
+
+	rbsp->pos += i * 8;
+}
+
+static void nal_hevc_write_filler_data(struct rbsp *rbsp)
+{
+	u8 *p = rbsp->data + DIV_ROUND_UP(rbsp->pos, 8);
+	int i;
+
+	/* Keep 1 byte extra for terminating the NAL unit */
+	i = rbsp->size - DIV_ROUND_UP(rbsp->pos, 8) - 1;
+	memset(p, 0xff, i);
+	rbsp->pos += i * 8;
+}
+
+static void nal_hevc_read_filler_data(struct rbsp *rbsp)
+{
+	u8 *p = rbsp->data + DIV_ROUND_UP(rbsp->pos, 8);
+
+	while (*p == 0xff) {
+		if (DIV_ROUND_UP(rbsp->pos, 8) > rbsp->size) {
+			rbsp->error = -EINVAL;
+			return;
+		}
+
+		p++;
+		rbsp->pos += 8;
+	}
+}
+
+static void nal_hevc_rbsp_profile_tier_level(struct rbsp *rbsp,
+					     struct nal_hevc_profile_tier_level *ptl)
+{
+	unsigned int i;
+	unsigned int max_num_sub_layers_minus_1 = 0;
+
+	rbsp_bits(rbsp, 2, &ptl->general_profile_space);
+	rbsp_bit(rbsp, &ptl->general_tier_flag);
+	rbsp_bits(rbsp, 5, &ptl->general_profile_idc);
+	for (i = 0; i < 32; i++)
+		rbsp_bit(rbsp, &ptl->general_profile_compatibility_flag[i]);
+	rbsp_bit(rbsp, &ptl->general_progressive_source_flag);
+	rbsp_bit(rbsp, &ptl->general_interlaced_source_flag);
+	rbsp_bit(rbsp, &ptl->general_non_packed_constraint_flag);
+	rbsp_bit(rbsp, &ptl->general_frame_only_constraint_flag);
+	if (ptl->general_profile_idc == 4 ||
+	    ptl->general_profile_compatibility_flag[4] ||
+	    ptl->general_profile_idc == 5 ||
+	    ptl->general_profile_compatibility_flag[5] ||
+	    ptl->general_profile_idc == 6 ||
+	    ptl->general_profile_compatibility_flag[6] ||
+	    ptl->general_profile_idc == 7 ||
+	    ptl->general_profile_compatibility_flag[7] ||
+	    ptl->general_profile_idc == 8 ||
+	    ptl->general_profile_compatibility_flag[8] ||
+	    ptl->general_profile_idc == 9 ||
+	    ptl->general_profile_compatibility_flag[9] ||
+	    ptl->general_profile_idc == 10 ||
+	    ptl->general_profile_compatibility_flag[10]) {
+		rbsp_bit(rbsp, &ptl->general_max_12bit_constraint_flag);
+		rbsp_bit(rbsp, &ptl->general_max_10bit_constraint_flag);
+		rbsp_bit(rbsp, &ptl->general_max_8bit_constraint_flag);
+		rbsp_bit(rbsp, &ptl->general_max_422chroma_constraint_flag);
+		rbsp_bit(rbsp, &ptl->general_max_420chroma_constraint_flag);
+		rbsp_bit(rbsp, &ptl->general_max_monochrome_constraint_flag);
+		rbsp_bit(rbsp, &ptl->general_intra_constraint_flag);
+		rbsp_bit(rbsp, &ptl->general_one_picture_only_constraint_flag);
+		rbsp_bit(rbsp, &ptl->general_lower_bit_rate_constraint_flag);
+		if (ptl->general_profile_idc == 5 ||
+		    ptl->general_profile_compatibility_flag[5] ||
+		    ptl->general_profile_idc == 9 ||
+		    ptl->general_profile_compatibility_flag[9] ||
+		    ptl->general_profile_idc == 10 ||
+		    ptl->general_profile_compatibility_flag[10]) {
+			rbsp_bit(rbsp, &ptl->general_max_14bit_constraint_flag);
+			rbsp_bits(rbsp, 32, &ptl->general_reserved_zero_33bits);
+			rbsp_bits(rbsp, 33 - 32, &ptl->general_reserved_zero_33bits);
+		} else {
+			rbsp_bits(rbsp, 32, &ptl->general_reserved_zero_34bits);
+			rbsp_bits(rbsp, 34 - 2, &ptl->general_reserved_zero_34bits);
+		}
+	} else if (ptl->general_profile_idc == 2 ||
+		   ptl->general_profile_compatibility_flag[2]) {
+		rbsp_bits(rbsp, 7, &ptl->general_reserved_zero_7bits);
+		rbsp_bit(rbsp, &ptl->general_one_picture_only_constraint_flag);
+		rbsp_bits(rbsp, 32, &ptl->general_reserved_zero_35bits);
+		rbsp_bits(rbsp, 35 - 32, &ptl->general_reserved_zero_35bits);
+	} else {
+		rbsp_bits(rbsp, 32, &ptl->general_reserved_zero_43bits);
+		rbsp_bits(rbsp, 43 - 32, &ptl->general_reserved_zero_43bits);
+	}
+	if ((ptl->general_profile_idc >= 1 && ptl->general_profile_idc <= 5) ||
+	    ptl->general_profile_idc == 9 ||
+	    ptl->general_profile_compatibility_flag[1] ||
+	    ptl->general_profile_compatibility_flag[2] ||
+	    ptl->general_profile_compatibility_flag[3] ||
+	    ptl->general_profile_compatibility_flag[4] ||
+	    ptl->general_profile_compatibility_flag[5] ||
+	    ptl->general_profile_compatibility_flag[9])
+		rbsp_bit(rbsp, &ptl->general_inbld_flag);
+	else
+		rbsp_bit(rbsp, &ptl->general_reserved_zero_bit);
+	rbsp_bits(rbsp, 8, &ptl->general_level_idc);
+	if (max_num_sub_layers_minus_1 > 0)
+		rbsp_unsupported(rbsp);
+}
+
+static void nal_hevc_rbsp_vps(struct rbsp *rbsp, struct nal_hevc_vps *vps)
+{
+	unsigned int i, j;
+	unsigned int reserved_0xffff_16bits = 0xffff;
+
+	rbsp_bits(rbsp, 4, &vps->video_parameter_set_id);
+	rbsp_bit(rbsp, &vps->base_layer_internal_flag);
+	rbsp_bit(rbsp, &vps->base_layer_available_flag);
+	rbsp_bits(rbsp, 6, &vps->max_layers_minus1);
+	rbsp_bits(rbsp, 3, &vps->max_sub_layers_minus1);
+	rbsp_bits(rbsp, 1, &vps->temporal_id_nesting_flag);
+	rbsp_bits(rbsp, 16, &reserved_0xffff_16bits);
+	nal_hevc_rbsp_profile_tier_level(rbsp, &vps->profile_tier_level);
+	rbsp_bit(rbsp, &vps->sub_layer_ordering_info_present_flag);
+	for (i = vps->sub_layer_ordering_info_present_flag ? 0 : vps->max_sub_layers_minus1;
+	     i <= vps->max_sub_layers_minus1; i++) {
+		rbsp_uev(rbsp, &vps->max_dec_pic_buffering_minus1[i]);
+		rbsp_uev(rbsp, &vps->max_num_reorder_pics[i]);
+		rbsp_uev(rbsp, &vps->max_latency_increase_plus1[i]);
+	}
+	rbsp_bits(rbsp, 6, &vps->max_layer_id);
+	rbsp_uev(rbsp, &vps->num_layer_sets_minus1);
+	for (i = 0; i <= vps->num_layer_sets_minus1; i++)
+		for (j = 0; j <= vps->max_layer_id; j++)
+			rbsp_bit(rbsp, &vps->layer_id_included_flag[i][j]);
+	rbsp_bit(rbsp, &vps->timing_info_present_flag);
+	if (vps->timing_info_present_flag)
+		rbsp_unsupported(rbsp);
+	rbsp_bit(rbsp, &vps->extension_flag);
+	if (vps->extension_flag)
+		rbsp_unsupported(rbsp);
+}
+
+static void nal_hevc_rbsp_sps(struct rbsp *rbsp, struct nal_hevc_sps *sps)
+{
+	unsigned int i;
+
+	rbsp_bits(rbsp, 4, &sps->video_parameter_set_id);
+	rbsp_bits(rbsp, 3, &sps->max_sub_layers_minus1);
+	rbsp_bit(rbsp, &sps->temporal_id_nesting_flag);
+	nal_hevc_rbsp_profile_tier_level(rbsp, &sps->profile_tier_level);
+	rbsp_uev(rbsp, &sps->seq_parameter_set_id);
+
+	rbsp_uev(rbsp, &sps->chroma_format_idc);
+	if (sps->chroma_format_idc == 3)
+		rbsp_bit(rbsp, &sps->separate_colour_plane_flag);
+	rbsp_uev(rbsp, &sps->pic_width_in_luma_samples);
+	rbsp_uev(rbsp, &sps->pic_height_in_luma_samples);
+	rbsp_bit(rbsp, &sps->conformance_window_flag);
+	if (sps->conformance_window_flag) {
+		rbsp_uev(rbsp, &sps->conf_win_left_offset);
+		rbsp_uev(rbsp, &sps->conf_win_right_offset);
+		rbsp_uev(rbsp, &sps->conf_win_top_offset);
+		rbsp_uev(rbsp, &sps->conf_win_bottom_offset);
+	}
+	rbsp_uev(rbsp, &sps->bit_depth_luma_minus8);
+	rbsp_uev(rbsp, &sps->bit_depth_chroma_minus8);
+
+	rbsp_uev(rbsp, &sps->log2_max_pic_order_cnt_lsb_minus4);
+
+	rbsp_bit(rbsp, &sps->sub_layer_ordering_info_present_flag);
+	for (i = (sps->sub_layer_ordering_info_present_flag ? 0 : sps->max_sub_layers_minus1);
+	     i <= sps->max_sub_layers_minus1; i++) {
+		rbsp_uev(rbsp, &sps->max_dec_pic_buffering_minus1[i]);
+		rbsp_uev(rbsp, &sps->max_num_reorder_pics[i]);
+		rbsp_uev(rbsp, &sps->max_latency_increase_plus1[i]);
+	}
+	rbsp_uev(rbsp, &sps->log2_min_luma_coding_block_size_minus3);
+	rbsp_uev(rbsp, &sps->log2_diff_max_min_luma_coding_block_size);
+	rbsp_uev(rbsp, &sps->log2_min_luma_transform_block_size_minus2);
+	rbsp_uev(rbsp, &sps->log2_diff_max_min_luma_transform_block_size);
+	rbsp_uev(rbsp, &sps->max_transform_hierarchy_depth_inter);
+	rbsp_uev(rbsp, &sps->max_transform_hierarchy_depth_intra);
+
+	rbsp_bit(rbsp, &sps->scaling_list_enabled_flag);
+	if (sps->scaling_list_enabled_flag)
+		rbsp_unsupported(rbsp);
+
+	rbsp_bit(rbsp, &sps->amp_enabled_flag);
+	rbsp_bit(rbsp, &sps->sample_adaptive_offset_enabled_flag);
+	rbsp_bit(rbsp, &sps->pcm_enabled_flag);
+	if (sps->pcm_enabled_flag) {
+		rbsp_bits(rbsp, 4, &sps->pcm_sample_bit_depth_luma_minus1);
+		rbsp_bits(rbsp, 4, &sps->pcm_sample_bit_depth_chroma_minus1);
+		rbsp_uev(rbsp, &sps->log2_min_pcm_luma_coding_block_size_minus3);
+		rbsp_uev(rbsp, &sps->log2_diff_max_min_pcm_luma_coding_block_size);
+		rbsp_bit(rbsp, &sps->pcm_loop_filter_disabled_flag);
+	}
+
+	rbsp_uev(rbsp, &sps->num_short_term_ref_pic_sets);
+	if (sps->num_short_term_ref_pic_sets > 0)
+		rbsp_unsupported(rbsp);
+
+	rbsp_bit(rbsp, &sps->long_term_ref_pics_present_flag);
+	if (sps->long_term_ref_pics_present_flag)
+		rbsp_unsupported(rbsp);
+
+	rbsp_bit(rbsp, &sps->sps_temporal_mvp_enabled_flag);
+	rbsp_bit(rbsp, &sps->strong_intra_smoothing_enabled_flag);
+	rbsp_bit(rbsp, &sps->vui_parameters_present_flag);
+	if (sps->vui_parameters_present_flag)
+		rbsp_unsupported(rbsp);
+
+	rbsp_bit(rbsp, &sps->extension_present_flag);
+	if (sps->extension_present_flag) {
+		rbsp_bit(rbsp, &sps->sps_range_extension_flag);
+		rbsp_bit(rbsp, &sps->sps_multilayer_extension_flag);
+		rbsp_bit(rbsp, &sps->sps_3d_extension_flag);
+		rbsp_bit(rbsp, &sps->sps_scc_extension_flag);
+		rbsp_bits(rbsp, 5, &sps->sps_extension_4bits);
+	}
+	if (sps->sps_range_extension_flag)
+		rbsp_unsupported(rbsp);
+	if (sps->sps_multilayer_extension_flag)
+		rbsp_unsupported(rbsp);
+	if (sps->sps_3d_extension_flag)
+		rbsp_unsupported(rbsp);
+	if (sps->sps_scc_extension_flag)
+		rbsp_unsupported(rbsp);
+	if (sps->sps_extension_4bits)
+		rbsp_unsupported(rbsp);
+}
+
+static void nal_hevc_rbsp_pps(struct rbsp *rbsp, struct nal_hevc_pps *pps)
+{
+	unsigned int i;
+
+	rbsp_uev(rbsp, &pps->pps_pic_parameter_set_id);
+	rbsp_uev(rbsp, &pps->pps_seq_parameter_set_id);
+	rbsp_bit(rbsp, &pps->dependent_slice_segments_enabled_flag);
+	rbsp_bit(rbsp, &pps->output_flag_present_flag);
+	rbsp_bits(rbsp, 3, &pps->num_extra_slice_header_bits);
+	rbsp_bit(rbsp, &pps->sign_data_hiding_enabled_flag);
+	rbsp_bit(rbsp, &pps->cabac_init_present_flag);
+	rbsp_uev(rbsp, &pps->num_ref_idx_l0_default_active_minus1);
+	rbsp_uev(rbsp, &pps->num_ref_idx_l1_default_active_minus1);
+	rbsp_sev(rbsp, &pps->init_qp_minus26);
+	rbsp_bit(rbsp, &pps->constrained_intra_pred_flag);
+	rbsp_bit(rbsp, &pps->transform_skip_enabled_flag);
+	rbsp_bit(rbsp, &pps->cu_qp_delta_enabled_flag);
+	if (pps->cu_qp_delta_enabled_flag)
+		rbsp_uev(rbsp, &pps->diff_cu_qp_delta_depth);
+	rbsp_sev(rbsp, &pps->pps_cb_qp_offset);
+	rbsp_sev(rbsp, &pps->pps_cr_qp_offset);
+	rbsp_bit(rbsp, &pps->pps_slice_chroma_qp_offsets_present_flag);
+	rbsp_bit(rbsp, &pps->weighted_pred_flag);
+	rbsp_bit(rbsp, &pps->weighted_bipred_flag);
+	rbsp_bit(rbsp, &pps->transquant_bypass_enabled_flag);
+	rbsp_bit(rbsp, &pps->tiles_enabled_flag);
+	rbsp_bit(rbsp, &pps->entropy_coding_sync_enabled_flag);
+	if (pps->tiles_enabled_flag) {
+		rbsp_uev(rbsp, &pps->num_tile_columns_minus1);
+		rbsp_uev(rbsp, &pps->num_tile_rows_minus1);
+		rbsp_bit(rbsp, &pps->uniform_spacing_flag);
+		if (!pps->uniform_spacing_flag) {
+			for (i = 0; i < pps->num_tile_columns_minus1; i++)
+				rbsp_uev(rbsp, &pps->column_width_minus1[i]);
+			for (i = 0; i < pps->num_tile_rows_minus1; i++)
+				rbsp_uev(rbsp, &pps->row_height_minus1[i]);
+		}
+		rbsp_bit(rbsp, &pps->loop_filter_across_tiles_enabled_flag);
+	}
+	rbsp_bit(rbsp, &pps->pps_loop_filter_across_slices_enabled_flag);
+	rbsp_bit(rbsp, &pps->deblocking_filter_control_present_flag);
+	if (pps->deblocking_filter_control_present_flag) {
+		rbsp_bit(rbsp, &pps->deblocking_filter_override_enabled_flag);
+		rbsp_bit(rbsp, &pps->pps_deblocking_filter_disabled_flag);
+		if (!pps->pps_deblocking_filter_disabled_flag) {
+			rbsp_sev(rbsp, &pps->pps_beta_offset_div2);
+			rbsp_sev(rbsp, &pps->pps_tc_offset_div2);
+		}
+	}
+	rbsp_bit(rbsp, &pps->pps_scaling_list_data_present_flag);
+	if (pps->pps_scaling_list_data_present_flag)
+		rbsp_unsupported(rbsp);
+	rbsp_bit(rbsp, &pps->lists_modification_present_flag);
+	rbsp_uev(rbsp, &pps->log2_parallel_merge_level_minus2);
+	rbsp_bit(rbsp, &pps->slice_segment_header_extension_present_flag);
+	rbsp_bit(rbsp, &pps->pps_extension_present_flag);
+	if (pps->pps_extension_present_flag) {
+		rbsp_bit(rbsp, &pps->pps_range_extension_flag);
+		rbsp_bit(rbsp, &pps->pps_multilayer_extension_flag);
+		rbsp_bit(rbsp, &pps->pps_3d_extension_flag);
+		rbsp_bit(rbsp, &pps->pps_scc_extension_flag);
+		rbsp_bits(rbsp, 4, &pps->pps_extension_4bits);
+	}
+	if (pps->pps_range_extension_flag)
+		rbsp_unsupported(rbsp);
+	if (pps->pps_multilayer_extension_flag)
+		rbsp_unsupported(rbsp);
+	if (pps->pps_3d_extension_flag)
+		rbsp_unsupported(rbsp);
+	if (pps->pps_scc_extension_flag)
+		rbsp_unsupported(rbsp);
+	if (pps->pps_extension_4bits)
+		rbsp_unsupported(rbsp);
+}
+
+/**
+ * nal_hevc_write_vps() - Write PPS NAL unit into RBSP format
+ * @dev: device pointer
+ * @dest: the buffer that is filled with RBSP data
+ * @n: maximum size of @dest in bytes
+ * @pps: &struct nal_hevc_vps to convert to RBSP
+ *
+ * Convert @vps to RBSP data and write it into @dest.
+ *
+ * The size of the VPS NAL unit is not known in advance and this function will
+ * fail, if @dest does not hold sufficient space for the VPS NAL unit.
+ *
+ * Return: number of bytes written to @dest or negative error code
+ */
+ssize_t nal_hevc_write_vps(const struct device *dev,
+			   void *dest, size_t n, struct nal_hevc_vps *vps)
+{
+	struct rbsp rbsp;
+	unsigned int forbidden_zero_bit = 0;
+	unsigned int nal_unit_type = VPS_NUT;
+	unsigned int nuh_layer_id = 0;
+	unsigned int nuh_temporal_id_plus1 = 1;
+
+	if (!dest)
+		return -EINVAL;
+
+	rbsp_init(&rbsp, dest, n, &write);
+
+	nal_hevc_write_start_code_prefix(&rbsp);
+
+	/* NAL unit header */
+	rbsp_bit(&rbsp, &forbidden_zero_bit);
+	rbsp_bits(&rbsp, 6, &nal_unit_type);
+	rbsp_bits(&rbsp, 6, &nuh_layer_id);
+	rbsp_bits(&rbsp, 3, &nuh_temporal_id_plus1);
+
+	nal_hevc_rbsp_vps(&rbsp, vps);
+
+	rbsp_trailing_bits(&rbsp);
+
+	if (rbsp.error)
+		return rbsp.error;
+
+	return DIV_ROUND_UP(rbsp.pos, 8);
+}
+EXPORT_SYMBOL_GPL(nal_hevc_write_vps);
+
+/**
+ * nal_hevc_read_vps() - Read VPS NAL unit from RBSP format
+ * @dev: device pointer
+ * @vps: the &struct nal_hevc_vps to fill from the RBSP data
+ * @src: the buffer that contains the RBSP data
+ * @n: size of @src in bytes
+ *
+ * Read RBSP data from @src and use it to fill @vps.
+ *
+ * Return: number of bytes read from @src or negative error code
+ */
+ssize_t nal_hevc_read_vps(const struct device *dev,
+			  struct nal_hevc_vps *vps, void *src, size_t n)
+{
+	struct rbsp rbsp;
+	unsigned int forbidden_zero_bit;
+	unsigned int nal_unit_type;
+	unsigned int nuh_layer_id;
+	unsigned int nuh_temporal_id_plus1;
+
+	if (!src)
+		return -EINVAL;
+
+	rbsp_init(&rbsp, src, n, &read);
+
+	nal_hevc_read_start_code_prefix(&rbsp);
+
+	rbsp_bit(&rbsp, &forbidden_zero_bit);
+	rbsp_bits(&rbsp, 6, &nal_unit_type);
+	rbsp_bits(&rbsp, 6, &nuh_layer_id);
+	rbsp_bits(&rbsp, 3, &nuh_temporal_id_plus1);
+
+	if (rbsp.error ||
+	    forbidden_zero_bit != 0 ||
+	    nal_unit_type != VPS_NUT)
+		return -EINVAL;
+
+	nal_hevc_rbsp_vps(&rbsp, vps);
+
+	rbsp_trailing_bits(&rbsp);
+
+	if (rbsp.error)
+		return rbsp.error;
+
+	return DIV_ROUND_UP(rbsp.pos, 8);
+}
+EXPORT_SYMBOL_GPL(nal_hevc_read_vps);
+
+/**
+ * nal_hevc_write_sps() - Write SPS NAL unit into RBSP format
+ * @dev: device pointer
+ * @dest: the buffer that is filled with RBSP data
+ * @n: maximum size of @dest in bytes
+ * @sps: &struct nal_hevc_sps to convert to RBSP
+ *
+ * Convert @sps to RBSP data and write it into @dest.
+ *
+ * The size of the SPS NAL unit is not known in advance and this function will
+ * fail, if @dest does not hold sufficient space for the SPS NAL unit.
+ *
+ * Return: number of bytes written to @dest or negative error code
+ */
+ssize_t nal_hevc_write_sps(const struct device *dev,
+			   void *dest, size_t n, struct nal_hevc_sps *sps)
+{
+	struct rbsp rbsp;
+	unsigned int forbidden_zero_bit = 0;
+	unsigned int nal_unit_type = SPS_NUT;
+	unsigned int nuh_layer_id = 0;
+	unsigned int nuh_temporal_id_plus1 = 1;
+
+	if (!dest)
+		return -EINVAL;
+
+	rbsp_init(&rbsp, dest, n, &write);
+
+	nal_hevc_write_start_code_prefix(&rbsp);
+
+	/* NAL unit header */
+	rbsp_bit(&rbsp, &forbidden_zero_bit);
+	rbsp_bits(&rbsp, 6, &nal_unit_type);
+	rbsp_bits(&rbsp, 6, &nuh_layer_id);
+	rbsp_bits(&rbsp, 3, &nuh_temporal_id_plus1);
+
+	nal_hevc_rbsp_sps(&rbsp, sps);
+
+	rbsp_trailing_bits(&rbsp);
+
+	if (rbsp.error)
+		return rbsp.error;
+
+	return DIV_ROUND_UP(rbsp.pos, 8);
+}
+EXPORT_SYMBOL_GPL(nal_hevc_write_sps);
+
+/**
+ * nal_hevc_read_sps() - Read SPS NAL unit from RBSP format
+ * @dev: device pointer
+ * @sps: the &struct nal_hevc_sps to fill from the RBSP data
+ * @src: the buffer that contains the RBSP data
+ * @n: size of @src in bytes
+ *
+ * Read RBSP data from @src and use it to fill @sps.
+ *
+ * Return: number of bytes read from @src or negative error code
+ */
+ssize_t nal_hevc_read_sps(const struct device *dev,
+			  struct nal_hevc_sps *sps, void *src, size_t n)
+{
+	struct rbsp rbsp;
+	unsigned int forbidden_zero_bit;
+	unsigned int nal_unit_type;
+	unsigned int nuh_layer_id;
+	unsigned int nuh_temporal_id_plus1;
+
+	if (!src)
+		return -EINVAL;
+
+	rbsp_init(&rbsp, src, n, &read);
+
+	nal_hevc_read_start_code_prefix(&rbsp);
+
+	rbsp_bit(&rbsp, &forbidden_zero_bit);
+	rbsp_bits(&rbsp, 6, &nal_unit_type);
+	rbsp_bits(&rbsp, 6, &nuh_layer_id);
+	rbsp_bits(&rbsp, 3, &nuh_temporal_id_plus1);
+
+	if (rbsp.error ||
+	    forbidden_zero_bit != 0 ||
+	    nal_unit_type != SPS_NUT)
+		return -EINVAL;
+
+	nal_hevc_rbsp_sps(&rbsp, sps);
+
+	rbsp_trailing_bits(&rbsp);
+
+	if (rbsp.error)
+		return rbsp.error;
+
+	return DIV_ROUND_UP(rbsp.pos, 8);
+}
+EXPORT_SYMBOL_GPL(nal_hevc_read_sps);
+
+/**
+ * nal_hevc_write_pps() - Write PPS NAL unit into RBSP format
+ * @dev: device pointer
+ * @dest: the buffer that is filled with RBSP data
+ * @n: maximum size of @dest in bytes
+ * @pps: &struct nal_hevc_pps to convert to RBSP
+ *
+ * Convert @pps to RBSP data and write it into @dest.
+ *
+ * The size of the PPS NAL unit is not known in advance and this function will
+ * fail, if @dest does not hold sufficient space for the PPS NAL unit.
+ *
+ * Return: number of bytes written to @dest or negative error code
+ */
+ssize_t nal_hevc_write_pps(const struct device *dev,
+			   void *dest, size_t n, struct nal_hevc_pps *pps)
+{
+	struct rbsp rbsp;
+	unsigned int forbidden_zero_bit = 0;
+	unsigned int nal_unit_type = PPS_NUT;
+	unsigned int nuh_layer_id = 0;
+	unsigned int nuh_temporal_id_plus1 = 1;
+
+	if (!dest)
+		return -EINVAL;
+
+	rbsp_init(&rbsp, dest, n, &write);
+
+	nal_hevc_write_start_code_prefix(&rbsp);
+
+	/* NAL unit header */
+	rbsp_bit(&rbsp, &forbidden_zero_bit);
+	rbsp_bits(&rbsp, 6, &nal_unit_type);
+	rbsp_bits(&rbsp, 6, &nuh_layer_id);
+	rbsp_bits(&rbsp, 3, &nuh_temporal_id_plus1);
+
+	nal_hevc_rbsp_pps(&rbsp, pps);
+
+	rbsp_trailing_bits(&rbsp);
+
+	if (rbsp.error)
+		return rbsp.error;
+
+	return DIV_ROUND_UP(rbsp.pos, 8);
+}
+EXPORT_SYMBOL_GPL(nal_hevc_write_pps);
+
+/**
+ * nal_hevc_read_pps() - Read PPS NAL unit from RBSP format
+ * @dev: device pointer
+ * @pps: the &struct nal_hevc_pps to fill from the RBSP data
+ * @src: the buffer that contains the RBSP data
+ * @n: size of @src in bytes
+ *
+ * Read RBSP data from @src and use it to fill @pps.
+ *
+ * Return: number of bytes read from @src or negative error code
+ */
+ssize_t nal_hevc_read_pps(const struct device *dev,
+			  struct nal_hevc_pps *pps, void *src, size_t n)
+{
+	struct rbsp rbsp;
+	unsigned int forbidden_zero_bit;
+	unsigned int nal_unit_type;
+	unsigned int nuh_layer_id;
+	unsigned int nuh_temporal_id_plus1;
+
+	if (!src)
+		return -EINVAL;
+
+	rbsp_init(&rbsp, src, n, &read);
+
+	nal_hevc_read_start_code_prefix(&rbsp);
+
+	/* NAL unit header */
+	rbsp_bit(&rbsp, &forbidden_zero_bit);
+	rbsp_bits(&rbsp, 6, &nal_unit_type);
+	rbsp_bits(&rbsp, 6, &nuh_layer_id);
+	rbsp_bits(&rbsp, 3, &nuh_temporal_id_plus1);
+
+	nal_hevc_rbsp_pps(&rbsp, pps);
+
+	rbsp_trailing_bits(&rbsp);
+
+	if (rbsp.error)
+		return rbsp.error;
+
+	return DIV_ROUND_UP(rbsp.pos, 8);
+}
+EXPORT_SYMBOL_GPL(nal_hevc_read_pps);
+
+/**
+ * nal_hevc_write_filler() - Write filler data RBSP
+ * @dev: device pointer
+ * @dest: buffer to fill with filler data
+ * @n: size of the buffer to fill with filler data
+ *
+ * Write a filler data RBSP to @dest with a size of @n bytes and return the
+ * number of written filler data bytes.
+ *
+ * Use this function to generate dummy data in an RBSP data stream that can be
+ * safely ignored by hevc decoders.
+ *
+ * The RBSP format of the filler data is specified in Rec. ITU-T H.265
+ * (02/2018) 7.3.2.8 Filler data RBSP syntax.
+ *
+ * Return: number of filler data bytes (including marker) or negative error
+ */
+ssize_t nal_hevc_write_filler(const struct device *dev, void *dest, size_t n)
+{
+	struct rbsp rbsp;
+	unsigned int forbidden_zero_bit = 0;
+	unsigned int nal_unit_type = FD_NUT;
+	unsigned int nuh_layer_id = 0;
+	unsigned int nuh_temporal_id_plus1 = 1;
+
+	if (!dest)
+		return -EINVAL;
+
+	rbsp_init(&rbsp, dest, n, &write);
+
+	nal_hevc_write_start_code_prefix(&rbsp);
+
+	rbsp_bit(&rbsp, &forbidden_zero_bit);
+	rbsp_bits(&rbsp, 6, &nal_unit_type);
+	rbsp_bits(&rbsp, 6, &nuh_layer_id);
+	rbsp_bits(&rbsp, 3, &nuh_temporal_id_plus1);
+
+	nal_hevc_write_filler_data(&rbsp);
+	rbsp_trailing_bits(&rbsp);
+
+	if (rbsp.error)
+		return rbsp.error;
+
+	return DIV_ROUND_UP(rbsp.pos, 8);
+}
+EXPORT_SYMBOL_GPL(nal_hevc_write_filler);
+
+/**
+ * nal_hevc_read_filler() - Read filler data RBSP
+ * @dev: device pointer
+ * @src: buffer with RBSP data that is read
+ * @n: maximum size of src that shall be read
+ *
+ * Read a filler data RBSP from @src up to a maximum size of @n bytes and
+ * return the size of the filler data in bytes including the marker.
+ *
+ * This function is used to parse filler data and skip the respective bytes in
+ * the RBSP data.
+ *
+ * The RBSP format of the filler data is specified in Rec. ITU-T H.265
+ * (02/2018) 7.3.2.8 Filler data RBSP syntax.
+ *
+ * Return: number of filler data bytes (including marker) or negative error
+ */
+ssize_t nal_hevc_read_filler(const struct device *dev, void *src, size_t n)
+{
+	struct rbsp rbsp;
+	unsigned int forbidden_zero_bit;
+	unsigned int nal_unit_type;
+	unsigned int nuh_layer_id;
+	unsigned int nuh_temporal_id_plus1;
+
+	if (!src)
+		return -EINVAL;
+
+	rbsp_init(&rbsp, src, n, &read);
+
+	nal_hevc_read_start_code_prefix(&rbsp);
+
+	rbsp_bit(&rbsp, &forbidden_zero_bit);
+	rbsp_bits(&rbsp, 6, &nal_unit_type);
+	rbsp_bits(&rbsp, 6, &nuh_layer_id);
+	rbsp_bits(&rbsp, 3, &nuh_temporal_id_plus1);
+
+	if (rbsp.error)
+		return rbsp.error;
+	if (forbidden_zero_bit != 0 ||
+	    nal_unit_type != FD_NUT)
+		return -EINVAL;
+
+	nal_hevc_read_filler_data(&rbsp);
+	rbsp_trailing_bits(&rbsp);
+
+	if (rbsp.error)
+		return rbsp.error;
+
+	return DIV_ROUND_UP(rbsp.pos, 8);
+}
+EXPORT_SYMBOL_GPL(nal_hevc_read_filler);
diff --git a/drivers/media/platform/allegro-dvt/nal-hevc.h b/drivers/media/platform/allegro-dvt/nal-hevc.h
new file mode 100644
index 000000000000..fc994d4242d8
--- /dev/null
+++ b/drivers/media/platform/allegro-dvt/nal-hevc.h
@@ -0,0 +1,350 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2019 Pengutronix, Michael Tretter <kernel@pengutronix.de>
+ *
+ * Convert NAL units between raw byte sequence payloads (RBSP) and C structs.
+ */
+
+#ifndef __NAL_HEVC_H__
+#define __NAL_HEVC_H__
+
+#include <linux/kernel.h>
+#include <linux/types.h>
+#include <media/v4l2-ctrls.h>
+
+struct nal_hevc_profile_tier_level {
+	unsigned int general_profile_space;
+	unsigned int general_tier_flag;
+	unsigned int general_profile_idc;
+	unsigned int general_profile_compatibility_flag[32];
+	unsigned int general_progressive_source_flag;
+	unsigned int general_interlaced_source_flag;
+	unsigned int general_non_packed_constraint_flag;
+	unsigned int general_frame_only_constraint_flag;
+	union {
+		struct {
+			unsigned int general_max_12bit_constraint_flag;
+			unsigned int general_max_10bit_constraint_flag;
+			unsigned int general_max_8bit_constraint_flag;
+			unsigned int general_max_422chroma_constraint_flag;
+			unsigned int general_max_420chroma_constraint_flag;
+			unsigned int general_max_monochrome_constraint_flag;
+			unsigned int general_intra_constraint_flag;
+			unsigned int general_one_picture_only_constraint_flag;
+			unsigned int general_lower_bit_rate_constraint_flag;
+			union {
+				struct {
+					unsigned int general_max_14bit_constraint_flag;
+					unsigned int general_reserved_zero_33bits;
+				};
+				unsigned int general_reserved_zero_34bits;
+			};
+		};
+		struct {
+			unsigned int general_reserved_zero_7bits;
+			/* unsigned int general_one_picture_only_constraint_flag; */
+			unsigned int general_reserved_zero_35bits;
+		};
+		unsigned int general_reserved_zero_43bits;
+	};
+	union {
+		unsigned int general_inbld_flag;
+		unsigned int general_reserved_zero_bit;
+	};
+	unsigned int general_level_idc;
+};
+
+/**
+ * struct nal_hevc_vps - Video parameter set
+ *
+ * C struct representation of the video parameter set NAL unit as defined by
+ * Rec. ITU-T H.265 (02/2018) 7.3.2.1 Video parameter set RBSP syntax
+ */
+struct nal_hevc_vps {
+	unsigned int video_parameter_set_id;
+	unsigned int base_layer_internal_flag;
+	unsigned int base_layer_available_flag;
+	unsigned int max_layers_minus1;
+	unsigned int max_sub_layers_minus1;
+	unsigned int temporal_id_nesting_flag;
+	struct nal_hevc_profile_tier_level profile_tier_level;
+	unsigned int sub_layer_ordering_info_present_flag;
+	struct {
+		unsigned int max_dec_pic_buffering_minus1[7];
+		unsigned int max_num_reorder_pics[7];
+		unsigned int max_latency_increase_plus1[7];
+	};
+	unsigned int max_layer_id;
+	unsigned int num_layer_sets_minus1;
+	unsigned int layer_id_included_flag[1024][64];
+	unsigned int timing_info_present_flag;
+	struct {
+		unsigned int num_units_in_tick;
+		unsigned int time_scale;
+		unsigned int poc_proportional_to_timing_flag;
+		unsigned int num_ticks_poc_diff_one_minus1;
+		unsigned int num_hrd_parameters;
+		struct {
+			unsigned int hrd_layer_set_idx[0];
+			unsigned int cprms_present_flag[0];
+		};
+		/* hrd_parameters( cprms_present_flag[ i ], max_sub_layers_minus1 ) */
+	};
+	unsigned int extension_flag;
+	unsigned int extension_data_flag;
+};
+
+struct nal_hevc_sub_layer_hrd_parameters {
+	unsigned int bit_rate_value_minus1[1];
+	unsigned int cpb_size_value_minus1[1];
+	unsigned int cbr_flag[1];
+};
+
+struct nal_hevc_hrd_parameters {
+	unsigned int nal_hrd_parameters_present_flag;
+	unsigned int vcl_hrd_parameters_present_flag;
+	struct {
+		unsigned int sub_pic_hrd_params_present_flag;
+		struct {
+			unsigned int tick_divisor_minus2;
+			unsigned int du_cpb_removal_delay_increment_length_minus1;
+			unsigned int sub_pic_cpb_params_in_pic_timing_sei_flag;
+			unsigned int dpb_output_delay_du_length_minus1;
+		};
+		unsigned int bit_rate_scale;
+		unsigned int cpb_size_scale;
+		unsigned int cpb_size_du_scale;
+		unsigned int initial_cpb_removal_delay_length_minus1;
+		unsigned int au_cpb_removal_delay_length_minus1;
+		unsigned int dpb_output_delay_length_minus1;
+	};
+	struct {
+		unsigned int fixed_pic_rate_general_flag[1];
+		unsigned int fixed_pic_rate_within_cvs_flag[1];
+		unsigned int elemental_duration_in_tc_minus1[1];
+		unsigned int low_delay_hrd_flag[1];
+		unsigned int cpb_cnt_minus1[1];
+		struct nal_hevc_sub_layer_hrd_parameters nal_hrd[1];
+		struct nal_hevc_sub_layer_hrd_parameters vcl_hrd[1];
+	};
+};
+
+/**
+ * struct nal_hevc_vui_parameters - VUI parameters
+ *
+ * C struct representation of the VUI parameters as defined by Rec. ITU-T
+ * H.265 (02/2018) E.2.1 VUI parameters syntax.
+ */
+struct nal_hevc_vui_parameters {
+	unsigned int aspect_ratio_info_present_flag;
+	struct {
+		unsigned int aspect_ratio_idc;
+		unsigned int sar_width;
+		unsigned int sar_height;
+	};
+	unsigned int overscan_info_present_flag;
+	unsigned int overscan_appropriate_flag;
+	unsigned int video_signal_type_present_flag;
+	struct {
+		unsigned int video_format;
+		unsigned int video_full_range_flag;
+		unsigned int colour_description_present_flag;
+		struct {
+			unsigned int colour_primaries;
+			unsigned int transfer_characteristics;
+			unsigned int matrix_coeffs;
+		};
+	};
+	unsigned int chroma_loc_info_present_flag;
+	struct {
+		unsigned int chroma_sample_loc_type_top_field;
+		unsigned int chroma_sample_loc_type_bottom_field;
+	};
+	unsigned int neutral_chroma_indication_flag;
+	unsigned int field_seq_flag;
+	unsigned int frame_field_info_present_flag;
+	unsigned int default_display_window_flag;
+	struct {
+		unsigned int def_disp_win_left_offset;
+		unsigned int def_disp_win_right_offset;
+		unsigned int def_disp_win_top_offset;
+		unsigned int def_disp_win_bottom_offset;
+	};
+	unsigned int vui_timing_info_present_flag;
+	struct {
+		unsigned int vui_num_units_in_tick;
+		unsigned int vui_time_scale;
+		unsigned int vui_poc_proportional_to_timing_flag;
+		unsigned int vui_num_ticks_poc_diff_one_minus1;
+		unsigned int vui_hrd_parameters_present_flag;
+		struct nal_hevc_hrd_parameters nal_hrd_parameters;
+	};
+	unsigned int bitstream_restriction_flag;
+	struct {
+		unsigned int tiles_fixed_structure_flag;
+		unsigned int motion_vectors_over_pic_boundaries_flag;
+		unsigned int restricted_ref_pic_lists_flag;
+		unsigned int min_spatial_segmentation_idc;
+		unsigned int max_bytes_per_pic_denom;
+		unsigned int max_bits_per_min_cu_denom;
+		unsigned int log2_max_mv_length_horizontal;
+		unsigned int log2_max_mv_length_vertical;
+	};
+};
+
+/**
+ * struct nal_hevc_sps - Sequence parameter set
+ *
+ * C struct representation of the video parameter set NAL unit as defined by
+ * Rec. ITU-T H.265 (02/2018) 7.3.2.2 Sequence parameter set RBSP syntax
+ */
+struct nal_hevc_sps {
+	unsigned int video_parameter_set_id;
+	unsigned int max_sub_layers_minus1;
+	unsigned int temporal_id_nesting_flag;
+	struct nal_hevc_profile_tier_level profile_tier_level;
+	unsigned int seq_parameter_set_id;
+	unsigned int chroma_format_idc;
+	unsigned int separate_colour_plane_flag;
+	unsigned int pic_width_in_luma_samples;
+	unsigned int pic_height_in_luma_samples;
+	unsigned int conformance_window_flag;
+	struct {
+		unsigned int conf_win_left_offset;
+		unsigned int conf_win_right_offset;
+		unsigned int conf_win_top_offset;
+		unsigned int conf_win_bottom_offset;
+	};
+
+	unsigned int bit_depth_luma_minus8;
+	unsigned int bit_depth_chroma_minus8;
+	unsigned int log2_max_pic_order_cnt_lsb_minus4;
+	unsigned int sub_layer_ordering_info_present_flag;
+	struct {
+		unsigned int max_dec_pic_buffering_minus1[7];
+		unsigned int max_num_reorder_pics[7];
+		unsigned int max_latency_increase_plus1[7];
+	};
+	unsigned int log2_min_luma_coding_block_size_minus3;
+	unsigned int log2_diff_max_min_luma_coding_block_size;
+	unsigned int log2_min_luma_transform_block_size_minus2;
+	unsigned int log2_diff_max_min_luma_transform_block_size;
+	unsigned int max_transform_hierarchy_depth_inter;
+	unsigned int max_transform_hierarchy_depth_intra;
+
+	unsigned int scaling_list_enabled_flag;
+	unsigned int scaling_list_data_present_flag;
+	unsigned int amp_enabled_flag;
+	unsigned int sample_adaptive_offset_enabled_flag;
+	unsigned int pcm_enabled_flag;
+	struct {
+		unsigned int pcm_sample_bit_depth_luma_minus1;
+		unsigned int pcm_sample_bit_depth_chroma_minus1;
+		unsigned int log2_min_pcm_luma_coding_block_size_minus3;
+		unsigned int log2_diff_max_min_pcm_luma_coding_block_size;
+		unsigned int pcm_loop_filter_disabled_flag;
+	};
+
+	unsigned int num_short_term_ref_pic_sets;
+	unsigned int long_term_ref_pics_present_flag;
+	unsigned int sps_temporal_mvp_enabled_flag;
+	unsigned int strong_intra_smoothing_enabled_flag;
+	unsigned int vui_parameters_present_flag;
+	struct nal_hevc_vui_parameters vui;
+	unsigned int extension_present_flag;
+	struct {
+		unsigned int sps_range_extension_flag;
+		unsigned int sps_multilayer_extension_flag;
+		unsigned int sps_3d_extension_flag;
+		unsigned int sps_scc_extension_flag;
+		unsigned int sps_extension_4bits;
+	};
+};
+
+struct nal_hevc_pps {
+	unsigned int pps_pic_parameter_set_id;
+	unsigned int pps_seq_parameter_set_id;
+	unsigned int dependent_slice_segments_enabled_flag;
+	unsigned int output_flag_present_flag;
+	unsigned int num_extra_slice_header_bits;
+	unsigned int sign_data_hiding_enabled_flag;
+	unsigned int cabac_init_present_flag;
+	unsigned int num_ref_idx_l0_default_active_minus1;
+	unsigned int num_ref_idx_l1_default_active_minus1;
+	int init_qp_minus26;
+	unsigned int constrained_intra_pred_flag;
+	unsigned int transform_skip_enabled_flag;
+	unsigned int cu_qp_delta_enabled_flag;
+	unsigned int diff_cu_qp_delta_depth;
+	int pps_cb_qp_offset;
+	int pps_cr_qp_offset;
+	unsigned int pps_slice_chroma_qp_offsets_present_flag;
+	unsigned int weighted_pred_flag;
+	unsigned int weighted_bipred_flag;
+	unsigned int transquant_bypass_enabled_flag;
+	unsigned int tiles_enabled_flag;
+	unsigned int entropy_coding_sync_enabled_flag;
+	struct {
+		unsigned int num_tile_columns_minus1;
+		unsigned int num_tile_rows_minus1;
+		unsigned int uniform_spacing_flag;
+		struct {
+			unsigned int column_width_minus1[1];
+			unsigned int row_height_minus1[1];
+		};
+		unsigned int loop_filter_across_tiles_enabled_flag;
+	};
+	unsigned int pps_loop_filter_across_slices_enabled_flag;
+	unsigned int deblocking_filter_control_present_flag;
+	struct {
+		unsigned int deblocking_filter_override_enabled_flag;
+		unsigned int pps_deblocking_filter_disabled_flag;
+		struct {
+			int pps_beta_offset_div2;
+			int pps_tc_offset_div2;
+		};
+	};
+	unsigned int pps_scaling_list_data_present_flag;
+	unsigned int lists_modification_present_flag;
+	unsigned int log2_parallel_merge_level_minus2;
+	unsigned int slice_segment_header_extension_present_flag;
+	unsigned int pps_extension_present_flag;
+	struct {
+		unsigned int pps_range_extension_flag;
+		unsigned int pps_multilayer_extension_flag;
+		unsigned int pps_3d_extension_flag;
+		unsigned int pps_scc_extension_flag;
+		unsigned int pps_extension_4bits;
+	};
+};
+
+int nal_hevc_profile_from_v4l2(enum v4l2_mpeg_video_hevc_profile profile);
+int nal_hevc_tier_from_v4l2(enum v4l2_mpeg_video_hevc_tier tier);
+int nal_hevc_level_from_v4l2(enum v4l2_mpeg_video_hevc_level level);
+
+int nal_range_from_v4l2(enum v4l2_quantization quantization);
+int nal_color_primaries_from_v4l2(enum v4l2_colorspace colorspace);
+int nal_transfer_characteristics_from_v4l2(enum v4l2_colorspace colorspace,
+					   enum v4l2_xfer_func xfer_func);
+int nal_matrix_coeffs_from_v4l2(enum v4l2_colorspace colorspace,
+				enum v4l2_ycbcr_encoding ycbcr_encoding);
+
+ssize_t nal_hevc_write_vps(const struct device *dev,
+			   void *dest, size_t n, struct nal_hevc_vps *vps);
+ssize_t nal_hevc_read_vps(const struct device *dev,
+			  struct nal_hevc_vps *vps, void *src, size_t n);
+
+ssize_t nal_hevc_write_sps(const struct device *dev,
+			   void *dest, size_t n, struct nal_hevc_sps *sps);
+ssize_t nal_hevc_read_sps(const struct device *dev,
+			  struct nal_hevc_sps *sps, void *src, size_t n);
+
+ssize_t nal_hevc_write_pps(const struct device *dev,
+			   void *dest, size_t n, struct nal_hevc_pps *pps);
+ssize_t nal_hevc_read_pps(const struct device *dev,
+			  struct nal_hevc_pps *pps, void *src, size_t n);
+
+ssize_t nal_hevc_write_filler(const struct device *dev, void *dest, size_t n);
+ssize_t nal_hevc_read_filler(const struct device *dev, void *src, size_t n);
+
+#endif /* __NAL_HEVC_H__ */
-- 
2.20.1

[PATCH 04/18] media: allegro: implement S_FMT for CAPTURE

[Michael Tretter]

In order to support different codecs, the driver must support changing
the format on CAPTURE. Therefore, the driver needs to handle S_FMT on
CAPTURE.

As the driver will have a different number of formats for OUTPUT and
CAPTURE, split the check for the format index in ENUM_FMT into CAPTURE
and OUTPUT.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 .../media/platform/allegro-dvt/allegro-core.c | 23 ++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c
index 6b5cbee05040..a733049c8727 100644
--- a/drivers/media/platform/allegro-dvt/allegro-core.c
+++ b/drivers/media/platform/allegro-dvt/allegro-core.c
@@ -2503,13 +2503,15 @@ static int allegro_querycap(struct file *file, void *fh,
 static int allegro_enum_fmt_vid(struct file *file, void *fh,
 				struct v4l2_fmtdesc *f)
 {
-	if (f->index)
-		return -EINVAL;
 	switch (f->type) {
 	case V4L2_BUF_TYPE_VIDEO_OUTPUT:
+		if (f->index >= 1)
+			return -EINVAL;
 		f->pixelformat = V4L2_PIX_FMT_NV12;
 		break;
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE:
+		if (f->index >= 1)
+			return -EINVAL;
 		f->pixelformat = V4L2_PIX_FMT_H264;
 		break;
 	default:
@@ -2557,6 +2559,21 @@ static int allegro_try_fmt_vid_cap(struct file *file, void *fh,
 	return 0;
 }
 
+static int allegro_s_fmt_vid_cap(struct file *file, void *fh,
+				 struct v4l2_format *f)
+{
+	struct allegro_channel *channel = fh_to_channel(fh);
+	int err;
+
+	err = allegro_try_fmt_vid_cap(file, fh, f);
+	if (err)
+		return err;
+
+	channel->codec = f->fmt.pix.pixelformat;
+
+	return 0;
+}
+
 static int allegro_g_fmt_vid_out(struct file *file, void *fh,
 				 struct v4l2_format *f)
 {
@@ -2769,7 +2786,7 @@ static const struct v4l2_ioctl_ops allegro_ioctl_ops = {
 	.vidioc_enum_fmt_vid_out = allegro_enum_fmt_vid,
 	.vidioc_g_fmt_vid_cap = allegro_g_fmt_vid_cap,
 	.vidioc_try_fmt_vid_cap = allegro_try_fmt_vid_cap,
-	.vidioc_s_fmt_vid_cap = allegro_try_fmt_vid_cap,
+	.vidioc_s_fmt_vid_cap = allegro_s_fmt_vid_cap,
 	.vidioc_g_fmt_vid_out = allegro_g_fmt_vid_out,
 	.vidioc_try_fmt_vid_out = allegro_try_fmt_vid_out,
 	.vidioc_s_fmt_vid_out = allegro_s_fmt_vid_out,
-- 
2.20.1

[PATCH 05/18] media: allegro: adjust channel after format change

[Michael Tretter]

A format change (i.e. the frame size or the codec) has multiple effects
on a channel:

- The available controls

- The limits of the available controls

- The default encoding options

To avoid scattering the changes all over the driver, add a new function
that goes over the channel and does all required adjustments.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 .../media/platform/allegro-dvt/allegro-core.c | 69 ++++++++++++++-----
 1 file changed, 52 insertions(+), 17 deletions(-)

diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c
index a733049c8727..3253262d56f9 100644
--- a/drivers/media/platform/allegro-dvt/allegro-core.c
+++ b/drivers/media/platform/allegro-dvt/allegro-core.c
@@ -1963,7 +1963,6 @@ static int allegro_create_channel(struct allegro_channel *channel)
 {
 	struct allegro_dev *dev = channel->dev;
 	unsigned long timeout;
-	enum v4l2_mpeg_video_h264_level min_level;
 
 	if (channel_exists(channel)) {
 		v4l2_warn(&dev->v4l2_dev,
@@ -1986,16 +1985,6 @@ static int allegro_create_channel(struct allegro_channel *channel)
 		 DIV_ROUND_UP(channel->framerate.numerator,
 			      channel->framerate.denominator));
 
-	min_level = select_minimum_h264_level(channel->width, channel->height);
-	if (channel->level < min_level) {
-		v4l2_warn(&dev->v4l2_dev,
-			  "user %d: selected Level %s too low: increasing to Level %s\n",
-			  channel->user_id,
-			  v4l2_ctrl_get_menu(V4L2_CID_MPEG_VIDEO_H264_LEVEL)[channel->level],
-			  v4l2_ctrl_get_menu(V4L2_CID_MPEG_VIDEO_H264_LEVEL)[min_level]);
-		channel->level = min_level;
-	}
-
 	v4l2_ctrl_grab(channel->mpeg_video_h264_profile, true);
 	v4l2_ctrl_grab(channel->mpeg_video_h264_level, true);
 	v4l2_ctrl_grab(channel->mpeg_video_h264_i_frame_qp, true);
@@ -2031,6 +2020,53 @@ static int allegro_create_channel(struct allegro_channel *channel)
 	return channel->error;
 }
 
+/**
+ * allegro_channel_adjust() - Adjust channel parameters to current format
+ * @channel: the channel to adjust
+ *
+ * Various parameters of a channel and their limits depend on the currently
+ * set format. Adjust the parameters after a format change in one go.
+ */
+static void allegro_channel_adjust(struct allegro_channel *channel)
+{
+	struct allegro_dev *dev = channel->dev;
+	struct v4l2_ctrl *ctrl;
+	s64 min;
+	s64 max;
+
+	channel->sizeimage_encoded =
+		estimate_stream_size(channel->width, channel->height);
+
+	ctrl = channel->mpeg_video_h264_level;
+	min = select_minimum_h264_level(channel->width, channel->height);
+	if (ctrl->minimum > min)
+		v4l2_dbg(1, debug, &dev->v4l2_dev,
+			 "%s.minimum: %lld -> %lld\n",
+			 v4l2_ctrl_get_name(ctrl->id), ctrl->minimum, min);
+	v4l2_ctrl_lock(ctrl);
+	__v4l2_ctrl_modify_range(ctrl, min, ctrl->maximum,
+				 ctrl->step, ctrl->default_value);
+	v4l2_ctrl_unlock(ctrl);
+	channel->level = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level);
+
+	ctrl = channel->mpeg_video_bitrate;
+	max = maximum_bitrate(v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level));
+	if (ctrl->maximum < max)
+		v4l2_dbg(1, debug, &dev->v4l2_dev,
+			 "%s: maximum: %lld -> %lld\n",
+			 v4l2_ctrl_get_name(ctrl->id), ctrl->maximum, max);
+	v4l2_ctrl_lock(ctrl);
+	__v4l2_ctrl_modify_range(ctrl, ctrl->minimum, max,
+				 ctrl->step, ctrl->default_value);
+	v4l2_ctrl_unlock(ctrl);
+
+	ctrl = channel->mpeg_video_bitrate_peak;
+	v4l2_ctrl_lock(ctrl);
+	__v4l2_ctrl_modify_range(ctrl, ctrl->minimum, max,
+				 ctrl->step, ctrl->default_value);
+	v4l2_ctrl_unlock(ctrl);
+}
+
 static void allegro_set_default_params(struct allegro_channel *channel)
 {
 	channel->width = ALLEGRO_WIDTH_DEFAULT;
@@ -2050,8 +2086,6 @@ static void allegro_set_default_params(struct allegro_channel *channel)
 	channel->profile = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE;
 	channel->level =
 		select_minimum_h264_level(channel->width, channel->height);
-	channel->sizeimage_encoded =
-		estimate_stream_size(channel->width, channel->height);
 
 	channel->bitrate = maximum_bitrate(channel->level);
 	channel->bitrate_peak = maximum_bitrate(channel->level);
@@ -2460,6 +2494,8 @@ static int allegro_open(struct file *file)
 	file->private_data = &channel->fh;
 	v4l2_fh_add(&channel->fh);
 
+	allegro_channel_adjust(channel);
+
 	return 0;
 
 error:
@@ -2571,6 +2607,8 @@ static int allegro_s_fmt_vid_cap(struct file *file, void *fh,
 
 	channel->codec = f->fmt.pix.pixelformat;
 
+	allegro_channel_adjust(channel);
+
 	return 0;
 }
 
@@ -2641,10 +2679,7 @@ static int allegro_s_fmt_vid_out(struct file *file, void *fh,
 	channel->quantization = f->fmt.pix.quantization;
 	channel->xfer_func = f->fmt.pix.xfer_func;
 
-	channel->level =
-		select_minimum_h264_level(channel->width, channel->height);
-	channel->sizeimage_encoded =
-		estimate_stream_size(channel->width, channel->height);
+	allegro_channel_adjust(channel);
 
 	return 0;
 }
-- 
2.20.1

[PATCH 06/18] media: allegro: move encoding options to channel

[Michael Tretter]

There are several encoding options that are hard coded in the parameter
that is used to configure the hardware codec. However, some of the
options must be written to the SPS/PPS by the driver. Furthermore, some
of the options depend on the codec that is used (i.e. H.264 or HEVC).

Therefore, move options that depend on the codec to the channel and add
constants for options that are independent of the codec but must be
written to the SPS/PPS.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 .../media/platform/allegro-dvt/allegro-core.c | 85 ++++++++++++++-----
 1 file changed, 65 insertions(+), 20 deletions(-)

diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c
index 3253262d56f9..a2a9000bbb15 100644
--- a/drivers/media/platform/allegro-dvt/allegro-core.c
+++ b/drivers/media/platform/allegro-dvt/allegro-core.c
@@ -94,6 +94,12 @@
 
 #define SIZE_MACROBLOCK 16
 
+/* Encoding options */
+#define LOG2_MAX_FRAME_NUM		4
+#define LOG2_MAX_PIC_ORDER_CNT		10
+#define BETA_OFFSET_DIV_2		-1
+#define TC_OFFSET_DIV_2			-1
+
 static int debug;
 module_param(debug, int, 0644);
 MODULE_PARM_DESC(debug, "Debug level (0-2)");
@@ -202,9 +208,30 @@ struct allegro_channel {
 
 	struct allegro_buffer config_blob;
 
+	unsigned int log2_max_frame_num;
+	bool temporal_mvp_enable;
+
+	bool enable_loop_filter_across_tiles;
+	bool enable_loop_filter_across_slices;
+	bool dbf_ovr_en;
+
 	unsigned int num_ref_idx_l0;
 	unsigned int num_ref_idx_l1;
 
+	/* Maximum range for motion estimation */
+	int b_hrz_me_range;
+	int b_vrt_me_range;
+	int p_hrz_me_range;
+	int p_vrt_me_range;
+	/* Size limits of coding unit */
+	int min_cu_size;
+	int max_cu_size;
+	/* Size limits of transform unit */
+	int min_tu_size;
+	int max_tu_size;
+	int max_transfo_depth_intra;
+	int max_transfo_depth_inter;
+
 	struct v4l2_ctrl *mpeg_video_h264_profile;
 	struct v4l2_ctrl *mpeg_video_h264_level;
 	struct v4l2_ctrl *mpeg_video_h264_i_frame_qp;
@@ -911,32 +938,32 @@ static int fill_create_channel_param(struct allegro_channel *channel,
 	param->level = v4l2_level_to_mcu_level(channel->level);
 	param->tier = 0;
 
-	param->log2_max_poc = 10;
-	param->log2_max_frame_num = 4;
-	param->temporal_mvp_enable = 1;
+	param->log2_max_poc = LOG2_MAX_PIC_ORDER_CNT;
+	param->log2_max_frame_num = channel->log2_max_frame_num;
+	param->temporal_mvp_enable = channel->temporal_mvp_enable;
 
-	param->dbf_ovr_en = 1;
+	param->dbf_ovr_en = channel->dbf_ovr_en;
 	param->rdo_cost_mode = 1;
 	param->custom_lda = 1;
 	param->lf = 1;
-	param->lf_x_tile = 1;
-	param->lf_x_slice = 1;
+	param->lf_x_tile = channel->enable_loop_filter_across_tiles;
+	param->lf_x_slice = channel->enable_loop_filter_across_slices;
 
 	param->src_bit_depth = 8;
 
-	param->beta_offset = -1;
-	param->tc_offset = -1;
+	param->beta_offset = BETA_OFFSET_DIV_2;
+	param->tc_offset = TC_OFFSET_DIV_2;
 	param->num_slices = 1;
-	param->me_range[0] = 8;
-	param->me_range[1] = 8;
-	param->me_range[2] = 16;
-	param->me_range[3] = 16;
-	param->max_cu_size = ilog2(SIZE_MACROBLOCK);
-	param->min_cu_size = ilog2(8);
-	param->max_tu_size = 2;
-	param->min_tu_size = 2;
-	param->max_transfo_depth_intra = 1;
-	param->max_transfo_depth_inter = 1;
+	param->me_range[0] = channel->b_hrz_me_range;
+	param->me_range[1] = channel->b_vrt_me_range;
+	param->me_range[2] = channel->p_hrz_me_range;
+	param->me_range[3] = channel->p_vrt_me_range;
+	param->max_cu_size = channel->max_cu_size;
+	param->min_cu_size = channel->min_cu_size;
+	param->max_tu_size = channel->max_tu_size;
+	param->min_tu_size = channel->min_tu_size;
+	param->max_transfo_depth_intra = channel->max_transfo_depth_intra;
+	param->max_transfo_depth_inter = channel->max_transfo_depth_inter;
 
 	param->prefetch_auto = 0;
 	param->prefetch_mem_offset = 0;
@@ -1266,9 +1293,9 @@ static ssize_t allegro_h264_write_sps(struct allegro_channel *channel,
 	sps->constraint_set5_flag = 0;
 	sps->level_idc = nal_h264_level_from_v4l2(channel->level);
 	sps->seq_parameter_set_id = 0;
-	sps->log2_max_frame_num_minus4 = 0;
+	sps->log2_max_frame_num_minus4 = LOG2_MAX_FRAME_NUM - 4;
 	sps->pic_order_cnt_type = 0;
-	sps->log2_max_pic_order_cnt_lsb_minus4 = 6;
+	sps->log2_max_pic_order_cnt_lsb_minus4 = LOG2_MAX_PIC_ORDER_CNT - 4;
 	sps->max_num_ref_frames = 3;
 	sps->gaps_in_frame_num_value_allowed_flag = 0;
 	sps->pic_width_in_mbs_minus1 =
@@ -2065,6 +2092,24 @@ static void allegro_channel_adjust(struct allegro_channel *channel)
 	__v4l2_ctrl_modify_range(ctrl, ctrl->minimum, max,
 				 ctrl->step, ctrl->default_value);
 	v4l2_ctrl_unlock(ctrl);
+
+	channel->log2_max_frame_num = LOG2_MAX_FRAME_NUM;
+	channel->temporal_mvp_enable = true;
+
+	channel->dbf_ovr_en = true;
+	channel->enable_loop_filter_across_tiles = true;
+	channel->enable_loop_filter_across_slices = true;
+
+	channel->b_hrz_me_range = 8;
+	channel->b_vrt_me_range = 8;
+	channel->p_hrz_me_range = 16;
+	channel->p_vrt_me_range = 16;
+	channel->max_cu_size = ilog2(16);
+	channel->min_cu_size = ilog2(8);
+	channel->max_tu_size = ilog2(4);
+	channel->min_tu_size = ilog2(4);
+	channel->max_transfo_depth_intra = 1;
+	channel->max_transfo_depth_inter = 1;
 }
 
 static void allegro_set_default_params(struct allegro_channel *channel)
-- 
2.20.1

[PATCH 07/18] media: allegro: fix log2_max_poc in firmware 2019.1

[Michael Tretter]

The log2_max_poc field is used to set log2_max_pic_order_cnt_lsb_minus4
for the coded stream. It has an allowed range of 0 to 12.

param contains the value without the minus4, but since firmware
version 2019.1, the value has to be log2_max_pic_order_cnt_lsb - 1,
presumably to fit the maximum value of 16 into a 4 bit field.

The driver does not support firmware version 2019.1. Thus, change the
behaviour starting from firmware version 2019.2.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 drivers/media/platform/allegro-dvt/allegro-mail.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/allegro-dvt/allegro-mail.c b/drivers/media/platform/allegro-dvt/allegro-mail.c
index 993e16f06305..5dbc1c029020 100644
--- a/drivers/media/platform/allegro-dvt/allegro-mail.c
+++ b/drivers/media/platform/allegro-dvt/allegro-mail.c
@@ -109,8 +109,11 @@ allegro_encode_config_blob(u32 *dst, struct create_channel_param *param)
 
 	val = 0;
 	val |= param->temporal_mvp_enable ? BIT(20) : 0;
-	val |= FIELD_PREP(GENMASK(7, 4), param->log2_max_frame_num) |
-	       FIELD_PREP(GENMASK(3, 0), param->log2_max_poc);
+	val |= FIELD_PREP(GENMASK(7, 4), param->log2_max_frame_num);
+	if (version >= MCU_MSG_VERSION_2019_2)
+		val |= FIELD_PREP(GENMASK(3, 0), param->log2_max_poc - 1);
+	else
+		val |= FIELD_PREP(GENMASK(3, 0), param->log2_max_poc);
 	dst[i++] = val;
 
 	val = 0;
-- 
2.20.1

[PATCH 08/18] media: allegro: use handler_setup to configure channel

[Michael Tretter]

v4l2_ctrl_handler_setup() calls s_ctrl for all controls of the handler.
This ensures that the channel is initialized using the default values of
the v4l2-ctrls.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 drivers/media/platform/allegro-dvt/allegro-core.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c
index a2a9000bbb15..859a2c81c782 100644
--- a/drivers/media/platform/allegro-dvt/allegro-core.c
+++ b/drivers/media/platform/allegro-dvt/allegro-core.c
@@ -2520,6 +2520,8 @@ static int allegro_open(struct file *file)
 
 	v4l2_ctrl_cluster(3, &channel->mpeg_video_bitrate_mode);
 
+	v4l2_ctrl_handler_setup(handler);
+
 	channel->mcu_channel_id = -1;
 	channel->user_id = -1;
 
-- 
2.20.1

[PATCH 09/18] media: allegro: initialize bitrate using v4l2_ctrl

[Michael Tretter]

As the driver now uses the v4l2-ctrls to setup the channel, there is no
need to explicitly set the bitrate. The initial bitrate is now set via
the same path as if it is set from userspace using the v4l2-ctrl.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 drivers/media/platform/allegro-dvt/allegro-core.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c
index 859a2c81c782..db504c4a81f6 100644
--- a/drivers/media/platform/allegro-dvt/allegro-core.c
+++ b/drivers/media/platform/allegro-dvt/allegro-core.c
@@ -2132,8 +2132,6 @@ static void allegro_set_default_params(struct allegro_channel *channel)
 	channel->level =
 		select_minimum_h264_level(channel->width, channel->height);
 
-	channel->bitrate = maximum_bitrate(channel->level);
-	channel->bitrate_peak = maximum_bitrate(channel->level);
 	channel->cpb_size = maximum_cpb_size(channel->level);
 	channel->gop_size = ALLEGRO_GOP_SIZE_DEFAULT;
 }
@@ -2421,6 +2419,8 @@ static int allegro_open(struct file *file)
 	struct v4l2_ctrl_handler *handler;
 	u64 mask;
 	int ret;
+	unsigned int bitrate_max;
+	unsigned int bitrate_def;
 
 	channel = kzalloc(sizeof(*channel), GFP_KERNEL);
 	if (!channel)
@@ -2486,16 +2486,17 @@ static int allegro_open(struct file *file)
 			V4L2_CID_MPEG_VIDEO_BITRATE_MODE,
 			V4L2_MPEG_VIDEO_BITRATE_MODE_CBR, 0,
 			V4L2_MPEG_VIDEO_BITRATE_MODE_CBR);
+
+	bitrate_max = maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
+	bitrate_def = maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
 	channel->mpeg_video_bitrate = v4l2_ctrl_new_std(handler,
 			&allegro_ctrl_ops,
 			V4L2_CID_MPEG_VIDEO_BITRATE,
-			0, maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1),
-			1, channel->bitrate);
+			0, bitrate_max, 1, bitrate_def);
 	channel->mpeg_video_bitrate_peak = v4l2_ctrl_new_std(handler,
 			&allegro_ctrl_ops,
 			V4L2_CID_MPEG_VIDEO_BITRATE_PEAK,
-			0, maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1),
-			1, channel->bitrate_peak);
+			0, bitrate_max, 1, bitrate_def);
 	channel->mpeg_video_cpb_size = v4l2_ctrl_new_std(handler,
 			&allegro_ctrl_ops,
 			V4L2_CID_MPEG_VIDEO_H264_CPB_SIZE,
-- 
2.20.1

[PATCH 10/18] media: allegro: implement scaling of cpb size in SPS

[Michael Tretter]

The cbp_size_value_minus1 in the SPS can scaled using cpb_size_scale to
reduce the number of bits necessary to encode the value. For simplicity,
the scaling was set to 1.

Restructure to the code to make it easier to drop the cbp_size from the
channel and as we are at it, also properly implement the scaling.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 drivers/media/platform/allegro-dvt/allegro-core.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c
index db504c4a81f6..4e5590e8f875 100644
--- a/drivers/media/platform/allegro-dvt/allegro-core.c
+++ b/drivers/media/platform/allegro-dvt/allegro-core.c
@@ -1279,6 +1279,8 @@ static ssize_t allegro_h264_write_sps(struct allegro_channel *channel,
 	/* Calculation of crop units in Rec. ITU-T H.264 (04/2017) p. 76 */
 	unsigned int crop_unit_x = 2;
 	unsigned int crop_unit_y = 2;
+	unsigned int cpb_size;
+	unsigned int cpb_size_scale;
 
 	sps = kzalloc(sizeof(*sps), GFP_KERNEL);
 	if (!sps)
@@ -1336,13 +1338,15 @@ static ssize_t allegro_h264_write_sps(struct allegro_channel *channel,
 	sps->vui.vcl_hrd_parameters_present_flag = 1;
 	sps->vui.vcl_hrd_parameters.cpb_cnt_minus1 = 0;
 	sps->vui.vcl_hrd_parameters.bit_rate_scale = 0;
-	sps->vui.vcl_hrd_parameters.cpb_size_scale = 1;
 	/* See Rec. ITU-T H.264 (04/2017) p. 410 E-53 */
 	sps->vui.vcl_hrd_parameters.bit_rate_value_minus1[0] =
 		channel->bitrate_peak / (1 << (6 + sps->vui.vcl_hrd_parameters.bit_rate_scale)) - 1;
 	/* See Rec. ITU-T H.264 (04/2017) p. 410 E-54 */
+	cpb_size = channel->cpb_size;
+	cpb_size_scale = ffs(cpb_size) - 4;
+	sps->vui.vcl_hrd_parameters.cpb_size_scale = cpb_size_scale;
 	sps->vui.vcl_hrd_parameters.cpb_size_value_minus1[0] =
-		(channel->cpb_size * 1000) / (1 << (4 + sps->vui.vcl_hrd_parameters.cpb_size_scale)) - 1;
+		(cpb_size * 1000) / (1 << (4 + cpb_size_scale)) - 1;
 	sps->vui.vcl_hrd_parameters.cbr_flag[0] =
 		!v4l2_ctrl_g_ctrl(channel->mpeg_video_frame_rc_enable);
 	sps->vui.vcl_hrd_parameters.initial_cpb_removal_delay_length_minus1 = 31;
-- 
2.20.1

[PATCH 11/18] media: allegro: remove cpb_size and gop_size from channel

[Michael Tretter]

The cpb_size and the gop_size are straight copies of the values in the
v4l2-ctrls. To avoid this duplication, directly get the values from the
v4l2-ctrls and remove the copies.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 .../media/platform/allegro-dvt/allegro-core.c | 30 +++++++------------
 1 file changed, 11 insertions(+), 19 deletions(-)

diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c
index 4e5590e8f875..8cabc6ba9f7e 100644
--- a/drivers/media/platform/allegro-dvt/allegro-core.c
+++ b/drivers/media/platform/allegro-dvt/allegro-core.c
@@ -203,8 +203,6 @@ struct allegro_channel {
 	bool frame_rc_enable;
 	unsigned int bitrate;
 	unsigned int bitrate_peak;
-	unsigned int cpb_size;
-	unsigned int gop_size;
 
 	struct allegro_buffer config_blob;
 
@@ -925,6 +923,7 @@ static int fill_create_channel_param(struct allegro_channel *channel,
 	int p_frame_qp = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_p_frame_qp);
 	int b_frame_qp = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_b_frame_qp);
 	int bitrate_mode = v4l2_ctrl_g_ctrl(channel->mpeg_video_bitrate_mode);
+	unsigned int cpb_size = v4l2_ctrl_g_ctrl(channel->mpeg_video_cpb_size);
 
 	param->width = channel->width;
 	param->height = channel->height;
@@ -972,8 +971,7 @@ static int fill_create_channel_param(struct allegro_channel *channel,
 	param->rate_control_mode = channel->frame_rc_enable ?
 		v4l2_bitrate_mode_to_mcu_mode(bitrate_mode) : 0;
 
-	param->cpb_size = v4l2_cpb_size_to_mcu(channel->cpb_size,
-					       channel->bitrate_peak);
+	param->cpb_size = v4l2_cpb_size_to_mcu(cpb_size, channel->bitrate_peak);
 	/* Shall be ]0;cpb_size in 90 kHz units]. Use maximum value. */
 	param->initial_rem_delay = param->cpb_size;
 	param->framerate = DIV_ROUND_UP(channel->framerate.numerator,
@@ -996,10 +994,10 @@ static int fill_create_channel_param(struct allegro_channel *channel,
 	param->max_pixel_value = 255;
 
 	param->gop_ctrl_mode = 0x00000002;
-	param->freq_idr = channel->gop_size;
+	param->freq_idr = v4l2_ctrl_g_ctrl(channel->mpeg_video_gop_size);
 	param->freq_lt = 0;
 	param->gdr_mode = 0x00000000;
-	param->gop_length = channel->gop_size;
+	param->gop_length = v4l2_ctrl_g_ctrl(channel->mpeg_video_gop_size);
 	param->subframe_latency = 0x00000000;
 
 	param->lda_factors[0] = 51;
@@ -1342,7 +1340,7 @@ static ssize_t allegro_h264_write_sps(struct allegro_channel *channel,
 	sps->vui.vcl_hrd_parameters.bit_rate_value_minus1[0] =
 		channel->bitrate_peak / (1 << (6 + sps->vui.vcl_hrd_parameters.bit_rate_scale)) - 1;
 	/* See Rec. ITU-T H.264 (04/2017) p. 410 E-54 */
-	cpb_size = channel->cpb_size;
+	cpb_size = v4l2_ctrl_g_ctrl(channel->mpeg_video_cpb_size);
 	cpb_size_scale = ffs(cpb_size) - 4;
 	sps->vui.vcl_hrd_parameters.cpb_size_scale = cpb_size_scale;
 	sps->vui.vcl_hrd_parameters.cpb_size_value_minus1[0] =
@@ -2135,9 +2133,6 @@ static void allegro_set_default_params(struct allegro_channel *channel)
 	channel->profile = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE;
 	channel->level =
 		select_minimum_h264_level(channel->width, channel->height);
-
-	channel->cpb_size = maximum_cpb_size(channel->level);
-	channel->gop_size = ALLEGRO_GOP_SIZE_DEFAULT;
 }
 
 static int allegro_queue_setup(struct vb2_queue *vq,
@@ -2394,12 +2389,6 @@ static int allegro_s_ctrl(struct v4l2_ctrl *ctrl)
 		v4l2_ctrl_activate(channel->mpeg_video_bitrate_peak,
 				   ctrl->val == V4L2_MPEG_VIDEO_BITRATE_MODE_VBR);
 		break;
-	case V4L2_CID_MPEG_VIDEO_H264_CPB_SIZE:
-		channel->cpb_size = ctrl->val;
-		break;
-	case V4L2_CID_MPEG_VIDEO_GOP_SIZE:
-		channel->gop_size = ctrl->val;
-		break;
 	case V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP:
 	case V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP:
 	case V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP:
@@ -2425,6 +2414,8 @@ static int allegro_open(struct file *file)
 	int ret;
 	unsigned int bitrate_max;
 	unsigned int bitrate_def;
+	unsigned int cpb_size_max;
+	unsigned int cpb_size_def;
 
 	channel = kzalloc(sizeof(*channel), GFP_KERNEL);
 	if (!channel)
@@ -2493,6 +2484,8 @@ static int allegro_open(struct file *file)
 
 	bitrate_max = maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
 	bitrate_def = maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
+	cpb_size_max = maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
+	cpb_size_def = maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
 	channel->mpeg_video_bitrate = v4l2_ctrl_new_std(handler,
 			&allegro_ctrl_ops,
 			V4L2_CID_MPEG_VIDEO_BITRATE,
@@ -2504,13 +2497,12 @@ static int allegro_open(struct file *file)
 	channel->mpeg_video_cpb_size = v4l2_ctrl_new_std(handler,
 			&allegro_ctrl_ops,
 			V4L2_CID_MPEG_VIDEO_H264_CPB_SIZE,
-			0, maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1),
-			1, channel->cpb_size);
+			0, cpb_size_max, 1, cpb_size_def);
 	channel->mpeg_video_gop_size = v4l2_ctrl_new_std(handler,
 			&allegro_ctrl_ops,
 			V4L2_CID_MPEG_VIDEO_GOP_SIZE,
 			0, ALLEGRO_GOP_SIZE_MAX,
-			1, channel->gop_size);
+			1, ALLEGRO_GOP_SIZE_DEFAULT);
 	v4l2_ctrl_new_std(handler,
 			  &allegro_ctrl_ops,
 			  V4L2_CID_MIN_BUFFERS_FOR_OUTPUT,
-- 
2.20.1

[PATCH 12/18] media: allegro: remove profile and level from channel

[Michael Tretter]

The profile and level are straight copies from the v4l2-ctrls. Avoid
duplication and directly read the value of the v4l2-ctrl.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 .../media/platform/allegro-dvt/allegro-core.c | 30 +++++++++----------
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c
index 8cabc6ba9f7e..c0519f7a67b8 100644
--- a/drivers/media/platform/allegro-dvt/allegro-core.c
+++ b/drivers/media/platform/allegro-dvt/allegro-core.c
@@ -195,8 +195,6 @@ struct allegro_channel {
 	unsigned int osequence;
 
 	u32 codec;
-	enum v4l2_mpeg_video_h264_profile profile;
-	enum v4l2_mpeg_video_h264_level level;
 	unsigned int sizeimage_encoded;
 	unsigned int csequence;
 
@@ -924,6 +922,8 @@ static int fill_create_channel_param(struct allegro_channel *channel,
 	int b_frame_qp = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_b_frame_qp);
 	int bitrate_mode = v4l2_ctrl_g_ctrl(channel->mpeg_video_bitrate_mode);
 	unsigned int cpb_size = v4l2_ctrl_g_ctrl(channel->mpeg_video_cpb_size);
+	enum v4l2_mpeg_video_h264_profile profile;
+	enum v4l2_mpeg_video_h264_level level;
 
 	param->width = channel->width;
 	param->height = channel->height;
@@ -931,11 +931,13 @@ static int fill_create_channel_param(struct allegro_channel *channel,
 	param->colorspace =
 		v4l2_colorspace_to_mcu_colorspace(channel->colorspace);
 	param->src_mode = 0x0;
-	param->profile = v4l2_profile_to_mcu_profile(channel->profile);
-	param->constraint_set_flags = BIT(1);
+
 	param->codec = channel->codec;
-	param->level = v4l2_level_to_mcu_level(channel->level);
-	param->tier = 0;
+	profile = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_profile);
+	level = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level);
+	param->profile = v4l2_profile_to_mcu_profile(profile);
+	param->constraint_set_flags = BIT(1);
+	param->level = v4l2_level_to_mcu_level(level);
 
 	param->log2_max_poc = LOG2_MAX_PIC_ORDER_CNT;
 	param->log2_max_frame_num = channel->log2_max_frame_num;
@@ -1277,6 +1279,8 @@ static ssize_t allegro_h264_write_sps(struct allegro_channel *channel,
 	/* Calculation of crop units in Rec. ITU-T H.264 (04/2017) p. 76 */
 	unsigned int crop_unit_x = 2;
 	unsigned int crop_unit_y = 2;
+	enum v4l2_mpeg_video_h264_profile profile;
+	enum v4l2_mpeg_video_h264_level level;
 	unsigned int cpb_size;
 	unsigned int cpb_size_scale;
 
@@ -1284,14 +1288,17 @@ static ssize_t allegro_h264_write_sps(struct allegro_channel *channel,
 	if (!sps)
 		return -ENOMEM;
 
-	sps->profile_idc = nal_h264_profile_from_v4l2(channel->profile);
+	profile = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_profile);
+	level = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level);
+
+	sps->profile_idc = nal_h264_profile_from_v4l2(profile);
 	sps->constraint_set0_flag = 0;
 	sps->constraint_set1_flag = 1;
 	sps->constraint_set2_flag = 0;
 	sps->constraint_set3_flag = 0;
 	sps->constraint_set4_flag = 0;
 	sps->constraint_set5_flag = 0;
-	sps->level_idc = nal_h264_level_from_v4l2(channel->level);
+	sps->level_idc = nal_h264_level_from_v4l2(level);
 	sps->seq_parameter_set_id = 0;
 	sps->log2_max_frame_num_minus4 = LOG2_MAX_FRAME_NUM - 4;
 	sps->pic_order_cnt_type = 0;
@@ -2076,7 +2083,6 @@ static void allegro_channel_adjust(struct allegro_channel *channel)
 	__v4l2_ctrl_modify_range(ctrl, min, ctrl->maximum,
 				 ctrl->step, ctrl->default_value);
 	v4l2_ctrl_unlock(ctrl);
-	channel->level = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level);
 
 	ctrl = channel->mpeg_video_bitrate;
 	max = maximum_bitrate(v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level));
@@ -2130,9 +2136,6 @@ static void allegro_set_default_params(struct allegro_channel *channel)
 	channel->sizeimage_raw = channel->stride * channel->height * 3 / 2;
 
 	channel->codec = V4L2_PIX_FMT_H264;
-	channel->profile = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE;
-	channel->level =
-		select_minimum_h264_level(channel->width, channel->height);
 }
 
 static int allegro_queue_setup(struct vb2_queue *vq,
@@ -2377,9 +2380,6 @@ static int allegro_s_ctrl(struct v4l2_ctrl *ctrl)
 		 "s_ctrl: %s = %d\n", v4l2_ctrl_get_name(ctrl->id), ctrl->val);
 
 	switch (ctrl->id) {
-	case V4L2_CID_MPEG_VIDEO_H264_LEVEL:
-		channel->level = ctrl->val;
-		break;
 	case V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE:
 		channel->frame_rc_enable = ctrl->val;
 		break;
-- 
2.20.1

[PATCH 13/18] media: allegro: use accessor functions for QP values

[Michael Tretter]

V4L2 specifies different controls for the QP values of different codecs.

Simplify users that just care for the QP values by providing accessor
function that return the correct control based on the currently set
codec.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 .../media/platform/allegro-dvt/allegro-core.c | 40 ++++++++++++++++---
 1 file changed, 35 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c
index c0519f7a67b8..004df6ea2430 100644
--- a/drivers/media/platform/allegro-dvt/allegro-core.c
+++ b/drivers/media/platform/allegro-dvt/allegro-core.c
@@ -264,6 +264,36 @@ struct allegro_channel {
 	unsigned int error;
 };
 
+static inline int
+allegro_channel_get_i_frame_qp(struct allegro_channel *channel)
+{
+	return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_i_frame_qp);
+}
+
+static inline int
+allegro_channel_get_p_frame_qp(struct allegro_channel *channel)
+{
+	return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_p_frame_qp);
+}
+
+static inline int
+allegro_channel_get_b_frame_qp(struct allegro_channel *channel)
+{
+	return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_b_frame_qp);
+}
+
+static inline int
+allegro_channel_get_min_qp(struct allegro_channel *channel)
+{
+	return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_min_qp);
+}
+
+static inline int
+allegro_channel_get_max_qp(struct allegro_channel *channel)
+{
+	return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_max_qp);
+}
+
 struct allegro_m2m_buffer {
 	struct v4l2_m2m_buffer buf;
 	struct list_head head;
@@ -917,9 +947,9 @@ static s16 get_qp_delta(int minuend, int subtrahend)
 static int fill_create_channel_param(struct allegro_channel *channel,
 				     struct create_channel_param *param)
 {
-	int i_frame_qp = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_i_frame_qp);
-	int p_frame_qp = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_p_frame_qp);
-	int b_frame_qp = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_b_frame_qp);
+	int i_frame_qp = allegro_channel_get_i_frame_qp(channel);
+	int p_frame_qp = allegro_channel_get_p_frame_qp(channel);
+	int b_frame_qp = allegro_channel_get_b_frame_qp(channel);
 	int bitrate_mode = v4l2_ctrl_g_ctrl(channel->mpeg_video_bitrate_mode);
 	unsigned int cpb_size = v4l2_ctrl_g_ctrl(channel->mpeg_video_cpb_size);
 	enum v4l2_mpeg_video_h264_profile profile;
@@ -982,8 +1012,8 @@ static int fill_create_channel_param(struct allegro_channel *channel,
 	param->target_bitrate = channel->bitrate;
 	param->max_bitrate = channel->bitrate_peak;
 	param->initial_qp = i_frame_qp;
-	param->min_qp = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_min_qp);
-	param->max_qp = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_max_qp);
+	param->min_qp = allegro_channel_get_min_qp(channel);
+	param->max_qp = allegro_channel_get_max_qp(channel);
 	param->ip_delta = get_qp_delta(i_frame_qp, p_frame_qp);
 	param->pb_delta = get_qp_delta(p_frame_qp, b_frame_qp);
 	param->golden_ref = 0;
-- 
2.20.1

[PATCH 14/18] media: allegro: add helper to get entropy mode

[Michael Tretter]

The driver only supports CAVLC for H.264, but HEVC only uses CABAC. As
the driver has to explicitly tell the MCU to use CABAC for HEVC, add a
helper function to get the entropy mode.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 drivers/media/platform/allegro-dvt/allegro-core.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c
index 004df6ea2430..016c31655467 100644
--- a/drivers/media/platform/allegro-dvt/allegro-core.c
+++ b/drivers/media/platform/allegro-dvt/allegro-core.c
@@ -944,6 +944,14 @@ static s16 get_qp_delta(int minuend, int subtrahend)
 		return minuend - subtrahend;
 }
 
+static u32 allegro_channel_get_entropy_mode(struct allegro_channel *channel)
+{
+#define ALLEGRO_ENTROPY_MODE_CAVLC 0
+#define ALLEGRO_ENTROPY_MODE_CABAC 1
+
+	return ALLEGRO_ENTROPY_MODE_CAVLC;
+}
+
 static int fill_create_channel_param(struct allegro_channel *channel,
 				     struct create_channel_param *param)
 {
@@ -974,6 +982,7 @@ static int fill_create_channel_param(struct allegro_channel *channel,
 	param->temporal_mvp_enable = channel->temporal_mvp_enable;
 
 	param->dbf_ovr_en = channel->dbf_ovr_en;
+	param->entropy_mode = allegro_channel_get_entropy_mode(channel);
 	param->rdo_cost_mode = 1;
 	param->custom_lda = 1;
 	param->lf = 1;
-- 
2.20.1

[PATCH 15/18] media: allegro: rename codec specific functions

[Michael Tretter]

The maximum bitrate and the size of the CPB are dependent on the level.
As the levels differ between the different codecs, the functions to get
the maximum bitrate and CPB must be codec specific, too.

Rename the functions to make it obvious that the function are only valid
for H.264.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 drivers/media/platform/allegro-dvt/allegro-core.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c
index 016c31655467..6a8196735eaf 100644
--- a/drivers/media/platform/allegro-dvt/allegro-core.c
+++ b/drivers/media/platform/allegro-dvt/allegro-core.c
@@ -507,7 +507,7 @@ select_minimum_h264_level(unsigned int width, unsigned int height)
 	return level;
 }
 
-static unsigned int maximum_bitrate(enum v4l2_mpeg_video_h264_level level)
+static unsigned int h264_maximum_bitrate(enum v4l2_mpeg_video_h264_level level)
 {
 	switch (level) {
 	case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
@@ -546,7 +546,7 @@ static unsigned int maximum_bitrate(enum v4l2_mpeg_video_h264_level level)
 	}
 }
 
-static unsigned int maximum_cpb_size(enum v4l2_mpeg_video_h264_level level)
+static unsigned int h264_maximum_cpb_size(enum v4l2_mpeg_video_h264_level level)
 {
 	switch (level) {
 	case V4L2_MPEG_VIDEO_H264_LEVEL_1_0:
@@ -2124,7 +2124,7 @@ static void allegro_channel_adjust(struct allegro_channel *channel)
 	v4l2_ctrl_unlock(ctrl);
 
 	ctrl = channel->mpeg_video_bitrate;
-	max = maximum_bitrate(v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level));
+	max = h264_maximum_bitrate(v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level));
 	if (ctrl->maximum < max)
 		v4l2_dbg(1, debug, &dev->v4l2_dev,
 			 "%s: maximum: %lld -> %lld\n",
@@ -2521,10 +2521,10 @@ static int allegro_open(struct file *file)
 			V4L2_MPEG_VIDEO_BITRATE_MODE_CBR, 0,
 			V4L2_MPEG_VIDEO_BITRATE_MODE_CBR);
 
-	bitrate_max = maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
-	bitrate_def = maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
-	cpb_size_max = maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
-	cpb_size_def = maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
+	bitrate_max = h264_maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
+	bitrate_def = h264_maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
+	cpb_size_max = h264_maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
+	cpb_size_def = h264_maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
 	channel->mpeg_video_bitrate = v4l2_ctrl_new_std(handler,
 			&allegro_ctrl_ops,
 			V4L2_CID_MPEG_VIDEO_BITRATE,
-- 
2.20.1

[PATCH 16/18] media: allegro: increase offset in CAPTURE buffer

[Michael Tretter]

The HEVC Non-VCL NAL units require more space than the H.264 Non-VCL NAL
units. Therefore, the driver needs to reserve more space in front of the
actual coded data that is written by the hardware codec.

Increase the offset that shall be used by the hardware codec from 64
bytes to 128 bytes which is as arbitrary as before.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 drivers/media/platform/allegro-dvt/allegro-core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c
index 6a8196735eaf..4eacad3d53cb 100644
--- a/drivers/media/platform/allegro-dvt/allegro-core.c
+++ b/drivers/media/platform/allegro-dvt/allegro-core.c
@@ -90,7 +90,7 @@
  * because it needs to write SPS/PPS NAL units. The encoder writes the actual
  * frame data after the offset.
  */
-#define ENCODER_STREAM_OFFSET SZ_64
+#define ENCODER_STREAM_OFFSET SZ_128
 
 #define SIZE_MACROBLOCK 16
 
-- 
2.20.1

[PATCH 17/18] media: allegro: activate v4l2-ctrls only for current codec

[Michael Tretter]

These controls are specific to H.264 and shall only be activated, if the
coded format is H.264.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 .../media/platform/allegro-dvt/allegro-core.c    | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c
index 4eacad3d53cb..5ff4509be13d 100644
--- a/drivers/media/platform/allegro-dvt/allegro-core.c
+++ b/drivers/media/platform/allegro-dvt/allegro-core.c
@@ -2105,6 +2105,7 @@ static int allegro_create_channel(struct allegro_channel *channel)
 static void allegro_channel_adjust(struct allegro_channel *channel)
 {
 	struct allegro_dev *dev = channel->dev;
+	u32 codec = channel->codec;
 	struct v4l2_ctrl *ctrl;
 	s64 min;
 	s64 max;
@@ -2140,6 +2141,21 @@ static void allegro_channel_adjust(struct allegro_channel *channel)
 				 ctrl->step, ctrl->default_value);
 	v4l2_ctrl_unlock(ctrl);
 
+	v4l2_ctrl_activate(channel->mpeg_video_h264_profile,
+			   codec == V4L2_PIX_FMT_H264);
+	v4l2_ctrl_activate(channel->mpeg_video_h264_level,
+			   codec == V4L2_PIX_FMT_H264);
+	v4l2_ctrl_activate(channel->mpeg_video_h264_i_frame_qp,
+			   codec == V4L2_PIX_FMT_H264);
+	v4l2_ctrl_activate(channel->mpeg_video_h264_max_qp,
+			   codec == V4L2_PIX_FMT_H264);
+	v4l2_ctrl_activate(channel->mpeg_video_h264_min_qp,
+			   codec == V4L2_PIX_FMT_H264);
+	v4l2_ctrl_activate(channel->mpeg_video_h264_p_frame_qp,
+			   codec == V4L2_PIX_FMT_H264);
+	v4l2_ctrl_activate(channel->mpeg_video_h264_b_frame_qp,
+			   codec == V4L2_PIX_FMT_H264);
+
 	channel->log2_max_frame_num = LOG2_MAX_FRAME_NUM;
 	channel->temporal_mvp_enable = true;
 
-- 
2.20.1

[PATCH 18/18] media: allegro: add support for HEVC encoding

[Michael Tretter]

The Allegro Codec supports HEVC encoding. The messages to the MCU are
the same for H.264 and HEVC, but some options have to be changed. These
are actually only a few options.

The driver, however, must add the HEVC VPS/SPS/PPS NAL Units to the
coded stream and must properly provide the HEVC format and controls to
user space.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
 .../media/platform/allegro-dvt/allegro-core.c | 545 ++++++++++++++++--
 .../media/platform/allegro-dvt/allegro-mail.c |   6 +
 .../media/platform/allegro-dvt/allegro-mail.h |   1 +
 3 files changed, 516 insertions(+), 36 deletions(-)

diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c
index 5ff4509be13d..83370b0bd704 100644
--- a/drivers/media/platform/allegro-dvt/allegro-core.c
+++ b/drivers/media/platform/allegro-dvt/allegro-core.c
@@ -30,6 +30,7 @@
 
 #include "allegro-mail.h"
 #include "nal-h264.h"
+#include "nal-hevc.h"
 
 /*
  * Support up to 4k video streams. The hardware actually supports higher
@@ -209,6 +210,8 @@ struct allegro_channel {
 
 	bool enable_loop_filter_across_tiles;
 	bool enable_loop_filter_across_slices;
+	bool enable_deblocking_filter_override;
+	bool enable_reordering;
 	bool dbf_ovr_en;
 
 	unsigned int num_ref_idx_l0;
@@ -235,6 +238,16 @@ struct allegro_channel {
 	struct v4l2_ctrl *mpeg_video_h264_min_qp;
 	struct v4l2_ctrl *mpeg_video_h264_p_frame_qp;
 	struct v4l2_ctrl *mpeg_video_h264_b_frame_qp;
+
+	struct v4l2_ctrl *mpeg_video_hevc_profile;
+	struct v4l2_ctrl *mpeg_video_hevc_level;
+	struct v4l2_ctrl *mpeg_video_hevc_tier;
+	struct v4l2_ctrl *mpeg_video_hevc_i_frame_qp;
+	struct v4l2_ctrl *mpeg_video_hevc_max_qp;
+	struct v4l2_ctrl *mpeg_video_hevc_min_qp;
+	struct v4l2_ctrl *mpeg_video_hevc_p_frame_qp;
+	struct v4l2_ctrl *mpeg_video_hevc_b_frame_qp;
+
 	struct v4l2_ctrl *mpeg_video_frame_rc_enable;
 	struct { /* video bitrate mode control cluster */
 		struct v4l2_ctrl *mpeg_video_bitrate_mode;
@@ -267,31 +280,46 @@ struct allegro_channel {
 static inline int
 allegro_channel_get_i_frame_qp(struct allegro_channel *channel)
 {
-	return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_i_frame_qp);
+	if (channel->codec == V4L2_PIX_FMT_HEVC)
+		return v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_i_frame_qp);
+	else
+		return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_i_frame_qp);
 }
 
 static inline int
 allegro_channel_get_p_frame_qp(struct allegro_channel *channel)
 {
-	return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_p_frame_qp);
+	if (channel->codec == V4L2_PIX_FMT_HEVC)
+		return v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_p_frame_qp);
+	else
+		return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_p_frame_qp);
 }
 
 static inline int
 allegro_channel_get_b_frame_qp(struct allegro_channel *channel)
 {
-	return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_b_frame_qp);
+	if (channel->codec == V4L2_PIX_FMT_HEVC)
+		return v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_b_frame_qp);
+	else
+		return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_b_frame_qp);
 }
 
 static inline int
 allegro_channel_get_min_qp(struct allegro_channel *channel)
 {
-	return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_min_qp);
+	if (channel->codec == V4L2_PIX_FMT_HEVC)
+		return v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_min_qp);
+	else
+		return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_min_qp);
 }
 
 static inline int
 allegro_channel_get_max_qp(struct allegro_channel *channel)
 {
-	return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_max_qp);
+	if (channel->codec == V4L2_PIX_FMT_HEVC)
+		return v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_max_qp);
+	else
+		return v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_max_qp);
 }
 
 struct allegro_m2m_buffer {
@@ -585,6 +613,86 @@ static unsigned int h264_maximum_cpb_size(enum v4l2_mpeg_video_h264_level level)
 	}
 }
 
+static enum v4l2_mpeg_video_hevc_level
+select_minimum_hevc_level(unsigned int width, unsigned int height)
+{
+	unsigned int luma_picture_size = width * height;
+	enum v4l2_mpeg_video_hevc_level level;
+
+	if (luma_picture_size <= 36864)
+		level = V4L2_MPEG_VIDEO_HEVC_LEVEL_1;
+	else if (luma_picture_size <= 122880)
+		level = V4L2_MPEG_VIDEO_HEVC_LEVEL_2;
+	else if (luma_picture_size <= 245760)
+		level = V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1;
+	else if (luma_picture_size <= 552960)
+		level = V4L2_MPEG_VIDEO_HEVC_LEVEL_3;
+	else if (luma_picture_size <= 983040)
+		level = V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1;
+	else if (luma_picture_size <= 2228224)
+		level = V4L2_MPEG_VIDEO_HEVC_LEVEL_4;
+	else if (luma_picture_size <= 8912896)
+		level = V4L2_MPEG_VIDEO_HEVC_LEVEL_5;
+	else
+		level = V4L2_MPEG_VIDEO_HEVC_LEVEL_6;
+
+	return level;
+}
+
+static unsigned int hevc_maximum_bitrate(enum v4l2_mpeg_video_h264_level level)
+{
+	/*
+	 * See Rec. ITU-T H.265 v5 (02/2018), A.4.2 Profile-specific level
+	 * limits for the video profiles.
+	 */
+	switch (level) {
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_1:
+		return 128;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_2:
+		return 1500;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1:
+		return 3000;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_3:
+		return 6000;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1:
+		return 10000;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_4:
+		return 12000;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_4_1:
+		return 20000;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_5:
+		return 25000;
+	default:
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1:
+		return 40000;
+	}
+}
+
+static unsigned int hevc_maximum_cpb_size(enum v4l2_mpeg_video_h264_level level)
+{
+	switch (level) {
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_1:
+		return 350;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_2:
+		return 1500;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1:
+		return 3000;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_3:
+		return 6000;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1:
+		return 10000;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_4:
+		return 12000;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_4_1:
+		return 20000;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_5:
+		return 25000;
+	default:
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1:
+		return 40000;
+	}
+}
+
 static const struct fw_info *
 allegro_get_firmware_info(struct allegro_dev *dev,
 			  const struct firmware *fw,
@@ -908,6 +1016,55 @@ static u16 v4l2_level_to_mcu_level(enum v4l2_mpeg_video_h264_level level)
 	}
 }
 
+static u8 hevc_profile_to_mcu_profile(enum v4l2_mpeg_video_hevc_profile profile)
+{
+	switch (profile) {
+	default:
+	case V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN:
+		return 1;
+	case V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10:
+		return 2;
+	case V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE:
+		return 3;
+	}
+}
+
+static u16 hevc_level_to_mcu_level(enum v4l2_mpeg_video_hevc_level level)
+{
+	switch (level) {
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_1:
+		return 10;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_2:
+		return 20;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1:
+		return 21;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_3:
+		return 30;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1:
+		return 31;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_4:
+		return 40;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_4_1:
+		return 41;
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_5:
+		return 50;
+	default:
+	case V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1:
+		return 51;
+	}
+}
+
+static u8 hevc_tier_to_mcu_tier(enum v4l2_mpeg_video_hevc_tier tier)
+{
+	switch (tier) {
+	default:
+	case V4L2_MPEG_VIDEO_HEVC_TIER_MAIN:
+		return 0;
+	case V4L2_MPEG_VIDEO_HEVC_TIER_HIGH:
+		return 1;
+	}
+}
+
 static u32
 v4l2_bitrate_mode_to_mcu_mode(enum v4l2_mpeg_video_bitrate_mode mode)
 {
@@ -949,6 +1106,10 @@ static u32 allegro_channel_get_entropy_mode(struct allegro_channel *channel)
 #define ALLEGRO_ENTROPY_MODE_CAVLC 0
 #define ALLEGRO_ENTROPY_MODE_CABAC 1
 
+	/* HEVC always uses CABAC, but this has to be explicitly set */
+	if (channel->codec == V4L2_PIX_FMT_HEVC)
+		return ALLEGRO_ENTROPY_MODE_CABAC;
+
 	return ALLEGRO_ENTROPY_MODE_CAVLC;
 }
 
@@ -960,8 +1121,6 @@ static int fill_create_channel_param(struct allegro_channel *channel,
 	int b_frame_qp = allegro_channel_get_b_frame_qp(channel);
 	int bitrate_mode = v4l2_ctrl_g_ctrl(channel->mpeg_video_bitrate_mode);
 	unsigned int cpb_size = v4l2_ctrl_g_ctrl(channel->mpeg_video_cpb_size);
-	enum v4l2_mpeg_video_h264_profile profile;
-	enum v4l2_mpeg_video_h264_level level;
 
 	param->width = channel->width;
 	param->height = channel->height;
@@ -971,17 +1130,37 @@ static int fill_create_channel_param(struct allegro_channel *channel,
 	param->src_mode = 0x0;
 
 	param->codec = channel->codec;
-	profile = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_profile);
-	level = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level);
-	param->profile = v4l2_profile_to_mcu_profile(profile);
-	param->constraint_set_flags = BIT(1);
-	param->level = v4l2_level_to_mcu_level(level);
+	if (channel->codec == V4L2_PIX_FMT_H264) {
+		enum v4l2_mpeg_video_h264_profile profile;
+		enum v4l2_mpeg_video_h264_level level;
+
+		profile = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_profile);
+		level = v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level);
+
+		param->profile = v4l2_profile_to_mcu_profile(profile);
+		param->constraint_set_flags = BIT(1);
+		param->level = v4l2_level_to_mcu_level(level);
+	} else {
+		enum v4l2_mpeg_video_hevc_profile profile;
+		enum v4l2_mpeg_video_hevc_level level;
+		enum v4l2_mpeg_video_hevc_tier tier;
+
+		profile = v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_profile);
+		level = v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_level);
+		tier = v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_tier);
+
+		param->profile = hevc_profile_to_mcu_profile(profile);
+		param->level = hevc_level_to_mcu_level(level);
+		param->tier = hevc_tier_to_mcu_tier(tier);
+	}
 
 	param->log2_max_poc = LOG2_MAX_PIC_ORDER_CNT;
 	param->log2_max_frame_num = channel->log2_max_frame_num;
 	param->temporal_mvp_enable = channel->temporal_mvp_enable;
 
 	param->dbf_ovr_en = channel->dbf_ovr_en;
+	param->override_lf = channel->enable_deblocking_filter_override;
+	param->enable_reordering = channel->enable_reordering;
 	param->entropy_mode = allegro_channel_get_entropy_mode(channel);
 	param->rdo_cost_mode = 1;
 	param->custom_lda = 1;
@@ -1454,6 +1633,158 @@ static void allegro_channel_eos_event(struct allegro_channel *channel)
 	v4l2_event_queue_fh(&channel->fh, &eos_event);
 }
 
+static ssize_t allegro_hevc_write_vps(struct allegro_channel *channel,
+				      void *dest, size_t n)
+{
+	struct allegro_dev *dev = channel->dev;
+	struct nal_hevc_vps *vps;
+	struct nal_hevc_profile_tier_level *ptl;
+	ssize_t size;
+	unsigned int num_ref_frames = channel->num_ref_idx_l0;
+	s32 profile = v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_profile);
+	s32 level = v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_level);
+	s32 tier = v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_tier);
+
+	vps = kzalloc(sizeof(*vps), GFP_KERNEL);
+	if (!vps)
+		return -ENOMEM;
+
+	vps->base_layer_internal_flag = 1;
+	vps->base_layer_available_flag = 1;
+	vps->temporal_id_nesting_flag = 1;
+
+	ptl = &vps->profile_tier_level;
+	ptl->general_profile_idc = nal_hevc_profile_from_v4l2(profile);
+	ptl->general_profile_compatibility_flag[ptl->general_profile_idc] = 1;
+	ptl->general_tier_flag = nal_hevc_tier_from_v4l2(tier);
+	ptl->general_progressive_source_flag = 1;
+	ptl->general_frame_only_constraint_flag = 1;
+	ptl->general_level_idc = nal_hevc_level_from_v4l2(level);
+
+	vps->sub_layer_ordering_info_present_flag = 0;
+	vps->max_dec_pic_buffering_minus1[0] = num_ref_frames;
+	vps->max_num_reorder_pics[0] = num_ref_frames;
+
+	size = nal_hevc_write_vps(&dev->plat_dev->dev, dest, n, vps);
+
+	kfree(vps);
+
+	return size;
+}
+
+static ssize_t allegro_hevc_write_sps(struct allegro_channel *channel,
+				      void *dest, size_t n)
+{
+	struct allegro_dev *dev = channel->dev;
+	struct nal_hevc_sps *sps;
+	struct nal_hevc_profile_tier_level *ptl;
+	ssize_t size;
+	unsigned int num_ref_frames = channel->num_ref_idx_l0;
+	s32 profile = v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_profile);
+	s32 level = v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_level);
+	s32 tier = v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_tier);
+
+	sps = kzalloc(sizeof(*sps), GFP_KERNEL);
+	if (!sps)
+		return -ENOMEM;
+
+	sps->temporal_id_nesting_flag = 1;
+
+	ptl = &sps->profile_tier_level;
+	ptl->general_profile_idc = nal_hevc_profile_from_v4l2(profile);
+	ptl->general_profile_compatibility_flag[ptl->general_profile_idc] = 1;
+	ptl->general_tier_flag = nal_hevc_tier_from_v4l2(tier);
+	ptl->general_progressive_source_flag = 1;
+	ptl->general_frame_only_constraint_flag = 1;
+	ptl->general_level_idc = nal_hevc_level_from_v4l2(level);
+
+	sps->seq_parameter_set_id = 0;
+	sps->chroma_format_idc = 1; /* Only 4:2:0 sampling supported */
+	sps->pic_width_in_luma_samples = round_up(channel->width, 8);
+	sps->pic_height_in_luma_samples = round_up(channel->height, 8);
+	sps->conf_win_right_offset =
+		sps->pic_width_in_luma_samples - channel->width;
+	sps->conf_win_bottom_offset =
+		sps->pic_height_in_luma_samples - channel->height;
+	sps->conformance_window_flag =
+		sps->conf_win_right_offset || sps->conf_win_bottom_offset;
+
+	sps->log2_max_pic_order_cnt_lsb_minus4 = LOG2_MAX_PIC_ORDER_CNT - 4;
+
+	sps->sub_layer_ordering_info_present_flag = 1;
+	sps->max_dec_pic_buffering_minus1[0] = num_ref_frames;
+	sps->max_num_reorder_pics[0] = num_ref_frames;
+
+	sps->log2_min_luma_coding_block_size_minus3 =
+		channel->min_cu_size - 3;
+	sps->log2_diff_max_min_luma_coding_block_size =
+		channel->max_cu_size - channel->min_cu_size;
+	sps->log2_min_luma_transform_block_size_minus2 =
+		channel->min_tu_size - 2;
+	sps->log2_diff_max_min_luma_transform_block_size =
+		channel->max_tu_size - channel->min_tu_size;
+	sps->max_transform_hierarchy_depth_intra =
+		channel->max_transfo_depth_intra;
+	sps->max_transform_hierarchy_depth_inter =
+		channel->max_transfo_depth_inter;
+
+	sps->sps_temporal_mvp_enabled_flag = channel->temporal_mvp_enable;
+	sps->strong_intra_smoothing_enabled_flag = channel->max_cu_size > 4;
+
+	size = nal_hevc_write_sps(&dev->plat_dev->dev, dest, n, sps);
+
+	kfree(sps);
+
+	return size;
+}
+
+static ssize_t allegro_hevc_write_pps(struct allegro_channel *channel,
+				      struct mcu_msg_encode_frame_response *msg,
+				      void *dest, size_t n)
+{
+	struct allegro_dev *dev = channel->dev;
+	struct nal_hevc_pps *pps;
+	ssize_t size;
+	int i;
+
+	pps = kzalloc(sizeof(*pps), GFP_KERNEL);
+	if (!pps)
+		return -ENOMEM;
+
+	pps->pps_pic_parameter_set_id = 0;
+	pps->pps_seq_parameter_set_id = 0;
+
+	if (msg->num_column > 1 || msg->num_row > 1) {
+		pps->tiles_enabled_flag = 1;
+		pps->num_tile_columns_minus1 = msg->num_column - 1;
+		pps->num_tile_rows_minus1 = msg->num_row - 1;
+
+		for (i = 0; i < msg->num_column; i++)
+			pps->column_width_minus1[i] = msg->tile_width[i] - 1;
+
+		for (i = 0; i < msg->num_row; i++)
+			pps->row_height_minus1[i] = msg->tile_height[i] - 1;
+	}
+
+	pps->loop_filter_across_tiles_enabled_flag =
+		channel->enable_loop_filter_across_tiles;
+	pps->pps_loop_filter_across_slices_enabled_flag =
+		channel->enable_loop_filter_across_slices;
+	pps->deblocking_filter_control_present_flag = 1;
+	pps->deblocking_filter_override_enabled_flag =
+		channel->enable_deblocking_filter_override;
+	pps->pps_beta_offset_div2 = BETA_OFFSET_DIV_2;
+	pps->pps_tc_offset_div2 = TC_OFFSET_DIV_2;
+
+	pps->lists_modification_present_flag = channel->enable_reordering;
+
+	size = nal_hevc_write_pps(&dev->plat_dev->dev, dest, n, pps);
+
+	kfree(pps);
+
+	return size;
+}
+
 static u64 allegro_put_buffer(struct allegro_channel *channel,
 			      struct list_head *list,
 			      struct vb2_v4l2_buffer *buffer)
@@ -1577,8 +1908,27 @@ static void allegro_channel_finish_frame(struct allegro_channel *channel,
 
 	curr = vb2_plane_vaddr(&dst_buf->vb2_buf, 0);
 	free = partition->offset;
+
+	if (channel->codec == V4L2_PIX_FMT_HEVC && msg->is_idr) {
+		len = allegro_hevc_write_vps(channel, curr, free);
+		if (len < 0) {
+			v4l2_err(&dev->v4l2_dev,
+				 "not enough space for video parameter set: %zd left\n",
+				 free);
+			goto err;
+		}
+		curr += len;
+		free -= len;
+		v4l2_dbg(1, debug, &dev->v4l2_dev,
+			 "channel %d: wrote %zd byte VPS nal unit\n",
+			 channel->mcu_channel_id, len);
+	}
+
 	if (msg->is_idr) {
-		len = allegro_h264_write_sps(channel, curr, free);
+		if (channel->codec == V4L2_PIX_FMT_H264)
+			len = allegro_h264_write_sps(channel, curr, free);
+		else
+			len = allegro_hevc_write_sps(channel, curr, free);
 		if (len < 0) {
 			v4l2_err(&dev->v4l2_dev,
 				 "not enough space for sequence parameter set: %zd left\n",
@@ -1593,7 +1943,10 @@ static void allegro_channel_finish_frame(struct allegro_channel *channel,
 	}
 
 	if (msg->slice_type == AL_ENC_SLICE_TYPE_I) {
-		len = allegro_h264_write_pps(channel, curr, free);
+		if (channel->codec == V4L2_PIX_FMT_H264)
+			len = allegro_h264_write_pps(channel, curr, free);
+		else
+			len = allegro_hevc_write_pps(channel, msg, curr, free);
 		if (len < 0) {
 			v4l2_err(&dev->v4l2_dev,
 				 "not enough space for picture parameter set: %zd left\n",
@@ -1611,7 +1964,10 @@ static void allegro_channel_finish_frame(struct allegro_channel *channel,
 		dst_buf->vb2_buf.planes[0].data_offset = free;
 		free = 0;
 	} else {
-		len = nal_h264_write_filler(&dev->plat_dev->dev, curr, free);
+		if (channel->codec == V4L2_PIX_FMT_H264)
+			len = nal_h264_write_filler(&dev->plat_dev->dev, curr, free);
+		else
+			len = nal_hevc_write_filler(&dev->plat_dev->dev, curr, free);
 		if (len < 0) {
 			v4l2_err(&dev->v4l2_dev,
 				 "failed to write %zd filler data\n", free);
@@ -2011,6 +2367,16 @@ static void allegro_destroy_channel(struct allegro_channel *channel)
 	v4l2_ctrl_grab(channel->mpeg_video_h264_min_qp, false);
 	v4l2_ctrl_grab(channel->mpeg_video_h264_p_frame_qp, false);
 	v4l2_ctrl_grab(channel->mpeg_video_h264_b_frame_qp, false);
+
+	v4l2_ctrl_grab(channel->mpeg_video_hevc_profile, false);
+	v4l2_ctrl_grab(channel->mpeg_video_hevc_level, false);
+	v4l2_ctrl_grab(channel->mpeg_video_hevc_tier, false);
+	v4l2_ctrl_grab(channel->mpeg_video_hevc_i_frame_qp, false);
+	v4l2_ctrl_grab(channel->mpeg_video_hevc_max_qp, false);
+	v4l2_ctrl_grab(channel->mpeg_video_hevc_min_qp, false);
+	v4l2_ctrl_grab(channel->mpeg_video_hevc_p_frame_qp, false);
+	v4l2_ctrl_grab(channel->mpeg_video_hevc_b_frame_qp, false);
+
 	v4l2_ctrl_grab(channel->mpeg_video_frame_rc_enable, false);
 	v4l2_ctrl_grab(channel->mpeg_video_bitrate_mode, false);
 	v4l2_ctrl_grab(channel->mpeg_video_bitrate, false);
@@ -2067,6 +2433,16 @@ static int allegro_create_channel(struct allegro_channel *channel)
 	v4l2_ctrl_grab(channel->mpeg_video_h264_min_qp, true);
 	v4l2_ctrl_grab(channel->mpeg_video_h264_p_frame_qp, true);
 	v4l2_ctrl_grab(channel->mpeg_video_h264_b_frame_qp, true);
+
+	v4l2_ctrl_grab(channel->mpeg_video_hevc_profile, true);
+	v4l2_ctrl_grab(channel->mpeg_video_hevc_level, true);
+	v4l2_ctrl_grab(channel->mpeg_video_hevc_tier, true);
+	v4l2_ctrl_grab(channel->mpeg_video_hevc_i_frame_qp, true);
+	v4l2_ctrl_grab(channel->mpeg_video_hevc_max_qp, true);
+	v4l2_ctrl_grab(channel->mpeg_video_hevc_min_qp, true);
+	v4l2_ctrl_grab(channel->mpeg_video_hevc_p_frame_qp, true);
+	v4l2_ctrl_grab(channel->mpeg_video_hevc_b_frame_qp, true);
+
 	v4l2_ctrl_grab(channel->mpeg_video_frame_rc_enable, true);
 	v4l2_ctrl_grab(channel->mpeg_video_bitrate_mode, true);
 	v4l2_ctrl_grab(channel->mpeg_video_bitrate, true);
@@ -2113,8 +2489,13 @@ static void allegro_channel_adjust(struct allegro_channel *channel)
 	channel->sizeimage_encoded =
 		estimate_stream_size(channel->width, channel->height);
 
-	ctrl = channel->mpeg_video_h264_level;
-	min = select_minimum_h264_level(channel->width, channel->height);
+	if (codec == V4L2_PIX_FMT_H264) {
+		ctrl = channel->mpeg_video_h264_level;
+		min = select_minimum_h264_level(channel->width, channel->height);
+	} else {
+		ctrl = channel->mpeg_video_hevc_level;
+		min = select_minimum_hevc_level(channel->width, channel->height);
+	}
 	if (ctrl->minimum > min)
 		v4l2_dbg(1, debug, &dev->v4l2_dev,
 			 "%s.minimum: %lld -> %lld\n",
@@ -2125,7 +2506,10 @@ static void allegro_channel_adjust(struct allegro_channel *channel)
 	v4l2_ctrl_unlock(ctrl);
 
 	ctrl = channel->mpeg_video_bitrate;
-	max = h264_maximum_bitrate(v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level));
+	if (codec == V4L2_PIX_FMT_H264)
+		max = h264_maximum_bitrate(v4l2_ctrl_g_ctrl(channel->mpeg_video_h264_level));
+	else
+		max = hevc_maximum_bitrate(v4l2_ctrl_g_ctrl(channel->mpeg_video_hevc_level));
 	if (ctrl->maximum < max)
 		v4l2_dbg(1, debug, &dev->v4l2_dev,
 			 "%s: maximum: %lld -> %lld\n",
@@ -2156,21 +2540,51 @@ static void allegro_channel_adjust(struct allegro_channel *channel)
 	v4l2_ctrl_activate(channel->mpeg_video_h264_b_frame_qp,
 			   codec == V4L2_PIX_FMT_H264);
 
-	channel->log2_max_frame_num = LOG2_MAX_FRAME_NUM;
+	v4l2_ctrl_activate(channel->mpeg_video_hevc_profile,
+			   codec == V4L2_PIX_FMT_HEVC);
+	v4l2_ctrl_activate(channel->mpeg_video_hevc_level,
+			   codec == V4L2_PIX_FMT_HEVC);
+	v4l2_ctrl_activate(channel->mpeg_video_hevc_tier,
+			   codec == V4L2_PIX_FMT_HEVC);
+	v4l2_ctrl_activate(channel->mpeg_video_hevc_i_frame_qp,
+			   codec == V4L2_PIX_FMT_HEVC);
+	v4l2_ctrl_activate(channel->mpeg_video_hevc_max_qp,
+			   codec == V4L2_PIX_FMT_HEVC);
+	v4l2_ctrl_activate(channel->mpeg_video_hevc_min_qp,
+			   codec == V4L2_PIX_FMT_HEVC);
+	v4l2_ctrl_activate(channel->mpeg_video_hevc_p_frame_qp,
+			   codec == V4L2_PIX_FMT_HEVC);
+	v4l2_ctrl_activate(channel->mpeg_video_hevc_b_frame_qp,
+			   codec == V4L2_PIX_FMT_HEVC);
+
+	if (codec == V4L2_PIX_FMT_H264)
+		channel->log2_max_frame_num = LOG2_MAX_FRAME_NUM;
 	channel->temporal_mvp_enable = true;
-
-	channel->dbf_ovr_en = true;
+	channel->dbf_ovr_en = (codec == V4L2_PIX_FMT_H264);
+	channel->enable_deblocking_filter_override = (codec == V4L2_PIX_FMT_HEVC);
+	channel->enable_reordering = (codec == V4L2_PIX_FMT_HEVC);
 	channel->enable_loop_filter_across_tiles = true;
 	channel->enable_loop_filter_across_slices = true;
 
-	channel->b_hrz_me_range = 8;
-	channel->b_vrt_me_range = 8;
-	channel->p_hrz_me_range = 16;
-	channel->p_vrt_me_range = 16;
-	channel->max_cu_size = ilog2(16);
-	channel->min_cu_size = ilog2(8);
-	channel->max_tu_size = ilog2(4);
-	channel->min_tu_size = ilog2(4);
+	if (codec == V4L2_PIX_FMT_H264) {
+		channel->b_hrz_me_range = 8;
+		channel->b_vrt_me_range = 8;
+		channel->p_hrz_me_range = 16;
+		channel->p_vrt_me_range = 16;
+		channel->max_cu_size = ilog2(16);
+		channel->min_cu_size = ilog2(8);
+		channel->max_tu_size = ilog2(4);
+		channel->min_tu_size = ilog2(4);
+	} else {
+		channel->b_hrz_me_range = 16;
+		channel->b_vrt_me_range = 16;
+		channel->p_hrz_me_range = 32;
+		channel->p_vrt_me_range = 32;
+		channel->max_cu_size = ilog2(32);
+		channel->min_cu_size = ilog2(8);
+		channel->max_tu_size = ilog2(32);
+		channel->min_tu_size = ilog2(4);
+	}
 	channel->max_transfo_depth_intra = 1;
 	channel->max_transfo_depth_inter = 1;
 }
@@ -2525,6 +2939,51 @@ static int allegro_open(struct file *file)
 				  &allegro_ctrl_ops,
 				  V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP,
 				  0, 51, 1, 30);
+
+	channel->mpeg_video_hevc_profile =
+		v4l2_ctrl_new_std_menu(handler,
+				       &allegro_ctrl_ops,
+				       V4L2_CID_MPEG_VIDEO_HEVC_PROFILE,
+				       V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN, 0x0,
+				       V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN);
+	channel->mpeg_video_hevc_level =
+		v4l2_ctrl_new_std_menu(handler,
+				       &allegro_ctrl_ops,
+				       V4L2_CID_MPEG_VIDEO_HEVC_LEVEL,
+				       V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1, 0x0,
+				       V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1);
+	channel->mpeg_video_hevc_tier =
+		v4l2_ctrl_new_std_menu(handler,
+				       &allegro_ctrl_ops,
+				       V4L2_CID_MPEG_VIDEO_HEVC_TIER,
+				       V4L2_MPEG_VIDEO_HEVC_TIER_HIGH, 0x0,
+				       V4L2_MPEG_VIDEO_HEVC_TIER_MAIN);
+	channel->mpeg_video_hevc_i_frame_qp =
+		v4l2_ctrl_new_std(handler,
+				  &allegro_ctrl_ops,
+				  V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_QP,
+				  0, 51, 1, 30);
+	channel->mpeg_video_hevc_max_qp =
+		v4l2_ctrl_new_std(handler,
+				  &allegro_ctrl_ops,
+				  V4L2_CID_MPEG_VIDEO_HEVC_MAX_QP,
+				  0, 51, 1, 51);
+	channel->mpeg_video_hevc_min_qp =
+		v4l2_ctrl_new_std(handler,
+				  &allegro_ctrl_ops,
+				  V4L2_CID_MPEG_VIDEO_HEVC_MIN_QP,
+				  0, 51, 1, 0);
+	channel->mpeg_video_hevc_p_frame_qp =
+		v4l2_ctrl_new_std(handler,
+				  &allegro_ctrl_ops,
+				  V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_QP,
+				  0, 51, 1, 30);
+	channel->mpeg_video_hevc_b_frame_qp =
+		v4l2_ctrl_new_std(handler,
+				  &allegro_ctrl_ops,
+				  V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP,
+				  0, 51, 1, 30);
+
 	channel->mpeg_video_frame_rc_enable =
 		v4l2_ctrl_new_std(handler,
 				  &allegro_ctrl_ops,
@@ -2537,10 +2996,17 @@ static int allegro_open(struct file *file)
 			V4L2_MPEG_VIDEO_BITRATE_MODE_CBR, 0,
 			V4L2_MPEG_VIDEO_BITRATE_MODE_CBR);
 
-	bitrate_max = h264_maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
-	bitrate_def = h264_maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
-	cpb_size_max = h264_maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
-	cpb_size_def = h264_maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
+	if (channel->codec == V4L2_PIX_FMT_H264) {
+		bitrate_max = h264_maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
+		bitrate_def = h264_maximum_bitrate(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
+		cpb_size_max = h264_maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
+		cpb_size_def = h264_maximum_cpb_size(V4L2_MPEG_VIDEO_H264_LEVEL_5_1);
+	} else {
+		bitrate_max = hevc_maximum_bitrate(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1);
+		bitrate_def = hevc_maximum_bitrate(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1);
+		cpb_size_max = hevc_maximum_cpb_size(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1);
+		cpb_size_def = hevc_maximum_cpb_size(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1);
+	}
 	channel->mpeg_video_bitrate = v4l2_ctrl_new_std(handler,
 			&allegro_ctrl_ops,
 			V4L2_CID_MPEG_VIDEO_BITRATE,
@@ -2645,9 +3111,12 @@ static int allegro_enum_fmt_vid(struct file *file, void *fh,
 		f->pixelformat = V4L2_PIX_FMT_NV12;
 		break;
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE:
-		if (f->index >= 1)
+		if (f->index >= 2)
 			return -EINVAL;
-		f->pixelformat = V4L2_PIX_FMT_H264;
+		if (f->index == 0)
+			f->pixelformat = V4L2_PIX_FMT_H264;
+		if (f->index == 1)
+			f->pixelformat = V4L2_PIX_FMT_HEVC;
 		break;
 	default:
 		return -EINVAL;
@@ -2686,7 +3155,10 @@ static int allegro_try_fmt_vid_cap(struct file *file, void *fh,
 	f->fmt.pix.height = clamp_t(__u32, f->fmt.pix.height,
 				    ALLEGRO_HEIGHT_MIN, ALLEGRO_HEIGHT_MAX);
 
-	f->fmt.pix.pixelformat = V4L2_PIX_FMT_H264;
+	if (f->fmt.pix.pixelformat != V4L2_PIX_FMT_HEVC &&
+	    f->fmt.pix.pixelformat != V4L2_PIX_FMT_H264)
+		f->fmt.pix.pixelformat = V4L2_PIX_FMT_H264;
+
 	f->fmt.pix.bytesperline = 0;
 	f->fmt.pix.sizeimage =
 		estimate_stream_size(f->fmt.pix.width, f->fmt.pix.height);
@@ -2826,6 +3298,7 @@ static int allegro_enum_framesizes(struct file *file, void *fh,
 				   struct v4l2_frmsizeenum *fsize)
 {
 	switch (fsize->pixel_format) {
+	case V4L2_PIX_FMT_HEVC:
 	case V4L2_PIX_FMT_H264:
 	case V4L2_PIX_FMT_NV12:
 		break;
diff --git a/drivers/media/platform/allegro-dvt/allegro-mail.c b/drivers/media/platform/allegro-dvt/allegro-mail.c
index 5dbc1c029020..7e08c5050f2e 100644
--- a/drivers/media/platform/allegro-dvt/allegro-mail.c
+++ b/drivers/media/platform/allegro-dvt/allegro-mail.c
@@ -67,12 +67,16 @@ static inline u32 settings_get_mcu_codec(struct create_channel_param *param)
 
 	if (version < MCU_MSG_VERSION_2019_2) {
 		switch (pixelformat) {
+		case V4L2_PIX_FMT_HEVC:
+			return 2;
 		case V4L2_PIX_FMT_H264:
 		default:
 			return 1;
 		}
 	} else {
 		switch (pixelformat) {
+		case V4L2_PIX_FMT_HEVC:
+			return 1;
 		case V4L2_PIX_FMT_H264:
 		default:
 			return 0;
@@ -117,7 +121,9 @@ allegro_encode_config_blob(u32 *dst, struct create_channel_param *param)
 	dst[i++] = val;
 
 	val = 0;
+	val |= param->enable_reordering ? BIT(0) : 0;
 	val |= param->dbf_ovr_en ? BIT(2) : 0;
+	val |= param->override_lf ? BIT(12) : 0;
 	dst[i++] = val;
 
 	if (version >= MCU_MSG_VERSION_2019_2) {
diff --git a/drivers/media/platform/allegro-dvt/allegro-mail.h b/drivers/media/platform/allegro-dvt/allegro-mail.h
index f7485cf78c4f..2c7bc509eac3 100644
--- a/drivers/media/platform/allegro-dvt/allegro-mail.h
+++ b/drivers/media/platform/allegro-dvt/allegro-mail.h
@@ -65,6 +65,7 @@ struct create_channel_param {
 	u32 temporal_mvp_enable;
 	u32 enable_reordering;
 	u32 dbf_ovr_en;
+	u32 override_lf;
 	u32 num_ref_idx_l0;
 	u32 num_ref_idx_l1;
 	u32 custom_lda;
-- 
2.20.1

Re: [PATCH 04/18] media: allegro: implement S_FMT for CAPTURE

[Hans Verkuil]

On 03/12/2020 12:00, Michael Tretter wrote:
> In order to support different codecs, the driver must support changing
> the format on CAPTURE. Therefore, the driver needs to handle S_FMT on
> CAPTURE.
> 
> As the driver will have a different number of formats for OUTPUT and
> CAPTURE, split the check for the format index in ENUM_FMT into CAPTURE
> and OUTPUT.
> 
> Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
> ---
>  .../media/platform/allegro-dvt/allegro-core.c | 23 ++++++++++++++++---
>  1 file changed, 20 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c
> index 6b5cbee05040..a733049c8727 100644
> --- a/drivers/media/platform/allegro-dvt/allegro-core.c
> +++ b/drivers/media/platform/allegro-dvt/allegro-core.c
> @@ -2503,13 +2503,15 @@ static int allegro_querycap(struct file *file, void *fh,
>  static int allegro_enum_fmt_vid(struct file *file, void *fh,
>  				struct v4l2_fmtdesc *f)
>  {
> -	if (f->index)
> -		return -EINVAL;
>  	switch (f->type) {
>  	case V4L2_BUF_TYPE_VIDEO_OUTPUT:
> +		if (f->index >= 1)
> +			return -EINVAL;
>  		f->pixelformat = V4L2_PIX_FMT_NV12;
>  		break;
>  	case V4L2_BUF_TYPE_VIDEO_CAPTURE:
> +		if (f->index >= 1)
> +			return -EINVAL;
>  		f->pixelformat = V4L2_PIX_FMT_H264;
>  		break;
>  	default:
> @@ -2557,6 +2559,21 @@ static int allegro_try_fmt_vid_cap(struct file *file, void *fh,
>  	return 0;
>  }
>  
> +static int allegro_s_fmt_vid_cap(struct file *file, void *fh,
> +				 struct v4l2_format *f)
> +{
> +	struct allegro_channel *channel = fh_to_channel(fh);
> +	int err;
> +
> +	err = allegro_try_fmt_vid_cap(file, fh, f);
> +	if (err)
> +		return err;

This needs an additional check:

	struct v4l2_m2m_ctx *m2m_ctx = channel->fh.m2m_ctx;
	struct vb2_queue vq = v4l2_m2m_get_vq(m2m_ctx, f->type);
	if (vb2_is_busy(vq))
		return -EBUSY;

This wasn't necessary before since there was only one possible pixelformat,
but now there are two (H264 and HEVC), so you need to prevent switching
codec while the queue is busy.

Regards,

	Hans

> +
> +	channel->codec = f->fmt.pix.pixelformat;
> +
> +	return 0;
> +}
> +
>  static int allegro_g_fmt_vid_out(struct file *file, void *fh,
>  				 struct v4l2_format *f)
>  {
> @@ -2769,7 +2786,7 @@ static const struct v4l2_ioctl_ops allegro_ioctl_ops = {
>  	.vidioc_enum_fmt_vid_out = allegro_enum_fmt_vid,
>  	.vidioc_g_fmt_vid_cap = allegro_g_fmt_vid_cap,
>  	.vidioc_try_fmt_vid_cap = allegro_try_fmt_vid_cap,
> -	.vidioc_s_fmt_vid_cap = allegro_try_fmt_vid_cap,
> +	.vidioc_s_fmt_vid_cap = allegro_s_fmt_vid_cap,
>  	.vidioc_g_fmt_vid_out = allegro_g_fmt_vid_out,
>  	.vidioc_try_fmt_vid_out = allegro_try_fmt_vid_out,
>  	.vidioc_s_fmt_vid_out = allegro_s_fmt_vid_out,
> 

[PATCH v2 04/18] media: allegro: implement S_FMT for CAPTURE

[Michael Tretter]

In order to support different codecs, the driver must support changing
the format on CAPTURE. Therefore, the driver needs to handle S_FMT on
CAPTURE.

As the driver will have a different number of formats for OUTPUT and
CAPTURE, split the check for the format index in ENUM_FMT into CAPTURE
and OUTPUT.

Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
---
Changelog:

v2:
- Add check for busy queue when setting the format

 .../media/platform/allegro-dvt/allegro-core.c | 30 +++++++++++++++++--
 1 file changed, 27 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/allegro-dvt/allegro-core.c b/drivers/media/platform/allegro-dvt/allegro-core.c
index 6b5cbee05040..fb3269cc4789 100644
--- a/drivers/media/platform/allegro-dvt/allegro-core.c
+++ b/drivers/media/platform/allegro-dvt/allegro-core.c
@@ -2503,13 +2503,15 @@ static int allegro_querycap(struct file *file, void *fh,
 static int allegro_enum_fmt_vid(struct file *file, void *fh,
 				struct v4l2_fmtdesc *f)
 {
-	if (f->index)
-		return -EINVAL;
 	switch (f->type) {
 	case V4L2_BUF_TYPE_VIDEO_OUTPUT:
+		if (f->index >= 1)
+			return -EINVAL;
 		f->pixelformat = V4L2_PIX_FMT_NV12;
 		break;
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE:
+		if (f->index >= 1)
+			return -EINVAL;
 		f->pixelformat = V4L2_PIX_FMT_H264;
 		break;
 	default:
@@ -2557,6 +2559,28 @@ static int allegro_try_fmt_vid_cap(struct file *file, void *fh,
 	return 0;
 }
 
+static int allegro_s_fmt_vid_cap(struct file *file, void *fh,
+				 struct v4l2_format *f)
+{
+	struct allegro_channel *channel = fh_to_channel(fh);
+	struct vb2_queue *vq;
+	int err;
+
+	err = allegro_try_fmt_vid_cap(file, fh, f);
+	if (err)
+		return err;
+
+	vq = v4l2_m2m_get_vq(channel->fh.m2m_ctx, f->type);
+	if (!vq)
+		return -EINVAL;
+	if (vb2_is_busy(vq))
+		return -EBUSY;
+
+	channel->codec = f->fmt.pix.pixelformat;
+
+	return 0;
+}
+
 static int allegro_g_fmt_vid_out(struct file *file, void *fh,
 				 struct v4l2_format *f)
 {
@@ -2769,7 +2793,7 @@ static const struct v4l2_ioctl_ops allegro_ioctl_ops = {
 	.vidioc_enum_fmt_vid_out = allegro_enum_fmt_vid,
 	.vidioc_g_fmt_vid_cap = allegro_g_fmt_vid_cap,
 	.vidioc_try_fmt_vid_cap = allegro_try_fmt_vid_cap,
-	.vidioc_s_fmt_vid_cap = allegro_try_fmt_vid_cap,
+	.vidioc_s_fmt_vid_cap = allegro_s_fmt_vid_cap,
 	.vidioc_g_fmt_vid_out = allegro_g_fmt_vid_out,
 	.vidioc_try_fmt_vid_out = allegro_try_fmt_vid_out,
 	.vidioc_s_fmt_vid_out = allegro_s_fmt_vid_out,
-- 
2.20.1