All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCHv2 0/7] vicodec improvements
@ 2018-08-23  7:32 Hans Verkuil
  2018-08-23  7:32 ` [PATCHv2 1/7] vicodec: add QP controls Hans Verkuil
                   ` (7 more replies)
  0 siblings, 8 replies; 10+ messages in thread
From: Hans Verkuil @ 2018-08-23  7:32 UTC (permalink / raw)
  To: linux-media

From: Hans Verkuil <hans.verkuil@cisco.com>

- add support for quantization parameters
- support many more pixel formats
- code simplifications
- rename source and use proper prefixes for the codec: this makes it
  independent from the vicodec driver and easier to reuse in userspace
  (similar to what we do for the v4l2-tpg code).
- split off the v4l2 'frontend' code for the FWHT codec into its own
  source for easier re-use elsewhere (i.e. v4l2-ctl/qvidcap).

I made a v4l-utils branch that uses the FWHT codec to compress video
when streaming over the network:

https://git.linuxtv.org/hverkuil/v4l-utils.git/log/?h=qvidcap

You need to add the --stream-to-host-lossy flag to enable FWHT streaming.

Note: the FWHT codec clips R/G/B values for RGB formats. This will be
addressed later. I might have to convert the R/G/B values from full to
limited range before encoding them, but I want to discuss this with the
author of the codec (Tom aan de Wiel) first.

Regards,

	Hans

Changes since v1:

- added the last patch (split off v4l2 FWHT code)
- the GOP_SIZE and QP controls can now be set during streaming as
  well.

Hans Verkuil (7):
  vicodec: add QP controls
  vicodec: add support for more pixel formats
  vicodec: simplify flags handling
  vicodec: simplify blocktype checking
  vicodec: improve handling of uncompressable planes
  vicodec: rename and use proper fwht prefix for codec
  vicodec: split off v4l2 specific parts for the codec

 .../media/uapi/v4l/pixfmt-compressed.rst      |   2 +-
 drivers/media/platform/vicodec/Makefile       |   2 +-
 .../vicodec/{vicodec-codec.c => codec-fwht.c} | 148 ++++--
 .../vicodec/{vicodec-codec.h => codec-fwht.h} |  80 ++-
 .../media/platform/vicodec/codec-v4l2-fwht.c  | 325 ++++++++++++
 .../media/platform/vicodec/codec-v4l2-fwht.h  |  50 ++
 drivers/media/platform/vicodec/vicodec-core.c | 483 ++++++++----------
 7 files changed, 723 insertions(+), 367 deletions(-)
 rename drivers/media/platform/vicodec/{vicodec-codec.c => codec-fwht.c} (85%)
 rename drivers/media/platform/vicodec/{vicodec-codec.h => codec-fwht.h} (64%)
 create mode 100644 drivers/media/platform/vicodec/codec-v4l2-fwht.c
 create mode 100644 drivers/media/platform/vicodec/codec-v4l2-fwht.h

-- 
2.18.0

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [PATCHv2 1/7] vicodec: add QP controls
  2018-08-23  7:32 [PATCHv2 0/7] vicodec improvements Hans Verkuil
@ 2018-08-23  7:32 ` Hans Verkuil
  2018-08-23  7:33 ` [PATCHv2 2/7] vicodec: add support for more pixel formats Hans Verkuil
                   ` (6 subsequent siblings)
  7 siblings, 0 replies; 10+ messages in thread
From: Hans Verkuil @ 2018-08-23  7:32 UTC (permalink / raw)
  To: linux-media; +Cc: Hans Verkuil

From: Hans Verkuil <hans.verkuil@cisco.com>

Instead of hardcoding the quantization parameter (or 'DEADZONE_WIDTH'
as it was called in the codec) make this configurable through two
controls: one for I frames, one for P frames.

Also allow changing these parameters and the GOP_SIZE parameter while
streaming.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 .../media/platform/vicodec/vicodec-codec.c    | 17 +++--
 .../media/platform/vicodec/vicodec-codec.h    |  2 +
 drivers/media/platform/vicodec/vicodec-core.c | 66 ++++++++++++++++---
 3 files changed, 68 insertions(+), 17 deletions(-)

diff --git a/drivers/media/platform/vicodec/vicodec-codec.c b/drivers/media/platform/vicodec/vicodec-codec.c
index 2d047646f614..7163f11b7ee8 100644
--- a/drivers/media/platform/vicodec/vicodec-codec.c
+++ b/drivers/media/platform/vicodec/vicodec-codec.c
@@ -13,7 +13,6 @@
 #include "vicodec-codec.h"
 
 #define ALL_ZEROS 15
-#define DEADZONE_WIDTH 20
 
 static const uint8_t zigzag[64] = {
 	0,
@@ -164,7 +163,7 @@ static const int quant_table_p[] = {
 	3, 3, 3, 6, 6, 9,  9,  10,
 };
 
-static void quantize_intra(s16 *coeff, s16 *de_coeff)
+static void quantize_intra(s16 *coeff, s16 *de_coeff, u16 qp)
 {
 	const int *quant = quant_table;
 	int i, j;
@@ -172,8 +171,7 @@ static void quantize_intra(s16 *coeff, s16 *de_coeff)
 	for (j = 0; j < 8; j++) {
 		for (i = 0; i < 8; i++, quant++, coeff++, de_coeff++) {
 			*coeff >>= *quant;
-			if (*coeff >= -DEADZONE_WIDTH &&
-			    *coeff <= DEADZONE_WIDTH)
+			if (*coeff >= -qp && *coeff <= qp)
 				*coeff = *de_coeff = 0;
 			else
 				*de_coeff = *coeff << *quant;
@@ -191,7 +189,7 @@ static void dequantize_intra(s16 *coeff)
 			*coeff <<= *quant;
 }
 
-static void quantize_inter(s16 *coeff, s16 *de_coeff)
+static void quantize_inter(s16 *coeff, s16 *de_coeff, u16 qp)
 {
 	const int *quant = quant_table_p;
 	int i, j;
@@ -199,8 +197,7 @@ static void quantize_inter(s16 *coeff, s16 *de_coeff)
 	for (j = 0; j < 8; j++) {
 		for (i = 0; i < 8; i++, quant++, coeff++, de_coeff++) {
 			*coeff >>= *quant;
-			if (*coeff >= -DEADZONE_WIDTH &&
-			    *coeff <= DEADZONE_WIDTH)
+			if (*coeff >= -qp && *coeff <= qp)
 				*coeff = *de_coeff = 0;
 			else
 				*de_coeff = *coeff << *quant;
@@ -639,13 +636,15 @@ static u32 encode_plane(u8 *input, u8 *refp, __be16 **rlco, __be16 *rlco_max,
 					deltablock, width, input_step);
 			if (is_intra || blocktype == IBLOCK) {
 				fwht(input, cf->coeffs, width, input_step, 1);
-				quantize_intra(cf->coeffs, cf->de_coeffs);
+				quantize_intra(cf->coeffs, cf->de_coeffs,
+					       cf->i_frame_qp);
 				blocktype = IBLOCK;
 			} else {
 				/* inter code */
 				encoding |= FRAME_PCODED;
 				fwht16(deltablock, cf->coeffs, 8, 0);
-				quantize_inter(cf->coeffs, cf->de_coeffs);
+				quantize_inter(cf->coeffs, cf->de_coeffs,
+					       cf->p_frame_qp);
 			}
 			if (!next_is_intra) {
 				ifwht(cf->de_coeffs, cf->de_fwht, blocktype);
diff --git a/drivers/media/platform/vicodec/vicodec-codec.h b/drivers/media/platform/vicodec/vicodec-codec.h
index cdfad1332a3e..cabe7b98623b 100644
--- a/drivers/media/platform/vicodec/vicodec-codec.h
+++ b/drivers/media/platform/vicodec/vicodec-codec.h
@@ -103,6 +103,8 @@ struct cframe_hdr {
 
 struct cframe {
 	unsigned int width, height;
+	u16 i_frame_qp;
+	u16 p_frame_qp;
 	__be16 *rlc_data;
 	s16 coeffs[8 * 8];
 	s16 de_coeffs[8 * 8];
diff --git a/drivers/media/platform/vicodec/vicodec-core.c b/drivers/media/platform/vicodec/vicodec-core.c
index 408cd55d3580..1f14e94e61b4 100644
--- a/drivers/media/platform/vicodec/vicodec-core.c
+++ b/drivers/media/platform/vicodec/vicodec-core.c
@@ -96,9 +96,10 @@ struct vicodec_ctx {
 	spinlock_t		*lock;
 
 	struct v4l2_ctrl_handler hdl;
-	struct v4l2_ctrl	*ctrl_gop_size;
 	unsigned int		gop_size;
 	unsigned int		gop_cnt;
+	u16			i_frame_qp;
+	u16			p_frame_qp;
 
 	/* Abort requested by m2m */
 	int			aborting;
@@ -191,13 +192,15 @@ static void encode(struct vicodec_ctx *ctx,
 
 	cf.width = q_data->width;
 	cf.height = q_data->height;
+	cf.i_frame_qp = ctx->i_frame_qp;
+	cf.p_frame_qp = ctx->p_frame_qp;
 	cf.rlc_data = (__be16 *)(p_out + sizeof(*p_hdr));
 
 	encoding = encode_frame(&rf, &ctx->ref_frame, &cf, !ctx->gop_cnt,
 				ctx->gop_cnt == ctx->gop_size - 1);
 	if (encoding != FRAME_PCODED)
 		ctx->gop_cnt = 0;
-	if (++ctx->gop_cnt == ctx->gop_size)
+	if (++ctx->gop_cnt >= ctx->gop_size)
 		ctx->gop_cnt = 0;
 
 	p_hdr = (struct cframe_hdr *)p_out;
@@ -1140,8 +1143,6 @@ static int vicodec_start_streaming(struct vb2_queue *q,
 	ctx->ref_frame.cr = ctx->ref_frame.cb + size / 4;
 	ctx->last_src_buf = NULL;
 	ctx->last_dst_buf = NULL;
-	v4l2_ctrl_grab(ctx->ctrl_gop_size, true);
-	ctx->gop_size = v4l2_ctrl_g_ctrl(ctx->ctrl_gop_size);
 	ctx->gop_cnt = 0;
 	ctx->cur_buf_offset = 0;
 	ctx->comp_size = 0;
@@ -1162,7 +1163,6 @@ static void vicodec_stop_streaming(struct vb2_queue *q)
 
 	kvfree(ctx->ref_frame.luma);
 	kvfree(ctx->compressed_frame);
-	v4l2_ctrl_grab(ctx->ctrl_gop_size, false);
 }
 
 static const struct vb2_ops vicodec_qops = {
@@ -1211,6 +1211,55 @@ static int queue_init(void *priv, struct vb2_queue *src_vq,
 	return vb2_queue_init(dst_vq);
 }
 
+#define VICODEC_CID_CUSTOM_BASE		(V4L2_CID_MPEG_BASE | 0xf000)
+#define VICODEC_CID_I_FRAME_QP		(VICODEC_CID_CUSTOM_BASE + 0)
+#define VICODEC_CID_P_FRAME_QP		(VICODEC_CID_CUSTOM_BASE + 1)
+
+static int vicodec_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct vicodec_ctx *ctx = container_of(ctrl->handler,
+					       struct vicodec_ctx, hdl);
+
+	switch (ctrl->id) {
+	case V4L2_CID_MPEG_VIDEO_GOP_SIZE:
+		ctx->gop_size = ctrl->val;
+		return 0;
+	case VICODEC_CID_I_FRAME_QP:
+		ctx->i_frame_qp = ctrl->val;
+		return 0;
+	case VICODEC_CID_P_FRAME_QP:
+		ctx->p_frame_qp = ctrl->val;
+		return 0;
+	}
+	return -EINVAL;
+}
+
+static const struct v4l2_ctrl_ops vicodec_ctrl_ops = {
+	.s_ctrl = vicodec_s_ctrl,
+};
+
+static const struct v4l2_ctrl_config vicodec_ctrl_i_frame = {
+	.ops = &vicodec_ctrl_ops,
+	.id = VICODEC_CID_I_FRAME_QP,
+	.name = "FWHT I-Frame QP Value",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = 1,
+	.max = 31,
+	.def = 20,
+	.step = 1,
+};
+
+static const struct v4l2_ctrl_config vicodec_ctrl_p_frame = {
+	.ops = &vicodec_ctrl_ops,
+	.id = VICODEC_CID_P_FRAME_QP,
+	.name = "FWHT P-Frame QP Value",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = 1,
+	.max = 31,
+	.def = 20,
+	.step = 1,
+};
+
 /*
  * File operations
  */
@@ -1239,9 +1288,10 @@ static int vicodec_open(struct file *file)
 	ctx->dev = dev;
 	hdl = &ctx->hdl;
 	v4l2_ctrl_handler_init(hdl, 4);
-	ctx->ctrl_gop_size = v4l2_ctrl_new_std(hdl, NULL,
-					       V4L2_CID_MPEG_VIDEO_GOP_SIZE,
-					       1, 16, 1, 10);
+	v4l2_ctrl_new_std(hdl, &vicodec_ctrl_ops, V4L2_CID_MPEG_VIDEO_GOP_SIZE,
+			  1, 16, 1, 10);
+	v4l2_ctrl_new_custom(hdl, &vicodec_ctrl_i_frame, NULL);
+	v4l2_ctrl_new_custom(hdl, &vicodec_ctrl_p_frame, NULL);
 	if (hdl->error) {
 		rc = hdl->error;
 		v4l2_ctrl_handler_free(hdl);
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCHv2 2/7] vicodec: add support for more pixel formats
  2018-08-23  7:32 [PATCHv2 0/7] vicodec improvements Hans Verkuil
  2018-08-23  7:32 ` [PATCHv2 1/7] vicodec: add QP controls Hans Verkuil
@ 2018-08-23  7:33 ` Hans Verkuil
  2018-08-23  7:33 ` [PATCHv2 3/7] vicodec: simplify flags handling Hans Verkuil
                   ` (5 subsequent siblings)
  7 siblings, 0 replies; 10+ messages in thread
From: Hans Verkuil @ 2018-08-23  7:33 UTC (permalink / raw)
  To: linux-media; +Cc: Hans Verkuil

From: Hans Verkuil <hans.verkuil@cisco.com>

Add support for 4:2:2, 4:4:4 and RGB 24/32 bits formats.

This makes it a lot more useful, esp. as a simple video compression
codec for use with v4l2-ctl/qvidcap.

Note that it does not do any conversion between e.g. 4:2:2 and 4:2:0
or RGB and YUV: it still just compresses planes be they Y/U/V or R/G/B.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 .../media/platform/vicodec/vicodec-codec.c    |  62 ++-
 .../media/platform/vicodec/vicodec-codec.h    |   5 +
 drivers/media/platform/vicodec/vicodec-core.c | 357 +++++++++++++-----
 3 files changed, 324 insertions(+), 100 deletions(-)

diff --git a/drivers/media/platform/vicodec/vicodec-codec.c b/drivers/media/platform/vicodec/vicodec-codec.c
index 7163f11b7ee8..7bd11a974db0 100644
--- a/drivers/media/platform/vicodec/vicodec-codec.c
+++ b/drivers/media/platform/vicodec/vicodec-codec.c
@@ -229,7 +229,8 @@ static void fwht(const u8 *block, s16 *output_block, unsigned int stride,
 	stride *= input_step;
 
 	for (i = 0; i < 8; i++, tmp += stride, out += 8) {
-		if (input_step == 1) {
+		switch (input_step) {
+		case 1:
 			workspace1[0]  = tmp[0] + tmp[1] - add;
 			workspace1[1]  = tmp[0] - tmp[1];
 
@@ -241,7 +242,8 @@ static void fwht(const u8 *block, s16 *output_block, unsigned int stride,
 
 			workspace1[6]  = tmp[6] + tmp[7] - add;
 			workspace1[7]  = tmp[6] - tmp[7];
-		} else {
+			break;
+		case 2:
 			workspace1[0]  = tmp[0] + tmp[2] - add;
 			workspace1[1]  = tmp[0] - tmp[2];
 
@@ -253,6 +255,33 @@ static void fwht(const u8 *block, s16 *output_block, unsigned int stride,
 
 			workspace1[6]  = tmp[12] + tmp[14] - add;
 			workspace1[7]  = tmp[12] - tmp[14];
+			break;
+		case 3:
+			workspace1[0]  = tmp[0] + tmp[3] - add;
+			workspace1[1]  = tmp[0] - tmp[3];
+
+			workspace1[2]  = tmp[6] + tmp[9] - add;
+			workspace1[3]  = tmp[6] - tmp[9];
+
+			workspace1[4]  = tmp[12] + tmp[15] - add;
+			workspace1[5]  = tmp[12] - tmp[15];
+
+			workspace1[6]  = tmp[18] + tmp[21] - add;
+			workspace1[7]  = tmp[18] - tmp[21];
+			break;
+		default:
+			workspace1[0]  = tmp[0] + tmp[4] - add;
+			workspace1[1]  = tmp[0] - tmp[4];
+
+			workspace1[2]  = tmp[8] + tmp[12] - add;
+			workspace1[3]  = tmp[8] - tmp[12];
+
+			workspace1[4]  = tmp[16] + tmp[20] - add;
+			workspace1[5]  = tmp[16] - tmp[20];
+
+			workspace1[6]  = tmp[24] + tmp[28] - add;
+			workspace1[7]  = tmp[24] - tmp[28];
+			break;
 		}
 
 		/* stage 2 */
@@ -704,25 +733,28 @@ u32 encode_frame(struct raw_frame *frm, struct raw_frame *ref_frm,
 	__be16 *rlco = cf->rlc_data;
 	__be16 *rlco_max;
 	u32 encoding;
+	u32 chroma_h = frm->height / frm->height_div;
+	u32 chroma_w = frm->width / frm->width_div;
+	unsigned int chroma_size = chroma_h * chroma_w;
 
 	rlco_max = rlco + size / 2 - 256;
 	encoding = encode_plane(frm->luma, ref_frm->luma, &rlco, rlco_max, cf,
-				  frm->height, frm->width,
-				  1, is_intra, next_is_intra);
+				frm->height, frm->width,
+				frm->luma_step, is_intra, next_is_intra);
 	if (encoding & FRAME_UNENCODED)
 		encoding |= LUMA_UNENCODED;
 	encoding &= ~FRAME_UNENCODED;
-	rlco_max = rlco + size / 8 - 256;
+	rlco_max = rlco + chroma_size / 2 - 256;
 	encoding |= encode_plane(frm->cb, ref_frm->cb, &rlco, rlco_max, cf,
-				   frm->height / 2, frm->width / 2,
-				   frm->chroma_step, is_intra, next_is_intra);
+				 chroma_h, chroma_w,
+				 frm->chroma_step, is_intra, next_is_intra);
 	if (encoding & FRAME_UNENCODED)
 		encoding |= CB_UNENCODED;
 	encoding &= ~FRAME_UNENCODED;
-	rlco_max = rlco + size / 8 - 256;
+	rlco_max = rlco + chroma_size / 2 - 256;
 	encoding |= encode_plane(frm->cr, ref_frm->cr, &rlco, rlco_max, cf,
-				   frm->height / 2, frm->width / 2,
-				   frm->chroma_step, is_intra, next_is_intra);
+				 chroma_h, chroma_w,
+				 frm->chroma_step, is_intra, next_is_intra);
 	if (encoding & FRAME_UNENCODED)
 		encoding |= CR_UNENCODED;
 	encoding &= ~FRAME_UNENCODED;
@@ -786,11 +818,17 @@ static void decode_plane(struct cframe *cf, const __be16 **rlco, u8 *ref,
 void decode_frame(struct cframe *cf, struct raw_frame *ref, u32 hdr_flags)
 {
 	const __be16 *rlco = cf->rlc_data;
+	u32 h = cf->height / 2;
+	u32 w = cf->width / 2;
 
+	if (hdr_flags & VICODEC_FL_CHROMA_FULL_HEIGHT)
+		h *= 2;
+	if (hdr_flags & VICODEC_FL_CHROMA_FULL_WIDTH)
+		w *= 2;
 	decode_plane(cf, &rlco, ref->luma, cf->height, cf->width,
 		     hdr_flags & VICODEC_FL_LUMA_IS_UNCOMPRESSED);
-	decode_plane(cf, &rlco, ref->cb, cf->height / 2, cf->width / 2,
+	decode_plane(cf, &rlco, ref->cb, h, w,
 		     hdr_flags & VICODEC_FL_CB_IS_UNCOMPRESSED);
-	decode_plane(cf, &rlco, ref->cr, cf->height / 2, cf->width / 2,
+	decode_plane(cf, &rlco, ref->cr, h, w,
 		     hdr_flags & VICODEC_FL_CR_IS_UNCOMPRESSED);
 }
diff --git a/drivers/media/platform/vicodec/vicodec-codec.h b/drivers/media/platform/vicodec/vicodec-codec.h
index cabe7b98623b..ff69d9297ef4 100644
--- a/drivers/media/platform/vicodec/vicodec-codec.h
+++ b/drivers/media/platform/vicodec/vicodec-codec.h
@@ -87,6 +87,8 @@
 #define VICODEC_FL_LUMA_IS_UNCOMPRESSED	BIT(4)
 #define VICODEC_FL_CB_IS_UNCOMPRESSED	BIT(5)
 #define VICODEC_FL_CR_IS_UNCOMPRESSED	BIT(6)
+#define VICODEC_FL_CHROMA_FULL_HEIGHT	BIT(7)
+#define VICODEC_FL_CHROMA_FULL_WIDTH	BIT(8)
 
 struct cframe_hdr {
 	u32 magic1;
@@ -114,6 +116,9 @@ struct cframe {
 
 struct raw_frame {
 	unsigned int width, height;
+	unsigned int width_div;
+	unsigned int height_div;
+	unsigned int luma_step;
 	unsigned int chroma_step;
 	u8 *luma, *cb, *cr;
 };
diff --git a/drivers/media/platform/vicodec/vicodec-core.c b/drivers/media/platform/vicodec/vicodec-core.c
index 1f14e94e61b4..6253fd9e1d30 100644
--- a/drivers/media/platform/vicodec/vicodec-core.c
+++ b/drivers/media/platform/vicodec/vicodec-core.c
@@ -48,6 +48,46 @@ MODULE_PARM_DESC(debug, " activates debug info");
 	v4l2_dbg(1, debug, &dev->v4l2_dev, "%s: " fmt, __func__, ## arg)
 
 
+struct pixfmt_info {
+	u32 id;
+	unsigned int bytesperline_mult;
+	unsigned int sizeimage_mult;
+	unsigned int sizeimage_div;
+	unsigned int luma_step;
+	unsigned int chroma_step;
+	/* Chroma plane subsampling */
+	unsigned int width_div;
+	unsigned int height_div;
+};
+
+static const struct pixfmt_info pixfmts[] = {
+	{ V4L2_PIX_FMT_YUV420,  1, 3, 2, 1, 1, 2, 2 },
+	{ V4L2_PIX_FMT_YVU420,  1, 3, 2, 1, 1, 2, 2 },
+	{ V4L2_PIX_FMT_YUV422P, 1, 2, 1, 1, 1, 2, 1 },
+	{ V4L2_PIX_FMT_NV12,    1, 3, 2, 1, 2, 2, 2 },
+	{ V4L2_PIX_FMT_NV21,    1, 3, 2, 1, 2, 2, 2 },
+	{ V4L2_PIX_FMT_NV16,    1, 2, 1, 1, 2, 2, 1 },
+	{ V4L2_PIX_FMT_NV61,    1, 2, 1, 1, 2, 2, 1 },
+	{ V4L2_PIX_FMT_NV24,    1, 3, 1, 1, 2, 1, 1 },
+	{ V4L2_PIX_FMT_NV42,    1, 3, 1, 1, 2, 1, 1 },
+	{ V4L2_PIX_FMT_YUYV,    2, 2, 1, 2, 4, 2, 1 },
+	{ V4L2_PIX_FMT_YVYU,    2, 2, 1, 2, 4, 2, 1 },
+	{ V4L2_PIX_FMT_UYVY,    2, 2, 1, 2, 4, 2, 1 },
+	{ V4L2_PIX_FMT_VYUY,    2, 2, 1, 2, 4, 2, 1 },
+	{ V4L2_PIX_FMT_BGR24,   3, 3, 1, 3, 3, 1, 1 },
+	{ V4L2_PIX_FMT_RGB24,   3, 3, 1, 3, 3, 1, 1 },
+	{ V4L2_PIX_FMT_HSV24,   3, 3, 1, 3, 3, 1, 1 },
+	{ V4L2_PIX_FMT_BGR32,   4, 4, 1, 4, 4, 1, 1 },
+	{ V4L2_PIX_FMT_XBGR32,  4, 4, 1, 4, 4, 1, 1 },
+	{ V4L2_PIX_FMT_RGB32,   4, 4, 1, 4, 4, 1, 1 },
+	{ V4L2_PIX_FMT_XRGB32,  4, 4, 1, 4, 4, 1, 1 },
+	{ V4L2_PIX_FMT_HSV32,   4, 4, 1, 4, 4, 1, 1 },
+};
+
+static const struct pixfmt_info pixfmt_fwht = {
+	V4L2_PIX_FMT_FWHT, 0, 3, 1, 1, 1, 1, 1
+};
+
 static void vicodec_dev_release(struct device *dev)
 {
 }
@@ -64,7 +104,7 @@ struct vicodec_q_data {
 	unsigned int		flags;
 	unsigned int		sizeimage;
 	unsigned int		sequence;
-	u32			fourcc;
+	const struct pixfmt_info *info;
 };
 
 enum {
@@ -124,13 +164,6 @@ struct vicodec_ctx {
 	bool			comp_has_next_frame;
 };
 
-static const u32 pixfmts_yuv[] = {
-	V4L2_PIX_FMT_YUV420,
-	V4L2_PIX_FMT_YVU420,
-	V4L2_PIX_FMT_NV12,
-	V4L2_PIX_FMT_NV21,
-};
-
 static inline struct vicodec_ctx *file2ctx(struct file *file)
 {
 	return container_of(file->private_data, struct vicodec_ctx, fh);
@@ -158,6 +191,7 @@ static void encode(struct vicodec_ctx *ctx,
 		   u8 *p_in, u8 *p_out)
 {
 	unsigned int size = q_data->width * q_data->height;
+	const struct pixfmt_info *info = q_data->info;
 	struct cframe_hdr *p_hdr;
 	struct cframe cf;
 	struct raw_frame rf;
@@ -166,27 +200,77 @@ static void encode(struct vicodec_ctx *ctx,
 	rf.width = q_data->width;
 	rf.height = q_data->height;
 	rf.luma = p_in;
+	rf.width_div = info->width_div;
+	rf.height_div = info->height_div;
+	rf.luma_step = info->luma_step;
+	rf.chroma_step = info->chroma_step;
 
-	switch (q_data->fourcc) {
+	switch (info->id) {
 	case V4L2_PIX_FMT_YUV420:
 		rf.cb = rf.luma + size;
 		rf.cr = rf.cb + size / 4;
-		rf.chroma_step = 1;
 		break;
 	case V4L2_PIX_FMT_YVU420:
 		rf.cr = rf.luma + size;
 		rf.cb = rf.cr + size / 4;
-		rf.chroma_step = 1;
+		break;
+	case V4L2_PIX_FMT_YUV422P:
+		rf.cb = rf.luma + size;
+		rf.cr = rf.cb + size / 2;
 		break;
 	case V4L2_PIX_FMT_NV12:
+	case V4L2_PIX_FMT_NV16:
+	case V4L2_PIX_FMT_NV24:
 		rf.cb = rf.luma + size;
 		rf.cr = rf.cb + 1;
-		rf.chroma_step = 2;
 		break;
 	case V4L2_PIX_FMT_NV21:
+	case V4L2_PIX_FMT_NV61:
+	case V4L2_PIX_FMT_NV42:
 		rf.cr = rf.luma + size;
 		rf.cb = rf.cr + 1;
-		rf.chroma_step = 2;
+		break;
+	case V4L2_PIX_FMT_YUYV:
+		rf.cb = rf.luma + 1;
+		rf.cr = rf.cb + 2;
+		break;
+	case V4L2_PIX_FMT_YVYU:
+		rf.cr = rf.luma + 1;
+		rf.cb = rf.cr + 2;
+		break;
+	case V4L2_PIX_FMT_UYVY:
+		rf.cb = rf.luma;
+		rf.cr = rf.cb + 2;
+		rf.luma++;
+		break;
+	case V4L2_PIX_FMT_VYUY:
+		rf.cr = rf.luma;
+		rf.cb = rf.cr + 2;
+		rf.luma++;
+		break;
+	case V4L2_PIX_FMT_RGB24:
+	case V4L2_PIX_FMT_HSV24:
+		rf.cr = rf.luma;
+		rf.cb = rf.cr + 2;
+		rf.luma++;
+		break;
+	case V4L2_PIX_FMT_BGR24:
+		rf.cb = rf.luma;
+		rf.cr = rf.cb + 2;
+		rf.luma++;
+		break;
+	case V4L2_PIX_FMT_RGB32:
+	case V4L2_PIX_FMT_XRGB32:
+	case V4L2_PIX_FMT_HSV32:
+		rf.cr = rf.luma + 1;
+		rf.cb = rf.cr + 2;
+		rf.luma += 2;
+		break;
+	case V4L2_PIX_FMT_BGR32:
+	case V4L2_PIX_FMT_XBGR32:
+		rf.cb = rf.luma;
+		rf.cr = rf.cb + 2;
+		rf.luma++;
 		break;
 	}
 
@@ -216,6 +300,10 @@ static void encode(struct vicodec_ctx *ctx,
 		p_hdr->flags |= htonl(VICODEC_FL_CB_IS_UNCOMPRESSED);
 	if (encoding & CR_UNENCODED)
 		p_hdr->flags |= htonl(VICODEC_FL_CR_IS_UNCOMPRESSED);
+	if (rf.height_div == 1)
+		p_hdr->flags |= htonl(VICODEC_FL_CHROMA_FULL_HEIGHT);
+	if (rf.width_div == 1)
+		p_hdr->flags |= htonl(VICODEC_FL_CHROMA_FULL_WIDTH);
 	p_hdr->colorspace = htonl(ctx->colorspace);
 	p_hdr->xfer_func = htonl(ctx->xfer_func);
 	p_hdr->ycbcr_enc = htonl(ctx->ycbcr_enc);
@@ -230,6 +318,7 @@ static int decode(struct vicodec_ctx *ctx,
 		  u8 *p_in, u8 *p_out)
 {
 	unsigned int size = q_data->width * q_data->height;
+	unsigned int chroma_size = size;
 	unsigned int i;
 	struct cframe_hdr *p_hdr;
 	struct cframe cf;
@@ -259,32 +348,114 @@ static int decode(struct vicodec_ctx *ctx,
 	if (cf.width != q_data->width || cf.height != q_data->height)
 		return -EINVAL;
 
+	if (!(q_data->flags & VICODEC_FL_CHROMA_FULL_WIDTH))
+		chroma_size /= 2;
+	if (!(q_data->flags & VICODEC_FL_CHROMA_FULL_HEIGHT))
+		chroma_size /= 2;
+
 	decode_frame(&cf, &ctx->ref_frame, q_data->flags);
-	memcpy(p_out, ctx->ref_frame.luma, size);
-	p_out += size;
 
-	switch (q_data->fourcc) {
+	switch (q_data->info->id) {
 	case V4L2_PIX_FMT_YUV420:
-		memcpy(p_out, ctx->ref_frame.cb, size / 4);
-		p_out += size / 4;
-		memcpy(p_out, ctx->ref_frame.cr, size / 4);
+	case V4L2_PIX_FMT_YUV422P:
+		memcpy(p_out, ctx->ref_frame.luma, size);
+		p_out += size;
+		memcpy(p_out, ctx->ref_frame.cb, chroma_size);
+		p_out += chroma_size;
+		memcpy(p_out, ctx->ref_frame.cr, chroma_size);
 		break;
 	case V4L2_PIX_FMT_YVU420:
-		memcpy(p_out, ctx->ref_frame.cr, size / 4);
-		p_out += size / 4;
-		memcpy(p_out, ctx->ref_frame.cb, size / 4);
+		memcpy(p_out, ctx->ref_frame.luma, size);
+		p_out += size;
+		memcpy(p_out, ctx->ref_frame.cr, chroma_size);
+		p_out += chroma_size;
+		memcpy(p_out, ctx->ref_frame.cb, chroma_size);
 		break;
 	case V4L2_PIX_FMT_NV12:
-		for (i = 0, p = p_out; i < size / 4; i++, p += 2)
-			*p = ctx->ref_frame.cb[i];
-		for (i = 0, p = p_out + 1; i < size / 4; i++, p += 2)
-			*p = ctx->ref_frame.cr[i];
+	case V4L2_PIX_FMT_NV16:
+	case V4L2_PIX_FMT_NV24:
+		memcpy(p_out, ctx->ref_frame.luma, size);
+		p_out += size;
+		for (i = 0, p = p_out; i < chroma_size; i++) {
+			*p++ = ctx->ref_frame.cb[i];
+			*p++ = ctx->ref_frame.cr[i];
+		}
 		break;
 	case V4L2_PIX_FMT_NV21:
-		for (i = 0, p = p_out; i < size / 4; i++, p += 2)
-			*p = ctx->ref_frame.cr[i];
-		for (i = 0, p = p_out + 1; i < size / 4; i++, p += 2)
-			*p = ctx->ref_frame.cb[i];
+	case V4L2_PIX_FMT_NV61:
+	case V4L2_PIX_FMT_NV42:
+		memcpy(p_out, ctx->ref_frame.luma, size);
+		p_out += size;
+		for (i = 0, p = p_out; i < chroma_size; i++) {
+			*p++ = ctx->ref_frame.cr[i];
+			*p++ = ctx->ref_frame.cb[i];
+		}
+		break;
+	case V4L2_PIX_FMT_YUYV:
+		for (i = 0, p = p_out; i < size; i += 2) {
+			*p++ = ctx->ref_frame.luma[i];
+			*p++ = ctx->ref_frame.cb[i / 2];
+			*p++ = ctx->ref_frame.luma[i + 1];
+			*p++ = ctx->ref_frame.cr[i / 2];
+		}
+		break;
+	case V4L2_PIX_FMT_YVYU:
+		for (i = 0, p = p_out; i < size; i += 2) {
+			*p++ = ctx->ref_frame.luma[i];
+			*p++ = ctx->ref_frame.cr[i / 2];
+			*p++ = ctx->ref_frame.luma[i + 1];
+			*p++ = ctx->ref_frame.cb[i / 2];
+		}
+		break;
+	case V4L2_PIX_FMT_UYVY:
+		for (i = 0, p = p_out; i < size; i += 2) {
+			*p++ = ctx->ref_frame.cb[i / 2];
+			*p++ = ctx->ref_frame.luma[i];
+			*p++ = ctx->ref_frame.cr[i / 2];
+			*p++ = ctx->ref_frame.luma[i + 1];
+		}
+		break;
+	case V4L2_PIX_FMT_VYUY:
+		for (i = 0, p = p_out; i < size; i += 2) {
+			*p++ = ctx->ref_frame.cr[i / 2];
+			*p++ = ctx->ref_frame.luma[i];
+			*p++ = ctx->ref_frame.cb[i / 2];
+			*p++ = ctx->ref_frame.luma[i + 1];
+		}
+		break;
+	case V4L2_PIX_FMT_RGB24:
+	case V4L2_PIX_FMT_HSV24:
+		for (i = 0, p = p_out; i < size; i++) {
+			*p++ = ctx->ref_frame.cr[i];
+			*p++ = ctx->ref_frame.luma[i];
+			*p++ = ctx->ref_frame.cb[i];
+		}
+		break;
+	case V4L2_PIX_FMT_BGR24:
+		for (i = 0, p = p_out; i < size; i++) {
+			*p++ = ctx->ref_frame.cb[i];
+			*p++ = ctx->ref_frame.luma[i];
+			*p++ = ctx->ref_frame.cr[i];
+		}
+		break;
+	case V4L2_PIX_FMT_RGB32:
+	case V4L2_PIX_FMT_XRGB32:
+	case V4L2_PIX_FMT_HSV32:
+		for (i = 0, p = p_out; i < size; i++) {
+			*p++ = 0;
+			*p++ = ctx->ref_frame.cr[i];
+			*p++ = ctx->ref_frame.luma[i];
+			*p++ = ctx->ref_frame.cb[i];
+		}
+		break;
+	case V4L2_PIX_FMT_BGR32:
+	case V4L2_PIX_FMT_XBGR32:
+		for (i = 0, p = p_out; i < size; i++) {
+			*p++ = ctx->ref_frame.cb[i];
+			*p++ = ctx->ref_frame.luma[i];
+			*p++ = ctx->ref_frame.cr[i];
+			*p++ = 0;
+		}
 		break;
 	}
 	return 0;
@@ -322,8 +493,7 @@ static int device_process(struct vicodec_ctx *ctx,
 		ret = decode(ctx, q_cap, p_in, p_out);
 		if (ret)
 			return ret;
-		vb2_set_plane_payload(&out_vb->vb2_buf, 0,
-				      q_cap->width * q_cap->height * 3 / 2);
+		vb2_set_plane_payload(&out_vb->vb2_buf, 0, q_cap->sizeimage);
 	}
 
 	out_vb->sequence = q_cap->sequence++;
@@ -523,14 +693,14 @@ static void job_abort(void *priv)
  * video ioctls
  */
 
-static u32 find_fmt(u32 fmt)
+static const struct pixfmt_info *find_fmt(u32 fmt)
 {
 	unsigned int i;
 
-	for (i = 0; i < ARRAY_SIZE(pixfmts_yuv); i++)
-		if (pixfmts_yuv[i] == fmt)
-			return fmt;
-	return pixfmts_yuv[0];
+	for (i = 0; i < ARRAY_SIZE(pixfmts); i++)
+		if (pixfmts[i].id == fmt)
+			return &pixfmts[i];
+	return &pixfmts[0];
 }
 
 static int vidioc_querycap(struct file *file, void *priv,
@@ -550,17 +720,17 @@ static int vidioc_querycap(struct file *file, void *priv,
 
 static int enum_fmt(struct v4l2_fmtdesc *f, bool is_enc, bool is_out)
 {
-	bool is_yuv = (is_enc && is_out) || (!is_enc && !is_out);
+	bool is_uncomp = (is_enc && is_out) || (!is_enc && !is_out);
 
 	if (V4L2_TYPE_IS_MULTIPLANAR(f->type) && !multiplanar)
 		return -EINVAL;
 	if (!V4L2_TYPE_IS_MULTIPLANAR(f->type) && multiplanar)
 		return -EINVAL;
-	if (f->index >= (is_yuv ? ARRAY_SIZE(pixfmts_yuv) : 1))
+	if (f->index >= (is_uncomp ? ARRAY_SIZE(pixfmts) : 1))
 		return -EINVAL;
 
-	if (is_yuv)
-		f->pixelformat = pixfmts_yuv[f->index];
+	if (is_uncomp)
+		f->pixelformat = pixfmts[f->index].id;
 	else
 		f->pixelformat = V4L2_PIX_FMT_FWHT;
 	return 0;
@@ -588,12 +758,14 @@ static int vidioc_g_fmt(struct vicodec_ctx *ctx, struct v4l2_format *f)
 	struct vicodec_q_data *q_data;
 	struct v4l2_pix_format_mplane *pix_mp;
 	struct v4l2_pix_format *pix;
+	const struct pixfmt_info *info;
 
 	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type);
 	if (!vq)
 		return -EINVAL;
 
 	q_data = get_q_data(ctx, f->type);
+	info = q_data->info;
 
 	switch (f->type) {
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE:
@@ -604,11 +776,8 @@ static int vidioc_g_fmt(struct vicodec_ctx *ctx, struct v4l2_format *f)
 		pix->width = q_data->width;
 		pix->height = q_data->height;
 		pix->field = V4L2_FIELD_NONE;
-		pix->pixelformat = q_data->fourcc;
-		if (q_data->fourcc == V4L2_PIX_FMT_FWHT)
-			pix->bytesperline = 0;
-		else
-			pix->bytesperline = q_data->width;
+		pix->pixelformat = info->id;
+		pix->bytesperline = q_data->width * info->bytesperline_mult;
 		pix->sizeimage = q_data->sizeimage;
 		pix->colorspace = ctx->colorspace;
 		pix->xfer_func = ctx->xfer_func;
@@ -624,12 +793,10 @@ static int vidioc_g_fmt(struct vicodec_ctx *ctx, struct v4l2_format *f)
 		pix_mp->width = q_data->width;
 		pix_mp->height = q_data->height;
 		pix_mp->field = V4L2_FIELD_NONE;
-		pix_mp->pixelformat = q_data->fourcc;
+		pix_mp->pixelformat = info->id;
 		pix_mp->num_planes = 1;
-		if (q_data->fourcc == V4L2_PIX_FMT_FWHT)
-			pix_mp->plane_fmt[0].bytesperline = 0;
-		else
-			pix_mp->plane_fmt[0].bytesperline = q_data->width;
+		pix_mp->plane_fmt[0].bytesperline =
+				q_data->width * info->bytesperline_mult;
 		pix_mp->plane_fmt[0].sizeimage = q_data->sizeimage;
 		pix_mp->colorspace = ctx->colorspace;
 		pix_mp->xfer_func = ctx->xfer_func;
@@ -661,40 +828,44 @@ static int vidioc_try_fmt(struct vicodec_ctx *ctx, struct v4l2_format *f)
 {
 	struct v4l2_pix_format_mplane *pix_mp;
 	struct v4l2_pix_format *pix;
+	struct v4l2_plane_pix_format *plane;
+	const struct pixfmt_info *info = &pixfmt_fwht;
 
 	switch (f->type) {
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE:
 	case V4L2_BUF_TYPE_VIDEO_OUTPUT:
 		pix = &f->fmt.pix;
+		if (pix->pixelformat != V4L2_PIX_FMT_FWHT)
+			info = find_fmt(pix->pixelformat);
 		pix->width = clamp(pix->width, MIN_WIDTH, MAX_WIDTH) & ~7;
 		pix->height = clamp(pix->height, MIN_HEIGHT, MAX_HEIGHT) & ~7;
-		pix->bytesperline = pix->width;
-		pix->sizeimage = pix->width * pix->height * 3 / 2;
 		pix->field = V4L2_FIELD_NONE;
-		if (pix->pixelformat == V4L2_PIX_FMT_FWHT) {
-			pix->bytesperline = 0;
+		pix->bytesperline =
+			pix->width * info->bytesperline_mult;
+		pix->sizeimage = pix->width * pix->height *
+			info->sizeimage_mult / info->sizeimage_div;
+		if (pix->pixelformat == V4L2_PIX_FMT_FWHT)
 			pix->sizeimage += sizeof(struct cframe_hdr);
-		}
 		break;
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
 	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
 		pix_mp = &f->fmt.pix_mp;
+		plane = pix_mp->plane_fmt;
+		if (pix_mp->pixelformat != V4L2_PIX_FMT_FWHT)
+			info = find_fmt(pix_mp->pixelformat);
+		pix_mp->num_planes = 1;
 		pix_mp->width = clamp(pix_mp->width, MIN_WIDTH, MAX_WIDTH) & ~7;
 		pix_mp->height =
 			clamp(pix_mp->height, MIN_HEIGHT, MAX_HEIGHT) & ~7;
-		pix_mp->plane_fmt[0].bytesperline = pix_mp->width;
-		pix_mp->plane_fmt[0].sizeimage =
-			pix_mp->width * pix_mp->height * 3 / 2;
 		pix_mp->field = V4L2_FIELD_NONE;
-		pix_mp->num_planes = 1;
-		if (pix_mp->pixelformat == V4L2_PIX_FMT_FWHT) {
-			pix_mp->plane_fmt[0].bytesperline = 0;
-			pix_mp->plane_fmt[0].sizeimage +=
-					sizeof(struct cframe_hdr);
-		}
+		plane->bytesperline =
+			pix_mp->width * info->bytesperline_mult;
+		plane->sizeimage = pix_mp->width * pix_mp->height *
+			info->sizeimage_mult / info->sizeimage_div;
+		if (pix_mp->pixelformat == V4L2_PIX_FMT_FWHT)
+			plane->sizeimage += sizeof(struct cframe_hdr);
 		memset(pix_mp->reserved, 0, sizeof(pix_mp->reserved));
-		memset(pix_mp->plane_fmt[0].reserved, 0,
-		       sizeof(pix_mp->plane_fmt[0].reserved));
+		memset(plane->reserved, 0, sizeof(plane->reserved));
 		break;
 	default:
 		return -EINVAL;
@@ -716,7 +887,7 @@ static int vidioc_try_fmt_vid_cap(struct file *file, void *priv,
 			return -EINVAL;
 		pix = &f->fmt.pix;
 		pix->pixelformat = ctx->is_enc ? V4L2_PIX_FMT_FWHT :
-				   find_fmt(f->fmt.pix.pixelformat);
+				   find_fmt(f->fmt.pix.pixelformat)->id;
 		pix->colorspace = ctx->colorspace;
 		pix->xfer_func = ctx->xfer_func;
 		pix->ycbcr_enc = ctx->ycbcr_enc;
@@ -727,14 +898,11 @@ static int vidioc_try_fmt_vid_cap(struct file *file, void *priv,
 			return -EINVAL;
 		pix_mp = &f->fmt.pix_mp;
 		pix_mp->pixelformat = ctx->is_enc ? V4L2_PIX_FMT_FWHT :
-				      find_fmt(pix_mp->pixelformat);
+				      find_fmt(pix_mp->pixelformat)->id;
 		pix_mp->colorspace = ctx->colorspace;
 		pix_mp->xfer_func = ctx->xfer_func;
 		pix_mp->ycbcr_enc = ctx->ycbcr_enc;
 		pix_mp->quantization = ctx->quantization;
-		memset(pix_mp->reserved, 0, sizeof(pix_mp->reserved));
-		memset(pix_mp->plane_fmt[0].reserved, 0,
-		       sizeof(pix_mp->plane_fmt[0].reserved));
 		break;
 	default:
 		return -EINVAL;
@@ -756,7 +924,7 @@ static int vidioc_try_fmt_vid_out(struct file *file, void *priv,
 			return -EINVAL;
 		pix = &f->fmt.pix;
 		pix->pixelformat = !ctx->is_enc ? V4L2_PIX_FMT_FWHT :
-				   find_fmt(pix->pixelformat);
+				   find_fmt(pix->pixelformat)->id;
 		if (!pix->colorspace)
 			pix->colorspace = V4L2_COLORSPACE_REC709;
 		break;
@@ -765,7 +933,7 @@ static int vidioc_try_fmt_vid_out(struct file *file, void *priv,
 			return -EINVAL;
 		pix_mp = &f->fmt.pix_mp;
 		pix_mp->pixelformat = !ctx->is_enc ? V4L2_PIX_FMT_FWHT :
-				      find_fmt(pix_mp->pixelformat);
+				      find_fmt(pix_mp->pixelformat)->id;
 		if (!pix_mp->colorspace)
 			pix_mp->colorspace = V4L2_COLORSPACE_REC709;
 		break;
@@ -798,14 +966,17 @@ static int vidioc_s_fmt(struct vicodec_ctx *ctx, struct v4l2_format *f)
 		pix = &f->fmt.pix;
 		if (ctx->is_enc && V4L2_TYPE_IS_OUTPUT(f->type))
 			fmt_changed =
-				q_data->fourcc != pix->pixelformat ||
+				q_data->info->id != pix->pixelformat ||
 				q_data->width != pix->width ||
 				q_data->height != pix->height;
 
 		if (vb2_is_busy(vq) && fmt_changed)
 			return -EBUSY;
 
-		q_data->fourcc = pix->pixelformat;
+		if (pix->pixelformat == V4L2_PIX_FMT_FWHT)
+			q_data->info = &pixfmt_fwht;
+		else
+			q_data->info = find_fmt(pix->pixelformat);
 		q_data->width = pix->width;
 		q_data->height = pix->height;
 		q_data->sizeimage = pix->sizeimage;
@@ -815,14 +986,17 @@ static int vidioc_s_fmt(struct vicodec_ctx *ctx, struct v4l2_format *f)
 		pix_mp = &f->fmt.pix_mp;
 		if (ctx->is_enc && V4L2_TYPE_IS_OUTPUT(f->type))
 			fmt_changed =
-				q_data->fourcc != pix_mp->pixelformat ||
+				q_data->info->id != pix_mp->pixelformat ||
 				q_data->width != pix_mp->width ||
 				q_data->height != pix_mp->height;
 
 		if (vb2_is_busy(vq) && fmt_changed)
 			return -EBUSY;
 
-		q_data->fourcc = pix_mp->pixelformat;
+		if (pix_mp->pixelformat == V4L2_PIX_FMT_FWHT)
+			q_data->info = &pixfmt_fwht;
+		else
+			q_data->info = find_fmt(pix_mp->pixelformat);
 		q_data->width = pix_mp->width;
 		q_data->height = pix_mp->height;
 		q_data->sizeimage = pix_mp->plane_fmt[0].sizeimage;
@@ -833,7 +1007,7 @@ static int vidioc_s_fmt(struct vicodec_ctx *ctx, struct v4l2_format *f)
 
 	dprintk(ctx->dev,
 		"Setting format for type %d, wxh: %dx%d, fourcc: %08x\n",
-		f->type, q_data->width, q_data->height, q_data->fourcc);
+		f->type, q_data->width, q_data->height, q_data->info->id);
 
 	return 0;
 }
@@ -965,7 +1139,7 @@ static int vicodec_enum_framesizes(struct file *file, void *fh,
 	case V4L2_PIX_FMT_FWHT:
 		break;
 	default:
-		if (find_fmt(fsize->pixel_format) == fsize->pixel_format)
+		if (find_fmt(fsize->pixel_format)->id == fsize->pixel_format)
 			break;
 		return -EINVAL;
 	}
@@ -1123,6 +1297,8 @@ static int vicodec_start_streaming(struct vb2_queue *q,
 	struct vicodec_ctx *ctx = vb2_get_drv_priv(q);
 	struct vicodec_q_data *q_data = get_q_data(ctx, q->type);
 	unsigned int size = q_data->width * q_data->height;
+	const struct pixfmt_info *info = q_data->info;
+	unsigned int chroma_div = info->width_div * info->height_div;
 
 	q_data->sequence = 0;
 
@@ -1130,8 +1306,9 @@ static int vicodec_start_streaming(struct vb2_queue *q,
 		return 0;
 
 	ctx->ref_frame.width = ctx->ref_frame.height = 0;
-	ctx->ref_frame.luma = kvmalloc(size * 3 / 2, GFP_KERNEL);
-	ctx->comp_max_size = size * 3 / 2 + sizeof(struct cframe_hdr);
+	ctx->ref_frame.luma = kvmalloc(size + 2 * size / chroma_div, GFP_KERNEL);
+	ctx->comp_max_size = size + 2 * size / chroma_div +
+			     sizeof(struct cframe_hdr);
 	ctx->compressed_frame = kvmalloc(ctx->comp_max_size, GFP_KERNEL);
 	if (!ctx->ref_frame.luma || !ctx->compressed_frame) {
 		kvfree(ctx->ref_frame.luma);
@@ -1140,7 +1317,7 @@ static int vicodec_start_streaming(struct vb2_queue *q,
 		return -ENOMEM;
 	}
 	ctx->ref_frame.cb = ctx->ref_frame.luma + size;
-	ctx->ref_frame.cr = ctx->ref_frame.cb + size / 4;
+	ctx->ref_frame.cr = ctx->ref_frame.cb + size / chroma_div;
 	ctx->last_src_buf = NULL;
 	ctx->last_dst_buf = NULL;
 	ctx->gop_cnt = 0;
@@ -1301,15 +1478,19 @@ static int vicodec_open(struct file *file)
 	ctx->fh.ctrl_handler = hdl;
 	v4l2_ctrl_handler_setup(hdl);
 
-	ctx->q_data[V4L2_M2M_SRC].fourcc =
-		ctx->is_enc ? V4L2_PIX_FMT_YUV420 : V4L2_PIX_FMT_FWHT;
+	ctx->q_data[V4L2_M2M_SRC].info =
+		ctx->is_enc ? pixfmts : &pixfmt_fwht;
 	ctx->q_data[V4L2_M2M_SRC].width = 1280;
 	ctx->q_data[V4L2_M2M_SRC].height = 720;
-	size = 1280 * 720 * 3 / 2;
+	size = 1280 * 720 * ctx->q_data[V4L2_M2M_SRC].info->sizeimage_mult /
+		ctx->q_data[V4L2_M2M_SRC].info->sizeimage_div;
 	ctx->q_data[V4L2_M2M_SRC].sizeimage = size;
 	ctx->q_data[V4L2_M2M_DST] = ctx->q_data[V4L2_M2M_SRC];
-	ctx->q_data[V4L2_M2M_DST].fourcc =
-		ctx->is_enc ? V4L2_PIX_FMT_FWHT : V4L2_PIX_FMT_YUV420;
+	ctx->q_data[V4L2_M2M_DST].info =
+		ctx->is_enc ? &pixfmt_fwht : pixfmts;
+	size = 1280 * 720 * ctx->q_data[V4L2_M2M_DST].info->sizeimage_mult /
+		ctx->q_data[V4L2_M2M_DST].info->sizeimage_div;
+	ctx->q_data[V4L2_M2M_DST].sizeimage = size;
 	ctx->colorspace = V4L2_COLORSPACE_REC709;
 
 	size += sizeof(struct cframe_hdr);
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCHv2 3/7] vicodec: simplify flags handling
  2018-08-23  7:32 [PATCHv2 0/7] vicodec improvements Hans Verkuil
  2018-08-23  7:32 ` [PATCHv2 1/7] vicodec: add QP controls Hans Verkuil
  2018-08-23  7:33 ` [PATCHv2 2/7] vicodec: add support for more pixel formats Hans Verkuil
@ 2018-08-23  7:33 ` Hans Verkuil
  2018-08-23  7:33 ` [PATCHv2 4/7] vicodec: simplify blocktype checking Hans Verkuil
                   ` (4 subsequent siblings)
  7 siblings, 0 replies; 10+ messages in thread
From: Hans Verkuil @ 2018-08-23  7:33 UTC (permalink / raw)
  To: linux-media; +Cc: Hans Verkuil

From: Hans Verkuil <hans.verkuil@cisco.com>

The flags field can be removed from struct vicodec_q_data.
This simplifies the flags handling elsewhere.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/vicodec/vicodec-core.c | 26 +++++++++----------
 1 file changed, 13 insertions(+), 13 deletions(-)

diff --git a/drivers/media/platform/vicodec/vicodec-core.c b/drivers/media/platform/vicodec/vicodec-core.c
index 6253fd9e1d30..4680b3c9b9b2 100644
--- a/drivers/media/platform/vicodec/vicodec-core.c
+++ b/drivers/media/platform/vicodec/vicodec-core.c
@@ -101,7 +101,6 @@ static struct platform_device vicodec_pdev = {
 struct vicodec_q_data {
 	unsigned int		width;
 	unsigned int		height;
-	unsigned int		flags;
 	unsigned int		sizeimage;
 	unsigned int		sequence;
 	const struct pixfmt_info *info;
@@ -188,7 +187,7 @@ static struct vicodec_q_data *get_q_data(struct vicodec_ctx *ctx,
 
 static void encode(struct vicodec_ctx *ctx,
 		   struct vicodec_q_data *q_data,
-		   u8 *p_in, u8 *p_out)
+		   u8 *p_in, u8 *p_out, u32 flags)
 {
 	unsigned int size = q_data->width * q_data->height;
 	const struct pixfmt_info *info = q_data->info;
@@ -293,17 +292,17 @@ static void encode(struct vicodec_ctx *ctx,
 	p_hdr->version = htonl(VICODEC_VERSION);
 	p_hdr->width = htonl(cf.width);
 	p_hdr->height = htonl(cf.height);
-	p_hdr->flags = htonl(q_data->flags);
 	if (encoding & LUMA_UNENCODED)
-		p_hdr->flags |= htonl(VICODEC_FL_LUMA_IS_UNCOMPRESSED);
+		flags |= VICODEC_FL_LUMA_IS_UNCOMPRESSED;
 	if (encoding & CB_UNENCODED)
-		p_hdr->flags |= htonl(VICODEC_FL_CB_IS_UNCOMPRESSED);
+		flags |= VICODEC_FL_CB_IS_UNCOMPRESSED;
 	if (encoding & CR_UNENCODED)
-		p_hdr->flags |= htonl(VICODEC_FL_CR_IS_UNCOMPRESSED);
+		flags |= VICODEC_FL_CR_IS_UNCOMPRESSED;
 	if (rf.height_div == 1)
-		p_hdr->flags |= htonl(VICODEC_FL_CHROMA_FULL_HEIGHT);
+		flags |= VICODEC_FL_CHROMA_FULL_HEIGHT;
 	if (rf.width_div == 1)
-		p_hdr->flags |= htonl(VICODEC_FL_CHROMA_FULL_WIDTH);
+		flags |= VICODEC_FL_CHROMA_FULL_WIDTH;
+	p_hdr->flags = htonl(flags);
 	p_hdr->colorspace = htonl(ctx->colorspace);
 	p_hdr->xfer_func = htonl(ctx->xfer_func);
 	p_hdr->ycbcr_enc = htonl(ctx->ycbcr_enc);
@@ -320,6 +319,7 @@ static int decode(struct vicodec_ctx *ctx,
 	unsigned int size = q_data->width * q_data->height;
 	unsigned int chroma_size = size;
 	unsigned int i;
+	u32 flags;
 	struct cframe_hdr *p_hdr;
 	struct cframe cf;
 	u8 *p;
@@ -327,7 +327,7 @@ static int decode(struct vicodec_ctx *ctx,
 	p_hdr = (struct cframe_hdr *)p_in;
 	cf.width = ntohl(p_hdr->width);
 	cf.height = ntohl(p_hdr->height);
-	q_data->flags = ntohl(p_hdr->flags);
+	flags = ntohl(p_hdr->flags);
 	ctx->colorspace = ntohl(p_hdr->colorspace);
 	ctx->xfer_func = ntohl(p_hdr->xfer_func);
 	ctx->ycbcr_enc = ntohl(p_hdr->ycbcr_enc);
@@ -348,12 +348,12 @@ static int decode(struct vicodec_ctx *ctx,
 	if (cf.width != q_data->width || cf.height != q_data->height)
 		return -EINVAL;
 
-	if (!(q_data->flags & VICODEC_FL_CHROMA_FULL_WIDTH))
+	if (!(flags & VICODEC_FL_CHROMA_FULL_WIDTH))
 		chroma_size /= 2;
-	if (!(q_data->flags & VICODEC_FL_CHROMA_FULL_HEIGHT))
+	if (!(flags & VICODEC_FL_CHROMA_FULL_HEIGHT))
 		chroma_size /= 2;
 
-	decode_frame(&cf, &ctx->ref_frame, q_data->flags);
+	decode_frame(&cf, &ctx->ref_frame, flags);
 
 	switch (q_data->info->id) {
 	case V4L2_PIX_FMT_YUV420:
@@ -486,7 +486,7 @@ static int device_process(struct vicodec_ctx *ctx,
 	if (ctx->is_enc) {
 		struct cframe_hdr *p_hdr = (struct cframe_hdr *)p_out;
 
-		encode(ctx, q_out, p_in, p_out);
+		encode(ctx, q_out, p_in, p_out, 0);
 		vb2_set_plane_payload(&out_vb->vb2_buf, 0,
 				      sizeof(*p_hdr) + ntohl(p_hdr->size));
 	} else {
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCHv2 4/7] vicodec: simplify blocktype checking
  2018-08-23  7:32 [PATCHv2 0/7] vicodec improvements Hans Verkuil
                   ` (2 preceding siblings ...)
  2018-08-23  7:33 ` [PATCHv2 3/7] vicodec: simplify flags handling Hans Verkuil
@ 2018-08-23  7:33 ` Hans Verkuil
  2018-08-23  7:33 ` [PATCHv2 5/7] vicodec: improve handling of uncompressable planes Hans Verkuil
                   ` (3 subsequent siblings)
  7 siblings, 0 replies; 10+ messages in thread
From: Hans Verkuil @ 2018-08-23  7:33 UTC (permalink / raw)
  To: linux-media; +Cc: Hans Verkuil

From: Hans Verkuil <hans.verkuil@cisco.com>

Simplify some blocktype/is_intra checks.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/vicodec/vicodec-codec.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/media/platform/vicodec/vicodec-codec.c b/drivers/media/platform/vicodec/vicodec-codec.c
index 7bd11a974db0..e402d988f2ad 100644
--- a/drivers/media/platform/vicodec/vicodec-codec.c
+++ b/drivers/media/platform/vicodec/vicodec-codec.c
@@ -663,11 +663,10 @@ static u32 encode_plane(u8 *input, u8 *refp, __be16 **rlco, __be16 *rlco_max,
 			if (!is_intra)
 				blocktype = decide_blocktype(input, refp,
 					deltablock, width, input_step);
-			if (is_intra || blocktype == IBLOCK) {
+			if (blocktype == IBLOCK) {
 				fwht(input, cf->coeffs, width, input_step, 1);
 				quantize_intra(cf->coeffs, cf->de_coeffs,
 					       cf->i_frame_qp);
-				blocktype = IBLOCK;
 			} else {
 				/* inter code */
 				encoding |= FRAME_PCODED;
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCHv2 5/7] vicodec: improve handling of uncompressable planes
  2018-08-23  7:32 [PATCHv2 0/7] vicodec improvements Hans Verkuil
                   ` (3 preceding siblings ...)
  2018-08-23  7:33 ` [PATCHv2 4/7] vicodec: simplify blocktype checking Hans Verkuil
@ 2018-08-23  7:33 ` Hans Verkuil
  2018-08-23  7:33 ` [PATCHv2 6/7] vicodec: rename and use proper fwht prefix for codec Hans Verkuil
                   ` (2 subsequent siblings)
  7 siblings, 0 replies; 10+ messages in thread
From: Hans Verkuil @ 2018-08-23  7:33 UTC (permalink / raw)
  To: linux-media; +Cc: Hans Verkuil

From: Hans Verkuil <hans.verkuil@cisco.com>

Exit the loop immediately once it is clear that the plane
cannot be compressed. Also clear the PCODED bit and fix the
PCODED check (it should check for the bit) in the caller code.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/vicodec/vicodec-codec.c | 10 ++++++----
 drivers/media/platform/vicodec/vicodec-core.c  |  2 +-
 2 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/vicodec/vicodec-codec.c b/drivers/media/platform/vicodec/vicodec-codec.c
index e402d988f2ad..3547129c1163 100644
--- a/drivers/media/platform/vicodec/vicodec-codec.c
+++ b/drivers/media/platform/vicodec/vicodec-codec.c
@@ -685,9 +685,6 @@ static u32 encode_plane(u8 *input, u8 *refp, __be16 **rlco, __be16 *rlco_max,
 			input += 8 * input_step;
 			refp += 8 * 8;
 
-			if (encoding & FRAME_UNENCODED)
-				continue;
-
 			size = rlc(cf->coeffs, *rlco, blocktype);
 			if (last_size == size &&
 			    !memcmp(*rlco + 1, *rlco - size + 1, 2 * size - 2)) {
@@ -702,12 +699,16 @@ static u32 encode_plane(u8 *input, u8 *refp, __be16 **rlco, __be16 *rlco_max,
 			} else {
 				*rlco += size;
 			}
-			if (*rlco >= rlco_max)
+			if (*rlco >= rlco_max) {
 				encoding |= FRAME_UNENCODED;
+				goto exit_loop;
+			}
 			last_size = size;
 		}
 		input += width * 7 * input_step;
 	}
+
+exit_loop:
 	if (encoding & FRAME_UNENCODED) {
 		u8 *out = (u8 *)rlco_start;
 
@@ -721,6 +722,7 @@ static u32 encode_plane(u8 *input, u8 *refp, __be16 **rlco, __be16 *rlco_max,
 		for (i = 0; i < height * width; i++, input += input_step)
 			*out++ = (*input == 0xff) ? 0xfe : *input;
 		*rlco = (__be16 *)out;
+		encoding &= ~FRAME_PCODED;
 	}
 	return encoding;
 }
diff --git a/drivers/media/platform/vicodec/vicodec-core.c b/drivers/media/platform/vicodec/vicodec-core.c
index 4680b3c9b9b2..caff521d94c6 100644
--- a/drivers/media/platform/vicodec/vicodec-core.c
+++ b/drivers/media/platform/vicodec/vicodec-core.c
@@ -281,7 +281,7 @@ static void encode(struct vicodec_ctx *ctx,
 
 	encoding = encode_frame(&rf, &ctx->ref_frame, &cf, !ctx->gop_cnt,
 				ctx->gop_cnt == ctx->gop_size - 1);
-	if (encoding != FRAME_PCODED)
+	if (!(encoding & FRAME_PCODED))
 		ctx->gop_cnt = 0;
 	if (++ctx->gop_cnt >= ctx->gop_size)
 		ctx->gop_cnt = 0;
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCHv2 6/7] vicodec: rename and use proper fwht prefix for codec
  2018-08-23  7:32 [PATCHv2 0/7] vicodec improvements Hans Verkuil
                   ` (4 preceding siblings ...)
  2018-08-23  7:33 ` [PATCHv2 5/7] vicodec: improve handling of uncompressable planes Hans Verkuil
@ 2018-08-23  7:33 ` Hans Verkuil
  2018-08-23  7:33 ` [PATCHv2 7/7] vicodec: split off v4l2 specific parts for the codec Hans Verkuil
  2018-08-23  8:13 ` [PATCHv2 0/7] vicodec improvements Hans Verkuil
  7 siblings, 0 replies; 10+ messages in thread
From: Hans Verkuil @ 2018-08-23  7:33 UTC (permalink / raw)
  To: linux-media; +Cc: Hans Verkuil

From: Hans Verkuil <hans.verkuil@cisco.com>

The codec source is generic and not vicodec specific. It can be used
by other drivers or userspace as well. So rename the source and header
to something more generic (codec-fwht.c/h) and prefix the defines, types
and functions with fwht_.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 .../media/uapi/v4l/pixfmt-compressed.rst      |  2 +-
 drivers/media/platform/vicodec/Makefile       |  2 +-
 .../vicodec/{vicodec-codec.c => codec-fwht.c} | 62 ++++++++-----
 .../vicodec/{vicodec-codec.h => codec-fwht.h} | 77 +++++++---------
 drivers/media/platform/vicodec/vicodec-core.c | 89 ++++++++++---------
 5 files changed, 118 insertions(+), 114 deletions(-)
 rename drivers/media/platform/vicodec/{vicodec-codec.c => codec-fwht.c} (93%)
 rename drivers/media/platform/vicodec/{vicodec-codec.h => codec-fwht.h} (63%)

diff --git a/Documentation/media/uapi/v4l/pixfmt-compressed.rst b/Documentation/media/uapi/v4l/pixfmt-compressed.rst
index d382e7a5c38e..d04b18adac33 100644
--- a/Documentation/media/uapi/v4l/pixfmt-compressed.rst
+++ b/Documentation/media/uapi/v4l/pixfmt-compressed.rst
@@ -101,4 +101,4 @@ Compressed Formats
       - 'FWHT'
       - Video elementary stream using a codec based on the Fast Walsh Hadamard
         Transform. This codec is implemented by the vicodec ('Virtual Codec')
-	driver. See the vicodec-codec.h header for more details.
+	driver. See the codec-fwht.h header for more details.
diff --git a/drivers/media/platform/vicodec/Makefile b/drivers/media/platform/vicodec/Makefile
index 197229428953..a27242ff14ad 100644
--- a/drivers/media/platform/vicodec/Makefile
+++ b/drivers/media/platform/vicodec/Makefile
@@ -1,4 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0
-vicodec-objs := vicodec-core.o vicodec-codec.o
+vicodec-objs := vicodec-core.o codec-fwht.o
 
 obj-$(CONFIG_VIDEO_VICODEC) += vicodec.o
diff --git a/drivers/media/platform/vicodec/vicodec-codec.c b/drivers/media/platform/vicodec/codec-fwht.c
similarity index 93%
rename from drivers/media/platform/vicodec/vicodec-codec.c
rename to drivers/media/platform/vicodec/codec-fwht.c
index 3547129c1163..f91f90f7e5fc 100644
--- a/drivers/media/platform/vicodec/vicodec-codec.c
+++ b/drivers/media/platform/vicodec/codec-fwht.c
@@ -10,7 +10,18 @@
  */
 
 #include <linux/string.h>
-#include "vicodec-codec.h"
+#include "codec-fwht.h"
+
+/*
+ * Note: bit 0 of the header must always be 0. Otherwise it cannot
+ * be guaranteed that the magic 8 byte sequence (see below) can
+ * never occur in the rlc output.
+ */
+#define PFRAME_BIT BIT(15)
+#define DUPS_MASK 0x1ffe
+
+#define PBLOCK 0
+#define IBLOCK 1
 
 #define ALL_ZEROS 15
 
@@ -642,7 +653,7 @@ static void add_deltas(s16 *deltas, const u8 *ref, int stride)
 }
 
 static u32 encode_plane(u8 *input, u8 *refp, __be16 **rlco, __be16 *rlco_max,
-			struct cframe *cf, u32 height, u32 width,
+			struct fwht_cframe *cf, u32 height, u32 width,
 			unsigned int input_step,
 			bool is_intra, bool next_is_intra)
 {
@@ -669,7 +680,7 @@ static u32 encode_plane(u8 *input, u8 *refp, __be16 **rlco, __be16 *rlco_max,
 					       cf->i_frame_qp);
 			} else {
 				/* inter code */
-				encoding |= FRAME_PCODED;
+				encoding |= FWHT_FRAME_PCODED;
 				fwht16(deltablock, cf->coeffs, 8, 0);
 				quantize_inter(cf->coeffs, cf->de_coeffs,
 					       cf->p_frame_qp);
@@ -700,7 +711,7 @@ static u32 encode_plane(u8 *input, u8 *refp, __be16 **rlco, __be16 *rlco_max,
 				*rlco += size;
 			}
 			if (*rlco >= rlco_max) {
-				encoding |= FRAME_UNENCODED;
+				encoding |= FWHT_FRAME_UNENCODED;
 				goto exit_loop;
 			}
 			last_size = size;
@@ -709,7 +720,7 @@ static u32 encode_plane(u8 *input, u8 *refp, __be16 **rlco, __be16 *rlco_max,
 	}
 
 exit_loop:
-	if (encoding & FRAME_UNENCODED) {
+	if (encoding & FWHT_FRAME_UNENCODED) {
 		u8 *out = (u8 *)rlco_start;
 
 		input = input_start;
@@ -722,13 +733,15 @@ static u32 encode_plane(u8 *input, u8 *refp, __be16 **rlco, __be16 *rlco_max,
 		for (i = 0; i < height * width; i++, input += input_step)
 			*out++ = (*input == 0xff) ? 0xfe : *input;
 		*rlco = (__be16 *)out;
-		encoding &= ~FRAME_PCODED;
+		encoding &= ~FWHT_FRAME_PCODED;
 	}
 	return encoding;
 }
 
-u32 encode_frame(struct raw_frame *frm, struct raw_frame *ref_frm,
-		 struct cframe *cf, bool is_intra, bool next_is_intra)
+u32 fwht_encode_frame(struct fwht_raw_frame *frm,
+		      struct fwht_raw_frame *ref_frm,
+		      struct fwht_cframe *cf,
+		      bool is_intra, bool next_is_intra)
 {
 	unsigned int size = frm->height * frm->width;
 	__be16 *rlco = cf->rlc_data;
@@ -742,28 +755,28 @@ u32 encode_frame(struct raw_frame *frm, struct raw_frame *ref_frm,
 	encoding = encode_plane(frm->luma, ref_frm->luma, &rlco, rlco_max, cf,
 				frm->height, frm->width,
 				frm->luma_step, is_intra, next_is_intra);
-	if (encoding & FRAME_UNENCODED)
-		encoding |= LUMA_UNENCODED;
-	encoding &= ~FRAME_UNENCODED;
+	if (encoding & FWHT_FRAME_UNENCODED)
+		encoding |= FWHT_LUMA_UNENCODED;
+	encoding &= ~FWHT_FRAME_UNENCODED;
 	rlco_max = rlco + chroma_size / 2 - 256;
 	encoding |= encode_plane(frm->cb, ref_frm->cb, &rlco, rlco_max, cf,
 				 chroma_h, chroma_w,
 				 frm->chroma_step, is_intra, next_is_intra);
-	if (encoding & FRAME_UNENCODED)
-		encoding |= CB_UNENCODED;
-	encoding &= ~FRAME_UNENCODED;
+	if (encoding & FWHT_FRAME_UNENCODED)
+		encoding |= FWHT_CB_UNENCODED;
+	encoding &= ~FWHT_FRAME_UNENCODED;
 	rlco_max = rlco + chroma_size / 2 - 256;
 	encoding |= encode_plane(frm->cr, ref_frm->cr, &rlco, rlco_max, cf,
 				 chroma_h, chroma_w,
 				 frm->chroma_step, is_intra, next_is_intra);
-	if (encoding & FRAME_UNENCODED)
-		encoding |= CR_UNENCODED;
-	encoding &= ~FRAME_UNENCODED;
+	if (encoding & FWHT_FRAME_UNENCODED)
+		encoding |= FWHT_CR_UNENCODED;
+	encoding &= ~FWHT_FRAME_UNENCODED;
 	cf->size = (rlco - cf->rlc_data) * sizeof(*rlco);
 	return encoding;
 }
 
-static void decode_plane(struct cframe *cf, const __be16 **rlco, u8 *ref,
+static void decode_plane(struct fwht_cframe *cf, const __be16 **rlco, u8 *ref,
 			 u32 height, u32 width, bool uncompressed)
 {
 	unsigned int copies = 0;
@@ -816,20 +829,21 @@ static void decode_plane(struct cframe *cf, const __be16 **rlco, u8 *ref,
 	}
 }
 
-void decode_frame(struct cframe *cf, struct raw_frame *ref, u32 hdr_flags)
+void fwht_decode_frame(struct fwht_cframe *cf, struct fwht_raw_frame *ref,
+		       u32 hdr_flags)
 {
 	const __be16 *rlco = cf->rlc_data;
 	u32 h = cf->height / 2;
 	u32 w = cf->width / 2;
 
-	if (hdr_flags & VICODEC_FL_CHROMA_FULL_HEIGHT)
+	if (hdr_flags & FWHT_FL_CHROMA_FULL_HEIGHT)
 		h *= 2;
-	if (hdr_flags & VICODEC_FL_CHROMA_FULL_WIDTH)
+	if (hdr_flags & FWHT_FL_CHROMA_FULL_WIDTH)
 		w *= 2;
 	decode_plane(cf, &rlco, ref->luma, cf->height, cf->width,
-		     hdr_flags & VICODEC_FL_LUMA_IS_UNCOMPRESSED);
+		     hdr_flags & FWHT_FL_LUMA_IS_UNCOMPRESSED);
 	decode_plane(cf, &rlco, ref->cb, h, w,
-		     hdr_flags & VICODEC_FL_CB_IS_UNCOMPRESSED);
+		     hdr_flags & FWHT_FL_CB_IS_UNCOMPRESSED);
 	decode_plane(cf, &rlco, ref->cr, h, w,
-		     hdr_flags & VICODEC_FL_CR_IS_UNCOMPRESSED);
+		     hdr_flags & FWHT_FL_CR_IS_UNCOMPRESSED);
 }
diff --git a/drivers/media/platform/vicodec/vicodec-codec.h b/drivers/media/platform/vicodec/codec-fwht.h
similarity index 63%
rename from drivers/media/platform/vicodec/vicodec-codec.h
rename to drivers/media/platform/vicodec/codec-fwht.h
index ff69d9297ef4..1f9e47331197 100644
--- a/drivers/media/platform/vicodec/vicodec-codec.h
+++ b/drivers/media/platform/vicodec/codec-fwht.h
@@ -4,15 +4,15 @@
  * Copyright 2018 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
  */
 
-#ifndef VICODEC_RLC_H
-#define VICODEC_RLC_H
+#ifndef CODEC_FWHT_H
+#define CODEC_FWHT_H
 
 #include <linux/types.h>
 #include <linux/bitops.h>
 #include <asm/byteorder.h>
 
 /*
- * The compressed format consists of a cframe_hdr struct followed by the
+ * The compressed format consists of a fwht_cframe_hdr struct followed by the
  * compressed frame data. The header contains the size of that data.
  * Each Y, Cb and Cr plane is compressed separately. If the compressed
  * size of each plane becomes larger than the uncompressed size, then
@@ -35,7 +35,7 @@
  *
  * All 16 and 32 bit values are stored in big-endian (network) order.
  *
- * Each cframe_hdr starts with an 8 byte magic header that is
+ * Each fwht_cframe_hdr starts with an 8 byte magic header that is
  * guaranteed not to occur in the compressed frame data. This header
  * can be used to sync to the next frame.
  *
@@ -46,51 +46,37 @@
  * https://hverkuil.home.xs4all.nl/fwht.pdf
  */
 
-/*
- * Note: bit 0 of the header must always be 0. Otherwise it cannot
- * be guaranteed that the magic 8 byte sequence (see below) can
- * never occur in the rlc output.
- */
-#define PFRAME_BIT (1 << 15)
-#define DUPS_MASK 0x1ffe
-
 /*
  * This is a sequence of 8 bytes with the low 4 bits set to 0xf.
  *
  * This sequence cannot occur in the encoded data
+ *
+ * Note that these two magic values are symmetrical so endian issues here.
  */
-#define VICODEC_MAGIC1 0x4f4f4f4f
-#define VICODEC_MAGIC2 0xffffffff
-
-#define VICODEC_VERSION 1
-
-#define VICODEC_MAX_WIDTH 3840
-#define VICODEC_MAX_HEIGHT 2160
-#define VICODEC_MIN_WIDTH 640
-#define VICODEC_MIN_HEIGHT 480
+#define FWHT_MAGIC1 0x4f4f4f4f
+#define FWHT_MAGIC2 0xffffffff
 
-#define PBLOCK 0
-#define IBLOCK 1
+#define FWHT_VERSION 1
 
 /* Set if this is an interlaced format */
-#define VICODEC_FL_IS_INTERLACED	BIT(0)
+#define FWHT_FL_IS_INTERLACED		BIT(0)
 /* Set if this is a bottom-first (NTSC) interlaced format */
-#define VICODEC_FL_IS_BOTTOM_FIRST	BIT(1)
+#define FWHT_FL_IS_BOTTOM_FIRST		BIT(1)
 /* Set if each 'frame' contains just one field */
-#define VICODEC_FL_IS_ALTERNATE		BIT(2)
+#define FWHT_FL_IS_ALTERNATE		BIT(2)
 /*
- * If VICODEC_FL_IS_ALTERNATE was set, then this is set if this
+ * If FWHT_FL_IS_ALTERNATE was set, then this is set if this
  * 'frame' is the bottom field, else it is the top field.
  */
-#define VICODEC_FL_IS_BOTTOM_FIELD	BIT(3)
+#define FWHT_FL_IS_BOTTOM_FIELD		BIT(3)
 /* Set if this frame is uncompressed */
-#define VICODEC_FL_LUMA_IS_UNCOMPRESSED	BIT(4)
-#define VICODEC_FL_CB_IS_UNCOMPRESSED	BIT(5)
-#define VICODEC_FL_CR_IS_UNCOMPRESSED	BIT(6)
-#define VICODEC_FL_CHROMA_FULL_HEIGHT	BIT(7)
-#define VICODEC_FL_CHROMA_FULL_WIDTH	BIT(8)
+#define FWHT_FL_LUMA_IS_UNCOMPRESSED	BIT(4)
+#define FWHT_FL_CB_IS_UNCOMPRESSED	BIT(5)
+#define FWHT_FL_CR_IS_UNCOMPRESSED	BIT(6)
+#define FWHT_FL_CHROMA_FULL_HEIGHT	BIT(7)
+#define FWHT_FL_CHROMA_FULL_WIDTH	BIT(8)
 
-struct cframe_hdr {
+struct fwht_cframe_hdr {
 	u32 magic1;
 	u32 magic2;
 	__be32 version;
@@ -103,7 +89,7 @@ struct cframe_hdr {
 	__be32 size;
 };
 
-struct cframe {
+struct fwht_cframe {
 	unsigned int width, height;
 	u16 i_frame_qp;
 	u16 p_frame_qp;
@@ -114,7 +100,7 @@ struct cframe {
 	u32 size;
 };
 
-struct raw_frame {
+struct fwht_raw_frame {
 	unsigned int width, height;
 	unsigned int width_div;
 	unsigned int height_div;
@@ -123,14 +109,17 @@ struct raw_frame {
 	u8 *luma, *cb, *cr;
 };
 
-#define FRAME_PCODED	BIT(0)
-#define FRAME_UNENCODED	BIT(1)
-#define LUMA_UNENCODED	BIT(2)
-#define CB_UNENCODED	BIT(3)
-#define CR_UNENCODED	BIT(4)
+#define FWHT_FRAME_PCODED	BIT(0)
+#define FWHT_FRAME_UNENCODED	BIT(1)
+#define FWHT_LUMA_UNENCODED	BIT(2)
+#define FWHT_CB_UNENCODED	BIT(3)
+#define FWHT_CR_UNENCODED	BIT(4)
 
-u32 encode_frame(struct raw_frame *frm, struct raw_frame *ref_frm,
-		 struct cframe *cf, bool is_intra, bool next_is_intra);
-void decode_frame(struct cframe *cf, struct raw_frame *ref, u32 hdr_flags);
+u32 fwht_encode_frame(struct fwht_raw_frame *frm,
+		      struct fwht_raw_frame *ref_frm,
+		      struct fwht_cframe *cf,
+		      bool is_intra, bool next_is_intra);
+void fwht_decode_frame(struct fwht_cframe *cf, struct fwht_raw_frame *ref,
+		       u32 hdr_flags);
 
 #endif
diff --git a/drivers/media/platform/vicodec/vicodec-core.c b/drivers/media/platform/vicodec/vicodec-core.c
index caff521d94c6..4f2c35533e08 100644
--- a/drivers/media/platform/vicodec/vicodec-core.c
+++ b/drivers/media/platform/vicodec/vicodec-core.c
@@ -23,7 +23,7 @@
 #include <media/v4l2-event.h>
 #include <media/videobuf2-vmalloc.h>
 
-#include "vicodec-codec.h"
+#include "codec-fwht.h"
 
 MODULE_DESCRIPTION("Virtual codec device");
 MODULE_AUTHOR("Hans Verkuil <hans.verkuil@cisco.com>");
@@ -152,7 +152,7 @@ struct vicodec_ctx {
 
 	/* Source and destination queue data */
 	struct vicodec_q_data   q_data[2];
-	struct raw_frame	ref_frame;
+	struct fwht_raw_frame	ref_frame;
 	u8			*compressed_frame;
 	u32			cur_buf_offset;
 	u32			comp_max_size;
@@ -191,9 +191,9 @@ static void encode(struct vicodec_ctx *ctx,
 {
 	unsigned int size = q_data->width * q_data->height;
 	const struct pixfmt_info *info = q_data->info;
-	struct cframe_hdr *p_hdr;
-	struct cframe cf;
-	struct raw_frame rf;
+	struct fwht_cframe_hdr *p_hdr;
+	struct fwht_cframe cf;
+	struct fwht_raw_frame rf;
 	u32 encoding;
 
 	rf.width = q_data->width;
@@ -279,29 +279,29 @@ static void encode(struct vicodec_ctx *ctx,
 	cf.p_frame_qp = ctx->p_frame_qp;
 	cf.rlc_data = (__be16 *)(p_out + sizeof(*p_hdr));
 
-	encoding = encode_frame(&rf, &ctx->ref_frame, &cf, !ctx->gop_cnt,
-				ctx->gop_cnt == ctx->gop_size - 1);
-	if (!(encoding & FRAME_PCODED))
+	encoding = fwht_encode_frame(&rf, &ctx->ref_frame, &cf, !ctx->gop_cnt,
+				     ctx->gop_cnt == ctx->gop_size - 1);
+	if (!(encoding & FWHT_FRAME_PCODED))
 		ctx->gop_cnt = 0;
 	if (++ctx->gop_cnt >= ctx->gop_size)
 		ctx->gop_cnt = 0;
 
-	p_hdr = (struct cframe_hdr *)p_out;
-	p_hdr->magic1 = VICODEC_MAGIC1;
-	p_hdr->magic2 = VICODEC_MAGIC2;
-	p_hdr->version = htonl(VICODEC_VERSION);
+	p_hdr = (struct fwht_cframe_hdr *)p_out;
+	p_hdr->magic1 = FWHT_MAGIC1;
+	p_hdr->magic2 = FWHT_MAGIC2;
+	p_hdr->version = htonl(FWHT_VERSION);
 	p_hdr->width = htonl(cf.width);
 	p_hdr->height = htonl(cf.height);
-	if (encoding & LUMA_UNENCODED)
-		flags |= VICODEC_FL_LUMA_IS_UNCOMPRESSED;
-	if (encoding & CB_UNENCODED)
-		flags |= VICODEC_FL_CB_IS_UNCOMPRESSED;
-	if (encoding & CR_UNENCODED)
-		flags |= VICODEC_FL_CR_IS_UNCOMPRESSED;
+	if (encoding & FWHT_LUMA_UNENCODED)
+		flags |= FWHT_FL_LUMA_IS_UNCOMPRESSED;
+	if (encoding & FWHT_CB_UNENCODED)
+		flags |= FWHT_FL_CB_IS_UNCOMPRESSED;
+	if (encoding & FWHT_CR_UNENCODED)
+		flags |= FWHT_FL_CR_IS_UNCOMPRESSED;
 	if (rf.height_div == 1)
-		flags |= VICODEC_FL_CHROMA_FULL_HEIGHT;
+		flags |= FWHT_FL_CHROMA_FULL_HEIGHT;
 	if (rf.width_div == 1)
-		flags |= VICODEC_FL_CHROMA_FULL_WIDTH;
+		flags |= FWHT_FL_CHROMA_FULL_WIDTH;
 	p_hdr->flags = htonl(flags);
 	p_hdr->colorspace = htonl(ctx->colorspace);
 	p_hdr->xfer_func = htonl(ctx->xfer_func);
@@ -320,11 +320,11 @@ static int decode(struct vicodec_ctx *ctx,
 	unsigned int chroma_size = size;
 	unsigned int i;
 	u32 flags;
-	struct cframe_hdr *p_hdr;
-	struct cframe cf;
+	struct fwht_cframe_hdr *p_hdr;
+	struct fwht_cframe cf;
 	u8 *p;
 
-	p_hdr = (struct cframe_hdr *)p_in;
+	p_hdr = (struct fwht_cframe_hdr *)p_in;
 	cf.width = ntohl(p_hdr->width);
 	cf.height = ntohl(p_hdr->height);
 	flags = ntohl(p_hdr->flags);
@@ -334,13 +334,13 @@ static int decode(struct vicodec_ctx *ctx,
 	ctx->quantization = ntohl(p_hdr->quantization);
 	cf.rlc_data = (__be16 *)(p_in + sizeof(*p_hdr));
 
-	if (p_hdr->magic1 != VICODEC_MAGIC1 ||
-	    p_hdr->magic2 != VICODEC_MAGIC2 ||
-	    ntohl(p_hdr->version) != VICODEC_VERSION ||
-	    cf.width < VICODEC_MIN_WIDTH ||
-	    cf.width > VICODEC_MAX_WIDTH ||
-	    cf.height < VICODEC_MIN_HEIGHT ||
-	    cf.height > VICODEC_MAX_HEIGHT ||
+	if (p_hdr->magic1 != FWHT_MAGIC1 ||
+	    p_hdr->magic2 != FWHT_MAGIC2 ||
+	    ntohl(p_hdr->version) != FWHT_VERSION ||
+	    cf.width < MIN_WIDTH ||
+	    cf.width > MAX_WIDTH ||
+	    cf.height < MIN_HEIGHT ||
+	    cf.height > MAX_HEIGHT ||
 	    (cf.width & 7) || (cf.height & 7))
 		return -EINVAL;
 
@@ -348,12 +348,12 @@ static int decode(struct vicodec_ctx *ctx,
 	if (cf.width != q_data->width || cf.height != q_data->height)
 		return -EINVAL;
 
-	if (!(flags & VICODEC_FL_CHROMA_FULL_WIDTH))
+	if (!(flags & FWHT_FL_CHROMA_FULL_WIDTH))
 		chroma_size /= 2;
-	if (!(flags & VICODEC_FL_CHROMA_FULL_HEIGHT))
+	if (!(flags & FWHT_FL_CHROMA_FULL_HEIGHT))
 		chroma_size /= 2;
 
-	decode_frame(&cf, &ctx->ref_frame, flags);
+	fwht_decode_frame(&cf, &ctx->ref_frame, flags);
 
 	switch (q_data->info->id) {
 	case V4L2_PIX_FMT_YUV420:
@@ -484,7 +484,7 @@ static int device_process(struct vicodec_ctx *ctx,
 	}
 
 	if (ctx->is_enc) {
-		struct cframe_hdr *p_hdr = (struct cframe_hdr *)p_out;
+		struct fwht_cframe_hdr *p_hdr = (struct fwht_cframe_hdr *)p_out;
 
 		encode(ctx, q_out, p_in, p_out, 0);
 		vb2_set_plane_payload(&out_vb->vb2_buf, 0,
@@ -635,9 +635,10 @@ static int job_ready(void *priv)
 		}
 		ctx->comp_size = sizeof(magic);
 	}
-	if (ctx->comp_size < sizeof(struct cframe_hdr)) {
-		struct cframe_hdr *p_hdr = (struct cframe_hdr *)ctx->compressed_frame;
-		u32 copy = sizeof(struct cframe_hdr) - ctx->comp_size;
+	if (ctx->comp_size < sizeof(struct fwht_cframe_hdr)) {
+		struct fwht_cframe_hdr *p_hdr =
+			(struct fwht_cframe_hdr *)ctx->compressed_frame;
+		u32 copy = sizeof(struct fwht_cframe_hdr) - ctx->comp_size;
 
 		if (copy > p_out + sz - p)
 			copy = p_out + sz - p;
@@ -645,7 +646,7 @@ static int job_ready(void *priv)
 		       p, copy);
 		p += copy;
 		ctx->comp_size += copy;
-		if (ctx->comp_size < sizeof(struct cframe_hdr)) {
+		if (ctx->comp_size < sizeof(struct fwht_cframe_hdr)) {
 			job_remove_out_buf(ctx, state);
 			goto restart;
 		}
@@ -670,8 +671,8 @@ static int job_ready(void *priv)
 	ctx->cur_buf_offset = p - p_out;
 	ctx->comp_has_frame = true;
 	ctx->comp_has_next_frame = false;
-	if (sz - ctx->cur_buf_offset >= sizeof(struct cframe_hdr)) {
-		struct cframe_hdr *p_hdr = (struct cframe_hdr *)p;
+	if (sz - ctx->cur_buf_offset >= sizeof(struct fwht_cframe_hdr)) {
+		struct fwht_cframe_hdr *p_hdr = (struct fwht_cframe_hdr *)p;
 		u32 frame_size = ntohl(p_hdr->size);
 		u32 remaining = sz - ctx->cur_buf_offset - sizeof(*p_hdr);
 
@@ -845,7 +846,7 @@ static int vidioc_try_fmt(struct vicodec_ctx *ctx, struct v4l2_format *f)
 		pix->sizeimage = pix->width * pix->height *
 			info->sizeimage_mult / info->sizeimage_div;
 		if (pix->pixelformat == V4L2_PIX_FMT_FWHT)
-			pix->sizeimage += sizeof(struct cframe_hdr);
+			pix->sizeimage += sizeof(struct fwht_cframe_hdr);
 		break;
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
 	case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
@@ -863,7 +864,7 @@ static int vidioc_try_fmt(struct vicodec_ctx *ctx, struct v4l2_format *f)
 		plane->sizeimage = pix_mp->width * pix_mp->height *
 			info->sizeimage_mult / info->sizeimage_div;
 		if (pix_mp->pixelformat == V4L2_PIX_FMT_FWHT)
-			plane->sizeimage += sizeof(struct cframe_hdr);
+			plane->sizeimage += sizeof(struct fwht_cframe_hdr);
 		memset(pix_mp->reserved, 0, sizeof(pix_mp->reserved));
 		memset(plane->reserved, 0, sizeof(plane->reserved));
 		break;
@@ -1308,7 +1309,7 @@ static int vicodec_start_streaming(struct vb2_queue *q,
 	ctx->ref_frame.width = ctx->ref_frame.height = 0;
 	ctx->ref_frame.luma = kvmalloc(size + 2 * size / chroma_div, GFP_KERNEL);
 	ctx->comp_max_size = size + 2 * size / chroma_div +
-			     sizeof(struct cframe_hdr);
+			     sizeof(struct fwht_cframe_hdr);
 	ctx->compressed_frame = kvmalloc(ctx->comp_max_size, GFP_KERNEL);
 	if (!ctx->ref_frame.luma || !ctx->compressed_frame) {
 		kvfree(ctx->ref_frame.luma);
@@ -1493,7 +1494,7 @@ static int vicodec_open(struct file *file)
 	ctx->q_data[V4L2_M2M_DST].sizeimage = size;
 	ctx->colorspace = V4L2_COLORSPACE_REC709;
 
-	size += sizeof(struct cframe_hdr);
+	size += sizeof(struct fwht_cframe_hdr);
 	if (ctx->is_enc) {
 		ctx->q_data[V4L2_M2M_DST].sizeimage = size;
 		ctx->fh.m2m_ctx = v4l2_m2m_ctx_init(dev->enc_dev, ctx,
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCHv2 7/7] vicodec: split off v4l2 specific parts for the codec
  2018-08-23  7:32 [PATCHv2 0/7] vicodec improvements Hans Verkuil
                   ` (5 preceding siblings ...)
  2018-08-23  7:33 ` [PATCHv2 6/7] vicodec: rename and use proper fwht prefix for codec Hans Verkuil
@ 2018-08-23  7:33 ` Hans Verkuil
  2018-08-23  8:10   ` [PATCHv2 8/7] codec-fwht: fix out-of-range values when decoding Hans Verkuil
  2018-08-23  8:13 ` [PATCHv2 0/7] vicodec improvements Hans Verkuil
  7 siblings, 1 reply; 10+ messages in thread
From: Hans Verkuil @ 2018-08-23  7:33 UTC (permalink / raw)
  To: linux-media; +Cc: Hans Verkuil

From: Hans Verkuil <hans.verkuil@cisco.com>

Split off the decode and encode functions into a separate
source that can be reused elsewhere.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/vicodec/Makefile       |   2 +-
 .../media/platform/vicodec/codec-v4l2-fwht.c  | 325 ++++++++++++
 .../media/platform/vicodec/codec-v4l2-fwht.h  |  50 ++
 drivers/media/platform/vicodec/vicodec-core.c | 463 ++++--------------
 4 files changed, 458 insertions(+), 382 deletions(-)
 create mode 100644 drivers/media/platform/vicodec/codec-v4l2-fwht.c
 create mode 100644 drivers/media/platform/vicodec/codec-v4l2-fwht.h

diff --git a/drivers/media/platform/vicodec/Makefile b/drivers/media/platform/vicodec/Makefile
index a27242ff14ad..01bf7e9308a6 100644
--- a/drivers/media/platform/vicodec/Makefile
+++ b/drivers/media/platform/vicodec/Makefile
@@ -1,4 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0
-vicodec-objs := vicodec-core.o codec-fwht.o
+vicodec-objs := vicodec-core.o codec-fwht.o codec-v4l2-fwht.o
 
 obj-$(CONFIG_VIDEO_VICODEC) += vicodec.o
diff --git a/drivers/media/platform/vicodec/codec-v4l2-fwht.c b/drivers/media/platform/vicodec/codec-v4l2-fwht.c
new file mode 100644
index 000000000000..cfcf84b8574d
--- /dev/null
+++ b/drivers/media/platform/vicodec/codec-v4l2-fwht.c
@@ -0,0 +1,325 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * A V4L2 frontend for the FWHT codec
+ *
+ * Copyright 2018 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
+ */
+
+#include <linux/errno.h>
+#include <linux/string.h>
+#include <linux/videodev2.h>
+#include "codec-v4l2-fwht.h"
+
+static const struct v4l2_fwht_pixfmt_info v4l2_fwht_pixfmts[] = {
+	{ V4L2_PIX_FMT_YUV420,  1, 3, 2, 1, 1, 2, 2 },
+	{ V4L2_PIX_FMT_YVU420,  1, 3, 2, 1, 1, 2, 2 },
+	{ V4L2_PIX_FMT_YUV422P, 1, 2, 1, 1, 1, 2, 1 },
+	{ V4L2_PIX_FMT_NV12,    1, 3, 2, 1, 2, 2, 2 },
+	{ V4L2_PIX_FMT_NV21,    1, 3, 2, 1, 2, 2, 2 },
+	{ V4L2_PIX_FMT_NV16,    1, 2, 1, 1, 2, 2, 1 },
+	{ V4L2_PIX_FMT_NV61,    1, 2, 1, 1, 2, 2, 1 },
+	{ V4L2_PIX_FMT_NV24,    1, 3, 1, 1, 2, 1, 1 },
+	{ V4L2_PIX_FMT_NV42,    1, 3, 1, 1, 2, 1, 1 },
+	{ V4L2_PIX_FMT_YUYV,    2, 2, 1, 2, 4, 2, 1 },
+	{ V4L2_PIX_FMT_YVYU,    2, 2, 1, 2, 4, 2, 1 },
+	{ V4L2_PIX_FMT_UYVY,    2, 2, 1, 2, 4, 2, 1 },
+	{ V4L2_PIX_FMT_VYUY,    2, 2, 1, 2, 4, 2, 1 },
+	{ V4L2_PIX_FMT_BGR24,   3, 3, 1, 3, 3, 1, 1 },
+	{ V4L2_PIX_FMT_RGB24,   3, 3, 1, 3, 3, 1, 1 },
+	{ V4L2_PIX_FMT_HSV24,   3, 3, 1, 3, 3, 1, 1 },
+	{ V4L2_PIX_FMT_BGR32,   4, 4, 1, 4, 4, 1, 1 },
+	{ V4L2_PIX_FMT_XBGR32,  4, 4, 1, 4, 4, 1, 1 },
+	{ V4L2_PIX_FMT_RGB32,   4, 4, 1, 4, 4, 1, 1 },
+	{ V4L2_PIX_FMT_XRGB32,  4, 4, 1, 4, 4, 1, 1 },
+	{ V4L2_PIX_FMT_HSV32,   4, 4, 1, 4, 4, 1, 1 },
+};
+
+const struct v4l2_fwht_pixfmt_info *v4l2_fwht_find_pixfmt(u32 pixelformat)
+{
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(v4l2_fwht_pixfmts); i++)
+		if (v4l2_fwht_pixfmts[i].id == pixelformat)
+			return v4l2_fwht_pixfmts + i;
+	return NULL;
+}
+
+const struct v4l2_fwht_pixfmt_info *v4l2_fwht_get_pixfmt(u32 idx)
+{
+	if (idx >= ARRAY_SIZE(v4l2_fwht_pixfmts))
+		return NULL;
+	return v4l2_fwht_pixfmts + idx;
+}
+
+unsigned int v4l2_fwht_encode(struct v4l2_fwht_state *state,
+			      u8 *p_in, u8 *p_out)
+{
+	unsigned int size = state->width * state->height;
+	const struct v4l2_fwht_pixfmt_info *info = state->info;
+	struct fwht_cframe_hdr *p_hdr;
+	struct fwht_cframe cf;
+	struct fwht_raw_frame rf;
+	u32 encoding;
+	u32 flags = 0;
+
+	rf.width = state->width;
+	rf.height = state->height;
+	rf.luma = p_in;
+	rf.width_div = info->width_div;
+	rf.height_div = info->height_div;
+	rf.luma_step = info->luma_step;
+	rf.chroma_step = info->chroma_step;
+
+	switch (info->id) {
+	case V4L2_PIX_FMT_YUV420:
+		rf.cb = rf.luma + size;
+		rf.cr = rf.cb + size / 4;
+		break;
+	case V4L2_PIX_FMT_YVU420:
+		rf.cr = rf.luma + size;
+		rf.cb = rf.cr + size / 4;
+		break;
+	case V4L2_PIX_FMT_YUV422P:
+		rf.cb = rf.luma + size;
+		rf.cr = rf.cb + size / 2;
+		break;
+	case V4L2_PIX_FMT_NV12:
+	case V4L2_PIX_FMT_NV16:
+	case V4L2_PIX_FMT_NV24:
+		rf.cb = rf.luma + size;
+		rf.cr = rf.cb + 1;
+		break;
+	case V4L2_PIX_FMT_NV21:
+	case V4L2_PIX_FMT_NV61:
+	case V4L2_PIX_FMT_NV42:
+		rf.cr = rf.luma + size;
+		rf.cb = rf.cr + 1;
+		break;
+	case V4L2_PIX_FMT_YUYV:
+		rf.cb = rf.luma + 1;
+		rf.cr = rf.cb + 2;
+		break;
+	case V4L2_PIX_FMT_YVYU:
+		rf.cr = rf.luma + 1;
+		rf.cb = rf.cr + 2;
+		break;
+	case V4L2_PIX_FMT_UYVY:
+		rf.cb = rf.luma;
+		rf.cr = rf.cb + 2;
+		rf.luma++;
+		break;
+	case V4L2_PIX_FMT_VYUY:
+		rf.cr = rf.luma;
+		rf.cb = rf.cr + 2;
+		rf.luma++;
+		break;
+	case V4L2_PIX_FMT_RGB24:
+	case V4L2_PIX_FMT_HSV24:
+		rf.cr = rf.luma;
+		rf.cb = rf.cr + 2;
+		rf.luma++;
+		break;
+	case V4L2_PIX_FMT_BGR24:
+		rf.cb = rf.luma;
+		rf.cr = rf.cb + 2;
+		rf.luma++;
+		break;
+	case V4L2_PIX_FMT_RGB32:
+	case V4L2_PIX_FMT_XRGB32:
+	case V4L2_PIX_FMT_HSV32:
+		rf.cr = rf.luma + 1;
+		rf.cb = rf.cr + 2;
+		rf.luma += 2;
+		break;
+	case V4L2_PIX_FMT_BGR32:
+	case V4L2_PIX_FMT_XBGR32:
+		rf.cb = rf.luma;
+		rf.cr = rf.cb + 2;
+		rf.luma++;
+		break;
+	}
+
+	cf.width = state->width;
+	cf.height = state->height;
+	cf.i_frame_qp = state->i_frame_qp;
+	cf.p_frame_qp = state->p_frame_qp;
+	cf.rlc_data = (__be16 *)(p_out + sizeof(*p_hdr));
+
+	encoding = fwht_encode_frame(&rf, &state->ref_frame, &cf,
+				     !state->gop_cnt,
+				     state->gop_cnt == state->gop_size - 1);
+	if (!(encoding & FWHT_FRAME_PCODED))
+		state->gop_cnt = 0;
+	if (++state->gop_cnt >= state->gop_size)
+		state->gop_cnt = 0;
+
+	p_hdr = (struct fwht_cframe_hdr *)p_out;
+	p_hdr->magic1 = FWHT_MAGIC1;
+	p_hdr->magic2 = FWHT_MAGIC2;
+	p_hdr->version = htonl(FWHT_VERSION);
+	p_hdr->width = htonl(cf.width);
+	p_hdr->height = htonl(cf.height);
+	if (encoding & FWHT_LUMA_UNENCODED)
+		flags |= FWHT_FL_LUMA_IS_UNCOMPRESSED;
+	if (encoding & FWHT_CB_UNENCODED)
+		flags |= FWHT_FL_CB_IS_UNCOMPRESSED;
+	if (encoding & FWHT_CR_UNENCODED)
+		flags |= FWHT_FL_CR_IS_UNCOMPRESSED;
+	if (rf.height_div == 1)
+		flags |= FWHT_FL_CHROMA_FULL_HEIGHT;
+	if (rf.width_div == 1)
+		flags |= FWHT_FL_CHROMA_FULL_WIDTH;
+	p_hdr->flags = htonl(flags);
+	p_hdr->colorspace = htonl(state->colorspace);
+	p_hdr->xfer_func = htonl(state->xfer_func);
+	p_hdr->ycbcr_enc = htonl(state->ycbcr_enc);
+	p_hdr->quantization = htonl(state->quantization);
+	p_hdr->size = htonl(cf.size);
+	state->ref_frame.width = cf.width;
+	state->ref_frame.height = cf.height;
+	return cf.size + sizeof(*p_hdr);
+}
+
+int v4l2_fwht_decode(struct v4l2_fwht_state *state,
+		     u8 *p_in, u8 *p_out)
+{
+	unsigned int size = state->width * state->height;
+	unsigned int chroma_size = size;
+	unsigned int i;
+	u32 flags;
+	struct fwht_cframe_hdr *p_hdr;
+	struct fwht_cframe cf;
+	u8 *p;
+
+	p_hdr = (struct fwht_cframe_hdr *)p_in;
+	cf.width = ntohl(p_hdr->width);
+	cf.height = ntohl(p_hdr->height);
+	flags = ntohl(p_hdr->flags);
+	state->colorspace = ntohl(p_hdr->colorspace);
+	state->xfer_func = ntohl(p_hdr->xfer_func);
+	state->ycbcr_enc = ntohl(p_hdr->ycbcr_enc);
+	state->quantization = ntohl(p_hdr->quantization);
+	cf.rlc_data = (__be16 *)(p_in + sizeof(*p_hdr));
+
+	if (p_hdr->magic1 != FWHT_MAGIC1 ||
+	    p_hdr->magic2 != FWHT_MAGIC2 ||
+	    ntohl(p_hdr->version) != FWHT_VERSION ||
+	    (cf.width & 7) || (cf.height & 7))
+		return -EINVAL;
+
+	/* TODO: support resolution changes */
+	if (cf.width != state->width || cf.height != state->height)
+		return -EINVAL;
+
+	if (!(flags & FWHT_FL_CHROMA_FULL_WIDTH))
+		chroma_size /= 2;
+	if (!(flags & FWHT_FL_CHROMA_FULL_HEIGHT))
+		chroma_size /= 2;
+
+	fwht_decode_frame(&cf, &state->ref_frame, flags);
+
+	switch (state->info->id) {
+	case V4L2_PIX_FMT_YUV420:
+	case V4L2_PIX_FMT_YUV422P:
+		memcpy(p_out, state->ref_frame.luma, size);
+		p_out += size;
+		memcpy(p_out, state->ref_frame.cb, chroma_size);
+		p_out += chroma_size;
+		memcpy(p_out, state->ref_frame.cr, chroma_size);
+		break;
+	case V4L2_PIX_FMT_YVU420:
+		memcpy(p_out, state->ref_frame.luma, size);
+		p_out += size;
+		memcpy(p_out, state->ref_frame.cr, chroma_size);
+		p_out += chroma_size;
+		memcpy(p_out, state->ref_frame.cb, chroma_size);
+		break;
+	case V4L2_PIX_FMT_NV12:
+	case V4L2_PIX_FMT_NV16:
+	case V4L2_PIX_FMT_NV24:
+		memcpy(p_out, state->ref_frame.luma, size);
+		p_out += size;
+		for (i = 0, p = p_out; i < chroma_size; i++) {
+			*p++ = state->ref_frame.cb[i];
+			*p++ = state->ref_frame.cr[i];
+		}
+		break;
+	case V4L2_PIX_FMT_NV21:
+	case V4L2_PIX_FMT_NV61:
+	case V4L2_PIX_FMT_NV42:
+		memcpy(p_out, state->ref_frame.luma, size);
+		p_out += size;
+		for (i = 0, p = p_out; i < chroma_size; i++) {
+			*p++ = state->ref_frame.cr[i];
+			*p++ = state->ref_frame.cb[i];
+		}
+		break;
+	case V4L2_PIX_FMT_YUYV:
+		for (i = 0, p = p_out; i < size; i += 2) {
+			*p++ = state->ref_frame.luma[i];
+			*p++ = state->ref_frame.cb[i / 2];
+			*p++ = state->ref_frame.luma[i + 1];
+			*p++ = state->ref_frame.cr[i / 2];
+		}
+		break;
+	case V4L2_PIX_FMT_YVYU:
+		for (i = 0, p = p_out; i < size; i += 2) {
+			*p++ = state->ref_frame.luma[i];
+			*p++ = state->ref_frame.cr[i / 2];
+			*p++ = state->ref_frame.luma[i + 1];
+			*p++ = state->ref_frame.cb[i / 2];
+		}
+		break;
+	case V4L2_PIX_FMT_UYVY:
+		for (i = 0, p = p_out; i < size; i += 2) {
+			*p++ = state->ref_frame.cb[i / 2];
+			*p++ = state->ref_frame.luma[i];
+			*p++ = state->ref_frame.cr[i / 2];
+			*p++ = state->ref_frame.luma[i + 1];
+		}
+		break;
+	case V4L2_PIX_FMT_VYUY:
+		for (i = 0, p = p_out; i < size; i += 2) {
+			*p++ = state->ref_frame.cr[i / 2];
+			*p++ = state->ref_frame.luma[i];
+			*p++ = state->ref_frame.cb[i / 2];
+			*p++ = state->ref_frame.luma[i + 1];
+		}
+		break;
+	case V4L2_PIX_FMT_RGB24:
+	case V4L2_PIX_FMT_HSV24:
+		for (i = 0, p = p_out; i < size; i++) {
+			*p++ = state->ref_frame.cr[i];
+			*p++ = state->ref_frame.luma[i];
+			*p++ = state->ref_frame.cb[i];
+		}
+		break;
+	case V4L2_PIX_FMT_BGR24:
+		for (i = 0, p = p_out; i < size; i++) {
+			*p++ = state->ref_frame.cb[i];
+			*p++ = state->ref_frame.luma[i];
+			*p++ = state->ref_frame.cr[i];
+		}
+		break;
+	case V4L2_PIX_FMT_RGB32:
+	case V4L2_PIX_FMT_XRGB32:
+	case V4L2_PIX_FMT_HSV32:
+		for (i = 0, p = p_out; i < size; i++) {
+			*p++ = 0;
+			*p++ = state->ref_frame.cr[i];
+			*p++ = state->ref_frame.luma[i];
+			*p++ = state->ref_frame.cb[i];
+		}
+		break;
+	case V4L2_PIX_FMT_BGR32:
+	case V4L2_PIX_FMT_XBGR32:
+		for (i = 0, p = p_out; i < size; i++) {
+			*p++ = state->ref_frame.cb[i];
+			*p++ = state->ref_frame.luma[i];
+			*p++ = state->ref_frame.cr[i];
+			*p++ = 0;
+		}
+		break;
+	}
+	return 0;
+}
diff --git a/drivers/media/platform/vicodec/codec-v4l2-fwht.h b/drivers/media/platform/vicodec/codec-v4l2-fwht.h
new file mode 100644
index 000000000000..7794c186d905
--- /dev/null
+++ b/drivers/media/platform/vicodec/codec-v4l2-fwht.h
@@ -0,0 +1,50 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright 2018 Cisco Systems, Inc. and/or its affiliates. All rights reserved.
+ */
+
+#ifndef CODEC_V4L2_FWHT_H
+#define CODEC_V4L2_FWHT_H
+
+#include "codec-fwht.h"
+
+struct v4l2_fwht_pixfmt_info {
+	u32 id;
+	unsigned int bytesperline_mult;
+	unsigned int sizeimage_mult;
+	unsigned int sizeimage_div;
+	unsigned int luma_step;
+	unsigned int chroma_step;
+	/* Chroma plane subsampling */
+	unsigned int width_div;
+	unsigned int height_div;
+};
+
+struct v4l2_fwht_state {
+	const struct v4l2_fwht_pixfmt_info *info;
+	unsigned int width;
+	unsigned int height;
+	unsigned int gop_size;
+	unsigned int gop_cnt;
+	u16 i_frame_qp;
+	u16 p_frame_qp;
+
+	enum v4l2_colorspace colorspace;
+	enum v4l2_ycbcr_encoding ycbcr_enc;
+	enum v4l2_xfer_func xfer_func;
+	enum v4l2_quantization quantization;
+
+	struct fwht_raw_frame ref_frame;
+	u8 *compressed_frame;
+};
+
+const struct v4l2_fwht_pixfmt_info *v4l2_fwht_find_pixfmt(u32 pixelformat);
+const struct v4l2_fwht_pixfmt_info *v4l2_fwht_get_pixfmt(u32 idx);
+
+unsigned int v4l2_fwht_encode(struct v4l2_fwht_state *state,
+			      u8 *p_in, u8 *p_out);
+
+int v4l2_fwht_decode(struct v4l2_fwht_state *state,
+		     u8 *p_in, u8 *p_out);
+
+#endif
diff --git a/drivers/media/platform/vicodec/vicodec-core.c b/drivers/media/platform/vicodec/vicodec-core.c
index 4f2c35533e08..fdd77441a47b 100644
--- a/drivers/media/platform/vicodec/vicodec-core.c
+++ b/drivers/media/platform/vicodec/vicodec-core.c
@@ -23,7 +23,7 @@
 #include <media/v4l2-event.h>
 #include <media/videobuf2-vmalloc.h>
 
-#include "codec-fwht.h"
+#include "codec-v4l2-fwht.h"
 
 MODULE_DESCRIPTION("Virtual codec device");
 MODULE_AUTHOR("Hans Verkuil <hans.verkuil@cisco.com>");
@@ -60,31 +60,7 @@ struct pixfmt_info {
 	unsigned int height_div;
 };
 
-static const struct pixfmt_info pixfmts[] = {
-	{ V4L2_PIX_FMT_YUV420,  1, 3, 2, 1, 1, 2, 2 },
-	{ V4L2_PIX_FMT_YVU420,  1, 3, 2, 1, 1, 2, 2 },
-	{ V4L2_PIX_FMT_YUV422P, 1, 2, 1, 1, 1, 2, 1 },
-	{ V4L2_PIX_FMT_NV12,    1, 3, 2, 1, 2, 2, 2 },
-	{ V4L2_PIX_FMT_NV21,    1, 3, 2, 1, 2, 2, 2 },
-	{ V4L2_PIX_FMT_NV16,    1, 2, 1, 1, 2, 2, 1 },
-	{ V4L2_PIX_FMT_NV61,    1, 2, 1, 1, 2, 2, 1 },
-	{ V4L2_PIX_FMT_NV24,    1, 3, 1, 1, 2, 1, 1 },
-	{ V4L2_PIX_FMT_NV42,    1, 3, 1, 1, 2, 1, 1 },
-	{ V4L2_PIX_FMT_YUYV,    2, 2, 1, 2, 4, 2, 1 },
-	{ V4L2_PIX_FMT_YVYU,    2, 2, 1, 2, 4, 2, 1 },
-	{ V4L2_PIX_FMT_UYVY,    2, 2, 1, 2, 4, 2, 1 },
-	{ V4L2_PIX_FMT_VYUY,    2, 2, 1, 2, 4, 2, 1 },
-	{ V4L2_PIX_FMT_BGR24,   3, 3, 1, 3, 3, 1, 1 },
-	{ V4L2_PIX_FMT_RGB24,   3, 3, 1, 3, 3, 1, 1 },
-	{ V4L2_PIX_FMT_HSV24,   3, 3, 1, 3, 3, 1, 1 },
-	{ V4L2_PIX_FMT_BGR32,   4, 4, 1, 4, 4, 1, 1 },
-	{ V4L2_PIX_FMT_XBGR32,  4, 4, 1, 4, 4, 1, 1 },
-	{ V4L2_PIX_FMT_RGB32,   4, 4, 1, 4, 4, 1, 1 },
-	{ V4L2_PIX_FMT_XRGB32,  4, 4, 1, 4, 4, 1, 1 },
-	{ V4L2_PIX_FMT_HSV32,   4, 4, 1, 4, 4, 1, 1 },
-};
-
-static const struct pixfmt_info pixfmt_fwht = {
+static const struct v4l2_fwht_pixfmt_info pixfmt_fwht = {
 	V4L2_PIX_FMT_FWHT, 0, 3, 1, 1, 1, 1, 1
 };
 
@@ -103,7 +79,7 @@ struct vicodec_q_data {
 	unsigned int		height;
 	unsigned int		sizeimage;
 	unsigned int		sequence;
-	const struct pixfmt_info *info;
+	const struct v4l2_fwht_pixfmt_info *info;
 };
 
 enum {
@@ -135,25 +111,16 @@ struct vicodec_ctx {
 	spinlock_t		*lock;
 
 	struct v4l2_ctrl_handler hdl;
-	unsigned int		gop_size;
-	unsigned int		gop_cnt;
-	u16			i_frame_qp;
-	u16			p_frame_qp;
 
 	/* Abort requested by m2m */
 	int			aborting;
 	struct vb2_v4l2_buffer *last_src_buf;
 	struct vb2_v4l2_buffer *last_dst_buf;
 
-	enum v4l2_colorspace	colorspace;
-	enum v4l2_ycbcr_encoding ycbcr_enc;
-	enum v4l2_xfer_func	xfer_func;
-	enum v4l2_quantization	quantization;
-
 	/* Source and destination queue data */
 	struct vicodec_q_data   q_data[2];
-	struct fwht_raw_frame	ref_frame;
-	u8			*compressed_frame;
+	struct v4l2_fwht_state	state;
+
 	u32			cur_buf_offset;
 	u32			comp_max_size;
 	u32			comp_size;
@@ -185,288 +152,13 @@ static struct vicodec_q_data *get_q_data(struct vicodec_ctx *ctx,
 	return NULL;
 }
 
-static void encode(struct vicodec_ctx *ctx,
-		   struct vicodec_q_data *q_data,
-		   u8 *p_in, u8 *p_out, u32 flags)
-{
-	unsigned int size = q_data->width * q_data->height;
-	const struct pixfmt_info *info = q_data->info;
-	struct fwht_cframe_hdr *p_hdr;
-	struct fwht_cframe cf;
-	struct fwht_raw_frame rf;
-	u32 encoding;
-
-	rf.width = q_data->width;
-	rf.height = q_data->height;
-	rf.luma = p_in;
-	rf.width_div = info->width_div;
-	rf.height_div = info->height_div;
-	rf.luma_step = info->luma_step;
-	rf.chroma_step = info->chroma_step;
-
-	switch (info->id) {
-	case V4L2_PIX_FMT_YUV420:
-		rf.cb = rf.luma + size;
-		rf.cr = rf.cb + size / 4;
-		break;
-	case V4L2_PIX_FMT_YVU420:
-		rf.cr = rf.luma + size;
-		rf.cb = rf.cr + size / 4;
-		break;
-	case V4L2_PIX_FMT_YUV422P:
-		rf.cb = rf.luma + size;
-		rf.cr = rf.cb + size / 2;
-		break;
-	case V4L2_PIX_FMT_NV12:
-	case V4L2_PIX_FMT_NV16:
-	case V4L2_PIX_FMT_NV24:
-		rf.cb = rf.luma + size;
-		rf.cr = rf.cb + 1;
-		break;
-	case V4L2_PIX_FMT_NV21:
-	case V4L2_PIX_FMT_NV61:
-	case V4L2_PIX_FMT_NV42:
-		rf.cr = rf.luma + size;
-		rf.cb = rf.cr + 1;
-		break;
-	case V4L2_PIX_FMT_YUYV:
-		rf.cb = rf.luma + 1;
-		rf.cr = rf.cb + 2;
-		break;
-	case V4L2_PIX_FMT_YVYU:
-		rf.cr = rf.luma + 1;
-		rf.cb = rf.cr + 2;
-		break;
-	case V4L2_PIX_FMT_UYVY:
-		rf.cb = rf.luma;
-		rf.cr = rf.cb + 2;
-		rf.luma++;
-		break;
-	case V4L2_PIX_FMT_VYUY:
-		rf.cr = rf.luma;
-		rf.cb = rf.cr + 2;
-		rf.luma++;
-		break;
-	case V4L2_PIX_FMT_RGB24:
-	case V4L2_PIX_FMT_HSV24:
-		rf.cr = rf.luma;
-		rf.cb = rf.cr + 2;
-		rf.luma++;
-		break;
-	case V4L2_PIX_FMT_BGR24:
-		rf.cb = rf.luma;
-		rf.cr = rf.cb + 2;
-		rf.luma++;
-		break;
-	case V4L2_PIX_FMT_RGB32:
-	case V4L2_PIX_FMT_XRGB32:
-	case V4L2_PIX_FMT_HSV32:
-		rf.cr = rf.luma + 1;
-		rf.cb = rf.cr + 2;
-		rf.luma += 2;
-		break;
-	case V4L2_PIX_FMT_BGR32:
-	case V4L2_PIX_FMT_XBGR32:
-		rf.cb = rf.luma;
-		rf.cr = rf.cb + 2;
-		rf.luma++;
-		break;
-	}
-
-	cf.width = q_data->width;
-	cf.height = q_data->height;
-	cf.i_frame_qp = ctx->i_frame_qp;
-	cf.p_frame_qp = ctx->p_frame_qp;
-	cf.rlc_data = (__be16 *)(p_out + sizeof(*p_hdr));
-
-	encoding = fwht_encode_frame(&rf, &ctx->ref_frame, &cf, !ctx->gop_cnt,
-				     ctx->gop_cnt == ctx->gop_size - 1);
-	if (!(encoding & FWHT_FRAME_PCODED))
-		ctx->gop_cnt = 0;
-	if (++ctx->gop_cnt >= ctx->gop_size)
-		ctx->gop_cnt = 0;
-
-	p_hdr = (struct fwht_cframe_hdr *)p_out;
-	p_hdr->magic1 = FWHT_MAGIC1;
-	p_hdr->magic2 = FWHT_MAGIC2;
-	p_hdr->version = htonl(FWHT_VERSION);
-	p_hdr->width = htonl(cf.width);
-	p_hdr->height = htonl(cf.height);
-	if (encoding & FWHT_LUMA_UNENCODED)
-		flags |= FWHT_FL_LUMA_IS_UNCOMPRESSED;
-	if (encoding & FWHT_CB_UNENCODED)
-		flags |= FWHT_FL_CB_IS_UNCOMPRESSED;
-	if (encoding & FWHT_CR_UNENCODED)
-		flags |= FWHT_FL_CR_IS_UNCOMPRESSED;
-	if (rf.height_div == 1)
-		flags |= FWHT_FL_CHROMA_FULL_HEIGHT;
-	if (rf.width_div == 1)
-		flags |= FWHT_FL_CHROMA_FULL_WIDTH;
-	p_hdr->flags = htonl(flags);
-	p_hdr->colorspace = htonl(ctx->colorspace);
-	p_hdr->xfer_func = htonl(ctx->xfer_func);
-	p_hdr->ycbcr_enc = htonl(ctx->ycbcr_enc);
-	p_hdr->quantization = htonl(ctx->quantization);
-	p_hdr->size = htonl(cf.size);
-	ctx->ref_frame.width = cf.width;
-	ctx->ref_frame.height = cf.height;
-}
-
-static int decode(struct vicodec_ctx *ctx,
-		  struct vicodec_q_data *q_data,
-		  u8 *p_in, u8 *p_out)
-{
-	unsigned int size = q_data->width * q_data->height;
-	unsigned int chroma_size = size;
-	unsigned int i;
-	u32 flags;
-	struct fwht_cframe_hdr *p_hdr;
-	struct fwht_cframe cf;
-	u8 *p;
-
-	p_hdr = (struct fwht_cframe_hdr *)p_in;
-	cf.width = ntohl(p_hdr->width);
-	cf.height = ntohl(p_hdr->height);
-	flags = ntohl(p_hdr->flags);
-	ctx->colorspace = ntohl(p_hdr->colorspace);
-	ctx->xfer_func = ntohl(p_hdr->xfer_func);
-	ctx->ycbcr_enc = ntohl(p_hdr->ycbcr_enc);
-	ctx->quantization = ntohl(p_hdr->quantization);
-	cf.rlc_data = (__be16 *)(p_in + sizeof(*p_hdr));
-
-	if (p_hdr->magic1 != FWHT_MAGIC1 ||
-	    p_hdr->magic2 != FWHT_MAGIC2 ||
-	    ntohl(p_hdr->version) != FWHT_VERSION ||
-	    cf.width < MIN_WIDTH ||
-	    cf.width > MAX_WIDTH ||
-	    cf.height < MIN_HEIGHT ||
-	    cf.height > MAX_HEIGHT ||
-	    (cf.width & 7) || (cf.height & 7))
-		return -EINVAL;
-
-	/* TODO: support resolution changes */
-	if (cf.width != q_data->width || cf.height != q_data->height)
-		return -EINVAL;
-
-	if (!(flags & FWHT_FL_CHROMA_FULL_WIDTH))
-		chroma_size /= 2;
-	if (!(flags & FWHT_FL_CHROMA_FULL_HEIGHT))
-		chroma_size /= 2;
-
-	fwht_decode_frame(&cf, &ctx->ref_frame, flags);
-
-	switch (q_data->info->id) {
-	case V4L2_PIX_FMT_YUV420:
-	case V4L2_PIX_FMT_YUV422P:
-		memcpy(p_out, ctx->ref_frame.luma, size);
-		p_out += size;
-		memcpy(p_out, ctx->ref_frame.cb, chroma_size);
-		p_out += chroma_size;
-		memcpy(p_out, ctx->ref_frame.cr, chroma_size);
-		break;
-	case V4L2_PIX_FMT_YVU420:
-		memcpy(p_out, ctx->ref_frame.luma, size);
-		p_out += size;
-		memcpy(p_out, ctx->ref_frame.cr, chroma_size);
-		p_out += chroma_size;
-		memcpy(p_out, ctx->ref_frame.cb, chroma_size);
-		break;
-	case V4L2_PIX_FMT_NV12:
-	case V4L2_PIX_FMT_NV16:
-	case V4L2_PIX_FMT_NV24:
-		memcpy(p_out, ctx->ref_frame.luma, size);
-		p_out += size;
-		for (i = 0, p = p_out; i < chroma_size; i++) {
-			*p++ = ctx->ref_frame.cb[i];
-			*p++ = ctx->ref_frame.cr[i];
-		}
-		break;
-	case V4L2_PIX_FMT_NV21:
-	case V4L2_PIX_FMT_NV61:
-	case V4L2_PIX_FMT_NV42:
-		memcpy(p_out, ctx->ref_frame.luma, size);
-		p_out += size;
-		for (i = 0, p = p_out; i < chroma_size; i++) {
-			*p++ = ctx->ref_frame.cr[i];
-			*p++ = ctx->ref_frame.cb[i];
-		}
-		break;
-	case V4L2_PIX_FMT_YUYV:
-		for (i = 0, p = p_out; i < size; i += 2) {
-			*p++ = ctx->ref_frame.luma[i];
-			*p++ = ctx->ref_frame.cb[i / 2];
-			*p++ = ctx->ref_frame.luma[i + 1];
-			*p++ = ctx->ref_frame.cr[i / 2];
-		}
-		break;
-	case V4L2_PIX_FMT_YVYU:
-		for (i = 0, p = p_out; i < size; i += 2) {
-			*p++ = ctx->ref_frame.luma[i];
-			*p++ = ctx->ref_frame.cr[i / 2];
-			*p++ = ctx->ref_frame.luma[i + 1];
-			*p++ = ctx->ref_frame.cb[i / 2];
-		}
-		break;
-	case V4L2_PIX_FMT_UYVY:
-		for (i = 0, p = p_out; i < size; i += 2) {
-			*p++ = ctx->ref_frame.cb[i / 2];
-			*p++ = ctx->ref_frame.luma[i];
-			*p++ = ctx->ref_frame.cr[i / 2];
-			*p++ = ctx->ref_frame.luma[i + 1];
-		}
-		break;
-	case V4L2_PIX_FMT_VYUY:
-		for (i = 0, p = p_out; i < size; i += 2) {
-			*p++ = ctx->ref_frame.cr[i / 2];
-			*p++ = ctx->ref_frame.luma[i];
-			*p++ = ctx->ref_frame.cb[i / 2];
-			*p++ = ctx->ref_frame.luma[i + 1];
-		}
-		break;
-	case V4L2_PIX_FMT_RGB24:
-	case V4L2_PIX_FMT_HSV24:
-		for (i = 0, p = p_out; i < size; i++) {
-			*p++ = ctx->ref_frame.cr[i];
-			*p++ = ctx->ref_frame.luma[i];
-			*p++ = ctx->ref_frame.cb[i];
-		}
-		break;
-	case V4L2_PIX_FMT_BGR24:
-		for (i = 0, p = p_out; i < size; i++) {
-			*p++ = ctx->ref_frame.cb[i];
-			*p++ = ctx->ref_frame.luma[i];
-			*p++ = ctx->ref_frame.cr[i];
-		}
-		break;
-	case V4L2_PIX_FMT_RGB32:
-	case V4L2_PIX_FMT_XRGB32:
-	case V4L2_PIX_FMT_HSV32:
-		for (i = 0, p = p_out; i < size; i++) {
-			*p++ = 0;
-			*p++ = ctx->ref_frame.cr[i];
-			*p++ = ctx->ref_frame.luma[i];
-			*p++ = ctx->ref_frame.cb[i];
-		}
-		break;
-	case V4L2_PIX_FMT_BGR32:
-	case V4L2_PIX_FMT_XBGR32:
-		for (i = 0, p = p_out; i < size; i++) {
-			*p++ = ctx->ref_frame.cb[i];
-			*p++ = ctx->ref_frame.luma[i];
-			*p++ = ctx->ref_frame.cr[i];
-			*p++ = 0;
-		}
-		break;
-	}
-	return 0;
-}
-
 static int device_process(struct vicodec_ctx *ctx,
 			  struct vb2_v4l2_buffer *in_vb,
 			  struct vb2_v4l2_buffer *out_vb)
 {
 	struct vicodec_dev *dev = ctx->dev;
 	struct vicodec_q_data *q_out, *q_cap;
+	struct v4l2_fwht_state *state = &ctx->state;
 	u8 *p_in, *p_out;
 	int ret;
 
@@ -475,7 +167,7 @@ static int device_process(struct vicodec_ctx *ctx,
 	if (ctx->is_enc)
 		p_in = vb2_plane_vaddr(&in_vb->vb2_buf, 0);
 	else
-		p_in = ctx->compressed_frame;
+		p_in = state->compressed_frame;
 	p_out = vb2_plane_vaddr(&out_vb->vb2_buf, 0);
 	if (!p_in || !p_out) {
 		v4l2_err(&dev->v4l2_dev,
@@ -484,13 +176,11 @@ static int device_process(struct vicodec_ctx *ctx,
 	}
 
 	if (ctx->is_enc) {
-		struct fwht_cframe_hdr *p_hdr = (struct fwht_cframe_hdr *)p_out;
+		unsigned int size = v4l2_fwht_encode(state, p_in, p_out);
 
-		encode(ctx, q_out, p_in, p_out, 0);
-		vb2_set_plane_payload(&out_vb->vb2_buf, 0,
-				      sizeof(*p_hdr) + ntohl(p_hdr->size));
+		vb2_set_plane_payload(&out_vb->vb2_buf, 0, size);
 	} else {
-		ret = decode(ctx, q_cap, p_in, p_out);
+		ret = v4l2_fwht_decode(state, p_in, p_out);
 		if (ret)
 			return ret;
 		vb2_set_plane_payload(&out_vb->vb2_buf, 0, q_cap->sizeimage);
@@ -619,10 +309,11 @@ static int job_ready(void *priv)
 			copy = sizeof(magic) - ctx->comp_magic_cnt;
 			if (p_out + sz - p < copy)
 				copy = p_out + sz - p;
-			memcpy(ctx->compressed_frame + ctx->comp_magic_cnt,
+			memcpy(ctx->state.compressed_frame + ctx->comp_magic_cnt,
 			       p, copy);
 			ctx->comp_magic_cnt += copy;
-			if (!memcmp(ctx->compressed_frame, magic, ctx->comp_magic_cnt)) {
+			if (!memcmp(ctx->state.compressed_frame, magic,
+				    ctx->comp_magic_cnt)) {
 				p += copy;
 				state = VB2_BUF_STATE_DONE;
 				break;
@@ -637,12 +328,12 @@ static int job_ready(void *priv)
 	}
 	if (ctx->comp_size < sizeof(struct fwht_cframe_hdr)) {
 		struct fwht_cframe_hdr *p_hdr =
-			(struct fwht_cframe_hdr *)ctx->compressed_frame;
+			(struct fwht_cframe_hdr *)ctx->state.compressed_frame;
 		u32 copy = sizeof(struct fwht_cframe_hdr) - ctx->comp_size;
 
 		if (copy > p_out + sz - p)
 			copy = p_out + sz - p;
-		memcpy(ctx->compressed_frame + ctx->comp_size,
+		memcpy(ctx->state.compressed_frame + ctx->comp_size,
 		       p, copy);
 		p += copy;
 		ctx->comp_size += copy;
@@ -659,7 +350,7 @@ static int job_ready(void *priv)
 
 		if (copy > p_out + sz - p)
 			copy = p_out + sz - p;
-		memcpy(ctx->compressed_frame + ctx->comp_size,
+		memcpy(ctx->state.compressed_frame + ctx->comp_size,
 		       p, copy);
 		p += copy;
 		ctx->comp_size += copy;
@@ -694,14 +385,14 @@ static void job_abort(void *priv)
  * video ioctls
  */
 
-static const struct pixfmt_info *find_fmt(u32 fmt)
+static const struct v4l2_fwht_pixfmt_info *find_fmt(u32 fmt)
 {
-	unsigned int i;
+	const struct v4l2_fwht_pixfmt_info *info =
+		v4l2_fwht_find_pixfmt(fmt);
 
-	for (i = 0; i < ARRAY_SIZE(pixfmts); i++)
-		if (pixfmts[i].id == fmt)
-			return &pixfmts[i];
-	return &pixfmts[0];
+	if (!info)
+		info = v4l2_fwht_get_pixfmt(0);
+	return info;
 }
 
 static int vidioc_querycap(struct file *file, void *priv,
@@ -727,13 +418,19 @@ static int enum_fmt(struct v4l2_fmtdesc *f, bool is_enc, bool is_out)
 		return -EINVAL;
 	if (!V4L2_TYPE_IS_MULTIPLANAR(f->type) && multiplanar)
 		return -EINVAL;
-	if (f->index >= (is_uncomp ? ARRAY_SIZE(pixfmts) : 1))
-		return -EINVAL;
 
-	if (is_uncomp)
-		f->pixelformat = pixfmts[f->index].id;
-	else
+	if (is_uncomp) {
+		const struct v4l2_fwht_pixfmt_info *info =
+			v4l2_fwht_get_pixfmt(f->index);
+
+		if (!info)
+			return -EINVAL;
+		f->pixelformat = info->id;
+	} else {
+		if (f->index)
+			return -EINVAL;
 		f->pixelformat = V4L2_PIX_FMT_FWHT;
+	}
 	return 0;
 }
 
@@ -759,7 +456,7 @@ static int vidioc_g_fmt(struct vicodec_ctx *ctx, struct v4l2_format *f)
 	struct vicodec_q_data *q_data;
 	struct v4l2_pix_format_mplane *pix_mp;
 	struct v4l2_pix_format *pix;
-	const struct pixfmt_info *info;
+	const struct v4l2_fwht_pixfmt_info *info;
 
 	vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, f->type);
 	if (!vq)
@@ -780,10 +477,10 @@ static int vidioc_g_fmt(struct vicodec_ctx *ctx, struct v4l2_format *f)
 		pix->pixelformat = info->id;
 		pix->bytesperline = q_data->width * info->bytesperline_mult;
 		pix->sizeimage = q_data->sizeimage;
-		pix->colorspace = ctx->colorspace;
-		pix->xfer_func = ctx->xfer_func;
-		pix->ycbcr_enc = ctx->ycbcr_enc;
-		pix->quantization = ctx->quantization;
+		pix->colorspace = ctx->state.colorspace;
+		pix->xfer_func = ctx->state.xfer_func;
+		pix->ycbcr_enc = ctx->state.ycbcr_enc;
+		pix->quantization = ctx->state.quantization;
 		break;
 
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
@@ -799,10 +496,10 @@ static int vidioc_g_fmt(struct vicodec_ctx *ctx, struct v4l2_format *f)
 		pix_mp->plane_fmt[0].bytesperline =
 				q_data->width * info->bytesperline_mult;
 		pix_mp->plane_fmt[0].sizeimage = q_data->sizeimage;
-		pix_mp->colorspace = ctx->colorspace;
-		pix_mp->xfer_func = ctx->xfer_func;
-		pix_mp->ycbcr_enc = ctx->ycbcr_enc;
-		pix_mp->quantization = ctx->quantization;
+		pix_mp->colorspace = ctx->state.colorspace;
+		pix_mp->xfer_func = ctx->state.xfer_func;
+		pix_mp->ycbcr_enc = ctx->state.ycbcr_enc;
+		pix_mp->quantization = ctx->state.quantization;
 		memset(pix_mp->reserved, 0, sizeof(pix_mp->reserved));
 		memset(pix_mp->plane_fmt[0].reserved, 0,
 		       sizeof(pix_mp->plane_fmt[0].reserved));
@@ -830,7 +527,7 @@ static int vidioc_try_fmt(struct vicodec_ctx *ctx, struct v4l2_format *f)
 	struct v4l2_pix_format_mplane *pix_mp;
 	struct v4l2_pix_format *pix;
 	struct v4l2_plane_pix_format *plane;
-	const struct pixfmt_info *info = &pixfmt_fwht;
+	const struct v4l2_fwht_pixfmt_info *info = &pixfmt_fwht;
 
 	switch (f->type) {
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE:
@@ -889,10 +586,10 @@ static int vidioc_try_fmt_vid_cap(struct file *file, void *priv,
 		pix = &f->fmt.pix;
 		pix->pixelformat = ctx->is_enc ? V4L2_PIX_FMT_FWHT :
 				   find_fmt(f->fmt.pix.pixelformat)->id;
-		pix->colorspace = ctx->colorspace;
-		pix->xfer_func = ctx->xfer_func;
-		pix->ycbcr_enc = ctx->ycbcr_enc;
-		pix->quantization = ctx->quantization;
+		pix->colorspace = ctx->state.colorspace;
+		pix->xfer_func = ctx->state.xfer_func;
+		pix->ycbcr_enc = ctx->state.ycbcr_enc;
+		pix->quantization = ctx->state.quantization;
 		break;
 	case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
 		if (!multiplanar)
@@ -900,10 +597,10 @@ static int vidioc_try_fmt_vid_cap(struct file *file, void *priv,
 		pix_mp = &f->fmt.pix_mp;
 		pix_mp->pixelformat = ctx->is_enc ? V4L2_PIX_FMT_FWHT :
 				      find_fmt(pix_mp->pixelformat)->id;
-		pix_mp->colorspace = ctx->colorspace;
-		pix_mp->xfer_func = ctx->xfer_func;
-		pix_mp->ycbcr_enc = ctx->ycbcr_enc;
-		pix_mp->quantization = ctx->quantization;
+		pix_mp->colorspace = ctx->state.colorspace;
+		pix_mp->xfer_func = ctx->state.xfer_func;
+		pix_mp->ycbcr_enc = ctx->state.ycbcr_enc;
+		pix_mp->quantization = ctx->state.quantization;
 		break;
 	default:
 		return -EINVAL;
@@ -1043,18 +740,18 @@ static int vidioc_s_fmt_vid_out(struct file *file, void *priv,
 		case V4L2_BUF_TYPE_VIDEO_CAPTURE:
 		case V4L2_BUF_TYPE_VIDEO_OUTPUT:
 			pix = &f->fmt.pix;
-			ctx->colorspace = pix->colorspace;
-			ctx->xfer_func = pix->xfer_func;
-			ctx->ycbcr_enc = pix->ycbcr_enc;
-			ctx->quantization = pix->quantization;
+			ctx->state.colorspace = pix->colorspace;
+			ctx->state.xfer_func = pix->xfer_func;
+			ctx->state.ycbcr_enc = pix->ycbcr_enc;
+			ctx->state.quantization = pix->quantization;
 			break;
 		case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:
 		case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:
 			pix_mp = &f->fmt.pix_mp;
-			ctx->colorspace = pix_mp->colorspace;
-			ctx->xfer_func = pix_mp->xfer_func;
-			ctx->ycbcr_enc = pix_mp->ycbcr_enc;
-			ctx->quantization = pix_mp->quantization;
+			ctx->state.colorspace = pix_mp->colorspace;
+			ctx->state.xfer_func = pix_mp->xfer_func;
+			ctx->state.ycbcr_enc = pix_mp->ycbcr_enc;
+			ctx->state.quantization = pix_mp->quantization;
 			break;
 		default:
 			break;
@@ -1297,8 +994,9 @@ static int vicodec_start_streaming(struct vb2_queue *q,
 {
 	struct vicodec_ctx *ctx = vb2_get_drv_priv(q);
 	struct vicodec_q_data *q_data = get_q_data(ctx, q->type);
+	struct v4l2_fwht_state *state = &ctx->state;
 	unsigned int size = q_data->width * q_data->height;
-	const struct pixfmt_info *info = q_data->info;
+	const struct v4l2_fwht_pixfmt_info *info = q_data->info;
 	unsigned int chroma_div = info->width_div * info->height_div;
 
 	q_data->sequence = 0;
@@ -1306,22 +1004,25 @@ static int vicodec_start_streaming(struct vb2_queue *q,
 	if (!V4L2_TYPE_IS_OUTPUT(q->type))
 		return 0;
 
-	ctx->ref_frame.width = ctx->ref_frame.height = 0;
-	ctx->ref_frame.luma = kvmalloc(size + 2 * size / chroma_div, GFP_KERNEL);
+	state->width = q_data->width;
+	state->height = q_data->height;
+	state->ref_frame.width = state->ref_frame.height = 0;
+	state->ref_frame.luma = kvmalloc(size + 2 * size / chroma_div,
+					 GFP_KERNEL);
 	ctx->comp_max_size = size + 2 * size / chroma_div +
 			     sizeof(struct fwht_cframe_hdr);
-	ctx->compressed_frame = kvmalloc(ctx->comp_max_size, GFP_KERNEL);
-	if (!ctx->ref_frame.luma || !ctx->compressed_frame) {
-		kvfree(ctx->ref_frame.luma);
-		kvfree(ctx->compressed_frame);
+	state->compressed_frame = kvmalloc(ctx->comp_max_size, GFP_KERNEL);
+	if (!state->ref_frame.luma || !state->compressed_frame) {
+		kvfree(state->ref_frame.luma);
+		kvfree(state->compressed_frame);
 		vicodec_return_bufs(q, VB2_BUF_STATE_QUEUED);
 		return -ENOMEM;
 	}
-	ctx->ref_frame.cb = ctx->ref_frame.luma + size;
-	ctx->ref_frame.cr = ctx->ref_frame.cb + size / chroma_div;
+	state->ref_frame.cb = state->ref_frame.luma + size;
+	state->ref_frame.cr = state->ref_frame.cb + size / chroma_div;
 	ctx->last_src_buf = NULL;
 	ctx->last_dst_buf = NULL;
-	ctx->gop_cnt = 0;
+	state->gop_cnt = 0;
 	ctx->cur_buf_offset = 0;
 	ctx->comp_size = 0;
 	ctx->comp_magic_cnt = 0;
@@ -1339,8 +1040,8 @@ static void vicodec_stop_streaming(struct vb2_queue *q)
 	if (!V4L2_TYPE_IS_OUTPUT(q->type))
 		return;
 
-	kvfree(ctx->ref_frame.luma);
-	kvfree(ctx->compressed_frame);
+	kvfree(ctx->state.ref_frame.luma);
+	kvfree(ctx->state.compressed_frame);
 }
 
 static const struct vb2_ops vicodec_qops = {
@@ -1400,19 +1101,19 @@ static int vicodec_s_ctrl(struct v4l2_ctrl *ctrl)
 
 	switch (ctrl->id) {
 	case V4L2_CID_MPEG_VIDEO_GOP_SIZE:
-		ctx->gop_size = ctrl->val;
+		ctx->state.gop_size = ctrl->val;
 		return 0;
 	case VICODEC_CID_I_FRAME_QP:
-		ctx->i_frame_qp = ctrl->val;
+		ctx->state.i_frame_qp = ctrl->val;
 		return 0;
 	case VICODEC_CID_P_FRAME_QP:
-		ctx->p_frame_qp = ctrl->val;
+		ctx->state.p_frame_qp = ctrl->val;
 		return 0;
 	}
 	return -EINVAL;
 }
 
-static const struct v4l2_ctrl_ops vicodec_ctrl_ops = {
+static struct v4l2_ctrl_ops vicodec_ctrl_ops = {
 	.s_ctrl = vicodec_s_ctrl,
 };
 
@@ -1480,7 +1181,7 @@ static int vicodec_open(struct file *file)
 	v4l2_ctrl_handler_setup(hdl);
 
 	ctx->q_data[V4L2_M2M_SRC].info =
-		ctx->is_enc ? pixfmts : &pixfmt_fwht;
+		ctx->is_enc ? v4l2_fwht_get_pixfmt(0) : &pixfmt_fwht;
 	ctx->q_data[V4L2_M2M_SRC].width = 1280;
 	ctx->q_data[V4L2_M2M_SRC].height = 720;
 	size = 1280 * 720 * ctx->q_data[V4L2_M2M_SRC].info->sizeimage_mult /
@@ -1488,11 +1189,11 @@ static int vicodec_open(struct file *file)
 	ctx->q_data[V4L2_M2M_SRC].sizeimage = size;
 	ctx->q_data[V4L2_M2M_DST] = ctx->q_data[V4L2_M2M_SRC];
 	ctx->q_data[V4L2_M2M_DST].info =
-		ctx->is_enc ? &pixfmt_fwht : pixfmts;
+		ctx->is_enc ? &pixfmt_fwht : v4l2_fwht_get_pixfmt(0);
 	size = 1280 * 720 * ctx->q_data[V4L2_M2M_DST].info->sizeimage_mult /
 		ctx->q_data[V4L2_M2M_DST].info->sizeimage_div;
 	ctx->q_data[V4L2_M2M_DST].sizeimage = size;
-	ctx->colorspace = V4L2_COLORSPACE_REC709;
+	ctx->state.colorspace = V4L2_COLORSPACE_REC709;
 
 	size += sizeof(struct fwht_cframe_hdr);
 	if (ctx->is_enc) {
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCHv2 8/7] codec-fwht: fix out-of-range values when decoding
  2018-08-23  7:33 ` [PATCHv2 7/7] vicodec: split off v4l2 specific parts for the codec Hans Verkuil
@ 2018-08-23  8:10   ` Hans Verkuil
  0 siblings, 0 replies; 10+ messages in thread
From: Hans Verkuil @ 2018-08-23  8:10 UTC (permalink / raw)
  To: linux-media; +Cc: Hans Verkuil

While decoding you need to make sure you do not get values < 0
or > 255. Note that since this code will also be used in userspace
utilities the clamp macro isn't used since that is kernel-only.

Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/platform/vicodec/codec-fwht.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/vicodec/codec-fwht.c b/drivers/media/platform/vicodec/codec-fwht.c
index f91f90f7e5fc..47939160560e 100644
--- a/drivers/media/platform/vicodec/codec-fwht.c
+++ b/drivers/media/platform/vicodec/codec-fwht.c
@@ -625,8 +625,14 @@ static void fill_decoder_block(u8 *dst, const s16 *input, int stride)
 	int i, j;

 	for (i = 0; i < 8; i++) {
-		for (j = 0; j < 8; j++)
-			*dst++ = *input++;
+		for (j = 0; j < 8; j++, input++, dst++) {
+			if (*input < 0)
+				*dst = 0;
+			else if (*input > 255)
+				*dst = 255;
+			else
+				*dst = *input;
+		}
 		dst += stride - 8;
 	}
 }
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [PATCHv2 0/7] vicodec improvements
  2018-08-23  7:32 [PATCHv2 0/7] vicodec improvements Hans Verkuil
                   ` (6 preceding siblings ...)
  2018-08-23  7:33 ` [PATCHv2 7/7] vicodec: split off v4l2 specific parts for the codec Hans Verkuil
@ 2018-08-23  8:13 ` Hans Verkuil
  7 siblings, 0 replies; 10+ messages in thread
From: Hans Verkuil @ 2018-08-23  8:13 UTC (permalink / raw)
  To: linux-media

On 08/23/2018 09:32 AM, Hans Verkuil wrote:
> From: Hans Verkuil <hans.verkuil@cisco.com>
> 
> - add support for quantization parameters
> - support many more pixel formats
> - code simplifications
> - rename source and use proper prefixes for the codec: this makes it
>   independent from the vicodec driver and easier to reuse in userspace
>   (similar to what we do for the v4l2-tpg code).
> - split off the v4l2 'frontend' code for the FWHT codec into its own
>   source for easier re-use elsewhere (i.e. v4l2-ctl/qvidcap).
> 
> I made a v4l-utils branch that uses the FWHT codec to compress video
> when streaming over the network:
> 
> https://git.linuxtv.org/hverkuil/v4l-utils.git/log/?h=qvidcap
> 
> You need to add the --stream-to-host-lossy flag to enable FWHT streaming.
> 
> Note: the FWHT codec clips R/G/B values for RGB formats. This will be
> addressed later. I might have to convert the R/G/B values from full to
> limited range before encoding them, but I want to discuss this with the
> author of the codec (Tom aan de Wiel) first.

Figured out where the problem is and posted a patch 8/7 to fix this.

Since I no longer have these artifacts I also changed the --stream-to-host-lossy
argument to --stream-lossless: by default v4l2-ctl --stream-to-host will now
use the FWHT codec (if supported for the given pixelformat), unless --stream-lossless
is given.

Still not 100% certain about this, so this might change in the future.

Regards,

	Hans

> 
> Regards,
> 
> 	Hans
> 
> Changes since v1:
> 
> - added the last patch (split off v4l2 FWHT code)
> - the GOP_SIZE and QP controls can now be set during streaming as
>   well.
> 
> Hans Verkuil (7):
>   vicodec: add QP controls
>   vicodec: add support for more pixel formats
>   vicodec: simplify flags handling
>   vicodec: simplify blocktype checking
>   vicodec: improve handling of uncompressable planes
>   vicodec: rename and use proper fwht prefix for codec
>   vicodec: split off v4l2 specific parts for the codec
> 
>  .../media/uapi/v4l/pixfmt-compressed.rst      |   2 +-
>  drivers/media/platform/vicodec/Makefile       |   2 +-
>  .../vicodec/{vicodec-codec.c => codec-fwht.c} | 148 ++++--
>  .../vicodec/{vicodec-codec.h => codec-fwht.h} |  80 ++-
>  .../media/platform/vicodec/codec-v4l2-fwht.c  | 325 ++++++++++++
>  .../media/platform/vicodec/codec-v4l2-fwht.h  |  50 ++
>  drivers/media/platform/vicodec/vicodec-core.c | 483 ++++++++----------
>  7 files changed, 723 insertions(+), 367 deletions(-)
>  rename drivers/media/platform/vicodec/{vicodec-codec.c => codec-fwht.c} (85%)
>  rename drivers/media/platform/vicodec/{vicodec-codec.h => codec-fwht.h} (64%)
>  create mode 100644 drivers/media/platform/vicodec/codec-v4l2-fwht.c
>  create mode 100644 drivers/media/platform/vicodec/codec-v4l2-fwht.h
> 

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2018-08-23 11:42 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-08-23  7:32 [PATCHv2 0/7] vicodec improvements Hans Verkuil
2018-08-23  7:32 ` [PATCHv2 1/7] vicodec: add QP controls Hans Verkuil
2018-08-23  7:33 ` [PATCHv2 2/7] vicodec: add support for more pixel formats Hans Verkuil
2018-08-23  7:33 ` [PATCHv2 3/7] vicodec: simplify flags handling Hans Verkuil
2018-08-23  7:33 ` [PATCHv2 4/7] vicodec: simplify blocktype checking Hans Verkuil
2018-08-23  7:33 ` [PATCHv2 5/7] vicodec: improve handling of uncompressable planes Hans Verkuil
2018-08-23  7:33 ` [PATCHv2 6/7] vicodec: rename and use proper fwht prefix for codec Hans Verkuil
2018-08-23  7:33 ` [PATCHv2 7/7] vicodec: split off v4l2 specific parts for the codec Hans Verkuil
2018-08-23  8:10   ` [PATCHv2 8/7] codec-fwht: fix out-of-range values when decoding Hans Verkuil
2018-08-23  8:13 ` [PATCHv2 0/7] vicodec improvements Hans Verkuil

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.