[PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[Jammy Huang]

The aim of this series is to add aspeed-jpeg support for aspeed-video
driver.

To achieve this major goal some refactors are included.

In the last, debugfs information is also updated per this change.

Changes in v5:
 - Use model data to tell different soc

Changes in v4:
 - Add definition for the Aspeed JPEG format
 - Reserve controls for ASPEED
 - Use s_fmt to update format rather than new control
 - Update aspeed hq quality range, 1 ~ 12


Jammy Huang (10):
  media: aspeed: move err-handling together to the bottom
  media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log
  media: aspeed: add more debug log messages
  media: aspeed: refactor to gather format/compress settings
  media: v4l: Add definition for the Aspeed JPEG format
  media: v4l2-ctrls: Reserve controls for ASPEED
  media: aspeed: use model-data
  media: aspeed: Support aspeed mode to reduce compressed data
  media: aspeed: add comments and macro
  media: aspeed: Extend debug message

 .../media/uapi/v4l/pixfmt-reserved.rst        |  12 +
 drivers/media/platform/aspeed-video.c         | 534 ++++++++++++++----
 drivers/media/v4l2-core/v4l2-ioctl.c          |   1 +
 include/uapi/linux/aspeed-video.h             |  15 +
 include/uapi/linux/v4l2-controls.h            |   5 +
 include/uapi/linux/videodev2.h                |   1 +
 6 files changed, 467 insertions(+), 101 deletions(-)
 create mode 100644 include/uapi/linux/aspeed-video.h

-- 
2.25.1

[PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[Jammy Huang]

The aim of this series is to add aspeed-jpeg support for aspeed-video
driver.

To achieve this major goal some refactors are included.

In the last, debugfs information is also updated per this change.

Changes in v5:
 - Use model data to tell different soc

Changes in v4:
 - Add definition for the Aspeed JPEG format
 - Reserve controls for ASPEED
 - Use s_fmt to update format rather than new control
 - Update aspeed hq quality range, 1 ~ 12


Jammy Huang (10):
  media: aspeed: move err-handling together to the bottom
  media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log
  media: aspeed: add more debug log messages
  media: aspeed: refactor to gather format/compress settings
  media: v4l: Add definition for the Aspeed JPEG format
  media: v4l2-ctrls: Reserve controls for ASPEED
  media: aspeed: use model-data
  media: aspeed: Support aspeed mode to reduce compressed data
  media: aspeed: add comments and macro
  media: aspeed: Extend debug message

 .../media/uapi/v4l/pixfmt-reserved.rst        |  12 +
 drivers/media/platform/aspeed-video.c         | 534 ++++++++++++++----
 drivers/media/v4l2-core/v4l2-ioctl.c          |   1 +
 include/uapi/linux/aspeed-video.h             |  15 +
 include/uapi/linux/v4l2-controls.h            |   5 +
 include/uapi/linux/videodev2.h                |   1 +
 6 files changed, 467 insertions(+), 101 deletions(-)
 create mode 100644 include/uapi/linux/aspeed-video.h

-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v5 01/10] media: aspeed: move err-handling together to the bottom

[Jammy Huang]

refine aspeed_video_setup_video() flow.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5:
  - no update
v4:
  - no update
v3:
  - no update
v2:
  - no update
---
 drivers/media/platform/aspeed-video.c | 24 +++++++++++-------------
 1 file changed, 11 insertions(+), 13 deletions(-)

diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index fea5e4d0927e..f5c40d6b4ece 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -1641,11 +1641,8 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
 
 	rc = video->ctrl_handler.error;
 	if (rc) {
-		v4l2_ctrl_handler_free(&video->ctrl_handler);
-		v4l2_device_unregister(v4l2_dev);
-
 		dev_err(video->dev, "Failed to init controls: %d\n", rc);
-		return rc;
+		goto err_ctrl_init;
 	}
 
 	v4l2_dev->ctrl_handler = &video->ctrl_handler;
@@ -1663,11 +1660,8 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
 
 	rc = vb2_queue_init(vbq);
 	if (rc) {
-		v4l2_ctrl_handler_free(&video->ctrl_handler);
-		v4l2_device_unregister(v4l2_dev);
-
 		dev_err(video->dev, "Failed to init vb2 queue\n");
-		return rc;
+		goto err_vb2_init;
 	}
 
 	vdev->queue = vbq;
@@ -1685,15 +1679,19 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
 	video_set_drvdata(vdev, video);
 	rc = video_register_device(vdev, VFL_TYPE_GRABBER, 0);
 	if (rc) {
-		vb2_queue_release(vbq);
-		v4l2_ctrl_handler_free(&video->ctrl_handler);
-		v4l2_device_unregister(v4l2_dev);
-
 		dev_err(video->dev, "Failed to register video device\n");
-		return rc;
+		goto err_video_reg;
 	}
 
 	return 0;
+
+err_video_reg:
+	vb2_queue_release(vbq);
+err_vb2_init:
+err_ctrl_init:
+	v4l2_ctrl_handler_free(&video->ctrl_handler);
+	v4l2_device_unregister(v4l2_dev);
+	return rc;
 }
 
 static int aspeed_video_init(struct aspeed_video *video)
-- 
2.25.1

[PATCH v5 01/10] media: aspeed: move err-handling together to the bottom

[Jammy Huang]

refine aspeed_video_setup_video() flow.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5:
  - no update
v4:
  - no update
v3:
  - no update
v2:
  - no update
---
 drivers/media/platform/aspeed-video.c | 24 +++++++++++-------------
 1 file changed, 11 insertions(+), 13 deletions(-)

diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index fea5e4d0927e..f5c40d6b4ece 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -1641,11 +1641,8 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
 
 	rc = video->ctrl_handler.error;
 	if (rc) {
-		v4l2_ctrl_handler_free(&video->ctrl_handler);
-		v4l2_device_unregister(v4l2_dev);
-
 		dev_err(video->dev, "Failed to init controls: %d\n", rc);
-		return rc;
+		goto err_ctrl_init;
 	}
 
 	v4l2_dev->ctrl_handler = &video->ctrl_handler;
@@ -1663,11 +1660,8 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
 
 	rc = vb2_queue_init(vbq);
 	if (rc) {
-		v4l2_ctrl_handler_free(&video->ctrl_handler);
-		v4l2_device_unregister(v4l2_dev);
-
 		dev_err(video->dev, "Failed to init vb2 queue\n");
-		return rc;
+		goto err_vb2_init;
 	}
 
 	vdev->queue = vbq;
@@ -1685,15 +1679,19 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
 	video_set_drvdata(vdev, video);
 	rc = video_register_device(vdev, VFL_TYPE_GRABBER, 0);
 	if (rc) {
-		vb2_queue_release(vbq);
-		v4l2_ctrl_handler_free(&video->ctrl_handler);
-		v4l2_device_unregister(v4l2_dev);
-
 		dev_err(video->dev, "Failed to register video device\n");
-		return rc;
+		goto err_video_reg;
 	}
 
 	return 0;
+
+err_video_reg:
+	vb2_queue_release(vbq);
+err_vb2_init:
+err_ctrl_init:
+	v4l2_ctrl_handler_free(&video->ctrl_handler);
+	v4l2_device_unregister(v4l2_dev);
+	return rc;
 }
 
 static int aspeed_video_init(struct aspeed_video *video)
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v5 02/10] media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log

[Jammy Huang]

The debug log level, 0~3, is controlled by module_param, debug.
The higher the value, the more the information.
  0: off
  1: info
  2: debug
  3: register operations

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5:
  - no update
v4:
  - no update
v3:
  - update commit message
  - update MODULE_PARM_DESC for debug
v2:
  - use v4l2_xxx log rather than self-defined macro
---
 drivers/media/platform/aspeed-video.c | 40 +++++++++++++++------------
 1 file changed, 23 insertions(+), 17 deletions(-)

diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index f5c40d6b4ece..6af57467b6d4 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -30,6 +30,7 @@
 #include <media/v4l2-event.h>
 #include <media/v4l2-ioctl.h>
 #include <media/videobuf2-dma-contig.h>
+#include <linux/videodev2.h>
 
 #define DEVICE_NAME			"aspeed-video"
 
@@ -390,6 +391,8 @@ static const struct v4l2_dv_timings_cap aspeed_video_timings_cap = {
 	},
 };
 
+static unsigned int debug;
+
 static void aspeed_video_init_jpeg_table(u32 *table, bool yuv420)
 {
 	int i;
@@ -437,23 +440,23 @@ static void aspeed_video_update(struct aspeed_video *video, u32 reg, u32 clear,
 	t &= ~clear;
 	t |= bits;
 	writel(t, video->base + reg);
-	dev_dbg(video->dev, "update %03x[%08x -> %08x]\n", reg, before,
-		readl(video->base + reg));
+	v4l2_dbg(3, debug, &video->v4l2_dev, "update %03x[%08x -> %08x]\n",
+		 reg, before, readl(video->base + reg));
 }
 
 static u32 aspeed_video_read(struct aspeed_video *video, u32 reg)
 {
 	u32 t = readl(video->base + reg);
 
-	dev_dbg(video->dev, "read %03x[%08x]\n", reg, t);
+	v4l2_dbg(3, debug, &video->v4l2_dev, "read %03x[%08x]\n", reg, t);
 	return t;
 }
 
 static void aspeed_video_write(struct aspeed_video *video, u32 reg, u32 val)
 {
 	writel(val, video->base + reg);
-	dev_dbg(video->dev, "write %03x[%08x]\n", reg,
-		readl(video->base + reg));
+	v4l2_dbg(3, debug, &video->v4l2_dev, "write %03x[%08x]\n", reg,
+		 readl(video->base + reg));
 }
 
 static void update_perf(struct aspeed_video_perf *p)
@@ -474,13 +477,13 @@ static int aspeed_video_start_frame(struct aspeed_video *video)
 	u32 seq_ctrl = aspeed_video_read(video, VE_SEQ_CTRL);
 
 	if (video->v4l2_input_status) {
-		dev_dbg(video->dev, "No signal; don't start frame\n");
+		v4l2_warn(&video->v4l2_dev, "No signal; don't start frame\n");
 		return 0;
 	}
 
 	if (!(seq_ctrl & VE_SEQ_CTRL_COMP_BUSY) ||
 	    !(seq_ctrl & VE_SEQ_CTRL_CAP_BUSY)) {
-		dev_dbg(video->dev, "Engine busy; don't start frame\n");
+		v4l2_warn(&video->v4l2_dev, "Engine busy; don't start frame\n");
 		return -EBUSY;
 	}
 
@@ -489,7 +492,7 @@ static int aspeed_video_start_frame(struct aspeed_video *video)
 				       struct aspeed_video_buffer, link);
 	if (!buf) {
 		spin_unlock_irqrestore(&video->lock, flags);
-		dev_dbg(video->dev, "No buffers; don't start frame\n");
+		v4l2_warn(&video->v4l2_dev, "No buffers; don't start frame\n");
 		return -EPROTO;
 	}
 
@@ -569,7 +572,7 @@ static void aspeed_video_bufs_done(struct aspeed_video *video,
 
 static void aspeed_video_irq_res_change(struct aspeed_video *video, ulong delay)
 {
-	dev_dbg(video->dev, "Resolution changed; resetting\n");
+	v4l2_dbg(1, debug, &video->v4l2_dev, "Resolution changed; resetting\n");
 
 	set_bit(VIDEO_RES_CHANGE, &video->flags);
 	clear_bit(VIDEO_FRAME_INPRG, &video->flags);
@@ -770,8 +773,8 @@ static void aspeed_video_calc_compressed_size(struct aspeed_video *video,
 	aspeed_video_write(video, VE_STREAM_BUF_SIZE,
 			   compression_buffer_size_reg);
 
-	dev_dbg(video->dev, "Max compressed size: %x\n",
-		video->max_compressed_size);
+	v4l2_dbg(1, debug, &video->v4l2_dev, "Max compressed size: %#x\n",
+		 video->max_compressed_size);
 }
 
 #define res_check(v) test_and_clear_bit(VIDEO_MODE_DETECT_DONE, &(v)->flags)
@@ -808,7 +811,7 @@ static void aspeed_video_get_resolution(struct aspeed_video *video)
 						      res_check(video),
 						      MODE_DETECT_TIMEOUT);
 		if (!rc) {
-			dev_dbg(video->dev, "Timed out; first mode detect\n");
+			v4l2_warn(&video->v4l2_dev, "Timed out; first mode detect\n");
 			clear_bit(VIDEO_RES_DETECT, &video->flags);
 			return;
 		}
@@ -822,7 +825,7 @@ static void aspeed_video_get_resolution(struct aspeed_video *video)
 						      MODE_DETECT_TIMEOUT);
 		clear_bit(VIDEO_RES_DETECT, &video->flags);
 		if (!rc) {
-			dev_dbg(video->dev, "Timed out; second mode detect\n");
+			v4l2_warn(&video->v4l2_dev, "Timed out; second mode detect\n");
 			return;
 		}
 
@@ -856,7 +859,7 @@ static void aspeed_video_get_resolution(struct aspeed_video *video)
 	} while (invalid_resolution && (tries++ < INVALID_RESOLUTION_RETRIES));
 
 	if (invalid_resolution) {
-		dev_dbg(video->dev, "Invalid resolution detected\n");
+		v4l2_warn(&video->v4l2_dev, "Invalid resolution detected\n");
 		return;
 	}
 
@@ -873,8 +876,8 @@ static void aspeed_video_get_resolution(struct aspeed_video *video)
 	aspeed_video_update(video, VE_SEQ_CTRL, 0,
 			    VE_SEQ_CTRL_AUTO_COMP | VE_SEQ_CTRL_EN_WATCHDOG);
 
-	dev_dbg(video->dev, "Got resolution: %dx%d\n", det->width,
-		det->height);
+	v4l2_dbg(1, debug, &video->v4l2_dev, "Got resolution: %dx%d\n",
+		 det->width, det->height);
 }
 
 static void aspeed_video_set_resolution(struct aspeed_video *video)
@@ -1501,7 +1504,7 @@ static void aspeed_video_stop_streaming(struct vb2_queue *q)
 				!test_bit(VIDEO_FRAME_INPRG, &video->flags),
 				STOP_TIMEOUT);
 	if (!rc) {
-		dev_dbg(video->dev, "Timed out when stopping streaming\n");
+		v4l2_warn(&video->v4l2_dev, "Timed out when stopping streaming\n");
 
 		/*
 		 * Need to force stop any DMA and try and get HW into a good
@@ -1851,6 +1854,9 @@ static struct platform_driver aspeed_video_driver = {
 
 module_platform_driver(aspeed_video_driver);
 
+module_param(debug, int, 0644);
+MODULE_PARM_DESC(debug, "Debug level (0=off,1=info,2=debug,3=reg ops)");
+
 MODULE_DESCRIPTION("ASPEED Video Engine Driver");
 MODULE_AUTHOR("Eddie James");
 MODULE_LICENSE("GPL v2");
-- 
2.25.1

[PATCH v5 02/10] media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log

[Jammy Huang]

The debug log level, 0~3, is controlled by module_param, debug.
The higher the value, the more the information.
  0: off
  1: info
  2: debug
  3: register operations

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5:
  - no update
v4:
  - no update
v3:
  - update commit message
  - update MODULE_PARM_DESC for debug
v2:
  - use v4l2_xxx log rather than self-defined macro
---
 drivers/media/platform/aspeed-video.c | 40 +++++++++++++++------------
 1 file changed, 23 insertions(+), 17 deletions(-)

diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index f5c40d6b4ece..6af57467b6d4 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -30,6 +30,7 @@
 #include <media/v4l2-event.h>
 #include <media/v4l2-ioctl.h>
 #include <media/videobuf2-dma-contig.h>
+#include <linux/videodev2.h>
 
 #define DEVICE_NAME			"aspeed-video"
 
@@ -390,6 +391,8 @@ static const struct v4l2_dv_timings_cap aspeed_video_timings_cap = {
 	},
 };
 
+static unsigned int debug;
+
 static void aspeed_video_init_jpeg_table(u32 *table, bool yuv420)
 {
 	int i;
@@ -437,23 +440,23 @@ static void aspeed_video_update(struct aspeed_video *video, u32 reg, u32 clear,
 	t &= ~clear;
 	t |= bits;
 	writel(t, video->base + reg);
-	dev_dbg(video->dev, "update %03x[%08x -> %08x]\n", reg, before,
-		readl(video->base + reg));
+	v4l2_dbg(3, debug, &video->v4l2_dev, "update %03x[%08x -> %08x]\n",
+		 reg, before, readl(video->base + reg));
 }
 
 static u32 aspeed_video_read(struct aspeed_video *video, u32 reg)
 {
 	u32 t = readl(video->base + reg);
 
-	dev_dbg(video->dev, "read %03x[%08x]\n", reg, t);
+	v4l2_dbg(3, debug, &video->v4l2_dev, "read %03x[%08x]\n", reg, t);
 	return t;
 }
 
 static void aspeed_video_write(struct aspeed_video *video, u32 reg, u32 val)
 {
 	writel(val, video->base + reg);
-	dev_dbg(video->dev, "write %03x[%08x]\n", reg,
-		readl(video->base + reg));
+	v4l2_dbg(3, debug, &video->v4l2_dev, "write %03x[%08x]\n", reg,
+		 readl(video->base + reg));
 }
 
 static void update_perf(struct aspeed_video_perf *p)
@@ -474,13 +477,13 @@ static int aspeed_video_start_frame(struct aspeed_video *video)
 	u32 seq_ctrl = aspeed_video_read(video, VE_SEQ_CTRL);
 
 	if (video->v4l2_input_status) {
-		dev_dbg(video->dev, "No signal; don't start frame\n");
+		v4l2_warn(&video->v4l2_dev, "No signal; don't start frame\n");
 		return 0;
 	}
 
 	if (!(seq_ctrl & VE_SEQ_CTRL_COMP_BUSY) ||
 	    !(seq_ctrl & VE_SEQ_CTRL_CAP_BUSY)) {
-		dev_dbg(video->dev, "Engine busy; don't start frame\n");
+		v4l2_warn(&video->v4l2_dev, "Engine busy; don't start frame\n");
 		return -EBUSY;
 	}
 
@@ -489,7 +492,7 @@ static int aspeed_video_start_frame(struct aspeed_video *video)
 				       struct aspeed_video_buffer, link);
 	if (!buf) {
 		spin_unlock_irqrestore(&video->lock, flags);
-		dev_dbg(video->dev, "No buffers; don't start frame\n");
+		v4l2_warn(&video->v4l2_dev, "No buffers; don't start frame\n");
 		return -EPROTO;
 	}
 
@@ -569,7 +572,7 @@ static void aspeed_video_bufs_done(struct aspeed_video *video,
 
 static void aspeed_video_irq_res_change(struct aspeed_video *video, ulong delay)
 {
-	dev_dbg(video->dev, "Resolution changed; resetting\n");
+	v4l2_dbg(1, debug, &video->v4l2_dev, "Resolution changed; resetting\n");
 
 	set_bit(VIDEO_RES_CHANGE, &video->flags);
 	clear_bit(VIDEO_FRAME_INPRG, &video->flags);
@@ -770,8 +773,8 @@ static void aspeed_video_calc_compressed_size(struct aspeed_video *video,
 	aspeed_video_write(video, VE_STREAM_BUF_SIZE,
 			   compression_buffer_size_reg);
 
-	dev_dbg(video->dev, "Max compressed size: %x\n",
-		video->max_compressed_size);
+	v4l2_dbg(1, debug, &video->v4l2_dev, "Max compressed size: %#x\n",
+		 video->max_compressed_size);
 }
 
 #define res_check(v) test_and_clear_bit(VIDEO_MODE_DETECT_DONE, &(v)->flags)
@@ -808,7 +811,7 @@ static void aspeed_video_get_resolution(struct aspeed_video *video)
 						      res_check(video),
 						      MODE_DETECT_TIMEOUT);
 		if (!rc) {
-			dev_dbg(video->dev, "Timed out; first mode detect\n");
+			v4l2_warn(&video->v4l2_dev, "Timed out; first mode detect\n");
 			clear_bit(VIDEO_RES_DETECT, &video->flags);
 			return;
 		}
@@ -822,7 +825,7 @@ static void aspeed_video_get_resolution(struct aspeed_video *video)
 						      MODE_DETECT_TIMEOUT);
 		clear_bit(VIDEO_RES_DETECT, &video->flags);
 		if (!rc) {
-			dev_dbg(video->dev, "Timed out; second mode detect\n");
+			v4l2_warn(&video->v4l2_dev, "Timed out; second mode detect\n");
 			return;
 		}
 
@@ -856,7 +859,7 @@ static void aspeed_video_get_resolution(struct aspeed_video *video)
 	} while (invalid_resolution && (tries++ < INVALID_RESOLUTION_RETRIES));
 
 	if (invalid_resolution) {
-		dev_dbg(video->dev, "Invalid resolution detected\n");
+		v4l2_warn(&video->v4l2_dev, "Invalid resolution detected\n");
 		return;
 	}
 
@@ -873,8 +876,8 @@ static void aspeed_video_get_resolution(struct aspeed_video *video)
 	aspeed_video_update(video, VE_SEQ_CTRL, 0,
 			    VE_SEQ_CTRL_AUTO_COMP | VE_SEQ_CTRL_EN_WATCHDOG);
 
-	dev_dbg(video->dev, "Got resolution: %dx%d\n", det->width,
-		det->height);
+	v4l2_dbg(1, debug, &video->v4l2_dev, "Got resolution: %dx%d\n",
+		 det->width, det->height);
 }
 
 static void aspeed_video_set_resolution(struct aspeed_video *video)
@@ -1501,7 +1504,7 @@ static void aspeed_video_stop_streaming(struct vb2_queue *q)
 				!test_bit(VIDEO_FRAME_INPRG, &video->flags),
 				STOP_TIMEOUT);
 	if (!rc) {
-		dev_dbg(video->dev, "Timed out when stopping streaming\n");
+		v4l2_warn(&video->v4l2_dev, "Timed out when stopping streaming\n");
 
 		/*
 		 * Need to force stop any DMA and try and get HW into a good
@@ -1851,6 +1854,9 @@ static struct platform_driver aspeed_video_driver = {
 
 module_platform_driver(aspeed_video_driver);
 
+module_param(debug, int, 0644);
+MODULE_PARM_DESC(debug, "Debug level (0=off,1=info,2=debug,3=reg ops)");
+
 MODULE_DESCRIPTION("ASPEED Video Engine Driver");
 MODULE_AUTHOR("Eddie James");
 MODULE_LICENSE("GPL v2");
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v5 03/10] media: aspeed: add more debug log messages

[Jammy Huang]

The new messages are listed as below:
1. jpeg header and capture buffer information
2. information for each irq
3. current capture mode, sync or direct-fetch
4. time consumed for each frame
5. input timing changed information

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5:
  - no update
v4:
  - modify log level
v3:
  - update commit message
v2:
  - new
---
 drivers/media/platform/aspeed-video.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index 6af57467b6d4..e8dd0a7ebfc7 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -461,12 +461,17 @@ static void aspeed_video_write(struct aspeed_video *video, u32 reg, u32 val)
 
 static void update_perf(struct aspeed_video_perf *p)
 {
+	struct aspeed_video *v = container_of(p, struct aspeed_video,
+					      perf);
+
 	p->duration =
 		ktime_to_ms(ktime_sub(ktime_get(),  p->last_sample));
 	p->totaltime += p->duration;
 
 	p->duration_max = max(p->duration, p->duration_max);
 	p->duration_min = min(p->duration, p->duration_min);
+	v4l2_dbg(2, debug, &v->v4l2_dev, "time consumed: %d ms\n",
+		 p->duration);
 }
 
 static int aspeed_video_start_frame(struct aspeed_video *video)
@@ -597,6 +602,12 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
 	struct aspeed_video *video = arg;
 	u32 sts = aspeed_video_read(video, VE_INTERRUPT_STATUS);
 
+	v4l2_dbg(2, debug, &video->v4l2_dev, "irq sts=%#x %s%s%s%s\n", sts,
+		 sts & VE_INTERRUPT_MODE_DETECT_WD ? ", unlock" : "",
+		 sts & VE_INTERRUPT_MODE_DETECT ? ", lock" : "",
+		 sts & VE_INTERRUPT_CAPTURE_COMPLETE ? ", capture-done" : "",
+		 sts & VE_INTERRUPT_COMP_COMPLETE ? ", comp-done" : "");
+
 	/*
 	 * Resolution changed or signal was lost; reset the engine and
 	 * re-initialize
@@ -910,6 +921,7 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
 
 	/* Don't use direct mode below 1024 x 768 (irqs don't fire) */
 	if (size < DIRECT_FETCH_THRESHOLD) {
+		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Sync Mode\n");
 		aspeed_video_write(video, VE_TGS_0,
 				   FIELD_PREP(VE_TGS_FIRST,
 					      video->frame_left - 1) |
@@ -921,6 +933,7 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
 					      video->frame_bottom + 1));
 		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_INT_DE);
 	} else {
+		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Direct Mode\n");
 		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_DIRECT_FETCH);
 	}
 
@@ -937,6 +950,10 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
 		if (!aspeed_video_alloc_buf(video, &video->srcs[1], size))
 			goto err_mem;
 
+		v4l2_dbg(1, debug, &video->v4l2_dev, "src buf0 addr(%#x) size(%d)\n",
+			 video->srcs[0].dma, video->srcs[0].size);
+		v4l2_dbg(1, debug, &video->v4l2_dev, "src buf1 addr(%#x) size(%d)\n",
+			 video->srcs[1].dma, video->srcs[1].size);
 		aspeed_video_write(video, VE_SRC0_ADDR, video->srcs[0].dma);
 		aspeed_video_write(video, VE_SRC1_ADDR, video->srcs[1].dma);
 	}
@@ -1201,6 +1218,9 @@ static int aspeed_video_set_dv_timings(struct file *file, void *fh,
 
 	timings->type = V4L2_DV_BT_656_1120;
 
+	v4l2_dbg(1, debug, &video->v4l2_dev, "set new timings(%dx%d)\n",
+		 timings->bt.width, timings->bt.height);
+
 	return 0;
 }
 
@@ -1383,6 +1403,7 @@ static void aspeed_video_resolution_work(struct work_struct *work)
 			.u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION,
 		};
 
+		v4l2_dbg(1, debug, &video->v4l2_dev, "fire source change event\n");
 		v4l2_event_queue(&video->vdev, &ev);
 	} else if (test_bit(VIDEO_STREAMING, &video->flags)) {
 		/* No resolution change so just restart streaming */
@@ -1715,6 +1736,7 @@ static int aspeed_video_init(struct aspeed_video *video)
 		dev_err(dev, "Unable to request IRQ %d\n", irq);
 		return rc;
 	}
+	dev_info(video->dev, "irq %d\n", irq);
 
 	video->eclk = devm_clk_get(dev, "eclk");
 	if (IS_ERR(video->eclk)) {
@@ -1751,6 +1773,8 @@ static int aspeed_video_init(struct aspeed_video *video)
 		rc = -ENOMEM;
 		goto err_release_reserved_mem;
 	}
+	dev_info(video->dev, "alloc mem size(%d) at %#x for jpeg header\n",
+		 VE_JPEG_HEADER_SIZE, video->jpeg.dma);
 
 	aspeed_video_init_jpeg_table(video->jpeg.virt, video->yuv420);
 
-- 
2.25.1

[PATCH v5 03/10] media: aspeed: add more debug log messages

[Jammy Huang]

The new messages are listed as below:
1. jpeg header and capture buffer information
2. information for each irq
3. current capture mode, sync or direct-fetch
4. time consumed for each frame
5. input timing changed information

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5:
  - no update
v4:
  - modify log level
v3:
  - update commit message
v2:
  - new
---
 drivers/media/platform/aspeed-video.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index 6af57467b6d4..e8dd0a7ebfc7 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -461,12 +461,17 @@ static void aspeed_video_write(struct aspeed_video *video, u32 reg, u32 val)
 
 static void update_perf(struct aspeed_video_perf *p)
 {
+	struct aspeed_video *v = container_of(p, struct aspeed_video,
+					      perf);
+
 	p->duration =
 		ktime_to_ms(ktime_sub(ktime_get(),  p->last_sample));
 	p->totaltime += p->duration;
 
 	p->duration_max = max(p->duration, p->duration_max);
 	p->duration_min = min(p->duration, p->duration_min);
+	v4l2_dbg(2, debug, &v->v4l2_dev, "time consumed: %d ms\n",
+		 p->duration);
 }
 
 static int aspeed_video_start_frame(struct aspeed_video *video)
@@ -597,6 +602,12 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
 	struct aspeed_video *video = arg;
 	u32 sts = aspeed_video_read(video, VE_INTERRUPT_STATUS);
 
+	v4l2_dbg(2, debug, &video->v4l2_dev, "irq sts=%#x %s%s%s%s\n", sts,
+		 sts & VE_INTERRUPT_MODE_DETECT_WD ? ", unlock" : "",
+		 sts & VE_INTERRUPT_MODE_DETECT ? ", lock" : "",
+		 sts & VE_INTERRUPT_CAPTURE_COMPLETE ? ", capture-done" : "",
+		 sts & VE_INTERRUPT_COMP_COMPLETE ? ", comp-done" : "");
+
 	/*
 	 * Resolution changed or signal was lost; reset the engine and
 	 * re-initialize
@@ -910,6 +921,7 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
 
 	/* Don't use direct mode below 1024 x 768 (irqs don't fire) */
 	if (size < DIRECT_FETCH_THRESHOLD) {
+		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Sync Mode\n");
 		aspeed_video_write(video, VE_TGS_0,
 				   FIELD_PREP(VE_TGS_FIRST,
 					      video->frame_left - 1) |
@@ -921,6 +933,7 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
 					      video->frame_bottom + 1));
 		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_INT_DE);
 	} else {
+		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Direct Mode\n");
 		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_DIRECT_FETCH);
 	}
 
@@ -937,6 +950,10 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
 		if (!aspeed_video_alloc_buf(video, &video->srcs[1], size))
 			goto err_mem;
 
+		v4l2_dbg(1, debug, &video->v4l2_dev, "src buf0 addr(%#x) size(%d)\n",
+			 video->srcs[0].dma, video->srcs[0].size);
+		v4l2_dbg(1, debug, &video->v4l2_dev, "src buf1 addr(%#x) size(%d)\n",
+			 video->srcs[1].dma, video->srcs[1].size);
 		aspeed_video_write(video, VE_SRC0_ADDR, video->srcs[0].dma);
 		aspeed_video_write(video, VE_SRC1_ADDR, video->srcs[1].dma);
 	}
@@ -1201,6 +1218,9 @@ static int aspeed_video_set_dv_timings(struct file *file, void *fh,
 
 	timings->type = V4L2_DV_BT_656_1120;
 
+	v4l2_dbg(1, debug, &video->v4l2_dev, "set new timings(%dx%d)\n",
+		 timings->bt.width, timings->bt.height);
+
 	return 0;
 }
 
@@ -1383,6 +1403,7 @@ static void aspeed_video_resolution_work(struct work_struct *work)
 			.u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION,
 		};
 
+		v4l2_dbg(1, debug, &video->v4l2_dev, "fire source change event\n");
 		v4l2_event_queue(&video->vdev, &ev);
 	} else if (test_bit(VIDEO_STREAMING, &video->flags)) {
 		/* No resolution change so just restart streaming */
@@ -1715,6 +1736,7 @@ static int aspeed_video_init(struct aspeed_video *video)
 		dev_err(dev, "Unable to request IRQ %d\n", irq);
 		return rc;
 	}
+	dev_info(video->dev, "irq %d\n", irq);
 
 	video->eclk = devm_clk_get(dev, "eclk");
 	if (IS_ERR(video->eclk)) {
@@ -1751,6 +1773,8 @@ static int aspeed_video_init(struct aspeed_video *video)
 		rc = -ENOMEM;
 		goto err_release_reserved_mem;
 	}
+	dev_info(video->dev, "alloc mem size(%d) at %#x for jpeg header\n",
+		 VE_JPEG_HEADER_SIZE, video->jpeg.dma);
 
 	aspeed_video_init_jpeg_table(video->jpeg.virt, video->yuv420);
 
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v5 04/10] media: aspeed: refactor to gather format/compress settings

[Jammy Huang]

Add API, aspeed_video_update_regs(), to gather format/compress settings
which are controlled by user.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5:
  - no update
v4:
  - no update
v3:
  - no update
v2:
  - update commit message
  - update log to use v4l2_dbg rather than dprintk
---
 drivers/media/platform/aspeed-video.c | 69 ++++++++++++++-------------
 1 file changed, 35 insertions(+), 34 deletions(-)

diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index e8dd0a7ebfc7..d2335d669fb3 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -967,20 +967,42 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
 		aspeed_video_free_buf(video, &video->srcs[0]);
 }
 
-static void aspeed_video_init_regs(struct aspeed_video *video)
+static void aspeed_video_update_regs(struct aspeed_video *video)
 {
 	u32 comp_ctrl = VE_COMP_CTRL_RSVD |
 		FIELD_PREP(VE_COMP_CTRL_DCT_LUM, video->jpeg_quality) |
 		FIELD_PREP(VE_COMP_CTRL_DCT_CHR, video->jpeg_quality | 0x10);
-	u32 ctrl = VE_CTRL_AUTO_OR_CURSOR;
+	u32 ctrl = 0;
 	u32 seq_ctrl = VE_SEQ_CTRL_JPEG_MODE;
 
+	v4l2_dbg(1, debug, &video->v4l2_dev, "framerate(%d)\n",
+		 video->frame_rate);
+	v4l2_dbg(1, debug, &video->v4l2_dev, "subsample(%s)\n",
+		 video->yuv420 ? "420" : "444");
+	v4l2_dbg(1, debug, &video->v4l2_dev, "compression quality(%d)\n",
+		 video->jpeg_quality);
+
 	if (video->frame_rate)
 		ctrl |= FIELD_PREP(VE_CTRL_FRC, video->frame_rate);
 
 	if (video->yuv420)
 		seq_ctrl |= VE_SEQ_CTRL_YUV420;
 
+	if (video->jpeg.virt)
+		aspeed_video_update_jpeg_table(video->jpeg.virt, video->yuv420);
+
+	/* Set control registers */
+	aspeed_video_update(video, VE_SEQ_CTRL,
+			    VE_SEQ_CTRL_JPEG_MODE | VE_SEQ_CTRL_YUV420,
+			    seq_ctrl);
+	aspeed_video_update(video, VE_CTRL, VE_CTRL_FRC, ctrl);
+	aspeed_video_update(video, VE_COMP_CTRL,
+			    VE_COMP_CTRL_DCT_LUM | VE_COMP_CTRL_DCT_CHR,
+			    comp_ctrl);
+}
+
+static void aspeed_video_init_regs(struct aspeed_video *video)
+{
 	/* Unlock VE registers */
 	aspeed_video_write(video, VE_PROTECTION_KEY, VE_PROTECTION_KEY_UNLOCK);
 
@@ -995,9 +1017,8 @@ static void aspeed_video_init_regs(struct aspeed_video *video)
 	aspeed_video_write(video, VE_JPEG_ADDR, video->jpeg.dma);
 
 	/* Set control registers */
-	aspeed_video_write(video, VE_SEQ_CTRL, seq_ctrl);
-	aspeed_video_write(video, VE_CTRL, ctrl);
-	aspeed_video_write(video, VE_COMP_CTRL, comp_ctrl);
+	aspeed_video_write(video, VE_CTRL, VE_CTRL_AUTO_OR_CURSOR);
+	aspeed_video_write(video, VE_COMP_CTRL, VE_COMP_CTRL_RSVD);
 
 	/* Don't downscale */
 	aspeed_video_write(video, VE_SCALING_FACTOR, 0x10001000);
@@ -1326,27 +1347,6 @@ static const struct v4l2_ioctl_ops aspeed_video_ioctl_ops = {
 	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
 };
 
-static void aspeed_video_update_jpeg_quality(struct aspeed_video *video)
-{
-	u32 comp_ctrl = FIELD_PREP(VE_COMP_CTRL_DCT_LUM, video->jpeg_quality) |
-		FIELD_PREP(VE_COMP_CTRL_DCT_CHR, video->jpeg_quality | 0x10);
-
-	aspeed_video_update(video, VE_COMP_CTRL,
-			    VE_COMP_CTRL_DCT_LUM | VE_COMP_CTRL_DCT_CHR,
-			    comp_ctrl);
-}
-
-static void aspeed_video_update_subsampling(struct aspeed_video *video)
-{
-	if (video->jpeg.virt)
-		aspeed_video_update_jpeg_table(video->jpeg.virt, video->yuv420);
-
-	if (video->yuv420)
-		aspeed_video_update(video, VE_SEQ_CTRL, 0, VE_SEQ_CTRL_YUV420);
-	else
-		aspeed_video_update(video, VE_SEQ_CTRL, VE_SEQ_CTRL_YUV420, 0);
-}
-
 static int aspeed_video_set_ctrl(struct v4l2_ctrl *ctrl)
 {
 	struct aspeed_video *video = container_of(ctrl->handler,
@@ -1356,16 +1356,13 @@ static int aspeed_video_set_ctrl(struct v4l2_ctrl *ctrl)
 	switch (ctrl->id) {
 	case V4L2_CID_JPEG_COMPRESSION_QUALITY:
 		video->jpeg_quality = ctrl->val;
-		aspeed_video_update_jpeg_quality(video);
+		if (test_bit(VIDEO_STREAMING, &video->flags))
+			aspeed_video_update_regs(video);
 		break;
 	case V4L2_CID_JPEG_CHROMA_SUBSAMPLING:
-		if (ctrl->val == V4L2_JPEG_CHROMA_SUBSAMPLING_420) {
-			video->yuv420 = true;
-			aspeed_video_update_subsampling(video);
-		} else {
-			video->yuv420 = false;
-			aspeed_video_update_subsampling(video);
-		}
+		video->yuv420 = (ctrl->val == V4L2_JPEG_CHROMA_SUBSAMPLING_420);
+		if (test_bit(VIDEO_STREAMING, &video->flags))
+			aspeed_video_update_regs(video);
 		break;
 	default:
 		return -EINVAL;
@@ -1393,6 +1390,8 @@ static void aspeed_video_resolution_work(struct work_struct *work)
 
 	aspeed_video_init_regs(video);
 
+	aspeed_video_update_regs(video);
+
 	aspeed_video_get_resolution(video);
 
 	if (video->detected_timings.width != video->active_timings.width ||
@@ -1504,6 +1503,8 @@ static int aspeed_video_start_streaming(struct vb2_queue *q,
 	video->perf.duration_max = 0;
 	video->perf.duration_min = 0xffffffff;
 
+	aspeed_video_update_regs(video);
+
 	rc = aspeed_video_start_frame(video);
 	if (rc) {
 		aspeed_video_bufs_done(video, VB2_BUF_STATE_QUEUED);
-- 
2.25.1

[PATCH v5 04/10] media: aspeed: refactor to gather format/compress settings

[Jammy Huang]

Add API, aspeed_video_update_regs(), to gather format/compress settings
which are controlled by user.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5:
  - no update
v4:
  - no update
v3:
  - no update
v2:
  - update commit message
  - update log to use v4l2_dbg rather than dprintk
---
 drivers/media/platform/aspeed-video.c | 69 ++++++++++++++-------------
 1 file changed, 35 insertions(+), 34 deletions(-)

diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index e8dd0a7ebfc7..d2335d669fb3 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -967,20 +967,42 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
 		aspeed_video_free_buf(video, &video->srcs[0]);
 }
 
-static void aspeed_video_init_regs(struct aspeed_video *video)
+static void aspeed_video_update_regs(struct aspeed_video *video)
 {
 	u32 comp_ctrl = VE_COMP_CTRL_RSVD |
 		FIELD_PREP(VE_COMP_CTRL_DCT_LUM, video->jpeg_quality) |
 		FIELD_PREP(VE_COMP_CTRL_DCT_CHR, video->jpeg_quality | 0x10);
-	u32 ctrl = VE_CTRL_AUTO_OR_CURSOR;
+	u32 ctrl = 0;
 	u32 seq_ctrl = VE_SEQ_CTRL_JPEG_MODE;
 
+	v4l2_dbg(1, debug, &video->v4l2_dev, "framerate(%d)\n",
+		 video->frame_rate);
+	v4l2_dbg(1, debug, &video->v4l2_dev, "subsample(%s)\n",
+		 video->yuv420 ? "420" : "444");
+	v4l2_dbg(1, debug, &video->v4l2_dev, "compression quality(%d)\n",
+		 video->jpeg_quality);
+
 	if (video->frame_rate)
 		ctrl |= FIELD_PREP(VE_CTRL_FRC, video->frame_rate);
 
 	if (video->yuv420)
 		seq_ctrl |= VE_SEQ_CTRL_YUV420;
 
+	if (video->jpeg.virt)
+		aspeed_video_update_jpeg_table(video->jpeg.virt, video->yuv420);
+
+	/* Set control registers */
+	aspeed_video_update(video, VE_SEQ_CTRL,
+			    VE_SEQ_CTRL_JPEG_MODE | VE_SEQ_CTRL_YUV420,
+			    seq_ctrl);
+	aspeed_video_update(video, VE_CTRL, VE_CTRL_FRC, ctrl);
+	aspeed_video_update(video, VE_COMP_CTRL,
+			    VE_COMP_CTRL_DCT_LUM | VE_COMP_CTRL_DCT_CHR,
+			    comp_ctrl);
+}
+
+static void aspeed_video_init_regs(struct aspeed_video *video)
+{
 	/* Unlock VE registers */
 	aspeed_video_write(video, VE_PROTECTION_KEY, VE_PROTECTION_KEY_UNLOCK);
 
@@ -995,9 +1017,8 @@ static void aspeed_video_init_regs(struct aspeed_video *video)
 	aspeed_video_write(video, VE_JPEG_ADDR, video->jpeg.dma);
 
 	/* Set control registers */
-	aspeed_video_write(video, VE_SEQ_CTRL, seq_ctrl);
-	aspeed_video_write(video, VE_CTRL, ctrl);
-	aspeed_video_write(video, VE_COMP_CTRL, comp_ctrl);
+	aspeed_video_write(video, VE_CTRL, VE_CTRL_AUTO_OR_CURSOR);
+	aspeed_video_write(video, VE_COMP_CTRL, VE_COMP_CTRL_RSVD);
 
 	/* Don't downscale */
 	aspeed_video_write(video, VE_SCALING_FACTOR, 0x10001000);
@@ -1326,27 +1347,6 @@ static const struct v4l2_ioctl_ops aspeed_video_ioctl_ops = {
 	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
 };
 
-static void aspeed_video_update_jpeg_quality(struct aspeed_video *video)
-{
-	u32 comp_ctrl = FIELD_PREP(VE_COMP_CTRL_DCT_LUM, video->jpeg_quality) |
-		FIELD_PREP(VE_COMP_CTRL_DCT_CHR, video->jpeg_quality | 0x10);
-
-	aspeed_video_update(video, VE_COMP_CTRL,
-			    VE_COMP_CTRL_DCT_LUM | VE_COMP_CTRL_DCT_CHR,
-			    comp_ctrl);
-}
-
-static void aspeed_video_update_subsampling(struct aspeed_video *video)
-{
-	if (video->jpeg.virt)
-		aspeed_video_update_jpeg_table(video->jpeg.virt, video->yuv420);
-
-	if (video->yuv420)
-		aspeed_video_update(video, VE_SEQ_CTRL, 0, VE_SEQ_CTRL_YUV420);
-	else
-		aspeed_video_update(video, VE_SEQ_CTRL, VE_SEQ_CTRL_YUV420, 0);
-}
-
 static int aspeed_video_set_ctrl(struct v4l2_ctrl *ctrl)
 {
 	struct aspeed_video *video = container_of(ctrl->handler,
@@ -1356,16 +1356,13 @@ static int aspeed_video_set_ctrl(struct v4l2_ctrl *ctrl)
 	switch (ctrl->id) {
 	case V4L2_CID_JPEG_COMPRESSION_QUALITY:
 		video->jpeg_quality = ctrl->val;
-		aspeed_video_update_jpeg_quality(video);
+		if (test_bit(VIDEO_STREAMING, &video->flags))
+			aspeed_video_update_regs(video);
 		break;
 	case V4L2_CID_JPEG_CHROMA_SUBSAMPLING:
-		if (ctrl->val == V4L2_JPEG_CHROMA_SUBSAMPLING_420) {
-			video->yuv420 = true;
-			aspeed_video_update_subsampling(video);
-		} else {
-			video->yuv420 = false;
-			aspeed_video_update_subsampling(video);
-		}
+		video->yuv420 = (ctrl->val == V4L2_JPEG_CHROMA_SUBSAMPLING_420);
+		if (test_bit(VIDEO_STREAMING, &video->flags))
+			aspeed_video_update_regs(video);
 		break;
 	default:
 		return -EINVAL;
@@ -1393,6 +1390,8 @@ static void aspeed_video_resolution_work(struct work_struct *work)
 
 	aspeed_video_init_regs(video);
 
+	aspeed_video_update_regs(video);
+
 	aspeed_video_get_resolution(video);
 
 	if (video->detected_timings.width != video->active_timings.width ||
@@ -1504,6 +1503,8 @@ static int aspeed_video_start_streaming(struct vb2_queue *q,
 	video->perf.duration_max = 0;
 	video->perf.duration_min = 0xffffffff;
 
+	aspeed_video_update_regs(video);
+
 	rc = aspeed_video_start_frame(video);
 	if (rc) {
 		aspeed_video_bufs_done(video, VB2_BUF_STATE_QUEUED);
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

This introduces support for the Aspeed JPEG format, where the new frame
can refer to previous frame to reduce the amount of compressed data. The
concept is similar to I/P frame of video compression. I will compare the
new frame with previous one to decide which macroblock's data is
changed, and only the changed macroblocks will be compressed.

This Aspeed JPEG format is used by the video engine on Aspeed platforms,
which is generally adapted for remote KVM.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5:
  - no update
v4:
  - new
---
 Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
 include/uapi/linux/videodev2.h                   |  1 +
 3 files changed, 14 insertions(+)

diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
index b2cd155e691b..23c05063133d 100644
--- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
+++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
@@ -264,6 +264,18 @@ please make a proposal on the linux-media mailing list.
 	of tiles, resulting in 32-aligned resolutions for the luminance plane
 	and 16-aligned resolutions for the chrominance plane (with 2x2
 	subsampling).
+    * .. _V4L2-PIX-FMT-AJPG:
+
+      - ``V4L2_PIX_FMT_AJPG``
+      - 'AJPG'
+      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
+        which is generally adapted for remote KVM.
+        On each frame compression, I will compare the new frame with previous
+        one to decide which macroblock's data is changed, and only the changed
+        macroblocks will be compressed.
+
+        You could reference to chapter 36, Video Engine, of AST2600's datasheet
+        for more information.
 
 .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
 
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 24db33f803c0..00dde01d2f97 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 		case V4L2_PIX_FMT_S5C_UYVY_JPG:	descr = "S5C73MX interleaved UYVY/JPEG"; break;
 		case V4L2_PIX_FMT_MT21C:	descr = "Mediatek Compressed Format"; break;
 		case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
+		case V4L2_PIX_FMT_AJPG:		descr = "Aspeed JPEG"; break;
 		default:
 			if (fmt->description[0])
 				return;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 3210b3c82a4a..994eb6155ea9 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -726,6 +726,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
 #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
 #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
+#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
 
 /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
 #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
-- 
2.25.1

[PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

This introduces support for the Aspeed JPEG format, where the new frame
can refer to previous frame to reduce the amount of compressed data. The
concept is similar to I/P frame of video compression. I will compare the
new frame with previous one to decide which macroblock's data is
changed, and only the changed macroblocks will be compressed.

This Aspeed JPEG format is used by the video engine on Aspeed platforms,
which is generally adapted for remote KVM.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5:
  - no update
v4:
  - new
---
 Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
 include/uapi/linux/videodev2.h                   |  1 +
 3 files changed, 14 insertions(+)

diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
index b2cd155e691b..23c05063133d 100644
--- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
+++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
@@ -264,6 +264,18 @@ please make a proposal on the linux-media mailing list.
 	of tiles, resulting in 32-aligned resolutions for the luminance plane
 	and 16-aligned resolutions for the chrominance plane (with 2x2
 	subsampling).
+    * .. _V4L2-PIX-FMT-AJPG:
+
+      - ``V4L2_PIX_FMT_AJPG``
+      - 'AJPG'
+      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
+        which is generally adapted for remote KVM.
+        On each frame compression, I will compare the new frame with previous
+        one to decide which macroblock's data is changed, and only the changed
+        macroblocks will be compressed.
+
+        You could reference to chapter 36, Video Engine, of AST2600's datasheet
+        for more information.
 
 .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
 
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 24db33f803c0..00dde01d2f97 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 		case V4L2_PIX_FMT_S5C_UYVY_JPG:	descr = "S5C73MX interleaved UYVY/JPEG"; break;
 		case V4L2_PIX_FMT_MT21C:	descr = "Mediatek Compressed Format"; break;
 		case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
+		case V4L2_PIX_FMT_AJPG:		descr = "Aspeed JPEG"; break;
 		default:
 			if (fmt->description[0])
 				return;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 3210b3c82a4a..994eb6155ea9 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -726,6 +726,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
 #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
 #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
+#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
 
 /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
 #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v5 06/10] media: v4l2-ctrls: Reserve controls for ASPEED

[Jammy Huang]

Reserve controls for ASPEED video family. Aspeed video engine contains a
few features which improve video quality, reduce amount of compressed
data, and etc. Hence, 16 controls are reserved for these aspeed
proprietary features.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5:
  - no update
v4:
  - new
---
 include/uapi/linux/v4l2-controls.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index a2669b79b294..e7c425a6fa9f 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -192,6 +192,11 @@ enum v4l2_colorfx {
  * We reserve 16 controls for this driver. */
 #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
 
+/* The base for the aspeed driver controls.
+ * We reserve 16 controls for this driver.
+ */
+#define V4L2_CID_USER_ASPEED_BASE		(V4L2_CID_USER_BASE + 0x10c0)
+
 /* MPEG-class control IDs */
 /* The MPEG controls are applicable to all codec controls
  * and the 'MPEG' part of the define is historical */
-- 
2.25.1

[PATCH v5 06/10] media: v4l2-ctrls: Reserve controls for ASPEED

[Jammy Huang]

Reserve controls for ASPEED video family. Aspeed video engine contains a
few features which improve video quality, reduce amount of compressed
data, and etc. Hence, 16 controls are reserved for these aspeed
proprietary features.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5:
  - no update
v4:
  - new
---
 include/uapi/linux/v4l2-controls.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index a2669b79b294..e7c425a6fa9f 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -192,6 +192,11 @@ enum v4l2_colorfx {
  * We reserve 16 controls for this driver. */
 #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
 
+/* The base for the aspeed driver controls.
+ * We reserve 16 controls for this driver.
+ */
+#define V4L2_CID_USER_ASPEED_BASE		(V4L2_CID_USER_BASE + 0x10c0)
+
 /* MPEG-class control IDs */
 /* The MPEG controls are applicable to all codec controls
  * and the 'MPEG' part of the define is historical */
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v5 07/10] media: aspeed: Use runtime configuration

[Jammy Huang]

The aspeed video IP has some differences between SoC families. Currently
the driver decides which registers to use at compile time, which means
a single kernel can not be used between platforms.

Switch to using runtime configuration of the registers that vary between
SoC families.

Signed-off-by: Joel Stanley <joel@jms.id.au>
Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5
  - new
---
 drivers/media/platform/aspeed-video.c | 71 ++++++++++++++++++++-------
 1 file changed, 52 insertions(+), 19 deletions(-)

diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index d2335d669fb3..ba8ee82b38c3 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -75,11 +75,8 @@
 #define  VE_SEQ_CTRL_CAP_BUSY		BIT(16)
 #define  VE_SEQ_CTRL_COMP_BUSY		BIT(18)
 
-#ifdef CONFIG_MACH_ASPEED_G4
-#define  VE_SEQ_CTRL_JPEG_MODE		BIT(8)	/* AST2400 */
-#else
-#define  VE_SEQ_CTRL_JPEG_MODE		BIT(13)	/* AST2500/2600 */
-#endif
+#define AST2500_VE_SEQ_CTRL_JPEG_MODE	BIT(13)
+#define AST2400_VE_SEQ_CTRL_JPEG_MODE	BIT(8)
 
 #define VE_CTRL				0x008
 #define  VE_CTRL_HSYNC_POL		BIT(0)
@@ -136,9 +133,8 @@
 #define  VE_COMP_CTRL_HQ_DCT_CHR	GENMASK(26, 22)
 #define  VE_COMP_CTRL_HQ_DCT_LUM	GENMASK(31, 27)
 
-#define VE_OFFSET_COMP_STREAM		0x078
-
-#define VE_JPEG_COMP_SIZE_READ_BACK	0x084
+#define AST2400_VE_COMP_SIZE_READ_BACK	0x078
+#define AST2600_VE_COMP_SIZE_READ_BACK	0x084
 
 #define VE_SRC_LR_EDGE_DET		0x090
 #define  VE_SRC_LR_EDGE_DET_LEFT	GENMASK(11, 0)
@@ -233,6 +229,8 @@ struct aspeed_video {
 	struct video_device vdev;
 	struct mutex video_lock;	/* v4l2 and videobuf2 lock */
 
+	struct aspeed_video_config config;
+
 	wait_queue_head_t wait;
 	spinlock_t lock;		/* buffer list lock */
 	struct delayed_work res_work;
@@ -258,6 +256,30 @@ struct aspeed_video {
 
 #define to_aspeed_video(x) container_of((x), struct aspeed_video, v4l2_dev)
 
+struct aspeed_video_config {
+	u32 version;
+	u32 jpeg_mode;
+	u32 comp_size_read;
+};
+
+static const struct aspeed_video_config ast2400_config = {
+	.version = 4,
+	.jpeg_mode = AST2400_VE_SEQ_CTRL_JPEG_MODE,
+	.comp_size_read = AST2400_VE_COMP_SIZE_READ_BACK,
+};
+
+static const struct aspeed_video_config ast2500_config = {
+	.version = 5,
+	.jpeg_mode = AST2500_VE_SEQ_CTRL_JPEG_MODE,
+	.comp_size_read = AST2400_VE_COMP_SIZE_READ_BACK,
+};
+
+static const struct aspeed_video_config ast2600_config = {
+	.version = 6,
+	.jpeg_mode = AST2500_VE_SEQ_CTRL_JPEG_MODE,
+	.comp_size_read = AST2600_VE_COMP_SIZE_READ_BACK,
+};
+
 static const u32 aspeed_video_jpeg_header[ASPEED_VIDEO_JPEG_HEADER_SIZE] = {
 	0xe0ffd8ff, 0x464a1000, 0x01004649, 0x60000101, 0x00006000, 0x0f00feff,
 	0x00002d05, 0x00000000, 0x00000000, 0x00dbff00
@@ -640,7 +662,7 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
 	if (sts & VE_INTERRUPT_COMP_COMPLETE) {
 		struct aspeed_video_buffer *buf;
 		u32 frame_size = aspeed_video_read(video,
-						   VE_JPEG_COMP_SIZE_READ_BACK);
+						   video->config.comp_size_read);
 
 		update_perf(&video->perf);
 
@@ -973,7 +995,7 @@ static void aspeed_video_update_regs(struct aspeed_video *video)
 		FIELD_PREP(VE_COMP_CTRL_DCT_LUM, video->jpeg_quality) |
 		FIELD_PREP(VE_COMP_CTRL_DCT_CHR, video->jpeg_quality | 0x10);
 	u32 ctrl = 0;
-	u32 seq_ctrl = VE_SEQ_CTRL_JPEG_MODE;
+	u32 seq_ctrl = video->config.jpeg_mode;
 
 	v4l2_dbg(1, debug, &video->v4l2_dev, "framerate(%d)\n",
 		 video->frame_rate);
@@ -993,7 +1015,7 @@ static void aspeed_video_update_regs(struct aspeed_video *video)
 
 	/* Set control registers */
 	aspeed_video_update(video, VE_SEQ_CTRL,
-			    VE_SEQ_CTRL_JPEG_MODE | VE_SEQ_CTRL_YUV420,
+			    video->config.jpeg_mode | VE_SEQ_CTRL_YUV420,
 			    seq_ctrl);
 	aspeed_video_update(video, VE_CTRL, VE_CTRL_FRC, ctrl);
 	aspeed_video_update(video, VE_COMP_CTRL,
@@ -1790,8 +1812,18 @@ static int aspeed_video_init(struct aspeed_video *video)
 	return rc;
 }
 
+static const struct of_device_id aspeed_video_of_match[] = {
+	{ .compatible = "aspeed,ast2400-video-engine", .data = &ast2400_config },
+	{ .compatible = "aspeed,ast2500-video-engine", .data = &ast2500_config },
+	{ .compatible = "aspeed,ast2600-video-engine", .data = &ast2600_config },
+	{}
+};
+MODULE_DEVICE_TABLE(of, aspeed_video_of_match);
+
 static int aspeed_video_probe(struct platform_device *pdev)
 {
+	const struct aspeed_video_config *config;
+	const struct of_device_id *match;
 	int rc;
 	struct resource *res;
 	struct aspeed_video *video =
@@ -1815,6 +1847,13 @@ static int aspeed_video_probe(struct platform_device *pdev)
 	if (IS_ERR(video->base))
 		return PTR_ERR(video->base);
 
+	match = of_match_node(aspeed_video_of_match, pdev->dev.of_node);
+	if (!match)
+		return -EINVAL;
+
+	config = match->data;
+	video->config = *config;
+
 	rc = aspeed_video_init(video);
 	if (rc)
 		return rc;
@@ -1828,6 +1867,8 @@ static int aspeed_video_probe(struct platform_device *pdev)
 
 	aspeed_video_debugfs_create(video);
 
+	dev_info(video->dev, "compatible for g%d\n", config->version);
+
 	return 0;
 }
 
@@ -1860,14 +1901,6 @@ static int aspeed_video_remove(struct platform_device *pdev)
 	return 0;
 }
 
-static const struct of_device_id aspeed_video_of_match[] = {
-	{ .compatible = "aspeed,ast2400-video-engine" },
-	{ .compatible = "aspeed,ast2500-video-engine" },
-	{ .compatible = "aspeed,ast2600-video-engine" },
-	{}
-};
-MODULE_DEVICE_TABLE(of, aspeed_video_of_match);
-
 static struct platform_driver aspeed_video_driver = {
 	.driver = {
 		.name = DEVICE_NAME,
-- 
2.25.1

[PATCH v5 07/10] media: aspeed: Use runtime configuration

[Jammy Huang]

The aspeed video IP has some differences between SoC families. Currently
the driver decides which registers to use at compile time, which means
a single kernel can not be used between platforms.

Switch to using runtime configuration of the registers that vary between
SoC families.

Signed-off-by: Joel Stanley <joel@jms.id.au>
Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5
  - new
---
 drivers/media/platform/aspeed-video.c | 71 ++++++++++++++++++++-------
 1 file changed, 52 insertions(+), 19 deletions(-)

diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index d2335d669fb3..ba8ee82b38c3 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -75,11 +75,8 @@
 #define  VE_SEQ_CTRL_CAP_BUSY		BIT(16)
 #define  VE_SEQ_CTRL_COMP_BUSY		BIT(18)
 
-#ifdef CONFIG_MACH_ASPEED_G4
-#define  VE_SEQ_CTRL_JPEG_MODE		BIT(8)	/* AST2400 */
-#else
-#define  VE_SEQ_CTRL_JPEG_MODE		BIT(13)	/* AST2500/2600 */
-#endif
+#define AST2500_VE_SEQ_CTRL_JPEG_MODE	BIT(13)
+#define AST2400_VE_SEQ_CTRL_JPEG_MODE	BIT(8)
 
 #define VE_CTRL				0x008
 #define  VE_CTRL_HSYNC_POL		BIT(0)
@@ -136,9 +133,8 @@
 #define  VE_COMP_CTRL_HQ_DCT_CHR	GENMASK(26, 22)
 #define  VE_COMP_CTRL_HQ_DCT_LUM	GENMASK(31, 27)
 
-#define VE_OFFSET_COMP_STREAM		0x078
-
-#define VE_JPEG_COMP_SIZE_READ_BACK	0x084
+#define AST2400_VE_COMP_SIZE_READ_BACK	0x078
+#define AST2600_VE_COMP_SIZE_READ_BACK	0x084
 
 #define VE_SRC_LR_EDGE_DET		0x090
 #define  VE_SRC_LR_EDGE_DET_LEFT	GENMASK(11, 0)
@@ -233,6 +229,8 @@ struct aspeed_video {
 	struct video_device vdev;
 	struct mutex video_lock;	/* v4l2 and videobuf2 lock */
 
+	struct aspeed_video_config config;
+
 	wait_queue_head_t wait;
 	spinlock_t lock;		/* buffer list lock */
 	struct delayed_work res_work;
@@ -258,6 +256,30 @@ struct aspeed_video {
 
 #define to_aspeed_video(x) container_of((x), struct aspeed_video, v4l2_dev)
 
+struct aspeed_video_config {
+	u32 version;
+	u32 jpeg_mode;
+	u32 comp_size_read;
+};
+
+static const struct aspeed_video_config ast2400_config = {
+	.version = 4,
+	.jpeg_mode = AST2400_VE_SEQ_CTRL_JPEG_MODE,
+	.comp_size_read = AST2400_VE_COMP_SIZE_READ_BACK,
+};
+
+static const struct aspeed_video_config ast2500_config = {
+	.version = 5,
+	.jpeg_mode = AST2500_VE_SEQ_CTRL_JPEG_MODE,
+	.comp_size_read = AST2400_VE_COMP_SIZE_READ_BACK,
+};
+
+static const struct aspeed_video_config ast2600_config = {
+	.version = 6,
+	.jpeg_mode = AST2500_VE_SEQ_CTRL_JPEG_MODE,
+	.comp_size_read = AST2600_VE_COMP_SIZE_READ_BACK,
+};
+
 static const u32 aspeed_video_jpeg_header[ASPEED_VIDEO_JPEG_HEADER_SIZE] = {
 	0xe0ffd8ff, 0x464a1000, 0x01004649, 0x60000101, 0x00006000, 0x0f00feff,
 	0x00002d05, 0x00000000, 0x00000000, 0x00dbff00
@@ -640,7 +662,7 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
 	if (sts & VE_INTERRUPT_COMP_COMPLETE) {
 		struct aspeed_video_buffer *buf;
 		u32 frame_size = aspeed_video_read(video,
-						   VE_JPEG_COMP_SIZE_READ_BACK);
+						   video->config.comp_size_read);
 
 		update_perf(&video->perf);
 
@@ -973,7 +995,7 @@ static void aspeed_video_update_regs(struct aspeed_video *video)
 		FIELD_PREP(VE_COMP_CTRL_DCT_LUM, video->jpeg_quality) |
 		FIELD_PREP(VE_COMP_CTRL_DCT_CHR, video->jpeg_quality | 0x10);
 	u32 ctrl = 0;
-	u32 seq_ctrl = VE_SEQ_CTRL_JPEG_MODE;
+	u32 seq_ctrl = video->config.jpeg_mode;
 
 	v4l2_dbg(1, debug, &video->v4l2_dev, "framerate(%d)\n",
 		 video->frame_rate);
@@ -993,7 +1015,7 @@ static void aspeed_video_update_regs(struct aspeed_video *video)
 
 	/* Set control registers */
 	aspeed_video_update(video, VE_SEQ_CTRL,
-			    VE_SEQ_CTRL_JPEG_MODE | VE_SEQ_CTRL_YUV420,
+			    video->config.jpeg_mode | VE_SEQ_CTRL_YUV420,
 			    seq_ctrl);
 	aspeed_video_update(video, VE_CTRL, VE_CTRL_FRC, ctrl);
 	aspeed_video_update(video, VE_COMP_CTRL,
@@ -1790,8 +1812,18 @@ static int aspeed_video_init(struct aspeed_video *video)
 	return rc;
 }
 
+static const struct of_device_id aspeed_video_of_match[] = {
+	{ .compatible = "aspeed,ast2400-video-engine", .data = &ast2400_config },
+	{ .compatible = "aspeed,ast2500-video-engine", .data = &ast2500_config },
+	{ .compatible = "aspeed,ast2600-video-engine", .data = &ast2600_config },
+	{}
+};
+MODULE_DEVICE_TABLE(of, aspeed_video_of_match);
+
 static int aspeed_video_probe(struct platform_device *pdev)
 {
+	const struct aspeed_video_config *config;
+	const struct of_device_id *match;
 	int rc;
 	struct resource *res;
 	struct aspeed_video *video =
@@ -1815,6 +1847,13 @@ static int aspeed_video_probe(struct platform_device *pdev)
 	if (IS_ERR(video->base))
 		return PTR_ERR(video->base);
 
+	match = of_match_node(aspeed_video_of_match, pdev->dev.of_node);
+	if (!match)
+		return -EINVAL;
+
+	config = match->data;
+	video->config = *config;
+
 	rc = aspeed_video_init(video);
 	if (rc)
 		return rc;
@@ -1828,6 +1867,8 @@ static int aspeed_video_probe(struct platform_device *pdev)
 
 	aspeed_video_debugfs_create(video);
 
+	dev_info(video->dev, "compatible for g%d\n", config->version);
+
 	return 0;
 }
 
@@ -1860,14 +1901,6 @@ static int aspeed_video_remove(struct platform_device *pdev)
 	return 0;
 }
 
-static const struct of_device_id aspeed_video_of_match[] = {
-	{ .compatible = "aspeed,ast2400-video-engine" },
-	{ .compatible = "aspeed,ast2500-video-engine" },
-	{ .compatible = "aspeed,ast2600-video-engine" },
-	{}
-};
-MODULE_DEVICE_TABLE(of, aspeed_video_of_match);
-
 static struct platform_driver aspeed_video_driver = {
 	.driver = {
 		.name = DEVICE_NAME,
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v5 08/10] media: aspeed: Support aspeed mode to reduce compressed data

[Jammy Huang]

aspeed supports differential jpeg format which only compress the parts
which are changed. In this way, it reduces both the amount of data to be
transferred by network and those to be decoded on the client side.

4 new ctrls are added:
* Aspeed JPEG Format: to control aspeed's partial jpeg on/off
  0: standard jpeg, 1: aspeed jpeg
* Aspeed Compression Mode: to control aspeed's compression mode
  0: DCT Only, 1: DCT VQ mix 2-color, 2: DCT VQ mix 4-color
  This is AST2400 only. It will adapt JPEG or VQ encoding method according
  to the context automatically.
* Aspeed HQ Mode: to control aspeed's high quality(2-pass) compression mode
  This only works with yuv444 subsampling.
* Aspeed HQ Quality: to control the quality of aspeed's HQ mode
  only useful if Aspeed HQ mode is enabled

Aspeed JPEG Format requires an additional buffer, called bcd, to store
the information about which macro block in the new frame is different
from the previous one.

To have bcd correctly working, we need to swap the buffers for src0/1 to
make src1 refer to previous frame and src0 to the coming new frame.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5:
  - use runtime configuration to handle differences between soc
  - add aspeed_video_swap_src_buf().
v4
  - add include/uapi/linux/aspeed-video.h for V4L2_CID_ASPEED_XXX
  - add aspeed_video_set_format
  - remove aspeed_ctrl_format
  - add variable, format, to replace partial_jpeg
  - modify aspeed_ctrl_HQ_jpeg_quality's range
  - reduce bcd buffer size
  - update commit message
v3:
  - 'compression mode' only available for G4
v2:
  - update commit message
  - use v4l2_xxx log rather than self-defined macro
---
 drivers/media/platform/aspeed-video.c | 252 +++++++++++++++++++++++---
 include/uapi/linux/aspeed-video.h     |  15 ++
 2 files changed, 243 insertions(+), 24 deletions(-)
 create mode 100644 include/uapi/linux/aspeed-video.h

diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index ba8ee82b38c3..419825ed7737 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -31,6 +31,7 @@
 #include <media/v4l2-ioctl.h>
 #include <media/videobuf2-dma-contig.h>
 #include <linux/videodev2.h>
+#include <uapi/linux/aspeed-video.h>
 
 #define DEVICE_NAME			"aspeed-video"
 
@@ -55,6 +56,7 @@
 
 #define VE_MAX_SRC_BUFFER_SIZE		0x8ca000 /* 1920 * 1200, 32bpp */
 #define VE_JPEG_HEADER_SIZE		0x006000 /* 512 * 12 * 4 */
+#define VE_BCD_BUFF_SIZE		0x9000 /* (1920/8) * (1200/8) */
 
 #define VE_PROTECTION_KEY		0x000
 #define  VE_PROTECTION_KEY_UNLOCK	0x1a038aa8
@@ -105,6 +107,13 @@
 #define VE_SCALING_FILTER2		0x020
 #define VE_SCALING_FILTER3		0x024
 
+#define VE_BCD_CTRL			0x02C
+#define  VE_BCD_CTRL_EN_BCD		BIT(0)
+#define  VE_BCD_CTRL_EN_ABCD		BIT(1)
+#define  VE_BCD_CTRL_EN_CB		BIT(2)
+#define  VE_BCD_CTRL_THR		GENMASK(23, 16)
+#define  VE_BCD_CTRL_ABCD_THR		GENMASK(31, 24)
+
 #define VE_CAP_WINDOW			0x030
 #define VE_COMP_WINDOW			0x034
 #define VE_COMP_PROC_OFFSET		0x038
@@ -113,6 +122,7 @@
 #define VE_SRC0_ADDR			0x044
 #define VE_SRC_SCANLINE_OFFSET		0x048
 #define VE_SRC1_ADDR			0x04c
+#define VE_BCD_ADDR			0x050
 #define VE_COMP_ADDR			0x054
 
 #define VE_STREAM_BUF_SIZE		0x058
@@ -133,6 +143,8 @@
 #define  VE_COMP_CTRL_HQ_DCT_CHR	GENMASK(26, 22)
 #define  VE_COMP_CTRL_HQ_DCT_LUM	GENMASK(31, 27)
 
+#define VE_CB_ADDR			0x06C
+
 #define AST2400_VE_COMP_SIZE_READ_BACK	0x078
 #define AST2600_VE_COMP_SIZE_READ_BACK	0x084
 
@@ -191,6 +203,12 @@ enum {
 	VIDEO_CLOCKS_ON,
 };
 
+enum aspeed_video_format {
+	VIDEO_FMT_STANDARD = 0,
+	VIDEO_FMT_ASPEED,
+	VIDEO_FMT_MAX = VIDEO_FMT_ASPEED
+};
+
 struct aspeed_video_addr {
 	unsigned int size;
 	dma_addr_t dma;
@@ -241,10 +259,15 @@ struct aspeed_video {
 	unsigned int max_compressed_size;
 	struct aspeed_video_addr srcs[2];
 	struct aspeed_video_addr jpeg;
+	struct aspeed_video_addr bcd;
 
 	bool yuv420;
+	enum aspeed_video_format format;
+	bool hq_mode;
 	unsigned int frame_rate;
 	unsigned int jpeg_quality;
+	unsigned int jpeg_hq_quality;
+	unsigned int compression_mode;
 
 	unsigned int frame_bottom;
 	unsigned int frame_left;
@@ -413,8 +436,20 @@ static const struct v4l2_dv_timings_cap aspeed_video_timings_cap = {
 	},
 };
 
+static const char * const compress_mode_str[] = {"DCT Only",
+	"DCT VQ mix 2-color", "DCT VQ mix 4-color"};
+static const char * const format_str[] = {"Standard JPEG",
+	"Aspeed JPEG", "Partial JPEG"};
+
 static unsigned int debug;
 
+static bool aspeed_video_alloc_buf(struct aspeed_video *video,
+				   struct aspeed_video_addr *addr,
+				   unsigned int size);
+
+static void aspeed_video_free_buf(struct aspeed_video *video,
+				  struct aspeed_video_addr *addr);
+
 static void aspeed_video_init_jpeg_table(u32 *table, bool yuv420)
 {
 	int i;
@@ -502,6 +537,7 @@ static int aspeed_video_start_frame(struct aspeed_video *video)
 	unsigned long flags;
 	struct aspeed_video_buffer *buf;
 	u32 seq_ctrl = aspeed_video_read(video, VE_SEQ_CTRL);
+	bool bcd_buf_need = (video->format != VIDEO_FMT_STANDARD);
 
 	if (video->v4l2_input_status) {
 		v4l2_warn(&video->v4l2_dev, "No signal; don't start frame\n");
@@ -514,6 +550,20 @@ static int aspeed_video_start_frame(struct aspeed_video *video)
 		return -EBUSY;
 	}
 
+	if (bcd_buf_need && !video->bcd.size) {
+		if (!aspeed_video_alloc_buf(video, &video->bcd,
+					    VE_BCD_BUFF_SIZE)) {
+			dev_err(video->dev, "Failed to allocate BCD buffer\n");
+			dev_err(video->dev, "don't start frame\n");
+			return -ENOMEM;
+		}
+		aspeed_video_write(video, VE_BCD_ADDR, video->bcd.dma);
+		v4l2_dbg(1, debug, &video->v4l2_dev, "bcd addr(%#x) size(%d)\n",
+			 video->bcd.dma, video->bcd.size);
+	} else if (!bcd_buf_need && video->bcd.size) {
+		aspeed_video_free_buf(video, &video->bcd);
+	}
+
 	spin_lock_irqsave(&video->lock, flags);
 	buf = list_first_entry_or_null(&video->buffers,
 				       struct aspeed_video_buffer, link);
@@ -610,6 +660,20 @@ static void aspeed_video_irq_res_change(struct aspeed_video *video, ulong delay)
 	schedule_delayed_work(&video->res_work, delay);
 }
 
+static void aspeed_video_swap_src_buf(struct aspeed_video *v)
+{
+	if (v->format == VIDEO_FMT_STANDARD)
+		return;
+
+	if (v->sequence & 0x01) {
+		aspeed_video_write(v, VE_SRC0_ADDR, v->srcs[1].dma);
+		aspeed_video_write(v, VE_SRC1_ADDR, v->srcs[0].dma);
+	} else {
+		aspeed_video_write(v, VE_SRC0_ADDR, v->srcs[0].dma);
+		aspeed_video_write(v, VE_SRC1_ADDR, v->srcs[1].dma);
+	}
+}
+
 /*
  * Interrupts that we don't use but have to explicitly ignore because the
  * hardware asserts them even when they're disabled in the VE_INTERRUPT_CTRL
@@ -661,6 +725,7 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
 
 	if (sts & VE_INTERRUPT_COMP_COMPLETE) {
 		struct aspeed_video_buffer *buf;
+		bool empty = true;
 		u32 frame_size = aspeed_video_read(video,
 						   video->config.comp_size_read);
 
@@ -674,13 +739,23 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
 		if (buf) {
 			vb2_set_plane_payload(&buf->vb.vb2_buf, 0, frame_size);
 
-			if (!list_is_last(&buf->link, &video->buffers)) {
+			/*
+			 * partial_jpeg requires continuous update.
+			 * On the contrary, standard jpeg can keep last buffer
+			 * to always have the latest result.
+			 */
+			if (video->format == VIDEO_FMT_STANDARD &&
+			    list_is_last(&buf->link, &video->buffers)) {
+				empty = false;
+				v4l2_warn(&video->v4l2_dev, "skip to keep last frame updated\n");
+			} else {
 				buf->vb.vb2_buf.timestamp = ktime_get_ns();
 				buf->vb.sequence = video->sequence++;
 				buf->vb.field = V4L2_FIELD_NONE;
 				vb2_buffer_done(&buf->vb.vb2_buf,
 						VB2_BUF_STATE_DONE);
 				list_del(&buf->link);
+				empty = list_empty(&video->buffers);
 			}
 		}
 		spin_unlock(&video->lock);
@@ -694,7 +769,10 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
 		aspeed_video_write(video, VE_INTERRUPT_STATUS,
 				   VE_INTERRUPT_COMP_COMPLETE);
 		sts &= ~VE_INTERRUPT_COMP_COMPLETE;
-		if (test_bit(VIDEO_STREAMING, &video->flags) && buf)
+
+		aspeed_video_swap_src_buf(video);
+
+		if (test_bit(VIDEO_STREAMING, &video->flags) && !empty)
 			aspeed_video_start_frame(video);
 	}
 
@@ -953,10 +1031,14 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
 				   FIELD_PREP(VE_TGS_FIRST, video->frame_top) |
 				   FIELD_PREP(VE_TGS_LAST,
 					      video->frame_bottom + 1));
-		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_INT_DE);
+		aspeed_video_update(video, VE_CTRL,
+				    VE_CTRL_INT_DE | VE_CTRL_DIRECT_FETCH,
+				    VE_CTRL_INT_DE);
 	} else {
 		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Direct Mode\n");
-		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_DIRECT_FETCH);
+		aspeed_video_update(video, VE_CTRL,
+				    VE_CTRL_INT_DE | VE_CTRL_DIRECT_FETCH,
+				    VE_CTRL_DIRECT_FETCH);
 	}
 
 	size *= 4;
@@ -991,35 +1073,71 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
 
 static void aspeed_video_update_regs(struct aspeed_video *video)
 {
-	u32 comp_ctrl = VE_COMP_CTRL_RSVD |
-		FIELD_PREP(VE_COMP_CTRL_DCT_LUM, video->jpeg_quality) |
-		FIELD_PREP(VE_COMP_CTRL_DCT_CHR, video->jpeg_quality | 0x10);
+	u8 jpeg_hq_quality = clamp((int)video->jpeg_hq_quality - 1, 0,
+				   ASPEED_VIDEO_JPEG_NUM_QUALITIES - 1);
+	u32 comp_ctrl =	FIELD_PREP(VE_COMP_CTRL_DCT_LUM, video->jpeg_quality) |
+		FIELD_PREP(VE_COMP_CTRL_DCT_CHR, video->jpeg_quality | 0x10) |
+		FIELD_PREP(VE_COMP_CTRL_EN_HQ, video->hq_mode) |
+		FIELD_PREP(VE_COMP_CTRL_HQ_DCT_LUM, jpeg_hq_quality) |
+		FIELD_PREP(VE_COMP_CTRL_HQ_DCT_CHR, jpeg_hq_quality |
+			   0x10);
 	u32 ctrl = 0;
-	u32 seq_ctrl = video->config.jpeg_mode;
+	u32 seq_ctrl = 0;
 
-	v4l2_dbg(1, debug, &video->v4l2_dev, "framerate(%d)\n",
-		 video->frame_rate);
-	v4l2_dbg(1, debug, &video->v4l2_dev, "subsample(%s)\n",
+	v4l2_dbg(1, debug, &video->v4l2_dev, "framerate(%d)\n", video->frame_rate);
+	v4l2_dbg(1, debug, &video->v4l2_dev, "jpeg format(%s) subsample(%s)\n",
+		 format_str[video->format],
 		 video->yuv420 ? "420" : "444");
-	v4l2_dbg(1, debug, &video->v4l2_dev, "compression quality(%d)\n",
-		 video->jpeg_quality);
+	v4l2_dbg(1, debug, &video->v4l2_dev, "compression quality(%d) hq(%s) hq_quality(%d)\n",
+		 video->jpeg_quality, video->hq_mode ? "on" : "off",
+		 video->jpeg_hq_quality);
+	v4l2_dbg(1, debug, &video->v4l2_dev, "compression mode(%s)\n",
+		 compress_mode_str[video->compression_mode]);
+
+	if (video->format == VIDEO_FMT_ASPEED)
+		aspeed_video_update(video, VE_BCD_CTRL, 0, VE_BCD_CTRL_EN_BCD);
+	else
+		aspeed_video_update(video, VE_BCD_CTRL, VE_BCD_CTRL_EN_BCD, 0);
 
 	if (video->frame_rate)
 		ctrl |= FIELD_PREP(VE_CTRL_FRC, video->frame_rate);
 
+	if (video->format == VIDEO_FMT_STANDARD) {
+		comp_ctrl &= ~FIELD_PREP(VE_COMP_CTRL_EN_HQ, video->hq_mode);
+		seq_ctrl |= video->config.jpeg_mode;
+	}
+
 	if (video->yuv420)
 		seq_ctrl |= VE_SEQ_CTRL_YUV420;
 
 	if (video->jpeg.virt)
 		aspeed_video_update_jpeg_table(video->jpeg.virt, video->yuv420);
 
+	if (video->config.version == 4) {
+		switch (video->compression_mode) {
+		case 0:	//DCT only
+			comp_ctrl |= VE_COMP_CTRL_VQ_DCT_ONLY;
+			break;
+		case 1:	//DCT VQ mix 2-color
+			comp_ctrl &= ~(VE_COMP_CTRL_VQ_4COLOR |
+				     VE_COMP_CTRL_VQ_DCT_ONLY);
+			break;
+		case 2:	//DCT VQ mix 4-color
+			comp_ctrl |= VE_COMP_CTRL_VQ_4COLOR;
+			break;
+		}
+	}
+
 	/* Set control registers */
 	aspeed_video_update(video, VE_SEQ_CTRL,
 			    video->config.jpeg_mode | VE_SEQ_CTRL_YUV420,
 			    seq_ctrl);
 	aspeed_video_update(video, VE_CTRL, VE_CTRL_FRC, ctrl);
 	aspeed_video_update(video, VE_COMP_CTRL,
-			    VE_COMP_CTRL_DCT_LUM | VE_COMP_CTRL_DCT_CHR,
+			    VE_COMP_CTRL_DCT_LUM | VE_COMP_CTRL_DCT_CHR |
+			    VE_COMP_CTRL_EN_HQ | VE_COMP_CTRL_HQ_DCT_LUM |
+			    VE_COMP_CTRL_HQ_DCT_CHR | VE_COMP_CTRL_VQ_4COLOR |
+			    VE_COMP_CTRL_VQ_DCT_ONLY,
 			    comp_ctrl);
 }
 
@@ -1051,6 +1169,8 @@ static void aspeed_video_init_regs(struct aspeed_video *video)
 
 	/* Set mode detection defaults */
 	aspeed_video_write(video, VE_MODE_DETECT, 0x22666500);
+
+	aspeed_video_write(video, VE_BCD_CTRL, 0);
 }
 
 static void aspeed_video_start(struct aspeed_video *video)
@@ -1084,6 +1204,9 @@ static void aspeed_video_stop(struct aspeed_video *video)
 	if (video->srcs[1].size)
 		aspeed_video_free_buf(video, &video->srcs[1]);
 
+	if (video->bcd.size)
+		aspeed_video_free_buf(video, &video->bcd);
+
 	video->v4l2_input_status = V4L2_IN_ST_NO_SIGNAL;
 	video->flags = 0;
 }
@@ -1102,10 +1225,12 @@ static int aspeed_video_querycap(struct file *file, void *fh,
 static int aspeed_video_enum_format(struct file *file, void *fh,
 				    struct v4l2_fmtdesc *f)
 {
+	struct aspeed_video *video = video_drvdata(file);
+
 	if (f->index)
 		return -EINVAL;
 
-	f->pixelformat = V4L2_PIX_FMT_JPEG;
+	f->pixelformat = video->pix_fmt.pixelformat;
 
 	return 0;
 }
@@ -1120,6 +1245,29 @@ static int aspeed_video_get_format(struct file *file, void *fh,
 	return 0;
 }
 
+static int aspeed_video_set_format(struct file *file, void *fh,
+				   struct v4l2_format *f)
+{
+	struct aspeed_video *video = video_drvdata(file);
+
+	if (vb2_is_busy(&video->queue))
+		return -EBUSY;
+
+	switch (f->fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_JPEG:
+		video->format = VIDEO_FMT_STANDARD;
+		break;
+	case V4L2_PIX_FMT_AJPG:
+		video->format = VIDEO_FMT_ASPEED;
+		break;
+	default:
+		return -EINVAL;
+	}
+	video->pix_fmt.pixelformat = f->fmt.pix.pixelformat;
+
+	return 0;
+}
+
 static int aspeed_video_enum_input(struct file *file, void *fh,
 				   struct v4l2_input *inp)
 {
@@ -1337,7 +1485,7 @@ static const struct v4l2_ioctl_ops aspeed_video_ioctl_ops = {
 
 	.vidioc_enum_fmt_vid_cap = aspeed_video_enum_format,
 	.vidioc_g_fmt_vid_cap = aspeed_video_get_format,
-	.vidioc_s_fmt_vid_cap = aspeed_video_get_format,
+	.vidioc_s_fmt_vid_cap = aspeed_video_set_format,
 	.vidioc_try_fmt_vid_cap = aspeed_video_get_format,
 
 	.vidioc_reqbufs = vb2_ioctl_reqbufs,
@@ -1386,6 +1534,24 @@ static int aspeed_video_set_ctrl(struct v4l2_ctrl *ctrl)
 		if (test_bit(VIDEO_STREAMING, &video->flags))
 			aspeed_video_update_regs(video);
 		break;
+	case V4L2_CID_ASPEED_COMPRESSION_MODE:
+		if (video->config.version > 4)
+			return -EINVAL;
+
+		video->compression_mode = ctrl->val;
+		if (test_bit(VIDEO_STREAMING, &video->flags))
+			aspeed_video_update_regs(video);
+		break;
+	case V4L2_CID_ASPEED_HQ_MODE:
+		video->hq_mode = ctrl->val;
+		if (test_bit(VIDEO_STREAMING, &video->flags))
+			aspeed_video_update_regs(video);
+		break;
+	case V4L2_CID_ASPEED_HQ_JPEG_QUALITY:
+		video->jpeg_hq_quality = ctrl->val;
+		if (test_bit(VIDEO_STREAMING, &video->flags))
+			aspeed_video_update_regs(video);
+		break;
 	default:
 		return -EINVAL;
 	}
@@ -1397,6 +1563,39 @@ static const struct v4l2_ctrl_ops aspeed_video_ctrl_ops = {
 	.s_ctrl = aspeed_video_set_ctrl,
 };
 
+static const struct v4l2_ctrl_config aspeed_ctrl_compression_mode = {
+	.ops = &aspeed_video_ctrl_ops,
+	.id = V4L2_CID_ASPEED_COMPRESSION_MODE,
+	.name = "Aspeed Compression Mode",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = 0,
+	.max = 2,
+	.step = 1,
+	.def = 0,
+};
+
+static const struct v4l2_ctrl_config aspeed_ctrl_HQ_mode = {
+	.ops = &aspeed_video_ctrl_ops,
+	.id = V4L2_CID_ASPEED_HQ_MODE,
+	.name = "Aspeed HQ Mode",
+	.type = V4L2_CTRL_TYPE_BOOLEAN,
+	.min = false,
+	.max = true,
+	.step = 1,
+	.def = false,
+};
+
+static const struct v4l2_ctrl_config aspeed_ctrl_HQ_jpeg_quality = {
+	.ops = &aspeed_video_ctrl_ops,
+	.id = V4L2_CID_ASPEED_HQ_JPEG_QUALITY,
+	.name = "Aspeed HQ Quality",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = 1,
+	.max = ASPEED_VIDEO_JPEG_NUM_QUALITIES,
+	.step = 1,
+	.def = 1,
+};
+
 static void aspeed_video_resolution_work(struct work_struct *work)
 {
 	struct delayed_work *dwork = to_delayed_work(work);
@@ -1619,7 +1818,6 @@ static int aspeed_video_debugfs_show(struct seq_file *s, void *data)
 		   (v->perf.totaltime && v->sequence) ?
 		   1000/(v->perf.totaltime/v->sequence) : 0);
 
-
 	return 0;
 }
 
@@ -1663,6 +1861,7 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
 	struct v4l2_device *v4l2_dev = &video->v4l2_dev;
 	struct vb2_queue *vbq = &video->queue;
 	struct video_device *vdev = &video->vdev;
+	struct v4l2_ctrl_handler *hdl = &video->ctrl_handler;
 	int rc;
 
 	video->pix_fmt.pixelformat = V4L2_PIX_FMT_JPEG;
@@ -1677,22 +1876,26 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
 		return rc;
 	}
 
-	v4l2_ctrl_handler_init(&video->ctrl_handler, 2);
-	v4l2_ctrl_new_std(&video->ctrl_handler, &aspeed_video_ctrl_ops,
-			  V4L2_CID_JPEG_COMPRESSION_QUALITY, 0,
-			  ASPEED_VIDEO_JPEG_NUM_QUALITIES - 1, 1, 0);
-	v4l2_ctrl_new_std_menu(&video->ctrl_handler, &aspeed_video_ctrl_ops,
+	v4l2_ctrl_handler_init(hdl, 6);
+	v4l2_ctrl_new_std(hdl, &aspeed_video_ctrl_ops,
+			  V4L2_CID_JPEG_COMPRESSION_QUALITY, 1,
+			  ASPEED_VIDEO_JPEG_NUM_QUALITIES, 1, 1);
+	v4l2_ctrl_new_std_menu(hdl, &aspeed_video_ctrl_ops,
 			       V4L2_CID_JPEG_CHROMA_SUBSAMPLING,
 			       V4L2_JPEG_CHROMA_SUBSAMPLING_420, mask,
 			       V4L2_JPEG_CHROMA_SUBSAMPLING_444);
+	if (video->config.version == 4)
+		v4l2_ctrl_new_custom(hdl, &aspeed_ctrl_compression_mode, NULL);
+	v4l2_ctrl_new_custom(hdl, &aspeed_ctrl_HQ_mode, NULL);
+	v4l2_ctrl_new_custom(hdl, &aspeed_ctrl_HQ_jpeg_quality, NULL);
 
-	rc = video->ctrl_handler.error;
+	rc = hdl->error;
 	if (rc) {
 		dev_err(video->dev, "Failed to init controls: %d\n", rc);
 		goto err_ctrl_init;
 	}
 
-	v4l2_dev->ctrl_handler = &video->ctrl_handler;
+	v4l2_dev->ctrl_handler = hdl;
 
 	vbq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 	vbq->io_modes = VB2_MMAP | VB2_READ | VB2_DMABUF;
@@ -1833,6 +2036,7 @@ static int aspeed_video_probe(struct platform_device *pdev)
 		return -ENOMEM;
 
 	video->frame_rate = 30;
+	video->jpeg_hq_quality = 1;
 	video->dev = &pdev->dev;
 	spin_lock_init(&video->lock);
 	mutex_init(&video->video_lock);
diff --git a/include/uapi/linux/aspeed-video.h b/include/uapi/linux/aspeed-video.h
new file mode 100644
index 000000000000..117bfeaaa83a
--- /dev/null
+++ b/include/uapi/linux/aspeed-video.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2021 ASPEED Technology Inc.
+ */
+
+#ifndef _UAPI_LINUX_ASPEED_VIDEO_H
+#define _UAPI_LINUX_ASPEED_VIDEO_H
+
+#include <linux/v4l2-controls.h>
+
+#define V4L2_CID_ASPEED_COMPRESSION_MODE	(V4L2_CID_USER_ASPEED_BASE  + 1)
+#define V4L2_CID_ASPEED_HQ_MODE			(V4L2_CID_USER_ASPEED_BASE  + 2)
+#define V4L2_CID_ASPEED_HQ_JPEG_QUALITY		(V4L2_CID_USER_ASPEED_BASE  + 3)
+
+#endif /* _UAPI_LINUX_ASPEED_VIDEO_H */
-- 
2.25.1

[PATCH v5 08/10] media: aspeed: Support aspeed mode to reduce compressed data

[Jammy Huang]

aspeed supports differential jpeg format which only compress the parts
which are changed. In this way, it reduces both the amount of data to be
transferred by network and those to be decoded on the client side.

4 new ctrls are added:
* Aspeed JPEG Format: to control aspeed's partial jpeg on/off
  0: standard jpeg, 1: aspeed jpeg
* Aspeed Compression Mode: to control aspeed's compression mode
  0: DCT Only, 1: DCT VQ mix 2-color, 2: DCT VQ mix 4-color
  This is AST2400 only. It will adapt JPEG or VQ encoding method according
  to the context automatically.
* Aspeed HQ Mode: to control aspeed's high quality(2-pass) compression mode
  This only works with yuv444 subsampling.
* Aspeed HQ Quality: to control the quality of aspeed's HQ mode
  only useful if Aspeed HQ mode is enabled

Aspeed JPEG Format requires an additional buffer, called bcd, to store
the information about which macro block in the new frame is different
from the previous one.

To have bcd correctly working, we need to swap the buffers for src0/1 to
make src1 refer to previous frame and src0 to the coming new frame.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5:
  - use runtime configuration to handle differences between soc
  - add aspeed_video_swap_src_buf().
v4
  - add include/uapi/linux/aspeed-video.h for V4L2_CID_ASPEED_XXX
  - add aspeed_video_set_format
  - remove aspeed_ctrl_format
  - add variable, format, to replace partial_jpeg
  - modify aspeed_ctrl_HQ_jpeg_quality's range
  - reduce bcd buffer size
  - update commit message
v3:
  - 'compression mode' only available for G4
v2:
  - update commit message
  - use v4l2_xxx log rather than self-defined macro
---
 drivers/media/platform/aspeed-video.c | 252 +++++++++++++++++++++++---
 include/uapi/linux/aspeed-video.h     |  15 ++
 2 files changed, 243 insertions(+), 24 deletions(-)
 create mode 100644 include/uapi/linux/aspeed-video.h

diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index ba8ee82b38c3..419825ed7737 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -31,6 +31,7 @@
 #include <media/v4l2-ioctl.h>
 #include <media/videobuf2-dma-contig.h>
 #include <linux/videodev2.h>
+#include <uapi/linux/aspeed-video.h>
 
 #define DEVICE_NAME			"aspeed-video"
 
@@ -55,6 +56,7 @@
 
 #define VE_MAX_SRC_BUFFER_SIZE		0x8ca000 /* 1920 * 1200, 32bpp */
 #define VE_JPEG_HEADER_SIZE		0x006000 /* 512 * 12 * 4 */
+#define VE_BCD_BUFF_SIZE		0x9000 /* (1920/8) * (1200/8) */
 
 #define VE_PROTECTION_KEY		0x000
 #define  VE_PROTECTION_KEY_UNLOCK	0x1a038aa8
@@ -105,6 +107,13 @@
 #define VE_SCALING_FILTER2		0x020
 #define VE_SCALING_FILTER3		0x024
 
+#define VE_BCD_CTRL			0x02C
+#define  VE_BCD_CTRL_EN_BCD		BIT(0)
+#define  VE_BCD_CTRL_EN_ABCD		BIT(1)
+#define  VE_BCD_CTRL_EN_CB		BIT(2)
+#define  VE_BCD_CTRL_THR		GENMASK(23, 16)
+#define  VE_BCD_CTRL_ABCD_THR		GENMASK(31, 24)
+
 #define VE_CAP_WINDOW			0x030
 #define VE_COMP_WINDOW			0x034
 #define VE_COMP_PROC_OFFSET		0x038
@@ -113,6 +122,7 @@
 #define VE_SRC0_ADDR			0x044
 #define VE_SRC_SCANLINE_OFFSET		0x048
 #define VE_SRC1_ADDR			0x04c
+#define VE_BCD_ADDR			0x050
 #define VE_COMP_ADDR			0x054
 
 #define VE_STREAM_BUF_SIZE		0x058
@@ -133,6 +143,8 @@
 #define  VE_COMP_CTRL_HQ_DCT_CHR	GENMASK(26, 22)
 #define  VE_COMP_CTRL_HQ_DCT_LUM	GENMASK(31, 27)
 
+#define VE_CB_ADDR			0x06C
+
 #define AST2400_VE_COMP_SIZE_READ_BACK	0x078
 #define AST2600_VE_COMP_SIZE_READ_BACK	0x084
 
@@ -191,6 +203,12 @@ enum {
 	VIDEO_CLOCKS_ON,
 };
 
+enum aspeed_video_format {
+	VIDEO_FMT_STANDARD = 0,
+	VIDEO_FMT_ASPEED,
+	VIDEO_FMT_MAX = VIDEO_FMT_ASPEED
+};
+
 struct aspeed_video_addr {
 	unsigned int size;
 	dma_addr_t dma;
@@ -241,10 +259,15 @@ struct aspeed_video {
 	unsigned int max_compressed_size;
 	struct aspeed_video_addr srcs[2];
 	struct aspeed_video_addr jpeg;
+	struct aspeed_video_addr bcd;
 
 	bool yuv420;
+	enum aspeed_video_format format;
+	bool hq_mode;
 	unsigned int frame_rate;
 	unsigned int jpeg_quality;
+	unsigned int jpeg_hq_quality;
+	unsigned int compression_mode;
 
 	unsigned int frame_bottom;
 	unsigned int frame_left;
@@ -413,8 +436,20 @@ static const struct v4l2_dv_timings_cap aspeed_video_timings_cap = {
 	},
 };
 
+static const char * const compress_mode_str[] = {"DCT Only",
+	"DCT VQ mix 2-color", "DCT VQ mix 4-color"};
+static const char * const format_str[] = {"Standard JPEG",
+	"Aspeed JPEG", "Partial JPEG"};
+
 static unsigned int debug;
 
+static bool aspeed_video_alloc_buf(struct aspeed_video *video,
+				   struct aspeed_video_addr *addr,
+				   unsigned int size);
+
+static void aspeed_video_free_buf(struct aspeed_video *video,
+				  struct aspeed_video_addr *addr);
+
 static void aspeed_video_init_jpeg_table(u32 *table, bool yuv420)
 {
 	int i;
@@ -502,6 +537,7 @@ static int aspeed_video_start_frame(struct aspeed_video *video)
 	unsigned long flags;
 	struct aspeed_video_buffer *buf;
 	u32 seq_ctrl = aspeed_video_read(video, VE_SEQ_CTRL);
+	bool bcd_buf_need = (video->format != VIDEO_FMT_STANDARD);
 
 	if (video->v4l2_input_status) {
 		v4l2_warn(&video->v4l2_dev, "No signal; don't start frame\n");
@@ -514,6 +550,20 @@ static int aspeed_video_start_frame(struct aspeed_video *video)
 		return -EBUSY;
 	}
 
+	if (bcd_buf_need && !video->bcd.size) {
+		if (!aspeed_video_alloc_buf(video, &video->bcd,
+					    VE_BCD_BUFF_SIZE)) {
+			dev_err(video->dev, "Failed to allocate BCD buffer\n");
+			dev_err(video->dev, "don't start frame\n");
+			return -ENOMEM;
+		}
+		aspeed_video_write(video, VE_BCD_ADDR, video->bcd.dma);
+		v4l2_dbg(1, debug, &video->v4l2_dev, "bcd addr(%#x) size(%d)\n",
+			 video->bcd.dma, video->bcd.size);
+	} else if (!bcd_buf_need && video->bcd.size) {
+		aspeed_video_free_buf(video, &video->bcd);
+	}
+
 	spin_lock_irqsave(&video->lock, flags);
 	buf = list_first_entry_or_null(&video->buffers,
 				       struct aspeed_video_buffer, link);
@@ -610,6 +660,20 @@ static void aspeed_video_irq_res_change(struct aspeed_video *video, ulong delay)
 	schedule_delayed_work(&video->res_work, delay);
 }
 
+static void aspeed_video_swap_src_buf(struct aspeed_video *v)
+{
+	if (v->format == VIDEO_FMT_STANDARD)
+		return;
+
+	if (v->sequence & 0x01) {
+		aspeed_video_write(v, VE_SRC0_ADDR, v->srcs[1].dma);
+		aspeed_video_write(v, VE_SRC1_ADDR, v->srcs[0].dma);
+	} else {
+		aspeed_video_write(v, VE_SRC0_ADDR, v->srcs[0].dma);
+		aspeed_video_write(v, VE_SRC1_ADDR, v->srcs[1].dma);
+	}
+}
+
 /*
  * Interrupts that we don't use but have to explicitly ignore because the
  * hardware asserts them even when they're disabled in the VE_INTERRUPT_CTRL
@@ -661,6 +725,7 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
 
 	if (sts & VE_INTERRUPT_COMP_COMPLETE) {
 		struct aspeed_video_buffer *buf;
+		bool empty = true;
 		u32 frame_size = aspeed_video_read(video,
 						   video->config.comp_size_read);
 
@@ -674,13 +739,23 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
 		if (buf) {
 			vb2_set_plane_payload(&buf->vb.vb2_buf, 0, frame_size);
 
-			if (!list_is_last(&buf->link, &video->buffers)) {
+			/*
+			 * partial_jpeg requires continuous update.
+			 * On the contrary, standard jpeg can keep last buffer
+			 * to always have the latest result.
+			 */
+			if (video->format == VIDEO_FMT_STANDARD &&
+			    list_is_last(&buf->link, &video->buffers)) {
+				empty = false;
+				v4l2_warn(&video->v4l2_dev, "skip to keep last frame updated\n");
+			} else {
 				buf->vb.vb2_buf.timestamp = ktime_get_ns();
 				buf->vb.sequence = video->sequence++;
 				buf->vb.field = V4L2_FIELD_NONE;
 				vb2_buffer_done(&buf->vb.vb2_buf,
 						VB2_BUF_STATE_DONE);
 				list_del(&buf->link);
+				empty = list_empty(&video->buffers);
 			}
 		}
 		spin_unlock(&video->lock);
@@ -694,7 +769,10 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
 		aspeed_video_write(video, VE_INTERRUPT_STATUS,
 				   VE_INTERRUPT_COMP_COMPLETE);
 		sts &= ~VE_INTERRUPT_COMP_COMPLETE;
-		if (test_bit(VIDEO_STREAMING, &video->flags) && buf)
+
+		aspeed_video_swap_src_buf(video);
+
+		if (test_bit(VIDEO_STREAMING, &video->flags) && !empty)
 			aspeed_video_start_frame(video);
 	}
 
@@ -953,10 +1031,14 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
 				   FIELD_PREP(VE_TGS_FIRST, video->frame_top) |
 				   FIELD_PREP(VE_TGS_LAST,
 					      video->frame_bottom + 1));
-		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_INT_DE);
+		aspeed_video_update(video, VE_CTRL,
+				    VE_CTRL_INT_DE | VE_CTRL_DIRECT_FETCH,
+				    VE_CTRL_INT_DE);
 	} else {
 		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Direct Mode\n");
-		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_DIRECT_FETCH);
+		aspeed_video_update(video, VE_CTRL,
+				    VE_CTRL_INT_DE | VE_CTRL_DIRECT_FETCH,
+				    VE_CTRL_DIRECT_FETCH);
 	}
 
 	size *= 4;
@@ -991,35 +1073,71 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
 
 static void aspeed_video_update_regs(struct aspeed_video *video)
 {
-	u32 comp_ctrl = VE_COMP_CTRL_RSVD |
-		FIELD_PREP(VE_COMP_CTRL_DCT_LUM, video->jpeg_quality) |
-		FIELD_PREP(VE_COMP_CTRL_DCT_CHR, video->jpeg_quality | 0x10);
+	u8 jpeg_hq_quality = clamp((int)video->jpeg_hq_quality - 1, 0,
+				   ASPEED_VIDEO_JPEG_NUM_QUALITIES - 1);
+	u32 comp_ctrl =	FIELD_PREP(VE_COMP_CTRL_DCT_LUM, video->jpeg_quality) |
+		FIELD_PREP(VE_COMP_CTRL_DCT_CHR, video->jpeg_quality | 0x10) |
+		FIELD_PREP(VE_COMP_CTRL_EN_HQ, video->hq_mode) |
+		FIELD_PREP(VE_COMP_CTRL_HQ_DCT_LUM, jpeg_hq_quality) |
+		FIELD_PREP(VE_COMP_CTRL_HQ_DCT_CHR, jpeg_hq_quality |
+			   0x10);
 	u32 ctrl = 0;
-	u32 seq_ctrl = video->config.jpeg_mode;
+	u32 seq_ctrl = 0;
 
-	v4l2_dbg(1, debug, &video->v4l2_dev, "framerate(%d)\n",
-		 video->frame_rate);
-	v4l2_dbg(1, debug, &video->v4l2_dev, "subsample(%s)\n",
+	v4l2_dbg(1, debug, &video->v4l2_dev, "framerate(%d)\n", video->frame_rate);
+	v4l2_dbg(1, debug, &video->v4l2_dev, "jpeg format(%s) subsample(%s)\n",
+		 format_str[video->format],
 		 video->yuv420 ? "420" : "444");
-	v4l2_dbg(1, debug, &video->v4l2_dev, "compression quality(%d)\n",
-		 video->jpeg_quality);
+	v4l2_dbg(1, debug, &video->v4l2_dev, "compression quality(%d) hq(%s) hq_quality(%d)\n",
+		 video->jpeg_quality, video->hq_mode ? "on" : "off",
+		 video->jpeg_hq_quality);
+	v4l2_dbg(1, debug, &video->v4l2_dev, "compression mode(%s)\n",
+		 compress_mode_str[video->compression_mode]);
+
+	if (video->format == VIDEO_FMT_ASPEED)
+		aspeed_video_update(video, VE_BCD_CTRL, 0, VE_BCD_CTRL_EN_BCD);
+	else
+		aspeed_video_update(video, VE_BCD_CTRL, VE_BCD_CTRL_EN_BCD, 0);
 
 	if (video->frame_rate)
 		ctrl |= FIELD_PREP(VE_CTRL_FRC, video->frame_rate);
 
+	if (video->format == VIDEO_FMT_STANDARD) {
+		comp_ctrl &= ~FIELD_PREP(VE_COMP_CTRL_EN_HQ, video->hq_mode);
+		seq_ctrl |= video->config.jpeg_mode;
+	}
+
 	if (video->yuv420)
 		seq_ctrl |= VE_SEQ_CTRL_YUV420;
 
 	if (video->jpeg.virt)
 		aspeed_video_update_jpeg_table(video->jpeg.virt, video->yuv420);
 
+	if (video->config.version == 4) {
+		switch (video->compression_mode) {
+		case 0:	//DCT only
+			comp_ctrl |= VE_COMP_CTRL_VQ_DCT_ONLY;
+			break;
+		case 1:	//DCT VQ mix 2-color
+			comp_ctrl &= ~(VE_COMP_CTRL_VQ_4COLOR |
+				     VE_COMP_CTRL_VQ_DCT_ONLY);
+			break;
+		case 2:	//DCT VQ mix 4-color
+			comp_ctrl |= VE_COMP_CTRL_VQ_4COLOR;
+			break;
+		}
+	}
+
 	/* Set control registers */
 	aspeed_video_update(video, VE_SEQ_CTRL,
 			    video->config.jpeg_mode | VE_SEQ_CTRL_YUV420,
 			    seq_ctrl);
 	aspeed_video_update(video, VE_CTRL, VE_CTRL_FRC, ctrl);
 	aspeed_video_update(video, VE_COMP_CTRL,
-			    VE_COMP_CTRL_DCT_LUM | VE_COMP_CTRL_DCT_CHR,
+			    VE_COMP_CTRL_DCT_LUM | VE_COMP_CTRL_DCT_CHR |
+			    VE_COMP_CTRL_EN_HQ | VE_COMP_CTRL_HQ_DCT_LUM |
+			    VE_COMP_CTRL_HQ_DCT_CHR | VE_COMP_CTRL_VQ_4COLOR |
+			    VE_COMP_CTRL_VQ_DCT_ONLY,
 			    comp_ctrl);
 }
 
@@ -1051,6 +1169,8 @@ static void aspeed_video_init_regs(struct aspeed_video *video)
 
 	/* Set mode detection defaults */
 	aspeed_video_write(video, VE_MODE_DETECT, 0x22666500);
+
+	aspeed_video_write(video, VE_BCD_CTRL, 0);
 }
 
 static void aspeed_video_start(struct aspeed_video *video)
@@ -1084,6 +1204,9 @@ static void aspeed_video_stop(struct aspeed_video *video)
 	if (video->srcs[1].size)
 		aspeed_video_free_buf(video, &video->srcs[1]);
 
+	if (video->bcd.size)
+		aspeed_video_free_buf(video, &video->bcd);
+
 	video->v4l2_input_status = V4L2_IN_ST_NO_SIGNAL;
 	video->flags = 0;
 }
@@ -1102,10 +1225,12 @@ static int aspeed_video_querycap(struct file *file, void *fh,
 static int aspeed_video_enum_format(struct file *file, void *fh,
 				    struct v4l2_fmtdesc *f)
 {
+	struct aspeed_video *video = video_drvdata(file);
+
 	if (f->index)
 		return -EINVAL;
 
-	f->pixelformat = V4L2_PIX_FMT_JPEG;
+	f->pixelformat = video->pix_fmt.pixelformat;
 
 	return 0;
 }
@@ -1120,6 +1245,29 @@ static int aspeed_video_get_format(struct file *file, void *fh,
 	return 0;
 }
 
+static int aspeed_video_set_format(struct file *file, void *fh,
+				   struct v4l2_format *f)
+{
+	struct aspeed_video *video = video_drvdata(file);
+
+	if (vb2_is_busy(&video->queue))
+		return -EBUSY;
+
+	switch (f->fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_JPEG:
+		video->format = VIDEO_FMT_STANDARD;
+		break;
+	case V4L2_PIX_FMT_AJPG:
+		video->format = VIDEO_FMT_ASPEED;
+		break;
+	default:
+		return -EINVAL;
+	}
+	video->pix_fmt.pixelformat = f->fmt.pix.pixelformat;
+
+	return 0;
+}
+
 static int aspeed_video_enum_input(struct file *file, void *fh,
 				   struct v4l2_input *inp)
 {
@@ -1337,7 +1485,7 @@ static const struct v4l2_ioctl_ops aspeed_video_ioctl_ops = {
 
 	.vidioc_enum_fmt_vid_cap = aspeed_video_enum_format,
 	.vidioc_g_fmt_vid_cap = aspeed_video_get_format,
-	.vidioc_s_fmt_vid_cap = aspeed_video_get_format,
+	.vidioc_s_fmt_vid_cap = aspeed_video_set_format,
 	.vidioc_try_fmt_vid_cap = aspeed_video_get_format,
 
 	.vidioc_reqbufs = vb2_ioctl_reqbufs,
@@ -1386,6 +1534,24 @@ static int aspeed_video_set_ctrl(struct v4l2_ctrl *ctrl)
 		if (test_bit(VIDEO_STREAMING, &video->flags))
 			aspeed_video_update_regs(video);
 		break;
+	case V4L2_CID_ASPEED_COMPRESSION_MODE:
+		if (video->config.version > 4)
+			return -EINVAL;
+
+		video->compression_mode = ctrl->val;
+		if (test_bit(VIDEO_STREAMING, &video->flags))
+			aspeed_video_update_regs(video);
+		break;
+	case V4L2_CID_ASPEED_HQ_MODE:
+		video->hq_mode = ctrl->val;
+		if (test_bit(VIDEO_STREAMING, &video->flags))
+			aspeed_video_update_regs(video);
+		break;
+	case V4L2_CID_ASPEED_HQ_JPEG_QUALITY:
+		video->jpeg_hq_quality = ctrl->val;
+		if (test_bit(VIDEO_STREAMING, &video->flags))
+			aspeed_video_update_regs(video);
+		break;
 	default:
 		return -EINVAL;
 	}
@@ -1397,6 +1563,39 @@ static const struct v4l2_ctrl_ops aspeed_video_ctrl_ops = {
 	.s_ctrl = aspeed_video_set_ctrl,
 };
 
+static const struct v4l2_ctrl_config aspeed_ctrl_compression_mode = {
+	.ops = &aspeed_video_ctrl_ops,
+	.id = V4L2_CID_ASPEED_COMPRESSION_MODE,
+	.name = "Aspeed Compression Mode",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = 0,
+	.max = 2,
+	.step = 1,
+	.def = 0,
+};
+
+static const struct v4l2_ctrl_config aspeed_ctrl_HQ_mode = {
+	.ops = &aspeed_video_ctrl_ops,
+	.id = V4L2_CID_ASPEED_HQ_MODE,
+	.name = "Aspeed HQ Mode",
+	.type = V4L2_CTRL_TYPE_BOOLEAN,
+	.min = false,
+	.max = true,
+	.step = 1,
+	.def = false,
+};
+
+static const struct v4l2_ctrl_config aspeed_ctrl_HQ_jpeg_quality = {
+	.ops = &aspeed_video_ctrl_ops,
+	.id = V4L2_CID_ASPEED_HQ_JPEG_QUALITY,
+	.name = "Aspeed HQ Quality",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = 1,
+	.max = ASPEED_VIDEO_JPEG_NUM_QUALITIES,
+	.step = 1,
+	.def = 1,
+};
+
 static void aspeed_video_resolution_work(struct work_struct *work)
 {
 	struct delayed_work *dwork = to_delayed_work(work);
@@ -1619,7 +1818,6 @@ static int aspeed_video_debugfs_show(struct seq_file *s, void *data)
 		   (v->perf.totaltime && v->sequence) ?
 		   1000/(v->perf.totaltime/v->sequence) : 0);
 
-
 	return 0;
 }
 
@@ -1663,6 +1861,7 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
 	struct v4l2_device *v4l2_dev = &video->v4l2_dev;
 	struct vb2_queue *vbq = &video->queue;
 	struct video_device *vdev = &video->vdev;
+	struct v4l2_ctrl_handler *hdl = &video->ctrl_handler;
 	int rc;
 
 	video->pix_fmt.pixelformat = V4L2_PIX_FMT_JPEG;
@@ -1677,22 +1876,26 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
 		return rc;
 	}
 
-	v4l2_ctrl_handler_init(&video->ctrl_handler, 2);
-	v4l2_ctrl_new_std(&video->ctrl_handler, &aspeed_video_ctrl_ops,
-			  V4L2_CID_JPEG_COMPRESSION_QUALITY, 0,
-			  ASPEED_VIDEO_JPEG_NUM_QUALITIES - 1, 1, 0);
-	v4l2_ctrl_new_std_menu(&video->ctrl_handler, &aspeed_video_ctrl_ops,
+	v4l2_ctrl_handler_init(hdl, 6);
+	v4l2_ctrl_new_std(hdl, &aspeed_video_ctrl_ops,
+			  V4L2_CID_JPEG_COMPRESSION_QUALITY, 1,
+			  ASPEED_VIDEO_JPEG_NUM_QUALITIES, 1, 1);
+	v4l2_ctrl_new_std_menu(hdl, &aspeed_video_ctrl_ops,
 			       V4L2_CID_JPEG_CHROMA_SUBSAMPLING,
 			       V4L2_JPEG_CHROMA_SUBSAMPLING_420, mask,
 			       V4L2_JPEG_CHROMA_SUBSAMPLING_444);
+	if (video->config.version == 4)
+		v4l2_ctrl_new_custom(hdl, &aspeed_ctrl_compression_mode, NULL);
+	v4l2_ctrl_new_custom(hdl, &aspeed_ctrl_HQ_mode, NULL);
+	v4l2_ctrl_new_custom(hdl, &aspeed_ctrl_HQ_jpeg_quality, NULL);
 
-	rc = video->ctrl_handler.error;
+	rc = hdl->error;
 	if (rc) {
 		dev_err(video->dev, "Failed to init controls: %d\n", rc);
 		goto err_ctrl_init;
 	}
 
-	v4l2_dev->ctrl_handler = &video->ctrl_handler;
+	v4l2_dev->ctrl_handler = hdl;
 
 	vbq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 	vbq->io_modes = VB2_MMAP | VB2_READ | VB2_DMABUF;
@@ -1833,6 +2036,7 @@ static int aspeed_video_probe(struct platform_device *pdev)
 		return -ENOMEM;
 
 	video->frame_rate = 30;
+	video->jpeg_hq_quality = 1;
 	video->dev = &pdev->dev;
 	spin_lock_init(&video->lock);
 	mutex_init(&video->video_lock);
diff --git a/include/uapi/linux/aspeed-video.h b/include/uapi/linux/aspeed-video.h
new file mode 100644
index 000000000000..117bfeaaa83a
--- /dev/null
+++ b/include/uapi/linux/aspeed-video.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2021 ASPEED Technology Inc.
+ */
+
+#ifndef _UAPI_LINUX_ASPEED_VIDEO_H
+#define _UAPI_LINUX_ASPEED_VIDEO_H
+
+#include <linux/v4l2-controls.h>
+
+#define V4L2_CID_ASPEED_COMPRESSION_MODE	(V4L2_CID_USER_ASPEED_BASE  + 1)
+#define V4L2_CID_ASPEED_HQ_MODE			(V4L2_CID_USER_ASPEED_BASE  + 2)
+#define V4L2_CID_ASPEED_HQ_JPEG_QUALITY		(V4L2_CID_USER_ASPEED_BASE  + 3)
+
+#endif /* _UAPI_LINUX_ASPEED_VIDEO_H */
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v5 09/10] media: aspeed: add comments and macro

[Jammy Huang]

Add comments to describe video-stat and 'struct aspeed_video'.
Add macro, ASPEED_VIDEO_V4L2_MIN_BUF_REQ, to describe the buffers
needed.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5:
  - no update
v4:
  - update comments
v3:
  - no update
v2:
  - no update
---
 drivers/media/platform/aspeed-video.c | 39 ++++++++++++++++++++++++---
 1 file changed, 36 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index 419825ed7737..03bb6c46f2a1 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -33,6 +33,8 @@
 #include <linux/videodev2.h>
 #include <uapi/linux/aspeed-video.h>
 
+#define ASPEED_VIDEO_V4L2_MIN_BUF_REQ 3
+
 #define DEVICE_NAME			"aspeed-video"
 
 #define ASPEED_VIDEO_JPEG_NUM_QUALITIES	12
@@ -193,6 +195,15 @@
 #define VE_MEM_RESTRICT_START		0x310
 #define VE_MEM_RESTRICT_END		0x314
 
+/*
+ * @VIDEO_MODE_DETECT_DONE:	a flag raised if signal lock
+ * @VIDEO_RES_CHANGE:		a flag raised if res_change work on-going
+ * @VIDEO_RES_DETECT:		a flag raised if res. detection on-going
+ * @VIDEO_STREAMING:		a flag raised if user requires stream-on
+ * @VIDEO_FRAME_INPRG:		a flag raised if hw working on a frame
+ * @VIDEO_STOPPED:		a flag raised if device release
+ * @VIDEO_CLOCKS_ON:		a flag raised if clk is on
+ */
 enum {
 	VIDEO_MODE_DETECT_DONE,
 	VIDEO_RES_CHANGE,
@@ -231,6 +242,28 @@ struct aspeed_video_perf {
 #define to_aspeed_video_buffer(x) \
 	container_of((x), struct aspeed_video_buffer, vb)
 
+/**
+ * struct aspeed_video - driver data
+ *
+ * @flags:		holds the state of video
+ * @sequence:		holds the last number of frame completed
+ * @max_compressed_size:holds max compressed stream's size
+ * @srcs:		holds the buffer information for srcs
+ * @jpeg:		holds the buffer information for jpeg header
+ * @bcd:		holds the buffer information for bcd work
+ * @yuv420:		a flag raised if JPEG subsampling is 420
+ * @format:		holds the video format
+ * @hq_mode:		a flag raised if HQ is enabled. Only for VIDEO_FMT_ASPEED
+ * @frame_rate:		holds the frame_rate
+ * @jpeg_quality:	holds jpeq's quality (0~11)
+ * @jpeg_hq_quality:	holds hq's quality (0~11) only if hq_mode enabled
+ * @compression_mode:	holds jpeg compression mode
+ * @frame_bottom:	end position of video data in vertical direction
+ * @frame_left:		start position of video data in horizontal direction
+ * @frame_right:	end position of video data in horizontal direction
+ * @frame_top:		start position of video data in vertical direction
+ * @perf:		holds the statistics primary for debugfs
+ */
 struct aspeed_video {
 	void __iomem *base;
 	struct clk *eclk;
@@ -1305,7 +1338,7 @@ static int aspeed_video_get_parm(struct file *file, void *fh,
 	struct aspeed_video *video = video_drvdata(file);
 
 	a->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
-	a->parm.capture.readbuffers = 3;
+	a->parm.capture.readbuffers = ASPEED_VIDEO_V4L2_MIN_BUF_REQ;
 	a->parm.capture.timeperframe.numerator = 1;
 	if (!video->frame_rate)
 		a->parm.capture.timeperframe.denominator = MAX_FRAME_RATE;
@@ -1322,7 +1355,7 @@ static int aspeed_video_set_parm(struct file *file, void *fh,
 	struct aspeed_video *video = video_drvdata(file);
 
 	a->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
-	a->parm.capture.readbuffers = 3;
+	a->parm.capture.readbuffers = ASPEED_VIDEO_V4L2_MIN_BUF_REQ;
 
 	if (a->parm.capture.timeperframe.numerator)
 		frame_rate = a->parm.capture.timeperframe.denominator /
@@ -1906,7 +1939,7 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
 	vbq->drv_priv = video;
 	vbq->buf_struct_size = sizeof(struct aspeed_video_buffer);
 	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
-	vbq->min_buffers_needed = 3;
+	vbq->min_buffers_needed = ASPEED_VIDEO_V4L2_MIN_BUF_REQ;
 
 	rc = vb2_queue_init(vbq);
 	if (rc) {
-- 
2.25.1

[PATCH v5 09/10] media: aspeed: add comments and macro

[Jammy Huang]

Add comments to describe video-stat and 'struct aspeed_video'.
Add macro, ASPEED_VIDEO_V4L2_MIN_BUF_REQ, to describe the buffers
needed.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v5:
  - no update
v4:
  - update comments
v3:
  - no update
v2:
  - no update
---
 drivers/media/platform/aspeed-video.c | 39 ++++++++++++++++++++++++---
 1 file changed, 36 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index 419825ed7737..03bb6c46f2a1 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -33,6 +33,8 @@
 #include <linux/videodev2.h>
 #include <uapi/linux/aspeed-video.h>
 
+#define ASPEED_VIDEO_V4L2_MIN_BUF_REQ 3
+
 #define DEVICE_NAME			"aspeed-video"
 
 #define ASPEED_VIDEO_JPEG_NUM_QUALITIES	12
@@ -193,6 +195,15 @@
 #define VE_MEM_RESTRICT_START		0x310
 #define VE_MEM_RESTRICT_END		0x314
 
+/*
+ * @VIDEO_MODE_DETECT_DONE:	a flag raised if signal lock
+ * @VIDEO_RES_CHANGE:		a flag raised if res_change work on-going
+ * @VIDEO_RES_DETECT:		a flag raised if res. detection on-going
+ * @VIDEO_STREAMING:		a flag raised if user requires stream-on
+ * @VIDEO_FRAME_INPRG:		a flag raised if hw working on a frame
+ * @VIDEO_STOPPED:		a flag raised if device release
+ * @VIDEO_CLOCKS_ON:		a flag raised if clk is on
+ */
 enum {
 	VIDEO_MODE_DETECT_DONE,
 	VIDEO_RES_CHANGE,
@@ -231,6 +242,28 @@ struct aspeed_video_perf {
 #define to_aspeed_video_buffer(x) \
 	container_of((x), struct aspeed_video_buffer, vb)
 
+/**
+ * struct aspeed_video - driver data
+ *
+ * @flags:		holds the state of video
+ * @sequence:		holds the last number of frame completed
+ * @max_compressed_size:holds max compressed stream's size
+ * @srcs:		holds the buffer information for srcs
+ * @jpeg:		holds the buffer information for jpeg header
+ * @bcd:		holds the buffer information for bcd work
+ * @yuv420:		a flag raised if JPEG subsampling is 420
+ * @format:		holds the video format
+ * @hq_mode:		a flag raised if HQ is enabled. Only for VIDEO_FMT_ASPEED
+ * @frame_rate:		holds the frame_rate
+ * @jpeg_quality:	holds jpeq's quality (0~11)
+ * @jpeg_hq_quality:	holds hq's quality (0~11) only if hq_mode enabled
+ * @compression_mode:	holds jpeg compression mode
+ * @frame_bottom:	end position of video data in vertical direction
+ * @frame_left:		start position of video data in horizontal direction
+ * @frame_right:	end position of video data in horizontal direction
+ * @frame_top:		start position of video data in vertical direction
+ * @perf:		holds the statistics primary for debugfs
+ */
 struct aspeed_video {
 	void __iomem *base;
 	struct clk *eclk;
@@ -1305,7 +1338,7 @@ static int aspeed_video_get_parm(struct file *file, void *fh,
 	struct aspeed_video *video = video_drvdata(file);
 
 	a->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
-	a->parm.capture.readbuffers = 3;
+	a->parm.capture.readbuffers = ASPEED_VIDEO_V4L2_MIN_BUF_REQ;
 	a->parm.capture.timeperframe.numerator = 1;
 	if (!video->frame_rate)
 		a->parm.capture.timeperframe.denominator = MAX_FRAME_RATE;
@@ -1322,7 +1355,7 @@ static int aspeed_video_set_parm(struct file *file, void *fh,
 	struct aspeed_video *video = video_drvdata(file);
 
 	a->parm.capture.capability = V4L2_CAP_TIMEPERFRAME;
-	a->parm.capture.readbuffers = 3;
+	a->parm.capture.readbuffers = ASPEED_VIDEO_V4L2_MIN_BUF_REQ;
 
 	if (a->parm.capture.timeperframe.numerator)
 		frame_rate = a->parm.capture.timeperframe.denominator /
@@ -1906,7 +1939,7 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
 	vbq->drv_priv = video;
 	vbq->buf_struct_size = sizeof(struct aspeed_video_buffer);
 	vbq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
-	vbq->min_buffers_needed = 3;
+	vbq->min_buffers_needed = ASPEED_VIDEO_V4L2_MIN_BUF_REQ;
 
 	rc = vb2_queue_init(vbq);
 	if (rc) {
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 02/10] media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log

[Sakari Ailus]

Hi Jammy,

On Thu, Nov 18, 2021 at 03:40:23PM +0800, Jammy Huang wrote:
> The debug log level, 0~3, is controlled by module_param, debug.
> The higher the value, the more the information.
>   0: off
>   1: info
>   2: debug
>   3: register operations
> 
> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>

Just wondering: what's the purpose of switching to v4l2_*() functions for
printing when dev_*() equivalents already can do the same?

-- 
Sakari Ailus

Re: [PATCH v5 02/10] media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log

[Sakari Ailus]

Hi Jammy,

On Thu, Nov 18, 2021 at 03:40:23PM +0800, Jammy Huang wrote:
> The debug log level, 0~3, is controlled by module_param, debug.
> The higher the value, the more the information.
>   0: off
>   1: info
>   2: debug
>   3: register operations
> 
> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>

Just wondering: what's the purpose of switching to v4l2_*() functions for
printing when dev_*() equivalents already can do the same?

-- 
Sakari Ailus

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 02/10] media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log

[Sakari Ailus]

Hi Jammy,

On Thu, Nov 18, 2021 at 03:40:23PM +0800, Jammy Huang wrote:
> The debug log level, 0~3, is controlled by module_param, debug.
> The higher the value, the more the information.
>   0: off
>   1: info
>   2: debug
>   3: register operations
> 
> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>

Just wondering: what's the purpose of switching to v4l2_*() functions for
printing when dev_*() equivalents already can do the same?

-- 
Sakari Ailus

Re: [PATCH v5 03/10] media: aspeed: add more debug log messages

[Sakari Ailus]

Hi Jammy,

On Thu, Nov 18, 2021 at 03:40:24PM +0800, Jammy Huang wrote:
> The new messages are listed as below:
> 1. jpeg header and capture buffer information
> 2. information for each irq
> 3. current capture mode, sync or direct-fetch
> 4. time consumed for each frame
> 5. input timing changed information
> 
> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
> ---
> v5:
>   - no update
> v4:
>   - modify log level
> v3:
>   - update commit message
> v2:
>   - new
> ---
>  drivers/media/platform/aspeed-video.c | 24 ++++++++++++++++++++++++
>  1 file changed, 24 insertions(+)
> 
> diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
> index 6af57467b6d4..e8dd0a7ebfc7 100644
> --- a/drivers/media/platform/aspeed-video.c
> +++ b/drivers/media/platform/aspeed-video.c
> @@ -461,12 +461,17 @@ static void aspeed_video_write(struct aspeed_video *video, u32 reg, u32 val)
>  
>  static void update_perf(struct aspeed_video_perf *p)
>  {
> +	struct aspeed_video *v = container_of(p, struct aspeed_video,
> +					      perf);
> +
>  	p->duration =
>  		ktime_to_ms(ktime_sub(ktime_get(),  p->last_sample));
>  	p->totaltime += p->duration;
>  
>  	p->duration_max = max(p->duration, p->duration_max);
>  	p->duration_min = min(p->duration, p->duration_min);
> +	v4l2_dbg(2, debug, &v->v4l2_dev, "time consumed: %d ms\n",
> +		 p->duration);
>  }
>  
>  static int aspeed_video_start_frame(struct aspeed_video *video)
> @@ -597,6 +602,12 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
>  	struct aspeed_video *video = arg;
>  	u32 sts = aspeed_video_read(video, VE_INTERRUPT_STATUS);
>  
> +	v4l2_dbg(2, debug, &video->v4l2_dev, "irq sts=%#x %s%s%s%s\n", sts,
> +		 sts & VE_INTERRUPT_MODE_DETECT_WD ? ", unlock" : "",
> +		 sts & VE_INTERRUPT_MODE_DETECT ? ", lock" : "",
> +		 sts & VE_INTERRUPT_CAPTURE_COMPLETE ? ", capture-done" : "",
> +		 sts & VE_INTERRUPT_COMP_COMPLETE ? ", comp-done" : "");
> +
>  	/*
>  	 * Resolution changed or signal was lost; reset the engine and
>  	 * re-initialize
> @@ -910,6 +921,7 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>  
>  	/* Don't use direct mode below 1024 x 768 (irqs don't fire) */
>  	if (size < DIRECT_FETCH_THRESHOLD) {
> +		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Sync Mode\n");
>  		aspeed_video_write(video, VE_TGS_0,
>  				   FIELD_PREP(VE_TGS_FIRST,
>  					      video->frame_left - 1) |
> @@ -921,6 +933,7 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>  					      video->frame_bottom + 1));
>  		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_INT_DE);
>  	} else {
> +		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Direct Mode\n");
>  		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_DIRECT_FETCH);
>  	}
>  
> @@ -937,6 +950,10 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>  		if (!aspeed_video_alloc_buf(video, &video->srcs[1], size))
>  			goto err_mem;
>  
> +		v4l2_dbg(1, debug, &video->v4l2_dev, "src buf0 addr(%#x) size(%d)\n",
> +			 video->srcs[0].dma, video->srcs[0].size);
> +		v4l2_dbg(1, debug, &video->v4l2_dev, "src buf1 addr(%#x) size(%d)\n",
> +			 video->srcs[1].dma, video->srcs[1].size);
>  		aspeed_video_write(video, VE_SRC0_ADDR, video->srcs[0].dma);
>  		aspeed_video_write(video, VE_SRC1_ADDR, video->srcs[1].dma);
>  	}
> @@ -1201,6 +1218,9 @@ static int aspeed_video_set_dv_timings(struct file *file, void *fh,
>  
>  	timings->type = V4L2_DV_BT_656_1120;
>  
> +	v4l2_dbg(1, debug, &video->v4l2_dev, "set new timings(%dx%d)\n",
> +		 timings->bt.width, timings->bt.height);
> +
>  	return 0;
>  }
>  
> @@ -1383,6 +1403,7 @@ static void aspeed_video_resolution_work(struct work_struct *work)
>  			.u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION,
>  		};
>  
> +		v4l2_dbg(1, debug, &video->v4l2_dev, "fire source change event\n");
>  		v4l2_event_queue(&video->vdev, &ev);
>  	} else if (test_bit(VIDEO_STREAMING, &video->flags)) {
>  		/* No resolution change so just restart streaming */
> @@ -1715,6 +1736,7 @@ static int aspeed_video_init(struct aspeed_video *video)
>  		dev_err(dev, "Unable to request IRQ %d\n", irq);
>  		return rc;
>  	}
> +	dev_info(video->dev, "irq %d\n", irq);
>  
>  	video->eclk = devm_clk_get(dev, "eclk");
>  	if (IS_ERR(video->eclk)) {
> @@ -1751,6 +1773,8 @@ static int aspeed_video_init(struct aspeed_video *video)
>  		rc = -ENOMEM;
>  		goto err_release_reserved_mem;
>  	}
> +	dev_info(video->dev, "alloc mem size(%d) at %#x for jpeg header\n",
> +		 VE_JPEG_HEADER_SIZE, video->jpeg.dma);
>  
>  	aspeed_video_init_jpeg_table(video->jpeg.virt, video->yuv420);
>  

You're using both v4l2_*() and dev_*() functions for printing messages.
They come with different prefixes, and it'd be better to stick with either,
not both.

-- 
Sakari Ailus

Re: [PATCH v5 03/10] media: aspeed: add more debug log messages

[Sakari Ailus]

Hi Jammy,

On Thu, Nov 18, 2021 at 03:40:24PM +0800, Jammy Huang wrote:
> The new messages are listed as below:
> 1. jpeg header and capture buffer information
> 2. information for each irq
> 3. current capture mode, sync or direct-fetch
> 4. time consumed for each frame
> 5. input timing changed information
> 
> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
> ---
> v5:
>   - no update
> v4:
>   - modify log level
> v3:
>   - update commit message
> v2:
>   - new
> ---
>  drivers/media/platform/aspeed-video.c | 24 ++++++++++++++++++++++++
>  1 file changed, 24 insertions(+)
> 
> diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
> index 6af57467b6d4..e8dd0a7ebfc7 100644
> --- a/drivers/media/platform/aspeed-video.c
> +++ b/drivers/media/platform/aspeed-video.c
> @@ -461,12 +461,17 @@ static void aspeed_video_write(struct aspeed_video *video, u32 reg, u32 val)
>  
>  static void update_perf(struct aspeed_video_perf *p)
>  {
> +	struct aspeed_video *v = container_of(p, struct aspeed_video,
> +					      perf);
> +
>  	p->duration =
>  		ktime_to_ms(ktime_sub(ktime_get(),  p->last_sample));
>  	p->totaltime += p->duration;
>  
>  	p->duration_max = max(p->duration, p->duration_max);
>  	p->duration_min = min(p->duration, p->duration_min);
> +	v4l2_dbg(2, debug, &v->v4l2_dev, "time consumed: %d ms\n",
> +		 p->duration);
>  }
>  
>  static int aspeed_video_start_frame(struct aspeed_video *video)
> @@ -597,6 +602,12 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
>  	struct aspeed_video *video = arg;
>  	u32 sts = aspeed_video_read(video, VE_INTERRUPT_STATUS);
>  
> +	v4l2_dbg(2, debug, &video->v4l2_dev, "irq sts=%#x %s%s%s%s\n", sts,
> +		 sts & VE_INTERRUPT_MODE_DETECT_WD ? ", unlock" : "",
> +		 sts & VE_INTERRUPT_MODE_DETECT ? ", lock" : "",
> +		 sts & VE_INTERRUPT_CAPTURE_COMPLETE ? ", capture-done" : "",
> +		 sts & VE_INTERRUPT_COMP_COMPLETE ? ", comp-done" : "");
> +
>  	/*
>  	 * Resolution changed or signal was lost; reset the engine and
>  	 * re-initialize
> @@ -910,6 +921,7 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>  
>  	/* Don't use direct mode below 1024 x 768 (irqs don't fire) */
>  	if (size < DIRECT_FETCH_THRESHOLD) {
> +		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Sync Mode\n");
>  		aspeed_video_write(video, VE_TGS_0,
>  				   FIELD_PREP(VE_TGS_FIRST,
>  					      video->frame_left - 1) |
> @@ -921,6 +933,7 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>  					      video->frame_bottom + 1));
>  		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_INT_DE);
>  	} else {
> +		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Direct Mode\n");
>  		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_DIRECT_FETCH);
>  	}
>  
> @@ -937,6 +950,10 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>  		if (!aspeed_video_alloc_buf(video, &video->srcs[1], size))
>  			goto err_mem;
>  
> +		v4l2_dbg(1, debug, &video->v4l2_dev, "src buf0 addr(%#x) size(%d)\n",
> +			 video->srcs[0].dma, video->srcs[0].size);
> +		v4l2_dbg(1, debug, &video->v4l2_dev, "src buf1 addr(%#x) size(%d)\n",
> +			 video->srcs[1].dma, video->srcs[1].size);
>  		aspeed_video_write(video, VE_SRC0_ADDR, video->srcs[0].dma);
>  		aspeed_video_write(video, VE_SRC1_ADDR, video->srcs[1].dma);
>  	}
> @@ -1201,6 +1218,9 @@ static int aspeed_video_set_dv_timings(struct file *file, void *fh,
>  
>  	timings->type = V4L2_DV_BT_656_1120;
>  
> +	v4l2_dbg(1, debug, &video->v4l2_dev, "set new timings(%dx%d)\n",
> +		 timings->bt.width, timings->bt.height);
> +
>  	return 0;
>  }
>  
> @@ -1383,6 +1403,7 @@ static void aspeed_video_resolution_work(struct work_struct *work)
>  			.u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION,
>  		};
>  
> +		v4l2_dbg(1, debug, &video->v4l2_dev, "fire source change event\n");
>  		v4l2_event_queue(&video->vdev, &ev);
>  	} else if (test_bit(VIDEO_STREAMING, &video->flags)) {
>  		/* No resolution change so just restart streaming */
> @@ -1715,6 +1736,7 @@ static int aspeed_video_init(struct aspeed_video *video)
>  		dev_err(dev, "Unable to request IRQ %d\n", irq);
>  		return rc;
>  	}
> +	dev_info(video->dev, "irq %d\n", irq);
>  
>  	video->eclk = devm_clk_get(dev, "eclk");
>  	if (IS_ERR(video->eclk)) {
> @@ -1751,6 +1773,8 @@ static int aspeed_video_init(struct aspeed_video *video)
>  		rc = -ENOMEM;
>  		goto err_release_reserved_mem;
>  	}
> +	dev_info(video->dev, "alloc mem size(%d) at %#x for jpeg header\n",
> +		 VE_JPEG_HEADER_SIZE, video->jpeg.dma);
>  
>  	aspeed_video_init_jpeg_table(video->jpeg.virt, video->yuv420);
>  

You're using both v4l2_*() and dev_*() functions for printing messages.
They come with different prefixes, and it'd be better to stick with either,
not both.

-- 
Sakari Ailus

Re: [PATCH v5 03/10] media: aspeed: add more debug log messages

[Sakari Ailus]

Hi Jammy,

On Thu, Nov 18, 2021 at 03:40:24PM +0800, Jammy Huang wrote:
> The new messages are listed as below:
> 1. jpeg header and capture buffer information
> 2. information for each irq
> 3. current capture mode, sync or direct-fetch
> 4. time consumed for each frame
> 5. input timing changed information
> 
> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
> ---
> v5:
>   - no update
> v4:
>   - modify log level
> v3:
>   - update commit message
> v2:
>   - new
> ---
>  drivers/media/platform/aspeed-video.c | 24 ++++++++++++++++++++++++
>  1 file changed, 24 insertions(+)
> 
> diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
> index 6af57467b6d4..e8dd0a7ebfc7 100644
> --- a/drivers/media/platform/aspeed-video.c
> +++ b/drivers/media/platform/aspeed-video.c
> @@ -461,12 +461,17 @@ static void aspeed_video_write(struct aspeed_video *video, u32 reg, u32 val)
>  
>  static void update_perf(struct aspeed_video_perf *p)
>  {
> +	struct aspeed_video *v = container_of(p, struct aspeed_video,
> +					      perf);
> +
>  	p->duration =
>  		ktime_to_ms(ktime_sub(ktime_get(),  p->last_sample));
>  	p->totaltime += p->duration;
>  
>  	p->duration_max = max(p->duration, p->duration_max);
>  	p->duration_min = min(p->duration, p->duration_min);
> +	v4l2_dbg(2, debug, &v->v4l2_dev, "time consumed: %d ms\n",
> +		 p->duration);
>  }
>  
>  static int aspeed_video_start_frame(struct aspeed_video *video)
> @@ -597,6 +602,12 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
>  	struct aspeed_video *video = arg;
>  	u32 sts = aspeed_video_read(video, VE_INTERRUPT_STATUS);
>  
> +	v4l2_dbg(2, debug, &video->v4l2_dev, "irq sts=%#x %s%s%s%s\n", sts,
> +		 sts & VE_INTERRUPT_MODE_DETECT_WD ? ", unlock" : "",
> +		 sts & VE_INTERRUPT_MODE_DETECT ? ", lock" : "",
> +		 sts & VE_INTERRUPT_CAPTURE_COMPLETE ? ", capture-done" : "",
> +		 sts & VE_INTERRUPT_COMP_COMPLETE ? ", comp-done" : "");
> +
>  	/*
>  	 * Resolution changed or signal was lost; reset the engine and
>  	 * re-initialize
> @@ -910,6 +921,7 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>  
>  	/* Don't use direct mode below 1024 x 768 (irqs don't fire) */
>  	if (size < DIRECT_FETCH_THRESHOLD) {
> +		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Sync Mode\n");
>  		aspeed_video_write(video, VE_TGS_0,
>  				   FIELD_PREP(VE_TGS_FIRST,
>  					      video->frame_left - 1) |
> @@ -921,6 +933,7 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>  					      video->frame_bottom + 1));
>  		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_INT_DE);
>  	} else {
> +		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Direct Mode\n");
>  		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_DIRECT_FETCH);
>  	}
>  
> @@ -937,6 +950,10 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>  		if (!aspeed_video_alloc_buf(video, &video->srcs[1], size))
>  			goto err_mem;
>  
> +		v4l2_dbg(1, debug, &video->v4l2_dev, "src buf0 addr(%#x) size(%d)\n",
> +			 video->srcs[0].dma, video->srcs[0].size);
> +		v4l2_dbg(1, debug, &video->v4l2_dev, "src buf1 addr(%#x) size(%d)\n",
> +			 video->srcs[1].dma, video->srcs[1].size);
>  		aspeed_video_write(video, VE_SRC0_ADDR, video->srcs[0].dma);
>  		aspeed_video_write(video, VE_SRC1_ADDR, video->srcs[1].dma);
>  	}
> @@ -1201,6 +1218,9 @@ static int aspeed_video_set_dv_timings(struct file *file, void *fh,
>  
>  	timings->type = V4L2_DV_BT_656_1120;
>  
> +	v4l2_dbg(1, debug, &video->v4l2_dev, "set new timings(%dx%d)\n",
> +		 timings->bt.width, timings->bt.height);
> +
>  	return 0;
>  }
>  
> @@ -1383,6 +1403,7 @@ static void aspeed_video_resolution_work(struct work_struct *work)
>  			.u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION,
>  		};
>  
> +		v4l2_dbg(1, debug, &video->v4l2_dev, "fire source change event\n");
>  		v4l2_event_queue(&video->vdev, &ev);
>  	} else if (test_bit(VIDEO_STREAMING, &video->flags)) {
>  		/* No resolution change so just restart streaming */
> @@ -1715,6 +1736,7 @@ static int aspeed_video_init(struct aspeed_video *video)
>  		dev_err(dev, "Unable to request IRQ %d\n", irq);
>  		return rc;
>  	}
> +	dev_info(video->dev, "irq %d\n", irq);
>  
>  	video->eclk = devm_clk_get(dev, "eclk");
>  	if (IS_ERR(video->eclk)) {
> @@ -1751,6 +1773,8 @@ static int aspeed_video_init(struct aspeed_video *video)
>  		rc = -ENOMEM;
>  		goto err_release_reserved_mem;
>  	}
> +	dev_info(video->dev, "alloc mem size(%d) at %#x for jpeg header\n",
> +		 VE_JPEG_HEADER_SIZE, video->jpeg.dma);
>  
>  	aspeed_video_init_jpeg_table(video->jpeg.virt, video->yuv420);
>  

You're using both v4l2_*() and dev_*() functions for printing messages.
They come with different prefixes, and it'd be better to stick with either,
not both.

-- 
Sakari Ailus

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Sakari Ailus]

Hi Jammy,

On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
> This introduces support for the Aspeed JPEG format, where the new frame
> can refer to previous frame to reduce the amount of compressed data. The
> concept is similar to I/P frame of video compression. I will compare the
> new frame with previous one to decide which macroblock's data is
> changed, and only the changed macroblocks will be compressed.
> 
> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
> which is generally adapted for remote KVM.
> 
> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
> ---
> v5:
>   - no update
> v4:
>   - new
> ---
>  Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>  include/uapi/linux/videodev2.h                   |  1 +
>  3 files changed, 14 insertions(+)
> 
> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> index b2cd155e691b..23c05063133d 100644
> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> @@ -264,6 +264,18 @@ please make a proposal on the linux-media mailing list.
>  	of tiles, resulting in 32-aligned resolutions for the luminance plane
>  	and 16-aligned resolutions for the chrominance plane (with 2x2
>  	subsampling).
> +    * .. _V4L2-PIX-FMT-AJPG:
> +
> +      - ``V4L2_PIX_FMT_AJPG``
> +      - 'AJPG'
> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
> +        which is generally adapted for remote KVM.
> +        On each frame compression, I will compare the new frame with previous
> +        one to decide which macroblock's data is changed, and only the changed
> +        macroblocks will be compressed.
> +
> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
> +        for more information.

Is this datasheet publicly available? Do you have a URL?

-- 
Regards,

Sakari Ailus

Re: [PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Sakari Ailus]

Hi Jammy,

On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
> This introduces support for the Aspeed JPEG format, where the new frame
> can refer to previous frame to reduce the amount of compressed data. The
> concept is similar to I/P frame of video compression. I will compare the
> new frame with previous one to decide which macroblock's data is
> changed, and only the changed macroblocks will be compressed.
> 
> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
> which is generally adapted for remote KVM.
> 
> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
> ---
> v5:
>   - no update
> v4:
>   - new
> ---
>  Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>  include/uapi/linux/videodev2.h                   |  1 +
>  3 files changed, 14 insertions(+)
> 
> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> index b2cd155e691b..23c05063133d 100644
> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> @@ -264,6 +264,18 @@ please make a proposal on the linux-media mailing list.
>  	of tiles, resulting in 32-aligned resolutions for the luminance plane
>  	and 16-aligned resolutions for the chrominance plane (with 2x2
>  	subsampling).
> +    * .. _V4L2-PIX-FMT-AJPG:
> +
> +      - ``V4L2_PIX_FMT_AJPG``
> +      - 'AJPG'
> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
> +        which is generally adapted for remote KVM.
> +        On each frame compression, I will compare the new frame with previous
> +        one to decide which macroblock's data is changed, and only the changed
> +        macroblocks will be compressed.
> +
> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
> +        for more information.

Is this datasheet publicly available? Do you have a URL?

-- 
Regards,

Sakari Ailus

Re: [PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Sakari Ailus]

Hi Jammy,

On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
> This introduces support for the Aspeed JPEG format, where the new frame
> can refer to previous frame to reduce the amount of compressed data. The
> concept is similar to I/P frame of video compression. I will compare the
> new frame with previous one to decide which macroblock's data is
> changed, and only the changed macroblocks will be compressed.
> 
> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
> which is generally adapted for remote KVM.
> 
> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
> ---
> v5:
>   - no update
> v4:
>   - new
> ---
>  Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>  include/uapi/linux/videodev2.h                   |  1 +
>  3 files changed, 14 insertions(+)
> 
> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> index b2cd155e691b..23c05063133d 100644
> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> @@ -264,6 +264,18 @@ please make a proposal on the linux-media mailing list.
>  	of tiles, resulting in 32-aligned resolutions for the luminance plane
>  	and 16-aligned resolutions for the chrominance plane (with 2x2
>  	subsampling).
> +    * .. _V4L2-PIX-FMT-AJPG:
> +
> +      - ``V4L2_PIX_FMT_AJPG``
> +      - 'AJPG'
> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
> +        which is generally adapted for remote KVM.
> +        On each frame compression, I will compare the new frame with previous
> +        one to decide which macroblock's data is changed, and only the changed
> +        macroblocks will be compressed.
> +
> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
> +        for more information.

Is this datasheet publicly available? Do you have a URL?

-- 
Regards,

Sakari Ailus

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 02/10] media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log

[Jammy Huang]

[-- Attachment #1: Type: text/plain, Size: 1495 bytes --]

Hi Sakari,

On 2021/11/18 下午 07:56, Sakari Ailus wrote:
> Hi Jammy,
>
> On Thu, Nov 18, 2021 at 03:40:23PM +0800, Jammy Huang wrote:
>> The debug log level, 0~3, is controlled by module_param, debug.
>> The higher the value, the more the information.
>>    0: off
>>    1: info
>>    2: debug
>>    3: register operations
>>
>> Signed-off-by: Jammy Huang<jammy_huang@aspeedtech.com>
> Just wondering: what's the purpose of switching to v4l2_*() functions for
> printing when dev_*() equivalents already can do the same?
>
Indeed, most of them, such as dev_err/v4l2_err or dev_info/v4l2_info, are the same.
The reason why I want to adapt v4l2_*() is v4l2_dbg with which I can have module's
own debug level.
/* These three macros assume that the debug level is set with a module
parameter called 'debug'. */
#define v4l2_dbg 
<https://elixir.bootlin.com/linux/latest/C/ident/v4l2_dbg>(level 
<https://elixir.bootlin.com/linux/latest/C/ident/level>, debug 
<https://elixir.bootlin.com/linux/latest/C/ident/debug>, dev, fmt 
<https://elixir.bootlin.com/linux/latest/C/ident/fmt>, arg...) \
do { \
if (debug <https://elixir.bootlin.com/linux/latest/C/ident/debug> >= 
(level <https://elixir.bootlin.com/linux/latest/C/ident/level>)) \
v4l2_printk 
<https://elixir.bootlin.com/linux/latest/C/ident/v4l2_printk>(KERN_DEBUG 
<https://elixir.bootlin.com/linux/latest/C/ident/KERN_DEBUG>, dev, fmt 
<https://elixir.bootlin.com/linux/latest/C/ident/fmt> , ## arg); \
} while (0)
-- 
Best Regards
Jammy

[-- Attachment #2: Type: text/html, Size: 4904 bytes --]

Re: [PATCH v5 03/10] media: aspeed: add more debug log messages

[Jammy Huang]

Hi Sakari,

On 2021/11/18 下午 07:57, Sakari Ailus wrote:
> Hi Jammy,
>
> On Thu, Nov 18, 2021 at 03:40:24PM +0800, Jammy Huang wrote:
>> The new messages are listed as below:
>> 1. jpeg header and capture buffer information
>> 2. information for each irq
>> 3. current capture mode, sync or direct-fetch
>> 4. time consumed for each frame
>> 5. input timing changed information
>>
>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>> ---
>> v5:
>>    - no update
>> v4:
>>    - modify log level
>> v3:
>>    - update commit message
>> v2:
>>    - new
>> ---
>>   drivers/media/platform/aspeed-video.c | 24 ++++++++++++++++++++++++
>>   1 file changed, 24 insertions(+)
>>
>> diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
>> index 6af57467b6d4..e8dd0a7ebfc7 100644
>> --- a/drivers/media/platform/aspeed-video.c
>> +++ b/drivers/media/platform/aspeed-video.c
>> @@ -461,12 +461,17 @@ static void aspeed_video_write(struct aspeed_video *video, u32 reg, u32 val)
>>   
>>   static void update_perf(struct aspeed_video_perf *p)
>>   {
>> +	struct aspeed_video *v = container_of(p, struct aspeed_video,
>> +					      perf);
>> +
>>   	p->duration =
>>   		ktime_to_ms(ktime_sub(ktime_get(),  p->last_sample));
>>   	p->totaltime += p->duration;
>>   
>>   	p->duration_max = max(p->duration, p->duration_max);
>>   	p->duration_min = min(p->duration, p->duration_min);
>> +	v4l2_dbg(2, debug, &v->v4l2_dev, "time consumed: %d ms\n",
>> +		 p->duration);
>>   }
>>   
>>   static int aspeed_video_start_frame(struct aspeed_video *video)
>> @@ -597,6 +602,12 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
>>   	struct aspeed_video *video = arg;
>>   	u32 sts = aspeed_video_read(video, VE_INTERRUPT_STATUS);
>>   
>> +	v4l2_dbg(2, debug, &video->v4l2_dev, "irq sts=%#x %s%s%s%s\n", sts,
>> +		 sts & VE_INTERRUPT_MODE_DETECT_WD ? ", unlock" : "",
>> +		 sts & VE_INTERRUPT_MODE_DETECT ? ", lock" : "",
>> +		 sts & VE_INTERRUPT_CAPTURE_COMPLETE ? ", capture-done" : "",
>> +		 sts & VE_INTERRUPT_COMP_COMPLETE ? ", comp-done" : "");
>> +
>>   	/*
>>   	 * Resolution changed or signal was lost; reset the engine and
>>   	 * re-initialize
>> @@ -910,6 +921,7 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>>   
>>   	/* Don't use direct mode below 1024 x 768 (irqs don't fire) */
>>   	if (size < DIRECT_FETCH_THRESHOLD) {
>> +		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Sync Mode\n");
>>   		aspeed_video_write(video, VE_TGS_0,
>>   				   FIELD_PREP(VE_TGS_FIRST,
>>   					      video->frame_left - 1) |
>> @@ -921,6 +933,7 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>>   					      video->frame_bottom + 1));
>>   		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_INT_DE);
>>   	} else {
>> +		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Direct Mode\n");
>>   		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_DIRECT_FETCH);
>>   	}
>>   
>> @@ -937,6 +950,10 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>>   		if (!aspeed_video_alloc_buf(video, &video->srcs[1], size))
>>   			goto err_mem;
>>   
>> +		v4l2_dbg(1, debug, &video->v4l2_dev, "src buf0 addr(%#x) size(%d)\n",
>> +			 video->srcs[0].dma, video->srcs[0].size);
>> +		v4l2_dbg(1, debug, &video->v4l2_dev, "src buf1 addr(%#x) size(%d)\n",
>> +			 video->srcs[1].dma, video->srcs[1].size);
>>   		aspeed_video_write(video, VE_SRC0_ADDR, video->srcs[0].dma);
>>   		aspeed_video_write(video, VE_SRC1_ADDR, video->srcs[1].dma);
>>   	}
>> @@ -1201,6 +1218,9 @@ static int aspeed_video_set_dv_timings(struct file *file, void *fh,
>>   
>>   	timings->type = V4L2_DV_BT_656_1120;
>>   
>> +	v4l2_dbg(1, debug, &video->v4l2_dev, "set new timings(%dx%d)\n",
>> +		 timings->bt.width, timings->bt.height);
>> +
>>   	return 0;
>>   }
>>   
>> @@ -1383,6 +1403,7 @@ static void aspeed_video_resolution_work(struct work_struct *work)
>>   			.u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION,
>>   		};
>>   
>> +		v4l2_dbg(1, debug, &video->v4l2_dev, "fire source change event\n");
>>   		v4l2_event_queue(&video->vdev, &ev);
>>   	} else if (test_bit(VIDEO_STREAMING, &video->flags)) {
>>   		/* No resolution change so just restart streaming */
>> @@ -1715,6 +1736,7 @@ static int aspeed_video_init(struct aspeed_video *video)
>>   		dev_err(dev, "Unable to request IRQ %d\n", irq);
>>   		return rc;
>>   	}
>> +	dev_info(video->dev, "irq %d\n", irq);
>>   
>>   	video->eclk = devm_clk_get(dev, "eclk");
>>   	if (IS_ERR(video->eclk)) {
>> @@ -1751,6 +1773,8 @@ static int aspeed_video_init(struct aspeed_video *video)
>>   		rc = -ENOMEM;
>>   		goto err_release_reserved_mem;
>>   	}
>> +	dev_info(video->dev, "alloc mem size(%d) at %#x for jpeg header\n",
>> +		 VE_JPEG_HEADER_SIZE, video->jpeg.dma);
>>   
>>   	aspeed_video_init_jpeg_table(video->jpeg.virt, video->yuv420);
>>   
> You're using both v4l2_*() and dev_*() functions for printing messages.
> They come with different prefixes, and it'd be better to stick with either,
> not both.
Sure, I agree with you. Why I still have dev_*() here is v4l2_dev isn't 
ready yet before
v4l2_device_register(). I think it could better to keep dev_*() in dev's 
probe().

-- 
Best Regards
Jammy

Re: [PATCH v5 03/10] media: aspeed: add more debug log messages

[Jammy Huang]

Hi Sakari,

On 2021/11/18 下午 07:57, Sakari Ailus wrote:
> Hi Jammy,
>
> On Thu, Nov 18, 2021 at 03:40:24PM +0800, Jammy Huang wrote:
>> The new messages are listed as below:
>> 1. jpeg header and capture buffer information
>> 2. information for each irq
>> 3. current capture mode, sync or direct-fetch
>> 4. time consumed for each frame
>> 5. input timing changed information
>>
>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>> ---
>> v5:
>>    - no update
>> v4:
>>    - modify log level
>> v3:
>>    - update commit message
>> v2:
>>    - new
>> ---
>>   drivers/media/platform/aspeed-video.c | 24 ++++++++++++++++++++++++
>>   1 file changed, 24 insertions(+)
>>
>> diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
>> index 6af57467b6d4..e8dd0a7ebfc7 100644
>> --- a/drivers/media/platform/aspeed-video.c
>> +++ b/drivers/media/platform/aspeed-video.c
>> @@ -461,12 +461,17 @@ static void aspeed_video_write(struct aspeed_video *video, u32 reg, u32 val)
>>   
>>   static void update_perf(struct aspeed_video_perf *p)
>>   {
>> +	struct aspeed_video *v = container_of(p, struct aspeed_video,
>> +					      perf);
>> +
>>   	p->duration =
>>   		ktime_to_ms(ktime_sub(ktime_get(),  p->last_sample));
>>   	p->totaltime += p->duration;
>>   
>>   	p->duration_max = max(p->duration, p->duration_max);
>>   	p->duration_min = min(p->duration, p->duration_min);
>> +	v4l2_dbg(2, debug, &v->v4l2_dev, "time consumed: %d ms\n",
>> +		 p->duration);
>>   }
>>   
>>   static int aspeed_video_start_frame(struct aspeed_video *video)
>> @@ -597,6 +602,12 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
>>   	struct aspeed_video *video = arg;
>>   	u32 sts = aspeed_video_read(video, VE_INTERRUPT_STATUS);
>>   
>> +	v4l2_dbg(2, debug, &video->v4l2_dev, "irq sts=%#x %s%s%s%s\n", sts,
>> +		 sts & VE_INTERRUPT_MODE_DETECT_WD ? ", unlock" : "",
>> +		 sts & VE_INTERRUPT_MODE_DETECT ? ", lock" : "",
>> +		 sts & VE_INTERRUPT_CAPTURE_COMPLETE ? ", capture-done" : "",
>> +		 sts & VE_INTERRUPT_COMP_COMPLETE ? ", comp-done" : "");
>> +
>>   	/*
>>   	 * Resolution changed or signal was lost; reset the engine and
>>   	 * re-initialize
>> @@ -910,6 +921,7 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>>   
>>   	/* Don't use direct mode below 1024 x 768 (irqs don't fire) */
>>   	if (size < DIRECT_FETCH_THRESHOLD) {
>> +		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Sync Mode\n");
>>   		aspeed_video_write(video, VE_TGS_0,
>>   				   FIELD_PREP(VE_TGS_FIRST,
>>   					      video->frame_left - 1) |
>> @@ -921,6 +933,7 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>>   					      video->frame_bottom + 1));
>>   		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_INT_DE);
>>   	} else {
>> +		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Direct Mode\n");
>>   		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_DIRECT_FETCH);
>>   	}
>>   
>> @@ -937,6 +950,10 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>>   		if (!aspeed_video_alloc_buf(video, &video->srcs[1], size))
>>   			goto err_mem;
>>   
>> +		v4l2_dbg(1, debug, &video->v4l2_dev, "src buf0 addr(%#x) size(%d)\n",
>> +			 video->srcs[0].dma, video->srcs[0].size);
>> +		v4l2_dbg(1, debug, &video->v4l2_dev, "src buf1 addr(%#x) size(%d)\n",
>> +			 video->srcs[1].dma, video->srcs[1].size);
>>   		aspeed_video_write(video, VE_SRC0_ADDR, video->srcs[0].dma);
>>   		aspeed_video_write(video, VE_SRC1_ADDR, video->srcs[1].dma);
>>   	}
>> @@ -1201,6 +1218,9 @@ static int aspeed_video_set_dv_timings(struct file *file, void *fh,
>>   
>>   	timings->type = V4L2_DV_BT_656_1120;
>>   
>> +	v4l2_dbg(1, debug, &video->v4l2_dev, "set new timings(%dx%d)\n",
>> +		 timings->bt.width, timings->bt.height);
>> +
>>   	return 0;
>>   }
>>   
>> @@ -1383,6 +1403,7 @@ static void aspeed_video_resolution_work(struct work_struct *work)
>>   			.u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION,
>>   		};
>>   
>> +		v4l2_dbg(1, debug, &video->v4l2_dev, "fire source change event\n");
>>   		v4l2_event_queue(&video->vdev, &ev);
>>   	} else if (test_bit(VIDEO_STREAMING, &video->flags)) {
>>   		/* No resolution change so just restart streaming */
>> @@ -1715,6 +1736,7 @@ static int aspeed_video_init(struct aspeed_video *video)
>>   		dev_err(dev, "Unable to request IRQ %d\n", irq);
>>   		return rc;
>>   	}
>> +	dev_info(video->dev, "irq %d\n", irq);
>>   
>>   	video->eclk = devm_clk_get(dev, "eclk");
>>   	if (IS_ERR(video->eclk)) {
>> @@ -1751,6 +1773,8 @@ static int aspeed_video_init(struct aspeed_video *video)
>>   		rc = -ENOMEM;
>>   		goto err_release_reserved_mem;
>>   	}
>> +	dev_info(video->dev, "alloc mem size(%d) at %#x for jpeg header\n",
>> +		 VE_JPEG_HEADER_SIZE, video->jpeg.dma);
>>   
>>   	aspeed_video_init_jpeg_table(video->jpeg.virt, video->yuv420);
>>   
> You're using both v4l2_*() and dev_*() functions for printing messages.
> They come with different prefixes, and it'd be better to stick with either,
> not both.
Sure, I agree with you. Why I still have dev_*() here is v4l2_dev isn't 
ready yet before
v4l2_device_register(). I think it could better to keep dev_*() in dev's 
probe().

-- 
Best Regards
Jammy

Re: [PATCH v5 03/10] media: aspeed: add more debug log messages

[Jammy Huang]

Hi Sakari,

On 2021/11/18 下午 07:57, Sakari Ailus wrote:
> Hi Jammy,
>
> On Thu, Nov 18, 2021 at 03:40:24PM +0800, Jammy Huang wrote:
>> The new messages are listed as below:
>> 1. jpeg header and capture buffer information
>> 2. information for each irq
>> 3. current capture mode, sync or direct-fetch
>> 4. time consumed for each frame
>> 5. input timing changed information
>>
>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>> ---
>> v5:
>>    - no update
>> v4:
>>    - modify log level
>> v3:
>>    - update commit message
>> v2:
>>    - new
>> ---
>>   drivers/media/platform/aspeed-video.c | 24 ++++++++++++++++++++++++
>>   1 file changed, 24 insertions(+)
>>
>> diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
>> index 6af57467b6d4..e8dd0a7ebfc7 100644
>> --- a/drivers/media/platform/aspeed-video.c
>> +++ b/drivers/media/platform/aspeed-video.c
>> @@ -461,12 +461,17 @@ static void aspeed_video_write(struct aspeed_video *video, u32 reg, u32 val)
>>   
>>   static void update_perf(struct aspeed_video_perf *p)
>>   {
>> +	struct aspeed_video *v = container_of(p, struct aspeed_video,
>> +					      perf);
>> +
>>   	p->duration =
>>   		ktime_to_ms(ktime_sub(ktime_get(),  p->last_sample));
>>   	p->totaltime += p->duration;
>>   
>>   	p->duration_max = max(p->duration, p->duration_max);
>>   	p->duration_min = min(p->duration, p->duration_min);
>> +	v4l2_dbg(2, debug, &v->v4l2_dev, "time consumed: %d ms\n",
>> +		 p->duration);
>>   }
>>   
>>   static int aspeed_video_start_frame(struct aspeed_video *video)
>> @@ -597,6 +602,12 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
>>   	struct aspeed_video *video = arg;
>>   	u32 sts = aspeed_video_read(video, VE_INTERRUPT_STATUS);
>>   
>> +	v4l2_dbg(2, debug, &video->v4l2_dev, "irq sts=%#x %s%s%s%s\n", sts,
>> +		 sts & VE_INTERRUPT_MODE_DETECT_WD ? ", unlock" : "",
>> +		 sts & VE_INTERRUPT_MODE_DETECT ? ", lock" : "",
>> +		 sts & VE_INTERRUPT_CAPTURE_COMPLETE ? ", capture-done" : "",
>> +		 sts & VE_INTERRUPT_COMP_COMPLETE ? ", comp-done" : "");
>> +
>>   	/*
>>   	 * Resolution changed or signal was lost; reset the engine and
>>   	 * re-initialize
>> @@ -910,6 +921,7 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>>   
>>   	/* Don't use direct mode below 1024 x 768 (irqs don't fire) */
>>   	if (size < DIRECT_FETCH_THRESHOLD) {
>> +		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Sync Mode\n");
>>   		aspeed_video_write(video, VE_TGS_0,
>>   				   FIELD_PREP(VE_TGS_FIRST,
>>   					      video->frame_left - 1) |
>> @@ -921,6 +933,7 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>>   					      video->frame_bottom + 1));
>>   		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_INT_DE);
>>   	} else {
>> +		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Direct Mode\n");
>>   		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_DIRECT_FETCH);
>>   	}
>>   
>> @@ -937,6 +950,10 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>>   		if (!aspeed_video_alloc_buf(video, &video->srcs[1], size))
>>   			goto err_mem;
>>   
>> +		v4l2_dbg(1, debug, &video->v4l2_dev, "src buf0 addr(%#x) size(%d)\n",
>> +			 video->srcs[0].dma, video->srcs[0].size);
>> +		v4l2_dbg(1, debug, &video->v4l2_dev, "src buf1 addr(%#x) size(%d)\n",
>> +			 video->srcs[1].dma, video->srcs[1].size);
>>   		aspeed_video_write(video, VE_SRC0_ADDR, video->srcs[0].dma);
>>   		aspeed_video_write(video, VE_SRC1_ADDR, video->srcs[1].dma);
>>   	}
>> @@ -1201,6 +1218,9 @@ static int aspeed_video_set_dv_timings(struct file *file, void *fh,
>>   
>>   	timings->type = V4L2_DV_BT_656_1120;
>>   
>> +	v4l2_dbg(1, debug, &video->v4l2_dev, "set new timings(%dx%d)\n",
>> +		 timings->bt.width, timings->bt.height);
>> +
>>   	return 0;
>>   }
>>   
>> @@ -1383,6 +1403,7 @@ static void aspeed_video_resolution_work(struct work_struct *work)
>>   			.u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION,
>>   		};
>>   
>> +		v4l2_dbg(1, debug, &video->v4l2_dev, "fire source change event\n");
>>   		v4l2_event_queue(&video->vdev, &ev);
>>   	} else if (test_bit(VIDEO_STREAMING, &video->flags)) {
>>   		/* No resolution change so just restart streaming */
>> @@ -1715,6 +1736,7 @@ static int aspeed_video_init(struct aspeed_video *video)
>>   		dev_err(dev, "Unable to request IRQ %d\n", irq);
>>   		return rc;
>>   	}
>> +	dev_info(video->dev, "irq %d\n", irq);
>>   
>>   	video->eclk = devm_clk_get(dev, "eclk");
>>   	if (IS_ERR(video->eclk)) {
>> @@ -1751,6 +1773,8 @@ static int aspeed_video_init(struct aspeed_video *video)
>>   		rc = -ENOMEM;
>>   		goto err_release_reserved_mem;
>>   	}
>> +	dev_info(video->dev, "alloc mem size(%d) at %#x for jpeg header\n",
>> +		 VE_JPEG_HEADER_SIZE, video->jpeg.dma);
>>   
>>   	aspeed_video_init_jpeg_table(video->jpeg.virt, video->yuv420);
>>   
> You're using both v4l2_*() and dev_*() functions for printing messages.
> They come with different prefixes, and it'd be better to stick with either,
> not both.
Sure, I agree with you. Why I still have dev_*() here is v4l2_dev isn't 
ready yet before
v4l2_device_register(). I think it could better to keep dev_*() in dev's 
probe().

-- 
Best Regards
Jammy


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

Hi Sakari,

On 2021/11/18 下午 08:33, Sakari Ailus wrote:
> Hi Jammy,
>
> On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
>> This introduces support for the Aspeed JPEG format, where the new frame
>> can refer to previous frame to reduce the amount of compressed data. The
>> concept is similar to I/P frame of video compression. I will compare the
>> new frame with previous one to decide which macroblock's data is
>> changed, and only the changed macroblocks will be compressed.
>>
>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>> which is generally adapted for remote KVM.
>>
>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>> ---
>> v5:
>>    - no update
>> v4:
>>    - new
>> ---
>>   Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>>   drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>>   include/uapi/linux/videodev2.h                   |  1 +
>>   3 files changed, 14 insertions(+)
>>
>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>> index b2cd155e691b..23c05063133d 100644
>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>> @@ -264,6 +264,18 @@ please make a proposal on the linux-media mailing list.
>>   	of tiles, resulting in 32-aligned resolutions for the luminance plane
>>   	and 16-aligned resolutions for the chrominance plane (with 2x2
>>   	subsampling).
>> +    * .. _V4L2-PIX-FMT-AJPG:
>> +
>> +      - ``V4L2_PIX_FMT_AJPG``
>> +      - 'AJPG'
>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>> +        which is generally adapted for remote KVM.
>> +        On each frame compression, I will compare the new frame with previous
>> +        one to decide which macroblock's data is changed, and only the changed
>> +        macroblocks will be compressed.
>> +
>> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
>> +        for more information.
> Is this datasheet publicly available? Do you have a URL?

Sorry, this datasheet is not publicly available.
Hans mentioned this as well in the discussion below:

https://lkml.org/lkml/2021/11/10/101

-- 
Best Regards
Jammy

Re: [PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

Hi Sakari,

On 2021/11/18 下午 08:33, Sakari Ailus wrote:
> Hi Jammy,
>
> On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
>> This introduces support for the Aspeed JPEG format, where the new frame
>> can refer to previous frame to reduce the amount of compressed data. The
>> concept is similar to I/P frame of video compression. I will compare the
>> new frame with previous one to decide which macroblock's data is
>> changed, and only the changed macroblocks will be compressed.
>>
>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>> which is generally adapted for remote KVM.
>>
>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>> ---
>> v5:
>>    - no update
>> v4:
>>    - new
>> ---
>>   Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>>   drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>>   include/uapi/linux/videodev2.h                   |  1 +
>>   3 files changed, 14 insertions(+)
>>
>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>> index b2cd155e691b..23c05063133d 100644
>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>> @@ -264,6 +264,18 @@ please make a proposal on the linux-media mailing list.
>>   	of tiles, resulting in 32-aligned resolutions for the luminance plane
>>   	and 16-aligned resolutions for the chrominance plane (with 2x2
>>   	subsampling).
>> +    * .. _V4L2-PIX-FMT-AJPG:
>> +
>> +      - ``V4L2_PIX_FMT_AJPG``
>> +      - 'AJPG'
>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>> +        which is generally adapted for remote KVM.
>> +        On each frame compression, I will compare the new frame with previous
>> +        one to decide which macroblock's data is changed, and only the changed
>> +        macroblocks will be compressed.
>> +
>> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
>> +        for more information.
> Is this datasheet publicly available? Do you have a URL?

Sorry, this datasheet is not publicly available.
Hans mentioned this as well in the discussion below:

https://lkml.org/lkml/2021/11/10/101

-- 
Best Regards
Jammy

Re: [PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

Hi Sakari,

On 2021/11/18 下午 08:33, Sakari Ailus wrote:
> Hi Jammy,
>
> On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
>> This introduces support for the Aspeed JPEG format, where the new frame
>> can refer to previous frame to reduce the amount of compressed data. The
>> concept is similar to I/P frame of video compression. I will compare the
>> new frame with previous one to decide which macroblock's data is
>> changed, and only the changed macroblocks will be compressed.
>>
>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>> which is generally adapted for remote KVM.
>>
>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>> ---
>> v5:
>>    - no update
>> v4:
>>    - new
>> ---
>>   Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>>   drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>>   include/uapi/linux/videodev2.h                   |  1 +
>>   3 files changed, 14 insertions(+)
>>
>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>> index b2cd155e691b..23c05063133d 100644
>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>> @@ -264,6 +264,18 @@ please make a proposal on the linux-media mailing list.
>>   	of tiles, resulting in 32-aligned resolutions for the luminance plane
>>   	and 16-aligned resolutions for the chrominance plane (with 2x2
>>   	subsampling).
>> +    * .. _V4L2-PIX-FMT-AJPG:
>> +
>> +      - ``V4L2_PIX_FMT_AJPG``
>> +      - 'AJPG'
>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>> +        which is generally adapted for remote KVM.
>> +        On each frame compression, I will compare the new frame with previous
>> +        one to decide which macroblock's data is changed, and only the changed
>> +        macroblocks will be compressed.
>> +
>> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
>> +        for more information.
> Is this datasheet publicly available? Do you have a URL?

Sorry, this datasheet is not publicly available.
Hans mentioned this as well in the discussion below:

https://lkml.org/lkml/2021/11/10/101

-- 
Best Regards
Jammy


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Paul Menzel]

Dear Jammy,


Am 19.11.21 um 03:02 schrieb Jammy Huang:

> On 2021/11/18 下午 08:33, Sakari Ailus wrote:

>> On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
>>> This introduces support for the Aspeed JPEG format, where the new frame
>>> can refer to previous frame to reduce the amount of compressed data. The
>>> concept is similar to I/P frame of video compression. I will compare the
>>> new frame with previous one to decide which macroblock's data is
>>> changed, and only the changed macroblocks will be compressed.
>>>
>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>> which is generally adapted for remote KVM.
>>>
>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>> ---
>>> v5:
>>>    - no update
>>> v4:
>>>    - new
>>> ---
>>>   Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>>>   drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>>>   include/uapi/linux/videodev2.h                   |  1 +
>>>   3 files changed, 14 insertions(+)
>>>
>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst 
>>> b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>> index b2cd155e691b..23c05063133d 100644
>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>> @@ -264,6 +264,18 @@ please make a proposal on the linux-media 
>>> mailing list.
>>>       of tiles, resulting in 32-aligned resolutions for the luminance 
>>> plane
>>>       and 16-aligned resolutions for the chrominance plane (with 2x2
>>>       subsampling).
>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>> +
>>> +      - ``V4L2_PIX_FMT_AJPG``
>>> +      - 'AJPG'
>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>> +        which is generally adapted for remote KVM.
>>> +        On each frame compression, I will compare the new frame with previous
>>> +        one to decide which macroblock's data is changed, and only the changed
>>> +        macroblocks will be compressed.
>>> +
>>> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
>>> +        for more information.
>> Is this datasheet publicly available? Do you have a URL?
> 
> Sorry, this datasheet is not publicly available.
> Hans mentioned this as well in the discussion below:
> 
> https://lkml.org/lkml/2021/11/10/101

If questions come up during review, please also add the answers to the 
commit message of the next iteration. ;-) Maybe:

> The implementation is based on datasheet *Name goes here*, revision
> X, which is not publicly available.

Kind regards,

Paul

Re: [PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Paul Menzel]

Dear Jammy,


Am 19.11.21 um 03:02 schrieb Jammy Huang:

> On 2021/11/18 下午 08:33, Sakari Ailus wrote:

>> On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
>>> This introduces support for the Aspeed JPEG format, where the new frame
>>> can refer to previous frame to reduce the amount of compressed data. The
>>> concept is similar to I/P frame of video compression. I will compare the
>>> new frame with previous one to decide which macroblock's data is
>>> changed, and only the changed macroblocks will be compressed.
>>>
>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>> which is generally adapted for remote KVM.
>>>
>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>> ---
>>> v5:
>>>    - no update
>>> v4:
>>>    - new
>>> ---
>>>   Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>>>   drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>>>   include/uapi/linux/videodev2.h                   |  1 +
>>>   3 files changed, 14 insertions(+)
>>>
>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst 
>>> b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>> index b2cd155e691b..23c05063133d 100644
>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>> @@ -264,6 +264,18 @@ please make a proposal on the linux-media 
>>> mailing list.
>>>       of tiles, resulting in 32-aligned resolutions for the luminance 
>>> plane
>>>       and 16-aligned resolutions for the chrominance plane (with 2x2
>>>       subsampling).
>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>> +
>>> +      - ``V4L2_PIX_FMT_AJPG``
>>> +      - 'AJPG'
>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>> +        which is generally adapted for remote KVM.
>>> +        On each frame compression, I will compare the new frame with previous
>>> +        one to decide which macroblock's data is changed, and only the changed
>>> +        macroblocks will be compressed.
>>> +
>>> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
>>> +        for more information.
>> Is this datasheet publicly available? Do you have a URL?
> 
> Sorry, this datasheet is not publicly available.
> Hans mentioned this as well in the discussion below:
> 
> https://lkml.org/lkml/2021/11/10/101

If questions come up during review, please also add the answers to the 
commit message of the next iteration. ;-) Maybe:

> The implementation is based on datasheet *Name goes here*, revision
> X, which is not publicly available.

Kind regards,

Paul

Re: [PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Paul Menzel]

Dear Jammy,


Am 19.11.21 um 03:02 schrieb Jammy Huang:

> On 2021/11/18 下午 08:33, Sakari Ailus wrote:

>> On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
>>> This introduces support for the Aspeed JPEG format, where the new frame
>>> can refer to previous frame to reduce the amount of compressed data. The
>>> concept is similar to I/P frame of video compression. I will compare the
>>> new frame with previous one to decide which macroblock's data is
>>> changed, and only the changed macroblocks will be compressed.
>>>
>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>> which is generally adapted for remote KVM.
>>>
>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>> ---
>>> v5:
>>>    - no update
>>> v4:
>>>    - new
>>> ---
>>>   Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>>>   drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>>>   include/uapi/linux/videodev2.h                   |  1 +
>>>   3 files changed, 14 insertions(+)
>>>
>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst 
>>> b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>> index b2cd155e691b..23c05063133d 100644
>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>> @@ -264,6 +264,18 @@ please make a proposal on the linux-media 
>>> mailing list.
>>>       of tiles, resulting in 32-aligned resolutions for the luminance 
>>> plane
>>>       and 16-aligned resolutions for the chrominance plane (with 2x2
>>>       subsampling).
>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>> +
>>> +      - ``V4L2_PIX_FMT_AJPG``
>>> +      - 'AJPG'
>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>> +        which is generally adapted for remote KVM.
>>> +        On each frame compression, I will compare the new frame with previous
>>> +        one to decide which macroblock's data is changed, and only the changed
>>> +        macroblocks will be compressed.
>>> +
>>> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
>>> +        for more information.
>> Is this datasheet publicly available? Do you have a URL?
> 
> Sorry, this datasheet is not publicly available.
> Hans mentioned this as well in the discussion below:
> 
> https://lkml.org/lkml/2021/11/10/101

If questions come up during review, please also add the answers to the 
commit message of the next iteration. ;-) Maybe:

> The implementation is based on datasheet *Name goes here*, revision
> X, which is not publicly available.

Kind regards,

Paul

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

Dear Paul,

OK, thanks for your suggestion.

On 2021/11/19 下午 04:39, Paul Menzel wrote:
> Dear Jammy,
>
>
> Am 19.11.21 um 03:02 schrieb Jammy Huang:
>
>> On 2021/11/18 下午 08:33, Sakari Ailus wrote:
>>> On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
>>>> This introduces support for the Aspeed JPEG format, where the new frame
>>>> can refer to previous frame to reduce the amount of compressed data. The
>>>> concept is similar to I/P frame of video compression. I will compare the
>>>> new frame with previous one to decide which macroblock's data is
>>>> changed, and only the changed macroblocks will be compressed.
>>>>
>>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>>> which is generally adapted for remote KVM.
>>>>
>>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>>> ---
>>>> v5:
>>>>     - no update
>>>> v4:
>>>>     - new
>>>> ---
>>>>    Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>>>>    drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>>>>    include/uapi/linux/videodev2.h                   |  1 +
>>>>    3 files changed, 14 insertions(+)
>>>>
>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> index b2cd155e691b..23c05063133d 100644
>>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> @@ -264,6 +264,18 @@ please make a proposal on the linux-media
>>>> mailing list.
>>>>        of tiles, resulting in 32-aligned resolutions for the luminance
>>>> plane
>>>>        and 16-aligned resolutions for the chrominance plane (with 2x2
>>>>        subsampling).
>>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>>> +
>>>> +      - ``V4L2_PIX_FMT_AJPG``
>>>> +      - 'AJPG'
>>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>>> +        which is generally adapted for remote KVM.
>>>> +        On each frame compression, I will compare the new frame with previous
>>>> +        one to decide which macroblock's data is changed, and only the changed
>>>> +        macroblocks will be compressed.
>>>> +
>>>> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
>>>> +        for more information.
>>> Is this datasheet publicly available? Do you have a URL?
>> Sorry, this datasheet is not publicly available.
>> Hans mentioned this as well in the discussion below:
>>
>> https://lkml.org/lkml/2021/11/10/101
> If questions come up during review, please also add the answers to the
> commit message of the next iteration. ;-) Maybe:
>
>> The implementation is based on datasheet *Name goes here*, revision
>> X, which is not publicly available.
> Kind regards,
>
> Paul

-- 
Best Regards
Jammy

Re: [PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

Dear Paul,

OK, thanks for your suggestion.

On 2021/11/19 下午 04:39, Paul Menzel wrote:
> Dear Jammy,
>
>
> Am 19.11.21 um 03:02 schrieb Jammy Huang:
>
>> On 2021/11/18 下午 08:33, Sakari Ailus wrote:
>>> On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
>>>> This introduces support for the Aspeed JPEG format, where the new frame
>>>> can refer to previous frame to reduce the amount of compressed data. The
>>>> concept is similar to I/P frame of video compression. I will compare the
>>>> new frame with previous one to decide which macroblock's data is
>>>> changed, and only the changed macroblocks will be compressed.
>>>>
>>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>>> which is generally adapted for remote KVM.
>>>>
>>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>>> ---
>>>> v5:
>>>>     - no update
>>>> v4:
>>>>     - new
>>>> ---
>>>>    Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>>>>    drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>>>>    include/uapi/linux/videodev2.h                   |  1 +
>>>>    3 files changed, 14 insertions(+)
>>>>
>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> index b2cd155e691b..23c05063133d 100644
>>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> @@ -264,6 +264,18 @@ please make a proposal on the linux-media
>>>> mailing list.
>>>>        of tiles, resulting in 32-aligned resolutions for the luminance
>>>> plane
>>>>        and 16-aligned resolutions for the chrominance plane (with 2x2
>>>>        subsampling).
>>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>>> +
>>>> +      - ``V4L2_PIX_FMT_AJPG``
>>>> +      - 'AJPG'
>>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>>> +        which is generally adapted for remote KVM.
>>>> +        On each frame compression, I will compare the new frame with previous
>>>> +        one to decide which macroblock's data is changed, and only the changed
>>>> +        macroblocks will be compressed.
>>>> +
>>>> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
>>>> +        for more information.
>>> Is this datasheet publicly available? Do you have a URL?
>> Sorry, this datasheet is not publicly available.
>> Hans mentioned this as well in the discussion below:
>>
>> https://lkml.org/lkml/2021/11/10/101
> If questions come up during review, please also add the answers to the
> commit message of the next iteration. ;-) Maybe:
>
>> The implementation is based on datasheet *Name goes here*, revision
>> X, which is not publicly available.
> Kind regards,
>
> Paul

-- 
Best Regards
Jammy

Re: [PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

Dear Paul,

OK, thanks for your suggestion.

On 2021/11/19 下午 04:39, Paul Menzel wrote:
> Dear Jammy,
>
>
> Am 19.11.21 um 03:02 schrieb Jammy Huang:
>
>> On 2021/11/18 下午 08:33, Sakari Ailus wrote:
>>> On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
>>>> This introduces support for the Aspeed JPEG format, where the new frame
>>>> can refer to previous frame to reduce the amount of compressed data. The
>>>> concept is similar to I/P frame of video compression. I will compare the
>>>> new frame with previous one to decide which macroblock's data is
>>>> changed, and only the changed macroblocks will be compressed.
>>>>
>>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>>> which is generally adapted for remote KVM.
>>>>
>>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>>> ---
>>>> v5:
>>>>     - no update
>>>> v4:
>>>>     - new
>>>> ---
>>>>    Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>>>>    drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>>>>    include/uapi/linux/videodev2.h                   |  1 +
>>>>    3 files changed, 14 insertions(+)
>>>>
>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> index b2cd155e691b..23c05063133d 100644
>>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> @@ -264,6 +264,18 @@ please make a proposal on the linux-media
>>>> mailing list.
>>>>        of tiles, resulting in 32-aligned resolutions for the luminance
>>>> plane
>>>>        and 16-aligned resolutions for the chrominance plane (with 2x2
>>>>        subsampling).
>>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>>> +
>>>> +      - ``V4L2_PIX_FMT_AJPG``
>>>> +      - 'AJPG'
>>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>>> +        which is generally adapted for remote KVM.
>>>> +        On each frame compression, I will compare the new frame with previous
>>>> +        one to decide which macroblock's data is changed, and only the changed
>>>> +        macroblocks will be compressed.
>>>> +
>>>> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
>>>> +        for more information.
>>> Is this datasheet publicly available? Do you have a URL?
>> Sorry, this datasheet is not publicly available.
>> Hans mentioned this as well in the discussion below:
>>
>> https://lkml.org/lkml/2021/11/10/101
> If questions come up during review, please also add the answers to the
> commit message of the next iteration. ;-) Maybe:
>
>> The implementation is based on datasheet *Name goes here*, revision
>> X, which is not publicly available.
> Kind regards,
>
> Paul

-- 
Best Regards
Jammy


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Sakari Ailus]

Hi Jammy,

On Fri, Nov 19, 2021 at 10:02:40AM +0800, Jammy Huang wrote:
> Hi Sakari,
> 
> On 2021/11/18 下午 08:33, Sakari Ailus wrote:
> > Hi Jammy,
> > 
> > On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
> > > This introduces support for the Aspeed JPEG format, where the new frame
> > > can refer to previous frame to reduce the amount of compressed data. The
> > > concept is similar to I/P frame of video compression. I will compare the
> > > new frame with previous one to decide which macroblock's data is
> > > changed, and only the changed macroblocks will be compressed.
> > > 
> > > This Aspeed JPEG format is used by the video engine on Aspeed platforms,
> > > which is generally adapted for remote KVM.
> > > 
> > > Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
> > > ---
> > > v5:
> > >    - no update
> > > v4:
> > >    - new
> > > ---
> > >   Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
> > >   drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
> > >   include/uapi/linux/videodev2.h                   |  1 +
> > >   3 files changed, 14 insertions(+)
> > > 
> > > diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> > > index b2cd155e691b..23c05063133d 100644
> > > --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> > > +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> > > @@ -264,6 +264,18 @@ please make a proposal on the linux-media mailing list.
> > >   	of tiles, resulting in 32-aligned resolutions for the luminance plane
> > >   	and 16-aligned resolutions for the chrominance plane (with 2x2
> > >   	subsampling).
> > > +    * .. _V4L2-PIX-FMT-AJPG:
> > > +
> > > +      - ``V4L2_PIX_FMT_AJPG``
> > > +      - 'AJPG'
> > > +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
> > > +        which is generally adapted for remote KVM.
> > > +        On each frame compression, I will compare the new frame with previous
> > > +        one to decide which macroblock's data is changed, and only the changed
> > > +        macroblocks will be compressed.
> > > +
> > > +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
> > > +        for more information.
> > Is this datasheet publicly available? Do you have a URL?
> 
> Sorry, this datasheet is not publicly available.
> Hans mentioned this as well in the discussion below:
> 
> https://lkml.org/lkml/2021/11/10/101

If the vendor documentation is not publicly available, you'll need to have
the format documented here. Alternatively an open source implementation
(e.g. LGPL) is presumably fine, too.

-- 
Regards,

Sakari Ailus

Re: [PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Sakari Ailus]

Hi Jammy,

On Fri, Nov 19, 2021 at 10:02:40AM +0800, Jammy Huang wrote:
> Hi Sakari,
> 
> On 2021/11/18 下午 08:33, Sakari Ailus wrote:
> > Hi Jammy,
> > 
> > On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
> > > This introduces support for the Aspeed JPEG format, where the new frame
> > > can refer to previous frame to reduce the amount of compressed data. The
> > > concept is similar to I/P frame of video compression. I will compare the
> > > new frame with previous one to decide which macroblock's data is
> > > changed, and only the changed macroblocks will be compressed.
> > > 
> > > This Aspeed JPEG format is used by the video engine on Aspeed platforms,
> > > which is generally adapted for remote KVM.
> > > 
> > > Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
> > > ---
> > > v5:
> > >    - no update
> > > v4:
> > >    - new
> > > ---
> > >   Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
> > >   drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
> > >   include/uapi/linux/videodev2.h                   |  1 +
> > >   3 files changed, 14 insertions(+)
> > > 
> > > diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> > > index b2cd155e691b..23c05063133d 100644
> > > --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> > > +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> > > @@ -264,6 +264,18 @@ please make a proposal on the linux-media mailing list.
> > >   	of tiles, resulting in 32-aligned resolutions for the luminance plane
> > >   	and 16-aligned resolutions for the chrominance plane (with 2x2
> > >   	subsampling).
> > > +    * .. _V4L2-PIX-FMT-AJPG:
> > > +
> > > +      - ``V4L2_PIX_FMT_AJPG``
> > > +      - 'AJPG'
> > > +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
> > > +        which is generally adapted for remote KVM.
> > > +        On each frame compression, I will compare the new frame with previous
> > > +        one to decide which macroblock's data is changed, and only the changed
> > > +        macroblocks will be compressed.
> > > +
> > > +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
> > > +        for more information.
> > Is this datasheet publicly available? Do you have a URL?
> 
> Sorry, this datasheet is not publicly available.
> Hans mentioned this as well in the discussion below:
> 
> https://lkml.org/lkml/2021/11/10/101

If the vendor documentation is not publicly available, you'll need to have
the format documented here. Alternatively an open source implementation
(e.g. LGPL) is presumably fine, too.

-- 
Regards,

Sakari Ailus

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Sakari Ailus]

Hi Jammy,

On Fri, Nov 19, 2021 at 10:02:40AM +0800, Jammy Huang wrote:
> Hi Sakari,
> 
> On 2021/11/18 下午 08:33, Sakari Ailus wrote:
> > Hi Jammy,
> > 
> > On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
> > > This introduces support for the Aspeed JPEG format, where the new frame
> > > can refer to previous frame to reduce the amount of compressed data. The
> > > concept is similar to I/P frame of video compression. I will compare the
> > > new frame with previous one to decide which macroblock's data is
> > > changed, and only the changed macroblocks will be compressed.
> > > 
> > > This Aspeed JPEG format is used by the video engine on Aspeed platforms,
> > > which is generally adapted for remote KVM.
> > > 
> > > Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
> > > ---
> > > v5:
> > >    - no update
> > > v4:
> > >    - new
> > > ---
> > >   Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
> > >   drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
> > >   include/uapi/linux/videodev2.h                   |  1 +
> > >   3 files changed, 14 insertions(+)
> > > 
> > > diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> > > index b2cd155e691b..23c05063133d 100644
> > > --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> > > +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> > > @@ -264,6 +264,18 @@ please make a proposal on the linux-media mailing list.
> > >   	of tiles, resulting in 32-aligned resolutions for the luminance plane
> > >   	and 16-aligned resolutions for the chrominance plane (with 2x2
> > >   	subsampling).
> > > +    * .. _V4L2-PIX-FMT-AJPG:
> > > +
> > > +      - ``V4L2_PIX_FMT_AJPG``
> > > +      - 'AJPG'
> > > +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
> > > +        which is generally adapted for remote KVM.
> > > +        On each frame compression, I will compare the new frame with previous
> > > +        one to decide which macroblock's data is changed, and only the changed
> > > +        macroblocks will be compressed.
> > > +
> > > +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
> > > +        for more information.
> > Is this datasheet publicly available? Do you have a URL?
> 
> Sorry, this datasheet is not publicly available.
> Hans mentioned this as well in the discussion below:
> 
> https://lkml.org/lkml/2021/11/10/101

If the vendor documentation is not publicly available, you'll need to have
the format documented here. Alternatively an open source implementation
(e.g. LGPL) is presumably fine, too.

-- 
Regards,

Sakari Ailus

Re: [PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[Hans Verkuil]

Hi Jammy,

I suggest posting a v3 that moves the patches that deal with the
Aspeed JPEG format to the end of the series. That way I can easily
merge the patches up to that point, and the Aspeed JPEG support can
be handled separately.

I'm not sure if it can be merged without that format being documented,
either in pixfmt-reserved.rst, by implementing support for it in
libv4lconvert from v4l-utils, or by providing a URL to some other
publicly available source code. We really don't like adding pixel
formats that cannot be interpreted by someone without access to the
datasheets.

Regards,

	Hans

On 18/11/2021 08:40, Jammy Huang wrote:
> The aim of this series is to add aspeed-jpeg support for aspeed-video
> driver.
> 
> To achieve this major goal some refactors are included.
> 
> In the last, debugfs information is also updated per this change.
> 
> Changes in v5:
>  - Use model data to tell different soc
> 
> Changes in v4:
>  - Add definition for the Aspeed JPEG format
>  - Reserve controls for ASPEED
>  - Use s_fmt to update format rather than new control
>  - Update aspeed hq quality range, 1 ~ 12
> 
> 
> Jammy Huang (10):
>   media: aspeed: move err-handling together to the bottom
>   media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log
>   media: aspeed: add more debug log messages
>   media: aspeed: refactor to gather format/compress settings
>   media: v4l: Add definition for the Aspeed JPEG format
>   media: v4l2-ctrls: Reserve controls for ASPEED
>   media: aspeed: use model-data
>   media: aspeed: Support aspeed mode to reduce compressed data
>   media: aspeed: add comments and macro
>   media: aspeed: Extend debug message
> 
>  .../media/uapi/v4l/pixfmt-reserved.rst        |  12 +
>  drivers/media/platform/aspeed-video.c         | 534 ++++++++++++++----
>  drivers/media/v4l2-core/v4l2-ioctl.c          |   1 +
>  include/uapi/linux/aspeed-video.h             |  15 +
>  include/uapi/linux/v4l2-controls.h            |   5 +
>  include/uapi/linux/videodev2.h                |   1 +
>  6 files changed, 467 insertions(+), 101 deletions(-)
>  create mode 100644 include/uapi/linux/aspeed-video.h
> 

Re: [PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[Hans Verkuil]

Hi Jammy,

I suggest posting a v3 that moves the patches that deal with the
Aspeed JPEG format to the end of the series. That way I can easily
merge the patches up to that point, and the Aspeed JPEG support can
be handled separately.

I'm not sure if it can be merged without that format being documented,
either in pixfmt-reserved.rst, by implementing support for it in
libv4lconvert from v4l-utils, or by providing a URL to some other
publicly available source code. We really don't like adding pixel
formats that cannot be interpreted by someone without access to the
datasheets.

Regards,

	Hans

On 18/11/2021 08:40, Jammy Huang wrote:
> The aim of this series is to add aspeed-jpeg support for aspeed-video
> driver.
> 
> To achieve this major goal some refactors are included.
> 
> In the last, debugfs information is also updated per this change.
> 
> Changes in v5:
>  - Use model data to tell different soc
> 
> Changes in v4:
>  - Add definition for the Aspeed JPEG format
>  - Reserve controls for ASPEED
>  - Use s_fmt to update format rather than new control
>  - Update aspeed hq quality range, 1 ~ 12
> 
> 
> Jammy Huang (10):
>   media: aspeed: move err-handling together to the bottom
>   media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log
>   media: aspeed: add more debug log messages
>   media: aspeed: refactor to gather format/compress settings
>   media: v4l: Add definition for the Aspeed JPEG format
>   media: v4l2-ctrls: Reserve controls for ASPEED
>   media: aspeed: use model-data
>   media: aspeed: Support aspeed mode to reduce compressed data
>   media: aspeed: add comments and macro
>   media: aspeed: Extend debug message
> 
>  .../media/uapi/v4l/pixfmt-reserved.rst        |  12 +
>  drivers/media/platform/aspeed-video.c         | 534 ++++++++++++++----
>  drivers/media/v4l2-core/v4l2-ioctl.c          |   1 +
>  include/uapi/linux/aspeed-video.h             |  15 +
>  include/uapi/linux/v4l2-controls.h            |   5 +
>  include/uapi/linux/videodev2.h                |   1 +
>  6 files changed, 467 insertions(+), 101 deletions(-)
>  create mode 100644 include/uapi/linux/aspeed-video.h
> 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 08/10] media: aspeed: Support aspeed mode to reduce compressed data

[Hans Verkuil]

On 18/11/2021 08:40, Jammy Huang wrote:
> aspeed supports differential jpeg format which only compress the parts
> which are changed. In this way, it reduces both the amount of data to be
> transferred by network and those to be decoded on the client side.
> 
> 4 new ctrls are added:

3 controls. This commit message is out of date!

> * Aspeed JPEG Format: to control aspeed's partial jpeg on/off
>   0: standard jpeg, 1: aspeed jpeg
> * Aspeed Compression Mode: to control aspeed's compression mode
>   0: DCT Only, 1: DCT VQ mix 2-color, 2: DCT VQ mix 4-color
>   This is AST2400 only. It will adapt JPEG or VQ encoding method according
>   to the context automatically.
> * Aspeed HQ Mode: to control aspeed's high quality(2-pass) compression mode
>   This only works with yuv444 subsampling.
> * Aspeed HQ Quality: to control the quality of aspeed's HQ mode
>   only useful if Aspeed HQ mode is enabled
> 
> Aspeed JPEG Format requires an additional buffer, called bcd, to store
> the information about which macro block in the new frame is different
> from the previous one.
> 
> To have bcd correctly working, we need to swap the buffers for src0/1 to
> make src1 refer to previous frame and src0 to the coming new frame.
> 
> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
> ---
> v5:
>   - use runtime configuration to handle differences between soc
>   - add aspeed_video_swap_src_buf().
> v4
>   - add include/uapi/linux/aspeed-video.h for V4L2_CID_ASPEED_XXX
>   - add aspeed_video_set_format
>   - remove aspeed_ctrl_format
>   - add variable, format, to replace partial_jpeg
>   - modify aspeed_ctrl_HQ_jpeg_quality's range
>   - reduce bcd buffer size
>   - update commit message
> v3:
>   - 'compression mode' only available for G4
> v2:
>   - update commit message
>   - use v4l2_xxx log rather than self-defined macro
> ---
>  drivers/media/platform/aspeed-video.c | 252 +++++++++++++++++++++++---
>  include/uapi/linux/aspeed-video.h     |  15 ++
>  2 files changed, 243 insertions(+), 24 deletions(-)
>  create mode 100644 include/uapi/linux/aspeed-video.h
> 
> diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
> index ba8ee82b38c3..419825ed7737 100644
> --- a/drivers/media/platform/aspeed-video.c
> +++ b/drivers/media/platform/aspeed-video.c
> @@ -31,6 +31,7 @@
>  #include <media/v4l2-ioctl.h>
>  #include <media/videobuf2-dma-contig.h>
>  #include <linux/videodev2.h>
> +#include <uapi/linux/aspeed-video.h>
>  
>  #define DEVICE_NAME			"aspeed-video"
>  
> @@ -55,6 +56,7 @@
>  
>  #define VE_MAX_SRC_BUFFER_SIZE		0x8ca000 /* 1920 * 1200, 32bpp */
>  #define VE_JPEG_HEADER_SIZE		0x006000 /* 512 * 12 * 4 */
> +#define VE_BCD_BUFF_SIZE		0x9000 /* (1920/8) * (1200/8) */
>  
>  #define VE_PROTECTION_KEY		0x000
>  #define  VE_PROTECTION_KEY_UNLOCK	0x1a038aa8
> @@ -105,6 +107,13 @@
>  #define VE_SCALING_FILTER2		0x020
>  #define VE_SCALING_FILTER3		0x024
>  
> +#define VE_BCD_CTRL			0x02C
> +#define  VE_BCD_CTRL_EN_BCD		BIT(0)
> +#define  VE_BCD_CTRL_EN_ABCD		BIT(1)
> +#define  VE_BCD_CTRL_EN_CB		BIT(2)
> +#define  VE_BCD_CTRL_THR		GENMASK(23, 16)
> +#define  VE_BCD_CTRL_ABCD_THR		GENMASK(31, 24)
> +
>  #define VE_CAP_WINDOW			0x030
>  #define VE_COMP_WINDOW			0x034
>  #define VE_COMP_PROC_OFFSET		0x038
> @@ -113,6 +122,7 @@
>  #define VE_SRC0_ADDR			0x044
>  #define VE_SRC_SCANLINE_OFFSET		0x048
>  #define VE_SRC1_ADDR			0x04c
> +#define VE_BCD_ADDR			0x050
>  #define VE_COMP_ADDR			0x054
>  
>  #define VE_STREAM_BUF_SIZE		0x058
> @@ -133,6 +143,8 @@
>  #define  VE_COMP_CTRL_HQ_DCT_CHR	GENMASK(26, 22)
>  #define  VE_COMP_CTRL_HQ_DCT_LUM	GENMASK(31, 27)
>  
> +#define VE_CB_ADDR			0x06C
> +
>  #define AST2400_VE_COMP_SIZE_READ_BACK	0x078
>  #define AST2600_VE_COMP_SIZE_READ_BACK	0x084
>  
> @@ -191,6 +203,12 @@ enum {
>  	VIDEO_CLOCKS_ON,
>  };
>  
> +enum aspeed_video_format {
> +	VIDEO_FMT_STANDARD = 0,
> +	VIDEO_FMT_ASPEED,
> +	VIDEO_FMT_MAX = VIDEO_FMT_ASPEED
> +};
> +
>  struct aspeed_video_addr {
>  	unsigned int size;
>  	dma_addr_t dma;
> @@ -241,10 +259,15 @@ struct aspeed_video {
>  	unsigned int max_compressed_size;
>  	struct aspeed_video_addr srcs[2];
>  	struct aspeed_video_addr jpeg;
> +	struct aspeed_video_addr bcd;
>  
>  	bool yuv420;
> +	enum aspeed_video_format format;
> +	bool hq_mode;
>  	unsigned int frame_rate;
>  	unsigned int jpeg_quality;
> +	unsigned int jpeg_hq_quality;
> +	unsigned int compression_mode;
>  
>  	unsigned int frame_bottom;
>  	unsigned int frame_left;
> @@ -413,8 +436,20 @@ static const struct v4l2_dv_timings_cap aspeed_video_timings_cap = {
>  	},
>  };
>  
> +static const char * const compress_mode_str[] = {"DCT Only",
> +	"DCT VQ mix 2-color", "DCT VQ mix 4-color"};

It's easier to read if it is written like this:

static const char * const compress_mode_str[] = {
	"DCT Only",
	"DCT VQ Mix 2-Color",
	"DCT VQ Mix 4-Color"};

Also capitalize 'Mix' and 'Color' (as done above).

> +static const char * const format_str[] = {"Standard JPEG",
> +	"Aspeed JPEG", "Partial JPEG"};

Same here: split with one entry per line.

For both arrays I'd also change _str to _ctrl_menu to clearly indicate that these
are used for control menus.

> +
>  static unsigned int debug;
>  
> +static bool aspeed_video_alloc_buf(struct aspeed_video *video,
> +				   struct aspeed_video_addr *addr,
> +				   unsigned int size);
> +
> +static void aspeed_video_free_buf(struct aspeed_video *video,
> +				  struct aspeed_video_addr *addr);
> +
>  static void aspeed_video_init_jpeg_table(u32 *table, bool yuv420)
>  {
>  	int i;
> @@ -502,6 +537,7 @@ static int aspeed_video_start_frame(struct aspeed_video *video)
>  	unsigned long flags;
>  	struct aspeed_video_buffer *buf;
>  	u32 seq_ctrl = aspeed_video_read(video, VE_SEQ_CTRL);
> +	bool bcd_buf_need = (video->format != VIDEO_FMT_STANDARD);
>  
>  	if (video->v4l2_input_status) {
>  		v4l2_warn(&video->v4l2_dev, "No signal; don't start frame\n");
> @@ -514,6 +550,20 @@ static int aspeed_video_start_frame(struct aspeed_video *video)
>  		return -EBUSY;
>  	}
>  
> +	if (bcd_buf_need && !video->bcd.size) {
> +		if (!aspeed_video_alloc_buf(video, &video->bcd,
> +					    VE_BCD_BUFF_SIZE)) {
> +			dev_err(video->dev, "Failed to allocate BCD buffer\n");
> +			dev_err(video->dev, "don't start frame\n");
> +			return -ENOMEM;
> +		}
> +		aspeed_video_write(video, VE_BCD_ADDR, video->bcd.dma);
> +		v4l2_dbg(1, debug, &video->v4l2_dev, "bcd addr(%#x) size(%d)\n",
> +			 video->bcd.dma, video->bcd.size);
> +	} else if (!bcd_buf_need && video->bcd.size) {
> +		aspeed_video_free_buf(video, &video->bcd);
> +	}
> +
>  	spin_lock_irqsave(&video->lock, flags);
>  	buf = list_first_entry_or_null(&video->buffers,
>  				       struct aspeed_video_buffer, link);
> @@ -610,6 +660,20 @@ static void aspeed_video_irq_res_change(struct aspeed_video *video, ulong delay)
>  	schedule_delayed_work(&video->res_work, delay);
>  }
>  
> +static void aspeed_video_swap_src_buf(struct aspeed_video *v)
> +{
> +	if (v->format == VIDEO_FMT_STANDARD)
> +		return;
> +
> +	if (v->sequence & 0x01) {
> +		aspeed_video_write(v, VE_SRC0_ADDR, v->srcs[1].dma);
> +		aspeed_video_write(v, VE_SRC1_ADDR, v->srcs[0].dma);
> +	} else {
> +		aspeed_video_write(v, VE_SRC0_ADDR, v->srcs[0].dma);
> +		aspeed_video_write(v, VE_SRC1_ADDR, v->srcs[1].dma);
> +	}
> +}
> +
>  /*
>   * Interrupts that we don't use but have to explicitly ignore because the
>   * hardware asserts them even when they're disabled in the VE_INTERRUPT_CTRL
> @@ -661,6 +725,7 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
>  
>  	if (sts & VE_INTERRUPT_COMP_COMPLETE) {
>  		struct aspeed_video_buffer *buf;
> +		bool empty = true;
>  		u32 frame_size = aspeed_video_read(video,
>  						   video->config.comp_size_read);
>  
> @@ -674,13 +739,23 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
>  		if (buf) {
>  			vb2_set_plane_payload(&buf->vb.vb2_buf, 0, frame_size);
>  
> -			if (!list_is_last(&buf->link, &video->buffers)) {
> +			/*
> +			 * partial_jpeg requires continuous update.
> +			 * On the contrary, standard jpeg can keep last buffer
> +			 * to always have the latest result.
> +			 */
> +			if (video->format == VIDEO_FMT_STANDARD &&
> +			    list_is_last(&buf->link, &video->buffers)) {
> +				empty = false;
> +				v4l2_warn(&video->v4l2_dev, "skip to keep last frame updated\n");
> +			} else {
>  				buf->vb.vb2_buf.timestamp = ktime_get_ns();
>  				buf->vb.sequence = video->sequence++;
>  				buf->vb.field = V4L2_FIELD_NONE;
>  				vb2_buffer_done(&buf->vb.vb2_buf,
>  						VB2_BUF_STATE_DONE);
>  				list_del(&buf->link);
> +				empty = list_empty(&video->buffers);
>  			}
>  		}
>  		spin_unlock(&video->lock);
> @@ -694,7 +769,10 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
>  		aspeed_video_write(video, VE_INTERRUPT_STATUS,
>  				   VE_INTERRUPT_COMP_COMPLETE);
>  		sts &= ~VE_INTERRUPT_COMP_COMPLETE;
> -		if (test_bit(VIDEO_STREAMING, &video->flags) && buf)
> +
> +		aspeed_video_swap_src_buf(video);
> +
> +		if (test_bit(VIDEO_STREAMING, &video->flags) && !empty)
>  			aspeed_video_start_frame(video);
>  	}
>  
> @@ -953,10 +1031,14 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>  				   FIELD_PREP(VE_TGS_FIRST, video->frame_top) |
>  				   FIELD_PREP(VE_TGS_LAST,
>  					      video->frame_bottom + 1));
> -		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_INT_DE);
> +		aspeed_video_update(video, VE_CTRL,
> +				    VE_CTRL_INT_DE | VE_CTRL_DIRECT_FETCH,
> +				    VE_CTRL_INT_DE);
>  	} else {
>  		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Direct Mode\n");
> -		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_DIRECT_FETCH);
> +		aspeed_video_update(video, VE_CTRL,
> +				    VE_CTRL_INT_DE | VE_CTRL_DIRECT_FETCH,
> +				    VE_CTRL_DIRECT_FETCH);
>  	}
>  
>  	size *= 4;
> @@ -991,35 +1073,71 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>  
>  static void aspeed_video_update_regs(struct aspeed_video *video)
>  {
> -	u32 comp_ctrl = VE_COMP_CTRL_RSVD |
> -		FIELD_PREP(VE_COMP_CTRL_DCT_LUM, video->jpeg_quality) |
> -		FIELD_PREP(VE_COMP_CTRL_DCT_CHR, video->jpeg_quality | 0x10);
> +	u8 jpeg_hq_quality = clamp((int)video->jpeg_hq_quality - 1, 0,
> +				   ASPEED_VIDEO_JPEG_NUM_QUALITIES - 1);
> +	u32 comp_ctrl =	FIELD_PREP(VE_COMP_CTRL_DCT_LUM, video->jpeg_quality) |
> +		FIELD_PREP(VE_COMP_CTRL_DCT_CHR, video->jpeg_quality | 0x10) |
> +		FIELD_PREP(VE_COMP_CTRL_EN_HQ, video->hq_mode) |
> +		FIELD_PREP(VE_COMP_CTRL_HQ_DCT_LUM, jpeg_hq_quality) |
> +		FIELD_PREP(VE_COMP_CTRL_HQ_DCT_CHR, jpeg_hq_quality |
> +			   0x10);
>  	u32 ctrl = 0;
> -	u32 seq_ctrl = video->config.jpeg_mode;
> +	u32 seq_ctrl = 0;
>  
> -	v4l2_dbg(1, debug, &video->v4l2_dev, "framerate(%d)\n",
> -		 video->frame_rate);
> -	v4l2_dbg(1, debug, &video->v4l2_dev, "subsample(%s)\n",
> +	v4l2_dbg(1, debug, &video->v4l2_dev, "framerate(%d)\n", video->frame_rate);
> +	v4l2_dbg(1, debug, &video->v4l2_dev, "jpeg format(%s) subsample(%s)\n",
> +		 format_str[video->format],
>  		 video->yuv420 ? "420" : "444");
> -	v4l2_dbg(1, debug, &video->v4l2_dev, "compression quality(%d)\n",
> -		 video->jpeg_quality);
> +	v4l2_dbg(1, debug, &video->v4l2_dev, "compression quality(%d) hq(%s) hq_quality(%d)\n",
> +		 video->jpeg_quality, video->hq_mode ? "on" : "off",
> +		 video->jpeg_hq_quality);
> +	v4l2_dbg(1, debug, &video->v4l2_dev, "compression mode(%s)\n",
> +		 compress_mode_str[video->compression_mode]);
> +
> +	if (video->format == VIDEO_FMT_ASPEED)
> +		aspeed_video_update(video, VE_BCD_CTRL, 0, VE_BCD_CTRL_EN_BCD);
> +	else
> +		aspeed_video_update(video, VE_BCD_CTRL, VE_BCD_CTRL_EN_BCD, 0);
>  
>  	if (video->frame_rate)
>  		ctrl |= FIELD_PREP(VE_CTRL_FRC, video->frame_rate);
>  
> +	if (video->format == VIDEO_FMT_STANDARD) {
> +		comp_ctrl &= ~FIELD_PREP(VE_COMP_CTRL_EN_HQ, video->hq_mode);
> +		seq_ctrl |= video->config.jpeg_mode;
> +	}
> +
>  	if (video->yuv420)
>  		seq_ctrl |= VE_SEQ_CTRL_YUV420;
>  
>  	if (video->jpeg.virt)
>  		aspeed_video_update_jpeg_table(video->jpeg.virt, video->yuv420);
>  
> +	if (video->config.version == 4) {
> +		switch (video->compression_mode) {
> +		case 0:	//DCT only
> +			comp_ctrl |= VE_COMP_CTRL_VQ_DCT_ONLY;
> +			break;
> +		case 1:	//DCT VQ mix 2-color
> +			comp_ctrl &= ~(VE_COMP_CTRL_VQ_4COLOR |
> +				     VE_COMP_CTRL_VQ_DCT_ONLY);
> +			break;
> +		case 2:	//DCT VQ mix 4-color
> +			comp_ctrl |= VE_COMP_CTRL_VQ_4COLOR;
> +			break;
> +		}
> +	}
> +
>  	/* Set control registers */
>  	aspeed_video_update(video, VE_SEQ_CTRL,
>  			    video->config.jpeg_mode | VE_SEQ_CTRL_YUV420,
>  			    seq_ctrl);
>  	aspeed_video_update(video, VE_CTRL, VE_CTRL_FRC, ctrl);
>  	aspeed_video_update(video, VE_COMP_CTRL,
> -			    VE_COMP_CTRL_DCT_LUM | VE_COMP_CTRL_DCT_CHR,
> +			    VE_COMP_CTRL_DCT_LUM | VE_COMP_CTRL_DCT_CHR |
> +			    VE_COMP_CTRL_EN_HQ | VE_COMP_CTRL_HQ_DCT_LUM |
> +			    VE_COMP_CTRL_HQ_DCT_CHR | VE_COMP_CTRL_VQ_4COLOR |
> +			    VE_COMP_CTRL_VQ_DCT_ONLY,
>  			    comp_ctrl);
>  }
>  
> @@ -1051,6 +1169,8 @@ static void aspeed_video_init_regs(struct aspeed_video *video)
>  
>  	/* Set mode detection defaults */
>  	aspeed_video_write(video, VE_MODE_DETECT, 0x22666500);
> +
> +	aspeed_video_write(video, VE_BCD_CTRL, 0);
>  }
>  
>  static void aspeed_video_start(struct aspeed_video *video)
> @@ -1084,6 +1204,9 @@ static void aspeed_video_stop(struct aspeed_video *video)
>  	if (video->srcs[1].size)
>  		aspeed_video_free_buf(video, &video->srcs[1]);
>  
> +	if (video->bcd.size)
> +		aspeed_video_free_buf(video, &video->bcd);
> +
>  	video->v4l2_input_status = V4L2_IN_ST_NO_SIGNAL;
>  	video->flags = 0;
>  }
> @@ -1102,10 +1225,12 @@ static int aspeed_video_querycap(struct file *file, void *fh,
>  static int aspeed_video_enum_format(struct file *file, void *fh,
>  				    struct v4l2_fmtdesc *f)
>  {
> +	struct aspeed_video *video = video_drvdata(file);
> +
>  	if (f->index)
>  		return -EINVAL;
>  
> -	f->pixelformat = V4L2_PIX_FMT_JPEG;
> +	f->pixelformat = video->pix_fmt.pixelformat;
>  
>  	return 0;
>  }
> @@ -1120,6 +1245,29 @@ static int aspeed_video_get_format(struct file *file, void *fh,
>  	return 0;
>  }
>  
> +static int aspeed_video_set_format(struct file *file, void *fh,
> +				   struct v4l2_format *f)
> +{
> +	struct aspeed_video *video = video_drvdata(file);
> +
> +	if (vb2_is_busy(&video->queue))
> +		return -EBUSY;
> +
> +	switch (f->fmt.pix.pixelformat) {
> +	case V4L2_PIX_FMT_JPEG:
> +		video->format = VIDEO_FMT_STANDARD;
> +		break;
> +	case V4L2_PIX_FMT_AJPG:
> +		video->format = VIDEO_FMT_ASPEED;
> +		break;
> +	default:
> +		return -EINVAL;
> +	}
> +	video->pix_fmt.pixelformat = f->fmt.pix.pixelformat;
> +
> +	return 0;
> +}
> +
>  static int aspeed_video_enum_input(struct file *file, void *fh,
>  				   struct v4l2_input *inp)
>  {
> @@ -1337,7 +1485,7 @@ static const struct v4l2_ioctl_ops aspeed_video_ioctl_ops = {
>  
>  	.vidioc_enum_fmt_vid_cap = aspeed_video_enum_format,
>  	.vidioc_g_fmt_vid_cap = aspeed_video_get_format,
> -	.vidioc_s_fmt_vid_cap = aspeed_video_get_format,
> +	.vidioc_s_fmt_vid_cap = aspeed_video_set_format,
>  	.vidioc_try_fmt_vid_cap = aspeed_video_get_format,
>  
>  	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> @@ -1386,6 +1534,24 @@ static int aspeed_video_set_ctrl(struct v4l2_ctrl *ctrl)
>  		if (test_bit(VIDEO_STREAMING, &video->flags))
>  			aspeed_video_update_regs(video);
>  		break;
> +	case V4L2_CID_ASPEED_COMPRESSION_MODE:
> +		if (video->config.version > 4)
> +			return -EINVAL;
> +
> +		video->compression_mode = ctrl->val;
> +		if (test_bit(VIDEO_STREAMING, &video->flags))
> +			aspeed_video_update_regs(video);
> +		break;
> +	case V4L2_CID_ASPEED_HQ_MODE:
> +		video->hq_mode = ctrl->val;
> +		if (test_bit(VIDEO_STREAMING, &video->flags))
> +			aspeed_video_update_regs(video);
> +		break;
> +	case V4L2_CID_ASPEED_HQ_JPEG_QUALITY:
> +		video->jpeg_hq_quality = ctrl->val;
> +		if (test_bit(VIDEO_STREAMING, &video->flags))
> +			aspeed_video_update_regs(video);
> +		break;
>  	default:
>  		return -EINVAL;
>  	}
> @@ -1397,6 +1563,39 @@ static const struct v4l2_ctrl_ops aspeed_video_ctrl_ops = {
>  	.s_ctrl = aspeed_video_set_ctrl,
>  };
>  
> +static const struct v4l2_ctrl_config aspeed_ctrl_compression_mode = {
> +	.ops = &aspeed_video_ctrl_ops,
> +	.id = V4L2_CID_ASPEED_COMPRESSION_MODE,
> +	.name = "Aspeed Compression Mode",
> +	.type = V4L2_CTRL_TYPE_INTEGER,
> +	.min = 0,
> +	.max = 2,
> +	.step = 1,
> +	.def = 0,
> +};
> +
> +static const struct v4l2_ctrl_config aspeed_ctrl_HQ_mode = {

Use lower case HQ, so: aspeed_ctrl_hq_mode (i.e.: no CamelCase)

> +	.ops = &aspeed_video_ctrl_ops,
> +	.id = V4L2_CID_ASPEED_HQ_MODE,
> +	.name = "Aspeed HQ Mode",
> +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> +	.min = false,
> +	.max = true,
> +	.step = 1,
> +	.def = false,
> +};
> +
> +static const struct v4l2_ctrl_config aspeed_ctrl_HQ_jpeg_quality = {

Ditto.

> +	.ops = &aspeed_video_ctrl_ops,
> +	.id = V4L2_CID_ASPEED_HQ_JPEG_QUALITY,
> +	.name = "Aspeed HQ Quality",
> +	.type = V4L2_CTRL_TYPE_INTEGER,
> +	.min = 1,
> +	.max = ASPEED_VIDEO_JPEG_NUM_QUALITIES,
> +	.step = 1,
> +	.def = 1,
> +};
> +
>  static void aspeed_video_resolution_work(struct work_struct *work)
>  {
>  	struct delayed_work *dwork = to_delayed_work(work);
> @@ -1619,7 +1818,6 @@ static int aspeed_video_debugfs_show(struct seq_file *s, void *data)
>  		   (v->perf.totaltime && v->sequence) ?
>  		   1000/(v->perf.totaltime/v->sequence) : 0);
>  
> -
>  	return 0;
>  }
>  
> @@ -1663,6 +1861,7 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
>  	struct v4l2_device *v4l2_dev = &video->v4l2_dev;
>  	struct vb2_queue *vbq = &video->queue;
>  	struct video_device *vdev = &video->vdev;
> +	struct v4l2_ctrl_handler *hdl = &video->ctrl_handler;
>  	int rc;
>  
>  	video->pix_fmt.pixelformat = V4L2_PIX_FMT_JPEG;
> @@ -1677,22 +1876,26 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
>  		return rc;
>  	}
>  
> -	v4l2_ctrl_handler_init(&video->ctrl_handler, 2);
> -	v4l2_ctrl_new_std(&video->ctrl_handler, &aspeed_video_ctrl_ops,
> -			  V4L2_CID_JPEG_COMPRESSION_QUALITY, 0,
> -			  ASPEED_VIDEO_JPEG_NUM_QUALITIES - 1, 1, 0);
> -	v4l2_ctrl_new_std_menu(&video->ctrl_handler, &aspeed_video_ctrl_ops,
> +	v4l2_ctrl_handler_init(hdl, 6);
> +	v4l2_ctrl_new_std(hdl, &aspeed_video_ctrl_ops,
> +			  V4L2_CID_JPEG_COMPRESSION_QUALITY, 1,
> +			  ASPEED_VIDEO_JPEG_NUM_QUALITIES, 1, 1);
> +	v4l2_ctrl_new_std_menu(hdl, &aspeed_video_ctrl_ops,
>  			       V4L2_CID_JPEG_CHROMA_SUBSAMPLING,
>  			       V4L2_JPEG_CHROMA_SUBSAMPLING_420, mask,
>  			       V4L2_JPEG_CHROMA_SUBSAMPLING_444);
> +	if (video->config.version == 4)
> +		v4l2_ctrl_new_custom(hdl, &aspeed_ctrl_compression_mode, NULL);
> +	v4l2_ctrl_new_custom(hdl, &aspeed_ctrl_HQ_mode, NULL);
> +	v4l2_ctrl_new_custom(hdl, &aspeed_ctrl_HQ_jpeg_quality, NULL);
>  
> -	rc = video->ctrl_handler.error;
> +	rc = hdl->error;
>  	if (rc) {
>  		dev_err(video->dev, "Failed to init controls: %d\n", rc);
>  		goto err_ctrl_init;
>  	}
>  
> -	v4l2_dev->ctrl_handler = &video->ctrl_handler;
> +	v4l2_dev->ctrl_handler = hdl;
>  
>  	vbq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
>  	vbq->io_modes = VB2_MMAP | VB2_READ | VB2_DMABUF;
> @@ -1833,6 +2036,7 @@ static int aspeed_video_probe(struct platform_device *pdev)
>  		return -ENOMEM;
>  
>  	video->frame_rate = 30;
> +	video->jpeg_hq_quality = 1;
>  	video->dev = &pdev->dev;
>  	spin_lock_init(&video->lock);
>  	mutex_init(&video->video_lock);
> diff --git a/include/uapi/linux/aspeed-video.h b/include/uapi/linux/aspeed-video.h
> new file mode 100644
> index 000000000000..117bfeaaa83a
> --- /dev/null
> +++ b/include/uapi/linux/aspeed-video.h
> @@ -0,0 +1,15 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +/*
> + * Copyright (C) 2021 ASPEED Technology Inc.
> + */
> +
> +#ifndef _UAPI_LINUX_ASPEED_VIDEO_H
> +#define _UAPI_LINUX_ASPEED_VIDEO_H
> +
> +#include <linux/v4l2-controls.h>
> +
> +#define V4L2_CID_ASPEED_COMPRESSION_MODE	(V4L2_CID_USER_ASPEED_BASE  + 1)
> +#define V4L2_CID_ASPEED_HQ_MODE			(V4L2_CID_USER_ASPEED_BASE  + 2)
> +#define V4L2_CID_ASPEED_HQ_JPEG_QUALITY		(V4L2_CID_USER_ASPEED_BASE  + 3)

Please document these controls here in detail!

This header is a good place to do this.

Regards,

	Hans

> +
> +#endif /* _UAPI_LINUX_ASPEED_VIDEO_H */
> 

Re: [PATCH v5 08/10] media: aspeed: Support aspeed mode to reduce compressed data

[Hans Verkuil]

On 18/11/2021 08:40, Jammy Huang wrote:
> aspeed supports differential jpeg format which only compress the parts
> which are changed. In this way, it reduces both the amount of data to be
> transferred by network and those to be decoded on the client side.
> 
> 4 new ctrls are added:

3 controls. This commit message is out of date!

> * Aspeed JPEG Format: to control aspeed's partial jpeg on/off
>   0: standard jpeg, 1: aspeed jpeg
> * Aspeed Compression Mode: to control aspeed's compression mode
>   0: DCT Only, 1: DCT VQ mix 2-color, 2: DCT VQ mix 4-color
>   This is AST2400 only. It will adapt JPEG or VQ encoding method according
>   to the context automatically.
> * Aspeed HQ Mode: to control aspeed's high quality(2-pass) compression mode
>   This only works with yuv444 subsampling.
> * Aspeed HQ Quality: to control the quality of aspeed's HQ mode
>   only useful if Aspeed HQ mode is enabled
> 
> Aspeed JPEG Format requires an additional buffer, called bcd, to store
> the information about which macro block in the new frame is different
> from the previous one.
> 
> To have bcd correctly working, we need to swap the buffers for src0/1 to
> make src1 refer to previous frame and src0 to the coming new frame.
> 
> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
> ---
> v5:
>   - use runtime configuration to handle differences between soc
>   - add aspeed_video_swap_src_buf().
> v4
>   - add include/uapi/linux/aspeed-video.h for V4L2_CID_ASPEED_XXX
>   - add aspeed_video_set_format
>   - remove aspeed_ctrl_format
>   - add variable, format, to replace partial_jpeg
>   - modify aspeed_ctrl_HQ_jpeg_quality's range
>   - reduce bcd buffer size
>   - update commit message
> v3:
>   - 'compression mode' only available for G4
> v2:
>   - update commit message
>   - use v4l2_xxx log rather than self-defined macro
> ---
>  drivers/media/platform/aspeed-video.c | 252 +++++++++++++++++++++++---
>  include/uapi/linux/aspeed-video.h     |  15 ++
>  2 files changed, 243 insertions(+), 24 deletions(-)
>  create mode 100644 include/uapi/linux/aspeed-video.h
> 
> diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
> index ba8ee82b38c3..419825ed7737 100644
> --- a/drivers/media/platform/aspeed-video.c
> +++ b/drivers/media/platform/aspeed-video.c
> @@ -31,6 +31,7 @@
>  #include <media/v4l2-ioctl.h>
>  #include <media/videobuf2-dma-contig.h>
>  #include <linux/videodev2.h>
> +#include <uapi/linux/aspeed-video.h>
>  
>  #define DEVICE_NAME			"aspeed-video"
>  
> @@ -55,6 +56,7 @@
>  
>  #define VE_MAX_SRC_BUFFER_SIZE		0x8ca000 /* 1920 * 1200, 32bpp */
>  #define VE_JPEG_HEADER_SIZE		0x006000 /* 512 * 12 * 4 */
> +#define VE_BCD_BUFF_SIZE		0x9000 /* (1920/8) * (1200/8) */
>  
>  #define VE_PROTECTION_KEY		0x000
>  #define  VE_PROTECTION_KEY_UNLOCK	0x1a038aa8
> @@ -105,6 +107,13 @@
>  #define VE_SCALING_FILTER2		0x020
>  #define VE_SCALING_FILTER3		0x024
>  
> +#define VE_BCD_CTRL			0x02C
> +#define  VE_BCD_CTRL_EN_BCD		BIT(0)
> +#define  VE_BCD_CTRL_EN_ABCD		BIT(1)
> +#define  VE_BCD_CTRL_EN_CB		BIT(2)
> +#define  VE_BCD_CTRL_THR		GENMASK(23, 16)
> +#define  VE_BCD_CTRL_ABCD_THR		GENMASK(31, 24)
> +
>  #define VE_CAP_WINDOW			0x030
>  #define VE_COMP_WINDOW			0x034
>  #define VE_COMP_PROC_OFFSET		0x038
> @@ -113,6 +122,7 @@
>  #define VE_SRC0_ADDR			0x044
>  #define VE_SRC_SCANLINE_OFFSET		0x048
>  #define VE_SRC1_ADDR			0x04c
> +#define VE_BCD_ADDR			0x050
>  #define VE_COMP_ADDR			0x054
>  
>  #define VE_STREAM_BUF_SIZE		0x058
> @@ -133,6 +143,8 @@
>  #define  VE_COMP_CTRL_HQ_DCT_CHR	GENMASK(26, 22)
>  #define  VE_COMP_CTRL_HQ_DCT_LUM	GENMASK(31, 27)
>  
> +#define VE_CB_ADDR			0x06C
> +
>  #define AST2400_VE_COMP_SIZE_READ_BACK	0x078
>  #define AST2600_VE_COMP_SIZE_READ_BACK	0x084
>  
> @@ -191,6 +203,12 @@ enum {
>  	VIDEO_CLOCKS_ON,
>  };
>  
> +enum aspeed_video_format {
> +	VIDEO_FMT_STANDARD = 0,
> +	VIDEO_FMT_ASPEED,
> +	VIDEO_FMT_MAX = VIDEO_FMT_ASPEED
> +};
> +
>  struct aspeed_video_addr {
>  	unsigned int size;
>  	dma_addr_t dma;
> @@ -241,10 +259,15 @@ struct aspeed_video {
>  	unsigned int max_compressed_size;
>  	struct aspeed_video_addr srcs[2];
>  	struct aspeed_video_addr jpeg;
> +	struct aspeed_video_addr bcd;
>  
>  	bool yuv420;
> +	enum aspeed_video_format format;
> +	bool hq_mode;
>  	unsigned int frame_rate;
>  	unsigned int jpeg_quality;
> +	unsigned int jpeg_hq_quality;
> +	unsigned int compression_mode;
>  
>  	unsigned int frame_bottom;
>  	unsigned int frame_left;
> @@ -413,8 +436,20 @@ static const struct v4l2_dv_timings_cap aspeed_video_timings_cap = {
>  	},
>  };
>  
> +static const char * const compress_mode_str[] = {"DCT Only",
> +	"DCT VQ mix 2-color", "DCT VQ mix 4-color"};

It's easier to read if it is written like this:

static const char * const compress_mode_str[] = {
	"DCT Only",
	"DCT VQ Mix 2-Color",
	"DCT VQ Mix 4-Color"};

Also capitalize 'Mix' and 'Color' (as done above).

> +static const char * const format_str[] = {"Standard JPEG",
> +	"Aspeed JPEG", "Partial JPEG"};

Same here: split with one entry per line.

For both arrays I'd also change _str to _ctrl_menu to clearly indicate that these
are used for control menus.

> +
>  static unsigned int debug;
>  
> +static bool aspeed_video_alloc_buf(struct aspeed_video *video,
> +				   struct aspeed_video_addr *addr,
> +				   unsigned int size);
> +
> +static void aspeed_video_free_buf(struct aspeed_video *video,
> +				  struct aspeed_video_addr *addr);
> +
>  static void aspeed_video_init_jpeg_table(u32 *table, bool yuv420)
>  {
>  	int i;
> @@ -502,6 +537,7 @@ static int aspeed_video_start_frame(struct aspeed_video *video)
>  	unsigned long flags;
>  	struct aspeed_video_buffer *buf;
>  	u32 seq_ctrl = aspeed_video_read(video, VE_SEQ_CTRL);
> +	bool bcd_buf_need = (video->format != VIDEO_FMT_STANDARD);
>  
>  	if (video->v4l2_input_status) {
>  		v4l2_warn(&video->v4l2_dev, "No signal; don't start frame\n");
> @@ -514,6 +550,20 @@ static int aspeed_video_start_frame(struct aspeed_video *video)
>  		return -EBUSY;
>  	}
>  
> +	if (bcd_buf_need && !video->bcd.size) {
> +		if (!aspeed_video_alloc_buf(video, &video->bcd,
> +					    VE_BCD_BUFF_SIZE)) {
> +			dev_err(video->dev, "Failed to allocate BCD buffer\n");
> +			dev_err(video->dev, "don't start frame\n");
> +			return -ENOMEM;
> +		}
> +		aspeed_video_write(video, VE_BCD_ADDR, video->bcd.dma);
> +		v4l2_dbg(1, debug, &video->v4l2_dev, "bcd addr(%#x) size(%d)\n",
> +			 video->bcd.dma, video->bcd.size);
> +	} else if (!bcd_buf_need && video->bcd.size) {
> +		aspeed_video_free_buf(video, &video->bcd);
> +	}
> +
>  	spin_lock_irqsave(&video->lock, flags);
>  	buf = list_first_entry_or_null(&video->buffers,
>  				       struct aspeed_video_buffer, link);
> @@ -610,6 +660,20 @@ static void aspeed_video_irq_res_change(struct aspeed_video *video, ulong delay)
>  	schedule_delayed_work(&video->res_work, delay);
>  }
>  
> +static void aspeed_video_swap_src_buf(struct aspeed_video *v)
> +{
> +	if (v->format == VIDEO_FMT_STANDARD)
> +		return;
> +
> +	if (v->sequence & 0x01) {
> +		aspeed_video_write(v, VE_SRC0_ADDR, v->srcs[1].dma);
> +		aspeed_video_write(v, VE_SRC1_ADDR, v->srcs[0].dma);
> +	} else {
> +		aspeed_video_write(v, VE_SRC0_ADDR, v->srcs[0].dma);
> +		aspeed_video_write(v, VE_SRC1_ADDR, v->srcs[1].dma);
> +	}
> +}
> +
>  /*
>   * Interrupts that we don't use but have to explicitly ignore because the
>   * hardware asserts them even when they're disabled in the VE_INTERRUPT_CTRL
> @@ -661,6 +725,7 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
>  
>  	if (sts & VE_INTERRUPT_COMP_COMPLETE) {
>  		struct aspeed_video_buffer *buf;
> +		bool empty = true;
>  		u32 frame_size = aspeed_video_read(video,
>  						   video->config.comp_size_read);
>  
> @@ -674,13 +739,23 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
>  		if (buf) {
>  			vb2_set_plane_payload(&buf->vb.vb2_buf, 0, frame_size);
>  
> -			if (!list_is_last(&buf->link, &video->buffers)) {
> +			/*
> +			 * partial_jpeg requires continuous update.
> +			 * On the contrary, standard jpeg can keep last buffer
> +			 * to always have the latest result.
> +			 */
> +			if (video->format == VIDEO_FMT_STANDARD &&
> +			    list_is_last(&buf->link, &video->buffers)) {
> +				empty = false;
> +				v4l2_warn(&video->v4l2_dev, "skip to keep last frame updated\n");
> +			} else {
>  				buf->vb.vb2_buf.timestamp = ktime_get_ns();
>  				buf->vb.sequence = video->sequence++;
>  				buf->vb.field = V4L2_FIELD_NONE;
>  				vb2_buffer_done(&buf->vb.vb2_buf,
>  						VB2_BUF_STATE_DONE);
>  				list_del(&buf->link);
> +				empty = list_empty(&video->buffers);
>  			}
>  		}
>  		spin_unlock(&video->lock);
> @@ -694,7 +769,10 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
>  		aspeed_video_write(video, VE_INTERRUPT_STATUS,
>  				   VE_INTERRUPT_COMP_COMPLETE);
>  		sts &= ~VE_INTERRUPT_COMP_COMPLETE;
> -		if (test_bit(VIDEO_STREAMING, &video->flags) && buf)
> +
> +		aspeed_video_swap_src_buf(video);
> +
> +		if (test_bit(VIDEO_STREAMING, &video->flags) && !empty)
>  			aspeed_video_start_frame(video);
>  	}
>  
> @@ -953,10 +1031,14 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>  				   FIELD_PREP(VE_TGS_FIRST, video->frame_top) |
>  				   FIELD_PREP(VE_TGS_LAST,
>  					      video->frame_bottom + 1));
> -		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_INT_DE);
> +		aspeed_video_update(video, VE_CTRL,
> +				    VE_CTRL_INT_DE | VE_CTRL_DIRECT_FETCH,
> +				    VE_CTRL_INT_DE);
>  	} else {
>  		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Direct Mode\n");
> -		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_DIRECT_FETCH);
> +		aspeed_video_update(video, VE_CTRL,
> +				    VE_CTRL_INT_DE | VE_CTRL_DIRECT_FETCH,
> +				    VE_CTRL_DIRECT_FETCH);
>  	}
>  
>  	size *= 4;
> @@ -991,35 +1073,71 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
>  
>  static void aspeed_video_update_regs(struct aspeed_video *video)
>  {
> -	u32 comp_ctrl = VE_COMP_CTRL_RSVD |
> -		FIELD_PREP(VE_COMP_CTRL_DCT_LUM, video->jpeg_quality) |
> -		FIELD_PREP(VE_COMP_CTRL_DCT_CHR, video->jpeg_quality | 0x10);
> +	u8 jpeg_hq_quality = clamp((int)video->jpeg_hq_quality - 1, 0,
> +				   ASPEED_VIDEO_JPEG_NUM_QUALITIES - 1);
> +	u32 comp_ctrl =	FIELD_PREP(VE_COMP_CTRL_DCT_LUM, video->jpeg_quality) |
> +		FIELD_PREP(VE_COMP_CTRL_DCT_CHR, video->jpeg_quality | 0x10) |
> +		FIELD_PREP(VE_COMP_CTRL_EN_HQ, video->hq_mode) |
> +		FIELD_PREP(VE_COMP_CTRL_HQ_DCT_LUM, jpeg_hq_quality) |
> +		FIELD_PREP(VE_COMP_CTRL_HQ_DCT_CHR, jpeg_hq_quality |
> +			   0x10);
>  	u32 ctrl = 0;
> -	u32 seq_ctrl = video->config.jpeg_mode;
> +	u32 seq_ctrl = 0;
>  
> -	v4l2_dbg(1, debug, &video->v4l2_dev, "framerate(%d)\n",
> -		 video->frame_rate);
> -	v4l2_dbg(1, debug, &video->v4l2_dev, "subsample(%s)\n",
> +	v4l2_dbg(1, debug, &video->v4l2_dev, "framerate(%d)\n", video->frame_rate);
> +	v4l2_dbg(1, debug, &video->v4l2_dev, "jpeg format(%s) subsample(%s)\n",
> +		 format_str[video->format],
>  		 video->yuv420 ? "420" : "444");
> -	v4l2_dbg(1, debug, &video->v4l2_dev, "compression quality(%d)\n",
> -		 video->jpeg_quality);
> +	v4l2_dbg(1, debug, &video->v4l2_dev, "compression quality(%d) hq(%s) hq_quality(%d)\n",
> +		 video->jpeg_quality, video->hq_mode ? "on" : "off",
> +		 video->jpeg_hq_quality);
> +	v4l2_dbg(1, debug, &video->v4l2_dev, "compression mode(%s)\n",
> +		 compress_mode_str[video->compression_mode]);
> +
> +	if (video->format == VIDEO_FMT_ASPEED)
> +		aspeed_video_update(video, VE_BCD_CTRL, 0, VE_BCD_CTRL_EN_BCD);
> +	else
> +		aspeed_video_update(video, VE_BCD_CTRL, VE_BCD_CTRL_EN_BCD, 0);
>  
>  	if (video->frame_rate)
>  		ctrl |= FIELD_PREP(VE_CTRL_FRC, video->frame_rate);
>  
> +	if (video->format == VIDEO_FMT_STANDARD) {
> +		comp_ctrl &= ~FIELD_PREP(VE_COMP_CTRL_EN_HQ, video->hq_mode);
> +		seq_ctrl |= video->config.jpeg_mode;
> +	}
> +
>  	if (video->yuv420)
>  		seq_ctrl |= VE_SEQ_CTRL_YUV420;
>  
>  	if (video->jpeg.virt)
>  		aspeed_video_update_jpeg_table(video->jpeg.virt, video->yuv420);
>  
> +	if (video->config.version == 4) {
> +		switch (video->compression_mode) {
> +		case 0:	//DCT only
> +			comp_ctrl |= VE_COMP_CTRL_VQ_DCT_ONLY;
> +			break;
> +		case 1:	//DCT VQ mix 2-color
> +			comp_ctrl &= ~(VE_COMP_CTRL_VQ_4COLOR |
> +				     VE_COMP_CTRL_VQ_DCT_ONLY);
> +			break;
> +		case 2:	//DCT VQ mix 4-color
> +			comp_ctrl |= VE_COMP_CTRL_VQ_4COLOR;
> +			break;
> +		}
> +	}
> +
>  	/* Set control registers */
>  	aspeed_video_update(video, VE_SEQ_CTRL,
>  			    video->config.jpeg_mode | VE_SEQ_CTRL_YUV420,
>  			    seq_ctrl);
>  	aspeed_video_update(video, VE_CTRL, VE_CTRL_FRC, ctrl);
>  	aspeed_video_update(video, VE_COMP_CTRL,
> -			    VE_COMP_CTRL_DCT_LUM | VE_COMP_CTRL_DCT_CHR,
> +			    VE_COMP_CTRL_DCT_LUM | VE_COMP_CTRL_DCT_CHR |
> +			    VE_COMP_CTRL_EN_HQ | VE_COMP_CTRL_HQ_DCT_LUM |
> +			    VE_COMP_CTRL_HQ_DCT_CHR | VE_COMP_CTRL_VQ_4COLOR |
> +			    VE_COMP_CTRL_VQ_DCT_ONLY,
>  			    comp_ctrl);
>  }
>  
> @@ -1051,6 +1169,8 @@ static void aspeed_video_init_regs(struct aspeed_video *video)
>  
>  	/* Set mode detection defaults */
>  	aspeed_video_write(video, VE_MODE_DETECT, 0x22666500);
> +
> +	aspeed_video_write(video, VE_BCD_CTRL, 0);
>  }
>  
>  static void aspeed_video_start(struct aspeed_video *video)
> @@ -1084,6 +1204,9 @@ static void aspeed_video_stop(struct aspeed_video *video)
>  	if (video->srcs[1].size)
>  		aspeed_video_free_buf(video, &video->srcs[1]);
>  
> +	if (video->bcd.size)
> +		aspeed_video_free_buf(video, &video->bcd);
> +
>  	video->v4l2_input_status = V4L2_IN_ST_NO_SIGNAL;
>  	video->flags = 0;
>  }
> @@ -1102,10 +1225,12 @@ static int aspeed_video_querycap(struct file *file, void *fh,
>  static int aspeed_video_enum_format(struct file *file, void *fh,
>  				    struct v4l2_fmtdesc *f)
>  {
> +	struct aspeed_video *video = video_drvdata(file);
> +
>  	if (f->index)
>  		return -EINVAL;
>  
> -	f->pixelformat = V4L2_PIX_FMT_JPEG;
> +	f->pixelformat = video->pix_fmt.pixelformat;
>  
>  	return 0;
>  }
> @@ -1120,6 +1245,29 @@ static int aspeed_video_get_format(struct file *file, void *fh,
>  	return 0;
>  }
>  
> +static int aspeed_video_set_format(struct file *file, void *fh,
> +				   struct v4l2_format *f)
> +{
> +	struct aspeed_video *video = video_drvdata(file);
> +
> +	if (vb2_is_busy(&video->queue))
> +		return -EBUSY;
> +
> +	switch (f->fmt.pix.pixelformat) {
> +	case V4L2_PIX_FMT_JPEG:
> +		video->format = VIDEO_FMT_STANDARD;
> +		break;
> +	case V4L2_PIX_FMT_AJPG:
> +		video->format = VIDEO_FMT_ASPEED;
> +		break;
> +	default:
> +		return -EINVAL;
> +	}
> +	video->pix_fmt.pixelformat = f->fmt.pix.pixelformat;
> +
> +	return 0;
> +}
> +
>  static int aspeed_video_enum_input(struct file *file, void *fh,
>  				   struct v4l2_input *inp)
>  {
> @@ -1337,7 +1485,7 @@ static const struct v4l2_ioctl_ops aspeed_video_ioctl_ops = {
>  
>  	.vidioc_enum_fmt_vid_cap = aspeed_video_enum_format,
>  	.vidioc_g_fmt_vid_cap = aspeed_video_get_format,
> -	.vidioc_s_fmt_vid_cap = aspeed_video_get_format,
> +	.vidioc_s_fmt_vid_cap = aspeed_video_set_format,
>  	.vidioc_try_fmt_vid_cap = aspeed_video_get_format,
>  
>  	.vidioc_reqbufs = vb2_ioctl_reqbufs,
> @@ -1386,6 +1534,24 @@ static int aspeed_video_set_ctrl(struct v4l2_ctrl *ctrl)
>  		if (test_bit(VIDEO_STREAMING, &video->flags))
>  			aspeed_video_update_regs(video);
>  		break;
> +	case V4L2_CID_ASPEED_COMPRESSION_MODE:
> +		if (video->config.version > 4)
> +			return -EINVAL;
> +
> +		video->compression_mode = ctrl->val;
> +		if (test_bit(VIDEO_STREAMING, &video->flags))
> +			aspeed_video_update_regs(video);
> +		break;
> +	case V4L2_CID_ASPEED_HQ_MODE:
> +		video->hq_mode = ctrl->val;
> +		if (test_bit(VIDEO_STREAMING, &video->flags))
> +			aspeed_video_update_regs(video);
> +		break;
> +	case V4L2_CID_ASPEED_HQ_JPEG_QUALITY:
> +		video->jpeg_hq_quality = ctrl->val;
> +		if (test_bit(VIDEO_STREAMING, &video->flags))
> +			aspeed_video_update_regs(video);
> +		break;
>  	default:
>  		return -EINVAL;
>  	}
> @@ -1397,6 +1563,39 @@ static const struct v4l2_ctrl_ops aspeed_video_ctrl_ops = {
>  	.s_ctrl = aspeed_video_set_ctrl,
>  };
>  
> +static const struct v4l2_ctrl_config aspeed_ctrl_compression_mode = {
> +	.ops = &aspeed_video_ctrl_ops,
> +	.id = V4L2_CID_ASPEED_COMPRESSION_MODE,
> +	.name = "Aspeed Compression Mode",
> +	.type = V4L2_CTRL_TYPE_INTEGER,
> +	.min = 0,
> +	.max = 2,
> +	.step = 1,
> +	.def = 0,
> +};
> +
> +static const struct v4l2_ctrl_config aspeed_ctrl_HQ_mode = {

Use lower case HQ, so: aspeed_ctrl_hq_mode (i.e.: no CamelCase)

> +	.ops = &aspeed_video_ctrl_ops,
> +	.id = V4L2_CID_ASPEED_HQ_MODE,
> +	.name = "Aspeed HQ Mode",
> +	.type = V4L2_CTRL_TYPE_BOOLEAN,
> +	.min = false,
> +	.max = true,
> +	.step = 1,
> +	.def = false,
> +};
> +
> +static const struct v4l2_ctrl_config aspeed_ctrl_HQ_jpeg_quality = {

Ditto.

> +	.ops = &aspeed_video_ctrl_ops,
> +	.id = V4L2_CID_ASPEED_HQ_JPEG_QUALITY,
> +	.name = "Aspeed HQ Quality",
> +	.type = V4L2_CTRL_TYPE_INTEGER,
> +	.min = 1,
> +	.max = ASPEED_VIDEO_JPEG_NUM_QUALITIES,
> +	.step = 1,
> +	.def = 1,
> +};
> +
>  static void aspeed_video_resolution_work(struct work_struct *work)
>  {
>  	struct delayed_work *dwork = to_delayed_work(work);
> @@ -1619,7 +1818,6 @@ static int aspeed_video_debugfs_show(struct seq_file *s, void *data)
>  		   (v->perf.totaltime && v->sequence) ?
>  		   1000/(v->perf.totaltime/v->sequence) : 0);
>  
> -
>  	return 0;
>  }
>  
> @@ -1663,6 +1861,7 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
>  	struct v4l2_device *v4l2_dev = &video->v4l2_dev;
>  	struct vb2_queue *vbq = &video->queue;
>  	struct video_device *vdev = &video->vdev;
> +	struct v4l2_ctrl_handler *hdl = &video->ctrl_handler;
>  	int rc;
>  
>  	video->pix_fmt.pixelformat = V4L2_PIX_FMT_JPEG;
> @@ -1677,22 +1876,26 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
>  		return rc;
>  	}
>  
> -	v4l2_ctrl_handler_init(&video->ctrl_handler, 2);
> -	v4l2_ctrl_new_std(&video->ctrl_handler, &aspeed_video_ctrl_ops,
> -			  V4L2_CID_JPEG_COMPRESSION_QUALITY, 0,
> -			  ASPEED_VIDEO_JPEG_NUM_QUALITIES - 1, 1, 0);
> -	v4l2_ctrl_new_std_menu(&video->ctrl_handler, &aspeed_video_ctrl_ops,
> +	v4l2_ctrl_handler_init(hdl, 6);
> +	v4l2_ctrl_new_std(hdl, &aspeed_video_ctrl_ops,
> +			  V4L2_CID_JPEG_COMPRESSION_QUALITY, 1,
> +			  ASPEED_VIDEO_JPEG_NUM_QUALITIES, 1, 1);
> +	v4l2_ctrl_new_std_menu(hdl, &aspeed_video_ctrl_ops,
>  			       V4L2_CID_JPEG_CHROMA_SUBSAMPLING,
>  			       V4L2_JPEG_CHROMA_SUBSAMPLING_420, mask,
>  			       V4L2_JPEG_CHROMA_SUBSAMPLING_444);
> +	if (video->config.version == 4)
> +		v4l2_ctrl_new_custom(hdl, &aspeed_ctrl_compression_mode, NULL);
> +	v4l2_ctrl_new_custom(hdl, &aspeed_ctrl_HQ_mode, NULL);
> +	v4l2_ctrl_new_custom(hdl, &aspeed_ctrl_HQ_jpeg_quality, NULL);
>  
> -	rc = video->ctrl_handler.error;
> +	rc = hdl->error;
>  	if (rc) {
>  		dev_err(video->dev, "Failed to init controls: %d\n", rc);
>  		goto err_ctrl_init;
>  	}
>  
> -	v4l2_dev->ctrl_handler = &video->ctrl_handler;
> +	v4l2_dev->ctrl_handler = hdl;
>  
>  	vbq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
>  	vbq->io_modes = VB2_MMAP | VB2_READ | VB2_DMABUF;
> @@ -1833,6 +2036,7 @@ static int aspeed_video_probe(struct platform_device *pdev)
>  		return -ENOMEM;
>  
>  	video->frame_rate = 30;
> +	video->jpeg_hq_quality = 1;
>  	video->dev = &pdev->dev;
>  	spin_lock_init(&video->lock);
>  	mutex_init(&video->video_lock);
> diff --git a/include/uapi/linux/aspeed-video.h b/include/uapi/linux/aspeed-video.h
> new file mode 100644
> index 000000000000..117bfeaaa83a
> --- /dev/null
> +++ b/include/uapi/linux/aspeed-video.h
> @@ -0,0 +1,15 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +/*
> + * Copyright (C) 2021 ASPEED Technology Inc.
> + */
> +
> +#ifndef _UAPI_LINUX_ASPEED_VIDEO_H
> +#define _UAPI_LINUX_ASPEED_VIDEO_H
> +
> +#include <linux/v4l2-controls.h>
> +
> +#define V4L2_CID_ASPEED_COMPRESSION_MODE	(V4L2_CID_USER_ASPEED_BASE  + 1)
> +#define V4L2_CID_ASPEED_HQ_MODE			(V4L2_CID_USER_ASPEED_BASE  + 2)
> +#define V4L2_CID_ASPEED_HQ_JPEG_QUALITY		(V4L2_CID_USER_ASPEED_BASE  + 3)

Please document these controls here in detail!

This header is a good place to do this.

Regards,

	Hans

> +
> +#endif /* _UAPI_LINUX_ASPEED_VIDEO_H */
> 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

Hi Sakari,

On 2021/11/23 下午 06:23, Sakari Ailus wrote:
> Hi Jammy,
>
> On Fri, Nov 19, 2021 at 10:02:40AM +0800, Jammy Huang wrote:
>> Hi Sakari,
>>
>> On 2021/11/18 下午 08:33, Sakari Ailus wrote:
>>> Hi Jammy,
>>>
>>> On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
>>>> This introduces support for the Aspeed JPEG format, where the new frame
>>>> can refer to previous frame to reduce the amount of compressed data. The
>>>> concept is similar to I/P frame of video compression. I will compare the
>>>> new frame with previous one to decide which macroblock's data is
>>>> changed, and only the changed macroblocks will be compressed.
>>>>
>>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>>> which is generally adapted for remote KVM.
>>>>
>>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>>> ---
>>>> v5:
>>>>     - no update
>>>> v4:
>>>>     - new
>>>> ---
>>>>    Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>>>>    drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>>>>    include/uapi/linux/videodev2.h                   |  1 +
>>>>    3 files changed, 14 insertions(+)
>>>>
>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> index b2cd155e691b..23c05063133d 100644
>>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> @@ -264,6 +264,18 @@ please make a proposal on the linux-media mailing list.
>>>>    	of tiles, resulting in 32-aligned resolutions for the luminance plane
>>>>    	and 16-aligned resolutions for the chrominance plane (with 2x2
>>>>    	subsampling).
>>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>>> +
>>>> +      - ``V4L2_PIX_FMT_AJPG``
>>>> +      - 'AJPG'
>>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>>> +        which is generally adapted for remote KVM.
>>>> +        On each frame compression, I will compare the new frame with previous
>>>> +        one to decide which macroblock's data is changed, and only the changed
>>>> +        macroblocks will be compressed.
>>>> +
>>>> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
>>>> +        for more information.
>>> Is this datasheet publicly available? Do you have a URL?
>> Sorry, this datasheet is not publicly available.
>> Hans mentioned this as well in the discussion below:
>>
>> https://lkml.org/lkml/2021/11/10/101
> If the vendor documentation is not publicly available, you'll need to have
> the format documented here. Alternatively an open source implementation
> (e.g. LGPL) is presumably fine, too.
Please refer to 
https://github.com/AspeedTech-BMC/openbmc/releases/download/v07.02/SDK_User_Guide_v07.02.pdf.
At Page 132, 'Video stream data format – ASPEED mode compression' 
describes the data format.
Is above information  enough??

Besides, I have already had openbmc's kvm support this aspeed format.
The source code of the implementation will be released after performance 
tuning finished.

-- 
Best Regards
Jammy

Re: [PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

Hi Sakari,

On 2021/11/23 下午 06:23, Sakari Ailus wrote:
> Hi Jammy,
>
> On Fri, Nov 19, 2021 at 10:02:40AM +0800, Jammy Huang wrote:
>> Hi Sakari,
>>
>> On 2021/11/18 下午 08:33, Sakari Ailus wrote:
>>> Hi Jammy,
>>>
>>> On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
>>>> This introduces support for the Aspeed JPEG format, where the new frame
>>>> can refer to previous frame to reduce the amount of compressed data. The
>>>> concept is similar to I/P frame of video compression. I will compare the
>>>> new frame with previous one to decide which macroblock's data is
>>>> changed, and only the changed macroblocks will be compressed.
>>>>
>>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>>> which is generally adapted for remote KVM.
>>>>
>>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>>> ---
>>>> v5:
>>>>     - no update
>>>> v4:
>>>>     - new
>>>> ---
>>>>    Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>>>>    drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>>>>    include/uapi/linux/videodev2.h                   |  1 +
>>>>    3 files changed, 14 insertions(+)
>>>>
>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> index b2cd155e691b..23c05063133d 100644
>>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> @@ -264,6 +264,18 @@ please make a proposal on the linux-media mailing list.
>>>>    	of tiles, resulting in 32-aligned resolutions for the luminance plane
>>>>    	and 16-aligned resolutions for the chrominance plane (with 2x2
>>>>    	subsampling).
>>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>>> +
>>>> +      - ``V4L2_PIX_FMT_AJPG``
>>>> +      - 'AJPG'
>>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>>> +        which is generally adapted for remote KVM.
>>>> +        On each frame compression, I will compare the new frame with previous
>>>> +        one to decide which macroblock's data is changed, and only the changed
>>>> +        macroblocks will be compressed.
>>>> +
>>>> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
>>>> +        for more information.
>>> Is this datasheet publicly available? Do you have a URL?
>> Sorry, this datasheet is not publicly available.
>> Hans mentioned this as well in the discussion below:
>>
>> https://lkml.org/lkml/2021/11/10/101
> If the vendor documentation is not publicly available, you'll need to have
> the format documented here. Alternatively an open source implementation
> (e.g. LGPL) is presumably fine, too.
Please refer to 
https://github.com/AspeedTech-BMC/openbmc/releases/download/v07.02/SDK_User_Guide_v07.02.pdf.
At Page 132, 'Video stream data format – ASPEED mode compression' 
describes the data format.
Is above information  enough??

Besides, I have already had openbmc's kvm support this aspeed format.
The source code of the implementation will be released after performance 
tuning finished.

-- 
Best Regards
Jammy

Re: [PATCH v5 05/10] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

Hi Sakari,

On 2021/11/23 下午 06:23, Sakari Ailus wrote:
> Hi Jammy,
>
> On Fri, Nov 19, 2021 at 10:02:40AM +0800, Jammy Huang wrote:
>> Hi Sakari,
>>
>> On 2021/11/18 下午 08:33, Sakari Ailus wrote:
>>> Hi Jammy,
>>>
>>> On Thu, Nov 18, 2021 at 03:40:26PM +0800, Jammy Huang wrote:
>>>> This introduces support for the Aspeed JPEG format, where the new frame
>>>> can refer to previous frame to reduce the amount of compressed data. The
>>>> concept is similar to I/P frame of video compression. I will compare the
>>>> new frame with previous one to decide which macroblock's data is
>>>> changed, and only the changed macroblocks will be compressed.
>>>>
>>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>>> which is generally adapted for remote KVM.
>>>>
>>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>>> ---
>>>> v5:
>>>>     - no update
>>>> v4:
>>>>     - new
>>>> ---
>>>>    Documentation/media/uapi/v4l/pixfmt-reserved.rst | 12 ++++++++++++
>>>>    drivers/media/v4l2-core/v4l2-ioctl.c             |  1 +
>>>>    include/uapi/linux/videodev2.h                   |  1 +
>>>>    3 files changed, 14 insertions(+)
>>>>
>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> index b2cd155e691b..23c05063133d 100644
>>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> @@ -264,6 +264,18 @@ please make a proposal on the linux-media mailing list.
>>>>    	of tiles, resulting in 32-aligned resolutions for the luminance plane
>>>>    	and 16-aligned resolutions for the chrominance plane (with 2x2
>>>>    	subsampling).
>>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>>> +
>>>> +      - ``V4L2_PIX_FMT_AJPG``
>>>> +      - 'AJPG'
>>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>>> +        which is generally adapted for remote KVM.
>>>> +        On each frame compression, I will compare the new frame with previous
>>>> +        one to decide which macroblock's data is changed, and only the changed
>>>> +        macroblocks will be compressed.
>>>> +
>>>> +        You could reference to chapter 36, Video Engine, of AST2600's datasheet
>>>> +        for more information.
>>> Is this datasheet publicly available? Do you have a URL?
>> Sorry, this datasheet is not publicly available.
>> Hans mentioned this as well in the discussion below:
>>
>> https://lkml.org/lkml/2021/11/10/101
> If the vendor documentation is not publicly available, you'll need to have
> the format documented here. Alternatively an open source implementation
> (e.g. LGPL) is presumably fine, too.
Please refer to 
https://github.com/AspeedTech-BMC/openbmc/releases/download/v07.02/SDK_User_Guide_v07.02.pdf.
At Page 132, 'Video stream data format – ASPEED mode compression' 
describes the data format.
Is above information  enough??

Besides, I have already had openbmc's kvm support this aspeed format.
The source code of the implementation will be released after performance 
tuning finished.

-- 
Best Regards
Jammy


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[Jammy Huang]

Hi Hans,

Please refer to Page 132, 'Video stream data format – ASPEED mode compression' of
https://github.com/AspeedTech-BMC/openbmc/releases/download/v07.02/SDK_User_Guide_v07.02.pdf

Is above information enough??
Or maybe we can postpone this series until the source code for this format ready.
I think I can accomplish it by the end of this year.

On 2021/11/25 下午 10:31, Hans Verkuil wrote:
> Hi Jammy,
>
> I suggest posting a v3 that moves the patches that deal with the
> Aspeed JPEG format to the end of the series. That way I can easily
> merge the patches up to that point, and the Aspeed JPEG support can
> be handled separately.
>
> I'm not sure if it can be merged without that format being documented,
> either in pixfmt-reserved.rst, by implementing support for it in
> libv4lconvert from v4l-utils, or by providing a URL to some other
> publicly available source code. We really don't like adding pixel
> formats that cannot be interpreted by someone without access to the
> datasheets.
>
> Regards,
>
> 	Hans
>
> On 18/11/2021 08:40, Jammy Huang wrote:
>> The aim of this series is to add aspeed-jpeg support for aspeed-video
>> driver.
>>
>> To achieve this major goal some refactors are included.
>>
>> In the last, debugfs information is also updated per this change.
>>
>> Changes in v5:
>>   - Use model data to tell different soc
>>
>> Changes in v4:
>>   - Add definition for the Aspeed JPEG format
>>   - Reserve controls for ASPEED
>>   - Use s_fmt to update format rather than new control
>>   - Update aspeed hq quality range, 1 ~ 12
>>
>>
>> Jammy Huang (10):
>>    media: aspeed: move err-handling together to the bottom
>>    media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log
>>    media: aspeed: add more debug log messages
>>    media: aspeed: refactor to gather format/compress settings
>>    media: v4l: Add definition for the Aspeed JPEG format
>>    media: v4l2-ctrls: Reserve controls for ASPEED
>>    media: aspeed: use model-data
>>    media: aspeed: Support aspeed mode to reduce compressed data
>>    media: aspeed: add comments and macro
>>    media: aspeed: Extend debug message
>>
>>   .../media/uapi/v4l/pixfmt-reserved.rst        |  12 +
>>   drivers/media/platform/aspeed-video.c         | 534 ++++++++++++++----
>>   drivers/media/v4l2-core/v4l2-ioctl.c          |   1 +
>>   include/uapi/linux/aspeed-video.h             |  15 +
>>   include/uapi/linux/v4l2-controls.h            |   5 +
>>   include/uapi/linux/videodev2.h                |   1 +
>>   6 files changed, 467 insertions(+), 101 deletions(-)
>>   create mode 100644 include/uapi/linux/aspeed-video.h
>>
-- 
Best Regards
Jammy

Re: [PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[Jammy Huang]

Hi Hans,

Please refer to Page 132, 'Video stream data format – ASPEED mode compression' of
https://github.com/AspeedTech-BMC/openbmc/releases/download/v07.02/SDK_User_Guide_v07.02.pdf

Is above information enough??
Or maybe we can postpone this series until the source code for this format ready.
I think I can accomplish it by the end of this year.

On 2021/11/25 下午 10:31, Hans Verkuil wrote:
> Hi Jammy,
>
> I suggest posting a v3 that moves the patches that deal with the
> Aspeed JPEG format to the end of the series. That way I can easily
> merge the patches up to that point, and the Aspeed JPEG support can
> be handled separately.
>
> I'm not sure if it can be merged without that format being documented,
> either in pixfmt-reserved.rst, by implementing support for it in
> libv4lconvert from v4l-utils, or by providing a URL to some other
> publicly available source code. We really don't like adding pixel
> formats that cannot be interpreted by someone without access to the
> datasheets.
>
> Regards,
>
> 	Hans
>
> On 18/11/2021 08:40, Jammy Huang wrote:
>> The aim of this series is to add aspeed-jpeg support for aspeed-video
>> driver.
>>
>> To achieve this major goal some refactors are included.
>>
>> In the last, debugfs information is also updated per this change.
>>
>> Changes in v5:
>>   - Use model data to tell different soc
>>
>> Changes in v4:
>>   - Add definition for the Aspeed JPEG format
>>   - Reserve controls for ASPEED
>>   - Use s_fmt to update format rather than new control
>>   - Update aspeed hq quality range, 1 ~ 12
>>
>>
>> Jammy Huang (10):
>>    media: aspeed: move err-handling together to the bottom
>>    media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log
>>    media: aspeed: add more debug log messages
>>    media: aspeed: refactor to gather format/compress settings
>>    media: v4l: Add definition for the Aspeed JPEG format
>>    media: v4l2-ctrls: Reserve controls for ASPEED
>>    media: aspeed: use model-data
>>    media: aspeed: Support aspeed mode to reduce compressed data
>>    media: aspeed: add comments and macro
>>    media: aspeed: Extend debug message
>>
>>   .../media/uapi/v4l/pixfmt-reserved.rst        |  12 +
>>   drivers/media/platform/aspeed-video.c         | 534 ++++++++++++++----
>>   drivers/media/v4l2-core/v4l2-ioctl.c          |   1 +
>>   include/uapi/linux/aspeed-video.h             |  15 +
>>   include/uapi/linux/v4l2-controls.h            |   5 +
>>   include/uapi/linux/videodev2.h                |   1 +
>>   6 files changed, 467 insertions(+), 101 deletions(-)
>>   create mode 100644 include/uapi/linux/aspeed-video.h
>>
-- 
Best Regards
Jammy


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[Hans Verkuil]

On 26/11/2021 09:17, Jammy Huang wrote:
> Hi Hans,
> 
> Please refer to Page 132, 'Video stream data format – ASPEED mode compression' of
> https://github.com/AspeedTech-BMC/openbmc/releases/download/v07.02/SDK_User_Guide_v07.02.pdf
> 
> Is above information enough??

Hmm, close. It would need some additional information, possibly documented in the source
code:

- What is the Frame Header? (the doc just says that it is different, but not what it actually is)
- What is the EOI code? Is it equal to 'FEND'? (not quite clear to me)

I think the documentation looks sufficient once those two items are addressed.

> Or maybe we can postpone this series until the source code for this format ready.
> I think I can accomplish it by the end of this year.

That's quite soon, so perhaps that would be the best approach. That way you also
know exactly what information is missing from the SDK User Guide, and if that is
documented, then this format can be used in the kernel.

But several of the patches in the series are independent of the new jpeg format,
so it might be useful to get those in first.

Regards,

	Hans

> 
> On 2021/11/25 下午 10:31, Hans Verkuil wrote:
>> Hi Jammy,
>>
>> I suggest posting a v3 that moves the patches that deal with the
>> Aspeed JPEG format to the end of the series. That way I can easily
>> merge the patches up to that point, and the Aspeed JPEG support can
>> be handled separately.
>>
>> I'm not sure if it can be merged without that format being documented,
>> either in pixfmt-reserved.rst, by implementing support for it in
>> libv4lconvert from v4l-utils, or by providing a URL to some other
>> publicly available source code. We really don't like adding pixel
>> formats that cannot be interpreted by someone without access to the
>> datasheets.
>>
>> Regards,
>>
>> 	Hans
>>
>> On 18/11/2021 08:40, Jammy Huang wrote:
>>> The aim of this series is to add aspeed-jpeg support for aspeed-video
>>> driver.
>>>
>>> To achieve this major goal some refactors are included.
>>>
>>> In the last, debugfs information is also updated per this change.
>>>
>>> Changes in v5:
>>>   - Use model data to tell different soc
>>>
>>> Changes in v4:
>>>   - Add definition for the Aspeed JPEG format
>>>   - Reserve controls for ASPEED
>>>   - Use s_fmt to update format rather than new control
>>>   - Update aspeed hq quality range, 1 ~ 12
>>>
>>>
>>> Jammy Huang (10):
>>>    media: aspeed: move err-handling together to the bottom
>>>    media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log
>>>    media: aspeed: add more debug log messages
>>>    media: aspeed: refactor to gather format/compress settings
>>>    media: v4l: Add definition for the Aspeed JPEG format
>>>    media: v4l2-ctrls: Reserve controls for ASPEED
>>>    media: aspeed: use model-data
>>>    media: aspeed: Support aspeed mode to reduce compressed data
>>>    media: aspeed: add comments and macro
>>>    media: aspeed: Extend debug message
>>>
>>>   .../media/uapi/v4l/pixfmt-reserved.rst        |  12 +
>>>   drivers/media/platform/aspeed-video.c         | 534 ++++++++++++++----
>>>   drivers/media/v4l2-core/v4l2-ioctl.c          |   1 +
>>>   include/uapi/linux/aspeed-video.h             |  15 +
>>>   include/uapi/linux/v4l2-controls.h            |   5 +
>>>   include/uapi/linux/videodev2.h                |   1 +
>>>   6 files changed, 467 insertions(+), 101 deletions(-)
>>>   create mode 100644 include/uapi/linux/aspeed-video.h
>>>

Re: [PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[Hans Verkuil]

On 26/11/2021 09:17, Jammy Huang wrote:
> Hi Hans,
> 
> Please refer to Page 132, 'Video stream data format – ASPEED mode compression' of
> https://github.com/AspeedTech-BMC/openbmc/releases/download/v07.02/SDK_User_Guide_v07.02.pdf
> 
> Is above information enough??

Hmm, close. It would need some additional information, possibly documented in the source
code:

- What is the Frame Header? (the doc just says that it is different, but not what it actually is)
- What is the EOI code? Is it equal to 'FEND'? (not quite clear to me)

I think the documentation looks sufficient once those two items are addressed.

> Or maybe we can postpone this series until the source code for this format ready.
> I think I can accomplish it by the end of this year.

That's quite soon, so perhaps that would be the best approach. That way you also
know exactly what information is missing from the SDK User Guide, and if that is
documented, then this format can be used in the kernel.

But several of the patches in the series are independent of the new jpeg format,
so it might be useful to get those in first.

Regards,

	Hans

> 
> On 2021/11/25 下午 10:31, Hans Verkuil wrote:
>> Hi Jammy,
>>
>> I suggest posting a v3 that moves the patches that deal with the
>> Aspeed JPEG format to the end of the series. That way I can easily
>> merge the patches up to that point, and the Aspeed JPEG support can
>> be handled separately.
>>
>> I'm not sure if it can be merged without that format being documented,
>> either in pixfmt-reserved.rst, by implementing support for it in
>> libv4lconvert from v4l-utils, or by providing a URL to some other
>> publicly available source code. We really don't like adding pixel
>> formats that cannot be interpreted by someone without access to the
>> datasheets.
>>
>> Regards,
>>
>> 	Hans
>>
>> On 18/11/2021 08:40, Jammy Huang wrote:
>>> The aim of this series is to add aspeed-jpeg support for aspeed-video
>>> driver.
>>>
>>> To achieve this major goal some refactors are included.
>>>
>>> In the last, debugfs information is also updated per this change.
>>>
>>> Changes in v5:
>>>   - Use model data to tell different soc
>>>
>>> Changes in v4:
>>>   - Add definition for the Aspeed JPEG format
>>>   - Reserve controls for ASPEED
>>>   - Use s_fmt to update format rather than new control
>>>   - Update aspeed hq quality range, 1 ~ 12
>>>
>>>
>>> Jammy Huang (10):
>>>    media: aspeed: move err-handling together to the bottom
>>>    media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log
>>>    media: aspeed: add more debug log messages
>>>    media: aspeed: refactor to gather format/compress settings
>>>    media: v4l: Add definition for the Aspeed JPEG format
>>>    media: v4l2-ctrls: Reserve controls for ASPEED
>>>    media: aspeed: use model-data
>>>    media: aspeed: Support aspeed mode to reduce compressed data
>>>    media: aspeed: add comments and macro
>>>    media: aspeed: Extend debug message
>>>
>>>   .../media/uapi/v4l/pixfmt-reserved.rst        |  12 +
>>>   drivers/media/platform/aspeed-video.c         | 534 ++++++++++++++----
>>>   drivers/media/v4l2-core/v4l2-ioctl.c          |   1 +
>>>   include/uapi/linux/aspeed-video.h             |  15 +
>>>   include/uapi/linux/v4l2-controls.h            |   5 +
>>>   include/uapi/linux/videodev2.h                |   1 +
>>>   6 files changed, 467 insertions(+), 101 deletions(-)
>>>   create mode 100644 include/uapi/linux/aspeed-video.h
>>>


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[Jammy Huang]

Hi Hans,

Thanks for your review and suggestions.

On 2021/11/26 下午 05:51, Hans Verkuil wrote:
> On 26/11/2021 09:17, Jammy Huang wrote:
>> Hi Hans,
>>
>> Please refer to Page 132, 'Video stream data format – ASPEED mode compression' of
>> https://github.com/AspeedTech-BMC/openbmc/releases/download/v07.02/SDK_User_Guide_v07.02.pdf
>>
>> Is above information enough??
> Hmm, close. It would need some additional information, possibly documented in the source
> code:
>
> - What is the Frame Header? (the doc just says that it is different, but not what it actually is)
> - What is the EOI code? Is it equal to 'FEND'? (not quite clear to me)
>
> I think the documentation looks sufficient once those two items are addressed.
Information you mentioned will be included in next revision, 
"SDK_User_Guide_v07.03.pdf".
The releases will be updated to 
https://github.com/AspeedTech-BMC/openbmc/releases
>> Or maybe we can postpone this series until the source code for this format ready.
>> I think I can accomplish it by the end of this year.
> That's quite soon, so perhaps that would be the best approach. That way you also
> know exactly what information is missing from the SDK User Guide, and if that is
> documented, then this format can be used in the kernel.
>
> But several of the patches in the series are independent of the new jpeg format,
> so it might be useful to get those in first.

Understood. For the patches below, I will resend except for patch 
5/6/8/10 which are closely
relative to aspeed jpeg format.

    1. media: aspeed: move err-handling together to the bottom
    2. media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log
    3. media: aspeed: add more debug log messages
    4. media: aspeed: refactor to gather format/compress settings
    5. media: v4l: Add definition for the Aspeed JPEG format
    6. media: v4l2-ctrls: Reserve controls for ASPEED
    7. media: aspeed: use model-data
    8. media: aspeed: Support aspeed mode to reduce compressed data
    9. media: aspeed: add comments and macro
    10.media: aspeed: Extend debug message



>
> Regards,
>
> 	Hans
>
>> On 2021/11/25 下午 10:31, Hans Verkuil wrote:
>>> Hi Jammy,
>>>
>>> I suggest posting a v3 that moves the patches that deal with the
>>> Aspeed JPEG format to the end of the series. That way I can easily
>>> merge the patches up to that point, and the Aspeed JPEG support can
>>> be handled separately.
>>>
>>> I'm not sure if it can be merged without that format being documented,
>>> either in pixfmt-reserved.rst, by implementing support for it in
>>> libv4lconvert from v4l-utils, or by providing a URL to some other
>>> publicly available source code. We really don't like adding pixel
>>> formats that cannot be interpreted by someone without access to the
>>> datasheets.
>>>
>>> Regards,
>>>
>>> 	Hans
>>>
>>> On 18/11/2021 08:40, Jammy Huang wrote:
>>>> The aim of this series is to add aspeed-jpeg support for aspeed-video
>>>> driver.
>>>>
>>>> To achieve this major goal some refactors are included.
>>>>
>>>> In the last, debugfs information is also updated per this change.
>>>>
>>>> Changes in v5:
>>>>    - Use model data to tell different soc
>>>>
>>>> Changes in v4:
>>>>    - Add definition for the Aspeed JPEG format
>>>>    - Reserve controls for ASPEED
>>>>    - Use s_fmt to update format rather than new control
>>>>    - Update aspeed hq quality range, 1 ~ 12
>>>>
>>>>
>>>> Jammy Huang (10):
>>>>     media: aspeed: move err-handling together to the bottom
>>>>     media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log
>>>>     media: aspeed: add more debug log messages
>>>>     media: aspeed: refactor to gather format/compress settings
>>>>     media: v4l: Add definition for the Aspeed JPEG format
>>>>     media: v4l2-ctrls: Reserve controls for ASPEED
>>>>     media: aspeed: use model-data
>>>>     media: aspeed: Support aspeed mode to reduce compressed data
>>>>     media: aspeed: add comments and macro
>>>>     media: aspeed: Extend debug message
>>>>
>>>>    .../media/uapi/v4l/pixfmt-reserved.rst        |  12 +
>>>>    drivers/media/platform/aspeed-video.c         | 534 ++++++++++++++----
>>>>    drivers/media/v4l2-core/v4l2-ioctl.c          |   1 +
>>>>    include/uapi/linux/aspeed-video.h             |  15 +
>>>>    include/uapi/linux/v4l2-controls.h            |   5 +
>>>>    include/uapi/linux/videodev2.h                |   1 +
>>>>    6 files changed, 467 insertions(+), 101 deletions(-)
>>>>    create mode 100644 include/uapi/linux/aspeed-video.h
>>>>
-- 
Best Regards
Jammy

Re: [PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[Jammy Huang]

Hi Hans,

Thanks for your review and suggestions.

On 2021/11/26 下午 05:51, Hans Verkuil wrote:
> On 26/11/2021 09:17, Jammy Huang wrote:
>> Hi Hans,
>>
>> Please refer to Page 132, 'Video stream data format – ASPEED mode compression' of
>> https://github.com/AspeedTech-BMC/openbmc/releases/download/v07.02/SDK_User_Guide_v07.02.pdf
>>
>> Is above information enough??
> Hmm, close. It would need some additional information, possibly documented in the source
> code:
>
> - What is the Frame Header? (the doc just says that it is different, but not what it actually is)
> - What is the EOI code? Is it equal to 'FEND'? (not quite clear to me)
>
> I think the documentation looks sufficient once those two items are addressed.
Information you mentioned will be included in next revision, 
"SDK_User_Guide_v07.03.pdf".
The releases will be updated to 
https://github.com/AspeedTech-BMC/openbmc/releases
>> Or maybe we can postpone this series until the source code for this format ready.
>> I think I can accomplish it by the end of this year.
> That's quite soon, so perhaps that would be the best approach. That way you also
> know exactly what information is missing from the SDK User Guide, and if that is
> documented, then this format can be used in the kernel.
>
> But several of the patches in the series are independent of the new jpeg format,
> so it might be useful to get those in first.

Understood. For the patches below, I will resend except for patch 
5/6/8/10 which are closely
relative to aspeed jpeg format.

    1. media: aspeed: move err-handling together to the bottom
    2. media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log
    3. media: aspeed: add more debug log messages
    4. media: aspeed: refactor to gather format/compress settings
    5. media: v4l: Add definition for the Aspeed JPEG format
    6. media: v4l2-ctrls: Reserve controls for ASPEED
    7. media: aspeed: use model-data
    8. media: aspeed: Support aspeed mode to reduce compressed data
    9. media: aspeed: add comments and macro
    10.media: aspeed: Extend debug message



>
> Regards,
>
> 	Hans
>
>> On 2021/11/25 下午 10:31, Hans Verkuil wrote:
>>> Hi Jammy,
>>>
>>> I suggest posting a v3 that moves the patches that deal with the
>>> Aspeed JPEG format to the end of the series. That way I can easily
>>> merge the patches up to that point, and the Aspeed JPEG support can
>>> be handled separately.
>>>
>>> I'm not sure if it can be merged without that format being documented,
>>> either in pixfmt-reserved.rst, by implementing support for it in
>>> libv4lconvert from v4l-utils, or by providing a URL to some other
>>> publicly available source code. We really don't like adding pixel
>>> formats that cannot be interpreted by someone without access to the
>>> datasheets.
>>>
>>> Regards,
>>>
>>> 	Hans
>>>
>>> On 18/11/2021 08:40, Jammy Huang wrote:
>>>> The aim of this series is to add aspeed-jpeg support for aspeed-video
>>>> driver.
>>>>
>>>> To achieve this major goal some refactors are included.
>>>>
>>>> In the last, debugfs information is also updated per this change.
>>>>
>>>> Changes in v5:
>>>>    - Use model data to tell different soc
>>>>
>>>> Changes in v4:
>>>>    - Add definition for the Aspeed JPEG format
>>>>    - Reserve controls for ASPEED
>>>>    - Use s_fmt to update format rather than new control
>>>>    - Update aspeed hq quality range, 1 ~ 12
>>>>
>>>>
>>>> Jammy Huang (10):
>>>>     media: aspeed: move err-handling together to the bottom
>>>>     media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log
>>>>     media: aspeed: add more debug log messages
>>>>     media: aspeed: refactor to gather format/compress settings
>>>>     media: v4l: Add definition for the Aspeed JPEG format
>>>>     media: v4l2-ctrls: Reserve controls for ASPEED
>>>>     media: aspeed: use model-data
>>>>     media: aspeed: Support aspeed mode to reduce compressed data
>>>>     media: aspeed: add comments and macro
>>>>     media: aspeed: Extend debug message
>>>>
>>>>    .../media/uapi/v4l/pixfmt-reserved.rst        |  12 +
>>>>    drivers/media/platform/aspeed-video.c         | 534 ++++++++++++++----
>>>>    drivers/media/v4l2-core/v4l2-ioctl.c          |   1 +
>>>>    include/uapi/linux/aspeed-video.h             |  15 +
>>>>    include/uapi/linux/v4l2-controls.h            |   5 +
>>>>    include/uapi/linux/videodev2.h                |   1 +
>>>>    6 files changed, 467 insertions(+), 101 deletions(-)
>>>>    create mode 100644 include/uapi/linux/aspeed-video.h
>>>>
-- 
Best Regards
Jammy


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[Jammy Huang]

Hi Hans,

The implementation of decoder for this format as been available here.
https://github.com/AspeedTech-BMC/aspeed_codec

On 2021/12/2 上午 09:29, Jammy Huang wrote:
> Hi Hans,
>
> Thanks for your review and suggestions.
>
> On 2021/11/26 下午 05:51, Hans Verkuil wrote:
>> On 26/11/2021 09:17, Jammy Huang wrote:
>>> Hi Hans,
>>>
>>> Please refer to Page 132, 'Video stream data format – ASPEED mode compression' of
>>> https://github.com/AspeedTech-BMC/openbmc/releases/download/v07.02/SDK_User_Guide_v07.02.pdf
>>>
>>> Is above information enough??
>> Hmm, close. It would need some additional information, possibly documented in the source
>> code:
>>
>> - What is the Frame Header? (the doc just says that it is different, but not what it actually is)
>> - What is the EOI code? Is it equal to 'FEND'? (not quite clear to me)
>>
>> I think the documentation looks sufficient once those two items are addressed.
> Information you mentioned will be included in next revision,
> "SDK_User_Guide_v07.03.pdf".
> The releases will be updated to
> https://github.com/AspeedTech-BMC/openbmc/releases
>>> Or maybe we can postpone this series until the source code for this format ready.
>>> I think I can accomplish it by the end of this year.
>> That's quite soon, so perhaps that would be the best approach. That way you also
>> know exactly what information is missing from the SDK User Guide, and if that is
>> documented, then this format can be used in the kernel.
>>
>> But several of the patches in the series are independent of the new jpeg format,
>> so it might be useful to get those in first.
> Understood. For the patches below, I will resend except for patch
> 5/6/8/10 which are closely
> relative to aspeed jpeg format.
>
>      1. media: aspeed: move err-handling together to the bottom
>      2. media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log
>      3. media: aspeed: add more debug log messages
>      4. media: aspeed: refactor to gather format/compress settings
>      5. media: v4l: Add definition for the Aspeed JPEG format
>      6. media: v4l2-ctrls: Reserve controls for ASPEED
>      7. media: aspeed: use model-data
>      8. media: aspeed: Support aspeed mode to reduce compressed data
>      9. media: aspeed: add comments and macro
>      10.media: aspeed: Extend debug message
>
>
>
>> Regards,
>>
>> 	Hans
>>
>>> On 2021/11/25 下午 10:31, Hans Verkuil wrote:
>>>> Hi Jammy,
>>>>
>>>> I suggest posting a v3 that moves the patches that deal with the
>>>> Aspeed JPEG format to the end of the series. That way I can easily
>>>> merge the patches up to that point, and the Aspeed JPEG support can
>>>> be handled separately.
>>>>
>>>> I'm not sure if it can be merged without that format being documented,
>>>> either in pixfmt-reserved.rst, by implementing support for it in
>>>> libv4lconvert from v4l-utils, or by providing a URL to some other
>>>> publicly available source code. We really don't like adding pixel
>>>> formats that cannot be interpreted by someone without access to the
>>>> datasheets.
>>>>
>>>> Regards,
>>>>
>>>> 	Hans
>>>>
>>>> On 18/11/2021 08:40, Jammy Huang wrote:
>>>>> The aim of this series is to add aspeed-jpeg support for aspeed-video
>>>>> driver.
>>>>>
>>>>> To achieve this major goal some refactors are included.
>>>>>
>>>>> In the last, debugfs information is also updated per this change.
>>>>>
>>>>> Changes in v5:
>>>>>     - Use model data to tell different soc
>>>>>
>>>>> Changes in v4:
>>>>>     - Add definition for the Aspeed JPEG format
>>>>>     - Reserve controls for ASPEED
>>>>>     - Use s_fmt to update format rather than new control
>>>>>     - Update aspeed hq quality range, 1 ~ 12
>>>>>
>>>>>
>>>>> Jammy Huang (10):
>>>>>      media: aspeed: move err-handling together to the bottom
>>>>>      media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log
>>>>>      media: aspeed: add more debug log messages
>>>>>      media: aspeed: refactor to gather format/compress settings
>>>>>      media: v4l: Add definition for the Aspeed JPEG format
>>>>>      media: v4l2-ctrls: Reserve controls for ASPEED
>>>>>      media: aspeed: use model-data
>>>>>      media: aspeed: Support aspeed mode to reduce compressed data
>>>>>      media: aspeed: add comments and macro
>>>>>      media: aspeed: Extend debug message
>>>>>
>>>>>     .../media/uapi/v4l/pixfmt-reserved.rst        |  12 +
>>>>>     drivers/media/platform/aspeed-video.c         | 534 ++++++++++++++----
>>>>>     drivers/media/v4l2-core/v4l2-ioctl.c          |   1 +
>>>>>     include/uapi/linux/aspeed-video.h             |  15 +
>>>>>     include/uapi/linux/v4l2-controls.h            |   5 +
>>>>>     include/uapi/linux/videodev2.h                |   1 +
>>>>>     6 files changed, 467 insertions(+), 101 deletions(-)
>>>>>     create mode 100644 include/uapi/linux/aspeed-video.h
>>>>>
-- 
Best Regards
Jammy

Re: [PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[Jammy Huang]

Hi Hans,

The implementation of decoder for this format as been available here.
https://github.com/AspeedTech-BMC/aspeed_codec

On 2021/12/2 上午 09:29, Jammy Huang wrote:
> Hi Hans,
>
> Thanks for your review and suggestions.
>
> On 2021/11/26 下午 05:51, Hans Verkuil wrote:
>> On 26/11/2021 09:17, Jammy Huang wrote:
>>> Hi Hans,
>>>
>>> Please refer to Page 132, 'Video stream data format – ASPEED mode compression' of
>>> https://github.com/AspeedTech-BMC/openbmc/releases/download/v07.02/SDK_User_Guide_v07.02.pdf
>>>
>>> Is above information enough??
>> Hmm, close. It would need some additional information, possibly documented in the source
>> code:
>>
>> - What is the Frame Header? (the doc just says that it is different, but not what it actually is)
>> - What is the EOI code? Is it equal to 'FEND'? (not quite clear to me)
>>
>> I think the documentation looks sufficient once those two items are addressed.
> Information you mentioned will be included in next revision,
> "SDK_User_Guide_v07.03.pdf".
> The releases will be updated to
> https://github.com/AspeedTech-BMC/openbmc/releases
>>> Or maybe we can postpone this series until the source code for this format ready.
>>> I think I can accomplish it by the end of this year.
>> That's quite soon, so perhaps that would be the best approach. That way you also
>> know exactly what information is missing from the SDK User Guide, and if that is
>> documented, then this format can be used in the kernel.
>>
>> But several of the patches in the series are independent of the new jpeg format,
>> so it might be useful to get those in first.
> Understood. For the patches below, I will resend except for patch
> 5/6/8/10 which are closely
> relative to aspeed jpeg format.
>
>      1. media: aspeed: move err-handling together to the bottom
>      2. media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log
>      3. media: aspeed: add more debug log messages
>      4. media: aspeed: refactor to gather format/compress settings
>      5. media: v4l: Add definition for the Aspeed JPEG format
>      6. media: v4l2-ctrls: Reserve controls for ASPEED
>      7. media: aspeed: use model-data
>      8. media: aspeed: Support aspeed mode to reduce compressed data
>      9. media: aspeed: add comments and macro
>      10.media: aspeed: Extend debug message
>
>
>
>> Regards,
>>
>> 	Hans
>>
>>> On 2021/11/25 下午 10:31, Hans Verkuil wrote:
>>>> Hi Jammy,
>>>>
>>>> I suggest posting a v3 that moves the patches that deal with the
>>>> Aspeed JPEG format to the end of the series. That way I can easily
>>>> merge the patches up to that point, and the Aspeed JPEG support can
>>>> be handled separately.
>>>>
>>>> I'm not sure if it can be merged without that format being documented,
>>>> either in pixfmt-reserved.rst, by implementing support for it in
>>>> libv4lconvert from v4l-utils, or by providing a URL to some other
>>>> publicly available source code. We really don't like adding pixel
>>>> formats that cannot be interpreted by someone without access to the
>>>> datasheets.
>>>>
>>>> Regards,
>>>>
>>>> 	Hans
>>>>
>>>> On 18/11/2021 08:40, Jammy Huang wrote:
>>>>> The aim of this series is to add aspeed-jpeg support for aspeed-video
>>>>> driver.
>>>>>
>>>>> To achieve this major goal some refactors are included.
>>>>>
>>>>> In the last, debugfs information is also updated per this change.
>>>>>
>>>>> Changes in v5:
>>>>>     - Use model data to tell different soc
>>>>>
>>>>> Changes in v4:
>>>>>     - Add definition for the Aspeed JPEG format
>>>>>     - Reserve controls for ASPEED
>>>>>     - Use s_fmt to update format rather than new control
>>>>>     - Update aspeed hq quality range, 1 ~ 12
>>>>>
>>>>>
>>>>> Jammy Huang (10):
>>>>>      media: aspeed: move err-handling together to the bottom
>>>>>      media: aspeed: use v4l2_info/v4l2_warn/v4l2_dbg for log
>>>>>      media: aspeed: add more debug log messages
>>>>>      media: aspeed: refactor to gather format/compress settings
>>>>>      media: v4l: Add definition for the Aspeed JPEG format
>>>>>      media: v4l2-ctrls: Reserve controls for ASPEED
>>>>>      media: aspeed: use model-data
>>>>>      media: aspeed: Support aspeed mode to reduce compressed data
>>>>>      media: aspeed: add comments and macro
>>>>>      media: aspeed: Extend debug message
>>>>>
>>>>>     .../media/uapi/v4l/pixfmt-reserved.rst        |  12 +
>>>>>     drivers/media/platform/aspeed-video.c         | 534 ++++++++++++++----
>>>>>     drivers/media/v4l2-core/v4l2-ioctl.c          |   1 +
>>>>>     include/uapi/linux/aspeed-video.h             |  15 +
>>>>>     include/uapi/linux/v4l2-controls.h            |   5 +
>>>>>     include/uapi/linux/videodev2.h                |   1 +
>>>>>     6 files changed, 467 insertions(+), 101 deletions(-)
>>>>>     create mode 100644 include/uapi/linux/aspeed-video.h
>>>>>
-- 
Best Regards
Jammy


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[sakari.ailus]

Hi Jammy,

On Tue, Dec 07, 2021 at 11:03:00AM +0800, Jammy Huang wrote:
> Hi Hans,
> 
> The implementation of decoder for this format as been available here.
> https://github.com/AspeedTech-BMC/aspeed_codec

The format documentation should point to this, as well as other
documentation there is.

I'm not sure there have been specific requirements of license, but it's the
first time I see MPL 2.0 being used in such context. It's eventually
convertible to GPL (or LGPL) although the route is inconvenient at best.
Or would people be happy with MPL code in v4l-utils?

I wonder what others think.

-- 
Regards,

Sakari Ailus

Re: [PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[sakari.ailus]

Hi Jammy,

On Tue, Dec 07, 2021 at 11:03:00AM +0800, Jammy Huang wrote:
> Hi Hans,
> 
> The implementation of decoder for this format as been available here.
> https://github.com/AspeedTech-BMC/aspeed_codec

The format documentation should point to this, as well as other
documentation there is.

I'm not sure there have been specific requirements of license, but it's the
first time I see MPL 2.0 being used in such context. It's eventually
convertible to GPL (or LGPL) although the route is inconvenient at best.
Or would people be happy with MPL code in v4l-utils?

I wonder what others think.

-- 
Regards,

Sakari Ailus

Re: [PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[sakari.ailus]

Hi Jammy,

On Tue, Dec 07, 2021 at 11:03:00AM +0800, Jammy Huang wrote:
> Hi Hans,
> 
> The implementation of decoder for this format as been available here.
> https://github.com/AspeedTech-BMC/aspeed_codec

The format documentation should point to this, as well as other
documentation there is.

I'm not sure there have been specific requirements of license, but it's the
first time I see MPL 2.0 being used in such context. It's eventually
convertible to GPL (or LGPL) although the route is inconvenient at best.
Or would people be happy with MPL code in v4l-utils?

I wonder what others think.

-- 
Regards,

Sakari Ailus

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[Jammy Huang]

Hi Sakari,

On 2021/12/21 上午 03:55, sakari.ailus@linux.intel.com wrote:
> Hi Jammy,
>
> On Tue, Dec 07, 2021 at 11:03:00AM +0800, Jammy Huang wrote:
>> Hi Hans,
>>
>> The implementation of decoder for this format as been available here.
>> https://github.com/AspeedTech-BMC/aspeed_codec
> The format documentation should point to this, as well as other
> documentation there is.
OK, I will add this into format documentation as well.
>
> I'm not sure there have been specific requirements of license, but it's the
> first time I see MPL 2.0 being used in such context. It's eventually
> convertible to GPL (or LGPL) although the route is inconvenient at best.
> Or would people be happy with MPL code in v4l-utils?

In order to have aspeed-jpeg format to work on openbmc's KVM, I was 
working on noVNC.
And this codec library is part of the code to make it work.

Before I put this source on github, I am not sure which kind of license 
is suitable. So I check
what noVNC used, https://github.com/novnc/noVNC/blob/master/LICENSE.txt. 
That's why I
adapted MPL 2.0.

If you think GPL(or LGPL) is more suitable, I can make a switch.


>
> I wonder what others think.
>
-- 
Best Regards
Jammy

Re: [PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[Jammy Huang]

Hi Sakari,

On 2021/12/21 上午 03:55, sakari.ailus@linux.intel.com wrote:
> Hi Jammy,
>
> On Tue, Dec 07, 2021 at 11:03:00AM +0800, Jammy Huang wrote:
>> Hi Hans,
>>
>> The implementation of decoder for this format as been available here.
>> https://github.com/AspeedTech-BMC/aspeed_codec
> The format documentation should point to this, as well as other
> documentation there is.
OK, I will add this into format documentation as well.
>
> I'm not sure there have been specific requirements of license, but it's the
> first time I see MPL 2.0 being used in such context. It's eventually
> convertible to GPL (or LGPL) although the route is inconvenient at best.
> Or would people be happy with MPL code in v4l-utils?

In order to have aspeed-jpeg format to work on openbmc's KVM, I was 
working on noVNC.
And this codec library is part of the code to make it work.

Before I put this source on github, I am not sure which kind of license 
is suitable. So I check
what noVNC used, https://github.com/novnc/noVNC/blob/master/LICENSE.txt. 
That's why I
adapted MPL 2.0.

If you think GPL(or LGPL) is more suitable, I can make a switch.


>
> I wonder what others think.
>
-- 
Best Regards
Jammy

Re: [PATCH v5 00/10] add aspeed-jpeg support for aspeed-video

[Jammy Huang]

Hi Sakari,

On 2021/12/21 上午 03:55, sakari.ailus@linux.intel.com wrote:
> Hi Jammy,
>
> On Tue, Dec 07, 2021 at 11:03:00AM +0800, Jammy Huang wrote:
>> Hi Hans,
>>
>> The implementation of decoder for this format as been available here.
>> https://github.com/AspeedTech-BMC/aspeed_codec
> The format documentation should point to this, as well as other
> documentation there is.
OK, I will add this into format documentation as well.
>
> I'm not sure there have been specific requirements of license, but it's the
> first time I see MPL 2.0 being used in such context. It's eventually
> convertible to GPL (or LGPL) although the route is inconvenient at best.
> Or would people be happy with MPL code in v4l-utils?

In order to have aspeed-jpeg format to work on openbmc's KVM, I was 
working on noVNC.
And this codec library is part of the code to make it work.

Before I put this source on github, I am not sure which kind of license 
is suitable. So I check
what noVNC used, https://github.com/novnc/noVNC/blob/master/LICENSE.txt. 
That's why I
adapted MPL 2.0.

If you think GPL(or LGPL) is more suitable, I can make a switch.


>
> I wonder what others think.
>
-- 
Best Regards
Jammy


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel