[PATCH v8 0/4] add aspeed-jpeg support for aspeed-video

[PATCH v8 0/4] add aspeed-jpeg support for aspeed-video

[Jammy Huang]

The aim of this series is to add aspeed-jpeg support for aspeed-video
driver. aspeed-jpeg is a per-frame differential jpeg format which only
compress the parts which are different from the previous frame. In this
way, it reduces both the amount of data to be transferred by network and
those to be decoded on the client side.

In the last, debugfs information is also updated per this change.

Changes in v8:
 - Add information of decoder's implementation
 
Changes in v7:
 - Separate other patches alone from aspeed-jpeg series
 - for Aspeed-jpeg, generate an I frame every 8 frames
 - rename compression_mode as compression_scheme
 - Add more reference for aspeed-jpeg
 - Update debugfs message

Changes in v6:
 - Update description for new format, aspeed-jpeg, in Documentation.

Changes in v5:
 - Use model data to tell different soc

Changes in v4:
 - Add definition for the Aspeed JPEG format
 - Reserve controls for ASPEED
 - Use s_fmt to update format rather than new control
 - Update aspeed hq quality range, 1 ~ 12


Jammy Huang (4):
  media: v4l: Add definition for the Aspeed JPEG format
  media: v4l2-ctrls: Reserve controls for ASPEED
  media: aspeed: Support aspeed mode to reduce compressed data
  media: aspeed: Extend debug message

 .../media/uapi/v4l/pixfmt-reserved.rst        |  17 ++
 drivers/media/platform/aspeed-video.c         | 287 ++++++++++++++++--
 drivers/media/v4l2-core/v4l2-ioctl.c          |   1 +
 include/uapi/linux/aspeed-video.h             |  15 +
 include/uapi/linux/v4l2-controls.h            |   5 +
 include/uapi/linux/videodev2.h                |   1 +
 6 files changed, 307 insertions(+), 19 deletions(-)
 create mode 100644 include/uapi/linux/aspeed-video.h

-- 
2.25.1

[PATCH v8 0/4] add aspeed-jpeg support for aspeed-video

[Jammy Huang]

The aim of this series is to add aspeed-jpeg support for aspeed-video
driver. aspeed-jpeg is a per-frame differential jpeg format which only
compress the parts which are different from the previous frame. In this
way, it reduces both the amount of data to be transferred by network and
those to be decoded on the client side.

In the last, debugfs information is also updated per this change.

Changes in v8:
 - Add information of decoder's implementation
 
Changes in v7:
 - Separate other patches alone from aspeed-jpeg series
 - for Aspeed-jpeg, generate an I frame every 8 frames
 - rename compression_mode as compression_scheme
 - Add more reference for aspeed-jpeg
 - Update debugfs message

Changes in v6:
 - Update description for new format, aspeed-jpeg, in Documentation.

Changes in v5:
 - Use model data to tell different soc

Changes in v4:
 - Add definition for the Aspeed JPEG format
 - Reserve controls for ASPEED
 - Use s_fmt to update format rather than new control
 - Update aspeed hq quality range, 1 ~ 12


Jammy Huang (4):
  media: v4l: Add definition for the Aspeed JPEG format
  media: v4l2-ctrls: Reserve controls for ASPEED
  media: aspeed: Support aspeed mode to reduce compressed data
  media: aspeed: Extend debug message

 .../media/uapi/v4l/pixfmt-reserved.rst        |  17 ++
 drivers/media/platform/aspeed-video.c         | 287 ++++++++++++++++--
 drivers/media/v4l2-core/v4l2-ioctl.c          |   1 +
 include/uapi/linux/aspeed-video.h             |  15 +
 include/uapi/linux/v4l2-controls.h            |   5 +
 include/uapi/linux/videodev2.h                |   1 +
 6 files changed, 307 insertions(+), 19 deletions(-)
 create mode 100644 include/uapi/linux/aspeed-video.h

-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 1/4] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

This introduces support for the Aspeed JPEG format, where the new frame
can refer to previous frame to reduce the amount of compressed data. The
concept is similar to I/P frame of video compression. It will compare the
new frame with previous one to decide which macroblock's data is
changed, and only the changed macroblocks will be compressed.

This Aspeed JPEG format is used by the video engine on Aspeed platforms,
which is generally adapted for remote KVM.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v8:
  - Add decoder information for aspeed-jpeg
v7:
  - Add more information for aspeed-jpeg
v6:
  - Update description for new format, aspeed-jpeg, in Documentation.
v5:
  - no update
v4:
  - new
---
 .../media/uapi/v4l/pixfmt-reserved.rst          | 17 +++++++++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c            |  1 +
 include/uapi/linux/videodev2.h                  |  1 +
 3 files changed, 19 insertions(+)

diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
index b2cd155e691b..1d0dc8d86ed7 100644
--- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
+++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
@@ -264,6 +264,23 @@ please make a proposal on the linux-media mailing list.
 	of tiles, resulting in 32-aligned resolutions for the luminance plane
 	and 16-aligned resolutions for the chrominance plane (with 2x2
 	subsampling).
+    * .. _V4L2-PIX-FMT-AJPG:
+
+      - ``V4L2_PIX_FMT_AJPG``
+      - 'AJPG'
+      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
+        which is generally adapted for remote KVM.
+        On each frame compression, I will compare the new frame with previous
+        one to decide which macroblock's data is changed, and only the changed
+        macroblocks will be compressed.
+
+        The implementation is based on AST2600 A3 datasheet, revision 0.9, which
+        is not publicly available. Or you can reference Video stream data format
+        – ASPEED mode compression of SDK_User_Guide which available on
+        AspeedTech-BMC/openbmc/releases.
+
+        Decoder's implementation can be found here,
+        `https://github.com/AspeedTech-BMC/aspeed_codec/ <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
 
 .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
 
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 24db33f803c0..00dde01d2f97 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 		case V4L2_PIX_FMT_S5C_UYVY_JPG:	descr = "S5C73MX interleaved UYVY/JPEG"; break;
 		case V4L2_PIX_FMT_MT21C:	descr = "Mediatek Compressed Format"; break;
 		case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
+		case V4L2_PIX_FMT_AJPG:		descr = "Aspeed JPEG"; break;
 		default:
 			if (fmt->description[0])
 				return;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 3210b3c82a4a..994eb6155ea9 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -726,6 +726,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
 #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
 #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
+#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
 
 /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
 #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
-- 
2.25.1

[PATCH v8 1/4] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

This introduces support for the Aspeed JPEG format, where the new frame
can refer to previous frame to reduce the amount of compressed data. The
concept is similar to I/P frame of video compression. It will compare the
new frame with previous one to decide which macroblock's data is
changed, and only the changed macroblocks will be compressed.

This Aspeed JPEG format is used by the video engine on Aspeed platforms,
which is generally adapted for remote KVM.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v8:
  - Add decoder information for aspeed-jpeg
v7:
  - Add more information for aspeed-jpeg
v6:
  - Update description for new format, aspeed-jpeg, in Documentation.
v5:
  - no update
v4:
  - new
---
 .../media/uapi/v4l/pixfmt-reserved.rst          | 17 +++++++++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c            |  1 +
 include/uapi/linux/videodev2.h                  |  1 +
 3 files changed, 19 insertions(+)

diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
index b2cd155e691b..1d0dc8d86ed7 100644
--- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
+++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
@@ -264,6 +264,23 @@ please make a proposal on the linux-media mailing list.
 	of tiles, resulting in 32-aligned resolutions for the luminance plane
 	and 16-aligned resolutions for the chrominance plane (with 2x2
 	subsampling).
+    * .. _V4L2-PIX-FMT-AJPG:
+
+      - ``V4L2_PIX_FMT_AJPG``
+      - 'AJPG'
+      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
+        which is generally adapted for remote KVM.
+        On each frame compression, I will compare the new frame with previous
+        one to decide which macroblock's data is changed, and only the changed
+        macroblocks will be compressed.
+
+        The implementation is based on AST2600 A3 datasheet, revision 0.9, which
+        is not publicly available. Or you can reference Video stream data format
+        – ASPEED mode compression of SDK_User_Guide which available on
+        AspeedTech-BMC/openbmc/releases.
+
+        Decoder's implementation can be found here,
+        `https://github.com/AspeedTech-BMC/aspeed_codec/ <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
 
 .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
 
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 24db33f803c0..00dde01d2f97 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 		case V4L2_PIX_FMT_S5C_UYVY_JPG:	descr = "S5C73MX interleaved UYVY/JPEG"; break;
 		case V4L2_PIX_FMT_MT21C:	descr = "Mediatek Compressed Format"; break;
 		case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
+		case V4L2_PIX_FMT_AJPG:		descr = "Aspeed JPEG"; break;
 		default:
 			if (fmt->description[0])
 				return;
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 3210b3c82a4a..994eb6155ea9 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -726,6 +726,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
 #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
 #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
+#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
 
 /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
 #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 1/4] media: v4l: Add definition for the Aspeed JPEG format

[Hans Verkuil]

On 24/12/2021 02:27, Jammy Huang wrote:
> This introduces support for the Aspeed JPEG format, where the new frame
> can refer to previous frame to reduce the amount of compressed data. The
> concept is similar to I/P frame of video compression. It will compare the
> new frame with previous one to decide which macroblock's data is
> changed, and only the changed macroblocks will be compressed.
> 
> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
> which is generally adapted for remote KVM.
> 
> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
> ---
> v8:
>   - Add decoder information for aspeed-jpeg
> v7:
>   - Add more information for aspeed-jpeg
> v6:
>   - Update description for new format, aspeed-jpeg, in Documentation.
> v5:
>   - no update
> v4:
>   - new
> ---
>  .../media/uapi/v4l/pixfmt-reserved.rst          | 17 +++++++++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c            |  1 +
>  include/uapi/linux/videodev2.h                  |  1 +
>  3 files changed, 19 insertions(+)
> 
> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst

This is the wrong file! It should be:

Documentation/userspace-api/media/v4l/pixfmt-reserved.rst

Regards,

	Hans

> index b2cd155e691b..1d0dc8d86ed7 100644
> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> @@ -264,6 +264,23 @@ please make a proposal on the linux-media mailing list.
>  	of tiles, resulting in 32-aligned resolutions for the luminance plane
>  	and 16-aligned resolutions for the chrominance plane (with 2x2
>  	subsampling).
> +    * .. _V4L2-PIX-FMT-AJPG:
> +
> +      - ``V4L2_PIX_FMT_AJPG``
> +      - 'AJPG'
> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
> +        which is generally adapted for remote KVM.
> +        On each frame compression, I will compare the new frame with previous
> +        one to decide which macroblock's data is changed, and only the changed
> +        macroblocks will be compressed.
> +
> +        The implementation is based on AST2600 A3 datasheet, revision 0.9, which
> +        is not publicly available. Or you can reference Video stream data format
> +        – ASPEED mode compression of SDK_User_Guide which available on
> +        AspeedTech-BMC/openbmc/releases.
> +
> +        Decoder's implementation can be found here,
> +        `https://github.com/AspeedTech-BMC/aspeed_codec/ <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
>  
>  .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
>  
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 24db33f803c0..00dde01d2f97 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  		case V4L2_PIX_FMT_S5C_UYVY_JPG:	descr = "S5C73MX interleaved UYVY/JPEG"; break;
>  		case V4L2_PIX_FMT_MT21C:	descr = "Mediatek Compressed Format"; break;
>  		case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
> +		case V4L2_PIX_FMT_AJPG:		descr = "Aspeed JPEG"; break;
>  		default:
>  			if (fmt->description[0])
>  				return;
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 3210b3c82a4a..994eb6155ea9 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -726,6 +726,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
>  #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
>  #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
> +#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
>  
>  /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
>  #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */

Re: [PATCH v8 1/4] media: v4l: Add definition for the Aspeed JPEG format

[Hans Verkuil]

On 24/12/2021 02:27, Jammy Huang wrote:
> This introduces support for the Aspeed JPEG format, where the new frame
> can refer to previous frame to reduce the amount of compressed data. The
> concept is similar to I/P frame of video compression. It will compare the
> new frame with previous one to decide which macroblock's data is
> changed, and only the changed macroblocks will be compressed.
> 
> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
> which is generally adapted for remote KVM.
> 
> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
> ---
> v8:
>   - Add decoder information for aspeed-jpeg
> v7:
>   - Add more information for aspeed-jpeg
> v6:
>   - Update description for new format, aspeed-jpeg, in Documentation.
> v5:
>   - no update
> v4:
>   - new
> ---
>  .../media/uapi/v4l/pixfmt-reserved.rst          | 17 +++++++++++++++++
>  drivers/media/v4l2-core/v4l2-ioctl.c            |  1 +
>  include/uapi/linux/videodev2.h                  |  1 +
>  3 files changed, 19 insertions(+)
> 
> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst

This is the wrong file! It should be:

Documentation/userspace-api/media/v4l/pixfmt-reserved.rst

Regards,

	Hans

> index b2cd155e691b..1d0dc8d86ed7 100644
> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> @@ -264,6 +264,23 @@ please make a proposal on the linux-media mailing list.
>  	of tiles, resulting in 32-aligned resolutions for the luminance plane
>  	and 16-aligned resolutions for the chrominance plane (with 2x2
>  	subsampling).
> +    * .. _V4L2-PIX-FMT-AJPG:
> +
> +      - ``V4L2_PIX_FMT_AJPG``
> +      - 'AJPG'
> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
> +        which is generally adapted for remote KVM.
> +        On each frame compression, I will compare the new frame with previous
> +        one to decide which macroblock's data is changed, and only the changed
> +        macroblocks will be compressed.
> +
> +        The implementation is based on AST2600 A3 datasheet, revision 0.9, which
> +        is not publicly available. Or you can reference Video stream data format
> +        – ASPEED mode compression of SDK_User_Guide which available on
> +        AspeedTech-BMC/openbmc/releases.
> +
> +        Decoder's implementation can be found here,
> +        `https://github.com/AspeedTech-BMC/aspeed_codec/ <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
>  
>  .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
>  
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 24db33f803c0..00dde01d2f97 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  		case V4L2_PIX_FMT_S5C_UYVY_JPG:	descr = "S5C73MX interleaved UYVY/JPEG"; break;
>  		case V4L2_PIX_FMT_MT21C:	descr = "Mediatek Compressed Format"; break;
>  		case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
> +		case V4L2_PIX_FMT_AJPG:		descr = "Aspeed JPEG"; break;
>  		default:
>  			if (fmt->description[0])
>  				return;
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 3210b3c82a4a..994eb6155ea9 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -726,6 +726,7 @@ struct v4l2_pix_format {
>  #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
>  #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
>  #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
> +#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
>  
>  /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
>  #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 1/4] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

Hi Hans,

On 2022/1/14 下午 04:11, Hans Verkuil wrote:
> On 24/12/2021 02:27, Jammy Huang wrote:
>> This introduces support for the Aspeed JPEG format, where the new frame
>> can refer to previous frame to reduce the amount of compressed data. The
>> concept is similar to I/P frame of video compression. It will compare the
>> new frame with previous one to decide which macroblock's data is
>> changed, and only the changed macroblocks will be compressed.
>>
>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>> which is generally adapted for remote KVM.
>>
>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>> ---
>> v8:
>>    - Add decoder information for aspeed-jpeg
>> v7:
>>    - Add more information for aspeed-jpeg
>> v6:
>>    - Update description for new format, aspeed-jpeg, in Documentation.
>> v5:
>>    - no update
>> v4:
>>    - new
>> ---
>>   .../media/uapi/v4l/pixfmt-reserved.rst          | 17 +++++++++++++++++
>>   drivers/media/v4l2-core/v4l2-ioctl.c            |  1 +
>>   include/uapi/linux/videodev2.h                  |  1 +
>>   3 files changed, 19 insertions(+)
>>
>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> This is the wrong file! It should be:
>
> Documentation/userspace-api/media/v4l/pixfmt-reserved.rst

Thanks, I just used git format-patch to generate the cover-letter and 
didn't notice this.

It looks like the file, pixfmt-reserved.rst, has different path in 
different Linux kernel,

* 5.4:           Documentation/media/uapi/v4l/pixfmt-reserved.rst

* 5.10/5.15: Documentation/userspace-api/media/v4l/pixfmt-reserved.rst

5.4 is the one I based to submit the patches.

Could you suggest the kernel that I should based to submit these patches??

I will need to change the number of V4L2_CID_USER_ASPEED_BASE per different

linux kernel as well.

>
> Regards,
>
> 	Hans
>
>> index b2cd155e691b..1d0dc8d86ed7 100644
>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>> @@ -264,6 +264,23 @@ please make a proposal on the linux-media mailing list.
>>   	of tiles, resulting in 32-aligned resolutions for the luminance plane
>>   	and 16-aligned resolutions for the chrominance plane (with 2x2
>>   	subsampling).
>> +    * .. _V4L2-PIX-FMT-AJPG:
>> +
>> +      - ``V4L2_PIX_FMT_AJPG``
>> +      - 'AJPG'
>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>> +        which is generally adapted for remote KVM.
>> +        On each frame compression, I will compare the new frame with previous
>> +        one to decide which macroblock's data is changed, and only the changed
>> +        macroblocks will be compressed.
>> +
>> +        The implementation is based on AST2600 A3 datasheet, revision 0.9, which
>> +        is not publicly available. Or you can reference Video stream data format
>> +        – ASPEED mode compression of SDK_User_Guide which available on
>> +        AspeedTech-BMC/openbmc/releases.
>> +
>> +        Decoder's implementation can be found here,
>> +        `https://github.com/AspeedTech-BMC/aspeed_codec/ <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
>>   
>>   .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
>>   
>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>> index 24db33f803c0..00dde01d2f97 100644
>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>> @@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>>   		case V4L2_PIX_FMT_S5C_UYVY_JPG:	descr = "S5C73MX interleaved UYVY/JPEG"; break;
>>   		case V4L2_PIX_FMT_MT21C:	descr = "Mediatek Compressed Format"; break;
>>   		case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
>> +		case V4L2_PIX_FMT_AJPG:		descr = "Aspeed JPEG"; break;
>>   		default:
>>   			if (fmt->description[0])
>>   				return;
>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>> index 3210b3c82a4a..994eb6155ea9 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -726,6 +726,7 @@ struct v4l2_pix_format {
>>   #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
>>   #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
>>   #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
>> +#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
>>   
>>   /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
>>   #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */

-- 
Best Regards
Jammy

Re: [PATCH v8 1/4] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

Hi Hans,

On 2022/1/14 下午 04:11, Hans Verkuil wrote:
> On 24/12/2021 02:27, Jammy Huang wrote:
>> This introduces support for the Aspeed JPEG format, where the new frame
>> can refer to previous frame to reduce the amount of compressed data. The
>> concept is similar to I/P frame of video compression. It will compare the
>> new frame with previous one to decide which macroblock's data is
>> changed, and only the changed macroblocks will be compressed.
>>
>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>> which is generally adapted for remote KVM.
>>
>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>> ---
>> v8:
>>    - Add decoder information for aspeed-jpeg
>> v7:
>>    - Add more information for aspeed-jpeg
>> v6:
>>    - Update description for new format, aspeed-jpeg, in Documentation.
>> v5:
>>    - no update
>> v4:
>>    - new
>> ---
>>   .../media/uapi/v4l/pixfmt-reserved.rst          | 17 +++++++++++++++++
>>   drivers/media/v4l2-core/v4l2-ioctl.c            |  1 +
>>   include/uapi/linux/videodev2.h                  |  1 +
>>   3 files changed, 19 insertions(+)
>>
>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> This is the wrong file! It should be:
>
> Documentation/userspace-api/media/v4l/pixfmt-reserved.rst

Thanks, I just used git format-patch to generate the cover-letter and 
didn't notice this.

It looks like the file, pixfmt-reserved.rst, has different path in 
different Linux kernel,

* 5.4:           Documentation/media/uapi/v4l/pixfmt-reserved.rst

* 5.10/5.15: Documentation/userspace-api/media/v4l/pixfmt-reserved.rst

5.4 is the one I based to submit the patches.

Could you suggest the kernel that I should based to submit these patches??

I will need to change the number of V4L2_CID_USER_ASPEED_BASE per different

linux kernel as well.

>
> Regards,
>
> 	Hans
>
>> index b2cd155e691b..1d0dc8d86ed7 100644
>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>> @@ -264,6 +264,23 @@ please make a proposal on the linux-media mailing list.
>>   	of tiles, resulting in 32-aligned resolutions for the luminance plane
>>   	and 16-aligned resolutions for the chrominance plane (with 2x2
>>   	subsampling).
>> +    * .. _V4L2-PIX-FMT-AJPG:
>> +
>> +      - ``V4L2_PIX_FMT_AJPG``
>> +      - 'AJPG'
>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>> +        which is generally adapted for remote KVM.
>> +        On each frame compression, I will compare the new frame with previous
>> +        one to decide which macroblock's data is changed, and only the changed
>> +        macroblocks will be compressed.
>> +
>> +        The implementation is based on AST2600 A3 datasheet, revision 0.9, which
>> +        is not publicly available. Or you can reference Video stream data format
>> +        – ASPEED mode compression of SDK_User_Guide which available on
>> +        AspeedTech-BMC/openbmc/releases.
>> +
>> +        Decoder's implementation can be found here,
>> +        `https://github.com/AspeedTech-BMC/aspeed_codec/ <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
>>   
>>   .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
>>   
>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>> index 24db33f803c0..00dde01d2f97 100644
>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>> @@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>>   		case V4L2_PIX_FMT_S5C_UYVY_JPG:	descr = "S5C73MX interleaved UYVY/JPEG"; break;
>>   		case V4L2_PIX_FMT_MT21C:	descr = "Mediatek Compressed Format"; break;
>>   		case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
>> +		case V4L2_PIX_FMT_AJPG:		descr = "Aspeed JPEG"; break;
>>   		default:
>>   			if (fmt->description[0])
>>   				return;
>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>> index 3210b3c82a4a..994eb6155ea9 100644
>> --- a/include/uapi/linux/videodev2.h
>> +++ b/include/uapi/linux/videodev2.h
>> @@ -726,6 +726,7 @@ struct v4l2_pix_format {
>>   #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
>>   #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
>>   #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
>> +#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
>>   
>>   /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
>>   #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */

-- 
Best Regards
Jammy


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 1/4] media: v4l: Add definition for the Aspeed JPEG format

[Joel Stanley]

On Mon, 17 Jan 2022 at 02:00, Jammy Huang <jammy_huang@aspeedtech.com> wrote:
>
> Hi Hans,
>
> On 2022/1/14 下午 04:11, Hans Verkuil wrote:
> > On 24/12/2021 02:27, Jammy Huang wrote:
> >> This introduces support for the Aspeed JPEG format, where the new frame
> >> can refer to previous frame to reduce the amount of compressed data. The
> >> concept is similar to I/P frame of video compression. It will compare the
> >> new frame with previous one to decide which macroblock's data is
> >> changed, and only the changed macroblocks will be compressed.
> >>
> >> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
> >> which is generally adapted for remote KVM.
> >>
> >> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
> >> ---
> >> v8:
> >>    - Add decoder information for aspeed-jpeg
> >> v7:
> >>    - Add more information for aspeed-jpeg
> >> v6:
> >>    - Update description for new format, aspeed-jpeg, in Documentation.
> >> v5:
> >>    - no update
> >> v4:
> >>    - new
> >> ---
> >>   .../media/uapi/v4l/pixfmt-reserved.rst          | 17 +++++++++++++++++
> >>   drivers/media/v4l2-core/v4l2-ioctl.c            |  1 +
> >>   include/uapi/linux/videodev2.h                  |  1 +
> >>   3 files changed, 19 insertions(+)
> >>
> >> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> > This is the wrong file! It should be:
> >
> > Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>
> Thanks, I just used git format-patch to generate the cover-letter and
> didn't notice this.
>
> It looks like the file, pixfmt-reserved.rst, has different path in
> different Linux kernel,
>
> * 5.4:           Documentation/media/uapi/v4l/pixfmt-reserved.rst
>
> * 5.10/5.15: Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>
> 5.4 is the one I based to submit the patches.
>
> Could you suggest the kernel that I should based to submit these patches??

You should always submit based on the latest release.

There are sometimes exceptions where you will base your patches on
something even newer, but generally you should grab the latest.

Be sure to re-test on the latest kernel before submitting.

Cheers,

Joel

>
> I will need to change the number of V4L2_CID_USER_ASPEED_BASE per different
>
> linux kernel as well.
>
> >
> > Regards,
> >
> >       Hans
> >
> >> index b2cd155e691b..1d0dc8d86ed7 100644
> >> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> >> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> >> @@ -264,6 +264,23 @@ please make a proposal on the linux-media mailing list.
> >>      of tiles, resulting in 32-aligned resolutions for the luminance plane
> >>      and 16-aligned resolutions for the chrominance plane (with 2x2
> >>      subsampling).
> >> +    * .. _V4L2-PIX-FMT-AJPG:
> >> +
> >> +      - ``V4L2_PIX_FMT_AJPG``
> >> +      - 'AJPG'
> >> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
> >> +        which is generally adapted for remote KVM.
> >> +        On each frame compression, I will compare the new frame with previous
> >> +        one to decide which macroblock's data is changed, and only the changed
> >> +        macroblocks will be compressed.
> >> +
> >> +        The implementation is based on AST2600 A3 datasheet, revision 0.9, which
> >> +        is not publicly available. Or you can reference Video stream data format
> >> +        – ASPEED mode compression of SDK_User_Guide which available on
> >> +        AspeedTech-BMC/openbmc/releases.
> >> +
> >> +        Decoder's implementation can be found here,
> >> +        `https://github.com/AspeedTech-BMC/aspeed_codec/ <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
> >>
> >>   .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
> >>
> >> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> >> index 24db33f803c0..00dde01d2f97 100644
> >> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> >> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> >> @@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> >>              case V4L2_PIX_FMT_S5C_UYVY_JPG: descr = "S5C73MX interleaved UYVY/JPEG"; break;
> >>              case V4L2_PIX_FMT_MT21C:        descr = "Mediatek Compressed Format"; break;
> >>              case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
> >> +            case V4L2_PIX_FMT_AJPG:         descr = "Aspeed JPEG"; break;
> >>              default:
> >>                      if (fmt->description[0])
> >>                              return;
> >> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> >> index 3210b3c82a4a..994eb6155ea9 100644
> >> --- a/include/uapi/linux/videodev2.h
> >> +++ b/include/uapi/linux/videodev2.h
> >> @@ -726,6 +726,7 @@ struct v4l2_pix_format {
> >>   #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
> >>   #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
> >>   #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
> >> +#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
> >>
> >>   /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
> >>   #define V4L2_PIX_FMT_IPU3_SBGGR10  v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
>
> --
> Best Regards
> Jammy
>

Re: [PATCH v8 1/4] media: v4l: Add definition for the Aspeed JPEG format

[Joel Stanley]

On Mon, 17 Jan 2022 at 02:00, Jammy Huang <jammy_huang@aspeedtech.com> wrote:
>
> Hi Hans,
>
> On 2022/1/14 下午 04:11, Hans Verkuil wrote:
> > On 24/12/2021 02:27, Jammy Huang wrote:
> >> This introduces support for the Aspeed JPEG format, where the new frame
> >> can refer to previous frame to reduce the amount of compressed data. The
> >> concept is similar to I/P frame of video compression. It will compare the
> >> new frame with previous one to decide which macroblock's data is
> >> changed, and only the changed macroblocks will be compressed.
> >>
> >> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
> >> which is generally adapted for remote KVM.
> >>
> >> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
> >> ---
> >> v8:
> >>    - Add decoder information for aspeed-jpeg
> >> v7:
> >>    - Add more information for aspeed-jpeg
> >> v6:
> >>    - Update description for new format, aspeed-jpeg, in Documentation.
> >> v5:
> >>    - no update
> >> v4:
> >>    - new
> >> ---
> >>   .../media/uapi/v4l/pixfmt-reserved.rst          | 17 +++++++++++++++++
> >>   drivers/media/v4l2-core/v4l2-ioctl.c            |  1 +
> >>   include/uapi/linux/videodev2.h                  |  1 +
> >>   3 files changed, 19 insertions(+)
> >>
> >> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> > This is the wrong file! It should be:
> >
> > Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>
> Thanks, I just used git format-patch to generate the cover-letter and
> didn't notice this.
>
> It looks like the file, pixfmt-reserved.rst, has different path in
> different Linux kernel,
>
> * 5.4:           Documentation/media/uapi/v4l/pixfmt-reserved.rst
>
> * 5.10/5.15: Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>
> 5.4 is the one I based to submit the patches.
>
> Could you suggest the kernel that I should based to submit these patches??

You should always submit based on the latest release.

There are sometimes exceptions where you will base your patches on
something even newer, but generally you should grab the latest.

Be sure to re-test on the latest kernel before submitting.

Cheers,

Joel

>
> I will need to change the number of V4L2_CID_USER_ASPEED_BASE per different
>
> linux kernel as well.
>
> >
> > Regards,
> >
> >       Hans
> >
> >> index b2cd155e691b..1d0dc8d86ed7 100644
> >> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> >> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> >> @@ -264,6 +264,23 @@ please make a proposal on the linux-media mailing list.
> >>      of tiles, resulting in 32-aligned resolutions for the luminance plane
> >>      and 16-aligned resolutions for the chrominance plane (with 2x2
> >>      subsampling).
> >> +    * .. _V4L2-PIX-FMT-AJPG:
> >> +
> >> +      - ``V4L2_PIX_FMT_AJPG``
> >> +      - 'AJPG'
> >> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
> >> +        which is generally adapted for remote KVM.
> >> +        On each frame compression, I will compare the new frame with previous
> >> +        one to decide which macroblock's data is changed, and only the changed
> >> +        macroblocks will be compressed.
> >> +
> >> +        The implementation is based on AST2600 A3 datasheet, revision 0.9, which
> >> +        is not publicly available. Or you can reference Video stream data format
> >> +        – ASPEED mode compression of SDK_User_Guide which available on
> >> +        AspeedTech-BMC/openbmc/releases.
> >> +
> >> +        Decoder's implementation can be found here,
> >> +        `https://github.com/AspeedTech-BMC/aspeed_codec/ <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
> >>
> >>   .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
> >>
> >> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> >> index 24db33f803c0..00dde01d2f97 100644
> >> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> >> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> >> @@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> >>              case V4L2_PIX_FMT_S5C_UYVY_JPG: descr = "S5C73MX interleaved UYVY/JPEG"; break;
> >>              case V4L2_PIX_FMT_MT21C:        descr = "Mediatek Compressed Format"; break;
> >>              case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
> >> +            case V4L2_PIX_FMT_AJPG:         descr = "Aspeed JPEG"; break;
> >>              default:
> >>                      if (fmt->description[0])
> >>                              return;
> >> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> >> index 3210b3c82a4a..994eb6155ea9 100644
> >> --- a/include/uapi/linux/videodev2.h
> >> +++ b/include/uapi/linux/videodev2.h
> >> @@ -726,6 +726,7 @@ struct v4l2_pix_format {
> >>   #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
> >>   #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
> >>   #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
> >> +#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
> >>
> >>   /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
> >>   #define V4L2_PIX_FMT_IPU3_SBGGR10  v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
>
> --
> Best Regards
> Jammy
>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 1/4] media: v4l: Add definition for the Aspeed JPEG format

[Joel Stanley]

On Mon, 17 Jan 2022 at 02:00, Jammy Huang <jammy_huang@aspeedtech.com> wrote:
>
> Hi Hans,
>
> On 2022/1/14 下午 04:11, Hans Verkuil wrote:
> > On 24/12/2021 02:27, Jammy Huang wrote:
> >> This introduces support for the Aspeed JPEG format, where the new frame
> >> can refer to previous frame to reduce the amount of compressed data. The
> >> concept is similar to I/P frame of video compression. It will compare the
> >> new frame with previous one to decide which macroblock's data is
> >> changed, and only the changed macroblocks will be compressed.
> >>
> >> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
> >> which is generally adapted for remote KVM.
> >>
> >> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
> >> ---
> >> v8:
> >>    - Add decoder information for aspeed-jpeg
> >> v7:
> >>    - Add more information for aspeed-jpeg
> >> v6:
> >>    - Update description for new format, aspeed-jpeg, in Documentation.
> >> v5:
> >>    - no update
> >> v4:
> >>    - new
> >> ---
> >>   .../media/uapi/v4l/pixfmt-reserved.rst          | 17 +++++++++++++++++
> >>   drivers/media/v4l2-core/v4l2-ioctl.c            |  1 +
> >>   include/uapi/linux/videodev2.h                  |  1 +
> >>   3 files changed, 19 insertions(+)
> >>
> >> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> > This is the wrong file! It should be:
> >
> > Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>
> Thanks, I just used git format-patch to generate the cover-letter and
> didn't notice this.
>
> It looks like the file, pixfmt-reserved.rst, has different path in
> different Linux kernel,
>
> * 5.4:           Documentation/media/uapi/v4l/pixfmt-reserved.rst
>
> * 5.10/5.15: Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>
> 5.4 is the one I based to submit the patches.
>
> Could you suggest the kernel that I should based to submit these patches??

You should always submit based on the latest release.

There are sometimes exceptions where you will base your patches on
something even newer, but generally you should grab the latest.

Be sure to re-test on the latest kernel before submitting.

Cheers,

Joel

>
> I will need to change the number of V4L2_CID_USER_ASPEED_BASE per different
>
> linux kernel as well.
>
> >
> > Regards,
> >
> >       Hans
> >
> >> index b2cd155e691b..1d0dc8d86ed7 100644
> >> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> >> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
> >> @@ -264,6 +264,23 @@ please make a proposal on the linux-media mailing list.
> >>      of tiles, resulting in 32-aligned resolutions for the luminance plane
> >>      and 16-aligned resolutions for the chrominance plane (with 2x2
> >>      subsampling).
> >> +    * .. _V4L2-PIX-FMT-AJPG:
> >> +
> >> +      - ``V4L2_PIX_FMT_AJPG``
> >> +      - 'AJPG'
> >> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
> >> +        which is generally adapted for remote KVM.
> >> +        On each frame compression, I will compare the new frame with previous
> >> +        one to decide which macroblock's data is changed, and only the changed
> >> +        macroblocks will be compressed.
> >> +
> >> +        The implementation is based on AST2600 A3 datasheet, revision 0.9, which
> >> +        is not publicly available. Or you can reference Video stream data format
> >> +        – ASPEED mode compression of SDK_User_Guide which available on
> >> +        AspeedTech-BMC/openbmc/releases.
> >> +
> >> +        Decoder's implementation can be found here,
> >> +        `https://github.com/AspeedTech-BMC/aspeed_codec/ <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
> >>
> >>   .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
> >>
> >> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> >> index 24db33f803c0..00dde01d2f97 100644
> >> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> >> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> >> @@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> >>              case V4L2_PIX_FMT_S5C_UYVY_JPG: descr = "S5C73MX interleaved UYVY/JPEG"; break;
> >>              case V4L2_PIX_FMT_MT21C:        descr = "Mediatek Compressed Format"; break;
> >>              case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
> >> +            case V4L2_PIX_FMT_AJPG:         descr = "Aspeed JPEG"; break;
> >>              default:
> >>                      if (fmt->description[0])
> >>                              return;
> >> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> >> index 3210b3c82a4a..994eb6155ea9 100644
> >> --- a/include/uapi/linux/videodev2.h
> >> +++ b/include/uapi/linux/videodev2.h
> >> @@ -726,6 +726,7 @@ struct v4l2_pix_format {
> >>   #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
> >>   #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
> >>   #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
> >> +#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
> >>
> >>   /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
> >>   #define V4L2_PIX_FMT_IPU3_SBGGR10  v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
>
> --
> Best Regards
> Jammy
>

Re: [PATCH v8 1/4] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

On 2022/1/17 上午 10:24, Joel Stanley wrote:
> On Mon, 17 Jan 2022 at 02:00, Jammy Huang <jammy_huang@aspeedtech.com> wrote:
>> Hi Hans,
>>
>> On 2022/1/14 下午 04:11, Hans Verkuil wrote:
>>> On 24/12/2021 02:27, Jammy Huang wrote:
>>>> This introduces support for the Aspeed JPEG format, where the new frame
>>>> can refer to previous frame to reduce the amount of compressed data. The
>>>> concept is similar to I/P frame of video compression. It will compare the
>>>> new frame with previous one to decide which macroblock's data is
>>>> changed, and only the changed macroblocks will be compressed.
>>>>
>>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>>> which is generally adapted for remote KVM.
>>>>
>>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>>> ---
>>>> v8:
>>>>     - Add decoder information for aspeed-jpeg
>>>> v7:
>>>>     - Add more information for aspeed-jpeg
>>>> v6:
>>>>     - Update description for new format, aspeed-jpeg, in Documentation.
>>>> v5:
>>>>     - no update
>>>> v4:
>>>>     - new
>>>> ---
>>>>    .../media/uapi/v4l/pixfmt-reserved.rst          | 17 +++++++++++++++++
>>>>    drivers/media/v4l2-core/v4l2-ioctl.c            |  1 +
>>>>    include/uapi/linux/videodev2.h                  |  1 +
>>>>    3 files changed, 19 insertions(+)
>>>>
>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>> This is the wrong file! It should be:
>>>
>>> Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>> Thanks, I just used git format-patch to generate the cover-letter and
>> didn't notice this.
>>
>> It looks like the file, pixfmt-reserved.rst, has different path in
>> different Linux kernel,
>>
>> * 5.4:           Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>
>> * 5.10/5.15: Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>>
>> 5.4 is the one I based to submit the patches.
>>
>> Could you suggest the kernel that I should based to submit these patches??
> You should always submit based on the latest release.
>
> There are sometimes exceptions where you will base your patches on
> something even newer, but generally you should grab the latest.
>
> Be sure to re-test on the latest kernel before submitting.
OK, thank you.
>
> Cheers,
>
> Joel
>
>> I will need to change the number of V4L2_CID_USER_ASPEED_BASE per different
>>
>> linux kernel as well.
>>
>>> Regards,
>>>
>>>        Hans
>>>
>>>> index b2cd155e691b..1d0dc8d86ed7 100644
>>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> @@ -264,6 +264,23 @@ please make a proposal on the linux-media mailing list.
>>>>       of tiles, resulting in 32-aligned resolutions for the luminance plane
>>>>       and 16-aligned resolutions for the chrominance plane (with 2x2
>>>>       subsampling).
>>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>>> +
>>>> +      - ``V4L2_PIX_FMT_AJPG``
>>>> +      - 'AJPG'
>>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>>> +        which is generally adapted for remote KVM.
>>>> +        On each frame compression, I will compare the new frame with previous
>>>> +        one to decide which macroblock's data is changed, and only the changed
>>>> +        macroblocks will be compressed.
>>>> +
>>>> +        The implementation is based on AST2600 A3 datasheet, revision 0.9, which
>>>> +        is not publicly available. Or you can reference Video stream data format
>>>> +        – ASPEED mode compression of SDK_User_Guide which available on
>>>> +        AspeedTech-BMC/openbmc/releases.
>>>> +
>>>> +        Decoder's implementation can be found here,
>>>> +        `https://github.com/AspeedTech-BMC/aspeed_codec/ <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
>>>>
>>>>    .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
>>>>
>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> index 24db33f803c0..00dde01d2f97 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> @@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>>>>               case V4L2_PIX_FMT_S5C_UYVY_JPG: descr = "S5C73MX interleaved UYVY/JPEG"; break;
>>>>               case V4L2_PIX_FMT_MT21C:        descr = "Mediatek Compressed Format"; break;
>>>>               case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
>>>> +            case V4L2_PIX_FMT_AJPG:         descr = "Aspeed JPEG"; break;
>>>>               default:
>>>>                       if (fmt->description[0])
>>>>                               return;
>>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>>> index 3210b3c82a4a..994eb6155ea9 100644
>>>> --- a/include/uapi/linux/videodev2.h
>>>> +++ b/include/uapi/linux/videodev2.h
>>>> @@ -726,6 +726,7 @@ struct v4l2_pix_format {
>>>>    #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
>>>>    #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
>>>>    #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
>>>> +#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
>>>>
>>>>    /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
>>>>    #define V4L2_PIX_FMT_IPU3_SBGGR10  v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
>> --
>> Best Regards
>> Jammy
>>
-- 
Best Regards
Jammy

Re: [PATCH v8 1/4] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

On 2022/1/17 上午 10:24, Joel Stanley wrote:
> On Mon, 17 Jan 2022 at 02:00, Jammy Huang <jammy_huang@aspeedtech.com> wrote:
>> Hi Hans,
>>
>> On 2022/1/14 下午 04:11, Hans Verkuil wrote:
>>> On 24/12/2021 02:27, Jammy Huang wrote:
>>>> This introduces support for the Aspeed JPEG format, where the new frame
>>>> can refer to previous frame to reduce the amount of compressed data. The
>>>> concept is similar to I/P frame of video compression. It will compare the
>>>> new frame with previous one to decide which macroblock's data is
>>>> changed, and only the changed macroblocks will be compressed.
>>>>
>>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>>> which is generally adapted for remote KVM.
>>>>
>>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>>> ---
>>>> v8:
>>>>     - Add decoder information for aspeed-jpeg
>>>> v7:
>>>>     - Add more information for aspeed-jpeg
>>>> v6:
>>>>     - Update description for new format, aspeed-jpeg, in Documentation.
>>>> v5:
>>>>     - no update
>>>> v4:
>>>>     - new
>>>> ---
>>>>    .../media/uapi/v4l/pixfmt-reserved.rst          | 17 +++++++++++++++++
>>>>    drivers/media/v4l2-core/v4l2-ioctl.c            |  1 +
>>>>    include/uapi/linux/videodev2.h                  |  1 +
>>>>    3 files changed, 19 insertions(+)
>>>>
>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>> This is the wrong file! It should be:
>>>
>>> Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>> Thanks, I just used git format-patch to generate the cover-letter and
>> didn't notice this.
>>
>> It looks like the file, pixfmt-reserved.rst, has different path in
>> different Linux kernel,
>>
>> * 5.4:           Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>
>> * 5.10/5.15: Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>>
>> 5.4 is the one I based to submit the patches.
>>
>> Could you suggest the kernel that I should based to submit these patches??
> You should always submit based on the latest release.
>
> There are sometimes exceptions where you will base your patches on
> something even newer, but generally you should grab the latest.
>
> Be sure to re-test on the latest kernel before submitting.
OK, thank you.
>
> Cheers,
>
> Joel
>
>> I will need to change the number of V4L2_CID_USER_ASPEED_BASE per different
>>
>> linux kernel as well.
>>
>>> Regards,
>>>
>>>        Hans
>>>
>>>> index b2cd155e691b..1d0dc8d86ed7 100644
>>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> @@ -264,6 +264,23 @@ please make a proposal on the linux-media mailing list.
>>>>       of tiles, resulting in 32-aligned resolutions for the luminance plane
>>>>       and 16-aligned resolutions for the chrominance plane (with 2x2
>>>>       subsampling).
>>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>>> +
>>>> +      - ``V4L2_PIX_FMT_AJPG``
>>>> +      - 'AJPG'
>>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>>> +        which is generally adapted for remote KVM.
>>>> +        On each frame compression, I will compare the new frame with previous
>>>> +        one to decide which macroblock's data is changed, and only the changed
>>>> +        macroblocks will be compressed.
>>>> +
>>>> +        The implementation is based on AST2600 A3 datasheet, revision 0.9, which
>>>> +        is not publicly available. Or you can reference Video stream data format
>>>> +        – ASPEED mode compression of SDK_User_Guide which available on
>>>> +        AspeedTech-BMC/openbmc/releases.
>>>> +
>>>> +        Decoder's implementation can be found here,
>>>> +        `https://github.com/AspeedTech-BMC/aspeed_codec/ <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
>>>>
>>>>    .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
>>>>
>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> index 24db33f803c0..00dde01d2f97 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> @@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>>>>               case V4L2_PIX_FMT_S5C_UYVY_JPG: descr = "S5C73MX interleaved UYVY/JPEG"; break;
>>>>               case V4L2_PIX_FMT_MT21C:        descr = "Mediatek Compressed Format"; break;
>>>>               case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
>>>> +            case V4L2_PIX_FMT_AJPG:         descr = "Aspeed JPEG"; break;
>>>>               default:
>>>>                       if (fmt->description[0])
>>>>                               return;
>>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>>> index 3210b3c82a4a..994eb6155ea9 100644
>>>> --- a/include/uapi/linux/videodev2.h
>>>> +++ b/include/uapi/linux/videodev2.h
>>>> @@ -726,6 +726,7 @@ struct v4l2_pix_format {
>>>>    #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
>>>>    #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
>>>>    #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
>>>> +#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
>>>>
>>>>    /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
>>>>    #define V4L2_PIX_FMT_IPU3_SBGGR10  v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
>> --
>> Best Regards
>> Jammy
>>
-- 
Best Regards
Jammy


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 1/4] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

On 2022/1/17 上午 10:24, Joel Stanley wrote:
> On Mon, 17 Jan 2022 at 02:00, Jammy Huang <jammy_huang@aspeedtech.com> wrote:
>> Hi Hans,
>>
>> On 2022/1/14 下午 04:11, Hans Verkuil wrote:
>>> On 24/12/2021 02:27, Jammy Huang wrote:
>>>> This introduces support for the Aspeed JPEG format, where the new frame
>>>> can refer to previous frame to reduce the amount of compressed data. The
>>>> concept is similar to I/P frame of video compression. It will compare the
>>>> new frame with previous one to decide which macroblock's data is
>>>> changed, and only the changed macroblocks will be compressed.
>>>>
>>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>>> which is generally adapted for remote KVM.
>>>>
>>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>>> ---
>>>> v8:
>>>>     - Add decoder information for aspeed-jpeg
>>>> v7:
>>>>     - Add more information for aspeed-jpeg
>>>> v6:
>>>>     - Update description for new format, aspeed-jpeg, in Documentation.
>>>> v5:
>>>>     - no update
>>>> v4:
>>>>     - new
>>>> ---
>>>>    .../media/uapi/v4l/pixfmt-reserved.rst          | 17 +++++++++++++++++
>>>>    drivers/media/v4l2-core/v4l2-ioctl.c            |  1 +
>>>>    include/uapi/linux/videodev2.h                  |  1 +
>>>>    3 files changed, 19 insertions(+)
>>>>
>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>> This is the wrong file! It should be:
>>>
>>> Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>> Thanks, I just used git format-patch to generate the cover-letter and
>> didn't notice this.
>>
>> It looks like the file, pixfmt-reserved.rst, has different path in
>> different Linux kernel,
>>
>> * 5.4:           Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>
>> * 5.10/5.15: Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>>
>> 5.4 is the one I based to submit the patches.
>>
>> Could you suggest the kernel that I should based to submit these patches??
> You should always submit based on the latest release.
>
> There are sometimes exceptions where you will base your patches on
> something even newer, but generally you should grab the latest.
>
> Be sure to re-test on the latest kernel before submitting.
OK, thank you.
>
> Cheers,
>
> Joel
>
>> I will need to change the number of V4L2_CID_USER_ASPEED_BASE per different
>>
>> linux kernel as well.
>>
>>> Regards,
>>>
>>>        Hans
>>>
>>>> index b2cd155e691b..1d0dc8d86ed7 100644
>>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> @@ -264,6 +264,23 @@ please make a proposal on the linux-media mailing list.
>>>>       of tiles, resulting in 32-aligned resolutions for the luminance plane
>>>>       and 16-aligned resolutions for the chrominance plane (with 2x2
>>>>       subsampling).
>>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>>> +
>>>> +      - ``V4L2_PIX_FMT_AJPG``
>>>> +      - 'AJPG'
>>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>>> +        which is generally adapted for remote KVM.
>>>> +        On each frame compression, I will compare the new frame with previous
>>>> +        one to decide which macroblock's data is changed, and only the changed
>>>> +        macroblocks will be compressed.
>>>> +
>>>> +        The implementation is based on AST2600 A3 datasheet, revision 0.9, which
>>>> +        is not publicly available. Or you can reference Video stream data format
>>>> +        – ASPEED mode compression of SDK_User_Guide which available on
>>>> +        AspeedTech-BMC/openbmc/releases.
>>>> +
>>>> +        Decoder's implementation can be found here,
>>>> +        `https://github.com/AspeedTech-BMC/aspeed_codec/ <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
>>>>
>>>>    .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
>>>>
>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> index 24db33f803c0..00dde01d2f97 100644
>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>> @@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>>>>               case V4L2_PIX_FMT_S5C_UYVY_JPG: descr = "S5C73MX interleaved UYVY/JPEG"; break;
>>>>               case V4L2_PIX_FMT_MT21C:        descr = "Mediatek Compressed Format"; break;
>>>>               case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
>>>> +            case V4L2_PIX_FMT_AJPG:         descr = "Aspeed JPEG"; break;
>>>>               default:
>>>>                       if (fmt->description[0])
>>>>                               return;
>>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>>> index 3210b3c82a4a..994eb6155ea9 100644
>>>> --- a/include/uapi/linux/videodev2.h
>>>> +++ b/include/uapi/linux/videodev2.h
>>>> @@ -726,6 +726,7 @@ struct v4l2_pix_format {
>>>>    #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
>>>>    #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
>>>>    #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
>>>> +#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
>>>>
>>>>    /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
>>>>    #define V4L2_PIX_FMT_IPU3_SBGGR10  v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
>> --
>> Best Regards
>> Jammy
>>
-- 
Best Regards
Jammy

Re: [PATCH v8 1/4] media: v4l: Add definition for the Aspeed JPEG format

[Hans Verkuil]

Hi Jammy,

On 1/17/22 08:18, Jammy Huang wrote:
> On 2022/1/17 上午 10:24, Joel Stanley wrote:
>> On Mon, 17 Jan 2022 at 02:00, Jammy Huang <jammy_huang@aspeedtech.com> wrote:
>>> Hi Hans,
>>>
>>> On 2022/1/14 下午 04:11, Hans Verkuil wrote:
>>>> On 24/12/2021 02:27, Jammy Huang wrote:
>>>>> This introduces support for the Aspeed JPEG format, where the new frame
>>>>> can refer to previous frame to reduce the amount of compressed data. The
>>>>> concept is similar to I/P frame of video compression. It will compare the
>>>>> new frame with previous one to decide which macroblock's data is
>>>>> changed, and only the changed macroblocks will be compressed.
>>>>>
>>>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>>>> which is generally adapted for remote KVM.
>>>>>
>>>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>>>> ---
>>>>> v8:
>>>>>     - Add decoder information for aspeed-jpeg
>>>>> v7:
>>>>>     - Add more information for aspeed-jpeg
>>>>> v6:
>>>>>     - Update description for new format, aspeed-jpeg, in Documentation.
>>>>> v5:
>>>>>     - no update
>>>>> v4:
>>>>>     - new
>>>>> ---
>>>>>    .../media/uapi/v4l/pixfmt-reserved.rst          | 17 +++++++++++++++++
>>>>>    drivers/media/v4l2-core/v4l2-ioctl.c            |  1 +
>>>>>    include/uapi/linux/videodev2.h                  |  1 +
>>>>>    3 files changed, 19 insertions(+)
>>>>>
>>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> This is the wrong file! It should be:
>>>>
>>>> Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>>> Thanks, I just used git format-patch to generate the cover-letter and
>>> didn't notice this.
>>>
>>> It looks like the file, pixfmt-reserved.rst, has different path in
>>> different Linux kernel,
>>>
>>> * 5.4:           Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>
>>> * 5.10/5.15: Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>>>
>>> 5.4 is the one I based to submit the patches.
>>>
>>> Could you suggest the kernel that I should based to submit these patches??
>> You should always submit based on the latest release.
>>
>> There are sometimes exceptions where you will base your patches on
>> something even newer, but generally you should grab the latest.
>>
>> Be sure to re-test on the latest kernel before submitting.
> OK, thank you.

I recommend waiting until later this week. I hope to have a PR for other outstanding
aspeed patches ready in a few days. It is easiest if you base your v9 on top of that
series, this to avoid merge conflicts.

Regards,

	Hans

>>
>> Cheers,
>>
>> Joel
>>
>>> I will need to change the number of V4L2_CID_USER_ASPEED_BASE per different
>>>
>>> linux kernel as well.
>>>
>>>> Regards,
>>>>
>>>>        Hans
>>>>
>>>>> index b2cd155e691b..1d0dc8d86ed7 100644
>>>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>>> @@ -264,6 +264,23 @@ please make a proposal on the linux-media mailing list.
>>>>>       of tiles, resulting in 32-aligned resolutions for the luminance plane
>>>>>       and 16-aligned resolutions for the chrominance plane (with 2x2
>>>>>       subsampling).
>>>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>>>> +
>>>>> +      - ``V4L2_PIX_FMT_AJPG``
>>>>> +      - 'AJPG'
>>>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>>>> +        which is generally adapted for remote KVM.
>>>>> +        On each frame compression, I will compare the new frame with previous
>>>>> +        one to decide which macroblock's data is changed, and only the changed
>>>>> +        macroblocks will be compressed.
>>>>> +
>>>>> +        The implementation is based on AST2600 A3 datasheet, revision 0.9, which
>>>>> +        is not publicly available. Or you can reference Video stream data format
>>>>> +        – ASPEED mode compression of SDK_User_Guide which available on
>>>>> +        AspeedTech-BMC/openbmc/releases.
>>>>> +
>>>>> +        Decoder's implementation can be found here,
>>>>> +        `https://github.com/AspeedTech-BMC/aspeed_codec/ <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
>>>>>
>>>>>    .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
>>>>>
>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> index 24db33f803c0..00dde01d2f97 100644
>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> @@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>>>>>               case V4L2_PIX_FMT_S5C_UYVY_JPG: descr = "S5C73MX interleaved UYVY/JPEG"; break;
>>>>>               case V4L2_PIX_FMT_MT21C:        descr = "Mediatek Compressed Format"; break;
>>>>>               case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
>>>>> +            case V4L2_PIX_FMT_AJPG:         descr = "Aspeed JPEG"; break;
>>>>>               default:
>>>>>                       if (fmt->description[0])
>>>>>                               return;
>>>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>>>> index 3210b3c82a4a..994eb6155ea9 100644
>>>>> --- a/include/uapi/linux/videodev2.h
>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>> @@ -726,6 +726,7 @@ struct v4l2_pix_format {
>>>>>    #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
>>>>>    #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
>>>>>    #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
>>>>> +#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
>>>>>
>>>>>    /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
>>>>>    #define V4L2_PIX_FMT_IPU3_SBGGR10  v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
>>> -- 
>>> Best Regards
>>> Jammy
>>>

Re: [PATCH v8 1/4] media: v4l: Add definition for the Aspeed JPEG format

[Hans Verkuil]

Hi Jammy,

On 1/17/22 08:18, Jammy Huang wrote:
> On 2022/1/17 上午 10:24, Joel Stanley wrote:
>> On Mon, 17 Jan 2022 at 02:00, Jammy Huang <jammy_huang@aspeedtech.com> wrote:
>>> Hi Hans,
>>>
>>> On 2022/1/14 下午 04:11, Hans Verkuil wrote:
>>>> On 24/12/2021 02:27, Jammy Huang wrote:
>>>>> This introduces support for the Aspeed JPEG format, where the new frame
>>>>> can refer to previous frame to reduce the amount of compressed data. The
>>>>> concept is similar to I/P frame of video compression. It will compare the
>>>>> new frame with previous one to decide which macroblock's data is
>>>>> changed, and only the changed macroblocks will be compressed.
>>>>>
>>>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>>>> which is generally adapted for remote KVM.
>>>>>
>>>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>>>> ---
>>>>> v8:
>>>>>     - Add decoder information for aspeed-jpeg
>>>>> v7:
>>>>>     - Add more information for aspeed-jpeg
>>>>> v6:
>>>>>     - Update description for new format, aspeed-jpeg, in Documentation.
>>>>> v5:
>>>>>     - no update
>>>>> v4:
>>>>>     - new
>>>>> ---
>>>>>    .../media/uapi/v4l/pixfmt-reserved.rst          | 17 +++++++++++++++++
>>>>>    drivers/media/v4l2-core/v4l2-ioctl.c            |  1 +
>>>>>    include/uapi/linux/videodev2.h                  |  1 +
>>>>>    3 files changed, 19 insertions(+)
>>>>>
>>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> This is the wrong file! It should be:
>>>>
>>>> Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>>> Thanks, I just used git format-patch to generate the cover-letter and
>>> didn't notice this.
>>>
>>> It looks like the file, pixfmt-reserved.rst, has different path in
>>> different Linux kernel,
>>>
>>> * 5.4:           Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>
>>> * 5.10/5.15: Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>>>
>>> 5.4 is the one I based to submit the patches.
>>>
>>> Could you suggest the kernel that I should based to submit these patches??
>> You should always submit based on the latest release.
>>
>> There are sometimes exceptions where you will base your patches on
>> something even newer, but generally you should grab the latest.
>>
>> Be sure to re-test on the latest kernel before submitting.
> OK, thank you.

I recommend waiting until later this week. I hope to have a PR for other outstanding
aspeed patches ready in a few days. It is easiest if you base your v9 on top of that
series, this to avoid merge conflicts.

Regards,

	Hans

>>
>> Cheers,
>>
>> Joel
>>
>>> I will need to change the number of V4L2_CID_USER_ASPEED_BASE per different
>>>
>>> linux kernel as well.
>>>
>>>> Regards,
>>>>
>>>>        Hans
>>>>
>>>>> index b2cd155e691b..1d0dc8d86ed7 100644
>>>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>>> @@ -264,6 +264,23 @@ please make a proposal on the linux-media mailing list.
>>>>>       of tiles, resulting in 32-aligned resolutions for the luminance plane
>>>>>       and 16-aligned resolutions for the chrominance plane (with 2x2
>>>>>       subsampling).
>>>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>>>> +
>>>>> +      - ``V4L2_PIX_FMT_AJPG``
>>>>> +      - 'AJPG'
>>>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>>>> +        which is generally adapted for remote KVM.
>>>>> +        On each frame compression, I will compare the new frame with previous
>>>>> +        one to decide which macroblock's data is changed, and only the changed
>>>>> +        macroblocks will be compressed.
>>>>> +
>>>>> +        The implementation is based on AST2600 A3 datasheet, revision 0.9, which
>>>>> +        is not publicly available. Or you can reference Video stream data format
>>>>> +        – ASPEED mode compression of SDK_User_Guide which available on
>>>>> +        AspeedTech-BMC/openbmc/releases.
>>>>> +
>>>>> +        Decoder's implementation can be found here,
>>>>> +        `https://github.com/AspeedTech-BMC/aspeed_codec/ <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
>>>>>
>>>>>    .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
>>>>>
>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> index 24db33f803c0..00dde01d2f97 100644
>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> @@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>>>>>               case V4L2_PIX_FMT_S5C_UYVY_JPG: descr = "S5C73MX interleaved UYVY/JPEG"; break;
>>>>>               case V4L2_PIX_FMT_MT21C:        descr = "Mediatek Compressed Format"; break;
>>>>>               case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
>>>>> +            case V4L2_PIX_FMT_AJPG:         descr = "Aspeed JPEG"; break;
>>>>>               default:
>>>>>                       if (fmt->description[0])
>>>>>                               return;
>>>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>>>> index 3210b3c82a4a..994eb6155ea9 100644
>>>>> --- a/include/uapi/linux/videodev2.h
>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>> @@ -726,6 +726,7 @@ struct v4l2_pix_format {
>>>>>    #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
>>>>>    #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
>>>>>    #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
>>>>> +#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
>>>>>
>>>>>    /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
>>>>>    #define V4L2_PIX_FMT_IPU3_SBGGR10  v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
>>> -- 
>>> Best Regards
>>> Jammy
>>>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 1/4] media: v4l: Add definition for the Aspeed JPEG format

[Hans Verkuil]

Hi Jammy,

On 1/17/22 08:18, Jammy Huang wrote:
> On 2022/1/17 上午 10:24, Joel Stanley wrote:
>> On Mon, 17 Jan 2022 at 02:00, Jammy Huang <jammy_huang@aspeedtech.com> wrote:
>>> Hi Hans,
>>>
>>> On 2022/1/14 下午 04:11, Hans Verkuil wrote:
>>>> On 24/12/2021 02:27, Jammy Huang wrote:
>>>>> This introduces support for the Aspeed JPEG format, where the new frame
>>>>> can refer to previous frame to reduce the amount of compressed data. The
>>>>> concept is similar to I/P frame of video compression. It will compare the
>>>>> new frame with previous one to decide which macroblock's data is
>>>>> changed, and only the changed macroblocks will be compressed.
>>>>>
>>>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>>>> which is generally adapted for remote KVM.
>>>>>
>>>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>>>> ---
>>>>> v8:
>>>>>     - Add decoder information for aspeed-jpeg
>>>>> v7:
>>>>>     - Add more information for aspeed-jpeg
>>>>> v6:
>>>>>     - Update description for new format, aspeed-jpeg, in Documentation.
>>>>> v5:
>>>>>     - no update
>>>>> v4:
>>>>>     - new
>>>>> ---
>>>>>    .../media/uapi/v4l/pixfmt-reserved.rst          | 17 +++++++++++++++++
>>>>>    drivers/media/v4l2-core/v4l2-ioctl.c            |  1 +
>>>>>    include/uapi/linux/videodev2.h                  |  1 +
>>>>>    3 files changed, 19 insertions(+)
>>>>>
>>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>> This is the wrong file! It should be:
>>>>
>>>> Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>>> Thanks, I just used git format-patch to generate the cover-letter and
>>> didn't notice this.
>>>
>>> It looks like the file, pixfmt-reserved.rst, has different path in
>>> different Linux kernel,
>>>
>>> * 5.4:           Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>
>>> * 5.10/5.15: Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>>>
>>> 5.4 is the one I based to submit the patches.
>>>
>>> Could you suggest the kernel that I should based to submit these patches??
>> You should always submit based on the latest release.
>>
>> There are sometimes exceptions where you will base your patches on
>> something even newer, but generally you should grab the latest.
>>
>> Be sure to re-test on the latest kernel before submitting.
> OK, thank you.

I recommend waiting until later this week. I hope to have a PR for other outstanding
aspeed patches ready in a few days. It is easiest if you base your v9 on top of that
series, this to avoid merge conflicts.

Regards,

	Hans

>>
>> Cheers,
>>
>> Joel
>>
>>> I will need to change the number of V4L2_CID_USER_ASPEED_BASE per different
>>>
>>> linux kernel as well.
>>>
>>>> Regards,
>>>>
>>>>        Hans
>>>>
>>>>> index b2cd155e691b..1d0dc8d86ed7 100644
>>>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>>> @@ -264,6 +264,23 @@ please make a proposal on the linux-media mailing list.
>>>>>       of tiles, resulting in 32-aligned resolutions for the luminance plane
>>>>>       and 16-aligned resolutions for the chrominance plane (with 2x2
>>>>>       subsampling).
>>>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>>>> +
>>>>> +      - ``V4L2_PIX_FMT_AJPG``
>>>>> +      - 'AJPG'
>>>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>>>> +        which is generally adapted for remote KVM.
>>>>> +        On each frame compression, I will compare the new frame with previous
>>>>> +        one to decide which macroblock's data is changed, and only the changed
>>>>> +        macroblocks will be compressed.
>>>>> +
>>>>> +        The implementation is based on AST2600 A3 datasheet, revision 0.9, which
>>>>> +        is not publicly available. Or you can reference Video stream data format
>>>>> +        – ASPEED mode compression of SDK_User_Guide which available on
>>>>> +        AspeedTech-BMC/openbmc/releases.
>>>>> +
>>>>> +        Decoder's implementation can be found here,
>>>>> +        `https://github.com/AspeedTech-BMC/aspeed_codec/ <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
>>>>>
>>>>>    .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
>>>>>
>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> index 24db33f803c0..00dde01d2f97 100644
>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> @@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>>>>>               case V4L2_PIX_FMT_S5C_UYVY_JPG: descr = "S5C73MX interleaved UYVY/JPEG"; break;
>>>>>               case V4L2_PIX_FMT_MT21C:        descr = "Mediatek Compressed Format"; break;
>>>>>               case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
>>>>> +            case V4L2_PIX_FMT_AJPG:         descr = "Aspeed JPEG"; break;
>>>>>               default:
>>>>>                       if (fmt->description[0])
>>>>>                               return;
>>>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>>>> index 3210b3c82a4a..994eb6155ea9 100644
>>>>> --- a/include/uapi/linux/videodev2.h
>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>> @@ -726,6 +726,7 @@ struct v4l2_pix_format {
>>>>>    #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
>>>>>    #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
>>>>>    #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
>>>>> +#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
>>>>>
>>>>>    /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
>>>>>    #define V4L2_PIX_FMT_IPU3_SBGGR10  v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
>>> -- 
>>> Best Regards
>>> Jammy
>>>

Re: [PATCH v8 1/4] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

Hi Hans,

On 2022/1/17 下午 05:28, Hans Verkuil wrote:
> Hi Jammy,
>
> On 1/17/22 08:18, Jammy Huang wrote:
>> On 2022/1/17 上午 10:24, Joel Stanley wrote:
>>> On Mon, 17 Jan 2022 at 02:00, Jammy Huang <jammy_huang@aspeedtech.com> wrote:
>>>> Hi Hans,
>>>>
>>>> On 2022/1/14 下午 04:11, Hans Verkuil wrote:
>>>>> On 24/12/2021 02:27, Jammy Huang wrote:
>>>>>> This introduces support for the Aspeed JPEG format, where the new frame
>>>>>> can refer to previous frame to reduce the amount of compressed data. The
>>>>>> concept is similar to I/P frame of video compression. It will compare the
>>>>>> new frame with previous one to decide which macroblock's data is
>>>>>> changed, and only the changed macroblocks will be compressed.
>>>>>>
>>>>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>>>>> which is generally adapted for remote KVM.
>>>>>>
>>>>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>>>>> ---
>>>>>> v8:
>>>>>>      - Add decoder information for aspeed-jpeg
>>>>>> v7:
>>>>>>      - Add more information for aspeed-jpeg
>>>>>> v6:
>>>>>>      - Update description for new format, aspeed-jpeg, in Documentation.
>>>>>> v5:
>>>>>>      - no update
>>>>>> v4:
>>>>>>      - new
>>>>>> ---
>>>>>>     .../media/uapi/v4l/pixfmt-reserved.rst          | 17 +++++++++++++++++
>>>>>>     drivers/media/v4l2-core/v4l2-ioctl.c            |  1 +
>>>>>>     include/uapi/linux/videodev2.h                  |  1 +
>>>>>>     3 files changed, 19 insertions(+)
>>>>>>
>>>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>>> This is the wrong file! It should be:
>>>>>
>>>>> Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>>>> Thanks, I just used git format-patch to generate the cover-letter and
>>>> didn't notice this.
>>>>
>>>> It looks like the file, pixfmt-reserved.rst, has different path in
>>>> different Linux kernel,
>>>>
>>>> * 5.4:           Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>>
>>>> * 5.10/5.15: Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>>>>
>>>> 5.4 is the one I based to submit the patches.
>>>>
>>>> Could you suggest the kernel that I should based to submit these patches??
>>> You should always submit based on the latest release.
>>>
>>> There are sometimes exceptions where you will base your patches on
>>> something even newer, but generally you should grab the latest.
>>>
>>> Be sure to re-test on the latest kernel before submitting.
>> OK, thank you.
> I recommend waiting until later this week. I hope to have a PR for other outstanding
> aspeed patches ready in a few days. It is easiest if you base your v9 on top of that
> series, this to avoid merge conflicts.

Sure, no problem. Thanks for your help.


Regards,

Jammy

>
> Regards,
>
> 	Hans
>
>>> Cheers,
>>>
>>> Joel
>>>
>>>> I will need to change the number of V4L2_CID_USER_ASPEED_BASE per different
>>>>
>>>> linux kernel as well.
>>>>
>>>>> Regards,
>>>>>
>>>>>         Hans
>>>>>
>>>>>> index b2cd155e691b..1d0dc8d86ed7 100644
>>>>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>>>> @@ -264,6 +264,23 @@ please make a proposal on the linux-media mailing list.
>>>>>>        of tiles, resulting in 32-aligned resolutions for the luminance plane
>>>>>>        and 16-aligned resolutions for the chrominance plane (with 2x2
>>>>>>        subsampling).
>>>>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>>>>> +
>>>>>> +      - ``V4L2_PIX_FMT_AJPG``
>>>>>> +      - 'AJPG'
>>>>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>>>>> +        which is generally adapted for remote KVM.
>>>>>> +        On each frame compression, I will compare the new frame with previous
>>>>>> +        one to decide which macroblock's data is changed, and only the changed
>>>>>> +        macroblocks will be compressed.
>>>>>> +
>>>>>> +        The implementation is based on AST2600 A3 datasheet, revision 0.9, which
>>>>>> +        is not publicly available. Or you can reference Video stream data format
>>>>>> +        – ASPEED mode compression of SDK_User_Guide which available on
>>>>>> +        AspeedTech-BMC/openbmc/releases.
>>>>>> +
>>>>>> +        Decoder's implementation can be found here,
>>>>>> +        `https://github.com/AspeedTech-BMC/aspeed_codec/ <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
>>>>>>
>>>>>>     .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
>>>>>>
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> index 24db33f803c0..00dde01d2f97 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> @@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>>>>>>                case V4L2_PIX_FMT_S5C_UYVY_JPG: descr = "S5C73MX interleaved UYVY/JPEG"; break;
>>>>>>                case V4L2_PIX_FMT_MT21C:        descr = "Mediatek Compressed Format"; break;
>>>>>>                case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
>>>>>> +            case V4L2_PIX_FMT_AJPG:         descr = "Aspeed JPEG"; break;
>>>>>>                default:
>>>>>>                        if (fmt->description[0])
>>>>>>                                return;
>>>>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>>>>> index 3210b3c82a4a..994eb6155ea9 100644
>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>> @@ -726,6 +726,7 @@ struct v4l2_pix_format {
>>>>>>     #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
>>>>>>     #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
>>>>>>     #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
>>>>>> +#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
>>>>>>
>>>>>>     /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
>>>>>>     #define V4L2_PIX_FMT_IPU3_SBGGR10  v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
>>>> -- 
>>>> Best Regards
>>>> Jammy
>>>>

Re: [PATCH v8 1/4] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

Hi Hans,

On 2022/1/17 下午 05:28, Hans Verkuil wrote:
> Hi Jammy,
>
> On 1/17/22 08:18, Jammy Huang wrote:
>> On 2022/1/17 上午 10:24, Joel Stanley wrote:
>>> On Mon, 17 Jan 2022 at 02:00, Jammy Huang <jammy_huang@aspeedtech.com> wrote:
>>>> Hi Hans,
>>>>
>>>> On 2022/1/14 下午 04:11, Hans Verkuil wrote:
>>>>> On 24/12/2021 02:27, Jammy Huang wrote:
>>>>>> This introduces support for the Aspeed JPEG format, where the new frame
>>>>>> can refer to previous frame to reduce the amount of compressed data. The
>>>>>> concept is similar to I/P frame of video compression. It will compare the
>>>>>> new frame with previous one to decide which macroblock's data is
>>>>>> changed, and only the changed macroblocks will be compressed.
>>>>>>
>>>>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>>>>> which is generally adapted for remote KVM.
>>>>>>
>>>>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>>>>> ---
>>>>>> v8:
>>>>>>      - Add decoder information for aspeed-jpeg
>>>>>> v7:
>>>>>>      - Add more information for aspeed-jpeg
>>>>>> v6:
>>>>>>      - Update description for new format, aspeed-jpeg, in Documentation.
>>>>>> v5:
>>>>>>      - no update
>>>>>> v4:
>>>>>>      - new
>>>>>> ---
>>>>>>     .../media/uapi/v4l/pixfmt-reserved.rst          | 17 +++++++++++++++++
>>>>>>     drivers/media/v4l2-core/v4l2-ioctl.c            |  1 +
>>>>>>     include/uapi/linux/videodev2.h                  |  1 +
>>>>>>     3 files changed, 19 insertions(+)
>>>>>>
>>>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>>> This is the wrong file! It should be:
>>>>>
>>>>> Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>>>> Thanks, I just used git format-patch to generate the cover-letter and
>>>> didn't notice this.
>>>>
>>>> It looks like the file, pixfmt-reserved.rst, has different path in
>>>> different Linux kernel,
>>>>
>>>> * 5.4:           Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>>
>>>> * 5.10/5.15: Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>>>>
>>>> 5.4 is the one I based to submit the patches.
>>>>
>>>> Could you suggest the kernel that I should based to submit these patches??
>>> You should always submit based on the latest release.
>>>
>>> There are sometimes exceptions where you will base your patches on
>>> something even newer, but generally you should grab the latest.
>>>
>>> Be sure to re-test on the latest kernel before submitting.
>> OK, thank you.
> I recommend waiting until later this week. I hope to have a PR for other outstanding
> aspeed patches ready in a few days. It is easiest if you base your v9 on top of that
> series, this to avoid merge conflicts.

Sure, no problem. Thanks for your help.


Regards,

Jammy

>
> Regards,
>
> 	Hans
>
>>> Cheers,
>>>
>>> Joel
>>>
>>>> I will need to change the number of V4L2_CID_USER_ASPEED_BASE per different
>>>>
>>>> linux kernel as well.
>>>>
>>>>> Regards,
>>>>>
>>>>>         Hans
>>>>>
>>>>>> index b2cd155e691b..1d0dc8d86ed7 100644
>>>>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>>>> @@ -264,6 +264,23 @@ please make a proposal on the linux-media mailing list.
>>>>>>        of tiles, resulting in 32-aligned resolutions for the luminance plane
>>>>>>        and 16-aligned resolutions for the chrominance plane (with 2x2
>>>>>>        subsampling).
>>>>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>>>>> +
>>>>>> +      - ``V4L2_PIX_FMT_AJPG``
>>>>>> +      - 'AJPG'
>>>>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>>>>> +        which is generally adapted for remote KVM.
>>>>>> +        On each frame compression, I will compare the new frame with previous
>>>>>> +        one to decide which macroblock's data is changed, and only the changed
>>>>>> +        macroblocks will be compressed.
>>>>>> +
>>>>>> +        The implementation is based on AST2600 A3 datasheet, revision 0.9, which
>>>>>> +        is not publicly available. Or you can reference Video stream data format
>>>>>> +        – ASPEED mode compression of SDK_User_Guide which available on
>>>>>> +        AspeedTech-BMC/openbmc/releases.
>>>>>> +
>>>>>> +        Decoder's implementation can be found here,
>>>>>> +        `https://github.com/AspeedTech-BMC/aspeed_codec/ <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
>>>>>>
>>>>>>     .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
>>>>>>
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> index 24db33f803c0..00dde01d2f97 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> @@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>>>>>>                case V4L2_PIX_FMT_S5C_UYVY_JPG: descr = "S5C73MX interleaved UYVY/JPEG"; break;
>>>>>>                case V4L2_PIX_FMT_MT21C:        descr = "Mediatek Compressed Format"; break;
>>>>>>                case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
>>>>>> +            case V4L2_PIX_FMT_AJPG:         descr = "Aspeed JPEG"; break;
>>>>>>                default:
>>>>>>                        if (fmt->description[0])
>>>>>>                                return;
>>>>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>>>>> index 3210b3c82a4a..994eb6155ea9 100644
>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>> @@ -726,6 +726,7 @@ struct v4l2_pix_format {
>>>>>>     #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
>>>>>>     #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
>>>>>>     #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
>>>>>> +#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
>>>>>>
>>>>>>     /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
>>>>>>     #define V4L2_PIX_FMT_IPU3_SBGGR10  v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
>>>> -- 
>>>> Best Regards
>>>> Jammy
>>>>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v8 1/4] media: v4l: Add definition for the Aspeed JPEG format

[Jammy Huang]

Hi Hans,

On 2022/1/17 下午 05:28, Hans Verkuil wrote:
> Hi Jammy,
>
> On 1/17/22 08:18, Jammy Huang wrote:
>> On 2022/1/17 上午 10:24, Joel Stanley wrote:
>>> On Mon, 17 Jan 2022 at 02:00, Jammy Huang <jammy_huang@aspeedtech.com> wrote:
>>>> Hi Hans,
>>>>
>>>> On 2022/1/14 下午 04:11, Hans Verkuil wrote:
>>>>> On 24/12/2021 02:27, Jammy Huang wrote:
>>>>>> This introduces support for the Aspeed JPEG format, where the new frame
>>>>>> can refer to previous frame to reduce the amount of compressed data. The
>>>>>> concept is similar to I/P frame of video compression. It will compare the
>>>>>> new frame with previous one to decide which macroblock's data is
>>>>>> changed, and only the changed macroblocks will be compressed.
>>>>>>
>>>>>> This Aspeed JPEG format is used by the video engine on Aspeed platforms,
>>>>>> which is generally adapted for remote KVM.
>>>>>>
>>>>>> Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
>>>>>> ---
>>>>>> v8:
>>>>>>      - Add decoder information for aspeed-jpeg
>>>>>> v7:
>>>>>>      - Add more information for aspeed-jpeg
>>>>>> v6:
>>>>>>      - Update description for new format, aspeed-jpeg, in Documentation.
>>>>>> v5:
>>>>>>      - no update
>>>>>> v4:
>>>>>>      - new
>>>>>> ---
>>>>>>     .../media/uapi/v4l/pixfmt-reserved.rst          | 17 +++++++++++++++++
>>>>>>     drivers/media/v4l2-core/v4l2-ioctl.c            |  1 +
>>>>>>     include/uapi/linux/videodev2.h                  |  1 +
>>>>>>     3 files changed, 19 insertions(+)
>>>>>>
>>>>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-reserved.rst b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>>> This is the wrong file! It should be:
>>>>>
>>>>> Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>>>> Thanks, I just used git format-patch to generate the cover-letter and
>>>> didn't notice this.
>>>>
>>>> It looks like the file, pixfmt-reserved.rst, has different path in
>>>> different Linux kernel,
>>>>
>>>> * 5.4:           Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>>
>>>> * 5.10/5.15: Documentation/userspace-api/media/v4l/pixfmt-reserved.rst
>>>>
>>>> 5.4 is the one I based to submit the patches.
>>>>
>>>> Could you suggest the kernel that I should based to submit these patches??
>>> You should always submit based on the latest release.
>>>
>>> There are sometimes exceptions where you will base your patches on
>>> something even newer, but generally you should grab the latest.
>>>
>>> Be sure to re-test on the latest kernel before submitting.
>> OK, thank you.
> I recommend waiting until later this week. I hope to have a PR for other outstanding
> aspeed patches ready in a few days. It is easiest if you base your v9 on top of that
> series, this to avoid merge conflicts.

Sure, no problem. Thanks for your help.


Regards,

Jammy

>
> Regards,
>
> 	Hans
>
>>> Cheers,
>>>
>>> Joel
>>>
>>>> I will need to change the number of V4L2_CID_USER_ASPEED_BASE per different
>>>>
>>>> linux kernel as well.
>>>>
>>>>> Regards,
>>>>>
>>>>>         Hans
>>>>>
>>>>>> index b2cd155e691b..1d0dc8d86ed7 100644
>>>>>> --- a/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>>>> +++ b/Documentation/media/uapi/v4l/pixfmt-reserved.rst
>>>>>> @@ -264,6 +264,23 @@ please make a proposal on the linux-media mailing list.
>>>>>>        of tiles, resulting in 32-aligned resolutions for the luminance plane
>>>>>>        and 16-aligned resolutions for the chrominance plane (with 2x2
>>>>>>        subsampling).
>>>>>> +    * .. _V4L2-PIX-FMT-AJPG:
>>>>>> +
>>>>>> +      - ``V4L2_PIX_FMT_AJPG``
>>>>>> +      - 'AJPG'
>>>>>> +      - ASPEED JPEG format used by the aspeed-video driver on Aspeed platforms,
>>>>>> +        which is generally adapted for remote KVM.
>>>>>> +        On each frame compression, I will compare the new frame with previous
>>>>>> +        one to decide which macroblock's data is changed, and only the changed
>>>>>> +        macroblocks will be compressed.
>>>>>> +
>>>>>> +        The implementation is based on AST2600 A3 datasheet, revision 0.9, which
>>>>>> +        is not publicly available. Or you can reference Video stream data format
>>>>>> +        – ASPEED mode compression of SDK_User_Guide which available on
>>>>>> +        AspeedTech-BMC/openbmc/releases.
>>>>>> +
>>>>>> +        Decoder's implementation can be found here,
>>>>>> +        `https://github.com/AspeedTech-BMC/aspeed_codec/ <https://github.com/AspeedTech-BMC/aspeed_codec/>`__
>>>>>>
>>>>>>     .. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
>>>>>>
>>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> index 24db33f803c0..00dde01d2f97 100644
>>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>>> @@ -1378,6 +1378,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>>>>>>                case V4L2_PIX_FMT_S5C_UYVY_JPG: descr = "S5C73MX interleaved UYVY/JPEG"; break;
>>>>>>                case V4L2_PIX_FMT_MT21C:        descr = "Mediatek Compressed Format"; break;
>>>>>>                case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
>>>>>> +            case V4L2_PIX_FMT_AJPG:         descr = "Aspeed JPEG"; break;
>>>>>>                default:
>>>>>>                        if (fmt->description[0])
>>>>>>                                return;
>>>>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>>>>> index 3210b3c82a4a..994eb6155ea9 100644
>>>>>> --- a/include/uapi/linux/videodev2.h
>>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>>> @@ -726,6 +726,7 @@ struct v4l2_pix_format {
>>>>>>     #define V4L2_PIX_FMT_INZI     v4l2_fourcc('I', 'N', 'Z', 'I') /* Intel Planar Greyscale 10-bit and Depth 16-bit */
>>>>>>     #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
>>>>>>     #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
>>>>>> +#define V4L2_PIX_FMT_AJPG     v4l2_fourcc('A', 'J', 'P', 'G') /* Aspeed JPEG */
>>>>>>
>>>>>>     /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
>>>>>>     #define V4L2_PIX_FMT_IPU3_SBGGR10  v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
>>>> -- 
>>>> Best Regards
>>>> Jammy
>>>>

[PATCH v8 2/4] media: v4l2-ctrls: Reserve controls for ASPEED

[Jammy Huang]

Reserve controls for ASPEED video family. Aspeed video engine contains a
few features which improve video quality, reduce amount of compressed
data, and etc. Hence, 16 controls are reserved for these aspeed
proprietary features.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v8:
  - no update
v7:
  - no update
v6:
  - no update
v5:
  - no update
v4:
  - new
---
 include/uapi/linux/v4l2-controls.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index a2669b79b294..e7c425a6fa9f 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -192,6 +192,11 @@ enum v4l2_colorfx {
  * We reserve 16 controls for this driver. */
 #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
 
+/* The base for the aspeed driver controls.
+ * We reserve 16 controls for this driver.
+ */
+#define V4L2_CID_USER_ASPEED_BASE		(V4L2_CID_USER_BASE + 0x10c0)
+
 /* MPEG-class control IDs */
 /* The MPEG controls are applicable to all codec controls
  * and the 'MPEG' part of the define is historical */
-- 
2.25.1

[PATCH v8 2/4] media: v4l2-ctrls: Reserve controls for ASPEED

[Jammy Huang]

Reserve controls for ASPEED video family. Aspeed video engine contains a
few features which improve video quality, reduce amount of compressed
data, and etc. Hence, 16 controls are reserved for these aspeed
proprietary features.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v8:
  - no update
v7:
  - no update
v6:
  - no update
v5:
  - no update
v4:
  - new
---
 include/uapi/linux/v4l2-controls.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index a2669b79b294..e7c425a6fa9f 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -192,6 +192,11 @@ enum v4l2_colorfx {
  * We reserve 16 controls for this driver. */
 #define V4L2_CID_USER_IMX_BASE			(V4L2_CID_USER_BASE + 0x10b0)
 
+/* The base for the aspeed driver controls.
+ * We reserve 16 controls for this driver.
+ */
+#define V4L2_CID_USER_ASPEED_BASE		(V4L2_CID_USER_BASE + 0x10c0)
+
 /* MPEG-class control IDs */
 /* The MPEG controls are applicable to all codec controls
  * and the 'MPEG' part of the define is historical */
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 3/4] media: aspeed: Support aspeed mode to reduce compressed data

[Jammy Huang]

aspeed supports differential jpeg format which only compress the parts
which are changed. In this way, it reduces both the amount of data to be
transferred by network and those to be decoded on the client side.

3 new ctrls are added:
* Aspeed Compression Scheme: to control aspeed's compression scheme
  0: DCT Only, 1: DCT VQ mix 2-color, 2: DCT VQ mix 4-color
  This is AST2400 only. It will adapt JPEG or VQ encoding method according
  to the context automatically.
* Aspeed HQ Mode: to control aspeed's high quality(2-pass) compression mode
  This only works with yuv444 subsampling.
* Aspeed HQ Quality: to control the quality of aspeed's HQ mode
  only useful if Aspeed HQ mode is enabled

Aspeed JPEG Format requires an additional buffer, called bcd, to store
the information about which macro block in the new frame is different
from the previous one.

To have bcd correctly working, we need to swap the buffers for src0/1 to
make src1 refer to previous frame and src0 to the coming new frame.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v8:
  - no update
v7:
  - rename compression_mode as compression_scheme
  - add reset bcd buffer every 8 frames to generate an I frame
  - update commit message
v6:
  - no update
v5:
  - use runtime configuration to handle differences between soc
  - add aspeed_video_swap_src_buf().
v4
  - add include/uapi/linux/aspeed-video.h for V4L2_CID_ASPEED_XXX
  - add aspeed_video_set_format
  - remove aspeed_ctrl_format
  - add variable, format, to replace partial_jpeg
  - modify aspeed_ctrl_HQ_jpeg_quality's range
  - reduce bcd buffer size
  - update commit message
v3:
  - 'compression mode' only available for G4
v2:
  - update commit message
  - use v4l2_xxx log rather than self-defined macro
---
 drivers/media/platform/aspeed-video.c | 252 ++++++++++++++++++++++++--
 include/uapi/linux/aspeed-video.h     |  15 ++
 2 files changed, 248 insertions(+), 19 deletions(-)
 create mode 100644 include/uapi/linux/aspeed-video.h

diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index 2f7451f06c24..c0cfcf98659d 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -31,6 +31,7 @@
 #include <media/v4l2-ioctl.h>
 #include <media/videobuf2-dma-contig.h>
 #include <linux/videodev2.h>
+#include <uapi/linux/aspeed-video.h>
 
 #define ASPEED_VIDEO_V4L2_MIN_BUF_REQ 3
 
@@ -57,6 +58,7 @@
 
 #define VE_MAX_SRC_BUFFER_SIZE		0x8ca000 /* 1920 * 1200, 32bpp */
 #define VE_JPEG_HEADER_SIZE		0x006000 /* 512 * 12 * 4 */
+#define VE_BCD_BUFF_SIZE		0x9000 /* (1920/8) * (1200/8) */
 
 #define VE_PROTECTION_KEY		0x000
 #define  VE_PROTECTION_KEY_UNLOCK	0x1a038aa8
@@ -107,6 +109,13 @@
 #define VE_SCALING_FILTER2		0x020
 #define VE_SCALING_FILTER3		0x024
 
+#define VE_BCD_CTRL			0x02C
+#define  VE_BCD_CTRL_EN_BCD		BIT(0)
+#define  VE_BCD_CTRL_EN_ABCD		BIT(1)
+#define  VE_BCD_CTRL_EN_CB		BIT(2)
+#define  VE_BCD_CTRL_THR		GENMASK(23, 16)
+#define  VE_BCD_CTRL_ABCD_THR		GENMASK(31, 24)
+
 #define VE_CAP_WINDOW			0x030
 #define VE_COMP_WINDOW			0x034
 #define VE_COMP_PROC_OFFSET		0x038
@@ -115,6 +124,7 @@
 #define VE_SRC0_ADDR			0x044
 #define VE_SRC_SCANLINE_OFFSET		0x048
 #define VE_SRC1_ADDR			0x04c
+#define VE_BCD_ADDR			0x050
 #define VE_COMP_ADDR			0x054
 
 #define VE_STREAM_BUF_SIZE		0x058
@@ -135,6 +145,8 @@
 #define  VE_COMP_CTRL_HQ_DCT_CHR	GENMASK(26, 22)
 #define  VE_COMP_CTRL_HQ_DCT_LUM	GENMASK(31, 27)
 
+#define VE_CB_ADDR			0x06C
+
 #define AST2400_VE_COMP_SIZE_READ_BACK	0x078
 #define AST2600_VE_COMP_SIZE_READ_BACK	0x084
 
@@ -202,6 +214,12 @@ enum {
 	VIDEO_CLOCKS_ON,
 };
 
+enum aspeed_video_format {
+	VIDEO_FMT_STANDARD = 0,
+	VIDEO_FMT_ASPEED,
+	VIDEO_FMT_MAX = VIDEO_FMT_ASPEED
+};
+
 struct aspeed_video_addr {
 	unsigned int size;
 	dma_addr_t dma;
@@ -234,9 +252,14 @@ struct aspeed_video_perf {
  * @max_compressed_size:holds max compressed stream's size
  * @srcs:		holds the buffer information for srcs
  * @jpeg:		holds the buffer information for jpeg header
+ * @bcd:		holds the buffer information for bcd work
  * @yuv420:		a flag raised if JPEG subsampling is 420
+ * @format:		holds the video format
+ * @hq_mode:		a flag raised if HQ is enabled. Only for VIDEO_FMT_ASPEED
  * @frame_rate:		holds the frame_rate
  * @jpeg_quality:	holds jpeq's quality (0~11)
+ * @jpeg_hq_quality:	holds hq's quality (1~12) only if hq_mode enabled
+ * @compression_scheme:	holds image coding scheme of aspeed jpeg
  * @frame_bottom:	end position of video data in vertical direction
  * @frame_left:		start position of video data in horizontal direction
  * @frame_right:	end position of video data in horizontal direction
@@ -271,10 +294,15 @@ struct aspeed_video {
 	unsigned int max_compressed_size;
 	struct aspeed_video_addr srcs[2];
 	struct aspeed_video_addr jpeg;
+	struct aspeed_video_addr bcd;
 
 	bool yuv420;
+	enum aspeed_video_format format;
+	bool hq_mode;
 	unsigned int frame_rate;
 	unsigned int jpeg_quality;
+	unsigned int jpeg_hq_quality;
+	unsigned int compression_scheme;
 
 	unsigned int frame_bottom;
 	unsigned int frame_left;
@@ -443,8 +471,20 @@ static const struct v4l2_dv_timings_cap aspeed_video_timings_cap = {
 	},
 };
 
+static const char * const compress_scheme_str[] = {"DCT Only",
+	"DCT VQ mix 2-color", "DCT VQ mix 4-color"};
+static const char * const format_str[] = {"Standard JPEG",
+	"Aspeed JPEG"};
+
 static unsigned int debug;
 
+static bool aspeed_video_alloc_buf(struct aspeed_video *video,
+				   struct aspeed_video_addr *addr,
+				   unsigned int size);
+
+static void aspeed_video_free_buf(struct aspeed_video *video,
+				  struct aspeed_video_addr *addr);
+
 static void aspeed_video_init_jpeg_table(u32 *table, bool yuv420)
 {
 	int i;
@@ -532,6 +572,7 @@ static int aspeed_video_start_frame(struct aspeed_video *video)
 	unsigned long flags;
 	struct aspeed_video_buffer *buf;
 	u32 seq_ctrl = aspeed_video_read(video, VE_SEQ_CTRL);
+	bool bcd_buf_need = (video->format != VIDEO_FMT_STANDARD);
 
 	if (video->v4l2_input_status) {
 		v4l2_warn(&video->v4l2_dev, "No signal; don't start frame\n");
@@ -544,6 +585,20 @@ static int aspeed_video_start_frame(struct aspeed_video *video)
 		return -EBUSY;
 	}
 
+	if (bcd_buf_need && !video->bcd.size) {
+		if (!aspeed_video_alloc_buf(video, &video->bcd,
+					    VE_BCD_BUFF_SIZE)) {
+			dev_err(video->dev, "Failed to allocate BCD buffer\n");
+			dev_err(video->dev, "don't start frame\n");
+			return -ENOMEM;
+		}
+		aspeed_video_write(video, VE_BCD_ADDR, video->bcd.dma);
+		v4l2_dbg(1, debug, &video->v4l2_dev, "bcd addr(%#x) size(%d)\n",
+			 video->bcd.dma, video->bcd.size);
+	} else if (!bcd_buf_need && video->bcd.size) {
+		aspeed_video_free_buf(video, &video->bcd);
+	}
+
 	spin_lock_irqsave(&video->lock, flags);
 	buf = list_first_entry_or_null(&video->buffers,
 				       struct aspeed_video_buffer, link);
@@ -640,6 +695,24 @@ static void aspeed_video_irq_res_change(struct aspeed_video *video, ulong delay)
 	schedule_delayed_work(&video->res_work, delay);
 }
 
+static void aspeed_video_swap_src_buf(struct aspeed_video *v)
+{
+	if (v->format == VIDEO_FMT_STANDARD)
+		return;
+
+	/* Reset bcd buffer to have a full frame update every 8 frames.  */
+	if (IS_ALIGNED(v->sequence, 8))
+		memset((u8 *)v->bcd.virt, 0x00, VE_BCD_BUFF_SIZE);
+
+	if (v->sequence & 0x01) {
+		aspeed_video_write(v, VE_SRC0_ADDR, v->srcs[1].dma);
+		aspeed_video_write(v, VE_SRC1_ADDR, v->srcs[0].dma);
+	} else {
+		aspeed_video_write(v, VE_SRC0_ADDR, v->srcs[0].dma);
+		aspeed_video_write(v, VE_SRC1_ADDR, v->srcs[1].dma);
+	}
+}
+
 /*
  * Interrupts that we don't use but have to explicitly ignore because the
  * hardware asserts them even when they're disabled in the VE_INTERRUPT_CTRL
@@ -691,6 +764,7 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
 
 	if (sts & VE_INTERRUPT_COMP_COMPLETE) {
 		struct aspeed_video_buffer *buf;
+		bool empty = true;
 		u32 frame_size = aspeed_video_read(video,
 						   video->config.comp_size_read);
 
@@ -704,13 +778,23 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
 		if (buf) {
 			vb2_set_plane_payload(&buf->vb.vb2_buf, 0, frame_size);
 
-			if (!list_is_last(&buf->link, &video->buffers)) {
+			/*
+			 * aspeed_jpeg requires continuous update.
+			 * On the contrary, standard jpeg can keep last buffer
+			 * to always have the latest result.
+			 */
+			if (video->format == VIDEO_FMT_STANDARD &&
+			    list_is_last(&buf->link, &video->buffers)) {
+				empty = false;
+				v4l2_warn(&video->v4l2_dev, "skip to keep last frame updated\n");
+			} else {
 				buf->vb.vb2_buf.timestamp = ktime_get_ns();
 				buf->vb.sequence = video->sequence++;
 				buf->vb.field = V4L2_FIELD_NONE;
 				vb2_buffer_done(&buf->vb.vb2_buf,
 						VB2_BUF_STATE_DONE);
 				list_del(&buf->link);
+				empty = list_empty(&video->buffers);
 			}
 		}
 		spin_unlock(&video->lock);
@@ -724,7 +808,10 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
 		aspeed_video_write(video, VE_INTERRUPT_STATUS,
 				   VE_INTERRUPT_COMP_COMPLETE);
 		sts &= ~VE_INTERRUPT_COMP_COMPLETE;
-		if (test_bit(VIDEO_STREAMING, &video->flags) && buf)
+
+		aspeed_video_swap_src_buf(video);
+
+		if (test_bit(VIDEO_STREAMING, &video->flags) && !empty)
 			aspeed_video_start_frame(video);
 	}
 
@@ -983,10 +1070,14 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
 				   FIELD_PREP(VE_TGS_FIRST, video->frame_top) |
 				   FIELD_PREP(VE_TGS_LAST,
 					      video->frame_bottom + 1));
-		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_INT_DE);
+		aspeed_video_update(video, VE_CTRL,
+				    VE_CTRL_INT_DE | VE_CTRL_DIRECT_FETCH,
+				    VE_CTRL_INT_DE);
 	} else {
 		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Direct Mode\n");
-		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_DIRECT_FETCH);
+		aspeed_video_update(video, VE_CTRL,
+				    VE_CTRL_INT_DE | VE_CTRL_DIRECT_FETCH,
+				    VE_CTRL_DIRECT_FETCH);
 	}
 
 	size *= 4;
@@ -1021,35 +1112,71 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
 
 static void aspeed_video_update_regs(struct aspeed_video *video)
 {
-	u32 comp_ctrl = VE_COMP_CTRL_RSVD |
-		FIELD_PREP(VE_COMP_CTRL_DCT_LUM, video->jpeg_quality) |
-		FIELD_PREP(VE_COMP_CTRL_DCT_CHR, video->jpeg_quality | 0x10);
+	u8 jpeg_hq_quality = clamp((int)video->jpeg_hq_quality - 1, 0,
+				   ASPEED_VIDEO_JPEG_NUM_QUALITIES - 1);
+	u32 comp_ctrl =	FIELD_PREP(VE_COMP_CTRL_DCT_LUM, video->jpeg_quality) |
+		FIELD_PREP(VE_COMP_CTRL_DCT_CHR, video->jpeg_quality | 0x10) |
+		FIELD_PREP(VE_COMP_CTRL_EN_HQ, video->hq_mode) |
+		FIELD_PREP(VE_COMP_CTRL_HQ_DCT_LUM, jpeg_hq_quality) |
+		FIELD_PREP(VE_COMP_CTRL_HQ_DCT_CHR, jpeg_hq_quality | 0x10);
 	u32 ctrl = 0;
-	u32 seq_ctrl = video->config.jpeg_mode;
+	u32 seq_ctrl = 0;
 
-	v4l2_dbg(1, debug, &video->v4l2_dev, "framerate(%d)\n",
-		 video->frame_rate);
-	v4l2_dbg(1, debug, &video->v4l2_dev, "subsample(%s)\n",
+	v4l2_dbg(1, debug, &video->v4l2_dev, "framerate(%d)\n", video->frame_rate);
+	v4l2_dbg(1, debug, &video->v4l2_dev, "jpeg format(%s) subsample(%s)\n",
+		 format_str[video->format],
 		 video->yuv420 ? "420" : "444");
 	v4l2_dbg(1, debug, &video->v4l2_dev, "compression quality(%d)\n",
 		 video->jpeg_quality);
+	v4l2_dbg(1, debug, &video->v4l2_dev, "hq_mode(%s) hq_quality(%d)\n",
+		 video->hq_mode ? "on" : "off", video->jpeg_hq_quality);
+	v4l2_dbg(1, debug, &video->v4l2_dev, "compression scheme(%s)\n",
+		 compress_scheme_str[video->compression_scheme]);
+
+	if (video->format == VIDEO_FMT_ASPEED)
+		aspeed_video_update(video, VE_BCD_CTRL, 0, VE_BCD_CTRL_EN_BCD);
+	else
+		aspeed_video_update(video, VE_BCD_CTRL, VE_BCD_CTRL_EN_BCD, 0);
 
 	if (video->frame_rate)
 		ctrl |= FIELD_PREP(VE_CTRL_FRC, video->frame_rate);
 
+	if (video->format == VIDEO_FMT_STANDARD) {
+		comp_ctrl &= ~FIELD_PREP(VE_COMP_CTRL_EN_HQ, video->hq_mode);
+		seq_ctrl |= video->config.jpeg_mode;
+	}
+
 	if (video->yuv420)
 		seq_ctrl |= VE_SEQ_CTRL_YUV420;
 
 	if (video->jpeg.virt)
 		aspeed_video_update_jpeg_table(video->jpeg.virt, video->yuv420);
 
+	if (video->config.version == 4) {
+		switch (video->compression_scheme) {
+		case 0:	//DCT only
+			comp_ctrl |= VE_COMP_CTRL_VQ_DCT_ONLY;
+			break;
+		case 1:	//DCT VQ mix 2-color
+			comp_ctrl &= ~(VE_COMP_CTRL_VQ_4COLOR |
+				     VE_COMP_CTRL_VQ_DCT_ONLY);
+			break;
+		case 2:	//DCT VQ mix 4-color
+			comp_ctrl |= VE_COMP_CTRL_VQ_4COLOR;
+			break;
+		}
+	}
+
 	/* Set control registers */
 	aspeed_video_update(video, VE_SEQ_CTRL,
 			    video->config.jpeg_mode | VE_SEQ_CTRL_YUV420,
 			    seq_ctrl);
 	aspeed_video_update(video, VE_CTRL, VE_CTRL_FRC, ctrl);
 	aspeed_video_update(video, VE_COMP_CTRL,
-			    VE_COMP_CTRL_DCT_LUM | VE_COMP_CTRL_DCT_CHR,
+			    VE_COMP_CTRL_DCT_LUM | VE_COMP_CTRL_DCT_CHR |
+			    VE_COMP_CTRL_EN_HQ | VE_COMP_CTRL_HQ_DCT_LUM |
+			    VE_COMP_CTRL_HQ_DCT_CHR | VE_COMP_CTRL_VQ_4COLOR |
+			    VE_COMP_CTRL_VQ_DCT_ONLY,
 			    comp_ctrl);
 }
 
@@ -1081,6 +1208,8 @@ static void aspeed_video_init_regs(struct aspeed_video *video)
 
 	/* Set mode detection defaults */
 	aspeed_video_write(video, VE_MODE_DETECT, 0x22666500);
+
+	aspeed_video_write(video, VE_BCD_CTRL, 0);
 }
 
 static void aspeed_video_start(struct aspeed_video *video)
@@ -1114,6 +1243,9 @@ static void aspeed_video_stop(struct aspeed_video *video)
 	if (video->srcs[1].size)
 		aspeed_video_free_buf(video, &video->srcs[1]);
 
+	if (video->bcd.size)
+		aspeed_video_free_buf(video, &video->bcd);
+
 	video->v4l2_input_status = V4L2_IN_ST_NO_SIGNAL;
 	video->flags = 0;
 }
@@ -1132,10 +1264,12 @@ static int aspeed_video_querycap(struct file *file, void *fh,
 static int aspeed_video_enum_format(struct file *file, void *fh,
 				    struct v4l2_fmtdesc *f)
 {
+	struct aspeed_video *video = video_drvdata(file);
+
 	if (f->index)
 		return -EINVAL;
 
-	f->pixelformat = V4L2_PIX_FMT_JPEG;
+	f->pixelformat = video->pix_fmt.pixelformat;
 
 	return 0;
 }
@@ -1150,6 +1284,29 @@ static int aspeed_video_get_format(struct file *file, void *fh,
 	return 0;
 }
 
+static int aspeed_video_set_format(struct file *file, void *fh,
+				   struct v4l2_format *f)
+{
+	struct aspeed_video *video = video_drvdata(file);
+
+	if (vb2_is_busy(&video->queue))
+		return -EBUSY;
+
+	switch (f->fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_JPEG:
+		video->format = VIDEO_FMT_STANDARD;
+		break;
+	case V4L2_PIX_FMT_AJPG:
+		video->format = VIDEO_FMT_ASPEED;
+		break;
+	default:
+		return -EINVAL;
+	}
+	video->pix_fmt.pixelformat = f->fmt.pix.pixelformat;
+
+	return 0;
+}
+
 static int aspeed_video_enum_input(struct file *file, void *fh,
 				   struct v4l2_input *inp)
 {
@@ -1367,7 +1524,7 @@ static const struct v4l2_ioctl_ops aspeed_video_ioctl_ops = {
 
 	.vidioc_enum_fmt_vid_cap = aspeed_video_enum_format,
 	.vidioc_g_fmt_vid_cap = aspeed_video_get_format,
-	.vidioc_s_fmt_vid_cap = aspeed_video_get_format,
+	.vidioc_s_fmt_vid_cap = aspeed_video_set_format,
 	.vidioc_try_fmt_vid_cap = aspeed_video_get_format,
 
 	.vidioc_reqbufs = vb2_ioctl_reqbufs,
@@ -1416,6 +1573,24 @@ static int aspeed_video_set_ctrl(struct v4l2_ctrl *ctrl)
 		if (test_bit(VIDEO_STREAMING, &video->flags))
 			aspeed_video_update_regs(video);
 		break;
+	case V4L2_CID_ASPEED_COMPRESSION_SCHEME:
+		if (video->config.version > 4)
+			return -EINVAL;
+
+		video->compression_scheme = ctrl->val;
+		if (test_bit(VIDEO_STREAMING, &video->flags))
+			aspeed_video_update_regs(video);
+		break;
+	case V4L2_CID_ASPEED_HQ_MODE:
+		video->hq_mode = ctrl->val;
+		if (test_bit(VIDEO_STREAMING, &video->flags))
+			aspeed_video_update_regs(video);
+		break;
+	case V4L2_CID_ASPEED_HQ_JPEG_QUALITY:
+		video->jpeg_hq_quality = ctrl->val;
+		if (test_bit(VIDEO_STREAMING, &video->flags))
+			aspeed_video_update_regs(video);
+		break;
 	default:
 		return -EINVAL;
 	}
@@ -1427,6 +1602,39 @@ static const struct v4l2_ctrl_ops aspeed_video_ctrl_ops = {
 	.s_ctrl = aspeed_video_set_ctrl,
 };
 
+static const struct v4l2_ctrl_config aspeed_ctrl_compression_scheme = {
+	.ops = &aspeed_video_ctrl_ops,
+	.id = V4L2_CID_ASPEED_COMPRESSION_SCHEME,
+	.name = "Aspeed Compression SCHEME",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = 0,
+	.max = 2,
+	.step = 1,
+	.def = 0,
+};
+
+static const struct v4l2_ctrl_config aspeed_ctrl_HQ_mode = {
+	.ops = &aspeed_video_ctrl_ops,
+	.id = V4L2_CID_ASPEED_HQ_MODE,
+	.name = "Aspeed HQ Mode",
+	.type = V4L2_CTRL_TYPE_BOOLEAN,
+	.min = false,
+	.max = true,
+	.step = 1,
+	.def = false,
+};
+
+static const struct v4l2_ctrl_config aspeed_ctrl_HQ_jpeg_quality = {
+	.ops = &aspeed_video_ctrl_ops,
+	.id = V4L2_CID_ASPEED_HQ_JPEG_QUALITY,
+	.name = "Aspeed HQ Quality",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = 1,
+	.max = ASPEED_VIDEO_JPEG_NUM_QUALITIES,
+	.step = 1,
+	.def = 1,
+};
+
 static void aspeed_video_resolution_work(struct work_struct *work)
 {
 	struct delayed_work *dwork = to_delayed_work(work);
@@ -1693,6 +1901,7 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
 	struct v4l2_device *v4l2_dev = &video->v4l2_dev;
 	struct vb2_queue *vbq = &video->queue;
 	struct video_device *vdev = &video->vdev;
+	struct v4l2_ctrl_handler *hdl = &video->ctrl_handler;
 	int rc;
 
 	video->pix_fmt.pixelformat = V4L2_PIX_FMT_JPEG;
@@ -1707,22 +1916,26 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
 		return rc;
 	}
 
-	v4l2_ctrl_handler_init(&video->ctrl_handler, 2);
-	v4l2_ctrl_new_std(&video->ctrl_handler, &aspeed_video_ctrl_ops,
+	v4l2_ctrl_handler_init(hdl, 5);
+	v4l2_ctrl_new_std(hdl, &aspeed_video_ctrl_ops,
 			  V4L2_CID_JPEG_COMPRESSION_QUALITY, 0,
 			  ASPEED_VIDEO_JPEG_NUM_QUALITIES - 1, 1, 0);
-	v4l2_ctrl_new_std_menu(&video->ctrl_handler, &aspeed_video_ctrl_ops,
+	v4l2_ctrl_new_std_menu(hdl, &aspeed_video_ctrl_ops,
 			       V4L2_CID_JPEG_CHROMA_SUBSAMPLING,
 			       V4L2_JPEG_CHROMA_SUBSAMPLING_420, mask,
 			       V4L2_JPEG_CHROMA_SUBSAMPLING_444);
+	if (video->config.version == 4)
+		v4l2_ctrl_new_custom(hdl, &aspeed_ctrl_compression_scheme, NULL);
+	v4l2_ctrl_new_custom(hdl, &aspeed_ctrl_HQ_mode, NULL);
+	v4l2_ctrl_new_custom(hdl, &aspeed_ctrl_HQ_jpeg_quality, NULL);
 
-	rc = video->ctrl_handler.error;
+	rc = hdl->error;
 	if (rc) {
 		dev_err(video->dev, "Failed to init controls: %d\n", rc);
 		goto err_ctrl_init;
 	}
 
-	v4l2_dev->ctrl_handler = &video->ctrl_handler;
+	v4l2_dev->ctrl_handler = hdl;
 
 	vbq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 	vbq->io_modes = VB2_MMAP | VB2_READ | VB2_DMABUF;
@@ -1863,6 +2076,7 @@ static int aspeed_video_probe(struct platform_device *pdev)
 		return -ENOMEM;
 
 	video->frame_rate = 30;
+	video->jpeg_hq_quality = 1;
 	video->dev = &pdev->dev;
 	spin_lock_init(&video->lock);
 	mutex_init(&video->video_lock);
diff --git a/include/uapi/linux/aspeed-video.h b/include/uapi/linux/aspeed-video.h
new file mode 100644
index 000000000000..63f0432192a5
--- /dev/null
+++ b/include/uapi/linux/aspeed-video.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2021 ASPEED Technology Inc.
+ */
+
+#ifndef _UAPI_LINUX_ASPEED_VIDEO_H
+#define _UAPI_LINUX_ASPEED_VIDEO_H
+
+#include <linux/v4l2-controls.h>
+
+#define V4L2_CID_ASPEED_COMPRESSION_SCHEME	(V4L2_CID_USER_ASPEED_BASE  + 1)
+#define V4L2_CID_ASPEED_HQ_MODE			(V4L2_CID_USER_ASPEED_BASE  + 2)
+#define V4L2_CID_ASPEED_HQ_JPEG_QUALITY		(V4L2_CID_USER_ASPEED_BASE  + 3)
+
+#endif /* _UAPI_LINUX_ASPEED_VIDEO_H */
-- 
2.25.1

[PATCH v8 3/4] media: aspeed: Support aspeed mode to reduce compressed data

[Jammy Huang]

aspeed supports differential jpeg format which only compress the parts
which are changed. In this way, it reduces both the amount of data to be
transferred by network and those to be decoded on the client side.

3 new ctrls are added:
* Aspeed Compression Scheme: to control aspeed's compression scheme
  0: DCT Only, 1: DCT VQ mix 2-color, 2: DCT VQ mix 4-color
  This is AST2400 only. It will adapt JPEG or VQ encoding method according
  to the context automatically.
* Aspeed HQ Mode: to control aspeed's high quality(2-pass) compression mode
  This only works with yuv444 subsampling.
* Aspeed HQ Quality: to control the quality of aspeed's HQ mode
  only useful if Aspeed HQ mode is enabled

Aspeed JPEG Format requires an additional buffer, called bcd, to store
the information about which macro block in the new frame is different
from the previous one.

To have bcd correctly working, we need to swap the buffers for src0/1 to
make src1 refer to previous frame and src0 to the coming new frame.

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v8:
  - no update
v7:
  - rename compression_mode as compression_scheme
  - add reset bcd buffer every 8 frames to generate an I frame
  - update commit message
v6:
  - no update
v5:
  - use runtime configuration to handle differences between soc
  - add aspeed_video_swap_src_buf().
v4
  - add include/uapi/linux/aspeed-video.h for V4L2_CID_ASPEED_XXX
  - add aspeed_video_set_format
  - remove aspeed_ctrl_format
  - add variable, format, to replace partial_jpeg
  - modify aspeed_ctrl_HQ_jpeg_quality's range
  - reduce bcd buffer size
  - update commit message
v3:
  - 'compression mode' only available for G4
v2:
  - update commit message
  - use v4l2_xxx log rather than self-defined macro
---
 drivers/media/platform/aspeed-video.c | 252 ++++++++++++++++++++++++--
 include/uapi/linux/aspeed-video.h     |  15 ++
 2 files changed, 248 insertions(+), 19 deletions(-)
 create mode 100644 include/uapi/linux/aspeed-video.h

diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index 2f7451f06c24..c0cfcf98659d 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -31,6 +31,7 @@
 #include <media/v4l2-ioctl.h>
 #include <media/videobuf2-dma-contig.h>
 #include <linux/videodev2.h>
+#include <uapi/linux/aspeed-video.h>
 
 #define ASPEED_VIDEO_V4L2_MIN_BUF_REQ 3
 
@@ -57,6 +58,7 @@
 
 #define VE_MAX_SRC_BUFFER_SIZE		0x8ca000 /* 1920 * 1200, 32bpp */
 #define VE_JPEG_HEADER_SIZE		0x006000 /* 512 * 12 * 4 */
+#define VE_BCD_BUFF_SIZE		0x9000 /* (1920/8) * (1200/8) */
 
 #define VE_PROTECTION_KEY		0x000
 #define  VE_PROTECTION_KEY_UNLOCK	0x1a038aa8
@@ -107,6 +109,13 @@
 #define VE_SCALING_FILTER2		0x020
 #define VE_SCALING_FILTER3		0x024
 
+#define VE_BCD_CTRL			0x02C
+#define  VE_BCD_CTRL_EN_BCD		BIT(0)
+#define  VE_BCD_CTRL_EN_ABCD		BIT(1)
+#define  VE_BCD_CTRL_EN_CB		BIT(2)
+#define  VE_BCD_CTRL_THR		GENMASK(23, 16)
+#define  VE_BCD_CTRL_ABCD_THR		GENMASK(31, 24)
+
 #define VE_CAP_WINDOW			0x030
 #define VE_COMP_WINDOW			0x034
 #define VE_COMP_PROC_OFFSET		0x038
@@ -115,6 +124,7 @@
 #define VE_SRC0_ADDR			0x044
 #define VE_SRC_SCANLINE_OFFSET		0x048
 #define VE_SRC1_ADDR			0x04c
+#define VE_BCD_ADDR			0x050
 #define VE_COMP_ADDR			0x054
 
 #define VE_STREAM_BUF_SIZE		0x058
@@ -135,6 +145,8 @@
 #define  VE_COMP_CTRL_HQ_DCT_CHR	GENMASK(26, 22)
 #define  VE_COMP_CTRL_HQ_DCT_LUM	GENMASK(31, 27)
 
+#define VE_CB_ADDR			0x06C
+
 #define AST2400_VE_COMP_SIZE_READ_BACK	0x078
 #define AST2600_VE_COMP_SIZE_READ_BACK	0x084
 
@@ -202,6 +214,12 @@ enum {
 	VIDEO_CLOCKS_ON,
 };
 
+enum aspeed_video_format {
+	VIDEO_FMT_STANDARD = 0,
+	VIDEO_FMT_ASPEED,
+	VIDEO_FMT_MAX = VIDEO_FMT_ASPEED
+};
+
 struct aspeed_video_addr {
 	unsigned int size;
 	dma_addr_t dma;
@@ -234,9 +252,14 @@ struct aspeed_video_perf {
  * @max_compressed_size:holds max compressed stream's size
  * @srcs:		holds the buffer information for srcs
  * @jpeg:		holds the buffer information for jpeg header
+ * @bcd:		holds the buffer information for bcd work
  * @yuv420:		a flag raised if JPEG subsampling is 420
+ * @format:		holds the video format
+ * @hq_mode:		a flag raised if HQ is enabled. Only for VIDEO_FMT_ASPEED
  * @frame_rate:		holds the frame_rate
  * @jpeg_quality:	holds jpeq's quality (0~11)
+ * @jpeg_hq_quality:	holds hq's quality (1~12) only if hq_mode enabled
+ * @compression_scheme:	holds image coding scheme of aspeed jpeg
  * @frame_bottom:	end position of video data in vertical direction
  * @frame_left:		start position of video data in horizontal direction
  * @frame_right:	end position of video data in horizontal direction
@@ -271,10 +294,15 @@ struct aspeed_video {
 	unsigned int max_compressed_size;
 	struct aspeed_video_addr srcs[2];
 	struct aspeed_video_addr jpeg;
+	struct aspeed_video_addr bcd;
 
 	bool yuv420;
+	enum aspeed_video_format format;
+	bool hq_mode;
 	unsigned int frame_rate;
 	unsigned int jpeg_quality;
+	unsigned int jpeg_hq_quality;
+	unsigned int compression_scheme;
 
 	unsigned int frame_bottom;
 	unsigned int frame_left;
@@ -443,8 +471,20 @@ static const struct v4l2_dv_timings_cap aspeed_video_timings_cap = {
 	},
 };
 
+static const char * const compress_scheme_str[] = {"DCT Only",
+	"DCT VQ mix 2-color", "DCT VQ mix 4-color"};
+static const char * const format_str[] = {"Standard JPEG",
+	"Aspeed JPEG"};
+
 static unsigned int debug;
 
+static bool aspeed_video_alloc_buf(struct aspeed_video *video,
+				   struct aspeed_video_addr *addr,
+				   unsigned int size);
+
+static void aspeed_video_free_buf(struct aspeed_video *video,
+				  struct aspeed_video_addr *addr);
+
 static void aspeed_video_init_jpeg_table(u32 *table, bool yuv420)
 {
 	int i;
@@ -532,6 +572,7 @@ static int aspeed_video_start_frame(struct aspeed_video *video)
 	unsigned long flags;
 	struct aspeed_video_buffer *buf;
 	u32 seq_ctrl = aspeed_video_read(video, VE_SEQ_CTRL);
+	bool bcd_buf_need = (video->format != VIDEO_FMT_STANDARD);
 
 	if (video->v4l2_input_status) {
 		v4l2_warn(&video->v4l2_dev, "No signal; don't start frame\n");
@@ -544,6 +585,20 @@ static int aspeed_video_start_frame(struct aspeed_video *video)
 		return -EBUSY;
 	}
 
+	if (bcd_buf_need && !video->bcd.size) {
+		if (!aspeed_video_alloc_buf(video, &video->bcd,
+					    VE_BCD_BUFF_SIZE)) {
+			dev_err(video->dev, "Failed to allocate BCD buffer\n");
+			dev_err(video->dev, "don't start frame\n");
+			return -ENOMEM;
+		}
+		aspeed_video_write(video, VE_BCD_ADDR, video->bcd.dma);
+		v4l2_dbg(1, debug, &video->v4l2_dev, "bcd addr(%#x) size(%d)\n",
+			 video->bcd.dma, video->bcd.size);
+	} else if (!bcd_buf_need && video->bcd.size) {
+		aspeed_video_free_buf(video, &video->bcd);
+	}
+
 	spin_lock_irqsave(&video->lock, flags);
 	buf = list_first_entry_or_null(&video->buffers,
 				       struct aspeed_video_buffer, link);
@@ -640,6 +695,24 @@ static void aspeed_video_irq_res_change(struct aspeed_video *video, ulong delay)
 	schedule_delayed_work(&video->res_work, delay);
 }
 
+static void aspeed_video_swap_src_buf(struct aspeed_video *v)
+{
+	if (v->format == VIDEO_FMT_STANDARD)
+		return;
+
+	/* Reset bcd buffer to have a full frame update every 8 frames.  */
+	if (IS_ALIGNED(v->sequence, 8))
+		memset((u8 *)v->bcd.virt, 0x00, VE_BCD_BUFF_SIZE);
+
+	if (v->sequence & 0x01) {
+		aspeed_video_write(v, VE_SRC0_ADDR, v->srcs[1].dma);
+		aspeed_video_write(v, VE_SRC1_ADDR, v->srcs[0].dma);
+	} else {
+		aspeed_video_write(v, VE_SRC0_ADDR, v->srcs[0].dma);
+		aspeed_video_write(v, VE_SRC1_ADDR, v->srcs[1].dma);
+	}
+}
+
 /*
  * Interrupts that we don't use but have to explicitly ignore because the
  * hardware asserts them even when they're disabled in the VE_INTERRUPT_CTRL
@@ -691,6 +764,7 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
 
 	if (sts & VE_INTERRUPT_COMP_COMPLETE) {
 		struct aspeed_video_buffer *buf;
+		bool empty = true;
 		u32 frame_size = aspeed_video_read(video,
 						   video->config.comp_size_read);
 
@@ -704,13 +778,23 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
 		if (buf) {
 			vb2_set_plane_payload(&buf->vb.vb2_buf, 0, frame_size);
 
-			if (!list_is_last(&buf->link, &video->buffers)) {
+			/*
+			 * aspeed_jpeg requires continuous update.
+			 * On the contrary, standard jpeg can keep last buffer
+			 * to always have the latest result.
+			 */
+			if (video->format == VIDEO_FMT_STANDARD &&
+			    list_is_last(&buf->link, &video->buffers)) {
+				empty = false;
+				v4l2_warn(&video->v4l2_dev, "skip to keep last frame updated\n");
+			} else {
 				buf->vb.vb2_buf.timestamp = ktime_get_ns();
 				buf->vb.sequence = video->sequence++;
 				buf->vb.field = V4L2_FIELD_NONE;
 				vb2_buffer_done(&buf->vb.vb2_buf,
 						VB2_BUF_STATE_DONE);
 				list_del(&buf->link);
+				empty = list_empty(&video->buffers);
 			}
 		}
 		spin_unlock(&video->lock);
@@ -724,7 +808,10 @@ static irqreturn_t aspeed_video_irq(int irq, void *arg)
 		aspeed_video_write(video, VE_INTERRUPT_STATUS,
 				   VE_INTERRUPT_COMP_COMPLETE);
 		sts &= ~VE_INTERRUPT_COMP_COMPLETE;
-		if (test_bit(VIDEO_STREAMING, &video->flags) && buf)
+
+		aspeed_video_swap_src_buf(video);
+
+		if (test_bit(VIDEO_STREAMING, &video->flags) && !empty)
 			aspeed_video_start_frame(video);
 	}
 
@@ -983,10 +1070,14 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
 				   FIELD_PREP(VE_TGS_FIRST, video->frame_top) |
 				   FIELD_PREP(VE_TGS_LAST,
 					      video->frame_bottom + 1));
-		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_INT_DE);
+		aspeed_video_update(video, VE_CTRL,
+				    VE_CTRL_INT_DE | VE_CTRL_DIRECT_FETCH,
+				    VE_CTRL_INT_DE);
 	} else {
 		v4l2_dbg(1, debug, &video->v4l2_dev, "Capture: Direct Mode\n");
-		aspeed_video_update(video, VE_CTRL, 0, VE_CTRL_DIRECT_FETCH);
+		aspeed_video_update(video, VE_CTRL,
+				    VE_CTRL_INT_DE | VE_CTRL_DIRECT_FETCH,
+				    VE_CTRL_DIRECT_FETCH);
 	}
 
 	size *= 4;
@@ -1021,35 +1112,71 @@ static void aspeed_video_set_resolution(struct aspeed_video *video)
 
 static void aspeed_video_update_regs(struct aspeed_video *video)
 {
-	u32 comp_ctrl = VE_COMP_CTRL_RSVD |
-		FIELD_PREP(VE_COMP_CTRL_DCT_LUM, video->jpeg_quality) |
-		FIELD_PREP(VE_COMP_CTRL_DCT_CHR, video->jpeg_quality | 0x10);
+	u8 jpeg_hq_quality = clamp((int)video->jpeg_hq_quality - 1, 0,
+				   ASPEED_VIDEO_JPEG_NUM_QUALITIES - 1);
+	u32 comp_ctrl =	FIELD_PREP(VE_COMP_CTRL_DCT_LUM, video->jpeg_quality) |
+		FIELD_PREP(VE_COMP_CTRL_DCT_CHR, video->jpeg_quality | 0x10) |
+		FIELD_PREP(VE_COMP_CTRL_EN_HQ, video->hq_mode) |
+		FIELD_PREP(VE_COMP_CTRL_HQ_DCT_LUM, jpeg_hq_quality) |
+		FIELD_PREP(VE_COMP_CTRL_HQ_DCT_CHR, jpeg_hq_quality | 0x10);
 	u32 ctrl = 0;
-	u32 seq_ctrl = video->config.jpeg_mode;
+	u32 seq_ctrl = 0;
 
-	v4l2_dbg(1, debug, &video->v4l2_dev, "framerate(%d)\n",
-		 video->frame_rate);
-	v4l2_dbg(1, debug, &video->v4l2_dev, "subsample(%s)\n",
+	v4l2_dbg(1, debug, &video->v4l2_dev, "framerate(%d)\n", video->frame_rate);
+	v4l2_dbg(1, debug, &video->v4l2_dev, "jpeg format(%s) subsample(%s)\n",
+		 format_str[video->format],
 		 video->yuv420 ? "420" : "444");
 	v4l2_dbg(1, debug, &video->v4l2_dev, "compression quality(%d)\n",
 		 video->jpeg_quality);
+	v4l2_dbg(1, debug, &video->v4l2_dev, "hq_mode(%s) hq_quality(%d)\n",
+		 video->hq_mode ? "on" : "off", video->jpeg_hq_quality);
+	v4l2_dbg(1, debug, &video->v4l2_dev, "compression scheme(%s)\n",
+		 compress_scheme_str[video->compression_scheme]);
+
+	if (video->format == VIDEO_FMT_ASPEED)
+		aspeed_video_update(video, VE_BCD_CTRL, 0, VE_BCD_CTRL_EN_BCD);
+	else
+		aspeed_video_update(video, VE_BCD_CTRL, VE_BCD_CTRL_EN_BCD, 0);
 
 	if (video->frame_rate)
 		ctrl |= FIELD_PREP(VE_CTRL_FRC, video->frame_rate);
 
+	if (video->format == VIDEO_FMT_STANDARD) {
+		comp_ctrl &= ~FIELD_PREP(VE_COMP_CTRL_EN_HQ, video->hq_mode);
+		seq_ctrl |= video->config.jpeg_mode;
+	}
+
 	if (video->yuv420)
 		seq_ctrl |= VE_SEQ_CTRL_YUV420;
 
 	if (video->jpeg.virt)
 		aspeed_video_update_jpeg_table(video->jpeg.virt, video->yuv420);
 
+	if (video->config.version == 4) {
+		switch (video->compression_scheme) {
+		case 0:	//DCT only
+			comp_ctrl |= VE_COMP_CTRL_VQ_DCT_ONLY;
+			break;
+		case 1:	//DCT VQ mix 2-color
+			comp_ctrl &= ~(VE_COMP_CTRL_VQ_4COLOR |
+				     VE_COMP_CTRL_VQ_DCT_ONLY);
+			break;
+		case 2:	//DCT VQ mix 4-color
+			comp_ctrl |= VE_COMP_CTRL_VQ_4COLOR;
+			break;
+		}
+	}
+
 	/* Set control registers */
 	aspeed_video_update(video, VE_SEQ_CTRL,
 			    video->config.jpeg_mode | VE_SEQ_CTRL_YUV420,
 			    seq_ctrl);
 	aspeed_video_update(video, VE_CTRL, VE_CTRL_FRC, ctrl);
 	aspeed_video_update(video, VE_COMP_CTRL,
-			    VE_COMP_CTRL_DCT_LUM | VE_COMP_CTRL_DCT_CHR,
+			    VE_COMP_CTRL_DCT_LUM | VE_COMP_CTRL_DCT_CHR |
+			    VE_COMP_CTRL_EN_HQ | VE_COMP_CTRL_HQ_DCT_LUM |
+			    VE_COMP_CTRL_HQ_DCT_CHR | VE_COMP_CTRL_VQ_4COLOR |
+			    VE_COMP_CTRL_VQ_DCT_ONLY,
 			    comp_ctrl);
 }
 
@@ -1081,6 +1208,8 @@ static void aspeed_video_init_regs(struct aspeed_video *video)
 
 	/* Set mode detection defaults */
 	aspeed_video_write(video, VE_MODE_DETECT, 0x22666500);
+
+	aspeed_video_write(video, VE_BCD_CTRL, 0);
 }
 
 static void aspeed_video_start(struct aspeed_video *video)
@@ -1114,6 +1243,9 @@ static void aspeed_video_stop(struct aspeed_video *video)
 	if (video->srcs[1].size)
 		aspeed_video_free_buf(video, &video->srcs[1]);
 
+	if (video->bcd.size)
+		aspeed_video_free_buf(video, &video->bcd);
+
 	video->v4l2_input_status = V4L2_IN_ST_NO_SIGNAL;
 	video->flags = 0;
 }
@@ -1132,10 +1264,12 @@ static int aspeed_video_querycap(struct file *file, void *fh,
 static int aspeed_video_enum_format(struct file *file, void *fh,
 				    struct v4l2_fmtdesc *f)
 {
+	struct aspeed_video *video = video_drvdata(file);
+
 	if (f->index)
 		return -EINVAL;
 
-	f->pixelformat = V4L2_PIX_FMT_JPEG;
+	f->pixelformat = video->pix_fmt.pixelformat;
 
 	return 0;
 }
@@ -1150,6 +1284,29 @@ static int aspeed_video_get_format(struct file *file, void *fh,
 	return 0;
 }
 
+static int aspeed_video_set_format(struct file *file, void *fh,
+				   struct v4l2_format *f)
+{
+	struct aspeed_video *video = video_drvdata(file);
+
+	if (vb2_is_busy(&video->queue))
+		return -EBUSY;
+
+	switch (f->fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_JPEG:
+		video->format = VIDEO_FMT_STANDARD;
+		break;
+	case V4L2_PIX_FMT_AJPG:
+		video->format = VIDEO_FMT_ASPEED;
+		break;
+	default:
+		return -EINVAL;
+	}
+	video->pix_fmt.pixelformat = f->fmt.pix.pixelformat;
+
+	return 0;
+}
+
 static int aspeed_video_enum_input(struct file *file, void *fh,
 				   struct v4l2_input *inp)
 {
@@ -1367,7 +1524,7 @@ static const struct v4l2_ioctl_ops aspeed_video_ioctl_ops = {
 
 	.vidioc_enum_fmt_vid_cap = aspeed_video_enum_format,
 	.vidioc_g_fmt_vid_cap = aspeed_video_get_format,
-	.vidioc_s_fmt_vid_cap = aspeed_video_get_format,
+	.vidioc_s_fmt_vid_cap = aspeed_video_set_format,
 	.vidioc_try_fmt_vid_cap = aspeed_video_get_format,
 
 	.vidioc_reqbufs = vb2_ioctl_reqbufs,
@@ -1416,6 +1573,24 @@ static int aspeed_video_set_ctrl(struct v4l2_ctrl *ctrl)
 		if (test_bit(VIDEO_STREAMING, &video->flags))
 			aspeed_video_update_regs(video);
 		break;
+	case V4L2_CID_ASPEED_COMPRESSION_SCHEME:
+		if (video->config.version > 4)
+			return -EINVAL;
+
+		video->compression_scheme = ctrl->val;
+		if (test_bit(VIDEO_STREAMING, &video->flags))
+			aspeed_video_update_regs(video);
+		break;
+	case V4L2_CID_ASPEED_HQ_MODE:
+		video->hq_mode = ctrl->val;
+		if (test_bit(VIDEO_STREAMING, &video->flags))
+			aspeed_video_update_regs(video);
+		break;
+	case V4L2_CID_ASPEED_HQ_JPEG_QUALITY:
+		video->jpeg_hq_quality = ctrl->val;
+		if (test_bit(VIDEO_STREAMING, &video->flags))
+			aspeed_video_update_regs(video);
+		break;
 	default:
 		return -EINVAL;
 	}
@@ -1427,6 +1602,39 @@ static const struct v4l2_ctrl_ops aspeed_video_ctrl_ops = {
 	.s_ctrl = aspeed_video_set_ctrl,
 };
 
+static const struct v4l2_ctrl_config aspeed_ctrl_compression_scheme = {
+	.ops = &aspeed_video_ctrl_ops,
+	.id = V4L2_CID_ASPEED_COMPRESSION_SCHEME,
+	.name = "Aspeed Compression SCHEME",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = 0,
+	.max = 2,
+	.step = 1,
+	.def = 0,
+};
+
+static const struct v4l2_ctrl_config aspeed_ctrl_HQ_mode = {
+	.ops = &aspeed_video_ctrl_ops,
+	.id = V4L2_CID_ASPEED_HQ_MODE,
+	.name = "Aspeed HQ Mode",
+	.type = V4L2_CTRL_TYPE_BOOLEAN,
+	.min = false,
+	.max = true,
+	.step = 1,
+	.def = false,
+};
+
+static const struct v4l2_ctrl_config aspeed_ctrl_HQ_jpeg_quality = {
+	.ops = &aspeed_video_ctrl_ops,
+	.id = V4L2_CID_ASPEED_HQ_JPEG_QUALITY,
+	.name = "Aspeed HQ Quality",
+	.type = V4L2_CTRL_TYPE_INTEGER,
+	.min = 1,
+	.max = ASPEED_VIDEO_JPEG_NUM_QUALITIES,
+	.step = 1,
+	.def = 1,
+};
+
 static void aspeed_video_resolution_work(struct work_struct *work)
 {
 	struct delayed_work *dwork = to_delayed_work(work);
@@ -1693,6 +1901,7 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
 	struct v4l2_device *v4l2_dev = &video->v4l2_dev;
 	struct vb2_queue *vbq = &video->queue;
 	struct video_device *vdev = &video->vdev;
+	struct v4l2_ctrl_handler *hdl = &video->ctrl_handler;
 	int rc;
 
 	video->pix_fmt.pixelformat = V4L2_PIX_FMT_JPEG;
@@ -1707,22 +1916,26 @@ static int aspeed_video_setup_video(struct aspeed_video *video)
 		return rc;
 	}
 
-	v4l2_ctrl_handler_init(&video->ctrl_handler, 2);
-	v4l2_ctrl_new_std(&video->ctrl_handler, &aspeed_video_ctrl_ops,
+	v4l2_ctrl_handler_init(hdl, 5);
+	v4l2_ctrl_new_std(hdl, &aspeed_video_ctrl_ops,
 			  V4L2_CID_JPEG_COMPRESSION_QUALITY, 0,
 			  ASPEED_VIDEO_JPEG_NUM_QUALITIES - 1, 1, 0);
-	v4l2_ctrl_new_std_menu(&video->ctrl_handler, &aspeed_video_ctrl_ops,
+	v4l2_ctrl_new_std_menu(hdl, &aspeed_video_ctrl_ops,
 			       V4L2_CID_JPEG_CHROMA_SUBSAMPLING,
 			       V4L2_JPEG_CHROMA_SUBSAMPLING_420, mask,
 			       V4L2_JPEG_CHROMA_SUBSAMPLING_444);
+	if (video->config.version == 4)
+		v4l2_ctrl_new_custom(hdl, &aspeed_ctrl_compression_scheme, NULL);
+	v4l2_ctrl_new_custom(hdl, &aspeed_ctrl_HQ_mode, NULL);
+	v4l2_ctrl_new_custom(hdl, &aspeed_ctrl_HQ_jpeg_quality, NULL);
 
-	rc = video->ctrl_handler.error;
+	rc = hdl->error;
 	if (rc) {
 		dev_err(video->dev, "Failed to init controls: %d\n", rc);
 		goto err_ctrl_init;
 	}
 
-	v4l2_dev->ctrl_handler = &video->ctrl_handler;
+	v4l2_dev->ctrl_handler = hdl;
 
 	vbq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
 	vbq->io_modes = VB2_MMAP | VB2_READ | VB2_DMABUF;
@@ -1863,6 +2076,7 @@ static int aspeed_video_probe(struct platform_device *pdev)
 		return -ENOMEM;
 
 	video->frame_rate = 30;
+	video->jpeg_hq_quality = 1;
 	video->dev = &pdev->dev;
 	spin_lock_init(&video->lock);
 	mutex_init(&video->video_lock);
diff --git a/include/uapi/linux/aspeed-video.h b/include/uapi/linux/aspeed-video.h
new file mode 100644
index 000000000000..63f0432192a5
--- /dev/null
+++ b/include/uapi/linux/aspeed-video.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2021 ASPEED Technology Inc.
+ */
+
+#ifndef _UAPI_LINUX_ASPEED_VIDEO_H
+#define _UAPI_LINUX_ASPEED_VIDEO_H
+
+#include <linux/v4l2-controls.h>
+
+#define V4L2_CID_ASPEED_COMPRESSION_SCHEME	(V4L2_CID_USER_ASPEED_BASE  + 1)
+#define V4L2_CID_ASPEED_HQ_MODE			(V4L2_CID_USER_ASPEED_BASE  + 2)
+#define V4L2_CID_ASPEED_HQ_JPEG_QUALITY		(V4L2_CID_USER_ASPEED_BASE  + 3)
+
+#endif /* _UAPI_LINUX_ASPEED_VIDEO_H */
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v8 4/4] media: aspeed: Extend debug message

[Jammy Huang]

updated as below:

Capture:
  Mode                : Direct fetch
  VGA bpp mode        : 32
  Signal              : lock
  Width               : 1920
  Height              : 1080
  FRC                 : 0

Compression:
  Format              : JPEG
  Subsampling         : 444
  Quality             : 4

Performance:
  Frame#              : 4
  Frame Duration(ms)  :
    Now               : 22
    Min               : 21
    Max               : 22
  FPS                 : 45

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v8:
  - no update
v7:
  - update debugfs message. Aspeed-jpeg's compression parameters only shown
    if it's aspeed jpeg now
v6:
  - no update
v5:
  - no update
v4:
  - update debugfs log
v3:
  - no update
v2:
  - update commit message
---
 drivers/media/platform/aspeed-video.c | 35 +++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index c0cfcf98659d..b388bc56ce81 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -1835,10 +1835,29 @@ static const struct vb2_ops aspeed_video_vb2_ops = {
 static int aspeed_video_debugfs_show(struct seq_file *s, void *data)
 {
 	struct aspeed_video *v = s->private;
+	u32 val08;
 
 	seq_puts(s, "\n");
 
 	seq_puts(s, "Capture:\n");
+	val08 = aspeed_video_read(v, VE_CTRL);
+	if (FIELD_GET(VE_CTRL_DIRECT_FETCH, val08)) {
+		seq_printf(s, "  %-20s:\tDirect fetch\n", "Mode");
+		seq_printf(s, "  %-20s:\t%s\n", "VGA bpp mode",
+			   FIELD_GET(VE_CTRL_INT_DE, val08) ? "16" : "32");
+	} else {
+		seq_printf(s, "  %-20s:\tSync\n", "Mode");
+		seq_printf(s, "  %-20s:\t%s\n", "Video source",
+			   FIELD_GET(VE_CTRL_SOURCE, val08) ?
+			   "external" : "internal");
+		seq_printf(s, "  %-20s:\t%s\n", "DE source",
+			   FIELD_GET(VE_CTRL_INT_DE, val08) ?
+			   "internal" : "external");
+		seq_printf(s, "  %-20s:\t%s\n", "Cursor overlay",
+			   FIELD_GET(VE_CTRL_AUTO_OR_CURSOR, val08) ?
+			   "Without" : "With");
+	}
+
 	seq_printf(s, "  %-20s:\t%s\n", "Signal",
 		   v->v4l2_input_status ? "Unlock" : "Lock");
 	seq_printf(s, "  %-20s:\t%d\n", "Width", v->pix_fmt.width);
@@ -1847,6 +1866,22 @@ static int aspeed_video_debugfs_show(struct seq_file *s, void *data)
 
 	seq_puts(s, "\n");
 
+	seq_puts(s, "Compression:\n");
+	seq_printf(s, "  %-20s:\t%s\n", "Format", format_str[v->format]);
+	seq_printf(s, "  %-20s:\t%s\n", "Subsampling",
+		   v->yuv420 ? "420" : "444");
+	seq_printf(s, "  %-20s:\t%d\n", "Quality", v->jpeg_quality);
+	if (v->format == VIDEO_FMT_ASPEED) {
+		seq_printf(s, "  %-20s:\t%s\n", "HQ Mode",
+			   v->hq_mode ? "on" : "off");
+		seq_printf(s, "  %-20s:\t%d\n", "HQ Quality",
+			   v->hq_mode ? v->jpeg_hq_quality : 0);
+		seq_printf(s, "  %-20s:\t%s\n", "Coding Scheme",
+			   compress_scheme_str[v->compression_scheme]);
+	}
+
+	seq_puts(s, "\n");
+
 	seq_puts(s, "Performance:\n");
 	seq_printf(s, "  %-20s:\t%d\n", "Frame#", v->sequence);
 	seq_printf(s, "  %-20s:\n", "Frame Duration(ms)");
-- 
2.25.1

[PATCH v8 4/4] media: aspeed: Extend debug message

[Jammy Huang]

updated as below:

Capture:
  Mode                : Direct fetch
  VGA bpp mode        : 32
  Signal              : lock
  Width               : 1920
  Height              : 1080
  FRC                 : 0

Compression:
  Format              : JPEG
  Subsampling         : 444
  Quality             : 4

Performance:
  Frame#              : 4
  Frame Duration(ms)  :
    Now               : 22
    Min               : 21
    Max               : 22
  FPS                 : 45

Signed-off-by: Jammy Huang <jammy_huang@aspeedtech.com>
---
v8:
  - no update
v7:
  - update debugfs message. Aspeed-jpeg's compression parameters only shown
    if it's aspeed jpeg now
v6:
  - no update
v5:
  - no update
v4:
  - update debugfs log
v3:
  - no update
v2:
  - update commit message
---
 drivers/media/platform/aspeed-video.c | 35 +++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/drivers/media/platform/aspeed-video.c b/drivers/media/platform/aspeed-video.c
index c0cfcf98659d..b388bc56ce81 100644
--- a/drivers/media/platform/aspeed-video.c
+++ b/drivers/media/platform/aspeed-video.c
@@ -1835,10 +1835,29 @@ static const struct vb2_ops aspeed_video_vb2_ops = {
 static int aspeed_video_debugfs_show(struct seq_file *s, void *data)
 {
 	struct aspeed_video *v = s->private;
+	u32 val08;
 
 	seq_puts(s, "\n");
 
 	seq_puts(s, "Capture:\n");
+	val08 = aspeed_video_read(v, VE_CTRL);
+	if (FIELD_GET(VE_CTRL_DIRECT_FETCH, val08)) {
+		seq_printf(s, "  %-20s:\tDirect fetch\n", "Mode");
+		seq_printf(s, "  %-20s:\t%s\n", "VGA bpp mode",
+			   FIELD_GET(VE_CTRL_INT_DE, val08) ? "16" : "32");
+	} else {
+		seq_printf(s, "  %-20s:\tSync\n", "Mode");
+		seq_printf(s, "  %-20s:\t%s\n", "Video source",
+			   FIELD_GET(VE_CTRL_SOURCE, val08) ?
+			   "external" : "internal");
+		seq_printf(s, "  %-20s:\t%s\n", "DE source",
+			   FIELD_GET(VE_CTRL_INT_DE, val08) ?
+			   "internal" : "external");
+		seq_printf(s, "  %-20s:\t%s\n", "Cursor overlay",
+			   FIELD_GET(VE_CTRL_AUTO_OR_CURSOR, val08) ?
+			   "Without" : "With");
+	}
+
 	seq_printf(s, "  %-20s:\t%s\n", "Signal",
 		   v->v4l2_input_status ? "Unlock" : "Lock");
 	seq_printf(s, "  %-20s:\t%d\n", "Width", v->pix_fmt.width);
@@ -1847,6 +1866,22 @@ static int aspeed_video_debugfs_show(struct seq_file *s, void *data)
 
 	seq_puts(s, "\n");
 
+	seq_puts(s, "Compression:\n");
+	seq_printf(s, "  %-20s:\t%s\n", "Format", format_str[v->format]);
+	seq_printf(s, "  %-20s:\t%s\n", "Subsampling",
+		   v->yuv420 ? "420" : "444");
+	seq_printf(s, "  %-20s:\t%d\n", "Quality", v->jpeg_quality);
+	if (v->format == VIDEO_FMT_ASPEED) {
+		seq_printf(s, "  %-20s:\t%s\n", "HQ Mode",
+			   v->hq_mode ? "on" : "off");
+		seq_printf(s, "  %-20s:\t%d\n", "HQ Quality",
+			   v->hq_mode ? v->jpeg_hq_quality : 0);
+		seq_printf(s, "  %-20s:\t%s\n", "Coding Scheme",
+			   compress_scheme_str[v->compression_scheme]);
+	}
+
+	seq_puts(s, "\n");
+
 	seq_puts(s, "Performance:\n");
 	seq_printf(s, "  %-20s:\t%d\n", "Frame#", v->sequence);
 	seq_printf(s, "  %-20s:\n", "Frame Duration(ms)");
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel