* Re: [PATCH v11 0/6] Wave5 codec driver
@ 2023-02-13 6:13 Som Qin
2023-02-13 14:13 ` Nicolas Dufresne
0 siblings, 1 reply; 7+ messages in thread
From: Som Qin @ 2023-02-13 6:13 UTC (permalink / raw)
To: Sebastian Fricke, linux-media
Cc: kernel, bob.beckett, hverkuil-cisco, nicolas.dufresne, nas.chung
On 12/7/22 20:13, Sebastian Fricke wrote:
>The Wave5 codec driver is a stateful encoder/decoder.
>It is found on the J721S2 SoC, JH7100 SoC, and ssd202d SoC. etc.
>But the current test report is based on J721S2 SoC and pre-silicon FPGA.
>
>The driver currently supports V4L2_PIX_FMT_HEVC and V4L2_PIX_FMT_H264.
>
>This driver has so far been tested on the J721S2 EVM board and
>pre-silicon FPGA.
>
>Testing on the J721S2 EVM board in combination with the GStreamer v4l2
>good plugin shows the decoder is working correctly, it has been tested
>with H264 & HEVC video clips with multiple different dimensions (4k,
>1080p, 720p, 540p, 360p, 270p) and various lengths. Additionally,
>Fluster conformance tests have been run for both supported codecs.
>
>The encoder can encode various videos but still has some issues causing
>it to fail occassionally.
>
>V4L2 compliance:
>
>Total for wave5-dec device /dev/video0: 44, Succeeded: 44, Failed: 0, Warnings: 0
>
>Buffer ioctls:
> fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(610): q.reqbufs(node, 1)
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(753): q.reqbufs(node, 2)
> test VIDIOC_EXPBUF: FAIL
> test Requests: OK (Not Supported)
>
>Total for wave5-enc device /dev/video1: 44, Succeeded: 42, Failed: 2, Warnings: 2
>
>changes since v10:
>
>* Remove structure member from the encoder and decoder output info
> structs, that have assigned values from the registers but aren't used
> in the driver, add comments to describe the register values in the
> register definitions
>* Fix issue with decoding videos with a dimension where the height is
> not a multiple of 16 (270, 360, 540, 1024 etc.)
>* Fix incorrect variable format identifiers in printks
>* Use debug logs in loops to avoid flooding the message log
>* Use the swap() function instead of manual swapping of two values
>* Add extended controls for the encoder
>* Fix control flow issue while handling bitstream buffers, where an
> error while writing the source buffer into the hardware ring buffer
> would result in skipping the problematic buffer, which in turn causes
> a reordering of source buffers
>* Use the rectangle format as described by the hardware, the hardware
> uses for rectangles like the display rectangle 4 offsets (top, bottom,
> left, right), which depict the offset from the respective edge. Use
> this format instead of implicitly converting the bottom and right
> attributes to width and height attributes.
>* Return an error upon reading the sequence header while STREAMON
>* Squash the VDI and the VPUAPI layer commits as they had circular
> dependencies
>
>changes since v9:
>
>* Move from staging to the media directory
> * Move coda driver to sub-directory
>
>* Fixes:
> * Use platform_get_irq instead of platform_get_resource to fetch the IRQ
>
>* General cleanups:
> * Add missing error messages to error conditions
> * Improve messages/variable names/comments, align parameter names across the driver
> * Use macros instead of magic numbers in multiple occassions
> * Reduce code duplication in multiple places
> * Fix whitespace, newline and tab alignment issues
> * Remove unused struct fields & commented out code
> * Convert signed integers to unsigned if signed is not necessary
> * Convert int/unsigned int to s32/u32, when the variable is assigned to the
> return of a register read or provided as a parameter for a register write
> (and vice versa)
> * Fix incorrect bitwise operators where logical operators are appropriate
> * Multiple smaller changes
>
>* Generalization:
> * Add new helper file providing generalized routines for vpu-dec & vpu-enc
> * Generalize luma & chroma table size calculation and stride calculation
>
>* Resource cleanup and error handling:
> * Add error handling to all calls with ignored return codes
> * Handle DMA resource cleanup properly
> * Fix insufficient instance cleanup while opening dec/enc
>
>changes since v8:
>
>* add 'wave5' to DEV_NAME
>* update to support Multi-stream
>* update to support loop test/dynamic resolution change
>* remove unnecessary memset, g_volatile, old version option
>
>changes since v7:
>
>* update v4l2-compliance test report
>* fix build error on linux-kernel 5.18.0-rc4
>
>changes since v6:
>
>* update TODO file
>* get sram info from device tree
>
>changes since v5:
>
>* support NV12/NV21 pixelformat for encoder and decoder
>* handle adnormal exit and EOS
>
>changes since v4:
>
>* refactor functions in wave5-hw and fix bug reported by Daniel Palmer
>* rename functions and variables to better names
>* change variable types such as replacing s32 with u32 and int with bool
>* as appropriate
>
>changes since v3:
>
>* Fixing all issues commented by Dan Carpenter
>* Change file names to have wave5- prefix
>* In wave5_vpu_probe, enable the clocks before reading registers, as
>* commented from Daniel Palmer
>* Add more to the TODO list,
>
>changes since v2:
>
>Main fixes includes:
>* change the yaml and dirver code to support up to 4 clks (instead of
>* one)
>* fix Kconfig format
>* remove unneeded cast,
>* change var types
>* change var names, func names
>* checkpatch fixes
>
>changes since v1:
>
>Fix changes due to comments from Ezequiel and Dan Carpenter. Main fixes
>inclueds:
>* move all files to one dir 'wave5'
>* replace private error codes with standard error codes
>* fix extra spaces
>* various checkpatch fixes
>* replace private 'DPRINTK' macro with standard 'dev_err/dbg ..'
>* fix error handling
>* add more possible fixes to the TODO file
>
We used the patchset on JH7110 SoC which is integrated with a wave511 VPU.
We see issue that the decoder output contains garbage data in the image
when decoding some of HEVC videos, in 1080p or 4k. But Most of the videos
we tested decoded well.
Beside, ffmpeg h264_v4l2m2m could not get the correct resolution via
VIDIOC_G_FMT or VIDIOC_TRY_FMT for 1080p AVC video.
>Nas Chung (3):
> media: chips-media: wave5: Add vpuapi layer
> media: chips-media: wave5: Add the v4l2 layer
> media: chips-media: wave5: Add TODO file
>
>Robert Beckett (2):
> dt-bindings: media: wave5: add yaml devicetree bindings
> media: chips-media: wave5: Add wave5 driver to maintainers file
>
>Sebastian Fricke (1):
> media: platform: chips-media: Move Coda to separate folder
>
> .../devicetree/bindings/cnm,wave5.yml | 72 +
> MAINTAINERS | 11 +-
> drivers/media/platform/chips-media/Kconfig | 18 +-
> drivers/media/platform/chips-media/Makefile | 6 +-
> .../media/platform/chips-media/coda/Kconfig | 18 +
> .../media/platform/chips-media/coda/Makefile | 6 +
> .../chips-media/{ => coda}/coda-bit.c | 0
> .../chips-media/{ => coda}/coda-common.c | 0
> .../chips-media/{ => coda}/coda-gdi.c | 0
> .../chips-media/{ => coda}/coda-h264.c | 0
> .../chips-media/{ => coda}/coda-jpeg.c | 0
> .../chips-media/{ => coda}/coda-mpeg2.c | 0
> .../chips-media/{ => coda}/coda-mpeg4.c | 0
> .../platform/chips-media/{ => coda}/coda.h | 0
> .../chips-media/{ => coda}/coda_regs.h | 0
> .../chips-media/{ => coda}/imx-vdoa.c | 0
> .../chips-media/{ => coda}/imx-vdoa.h | 0
> .../platform/chips-media/{ => coda}/trace.h | 0
> .../media/platform/chips-media/wave5/Kconfig | 12 +
> .../media/platform/chips-media/wave5/Makefile | 10 +
> drivers/media/platform/chips-media/wave5/TODO | 18 +
> .../platform/chips-media/wave5/wave5-helper.c | 175 +
> .../platform/chips-media/wave5/wave5-helper.h | 28 +
> .../platform/chips-media/wave5/wave5-hw.c | 3359 +++++++++++++++++
> .../chips-media/wave5/wave5-regdefine.h | 743 ++++
> .../platform/chips-media/wave5/wave5-vdi.c | 245 ++
> .../platform/chips-media/wave5/wave5-vdi.h | 67 +
> .../chips-media/wave5/wave5-vpu-dec.c | 1429 +++++++
> .../chips-media/wave5/wave5-vpu-enc.c | 1746 +++++++++
> .../platform/chips-media/wave5/wave5-vpu.c | 362 ++
> .../platform/chips-media/wave5/wave5-vpu.h | 72 +
> .../platform/chips-media/wave5/wave5-vpuapi.c | 1040 +++++
> .../platform/chips-media/wave5/wave5-vpuapi.h | 1136 ++++++
> .../chips-media/wave5/wave5-vpuconfig.h | 90 +
> .../chips-media/wave5/wave5-vpuerror.h | 454 +++
> .../media/platform/chips-media/wave5/wave5.h | 94 +
> 36 files changed, 11190 insertions(+), 21 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/cnm,wave5.yml
> create mode 100644 drivers/media/platform/chips-media/coda/Kconfig
> create mode 100644 drivers/media/platform/chips-media/coda/Makefile
> rename drivers/media/platform/chips-media/{ => coda}/coda-bit.c (100%)
> rename drivers/media/platform/chips-media/{ => coda}/coda-common.c (100%)
> rename drivers/media/platform/chips-media/{ => coda}/coda-gdi.c (100%)
> rename drivers/media/platform/chips-media/{ => coda}/coda-h264.c (100%)
> rename drivers/media/platform/chips-media/{ => coda}/coda-jpeg.c (100%)
> rename drivers/media/platform/chips-media/{ => coda}/coda-mpeg2.c (100%)
> rename drivers/media/platform/chips-media/{ => coda}/coda-mpeg4.c (100%)
> rename drivers/media/platform/chips-media/{ => coda}/coda.h (100%)
> rename drivers/media/platform/chips-media/{ => coda}/coda_regs.h (100%)
> rename drivers/media/platform/chips-media/{ => coda}/imx-vdoa.c (100%)
> rename drivers/media/platform/chips-media/{ => coda}/imx-vdoa.h (100%)
> rename drivers/media/platform/chips-media/{ => coda}/trace.h (100%)
> create mode 100644 drivers/media/platform/chips-media/wave5/Kconfig
> create mode 100644 drivers/media/platform/chips-media/wave5/Makefile
> create mode 100644 drivers/media/platform/chips-media/wave5/TODO
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-helper.c
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-helper.h
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-hw.c
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-regdefine.h
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vdi.c
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vdi.h
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu.c
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu.h
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuapi.c
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuapi.h
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuerror.h
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5.h
>
>--
>2.25.1
>
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v11 0/6] Wave5 codec driver
2023-02-13 6:13 [PATCH v11 0/6] Wave5 codec driver Som Qin
@ 2023-02-13 14:13 ` Nicolas Dufresne
2023-02-14 7:11 ` Som Qin
0 siblings, 1 reply; 7+ messages in thread
From: Nicolas Dufresne @ 2023-02-13 14:13 UTC (permalink / raw)
To: Som Qin, Sebastian Fricke, linux-media
Cc: kernel, bob.beckett, hverkuil-cisco, nas.chung
Le lundi 13 février 2023 à 06:13 +0000, Som Qin a écrit :
> On 12/7/22 20:13, Sebastian Fricke wrote:
> > The Wave5 codec driver is a stateful encoder/decoder.
> > It is found on the J721S2 SoC, JH7100 SoC, and ssd202d SoC. etc.
> > But the current test report is based on J721S2 SoC and pre-silicon FPGA.
> >
> > The driver currently supports V4L2_PIX_FMT_HEVC and V4L2_PIX_FMT_H264.
> >
> > This driver has so far been tested on the J721S2 EVM board and
> > pre-silicon FPGA.
> >
> > Testing on the J721S2 EVM board in combination with the GStreamer v4l2
> > good plugin shows the decoder is working correctly, it has been tested
> > with H264 & HEVC video clips with multiple different dimensions (4k,
> > 1080p, 720p, 540p, 360p, 270p) and various lengths. Additionally,
> > Fluster conformance tests have been run for both supported codecs.
> >
> > The encoder can encode various videos but still has some issues causing
> > it to fail occassionally.
> >
> > V4L2 compliance:
> >
> > Total for wave5-dec device /dev/video0: 44, Succeeded: 44, Failed: 0, Warnings: 0
> >
> > Buffer ioctls:
> > fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(610): q.reqbufs(node, 1)
> > test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> > fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(753): q.reqbufs(node, 2)
> > test VIDIOC_EXPBUF: FAIL
> > test Requests: OK (Not Supported)
> >
> > Total for wave5-enc device /dev/video1: 44, Succeeded: 42, Failed: 2, Warnings: 2
> >
> > changes since v10:
> >
> > * Remove structure member from the encoder and decoder output info
> > structs, that have assigned values from the registers but aren't used
> > in the driver, add comments to describe the register values in the
> > register definitions
> > * Fix issue with decoding videos with a dimension where the height is
> > not a multiple of 16 (270, 360, 540, 1024 etc.)
> > * Fix incorrect variable format identifiers in printks
> > * Use debug logs in loops to avoid flooding the message log
> > * Use the swap() function instead of manual swapping of two values
> > * Add extended controls for the encoder
> > * Fix control flow issue while handling bitstream buffers, where an
> > error while writing the source buffer into the hardware ring buffer
> > would result in skipping the problematic buffer, which in turn causes
> > a reordering of source buffers
> > * Use the rectangle format as described by the hardware, the hardware
> > uses for rectangles like the display rectangle 4 offsets (top, bottom,
> > left, right), which depict the offset from the respective edge. Use
> > this format instead of implicitly converting the bottom and right
> > attributes to width and height attributes.
> > * Return an error upon reading the sequence header while STREAMON
> > * Squash the VDI and the VPUAPI layer commits as they had circular
> > dependencies
> >
> > changes since v9:
> >
> > * Move from staging to the media directory
> > * Move coda driver to sub-directory
> >
> > * Fixes:
> > * Use platform_get_irq instead of platform_get_resource to fetch the IRQ
> >
> > * General cleanups:
> > * Add missing error messages to error conditions
> > * Improve messages/variable names/comments, align parameter names across the driver
> > * Use macros instead of magic numbers in multiple occassions
> > * Reduce code duplication in multiple places
> > * Fix whitespace, newline and tab alignment issues
> > * Remove unused struct fields & commented out code
> > * Convert signed integers to unsigned if signed is not necessary
> > * Convert int/unsigned int to s32/u32, when the variable is assigned to the
> > return of a register read or provided as a parameter for a register write
> > (and vice versa)
> > * Fix incorrect bitwise operators where logical operators are appropriate
> > * Multiple smaller changes
> >
> > * Generalization:
> > * Add new helper file providing generalized routines for vpu-dec & vpu-enc
> > * Generalize luma & chroma table size calculation and stride calculation
> >
> > * Resource cleanup and error handling:
> > * Add error handling to all calls with ignored return codes
> > * Handle DMA resource cleanup properly
> > * Fix insufficient instance cleanup while opening dec/enc
> >
> > changes since v8:
> >
> > * add 'wave5' to DEV_NAME
> > * update to support Multi-stream
> > * update to support loop test/dynamic resolution change
> > * remove unnecessary memset, g_volatile, old version option
> >
> > changes since v7:
> >
> > * update v4l2-compliance test report
> > * fix build error on linux-kernel 5.18.0-rc4
> >
> > changes since v6:
> >
> > * update TODO file
> > * get sram info from device tree
> >
> > changes since v5:
> >
> > * support NV12/NV21 pixelformat for encoder and decoder
> > * handle adnormal exit and EOS
> >
> > changes since v4:
> >
> > * refactor functions in wave5-hw and fix bug reported by Daniel Palmer
> > * rename functions and variables to better names
> > * change variable types such as replacing s32 with u32 and int with bool
> > * as appropriate
> >
> > changes since v3:
> >
> > * Fixing all issues commented by Dan Carpenter
> > * Change file names to have wave5- prefix
> > * In wave5_vpu_probe, enable the clocks before reading registers, as
> > * commented from Daniel Palmer
> > * Add more to the TODO list,
> >
> > changes since v2:
> >
> > Main fixes includes:
> > * change the yaml and dirver code to support up to 4 clks (instead of
> > * one)
> > * fix Kconfig format
> > * remove unneeded cast,
> > * change var types
> > * change var names, func names
> > * checkpatch fixes
> >
> > changes since v1:
> >
> > Fix changes due to comments from Ezequiel and Dan Carpenter. Main fixes
> > inclueds:
> > * move all files to one dir 'wave5'
> > * replace private error codes with standard error codes
> > * fix extra spaces
> > * various checkpatch fixes
> > * replace private 'DPRINTK' macro with standard 'dev_err/dbg ..'
> > * fix error handling
> > * add more possible fixes to the TODO file
> >
>
> We used the patchset on JH7110 SoC which is integrated with a wave511 VPU.
> We see issue that the decoder output contains garbage data in the image
> when decoding some of HEVC videos, in 1080p or 4k. But Most of the videos
> we tested decoded well.
>
> Beside, ffmpeg h264_v4l2m2m could not get the correct resolution via
> VIDIOC_G_FMT or VIDIOC_TRY_FMT for 1080p AVC video.
Thanks for the report, Sebastian has been working on a v12, which includes
important rework regarding format negotiation, hopefully this will help. One of
the pressing issue was that the original driver this effort was derived on was
designed to work on ancient GStreamer (1.16) and would simply not work at all on
new version, so this has been taking some time. It was also highlighted that
h264_v4l2m2m may run out of ram on JH7110 in absence of V4L2_CID_MIN_BUFFER*
controls, so we will be adding these too.
p.s. if you are allowed to share the streams (or a sample of it) you have had
issue with, Sebastian can certainly add this to his validation.
regards,
Nicolas
>
> > Nas Chung (3):
> > media: chips-media: wave5: Add vpuapi layer
> > media: chips-media: wave5: Add the v4l2 layer
> > media: chips-media: wave5: Add TODO file
> >
> > Robert Beckett (2):
> > dt-bindings: media: wave5: add yaml devicetree bindings
> > media: chips-media: wave5: Add wave5 driver to maintainers file
> >
> > Sebastian Fricke (1):
> > media: platform: chips-media: Move Coda to separate folder
> >
> > .../devicetree/bindings/cnm,wave5.yml | 72 +
> > MAINTAINERS | 11 +-
> > drivers/media/platform/chips-media/Kconfig | 18 +-
> > drivers/media/platform/chips-media/Makefile | 6 +-
> > .../media/platform/chips-media/coda/Kconfig | 18 +
> > .../media/platform/chips-media/coda/Makefile | 6 +
> > .../chips-media/{ => coda}/coda-bit.c | 0
> > .../chips-media/{ => coda}/coda-common.c | 0
> > .../chips-media/{ => coda}/coda-gdi.c | 0
> > .../chips-media/{ => coda}/coda-h264.c | 0
> > .../chips-media/{ => coda}/coda-jpeg.c | 0
> > .../chips-media/{ => coda}/coda-mpeg2.c | 0
> > .../chips-media/{ => coda}/coda-mpeg4.c | 0
> > .../platform/chips-media/{ => coda}/coda.h | 0
> > .../chips-media/{ => coda}/coda_regs.h | 0
> > .../chips-media/{ => coda}/imx-vdoa.c | 0
> > .../chips-media/{ => coda}/imx-vdoa.h | 0
> > .../platform/chips-media/{ => coda}/trace.h | 0
> > .../media/platform/chips-media/wave5/Kconfig | 12 +
> > .../media/platform/chips-media/wave5/Makefile | 10 +
> > drivers/media/platform/chips-media/wave5/TODO | 18 +
> > .../platform/chips-media/wave5/wave5-helper.c | 175 +
> > .../platform/chips-media/wave5/wave5-helper.h | 28 +
> > .../platform/chips-media/wave5/wave5-hw.c | 3359 +++++++++++++++++
> > .../chips-media/wave5/wave5-regdefine.h | 743 ++++
> > .../platform/chips-media/wave5/wave5-vdi.c | 245 ++
> > .../platform/chips-media/wave5/wave5-vdi.h | 67 +
> > .../chips-media/wave5/wave5-vpu-dec.c | 1429 +++++++
> > .../chips-media/wave5/wave5-vpu-enc.c | 1746 +++++++++
> > .../platform/chips-media/wave5/wave5-vpu.c | 362 ++
> > .../platform/chips-media/wave5/wave5-vpu.h | 72 +
> > .../platform/chips-media/wave5/wave5-vpuapi.c | 1040 +++++
> > .../platform/chips-media/wave5/wave5-vpuapi.h | 1136 ++++++
> > .../chips-media/wave5/wave5-vpuconfig.h | 90 +
> > .../chips-media/wave5/wave5-vpuerror.h | 454 +++
> > .../media/platform/chips-media/wave5/wave5.h | 94 +
> > 36 files changed, 11190 insertions(+), 21 deletions(-)
> > create mode 100644 Documentation/devicetree/bindings/cnm,wave5.yml
> > create mode 100644 drivers/media/platform/chips-media/coda/Kconfig
> > create mode 100644 drivers/media/platform/chips-media/coda/Makefile
> > rename drivers/media/platform/chips-media/{ => coda}/coda-bit.c (100%)
> > rename drivers/media/platform/chips-media/{ => coda}/coda-common.c (100%)
> > rename drivers/media/platform/chips-media/{ => coda}/coda-gdi.c (100%)
> > rename drivers/media/platform/chips-media/{ => coda}/coda-h264.c (100%)
> > rename drivers/media/platform/chips-media/{ => coda}/coda-jpeg.c (100%)
> > rename drivers/media/platform/chips-media/{ => coda}/coda-mpeg2.c (100%)
> > rename drivers/media/platform/chips-media/{ => coda}/coda-mpeg4.c (100%)
> > rename drivers/media/platform/chips-media/{ => coda}/coda.h (100%)
> > rename drivers/media/platform/chips-media/{ => coda}/coda_regs.h (100%)
> > rename drivers/media/platform/chips-media/{ => coda}/imx-vdoa.c (100%)
> > rename drivers/media/platform/chips-media/{ => coda}/imx-vdoa.h (100%)
> > rename drivers/media/platform/chips-media/{ => coda}/trace.h (100%)
> > create mode 100644 drivers/media/platform/chips-media/wave5/Kconfig
> > create mode 100644 drivers/media/platform/chips-media/wave5/Makefile
> > create mode 100644 drivers/media/platform/chips-media/wave5/TODO
> > create mode 100644 drivers/media/platform/chips-media/wave5/wave5-helper.c
> > create mode 100644 drivers/media/platform/chips-media/wave5/wave5-helper.h
> > create mode 100644 drivers/media/platform/chips-media/wave5/wave5-hw.c
> > create mode 100644 drivers/media/platform/chips-media/wave5/wave5-regdefine.h
> > create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vdi.c
> > create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vdi.h
> > create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
> > create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> > create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu.c
> > create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu.h
> > create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuapi.c
> > create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuapi.h
> > create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
> > create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuerror.h
> > create mode 100644 drivers/media/platform/chips-media/wave5/wave5.h
> >
> > --
> > 2.25.1
> >
>
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v11 0/6] Wave5 codec driver
2023-02-13 14:13 ` Nicolas Dufresne
@ 2023-02-14 7:11 ` Som Qin
2023-02-14 20:12 ` Nicolas Dufresne
0 siblings, 1 reply; 7+ messages in thread
From: Som Qin @ 2023-02-14 7:11 UTC (permalink / raw)
To: Nicolas Dufresne, Sebastian Fricke, linux-media
Cc: kernel, bob.beckett, hverkuil-cisco, nas.chung
On 2023/2/13 22:13, Nicolas Dufresne wrote:
> Le lundi 13 février 2023 à 06:13 +0000, Som Qin a écrit :
>> On 12/7/22 20:13, Sebastian Fricke wrote:
>>> The Wave5 codec driver is a stateful encoder/decoder.
>>> It is found on the J721S2 SoC, JH7100 SoC, and ssd202d SoC. etc.
>>> But the current test report is based on J721S2 SoC and pre-silicon FPGA.
>>>
>>> The driver currently supports V4L2_PIX_FMT_HEVC and V4L2_PIX_FMT_H264.
>>>
>>> This driver has so far been tested on the J721S2 EVM board and
>>> pre-silicon FPGA.
>>>
>>> Testing on the J721S2 EVM board in combination with the GStreamer v4l2
>>> good plugin shows the decoder is working correctly, it has been tested
>>> with H264 & HEVC video clips with multiple different dimensions (4k,
>>> 1080p, 720p, 540p, 360p, 270p) and various lengths. Additionally,
>>> Fluster conformance tests have been run for both supported codecs.
>>>
>>> The encoder can encode various videos but still has some issues causing
>>> it to fail occassionally.
>>>
>>> V4L2 compliance:
>>>
>>> Total for wave5-dec device /dev/video0: 44, Succeeded: 44, Failed: 0, Warnings: 0
>>>
>>> Buffer ioctls:
>>> fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(610): q.reqbufs(node, 1)
>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(753): q.reqbufs(node, 2)
>>> test VIDIOC_EXPBUF: FAIL
>>> test Requests: OK (Not Supported)
>>>
>>> Total for wave5-enc device /dev/video1: 44, Succeeded: 42, Failed: 2, Warnings: 2
>>>
>>> changes since v10:
>>>
>>> * Remove structure member from the encoder and decoder output info
>>> structs, that have assigned values from the registers but aren't used
>>> in the driver, add comments to describe the register values in the
>>> register definitions
>>> * Fix issue with decoding videos with a dimension where the height is
>>> not a multiple of 16 (270, 360, 540, 1024 etc.)
>>> * Fix incorrect variable format identifiers in printks
>>> * Use debug logs in loops to avoid flooding the message log
>>> * Use the swap() function instead of manual swapping of two values
>>> * Add extended controls for the encoder
>>> * Fix control flow issue while handling bitstream buffers, where an
>>> error while writing the source buffer into the hardware ring buffer
>>> would result in skipping the problematic buffer, which in turn causes
>>> a reordering of source buffers
>>> * Use the rectangle format as described by the hardware, the hardware
>>> uses for rectangles like the display rectangle 4 offsets (top, bottom,
>>> left, right), which depict the offset from the respective edge. Use
>>> this format instead of implicitly converting the bottom and right
>>> attributes to width and height attributes.
>>> * Return an error upon reading the sequence header while STREAMON
>>> * Squash the VDI and the VPUAPI layer commits as they had circular
>>> dependencies
>>>
>>> changes since v9:
>>>
>>> * Move from staging to the media directory
>>> * Move coda driver to sub-directory
>>>
>>> * Fixes:
>>> * Use platform_get_irq instead of platform_get_resource to fetch the IRQ
>>>
>>> * General cleanups:
>>> * Add missing error messages to error conditions
>>> * Improve messages/variable names/comments, align parameter names across the driver
>>> * Use macros instead of magic numbers in multiple occassions
>>> * Reduce code duplication in multiple places
>>> * Fix whitespace, newline and tab alignment issues
>>> * Remove unused struct fields & commented out code
>>> * Convert signed integers to unsigned if signed is not necessary
>>> * Convert int/unsigned int to s32/u32, when the variable is assigned to the
>>> return of a register read or provided as a parameter for a register write
>>> (and vice versa)
>>> * Fix incorrect bitwise operators where logical operators are appropriate
>>> * Multiple smaller changes
>>>
>>> * Generalization:
>>> * Add new helper file providing generalized routines for vpu-dec & vpu-enc
>>> * Generalize luma & chroma table size calculation and stride calculation
>>>
>>> * Resource cleanup and error handling:
>>> * Add error handling to all calls with ignored return codes
>>> * Handle DMA resource cleanup properly
>>> * Fix insufficient instance cleanup while opening dec/enc
>>>
>>> changes since v8:
>>>
>>> * add 'wave5' to DEV_NAME
>>> * update to support Multi-stream
>>> * update to support loop test/dynamic resolution change
>>> * remove unnecessary memset, g_volatile, old version option
>>>
>>> changes since v7:
>>>
>>> * update v4l2-compliance test report
>>> * fix build error on linux-kernel 5.18.0-rc4
>>>
>>> changes since v6:
>>>
>>> * update TODO file
>>> * get sram info from device tree
>>>
>>> changes since v5:
>>>
>>> * support NV12/NV21 pixelformat for encoder and decoder
>>> * handle adnormal exit and EOS
>>>
>>> changes since v4:
>>>
>>> * refactor functions in wave5-hw and fix bug reported by Daniel Palmer
>>> * rename functions and variables to better names
>>> * change variable types such as replacing s32 with u32 and int with bool
>>> * as appropriate
>>>
>>> changes since v3:
>>>
>>> * Fixing all issues commented by Dan Carpenter
>>> * Change file names to have wave5- prefix
>>> * In wave5_vpu_probe, enable the clocks before reading registers, as
>>> * commented from Daniel Palmer
>>> * Add more to the TODO list,
>>>
>>> changes since v2:
>>>
>>> Main fixes includes:
>>> * change the yaml and dirver code to support up to 4 clks (instead of
>>> * one)
>>> * fix Kconfig format
>>> * remove unneeded cast,
>>> * change var types
>>> * change var names, func names
>>> * checkpatch fixes
>>>
>>> changes since v1:
>>>
>>> Fix changes due to comments from Ezequiel and Dan Carpenter. Main fixes
>>> inclueds:
>>> * move all files to one dir 'wave5'
>>> * replace private error codes with standard error codes
>>> * fix extra spaces
>>> * various checkpatch fixes
>>> * replace private 'DPRINTK' macro with standard 'dev_err/dbg ..'
>>> * fix error handling
>>> * add more possible fixes to the TODO file
>>>
>>
>> We used the patchset on JH7110 SoC which is integrated with a wave511 VPU.
>> We see issue that the decoder output contains garbage data in the image
>> when decoding some of HEVC videos, in 1080p or 4k. But Most of the videos
>> we tested decoded well.
>>
>> Beside, ffmpeg h264_v4l2m2m could not get the correct resolution via
>> VIDIOC_G_FMT or VIDIOC_TRY_FMT for 1080p AVC video.
>
> Thanks for the report, Sebastian has been working on a v12, which includes
> important rework regarding format negotiation, hopefully this will help. One of
> the pressing issue was that the original driver this effort was derived on was
> designed to work on ancient GStreamer (1.16) and would simply not work at all on
> new version, so this has been taking some time. It was also highlighted that
> h264_v4l2m2m may run out of ram on JH7110 in absence of V4L2_CID_MIN_BUFFER*
> controls, so we will be adding these too.
>
> p.s. if you are allowed to share the streams (or a sample of it) you have had
> issue with, Sebastian can certainly add this to his validation.
>
> regards,
> Nicolas
>
Hi,Nicolas
Sure we will share the test stream we met issue, via Wetransfer net
disk. I'm not sure if this is a driver issue or a soc cache issue, it
not occurs with any stream. Hope you can clarify it.
It did run out the ram with h264_v4l2m2m if not do some modify, it
should be due to alignment.
p.s. are you interested in using 7110soc for verification? We can
provide the board for it.
Thanks.
Som
>>
>>> Nas Chung (3):
>>> media: chips-media: wave5: Add vpuapi layer
>>> media: chips-media: wave5: Add the v4l2 layer
>>> media: chips-media: wave5: Add TODO file
>>>
>>> Robert Beckett (2):
>>> dt-bindings: media: wave5: add yaml devicetree bindings
>>> media: chips-media: wave5: Add wave5 driver to maintainers file
>>>
>>> Sebastian Fricke (1):
>>> media: platform: chips-media: Move Coda to separate folder
>>>
>>> .../devicetree/bindings/cnm,wave5.yml | 72 +
>>> MAINTAINERS | 11 +-
>>> drivers/media/platform/chips-media/Kconfig | 18 +-
>>> drivers/media/platform/chips-media/Makefile | 6 +-
>>> .../media/platform/chips-media/coda/Kconfig | 18 +
>>> .../media/platform/chips-media/coda/Makefile | 6 +
>>> .../chips-media/{ => coda}/coda-bit.c | 0
>>> .../chips-media/{ => coda}/coda-common.c | 0
>>> .../chips-media/{ => coda}/coda-gdi.c | 0
>>> .../chips-media/{ => coda}/coda-h264.c | 0
>>> .../chips-media/{ => coda}/coda-jpeg.c | 0
>>> .../chips-media/{ => coda}/coda-mpeg2.c | 0
>>> .../chips-media/{ => coda}/coda-mpeg4.c | 0
>>> .../platform/chips-media/{ => coda}/coda.h | 0
>>> .../chips-media/{ => coda}/coda_regs.h | 0
>>> .../chips-media/{ => coda}/imx-vdoa.c | 0
>>> .../chips-media/{ => coda}/imx-vdoa.h | 0
>>> .../platform/chips-media/{ => coda}/trace.h | 0
>>> .../media/platform/chips-media/wave5/Kconfig | 12 +
>>> .../media/platform/chips-media/wave5/Makefile | 10 +
>>> drivers/media/platform/chips-media/wave5/TODO | 18 +
>>> .../platform/chips-media/wave5/wave5-helper.c | 175 +
>>> .../platform/chips-media/wave5/wave5-helper.h | 28 +
>>> .../platform/chips-media/wave5/wave5-hw.c | 3359 +++++++++++++++++
>>> .../chips-media/wave5/wave5-regdefine.h | 743 ++++
>>> .../platform/chips-media/wave5/wave5-vdi.c | 245 ++
>>> .../platform/chips-media/wave5/wave5-vdi.h | 67 +
>>> .../chips-media/wave5/wave5-vpu-dec.c | 1429 +++++++
>>> .../chips-media/wave5/wave5-vpu-enc.c | 1746 +++++++++
>>> .../platform/chips-media/wave5/wave5-vpu.c | 362 ++
>>> .../platform/chips-media/wave5/wave5-vpu.h | 72 +
>>> .../platform/chips-media/wave5/wave5-vpuapi.c | 1040 +++++
>>> .../platform/chips-media/wave5/wave5-vpuapi.h | 1136 ++++++
>>> .../chips-media/wave5/wave5-vpuconfig.h | 90 +
>>> .../chips-media/wave5/wave5-vpuerror.h | 454 +++
>>> .../media/platform/chips-media/wave5/wave5.h | 94 +
>>> 36 files changed, 11190 insertions(+), 21 deletions(-)
>>> create mode 100644 Documentation/devicetree/bindings/cnm,wave5.yml
>>> create mode 100644 drivers/media/platform/chips-media/coda/Kconfig
>>> create mode 100644 drivers/media/platform/chips-media/coda/Makefile
>>> rename drivers/media/platform/chips-media/{ => coda}/coda-bit.c (100%)
>>> rename drivers/media/platform/chips-media/{ => coda}/coda-common.c (100%)
>>> rename drivers/media/platform/chips-media/{ => coda}/coda-gdi.c (100%)
>>> rename drivers/media/platform/chips-media/{ => coda}/coda-h264.c (100%)
>>> rename drivers/media/platform/chips-media/{ => coda}/coda-jpeg.c (100%)
>>> rename drivers/media/platform/chips-media/{ => coda}/coda-mpeg2.c (100%)
>>> rename drivers/media/platform/chips-media/{ => coda}/coda-mpeg4.c (100%)
>>> rename drivers/media/platform/chips-media/{ => coda}/coda.h (100%)
>>> rename drivers/media/platform/chips-media/{ => coda}/coda_regs.h (100%)
>>> rename drivers/media/platform/chips-media/{ => coda}/imx-vdoa.c (100%)
>>> rename drivers/media/platform/chips-media/{ => coda}/imx-vdoa.h (100%)
>>> rename drivers/media/platform/chips-media/{ => coda}/trace.h (100%)
>>> create mode 100644 drivers/media/platform/chips-media/wave5/Kconfig
>>> create mode 100644 drivers/media/platform/chips-media/wave5/Makefile
>>> create mode 100644 drivers/media/platform/chips-media/wave5/TODO
>>> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-helper.c
>>> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-helper.h
>>> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-hw.c
>>> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-regdefine.h
>>> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vdi.c
>>> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vdi.h
>>> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
>>> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
>>> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu.c
>>> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu.h
>>> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuapi.c
>>> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuapi.h
>>> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
>>> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuerror.h
>>> create mode 100644 drivers/media/platform/chips-media/wave5/wave5.h
>>>
>>> --
>>> 2.25.1
>>>
>>
>
>
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v11 0/6] Wave5 codec driver
2023-02-14 7:11 ` Som Qin
@ 2023-02-14 20:12 ` Nicolas Dufresne
2023-02-15 2:44 ` Som Qin
0 siblings, 1 reply; 7+ messages in thread
From: Nicolas Dufresne @ 2023-02-14 20:12 UTC (permalink / raw)
To: Som Qin, Sebastian Fricke, linux-media
Cc: kernel, bob.beckett, hverkuil-cisco, nas.chung
Hi Som,
Le mardi 14 février 2023 à 15:11 +0800, Som Qin a écrit :
> Sure we will share the test stream we met issue, via Wetransfer net
> disk. I'm not sure if this is a driver issue or a soc cache issue, it
> not occurs with any stream. Hope you can clarify it.
thanks for the stream, I have shared them internally with Sebastian who will
test them on his board (a TI board). We'll be able to confirm with you if we can
also reproduce the issues there, and that will help regarding possible SoC cache
issues.
>
> It did run out the ram with h264_v4l2m2m if not do some modify, it
> should be due to alignment.
What LibreELEC dev told me is that in absence of V4L2_CID_MIN_BUFFER_FOR*
controls, ffmpeg will allocate too many buffers and cause ram issue. We will
have this implemented in later update, so hopefully this problem will go away.
>
> p.s. are you interested in using 7110soc for verification? We can
> provide the board for it.
Can you provide a status of what has been fixed from the SoC revision found on
BeagleV ? We stopped using the BeagleV for testing has it has SoC issue and no
known workaround for.
regards,
Nicolas
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH v11 0/6] Wave5 codec driver
2023-02-14 20:12 ` Nicolas Dufresne
@ 2023-02-15 2:44 ` Som Qin
0 siblings, 0 replies; 7+ messages in thread
From: Som Qin @ 2023-02-15 2:44 UTC (permalink / raw)
To: Nicolas Dufresne, Sebastian Fricke, linux-media
Cc: kernel, bob.beckett, hverkuil-cisco, nas.chung
On 2023/2/15 4:12, Nicolas Dufresne wrote:
> Hi Som,
>
> Le mardi 14 février 2023 à 15:11 +0800, Som Qin a écrit :
>> Sure we will share the test stream we met issue, via Wetransfer net
>> disk. I'm not sure if this is a driver issue or a soc cache issue, it
>> not occurs with any stream. Hope you can clarify it.
>
> thanks for the stream, I have shared them internally with Sebastian who will
> test them on his board (a TI board). We'll be able to confirm with you if we can
> also reproduce the issues there, and that will help regarding possible SoC cache
> issues.
>
Thank you for help.
>>
>> It did run out the ram with h264_v4l2m2m if not do some modify, it
>> should be due to alignment.
>
> What LibreELEC dev told me is that in absence of V4L2_CID_MIN_BUFFER_FOR*
> controls, ffmpeg will allocate too many buffers and cause ram issue. We will
> have this implemented in later update, so hopefully this problem will go away.
>
Based on what I've tried, ffmpeg re-allocate buffers as the width/height
was changed after VIDIOC_S_FMT/VIDIOC_G_FMT, with 1080P AVC.
>>
>> p.s. are you interested in using 7110soc for verification? We can
>> provide the board for it.
>
> Can you provide a status of what has been fixed from the SoC revision found on
> BeagleV ? We stopped using the BeagleV for testing has it has SoC issue and no
> known workaround for.
>
We are using board VisionFive 2, basically fixed the known soc issues in
the old BeagleV. Wave511 work quite well with omx-il on the board.
Thanks.
Som
> regards,
> Nicolas
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH v11 0/6] Wave5 codec driver
@ 2022-12-07 12:13 Sebastian Fricke
2023-04-26 13:16 ` Nicolas Dufresne
0 siblings, 1 reply; 7+ messages in thread
From: Sebastian Fricke @ 2022-12-07 12:13 UTC (permalink / raw)
To: linux-media
Cc: kernel, bob.beckett, hverkuil-cisco, nicolas.dufresne, nas.chung,
Sebastian Fricke
The Wave5 codec driver is a stateful encoder/decoder.
It is found on the J721S2 SoC, JH7100 SoC, and ssd202d SoC. etc.
But the current test report is based on J721S2 SoC and pre-silicon FPGA.
The driver currently supports V4L2_PIX_FMT_HEVC and V4L2_PIX_FMT_H264.
This driver has so far been tested on the J721S2 EVM board and
pre-silicon FPGA.
Testing on the J721S2 EVM board in combination with the GStreamer v4l2
good plugin shows the decoder is working correctly, it has been tested
with H264 & HEVC video clips with multiple different dimensions (4k,
1080p, 720p, 540p, 360p, 270p) and various lengths. Additionally,
Fluster conformance tests have been run for both supported codecs.
The encoder can encode various videos but still has some issues causing
it to fail occassionally.
V4L2 compliance:
Total for wave5-dec device /dev/video0: 44, Succeeded: 44, Failed: 0, Warnings: 0
Buffer ioctls:
fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(610): q.reqbufs(node, 1)
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(753): q.reqbufs(node, 2)
test VIDIOC_EXPBUF: FAIL
test Requests: OK (Not Supported)
Total for wave5-enc device /dev/video1: 44, Succeeded: 42, Failed: 2, Warnings: 2
changes since v10:
* Remove structure member from the encoder and decoder output info
structs, that have assigned values from the registers but aren't used
in the driver, add comments to describe the register values in the
register definitions
* Fix issue with decoding videos with a dimension where the height is
not a multiple of 16 (270, 360, 540, 1024 etc.)
* Fix incorrect variable format identifiers in printks
* Use debug logs in loops to avoid flooding the message log
* Use the swap() function instead of manual swapping of two values
* Add extended controls for the encoder
* Fix control flow issue while handling bitstream buffers, where an
error while writing the source buffer into the hardware ring buffer
would result in skipping the problematic buffer, which in turn causes
a reordering of source buffers
* Use the rectangle format as described by the hardware, the hardware
uses for rectangles like the display rectangle 4 offsets (top, bottom,
left, right), which depict the offset from the respective edge. Use
this format instead of implicitly converting the bottom and right
attributes to width and height attributes.
* Return an error upon reading the sequence header while STREAMON
* Squash the VDI and the VPUAPI layer commits as they had circular
dependencies
changes since v9:
* Move from staging to the media directory
* Move coda driver to sub-directory
* Fixes:
* Use platform_get_irq instead of platform_get_resource to fetch the IRQ
* General cleanups:
* Add missing error messages to error conditions
* Improve messages/variable names/comments, align parameter names across the driver
* Use macros instead of magic numbers in multiple occassions
* Reduce code duplication in multiple places
* Fix whitespace, newline and tab alignment issues
* Remove unused struct fields & commented out code
* Convert signed integers to unsigned if signed is not necessary
* Convert int/unsigned int to s32/u32, when the variable is assigned to the
return of a register read or provided as a parameter for a register write
(and vice versa)
* Fix incorrect bitwise operators where logical operators are appropriate
* Multiple smaller changes
* Generalization:
* Add new helper file providing generalized routines for vpu-dec & vpu-enc
* Generalize luma & chroma table size calculation and stride calculation
* Resource cleanup and error handling:
* Add error handling to all calls with ignored return codes
* Handle DMA resource cleanup properly
* Fix insufficient instance cleanup while opening dec/enc
changes since v8:
* add 'wave5' to DEV_NAME
* update to support Multi-stream
* update to support loop test/dynamic resolution change
* remove unnecessary memset, g_volatile, old version option
changes since v7:
* update v4l2-compliance test report
* fix build error on linux-kernel 5.18.0-rc4
changes since v6:
* update TODO file
* get sram info from device tree
changes since v5:
* support NV12/NV21 pixelformat for encoder and decoder
* handle adnormal exit and EOS
changes since v4:
* refactor functions in wave5-hw and fix bug reported by Daniel Palmer
* rename functions and variables to better names
* change variable types such as replacing s32 with u32 and int with bool
* as appropriate
changes since v3:
* Fixing all issues commented by Dan Carpenter
* Change file names to have wave5- prefix
* In wave5_vpu_probe, enable the clocks before reading registers, as
* commented from Daniel Palmer
* Add more to the TODO list,
changes since v2:
Main fixes includes:
* change the yaml and dirver code to support up to 4 clks (instead of
* one)
* fix Kconfig format
* remove unneeded cast,
* change var types
* change var names, func names
* checkpatch fixes
changes since v1:
Fix changes due to comments from Ezequiel and Dan Carpenter. Main fixes
inclueds:
* move all files to one dir 'wave5'
* replace private error codes with standard error codes
* fix extra spaces
* various checkpatch fixes
* replace private 'DPRINTK' macro with standard 'dev_err/dbg ..'
* fix error handling
* add more possible fixes to the TODO file
Nas Chung (3):
media: chips-media: wave5: Add vpuapi layer
media: chips-media: wave5: Add the v4l2 layer
media: chips-media: wave5: Add TODO file
Robert Beckett (2):
dt-bindings: media: wave5: add yaml devicetree bindings
media: chips-media: wave5: Add wave5 driver to maintainers file
Sebastian Fricke (1):
media: platform: chips-media: Move Coda to separate folder
.../devicetree/bindings/cnm,wave5.yml | 72 +
MAINTAINERS | 11 +-
drivers/media/platform/chips-media/Kconfig | 18 +-
drivers/media/platform/chips-media/Makefile | 6 +-
.../media/platform/chips-media/coda/Kconfig | 18 +
.../media/platform/chips-media/coda/Makefile | 6 +
.../chips-media/{ => coda}/coda-bit.c | 0
.../chips-media/{ => coda}/coda-common.c | 0
.../chips-media/{ => coda}/coda-gdi.c | 0
.../chips-media/{ => coda}/coda-h264.c | 0
.../chips-media/{ => coda}/coda-jpeg.c | 0
.../chips-media/{ => coda}/coda-mpeg2.c | 0
.../chips-media/{ => coda}/coda-mpeg4.c | 0
.../platform/chips-media/{ => coda}/coda.h | 0
.../chips-media/{ => coda}/coda_regs.h | 0
.../chips-media/{ => coda}/imx-vdoa.c | 0
.../chips-media/{ => coda}/imx-vdoa.h | 0
.../platform/chips-media/{ => coda}/trace.h | 0
.../media/platform/chips-media/wave5/Kconfig | 12 +
.../media/platform/chips-media/wave5/Makefile | 10 +
drivers/media/platform/chips-media/wave5/TODO | 18 +
.../platform/chips-media/wave5/wave5-helper.c | 175 +
.../platform/chips-media/wave5/wave5-helper.h | 28 +
.../platform/chips-media/wave5/wave5-hw.c | 3359 +++++++++++++++++
.../chips-media/wave5/wave5-regdefine.h | 743 ++++
.../platform/chips-media/wave5/wave5-vdi.c | 245 ++
.../platform/chips-media/wave5/wave5-vdi.h | 67 +
.../chips-media/wave5/wave5-vpu-dec.c | 1429 +++++++
.../chips-media/wave5/wave5-vpu-enc.c | 1746 +++++++++
.../platform/chips-media/wave5/wave5-vpu.c | 362 ++
.../platform/chips-media/wave5/wave5-vpu.h | 72 +
.../platform/chips-media/wave5/wave5-vpuapi.c | 1040 +++++
.../platform/chips-media/wave5/wave5-vpuapi.h | 1136 ++++++
.../chips-media/wave5/wave5-vpuconfig.h | 90 +
.../chips-media/wave5/wave5-vpuerror.h | 454 +++
.../media/platform/chips-media/wave5/wave5.h | 94 +
36 files changed, 11190 insertions(+), 21 deletions(-)
create mode 100644 Documentation/devicetree/bindings/cnm,wave5.yml
create mode 100644 drivers/media/platform/chips-media/coda/Kconfig
create mode 100644 drivers/media/platform/chips-media/coda/Makefile
rename drivers/media/platform/chips-media/{ => coda}/coda-bit.c (100%)
rename drivers/media/platform/chips-media/{ => coda}/coda-common.c (100%)
rename drivers/media/platform/chips-media/{ => coda}/coda-gdi.c (100%)
rename drivers/media/platform/chips-media/{ => coda}/coda-h264.c (100%)
rename drivers/media/platform/chips-media/{ => coda}/coda-jpeg.c (100%)
rename drivers/media/platform/chips-media/{ => coda}/coda-mpeg2.c (100%)
rename drivers/media/platform/chips-media/{ => coda}/coda-mpeg4.c (100%)
rename drivers/media/platform/chips-media/{ => coda}/coda.h (100%)
rename drivers/media/platform/chips-media/{ => coda}/coda_regs.h (100%)
rename drivers/media/platform/chips-media/{ => coda}/imx-vdoa.c (100%)
rename drivers/media/platform/chips-media/{ => coda}/imx-vdoa.h (100%)
rename drivers/media/platform/chips-media/{ => coda}/trace.h (100%)
create mode 100644 drivers/media/platform/chips-media/wave5/Kconfig
create mode 100644 drivers/media/platform/chips-media/wave5/Makefile
create mode 100644 drivers/media/platform/chips-media/wave5/TODO
create mode 100644 drivers/media/platform/chips-media/wave5/wave5-helper.c
create mode 100644 drivers/media/platform/chips-media/wave5/wave5-helper.h
create mode 100644 drivers/media/platform/chips-media/wave5/wave5-hw.c
create mode 100644 drivers/media/platform/chips-media/wave5/wave5-regdefine.h
create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vdi.c
create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vdi.h
create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu.c
create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu.h
create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuapi.c
create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuapi.h
create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuerror.h
create mode 100644 drivers/media/platform/chips-media/wave5/wave5.h
--
2.25.1
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v11 0/6] Wave5 codec driver
2022-12-07 12:13 Sebastian Fricke
@ 2023-04-26 13:16 ` Nicolas Dufresne
0 siblings, 0 replies; 7+ messages in thread
From: Nicolas Dufresne @ 2023-04-26 13:16 UTC (permalink / raw)
To: Sebastian Fricke, linux-media
Cc: kernel, bob.beckett, hverkuil-cisco, nas.chung
Hi Sebastien and all,
quick disclosure that I have started to work with Sebastien on reworking this
driver. But to ease the work, I'll start with code review, trying to identify
as many of the found issues as possible.
Le mercredi 07 décembre 2022 à 13:13 +0100, Sebastian Fricke a écrit :
> The Wave5 codec driver is a stateful encoder/decoder.
> It is found on the J721S2 SoC, JH7100 SoC, and ssd202d SoC. etc.
> But the current test report is based on J721S2 SoC and pre-silicon FPGA.
>
> The driver currently supports V4L2_PIX_FMT_HEVC and V4L2_PIX_FMT_H264.
>
> This driver has so far been tested on the J721S2 EVM board and
> pre-silicon FPGA.
>
> Testing on the J721S2 EVM board in combination with the GStreamer v4l2
> good plugin shows the decoder is working correctly, it has been tested
> with H264 & HEVC video clips with multiple different dimensions (4k,
> 1080p, 720p, 540p, 360p, 270p) and various lengths. Additionally,
> Fluster conformance tests have been run for both supported codecs.
This comment is missing which GStreamer version has been tested. What was
reported to me is that GStreamer 1.16 from TI BSP was the only tested software
version. As a consequence, the SOURCE_CHANGE implementation in this series is
largely untested. In order to make 1.16 to work with streams that needs larger
DPB, some synchronous operation (can be in streamon(output), QBUF or even G_FMT
like Samsung MFC do (for legacy support reason) must exist in this driver. This
is not quite acceptable for a newly written driver. Newly written driver should
follow the spec, and application needs to be modernized to work appropriately.
In short, we don't do forward compatibility.
For those that would ask why CODA960 driver works (in most cases) without
SOURCE_CHANGE, this is because that driver does not expose its reference frames
as the display frames. As a side effect, the DPB count is managed internally.
The delivered frames are transformed copies, hence we don't need to match the
DPB size when allocating these in the applicaiton.
>
> The encoder can encode various videos but still has some issues causing
> it to fail occassionally.
>
> V4L2 compliance:
>
> Total for wave5-dec device /dev/video0: 44, Succeeded: 44, Failed: 0, Warnings: 0
>
> Buffer ioctls:
> fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(610): q.reqbufs(node, 1)
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
> fail: ../../../v4l-utils-1.18.1/utils/v4l2-compliance/v4l2-test-buffers.cpp(753): q.reqbufs(node, 2)
> test VIDIOC_EXPBUF: FAIL
> test Requests: OK (Not Supported)
>
> Total for wave5-enc device /dev/video1: 44, Succeeded: 42, Failed: 2, Warnings: 2
>
> changes since v10:
>
> * Remove structure member from the encoder and decoder output info
> structs, that have assigned values from the registers but aren't used
> in the driver, add comments to describe the register values in the
> register definitions
> * Fix issue with decoding videos with a dimension where the height is
> not a multiple of 16 (270, 360, 540, 1024 etc.)
> * Fix incorrect variable format identifiers in printks
> * Use debug logs in loops to avoid flooding the message log
> * Use the swap() function instead of manual swapping of two values
> * Add extended controls for the encoder
> * Fix control flow issue while handling bitstream buffers, where an
> error while writing the source buffer into the hardware ring buffer
> would result in skipping the problematic buffer, which in turn causes
> a reordering of source buffers
> * Use the rectangle format as described by the hardware, the hardware
> uses for rectangles like the display rectangle 4 offsets (top, bottom,
> left, right), which depict the offset from the respective edge. Use
> this format instead of implicitly converting the bottom and right
> attributes to width and height attributes.
> * Return an error upon reading the sequence header while STREAMON
> * Squash the VDI and the VPUAPI layer commits as they had circular
> dependencies
>
> changes since v9:
>
> * Move from staging to the media directory
> * Move coda driver to sub-directory
>
> * Fixes:
> * Use platform_get_irq instead of platform_get_resource to fetch the IRQ
>
> * General cleanups:
> * Add missing error messages to error conditions
> * Improve messages/variable names/comments, align parameter names across the driver
> * Use macros instead of magic numbers in multiple occassions
> * Reduce code duplication in multiple places
> * Fix whitespace, newline and tab alignment issues
> * Remove unused struct fields & commented out code
> * Convert signed integers to unsigned if signed is not necessary
> * Convert int/unsigned int to s32/u32, when the variable is assigned to the
> return of a register read or provided as a parameter for a register write
> (and vice versa)
> * Fix incorrect bitwise operators where logical operators are appropriate
> * Multiple smaller changes
>
> * Generalization:
> * Add new helper file providing generalized routines for vpu-dec & vpu-enc
> * Generalize luma & chroma table size calculation and stride calculation
>
> * Resource cleanup and error handling:
> * Add error handling to all calls with ignored return codes
> * Handle DMA resource cleanup properly
> * Fix insufficient instance cleanup while opening dec/enc
>
> changes since v8:
>
> * add 'wave5' to DEV_NAME
> * update to support Multi-stream
> * update to support loop test/dynamic resolution change
> * remove unnecessary memset, g_volatile, old version option
>
> changes since v7:
>
> * update v4l2-compliance test report
> * fix build error on linux-kernel 5.18.0-rc4
>
> changes since v6:
>
> * update TODO file
> * get sram info from device tree
>
> changes since v5:
>
> * support NV12/NV21 pixelformat for encoder and decoder
> * handle adnormal exit and EOS
>
> changes since v4:
>
> * refactor functions in wave5-hw and fix bug reported by Daniel Palmer
> * rename functions and variables to better names
> * change variable types such as replacing s32 with u32 and int with bool
> * as appropriate
>
> changes since v3:
>
> * Fixing all issues commented by Dan Carpenter
> * Change file names to have wave5- prefix
> * In wave5_vpu_probe, enable the clocks before reading registers, as
> * commented from Daniel Palmer
> * Add more to the TODO list,
>
> changes since v2:
>
> Main fixes includes:
> * change the yaml and dirver code to support up to 4 clks (instead of
> * one)
> * fix Kconfig format
> * remove unneeded cast,
> * change var types
> * change var names, func names
> * checkpatch fixes
>
> changes since v1:
>
> Fix changes due to comments from Ezequiel and Dan Carpenter. Main fixes
> inclueds:
> * move all files to one dir 'wave5'
> * replace private error codes with standard error codes
> * fix extra spaces
> * various checkpatch fixes
> * replace private 'DPRINTK' macro with standard 'dev_err/dbg ..'
> * fix error handling
> * add more possible fixes to the TODO file
>
> Nas Chung (3):
> media: chips-media: wave5: Add vpuapi layer
> media: chips-media: wave5: Add the v4l2 layer
> media: chips-media: wave5: Add TODO file
>
> Robert Beckett (2):
> dt-bindings: media: wave5: add yaml devicetree bindings
> media: chips-media: wave5: Add wave5 driver to maintainers file
>
> Sebastian Fricke (1):
> media: platform: chips-media: Move Coda to separate folder
>
> .../devicetree/bindings/cnm,wave5.yml | 72 +
> MAINTAINERS | 11 +-
> drivers/media/platform/chips-media/Kconfig | 18 +-
> drivers/media/platform/chips-media/Makefile | 6 +-
> .../media/platform/chips-media/coda/Kconfig | 18 +
> .../media/platform/chips-media/coda/Makefile | 6 +
> .../chips-media/{ => coda}/coda-bit.c | 0
> .../chips-media/{ => coda}/coda-common.c | 0
> .../chips-media/{ => coda}/coda-gdi.c | 0
> .../chips-media/{ => coda}/coda-h264.c | 0
> .../chips-media/{ => coda}/coda-jpeg.c | 0
> .../chips-media/{ => coda}/coda-mpeg2.c | 0
> .../chips-media/{ => coda}/coda-mpeg4.c | 0
> .../platform/chips-media/{ => coda}/coda.h | 0
> .../chips-media/{ => coda}/coda_regs.h | 0
> .../chips-media/{ => coda}/imx-vdoa.c | 0
> .../chips-media/{ => coda}/imx-vdoa.h | 0
> .../platform/chips-media/{ => coda}/trace.h | 0
> .../media/platform/chips-media/wave5/Kconfig | 12 +
> .../media/platform/chips-media/wave5/Makefile | 10 +
> drivers/media/platform/chips-media/wave5/TODO | 18 +
> .../platform/chips-media/wave5/wave5-helper.c | 175 +
> .../platform/chips-media/wave5/wave5-helper.h | 28 +
> .../platform/chips-media/wave5/wave5-hw.c | 3359 +++++++++++++++++
> .../chips-media/wave5/wave5-regdefine.h | 743 ++++
> .../platform/chips-media/wave5/wave5-vdi.c | 245 ++
> .../platform/chips-media/wave5/wave5-vdi.h | 67 +
> .../chips-media/wave5/wave5-vpu-dec.c | 1429 +++++++
> .../chips-media/wave5/wave5-vpu-enc.c | 1746 +++++++++
> .../platform/chips-media/wave5/wave5-vpu.c | 362 ++
> .../platform/chips-media/wave5/wave5-vpu.h | 72 +
> .../platform/chips-media/wave5/wave5-vpuapi.c | 1040 +++++
> .../platform/chips-media/wave5/wave5-vpuapi.h | 1136 ++++++
> .../chips-media/wave5/wave5-vpuconfig.h | 90 +
> .../chips-media/wave5/wave5-vpuerror.h | 454 +++
> .../media/platform/chips-media/wave5/wave5.h | 94 +
> 36 files changed, 11190 insertions(+), 21 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/cnm,wave5.yml
> create mode 100644 drivers/media/platform/chips-media/coda/Kconfig
> create mode 100644 drivers/media/platform/chips-media/coda/Makefile
> rename drivers/media/platform/chips-media/{ => coda}/coda-bit.c (100%)
> rename drivers/media/platform/chips-media/{ => coda}/coda-common.c (100%)
> rename drivers/media/platform/chips-media/{ => coda}/coda-gdi.c (100%)
> rename drivers/media/platform/chips-media/{ => coda}/coda-h264.c (100%)
> rename drivers/media/platform/chips-media/{ => coda}/coda-jpeg.c (100%)
> rename drivers/media/platform/chips-media/{ => coda}/coda-mpeg2.c (100%)
> rename drivers/media/platform/chips-media/{ => coda}/coda-mpeg4.c (100%)
> rename drivers/media/platform/chips-media/{ => coda}/coda.h (100%)
> rename drivers/media/platform/chips-media/{ => coda}/coda_regs.h (100%)
> rename drivers/media/platform/chips-media/{ => coda}/imx-vdoa.c (100%)
> rename drivers/media/platform/chips-media/{ => coda}/imx-vdoa.h (100%)
> rename drivers/media/platform/chips-media/{ => coda}/trace.h (100%)
> create mode 100644 drivers/media/platform/chips-media/wave5/Kconfig
> create mode 100644 drivers/media/platform/chips-media/wave5/Makefile
> create mode 100644 drivers/media/platform/chips-media/wave5/TODO
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-helper.c
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-helper.h
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-hw.c
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-regdefine.h
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vdi.c
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vdi.h
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu-dec.c
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu.c
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpu.h
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuapi.c
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuapi.h
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuconfig.h
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5-vpuerror.h
> create mode 100644 drivers/media/platform/chips-media/wave5/wave5.h
>
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2023-04-26 13:17 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-02-13 6:13 [PATCH v11 0/6] Wave5 codec driver Som Qin
2023-02-13 14:13 ` Nicolas Dufresne
2023-02-14 7:11 ` Som Qin
2023-02-14 20:12 ` Nicolas Dufresne
2023-02-15 2:44 ` Som Qin
-- strict thread matches above, loose matches on Subject: below --
2022-12-07 12:13 Sebastian Fricke
2023-04-26 13:16 ` Nicolas Dufresne
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.