From: Hans Verkuil <hverkuil@xs4all.nl>
To: Shengjiu Wang <shengjiu.wang@nxp.com>,
sakari.ailus@iki.fi, tfiga@chromium.org,
m.szyprowski@samsung.com, mchehab@kernel.org,
linux-media@vger.kernel.org, linux-kernel@vger.kernel.org,
shengjiu.wang@gmail.com, Xiubo.Lee@gmail.com, festevam@gmail.com,
nicoleotsuka@gmail.com, lgirdwood@gmail.com, broonie@kernel.org,
perex@perex.cz, tiwai@suse.com, alsa-devel@alsa-project.org,
linuxppc-dev@lists.ozlabs.org
Subject: Re: [PATCH v14 00/16] Add audio support in v4l2 framework
Date: Mon, 18 Mar 2024 16:56:47 +0100 [thread overview]
Message-ID: <399da377-c2b6-4226-9108-8a1e776b6f3b@xs4all.nl> (raw)
In-Reply-To: <1710150841-26991-1-git-send-email-shengjiu.wang@nxp.com>
Hi Shengjiu,
I was about to prepare a pull request for this when I realized that
entries for the new drivers need to be added to the MAINTAINERS file.
Can you post patches 'v14 17/16' and 'v14/18/16' adding entries for each
new driver?
Thank you,
Hans
On 11/03/2024 10:53 am, Shengjiu Wang wrote:
> Audio signal processing also has the requirement for memory to
> memory similar as Video.
>
> This asrc memory to memory (memory ->asrc->memory) case is a non
> real time use case.
>
> User fills the input buffer to the asrc module, after conversion, then asrc
> sends back the output buffer to user. So it is not a traditional ALSA playback
> and capture case.
>
> It is a specific use case, there is no reference in current kernel.
> v4l2 memory to memory is the closed implementation, v4l2 current
> support video, image, radio, tuner, touch devices, so it is not
> complicated to add support for this specific audio case.
>
> Because we had implemented the "memory -> asrc ->i2s device-> codec"
> use case in ALSA. Now the "memory->asrc->memory" needs
> to reuse the code in asrc driver, so the first 3 patches is for refining
> the code to make it can be shared by the "memory->asrc->memory"
> driver.
>
> The main change is in the v4l2 side, A /dev/vl4-audioX will be created,
> user applications only use the ioctl of v4l2 framework.
>
> Other change is to add memory to memory support for two kinds of i.MX ASRC
> module.
>
> changes in v14:
> - document the reservation of 'AUXX' fourcc format.
> - add v4l2_audfmt_to_fourcc() definition.
>
> changes in v13
> - change 'pixelformat' to 'audioformat' in dev-audio-mem2mem.rst
> - add more description for clock drift in ext-ctrls-audio-m2m.rst
> - Add "media: v4l2-ctrls: add support for fraction_bits" from Hans
> to avoid build issue for kernel test robot
>
> changes in v12
> - minor changes according to comments
> - drop min_buffers_needed = 1 and V4L2_CTRL_FLAG_UPDATE flag
> - drop bus_info
>
> changes in v11
> - add add-fixed-point-test-controls in vivid.
> - add v4l2_ctrl_fp_compose() helper function for min and max
>
> changes in v10
> - remove FIXED_POINT type
> - change code base on media: v4l2-ctrls: add support for fraction_bits
> - fix issue reported by kernel test robot
> - remove module_alias
>
> changes in v9:
> - add MEDIA_ENT_F_PROC_AUDIO_RESAMPLER.
> - add MEDIA_INTF_T_V4L_AUDIO
> - add media controller support
> - refine the vim2m-audio to support 8k<->16k conversion.
>
> changes in v8:
> - refine V4L2_CAP_AUDIO_M2M to be 0x00000008
> - update doc for FIXED_POINT
> - address comments for imx-asrc
>
> changes in v7:
> - add acked-by from Mark
> - separate commit for fixed point, m2m audio class, audio rate controls
> - use INTEGER_MENU for rate, FIXED_POINT for rate offset
> - remove used fmts
> - address other comments for Hans
>
> changes in v6:
> - use m2m_prepare/m2m_unprepare/m2m_start/m2m_stop to replace
> m2m_start_part_one/m2m_stop_part_one, m2m_start_part_two/m2m_stop_part_two.
> - change V4L2_CTRL_TYPE_ASRC_RATE to V4L2_CTRL_TYPE_FIXED_POINT
> - fix warning by kernel test rebot
> - remove some unused format V4L2_AUDIO_FMT_XX
> - Get SNDRV_PCM_FORMAT from V4L2_AUDIO_FMT in driver.
> - rename audm2m to viaudm2m.
>
> changes in v5:
> - remove V4L2_AUDIO_FMT_LPCM
> - define audio pixel format like V4L2_AUDIO_FMT_S8...
> - remove rate and format in struct v4l2_audio_format.
> - Add V4L2_CID_ASRC_SOURCE_RATE and V4L2_CID_ASRC_DEST_RATE controls
> - updata document accordingly.
>
> changes in v4:
> - update document style
> - separate V4L2_AUDIO_FMT_LPCM and V4L2_CAP_AUDIO_M2M in separate commit
>
> changes in v3:
> - Modify documents for adding audio m2m support
> - Add audio virtual m2m driver
> - Defined V4L2_AUDIO_FMT_LPCM format type for audio.
> - Defined V4L2_CAP_AUDIO_M2M capability type for audio m2m case.
> - with modification in v4l-utils, pass v4l2-compliance test.
>
> changes in v2:
> - decouple the implementation in v4l2 and ALSA
> - implement the memory to memory driver as a platfrom driver
> and move it to driver/media
> - move fsl_asrc_common.h to include/sound folder
>
> Hans Verkuil (1):
> media: v4l2-ctrls: add support for fraction_bits
>
> Shengjiu Wang (15):
> ASoC: fsl_asrc: define functions for memory to memory usage
> ASoC: fsl_easrc: define functions for memory to memory usage
> ASoC: fsl_asrc: move fsl_asrc_common.h to include/sound
> ASoC: fsl_asrc: register m2m platform device
> ASoC: fsl_easrc: register m2m platform device
> media: uapi: Add V4L2_CAP_AUDIO_M2M capability flag
> media: v4l2: Add audio capture and output support
> media: uapi: Define audio sample format fourcc type
> media: uapi: Add V4L2_CTRL_CLASS_M2M_AUDIO
> media: uapi: Add audio rate controls support
> media: uapi: Declare interface types for Audio
> media: uapi: Add an entity type for audio resampler
> media: vivid: add fixed point test controls
> media: imx-asrc: Add memory to memory driver
> media: vim2m-audio: add virtual driver for audio memory to memory
>
> .../media/mediactl/media-types.rst | 11 +
> .../userspace-api/media/v4l/buffer.rst | 6 +
> .../userspace-api/media/v4l/common.rst | 1 +
> .../media/v4l/dev-audio-mem2mem.rst | 71 +
> .../userspace-api/media/v4l/devices.rst | 1 +
> .../media/v4l/ext-ctrls-audio-m2m.rst | 59 +
> .../userspace-api/media/v4l/pixfmt-audio.rst | 100 ++
> .../userspace-api/media/v4l/pixfmt.rst | 1 +
> .../media/v4l/vidioc-enum-fmt.rst | 2 +
> .../media/v4l/vidioc-g-ext-ctrls.rst | 4 +
> .../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 +
> .../media/v4l/vidioc-querycap.rst | 3 +
> .../media/v4l/vidioc-queryctrl.rst | 11 +-
> .../media/videodev2.h.rst.exceptions | 3 +
> .../media/common/videobuf2/videobuf2-v4l2.c | 4 +
> drivers/media/platform/nxp/Kconfig | 13 +
> drivers/media/platform/nxp/Makefile | 1 +
> drivers/media/platform/nxp/imx-asrc.c | 1256 +++++++++++++++++
> drivers/media/test-drivers/Kconfig | 10 +
> drivers/media/test-drivers/Makefile | 1 +
> drivers/media/test-drivers/vim2m-audio.c | 793 +++++++++++
> drivers/media/test-drivers/vivid/vivid-core.h | 2 +
> .../media/test-drivers/vivid/vivid-ctrls.c | 26 +
> drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 +
> drivers/media/v4l2-core/v4l2-ctrls-api.c | 1 +
> drivers/media/v4l2-core/v4l2-ctrls-core.c | 93 +-
> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 10 +
> drivers/media/v4l2-core/v4l2-dev.c | 21 +
> drivers/media/v4l2-core/v4l2-ioctl.c | 66 +
> drivers/media/v4l2-core/v4l2-mem2mem.c | 13 +-
> include/media/v4l2-ctrls.h | 13 +-
> include/media/v4l2-dev.h | 2 +
> include/media/v4l2-ioctl.h | 34 +
> .../fsl => include/sound}/fsl_asrc_common.h | 60 +
> include/uapi/linux/media.h | 2 +
> include/uapi/linux/v4l2-controls.h | 9 +
> include/uapi/linux/videodev2.h | 50 +-
> sound/soc/fsl/fsl_asrc.c | 144 ++
> sound/soc/fsl/fsl_asrc.h | 4 +-
> sound/soc/fsl/fsl_asrc_dma.c | 2 +-
> sound/soc/fsl/fsl_easrc.c | 233 +++
> sound/soc/fsl/fsl_easrc.h | 6 +-
> 42 files changed, 3128 insertions(+), 27 deletions(-)
> create mode 100644 Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> create mode 100644 Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
> create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> create mode 100644 drivers/media/platform/nxp/imx-asrc.c
> create mode 100644 drivers/media/test-drivers/vim2m-audio.c
> rename {sound/soc/fsl => include/sound}/fsl_asrc_common.h (60%)
>
next prev parent reply other threads:[~2024-03-18 15:57 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-11 9:53 [PATCH v14 00/16] Add audio support in v4l2 framework Shengjiu Wang
2024-03-11 9:53 ` [PATCH v14 01/16] media: v4l2-ctrls: add support for fraction_bits Shengjiu Wang
2024-03-11 9:53 ` [PATCH v14 02/16] ASoC: fsl_asrc: define functions for memory to memory usage Shengjiu Wang
2024-03-11 9:53 ` [PATCH v14 03/16] ASoC: fsl_easrc: " Shengjiu Wang
2024-03-11 9:53 ` [PATCH v14 04/16] ASoC: fsl_asrc: move fsl_asrc_common.h to include/sound Shengjiu Wang
2024-03-11 9:53 ` [PATCH v14 05/16] ASoC: fsl_asrc: register m2m platform device Shengjiu Wang
2024-03-11 9:53 ` [PATCH v14 06/16] ASoC: fsl_easrc: " Shengjiu Wang
2024-03-11 9:53 ` [PATCH v14 07/16] media: uapi: Add V4L2_CAP_AUDIO_M2M capability flag Shengjiu Wang
2024-03-11 9:53 ` [PATCH v14 08/16] media: v4l2: Add audio capture and output support Shengjiu Wang
2024-03-11 9:53 ` [PATCH v14 09/16] media: uapi: Define audio sample format fourcc type Shengjiu Wang
2024-03-11 9:53 ` [PATCH v14 10/16] media: uapi: Add V4L2_CTRL_CLASS_M2M_AUDIO Shengjiu Wang
2024-03-11 9:53 ` [PATCH v14 11/16] media: uapi: Add audio rate controls support Shengjiu Wang
2024-03-11 9:53 ` [PATCH v14 12/16] media: uapi: Declare interface types for Audio Shengjiu Wang
2024-03-11 9:53 ` [PATCH v14 13/16] media: uapi: Add an entity type for audio resampler Shengjiu Wang
2024-03-11 9:53 ` [PATCH v14 14/16] media: vivid: add fixed point test controls Shengjiu Wang
2024-03-11 9:54 ` [PATCH v14 15/16] media: imx-asrc: Add memory to memory driver Shengjiu Wang
2024-03-11 9:54 ` [PATCH v14 16/16] media: vim2m-audio: add virtual driver for audio memory to memory Shengjiu Wang
2024-03-18 15:56 ` Hans Verkuil [this message]
2024-03-19 2:24 ` [PATCH v14 00/16] Add audio support in v4l2 framework Shengjiu Wang
2024-03-19 3:14 ` Shengjiu Wang
2024-03-19 7:21 ` Hans Verkuil
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=399da377-c2b6-4226-9108-8a1e776b6f3b@xs4all.nl \
--to=hverkuil@xs4all.nl \
--cc=Xiubo.Lee@gmail.com \
--cc=alsa-devel@alsa-project.org \
--cc=broonie@kernel.org \
--cc=festevam@gmail.com \
--cc=lgirdwood@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=m.szyprowski@samsung.com \
--cc=mchehab@kernel.org \
--cc=nicoleotsuka@gmail.com \
--cc=perex@perex.cz \
--cc=sakari.ailus@iki.fi \
--cc=shengjiu.wang@gmail.com \
--cc=shengjiu.wang@nxp.com \
--cc=tfiga@chromium.org \
--cc=tiwai@suse.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).