From: Paul Cercueil <paul@crapouillou.net>
To: Daniel Vetter <daniel@ffwll.ch>
Cc: "Jonathan Cameron" <jic23@kernel.org>,
"Michael Hennerich" <Michael.Hennerich@analog.com>,
"Jonathan Corbet" <corbet@lwn.net>,
linux-iio@vger.kernel.org, linux-doc@vger.kernel.org,
linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org,
linaro-mm-sig@lists.linaro.org,
"Alexandru Ardelean" <ardeleanalex@gmail.com>,
"Christian König" <christian.koenig@amd.com>
Subject: Re: [PATCH v2 12/12] Documentation: iio: Document high-speed DMABUF based API
Date: Tue, 29 Mar 2022 10:47:23 +0100 [thread overview]
Message-ID: <Z63I9R.MKYUKBH4V8L41@crapouillou.net> (raw)
In-Reply-To: <YkLJU7Pp98CPIHfY@phenom.ffwll.local>
Hi Daniel,
Le mar., mars 29 2022 at 10:54:43 +0200, Daniel Vetter
<daniel@ffwll.ch> a écrit :
> On Mon, Feb 07, 2022 at 01:01:40PM +0000, Paul Cercueil wrote:
>> Document the new DMABUF based API.
>>
>> v2: - Explicitly state that the new interface is optional and is
>> not implemented by all drivers.
>> - The IOCTLs can now only be called on the buffer FD returned by
>> IIO_BUFFER_GET_FD_IOCTL.
>> - Move the page up a bit in the index since it is core stuff
>> and not
>> driver-specific.
>>
>> Signed-off-by: Paul Cercueil <paul@crapouillou.net>
>> ---
>> Documentation/driver-api/dma-buf.rst | 2 +
>> Documentation/iio/dmabuf_api.rst | 94
>> ++++++++++++++++++++++++++++
>> Documentation/iio/index.rst | 2 +
>> 3 files changed, 98 insertions(+)
>> create mode 100644 Documentation/iio/dmabuf_api.rst
>>
>> diff --git a/Documentation/driver-api/dma-buf.rst
>> b/Documentation/driver-api/dma-buf.rst
>> index 2cd7db82d9fe..d3c9b58d2706 100644
>> --- a/Documentation/driver-api/dma-buf.rst
>> +++ b/Documentation/driver-api/dma-buf.rst
>> @@ -1,3 +1,5 @@
>> +.. _dma-buf:
>> +
>> Buffer Sharing and Synchronization
>> ==================================
>>
>> diff --git a/Documentation/iio/dmabuf_api.rst
>> b/Documentation/iio/dmabuf_api.rst
>> new file mode 100644
>> index 000000000000..43bb2c1b9fdc
>> --- /dev/null
>> +++ b/Documentation/iio/dmabuf_api.rst
>> @@ -0,0 +1,94 @@
>> +===================================
>> +High-speed DMABUF interface for IIO
>> +===================================
>> +
>> +1. Overview
>> +===========
>> +
>> +The Industrial I/O subsystem supports access to buffers through a
>> file-based
>> +interface, with read() and write() access calls through the IIO
>> device's dev
>> +node.
>> +
>> +It additionally supports a DMABUF based interface, where the
>> userspace
>> +application can allocate and append DMABUF objects to the buffer's
>> queue.
>> +This interface is however optional and is not available in all
>> drivers.
>> +
>> +The advantage of this DMABUF based interface vs. the read()
>> +interface, is that it avoids an extra copy of the data between the
>> +kernel and userspace. This is particularly useful for high-speed
>> +devices which produce several megabytes or even gigabytes of data
>> per
>> +second.
>> +
>> +The data in this DMABUF interface is managed at the granularity of
>> +DMABUF objects. Reducing the granularity from byte level to block
>> level
>> +is done to reduce the userspace-kernelspace synchronization
>> overhead
>> +since performing syscalls for each byte at a few Mbps is just not
>> +feasible.
>> +
>> +This of course leads to a slightly increased latency. For this
>> reason an
>> +application can choose the size of the DMABUFs as well as how many
>> it
>> +allocates. E.g. two DMABUFs would be a traditional double buffering
>> +scheme. But using a higher number might be necessary to avoid
>> +underflow/overflow situations in the presence of scheduling
>> latencies.
>
> So this reads a lot like reinventing io-uring with pre-registered
> O_DIRECT
> memory ranges. Except it's using dma-buf and hand-rolling a lot of
> pieces
> instead of io-uring and O_DIRECT.
I don't see how io_uring would help us. It's an async I/O framework,
does it allow us to access a kernel buffer without copying the data?
Does it allow us to zero-copy the data to a network interface?
> At least if the entire justification for dma-buf support is zero-copy
> support between the driver and userspace it's _really_ not the right
> tool
> for the job. dma-buf is for zero-copy between devices, with cpu access
> from userpace (or kernel fwiw) being very much the exception (and
> often
> flat-out not supported at all).
We want both. Using dma-bufs for the driver/userspace interface is a
convenience as we then have a unique API instead of two distinct ones.
Why should CPU access from userspace be the exception? It works fine
for IIO dma-bufs. You keep warning about this being a terrible design,
but I simply don't see it.
Cheers,
-Paul
>> +
>> +2. User API
>> +===========
>> +
>> +``IIO_BUFFER_DMABUF_ALLOC_IOCTL(struct iio_dmabuf_alloc_req *)``
>> +----------------------------------------------------------------
>> +
>> +Each call will allocate a new DMABUF object. The return value (if
>> not
>> +a negative errno value as error) will be the file descriptor of
>> the new
>> +DMABUF.
>> +
>> +``IIO_BUFFER_DMABUF_ENQUEUE_IOCTL(struct iio_dmabuf *)``
>> +--------------------------------------------------------
>> +
>> +Place the DMABUF object into the queue pending for hardware
>> process.
>> +
>> +These two IOCTLs have to be performed on the IIO buffer's file
>> +descriptor, obtained using the `IIO_BUFFER_GET_FD_IOCTL` ioctl.
>> +
>> +3. Usage
>> +========
>> +
>> +To access the data stored in a block by userspace the block must be
>> +mapped to the process's memory. This is done by calling mmap() on
>> the
>> +DMABUF's file descriptor.
>> +
>> +Before accessing the data through the map, you must use the
>> +DMA_BUF_IOCTL_SYNC(struct dma_buf_sync *) ioctl, with the
>> +DMA_BUF_SYNC_START flag, to make sure that the data is available.
>> +This call may block until the hardware is done with this block.
>> Once
>> +you are done reading or writing the data, you must use this ioctl
>> again
>> +with the DMA_BUF_SYNC_END flag, before enqueueing the DMABUF to the
>> +kernel's queue.
>> +
>> +If you need to know when the hardware is done with a DMABUF, you
>> can
>> +poll its file descriptor for the EPOLLOUT event.
>> +
>> +Finally, to destroy a DMABUF object, simply call close() on its
>> file
>> +descriptor.
>> +
>> +For more information about manipulating DMABUF objects, see:
>> :ref:`dma-buf`.
>> +
>> +A typical workflow for the new interface is:
>> +
>> + for block in blocks:
>> + DMABUF_ALLOC block
>> + mmap block
>> +
>> + enable buffer
>> +
>> + while !done
>> + for block in blocks:
>> + DMABUF_ENQUEUE block
>> +
>> + DMABUF_SYNC_START block
>> + process data
>> + DMABUF_SYNC_END block
>> +
>> + disable buffer
>> +
>> + for block in blocks:
>> + close block
>> diff --git a/Documentation/iio/index.rst
>> b/Documentation/iio/index.rst
>> index 58b7a4ebac51..669deb67ddee 100644
>> --- a/Documentation/iio/index.rst
>> +++ b/Documentation/iio/index.rst
>> @@ -9,4 +9,6 @@ Industrial I/O
>>
>> iio_configfs
>>
>> + dmabuf_api
>> +
>> ep93xx_adc
>> --
>> 2.34.1
>>
>
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch
next prev parent reply other threads:[~2022-03-29 9:47 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-07 12:59 [PATCH v2 00/12] iio: buffer-dma: write() and new DMABUF based API Paul Cercueil
2022-02-07 12:59 ` [PATCH v2 01/12] iio: buffer-dma: Get rid of outgoing queue Paul Cercueil
2022-02-13 18:57 ` Jonathan Cameron
2022-02-13 19:25 ` Paul Cercueil
2022-03-28 17:17 ` Jonathan Cameron
2022-02-07 12:59 ` [PATCH v2 02/12] iio: buffer-dma: Enable buffer write support Paul Cercueil
2022-03-28 17:24 ` Jonathan Cameron
2022-03-28 18:39 ` Paul Cercueil
2022-03-29 7:11 ` Nuno Sá
2022-03-28 20:38 ` Andy Shevchenko
2022-02-07 12:59 ` [PATCH v2 03/12] iio: buffer-dmaengine: Support specifying buffer direction Paul Cercueil
2022-02-07 12:59 ` [PATCH v2 04/12] iio: buffer-dmaengine: Enable write support Paul Cercueil
2022-03-28 17:28 ` Jonathan Cameron
2022-02-07 12:59 ` [PATCH v2 05/12] iio: core: Add new DMABUF interface infrastructure Paul Cercueil
2022-03-28 17:37 ` Jonathan Cameron
2022-03-28 18:44 ` Paul Cercueil
2022-03-29 13:36 ` Jonathan Cameron
2022-03-28 20:46 ` Andy Shevchenko
2022-02-07 12:59 ` [PATCH v2 06/12] iio: buffer-dma: split iio_dma_buffer_fileio_free() function Paul Cercueil
2022-02-07 12:59 ` [PATCH v2 07/12] iio: buffer-dma: Use DMABUFs instead of custom solution Paul Cercueil
2022-03-28 17:54 ` Jonathan Cameron
2022-03-28 17:54 ` Christian König
2022-03-28 19:16 ` Paul Cercueil
2022-03-28 20:58 ` Andy Shevchenko
2022-02-07 12:59 ` [PATCH v2 08/12] iio: buffer-dma: Implement new DMABUF based userspace API Paul Cercueil
2022-02-07 12:59 ` [PATCH v2 09/12] iio: buffer-dmaengine: Support " Paul Cercueil
2022-02-07 12:59 ` [PATCH v2 10/12] iio: core: Add support for cyclic buffers Paul Cercueil
2022-02-07 13:01 ` [PATCH v2 11/12] iio: buffer-dmaengine: " Paul Cercueil
2022-02-07 13:01 ` [PATCH v2 12/12] Documentation: iio: Document high-speed DMABUF based API Paul Cercueil
2022-03-29 8:54 ` Daniel Vetter
2022-03-29 9:47 ` Paul Cercueil [this message]
2022-03-29 14:07 ` Daniel Vetter
2022-03-29 17:34 ` Paul Cercueil
2022-03-30 9:22 ` Daniel Vetter
2022-02-13 18:46 ` [PATCH v2 00/12] iio: buffer-dma: write() and new " Jonathan Cameron
2022-02-15 17:43 ` Paul Cercueil
2022-03-29 8:33 ` Daniel Vetter
2022-03-29 9:11 ` Paul Cercueil
2022-03-29 14:10 ` Daniel Vetter
2022-03-29 17:16 ` Paul Cercueil
2022-03-30 9:19 ` Daniel Vetter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Z63I9R.MKYUKBH4V8L41@crapouillou.net \
--to=paul@crapouillou.net \
--cc=Michael.Hennerich@analog.com \
--cc=ardeleanalex@gmail.com \
--cc=christian.koenig@amd.com \
--cc=corbet@lwn.net \
--cc=daniel@ffwll.ch \
--cc=dri-devel@lists.freedesktop.org \
--cc=jic23@kernel.org \
--cc=linaro-mm-sig@lists.linaro.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-iio@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).