linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jens Axboe <axboe@kernel.dk>
To: linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org
Cc: viro@zeniv.linux.org.uk
Subject: [PATCHSET 0/3] io_uring: support for linked SQEs
Date: Fri, 17 May 2019 15:41:28 -0600	[thread overview]
Message-ID: <20190517214131.5925-1-axboe@kernel.dk> (raw)

We currently support a barrier operation, which ensures that previous
commands have finished before they are executed. This patchset
introduces support for linked commands. Linked commands form a
dependency between commands, but not for the execution pipeline in
general. Example:

[[ ReadA, WriteA], [ReadB, WriteB], [ReadC, WriteC]]

This is a weak ascii attempt at showing a series of reads and writes,
where each write depends on the previous read. WriteA will not be
started before ReadA is complete. Think of a copy like operation, where
ReadA is "read X bytes from file Y at offset Z" and WriteB is then
"write X bytes to file A at offset B", with the two sharing the same
iovec. While WriteA depends on the completion of ReadA, there are no
dependencies between ReadA and ReadB, and they may execute in parallel.

In terms of user API, ReadA will have IOSQE_IO_LINK set. When that is
set, the next command depends on it. If the next one also has
IOSQE_IO_LINK set, then the dependency continues. If it doesn't, then
the chain stops with the next command.

I wrote a trivial cp(2) implementation using linked reads and writes,
you can find that here:

http://git.kernel.dk/cgit/liburing/tree/examples/link-cp.c

in the liburing git repo. There's also a test program in there
exercising various types of chains.

I think this is a very powerful concept, and something that could even
be extended to be BPF driven at completion time if dependencies need
further linkage outside of what can be statically provided initially.
A potential series could be [Open file, Read, Close file] where the
fd needs to be carried forward, for example.

Patches are against current master from Linus, and are also in my
io_uring-next branch.

-- 
Jens Axboe



             reply	other threads:[~2019-05-17 21:41 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-05-17 21:41 Jens Axboe [this message]
2019-05-17 21:41 ` [PATCH 1/3] uio: make import_iovec()/compat_import_iovec() return bytes on success Jens Axboe
2019-05-17 21:41 ` [PATCH 2/3] io_uring: punt short reads to async context Jens Axboe
2019-05-17 21:41 ` [PATCH 3/3] io_uring: add support for sqe links Jens Axboe
2019-05-22 16:34 ` [PATCHSET 0/3] io_uring: support for linked SQEs Christoph Hellwig
2019-05-22 16:49   ` Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190517214131.5925-1-axboe@kernel.dk \
    --to=axboe@kernel.dk \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=viro@zeniv.linux.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).