IO-Uring Archive on lore.kernel.org
 help / color / Atom feed
* how is register_(buffer|file) supposed to work?
@ 2020-02-12  4:12 Glauber Costa
  2020-02-12 14:20 ` Jens Axboe
  0 siblings, 1 reply; 3+ messages in thread
From: Glauber Costa @ 2020-02-12  4:12 UTC (permalink / raw)
  To: io-uring, Avi Kivity, Jens Axboe

Hi,

I am trying to experiment with the interface for registering files and buffers.

(almost) Every time I call io_uring_register with those opcodes, my
application hangs.

It's easy to see the reason. I am blocking here:

                mutex_unlock(&ctx->uring_lock);
                ret = wait_for_completion_interruptible(&ctx->completions[0]);
                mutex_lock(&ctx->uring_lock);

Am I right in my understanding that this is waiting for everything
that was submitted to complete? Some things in my ring may never
complete: for instance one may be polling for file descriptors that
may never really become ready.

This sounds a bit too restrictive to me. Is this really the intended
use of the interface?

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: how is register_(buffer|file) supposed to work?
  2020-02-12  4:12 how is register_(buffer|file) supposed to work? Glauber Costa
@ 2020-02-12 14:20 ` Jens Axboe
  2020-02-12 14:23   ` Glauber Costa
  0 siblings, 1 reply; 3+ messages in thread
From: Jens Axboe @ 2020-02-12 14:20 UTC (permalink / raw)
  To: Glauber Costa, io-uring, Avi Kivity

On 2/11/20 9:12 PM, Glauber Costa wrote:
> Hi,
> 
> I am trying to experiment with the interface for registering files and buffers.
> 
> (almost) Every time I call io_uring_register with those opcodes, my
> application hangs.
> 
> It's easy to see the reason. I am blocking here:
> 
>                 mutex_unlock(&ctx->uring_lock);
>                 ret = wait_for_completion_interruptible(&ctx->completions[0]);
>                 mutex_lock(&ctx->uring_lock);
> 
> Am I right in my understanding that this is waiting for everything
> that was submitted to complete? Some things in my ring may never
> complete: for instance one may be polling for file descriptors that
> may never really become ready.
> 
> This sounds a bit too restrictive to me. Is this really the intended
> use of the interface?

For files, this was added in the current merge window:

commit 05f3fb3c5397524feae2e73ee8e150a9090a7da2
Author: Jens Axboe <axboe@kernel.dk>
Date:   Mon Dec 9 11:22:50 2019 -0700

    io_uring: avoid ring quiesce for fixed file set unregister and update

which allows you to call IORING_REGISTER_FILES_UPDATE without having to
quiesce the ring. File sets can be sparse, you can register with an fd
of -1 and then later use FILES_UPDATE (or IORING_OP_FILES_UPDATE) to
replace it with a real entry. You can also replace a real entry with a
new one, or switch it to sparse again.

This helps the file case, but the buffer case is the same as originally,
it is intended to be used at ring setup time, not really during runtime.
As you have found, it needs to quiesce the ring, hence if you have any
requests pending, it'll wait for those. If they are unbounded requests
(like poll, for instance), then it won't really work that well.

That said, there's absolutely no reason why buffer registration can't
mimick what was done for file updates - allow registering a larger
sparse set, and update buffers as you need to. The latter without
needing a quiesce. The above commit basically just needs to be
implemented for buffers as well, which would be pretty trivial.

-- 
Jens Axboe


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: how is register_(buffer|file) supposed to work?
  2020-02-12 14:20 ` Jens Axboe
@ 2020-02-12 14:23   ` Glauber Costa
  0 siblings, 0 replies; 3+ messages in thread
From: Glauber Costa @ 2020-02-12 14:23 UTC (permalink / raw)
  To: Jens Axboe; +Cc: io-uring, Avi Kivity

On Wed, Feb 12, 2020 at 9:20 AM Jens Axboe <axboe@kernel.dk> wrote:
>
> On 2/11/20 9:12 PM, Glauber Costa wrote:
> > Hi,
> >
> > I am trying to experiment with the interface for registering files and buffers.
> >
> > (almost) Every time I call io_uring_register with those opcodes, my
> > application hangs.
> >
> > It's easy to see the reason. I am blocking here:
> >
> >                 mutex_unlock(&ctx->uring_lock);
> >                 ret = wait_for_completion_interruptible(&ctx->completions[0]);
> >                 mutex_lock(&ctx->uring_lock);
> >
> > Am I right in my understanding that this is waiting for everything
> > that was submitted to complete? Some things in my ring may never
> > complete: for instance one may be polling for file descriptors that
> > may never really become ready.
> >
> > This sounds a bit too restrictive to me. Is this really the intended
> > use of the interface?
>
> For files, this was added in the current merge window:
>

Ok, so this is what I was missing:

> commit 05f3fb3c5397524feae2e73ee8e150a9090a7da2
> Author: Jens Axboe <axboe@kernel.dk>
> Date:   Mon Dec 9 11:22:50 2019 -0700
>
>     io_uring: avoid ring quiesce for fixed file set unregister and update
>
> which allows you to call IORING_REGISTER_FILES_UPDATE without having to
> quiesce the ring. File sets can be sparse, you can register with an fd
> of -1 and then later use FILES_UPDATE (or IORING_OP_FILES_UPDATE) to
> replace it with a real entry. You can also replace a real entry with a
> new one, or switch it to sparse again.

^^^ this

I thought I've seen EBADF errors when registering -1, but maybe it
wasn't -1, it was
just really an invalid fd.

For memory I guess I could register early on and draw from a poll.
That's a bit inconvenient
as ideally that poll would grow and shrink dynamically, but it works

I'll give it a try

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, back to index

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-12  4:12 how is register_(buffer|file) supposed to work? Glauber Costa
2020-02-12 14:20 ` Jens Axboe
2020-02-12 14:23   ` Glauber Costa

IO-Uring Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/io-uring/0 io-uring/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 io-uring io-uring/ https://lore.kernel.org/io-uring \
		io-uring@vger.kernel.org
	public-inbox-index io-uring

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.io-uring


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git