Confusion regarding the use of OP_TIMEOUT

Confusion regarding the use of OP_TIMEOUT

[Drew DeVault]

Hi folks! I'm trying to use IO_TIMEOUT to insert a pause in the middle
of my SQ. I set the off (desired number of events to wait for) to zero,
which according to the docs just makes it behave like a timer.

Essentially, I want the following:

[operations...]
OP_TIMEOUT
[operations...]

To be well-ordered, so that the second batch executes after the first.
To accomplish this, I've tried to submit the first operation of the
second batch with IO_DRAIN, which causes the CQE to be delayed, but
ultimately it fails with EINTR instead of just waiting to execute.

I understand that the primary motivator for OP_TIMEOUT is to provide a
timeout functionality for other CQEs. Is my use-case not accomodated by
io_uring?

Re: Confusion regarding the use of OP_TIMEOUT

[Pavel Begunkov]

On 5/19/21 8:51 PM, Drew DeVault wrote:
> Hi folks! I'm trying to use IO_TIMEOUT to insert a pause in the middle
> of my SQ. I set the off (desired number of events to wait for) to zero,
> which according to the docs just makes it behave like a timer.

Right

> 
> Essentially, I want the following:
> 
> [operations...]
> OP_TIMEOUT
> [operations...]
> 
> To be well-ordered, so that the second batch executes after the first.
> To accomplish this, I've tried to submit the first operation of the
> second batch with IO_DRAIN, which causes the CQE to be delayed, but

...causes request submission (i.e. execution) to be delayed to be
exact, not CQE. But anyway sounds workable to me. (if timeout works
well) the second should not be submitted earlier than
submission_time + timeout. Or if timeout is marked DRAIN as well
would be batch1_completion_time + timeout.

> ultimately it fails with EINTR instead of just waiting to execute.

Does some request fails and you find such a CQE (which request?)?
Or a syscall? submission or waiting?

> I understand that the primary motivator for OP_TIMEOUT is to provide a
> timeout functionality for other CQEs. Is my use-case not accomodated by
> io_uring?

-- 
Pavel Begunkov

Re: Confusion regarding the use of OP_TIMEOUT

[Drew DeVault]

On Thu May 20, 2021 at 6:11 AM EDT, Pavel Begunkov wrote:
> > Essentially, I want the following:
> > 
> > [operations...]
> > OP_TIMEOUT
> > [operations...]
> > 
> > To be well-ordered, so that the second batch executes after the first.
> > To accomplish this, I've tried to submit the first operation of the
> > second batch with IO_DRAIN, which causes the CQE to be delayed, but
>
> ...causes request submission (i.e. execution) to be delayed to be
> exact, not CQE.

Er, right. But that's what I want, anyway.

> > ultimately it fails with EINTR instead of just waiting to execute.
>
> Does some request fails and you find such a CQE (which request?)?
> Or a syscall? submission or waiting?

So I have a test program which does the following:

1. Prep a write("hello world"), then a timeout, and a write("hello
   world"), in the SQ, in that order.
2. Submit all three
3. Process 3 SQEs, waiting if necessary, and display the result.

This prints "hello world", then waits 2 seconds (the timeout duration),
and then the final CQE shows EINTR as the result (and "hello world" is
not printed).

Here's an x86_64 binary which reproduces this:

https://l.sr.ht/ZQBh.test

Here's the code, though it's written in an as-of-yet unrelated
programming langauge, so not much help beyond illustrative purposes:

use errors;
use fmt;
use linux::io_uring::{flags};
use linux::io_uring;
use rt;
use strings;
use time;

export fn main() void = {
	let params = io_uring::params { ... };
	let ring = match (io_uring::setup(32, &params)) {
		ring: io_uring::io_uring => ring,
		err: io_uring::error => fmt::fatal(io_uring::strerror(err)),
	};
	defer io_uring::finish(&ring);

	let buf = strings::toutf8("Hello world!\n");
	let sqe = io_uring::must_get_sqe(&ring);
	io_uring::write(sqe, 1, buf: *[*]u8, len(buf));

	let ts = rt::timespec { ... };
	time::duration_to_timespec(time::SECOND * 2, &ts);
	let sqe = io_uring::must_get_sqe(&ring);
	io_uring::timeout(sqe, &ts, 0, 0);

	let sqe = io_uring::must_get_sqe(&ring);
	io_uring::write(sqe, 1, buf: *[*]u8, len(buf), flags::IO_DRAIN);

	io_uring::submit(&ring)!;

	for (let i = 0z; i < 3; i += 1) {
		let cqe = match (io_uring::wait(&ring)) {
			err: io_uring::error => fmt::fatal("Error: {}",
				io_uring::strerror(err)),
			cqe: *io_uring::cqe => cqe,
		};
		defer io_uring::cqe_seen(&ring, cqe);

		let result = match (io_uring::result(cqe)) {
			err: io_uring::error => match (err) {
				errors::timeout => continue,
				* => fmt::fatal("Error: {}",
					io_uring::strerror(err)),
			},
			r: int => r,
		};
		fmt::errorfln("result: {} ({})", result, cqe.res)!;
	};
};

Re: Confusion regarding the use of OP_TIMEOUT

[Alex O'Brien]

On 5/20/21 5:51 AM, Drew DeVault wrote:
> Hi folks! I'm trying to use IO_TIMEOUT to insert a pause in the middle
> of my SQ. I set the off (desired number of events to wait for) to zero,
> which according to the docs just makes it behave like a timer.
> 
> Essentially, I want the following:
> 
> [operations...]
> OP_TIMEOUT
> [operations...]
> 
> To be well-ordered, so that the second batch executes after the first.
> To accomplish this, I've tried to submit the first operation of the
> second batch with IO_DRAIN, which causes the CQE to be delayed, but
> ultimately it fails with EINTR instead of just waiting to execute.
> 
> I understand that the primary motivator for OP_TIMEOUT is to provide a
> timeout functionality for other CQEs. Is my use-case not accomodated by
> io_uring?
> 

Have you tried setting `IO_DRAIN` on the timeout operation itself?

-- 
- Alex O'Brien
<alex@emobrien.com>

Re: Confusion regarding the use of OP_TIMEOUT

[Alex O'Brien]

On 5/20/21 10:19 AM, Drew DeVault wrote:
> On Wed May 19, 2021 at 8:18 PM EDT, Alex O'Brien wrote:
>> On 5/20/21 5:51 AM, Drew DeVault wrote:
>>> Hi folks! I'm trying to use IO_TIMEOUT to insert a pause in the middle
>>> of my SQ. I set the off (desired number of events to wait for) to zero,
>>> which according to the docs just makes it behave like a timer.
>>>
>>> Essentially, I want the following:
>>>
>>> [operations...]
>>> OP_TIMEOUT
>>> [operations...]
>>>
>>> To be well-ordered, so that the second batch executes after the first.
>>> To accomplish this, I've tried to submit the first operation of the
>>> second batch with IO_DRAIN, which causes the CQE to be delayed, but
>>> ultimately it fails with EINTR instead of just waiting to execute.
>>>
>>> I understand that the primary motivator for OP_TIMEOUT is to provide a
>>> timeout functionality for other CQEs. Is my use-case not accomodated by
>>> io_uring?
>>
>> Have you tried setting `IO_DRAIN` on the timeout operation itself?
> 
> Tried it just now. Does not appear to change the results.
> 

Hm. Sorry, that was my one concrete idea. (I do think that's more what
you want, since it should guarantee that the timeout only starts after
the first batch completes). Maybe there's some weirdness due to the fact
that OP_TIMEOUT actually returns -ETIME on expiration, rather than
"succeeding", though I don't actually know how IO_DRAIN behaves in the
presence of preceding failures.

-- 
- Alex O'Brien
<alex@emobrien.com>