SQPOLL behaviour with openat

SQPOLL behaviour with openat

[wdauchy]

Hello,

I am trying to understand SQPOLL behaviour using liburing. I modified the
test in liburing (see below). The test is failing when we use `openat` 
with SQPOLL:

cqe res -9
test_io failed 0/0/1/0/0

Is `openat` supported with SQPOLL? If not I would expect -EINVAL as a
return value, but maybe I'm missing something.
note: I also tested without io_uring_register_files call.

-- 
William

---
 test/read-write.c | 47 ++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 44 insertions(+), 3 deletions(-)

diff --git a/test/read-write.c b/test/read-write.c
index d150f44..05baf0f 100644
--- a/test/read-write.c
+++ b/test/read-write.c
@@ -58,7 +58,7 @@ static int test_io(const char *file, int write, int buffered, int sqthread,
 	struct io_uring_cqe *cqe;
 	struct io_uring ring;
 	int open_flags, ring_flags;
-	int i, fd, ret;
+	int i, dirfd, fd, use_fd, ret;
 
 #ifdef VERBOSE
 	fprintf(stdout, "%s: start %d/%d/%d/%d/%d: ", __FUNCTION__, write,
@@ -79,7 +79,7 @@ static int test_io(const char *file, int write, int buffered, int sqthread,
 	if (!buffered)
 		open_flags |= O_DIRECT;
 
-	fd = open(file, open_flags);
+	dirfd = open(".", O_RDONLY);
 	if (fd < 0) {
 		perror("file open");
 		goto err;
@@ -95,6 +95,46 @@ static int test_io(const char *file, int write, int buffered, int sqthread,
 		goto err;
 	}
 
+	use_fd = dirfd;
+	if (sqthread) {
+		ret = io_uring_register_files(&ring, &dirfd, 1);
+		if (ret) {
+			fprintf(stderr, "file reg failed: %d\n", ret);
+			goto err;
+		}
+		use_fd = 0;
+	}
+	sqe = io_uring_get_sqe(&ring);
+	if (!sqe) {
+		fprintf(stderr, "sqe get failed\n");
+		goto err;
+	}
+	io_uring_prep_openat(sqe, use_fd, file, open_flags, 0);
+	ret = io_uring_submit(&ring);
+	if (ret != 1) {
+		fprintf(stderr, "submit got %d\n", ret);
+		goto err;
+	}
+	ret = io_uring_wait_cqe(&ring, &cqe);
+	if (ret) {
+		fprintf(stderr, "wait_cqe=%d\n", ret);
+		goto err;
+	}
+	if (cqe->res < 0) {
+		fprintf(stderr, "cqe res %d\n", cqe->res);
+		goto err;
+	}
+	fd = cqe->res;
+	io_uring_cqe_seen(&ring, cqe);
+
+	if (sqthread) {
+		ret = io_uring_unregister_files(&ring);
+		if (ret) {
+			fprintf(stderr, "file unreg failed: %d\n", ret);
+			goto err;
+		}
+	}
+
 	if (fixed) {
 		ret = io_uring_register_buffers(&ring, vecs, BUFFERS);
 		if (ret) {
@@ -121,7 +161,7 @@ static int test_io(const char *file, int write, int buffered, int sqthread,
 		offset = BS * (rand() % BUFFERS);
 		if (write) {
 			int do_fixed = fixed;
-			int use_fd = fd;
+			use_fd = fd;
 
 			if (sqthread)
 				use_fd = 0;
@@ -193,6 +233,7 @@ static int test_io(const char *file, int write, int buffered, int sqthread,
 
 	io_uring_queue_exit(&ring);
 	close(fd);
+	close(dirfd);
 #ifdef VERBOSE
 	printf("PASS\n");
 #endif
-- 
2.24.1

Re: SQPOLL behaviour with openat

[Jens Axboe]

On 1/4/20 9:22 AM, wdauchy@gmail.com wrote:
> Hello,
> 
> I am trying to understand SQPOLL behaviour using liburing. I modified the
> test in liburing (see below). The test is failing when we use `openat` 
> with SQPOLL:
> 
> cqe res -9
> test_io failed 0/0/1/0/0
> 
> Is `openat` supported with SQPOLL? If not I would expect -EINVAL as a
> return value, but maybe I'm missing something.
> note: I also tested without io_uring_register_files call.

sqpoll requires the use of fixed files for any sqe, at least for now.
That's why it returns -EBADF if the request doesn't have fixed files
specified.

So it cannot be used with opcodes that create (or close) file
descriptors either.

-- 
Jens Axboe

Re: SQPOLL behaviour with openat

[William Dauchy]

Hello Jens,

Thank you for the quick answer.

On Sat, Jan 4, 2020 at 5:28 PM Jens Axboe <axboe@kernel.dk> wrote:
> sqpoll requires the use of fixed files for any sqe, at least for now.
> That's why it returns -EBADF if the request doesn't have fixed files
> specified.

 I indeed forgot sqe->flags |= IOSQE_FIXED_FILE; in my modified test.

> So it cannot be used with opcodes that create (or close) file
> descriptors either.

ok, I thought `dirfd` could have been the index of my registered fd
for `openat` call. It was not very clear to me this was not possible
to use fixed files for those opcodes.

Thanks,
-- 
William

Re: SQPOLL behaviour with openat

[Jens Axboe]

On 1/4/20 9:41 AM, William Dauchy wrote:
> Hello Jens,
> 
> Thank you for the quick answer.
> 
> On Sat, Jan 4, 2020 at 5:28 PM Jens Axboe <axboe@kernel.dk> wrote:
>> sqpoll requires the use of fixed files for any sqe, at least for now.
>> That's why it returns -EBADF if the request doesn't have fixed files
>> specified.
> 
>  I indeed forgot sqe->flags |= IOSQE_FIXED_FILE; in my modified test.
> 
>> So it cannot be used with opcodes that create (or close) file
>> descriptors either.
> 
> ok, I thought `dirfd` could have been the index of my registered fd
> for `openat` call. It was not very clear to me this was not possible
> to use fixed files for those opcodes.

I should probably clarify that in the man pages! I'll get that done.
But yes, that it is how it _should_ work for registered files. I haven't
spent too much time on sqpoll lately, but there's really no reason why
it cannot do everything that the regular path can. Just need to ensure
the proper inheritance is done, it's a kernel thread just like the async
workers.

-- 
Jens Axboe