Re: Buffered IO async context overhead

[Andres Freund <andres@anarazel.de>]

Hi,

On 2020-02-14 13:13:35 -0700, Jens Axboe wrote:
> On 2/14/20 12:50 PM, Andres Freund wrote:
> > which I think is pretty clear evidence we're hitting fairly significant
> > contention on the queue lock.
> > 
> > 
> > I am hitting this in postgres originally, not fio, but I thought it's
> > easier to reproduce this way.  There's obviously benefit to doing things
> > in the background - but it requires odd logic around deciding when to
> > use io_uring, and when not.
> > 
> > To be clear, none of this happens with DIO, but I don't forsee switching
> > to DIO for all IO by default ever (too high demands on accurate
> > configuration).
> 
> Can you try with this added?
> 
> 
> diff --git a/fs/io_uring.c b/fs/io_uring.c
> index 76cbf474c184..207daf83f209 100644
> --- a/fs/io_uring.c
> +++ b/fs/io_uring.c
> @@ -620,6 +620,7 @@ static const struct io_op_def io_op_defs[] = {
>  		.async_ctx		= 1,
>  		.needs_mm		= 1,
>  		.needs_file		= 1,
> +		.hash_reg_file		= 1,
>  		.unbound_nonreg_file	= 1,
>  	},
>  	[IORING_OP_WRITEV] = {
> @@ -634,6 +635,7 @@ static const struct io_op_def io_op_defs[] = {
>  	},
>  	[IORING_OP_READ_FIXED] = {
>  		.needs_file		= 1,
> +		.hash_reg_file		= 1,
>  		.unbound_nonreg_file	= 1,
>  	},
>  	[IORING_OP_WRITE_FIXED] = {
> @@ -711,11 +713,13 @@ static const struct io_op_def io_op_defs[] = {
>  	[IORING_OP_READ] = {
>  		.needs_mm		= 1,
>  		.needs_file		= 1,
> +		.hash_reg_file		= 1,
>  		.unbound_nonreg_file	= 1,
>  	},
>  	[IORING_OP_WRITE] = {
>  		.needs_mm		= 1,
>  		.needs_file		= 1,
> +		.hash_reg_file		= 1,
>  		.unbound_nonreg_file	= 1,
>  	},
>  	[IORING_OP_FADVISE] = {
> @@ -955,7 +959,7 @@ static inline bool io_prep_async_work(struct io_kiocb *req,
>  	bool do_hashed = false;
>  
>  	if (req->flags & REQ_F_ISREG) {
> -		if (def->hash_reg_file)
> +		if (!(req->kiocb->ki_flags & IOCB_DIRECT) && def->hash_reg_file)
>  			do_hashed = true;
>  	} else {
>  		if (def->unbound_nonreg_file)

I can (will do Sunday, on the road till then). But I'm a bit doubtful
it'll help. This is using WRITEV after all, and I only see a single
worker?

Greetings,

Andres Freund