All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jens Axboe <axboe@kernel.dk>
To: Andres Freund <andres@anarazel.de>
Cc: io-uring@vger.kernel.org, linux-fsdevel@vger.kernel.org,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	akpm@linux-foundation.org
Subject: Re: [PATCHSET v5 0/12] Add support for async buffered reads
Date: Fri, 5 Jun 2020 08:42:28 -0600	[thread overview]
Message-ID: <6eeff14f-befc-a5cc-08da-cb77f811fbdf@kernel.dk> (raw)
In-Reply-To: <e3072371-1d6b-8ae5-d946-d83e60427cb0@kernel.dk>

On 6/3/20 7:04 PM, Jens Axboe wrote:
> On 6/3/20 6:59 PM, Andres Freund wrote:
>> Hi,
>>
>> I was trying to benchmark the benefits of this for the io_uring using
>> postgres I am working on. The initial results where quite promising
>> (reducing cpu usage significantly, to lower than non-uring sync io). But
>> unfortunately trying another workload triggered both panics and before
>> that seemingly returned wrong data.
>>
>> I first saw that problem with b360d424ce02, which was
>> linux-block/async-buffered.6 at the time. After hitting the issue, I
>> updated to the current linux-block/async-buffered.6, but the problem
>> persists.
>>
>> The workload that triggers the bug within a few seconds is postgres
>> doing a parallel sequential scan of a large table (and aggregating the
>> data, but that shouldn't matter). In the triggering case that boils down
>> to 9 processes sequentially reading a number of 1GB files (we chunk
>> tables internally into smaller files). Each process will read a 512kB
>> chunk of the file on its own, and then claim the next 512kB from a
>> shared memory location. Most of the IO will be READV requests, reading
>> 16 * 8kB into postgres' buffer pool (which may or may not be neighboring
>> 8kB pages).
> 
> I'll try and reproduce this, any chance you have a test case that can
> be run so I don't have to write one from scratch? The more detailed
> instructions the better.

Can you try with async-buffered.7? I've rebased it on a new mechanism,
and doing something like what you describe above I haven't been able
to trigger anything bad. I'd try your test case specifically, so do let
know if it's something I can run.

-- 
Jens Axboe


  parent reply	other threads:[~2020-06-05 14:42 UTC|newest]

Thread overview: 81+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-26 19:51 [PATCHSET v5 0/12] Add support for async buffered reads Jens Axboe
2020-05-26 19:51 ` [PATCH 01/12] block: read-ahead submission should imply no-wait as well Jens Axboe
2020-05-26 19:51 ` [PATCH 02/12] mm: allow read-ahead with IOCB_NOWAIT set Jens Axboe
2020-05-26 20:45   ` Johannes Weiner
2020-05-26 19:51 ` [PATCH 03/12] mm: abstract out wake_page_match() from wake_page_function() Jens Axboe
2020-05-26 21:02   ` Johannes Weiner
2020-06-01 14:16   ` Matthew Wilcox
2020-05-26 19:51 ` [PATCH 04/12] mm: add support for async page locking Jens Axboe
2020-05-26 21:59   ` Johannes Weiner
2020-05-26 22:01     ` Jens Axboe
2020-05-27 16:02       ` Johannes Weiner
2020-06-01 14:26   ` Matthew Wilcox
2020-06-01 17:15     ` Jens Axboe
2020-05-26 19:51 ` [PATCH 05/12] mm: support async buffered reads in generic_file_buffered_read() Jens Axboe
2020-05-26 22:00   ` Johannes Weiner
2020-05-26 19:51 ` [PATCH 06/12] fs: add FMODE_BUF_RASYNC Jens Axboe
2020-05-26 19:51 ` [PATCH 07/12] ext4: flag as supporting buffered async reads Jens Axboe
2020-05-26 19:51 ` [PATCH 08/12] block: flag block devices as supporting IOCB_WAITQ Jens Axboe
2020-05-26 19:51 ` [PATCH 09/12] xfs: flag files as supporting buffered async reads Jens Axboe
2020-05-28 17:53   ` Darrick J. Wong
2020-05-28 19:23     ` Jens Axboe
2020-05-26 19:51 ` [PATCH 10/12] btrfs: " Jens Axboe
2020-05-26 19:57   ` Chris Mason
2020-05-26 19:51 ` [PATCH 11/12] mm: add kiocb_wait_page_queue_init() helper Jens Axboe
2020-05-26 22:01   ` Johannes Weiner
2020-05-26 19:51 ` [PATCH 12/12] io_uring: support true async buffered reads, if file provides it Jens Axboe
2020-05-28 17:02 ` [PATCHSET v5 0/12] Add support for async buffered reads Sedat Dilek
2020-05-28 17:02   ` Sedat Dilek
2020-05-28 17:06   ` Jens Axboe
2020-05-28 17:12     ` Sedat Dilek
2020-05-28 17:12       ` Sedat Dilek
2020-05-28 17:14       ` Jens Axboe
2020-05-28 18:20         ` Sedat Dilek
2020-05-28 18:20           ` Sedat Dilek
2020-05-29 10:02     ` Sedat Dilek
2020-05-29 10:02       ` Sedat Dilek
2020-05-29 11:22       ` Sedat Dilek
2020-05-29 11:22         ` Sedat Dilek
2020-05-30 13:36         ` Sedat Dilek
2020-05-30 13:36           ` Sedat Dilek
2020-05-30 18:57           ` Sedat Dilek
2020-05-30 18:57             ` Sedat Dilek
2020-05-31  1:57             ` Jens Axboe
2020-05-31  7:04               ` Sedat Dilek
2020-05-31  7:04                 ` Sedat Dilek
2020-05-31  7:12                 ` Sedat Dilek
2020-05-31  7:12                   ` Sedat Dilek
2020-06-01 13:35                   ` Sedat Dilek
2020-06-01 13:35                     ` Sedat Dilek
2020-06-01 14:04                     ` Jens Axboe
2020-06-01 14:13                       ` Sedat Dilek
2020-06-01 14:13                         ` Sedat Dilek
2020-06-01 14:14                         ` Jens Axboe
2020-06-01 14:35                           ` Jens Axboe
2020-06-01 14:43                             ` Sedat Dilek
2020-06-01 14:43                               ` Sedat Dilek
2020-06-01 14:46                               ` Jens Axboe
2020-06-01 14:51                                 ` Sedat Dilek
2020-06-01 14:51                                   ` Sedat Dilek
2020-06-01 20:18                                 ` Sedat Dilek
2020-06-01 20:18                                   ` Sedat Dilek
2020-06-04  0:59 ` Andres Freund
2020-06-04  1:04   ` Jens Axboe
2020-06-04  1:30     ` Andres Freund
2020-06-05 19:56       ` Andres Freund
2020-06-05 14:42     ` Jens Axboe [this message]
2020-06-05 20:20       ` Andres Freund
2020-06-05 20:21         ` Jens Axboe
2020-06-05 20:36         ` Andres Freund
2020-06-05 20:53           ` Jens Axboe
2020-06-05 21:13             ` Jens Axboe
2020-06-05 21:21               ` Jens Axboe
2020-06-05 22:30                 ` Andres Freund
2020-06-05 22:36                   ` Andres Freund
2020-06-05 22:49                     ` Jens Axboe
2020-06-05 22:54                       ` Andres Freund
2020-06-05 22:56                         ` Jens Axboe
2020-06-05 23:02                           ` Andres Freund
2020-06-06  0:33                 ` Sedat Dilek
2020-06-06  0:33                   ` Sedat Dilek
2020-06-06 16:04                   ` Jens Axboe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=6eeff14f-befc-a5cc-08da-cb77f811fbdf@kernel.dk \
    --to=axboe@kernel.dk \
    --cc=akpm@linux-foundation.org \
    --cc=andres@anarazel.de \
    --cc=io-uring@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.