linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Nikolay Borisov <nborisov@suse.com>
To: Josef Bacik <josef@toxicpanda.com>,
	linux-btrfs@vger.kernel.org, kernel-team@fb.com
Subject: Re: [PATCH v3 05/12] btrfs: improve preemptive background space flushing
Date: Tue, 13 Oct 2020 14:29:19 +0300	[thread overview]
Message-ID: <2f95c845-5c0f-7574-7715-c425e10709d6@suse.com> (raw)
In-Reply-To: <26c66e1d02a0fee72d79ed92e24d2d2f4620d487.1602249928.git.josef@toxicpanda.com>



On 9.10.20 г. 16:28 ч., Josef Bacik wrote:
> Currently if we ever have to flush space because we do not have enough
> we allocate a ticket and attach it to the space_info, and then
> systematically flush things in the file system that hold space
> reservations until our space is reclaimed.
> 
> However this has a latency cost, we must go to sleep and wait for the
> flushing to make progress before we are woken up and allowed to continue
> doing our work.
> 
> In order to address that we used to kick off the async worker to flush
> space preemptively, so that we could be reclaiming space hopefully
> before any tasks needed to stop and wait for space to reclaim.
> 
> When I introduced the ticketed ENOSPC stuff this broke slightly in the
> fact that we were using tickets to indicate if we were done flushing.
> No tickets, no more flushing.  However this meant that we essentially
> never preemptively flushed.  This caused a write performance regression
> that Nikolay noticed in an unrelated patch that removed the committing
> of the transaction during btrfs_end_transaction.
> 
> The behavior that happened pre that patch was btrfs_end_transaction()
> would see that we were low on space, and it would commit the
> transaction.  This was bad because in this particular case you could end
> up with thousands and thousands of transactions being committed during
> the 5 minute reproducer.  With the patch to remove this behavior you got
> much more sane transaction commits, but we ended up slower because we
> would write for a while, flush, write for a while, flush again.
> 
> To address this we need to reinstate a preemptive flushing mechanism.
> However it is distinctly different from our ticketing flushing in that
> it doesn't have tickets to base it's decisions on.  Instead of bolting
> this logic into our existing flushing work, add another worker to handle
> this preemptive flushing.  Here we will attempt to be slightly
> intelligent about the things that we flushing, attempting to balance
> between whichever pool is taking up the most space.
> 
> Signed-off-by: Josef Bacik <josef@toxicpanda.com>

Reviewed-by: Nikolay Borisov <nborisov@suse.com>

  reply	other threads:[~2020-10-13 11:29 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-09 13:28 [PATCH v3 00/12] Improve preemptive ENOSPC flushing Josef Bacik
2020-10-09 13:28 ` [PATCH v3 01/12] btrfs: make flush_space take a enum btrfs_flush_state instead of int Josef Bacik
2020-10-12 13:49   ` Nikolay Borisov
2021-01-26 18:36   ` David Sterba
2021-01-26 20:32     ` Josef Bacik
2021-01-27 15:27       ` David Sterba
2020-10-09 13:28 ` [PATCH v3 02/12] btrfs: add a trace point for reserve tickets Josef Bacik
2021-01-26 19:41   ` David Sterba
2020-10-09 13:28 ` [PATCH v3 03/12] btrfs: track ordered bytes instead of just dio ordered bytes Josef Bacik
2020-10-12 13:50   ` Nikolay Borisov
2020-10-09 13:28 ` [PATCH v3 04/12] btrfs: introduce a FORCE_COMMIT_TRANS flush operation Josef Bacik
2020-10-12 13:50   ` Nikolay Borisov
2020-10-29 17:03   ` David Sterba
2021-01-26 18:41     ` David Sterba
2020-10-09 13:28 ` [PATCH v3 05/12] btrfs: improve preemptive background space flushing Josef Bacik
2020-10-13 11:29   ` Nikolay Borisov [this message]
2020-10-09 13:28 ` [PATCH v3 06/12] btrfs: rename need_do_async_reclaim Josef Bacik
2021-01-26 18:51   ` David Sterba
2020-10-09 13:28 ` [PATCH v3 07/12] btrfs: check reclaim_size in need_preemptive_reclaim Josef Bacik
2020-10-09 13:28 ` [PATCH v3 08/12] btrfs: rework btrfs_calc_reclaim_metadata_size Josef Bacik
2020-10-09 13:28 ` [PATCH v3 09/12] btrfs: simplify the logic in need_preemptive_flushing Josef Bacik
2020-10-13 12:18   ` Nikolay Borisov
2020-10-09 13:28 ` [PATCH v3 10/12] btrfs: implement space clamping for preemptive flushing Josef Bacik
2020-10-29 17:48   ` David Sterba
2020-10-09 13:28 ` [PATCH v3 11/12] btrfs: adjust the flush trace point to include the source Josef Bacik
2021-01-26 19:13   ` David Sterba
2020-10-09 13:28 ` [PATCH v3 12/12] btrfs: add a trace class for dumping the current ENOSPC state Josef Bacik

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=2f95c845-5c0f-7574-7715-c425e10709d6@suse.com \
    --to=nborisov@suse.com \
    --cc=josef@toxicpanda.com \
    --cc=kernel-team@fb.com \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).