From: David Sterba <firstname.lastname@example.org> To: Dennis Zhou <email@example.com> Cc: David Sterba <firstname.lastname@example.org>, David Sterba <email@example.com>, Chris Mason <firstname.lastname@example.org>, Josef Bacik <email@example.com>, Omar Sandoval <firstname.lastname@example.org>, email@example.com, firstname.lastname@example.org Subject: Re: [PATCH v6 00/22] btrfs: async discard support Date: Thu, 19 Dec 2019 21:34:38 +0100 Message-ID: <20191219203438.GS3929@twin.jikos.cz> (raw) In-Reply-To: <20191218000600.GB2823@dennisz-mbp> On Tue, Dec 17, 2019 at 07:06:00PM -0500, Dennis Zhou wrote: > > Regarding the slow io submission, I tried to increase the iops value, > > default was 10, but 100 and 1000 made no change. Increasing the maximum > > discard request size to 128M worked (when there was such long extent > > ready). I was expecting a burst of like 4 consecutive IOs after a 600MB > > file is deleted. I did not try to tweak bps_limit because there was > > nothing to limit. > > Ah so there's actually a max time between discards set to 10 seconds as > the maximum timeout is calculated over 6 hours. So if we only have 6 > extents, we'd discard 1 per hour(ish given it decays), but this is > clamped to 10 seconds. > > At least on our servers, we seem to discard at a reasonable rate to > prevent performance penalties during a large number of reads and writes > while maintaining reasonable write amplification performance. Also, > metadata blocks aren't tracked, so on freeing of a whole metadata block > group (minus relocation), we'll trickle discards slightly slower than > expected. So after watching the sysfs numbers, my observation is that the overall strategy of the async discard is to wait for larger ranges and discard one range every 10 seconds. This is a slow process, but this makes sense when there are reads or writes going on so the discard IO penalty is marginal. Running full fstrim will flush all the discardable extents so there's a way to reset the discardable queue. What I still don't see as optimal is the single discard request sent per one period. Namely because there's the iops_limit knob. My idea is that each timeout, 'iops_limit' times 'max_discard_size' is called, so the discard batches are large in total. However, this has impact on reads and writes and also on the device itself, I'm not sure if the too frequent discards are not making things worse (as this is a known problem). I'm interested in more strategies that you could have tested in your setups, either bps based or rate limited etc. The current one seems ok for first implementation but we might want to tune it once we get feedback from more users.
next prev parent reply index Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-12-14 0:22 Dennis Zhou 2019-12-14 0:22 ` [PATCH 01/22] bitmap: genericize percpu bitmap region iterators Dennis Zhou 2019-12-14 0:22 ` [PATCH 02/22] btrfs: rename DISCARD opt to DISCARD_SYNC Dennis Zhou 2019-12-14 0:22 ` [PATCH 03/22] btrfs: keep track of which extents have been discarded Dennis Zhou 2019-12-14 0:22 ` [PATCH 04/22] btrfs: keep track of cleanliness of the bitmap Dennis Zhou 2019-12-14 0:22 ` [PATCH 05/22] btrfs: add the beginning of async discard, discard workqueue Dennis Zhou 2019-12-14 0:22 ` [PATCH 06/22] btrfs: handle empty block_group removal Dennis Zhou 2019-12-14 0:22 ` [PATCH 07/22] btrfs: discard one region at a time in async discard Dennis Zhou 2019-12-14 0:22 ` [PATCH 08/22] btrfs: add removal calls for sysfs debug/ Dennis Zhou 2019-12-18 11:45 ` Anand Jain 2019-12-14 0:22 ` [PATCH 09/22] btrfs: make UUID/debug have its own kobject Dennis Zhou 2019-12-18 11:45 ` Anand Jain 2019-12-14 0:22 ` [PATCH 10/22] btrfs: add discard sysfs directory Dennis Zhou 2019-12-18 11:45 ` Anand Jain 2019-12-14 0:22 ` [PATCH 11/22] btrfs: track discardable extents for async discard Dennis Zhou 2019-12-14 0:22 ` [PATCH 12/22] btrfs: keep track of discardable_bytes Dennis Zhou 2019-12-14 0:22 ` [PATCH 13/22] btrfs: calculate discard delay based on number of extents Dennis Zhou 2019-12-30 16:50 ` David Sterba 2020-01-02 16:45 ` Dennis Zhou 2019-12-14 0:22 ` [PATCH 14/22] btrfs: add bps discard rate limit Dennis Zhou 2019-12-30 17:58 ` David Sterba 2020-01-02 16:46 ` Dennis Zhou 2019-12-14 0:22 ` [PATCH 15/22] btrfs: limit max discard size for async discard Dennis Zhou 2019-12-30 18:00 ` David Sterba 2019-12-30 18:08 ` David Sterba 2020-01-02 16:48 ` Dennis Zhou 2019-12-14 0:22 ` [PATCH 16/22] btrfs: make max async discard size tunable Dennis Zhou 2019-12-30 18:05 ` David Sterba 2020-01-02 16:50 ` Dennis Zhou 2019-12-14 0:22 ` [PATCH 17/22] btrfs: have multiple discard lists Dennis Zhou 2019-12-14 0:22 ` [PATCH 18/22] btrfs: only keep track of data extents for async discard Dennis Zhou 2019-12-30 17:39 ` David Sterba 2020-01-02 16:55 ` Dennis Zhou 2019-12-14 0:22 ` [PATCH 19/22] btrfs: keep track of discard reuse stats Dennis Zhou 2019-12-30 17:33 ` David Sterba 2020-01-02 16:57 ` Dennis Zhou 2019-12-14 0:22 ` [PATCH 20/22] btrfs: add async discard header Dennis Zhou 2019-12-14 0:22 ` [PATCH 21/22] btrfs: increase the metadata allowance for the free_space_cache Dennis Zhou 2019-12-14 0:22 ` [PATCH 22/22] btrfs: make smaller extents more likely to go into bitmaps Dennis Zhou 2019-12-17 14:55 ` [PATCH v6 00/22] btrfs: async discard support David Sterba 2019-12-18 0:06 ` Dennis Zhou 2019-12-19 2:03 ` Dennis Zhou 2019-12-19 20:06 ` David Sterba 2019-12-19 21:22 ` Dennis Zhou 2019-12-19 20:34 ` David Sterba [this message] 2019-12-19 21:17 ` Dennis Zhou 2019-12-30 18:13 ` David Sterba 2019-12-30 18:49 ` Dennis Zhou 2020-01-02 13:22 ` David Sterba
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20191219203438.GS3929@twin.jikos.cz \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Linux-BTRFS Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/linux-btrfs/0 linux-btrfs/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 linux-btrfs linux-btrfs/ https://lore.kernel.org/linux-btrfs \ email@example.com public-inbox-index linux-btrfs Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.linux-btrfs AGPL code for this site: git clone https://public-inbox.org/public-inbox.git