From: Jens Axboe <axboe@kernel.dk> To: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: hannes@cmpxchg.org, clm@fb.com, jack@suse.cz Subject: [PATCH 0/7 v2] More graceful flusher thread memory reclaim wakeup Date: Wed, 20 Sep 2017 09:32:55 -0600 [thread overview] Message-ID: <1505921582-26709-1-git-send-email-axboe@kernel.dk> (raw) We've had some issues with writeback in presence of memory reclaim at Facebook, and this patch set attempts to fix it up. The real functional change is the last patch in the series, the first 5 are prep and cleanup patches. The basic idea is that we have callers that call wakeup_flusher_threads() with nr_pages == 0. This means 'writeback everything'. For memory reclaim situations, we can end up queuing a TON of these kinds of writeback units. This can cause softlockups and further memory issues, since we allocate huge amounts of struct wb_writeback_work to handle this writeback. Handle this situation more gracefully. Changes since v1: - Rename WB_zero_pages to WB_start_all (Amir). - Remove a test_bit() for a condition where we always expect the bit to be set. - Remove 'nr_pages' from the wakeup flusher threads helpers, since everybody now passes in zero. Enables further cleanups in later patches too (Jan). - Fix a case where I forgot to clear WB_start_all if 'work' allocation failed. - Get rid of cond_resched() in the wb_do_writeback() loop. -- Jens Axboe
WARNING: multiple messages have this Message-ID (diff)
From: Jens Axboe <axboe@kernel.dk> To: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Cc: hannes@cmpxchg.org, clm@fb.com, jack@suse.cz Subject: [PATCH 0/7 v2] More graceful flusher thread memory reclaim wakeup Date: Wed, 20 Sep 2017 09:32:55 -0600 [thread overview] Message-ID: <1505921582-26709-1-git-send-email-axboe@kernel.dk> (raw) We've had some issues with writeback in presence of memory reclaim at Facebook, and this patch set attempts to fix it up. The real functional change is the last patch in the series, the first 5 are prep and cleanup patches. The basic idea is that we have callers that call wakeup_flusher_threads() with nr_pages == 0. This means 'writeback everything'. For memory reclaim situations, we can end up queuing a TON of these kinds of writeback units. This can cause softlockups and further memory issues, since we allocate huge amounts of struct wb_writeback_work to handle this writeback. Handle this situation more gracefully. Changes since v1: - Rename WB_zero_pages to WB_start_all (Amir). - Remove a test_bit() for a condition where we always expect the bit to be set. - Remove 'nr_pages' from the wakeup flusher threads helpers, since everybody now passes in zero. Enables further cleanups in later patches too (Jan). - Fix a case where I forgot to clear WB_start_all if 'work' allocation failed. - Get rid of cond_resched() in the wb_do_writeback() loop. -- Jens Axboe -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next reply other threads:[~2017-09-20 15:33 UTC|newest] Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-09-20 15:32 Jens Axboe [this message] 2017-09-20 15:32 ` [PATCH 0/7 v2] More graceful flusher thread memory reclaim wakeup Jens Axboe 2017-09-20 15:32 ` [PATCH 1/7] buffer: cleanup free_more_memory() flusher wakeup Jens Axboe 2017-09-20 15:32 ` Jens Axboe 2017-09-21 14:55 ` Christoph Hellwig 2017-09-21 14:55 ` Christoph Hellwig 2017-09-20 15:32 ` [PATCH 2/7] fs: kill 'nr_pages' argument from wakeup_flusher_threads() Jens Axboe 2017-09-20 15:32 ` Jens Axboe 2017-09-21 14:55 ` Christoph Hellwig 2017-09-21 14:55 ` Christoph Hellwig 2017-09-22 13:12 ` Jan Kara 2017-09-22 13:12 ` Jan Kara 2017-09-20 15:32 ` [PATCH 3/7] fs-writeback: provide a wakeup_flusher_threads_bdi() Jens Axboe 2017-09-20 15:32 ` Jens Axboe 2017-09-21 14:56 ` Christoph Hellwig 2017-09-21 14:56 ` Christoph Hellwig 2017-09-20 15:32 ` [PATCH 4/7] page-writeback: pass in '0' for nr_pages writeback in laptop mode Jens Axboe 2017-09-20 15:32 ` Jens Axboe 2017-09-21 14:59 ` Christoph Hellwig 2017-09-21 14:59 ` Christoph Hellwig 2017-09-25 14:57 ` Jens Axboe 2017-09-25 14:57 ` Jens Axboe 2017-09-21 15:00 ` Christoph Hellwig 2017-09-21 15:00 ` Christoph Hellwig 2017-09-22 13:14 ` Jan Kara 2017-09-22 13:14 ` Jan Kara 2017-09-20 15:33 ` [PATCH 5/7] fs-writeback: make wb_start_writeback() static Jens Axboe 2017-09-20 15:33 ` Jens Axboe 2017-09-21 15:01 ` Christoph Hellwig 2017-09-21 15:01 ` Christoph Hellwig 2017-09-20 15:33 ` [PATCH 6/7] fs-writeback: move nr_pages == 0 logic to one location Jens Axboe 2017-09-20 15:33 ` Jens Axboe 2017-09-21 15:02 ` Christoph Hellwig 2017-09-21 15:02 ` Christoph Hellwig 2017-09-22 13:17 ` Jan Kara 2017-09-22 13:17 ` Jan Kara 2017-09-20 15:33 ` [PATCH 7/7] fs-writeback: only allow one inflight and pending full flush Jens Axboe 2017-09-20 15:33 ` Jens Axboe 2017-09-21 15:05 ` Christoph Hellwig 2017-09-21 15:05 ` Christoph Hellwig 2017-09-21 15:36 ` Jens Axboe 2017-09-21 15:36 ` Jens Axboe 2017-09-21 16:00 ` Jens Axboe 2017-09-21 16:00 ` Jens Axboe 2017-09-21 17:33 ` Christoph Hellwig 2017-09-21 17:33 ` Christoph Hellwig 2017-09-25 9:35 ` Jan Kara 2017-09-25 9:35 ` Jan Kara 2017-09-25 14:48 ` Jens Axboe 2017-09-25 14:48 ` Jens Axboe 2017-09-28 18:09 ` Jens Axboe 2017-09-28 18:09 ` Jens Axboe 2017-09-29 23:20 ` Jens Axboe 2017-09-29 23:20 ` Jens Axboe 2017-10-02 14:53 ` Jan Kara 2017-10-02 14:53 ` Jan Kara
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1505921582-26709-1-git-send-email-axboe@kernel.dk \ --to=axboe@kernel.dk \ --cc=clm@fb.com \ --cc=hannes@cmpxchg.org \ --cc=jack@suse.cz \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.