From: Tejun Heo <tj@kernel.org> To: axboe@kernel.dk Cc: linux-kernel@vger.kernel.org, jack@suse.cz, hch@infradead.org, hannes@cmpxchg.org, linux-fsdevel@vger.kernel.org, vgoyal@redhat.com, lizefan@huawei.com, cgroups@vger.kernel.org, linux-mm@kvack.org, mhocko@suse.cz, clm@fb.com, fengguang.wu@intel.com, david@fromorbit.com, gthelen@google.com, Tejun Heo <tj@kernel.org> Subject: [PATCH 43/49] writeback: implement wb_wait_for_single_work() Date: Mon, 6 Apr 2015 15:58:32 -0400 [thread overview] Message-ID: <1428350318-8215-44-git-send-email-tj@kernel.org> (raw) In-Reply-To: <1428350318-8215-1-git-send-email-tj@kernel.org> For cgroup writeback, multiple wb_writeback_work items may need to be issuedto accomplish a single task. The previous patch updated the waiting mechanism such that wb_wait_for_completion() can wait for multiple work items. Issuing mulitple work items involves memory allocation which may fail. As most writeback operations can't fail or blocked on memory allocation, in such cases, we'll fall back to sequential issuing of an on-stack work item, which would need to be waited upon sequentially. This patch implements wb_wait_for_single_work() which waits for a single work item independently from wb_completion waiting so that such fallback mechanism can be used without getting tangled with the usual issuing / completion operation. Signed-off-by: Tejun Heo <tj@kernel.org> Cc: Jens Axboe <axboe@kernel.dk> Cc: Jan Kara <jack@suse.cz> --- fs/fs-writeback.c | 47 +++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 45 insertions(+), 2 deletions(-) diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c index faa9272..2a3cd9c 100644 --- a/fs/fs-writeback.c +++ b/fs/fs-writeback.c @@ -52,6 +52,8 @@ struct wb_writeback_work { unsigned int for_background:1; unsigned int for_sync:1; /* sync(2) WB_SYNC_ALL writeback */ unsigned int auto_free:1; /* free on completion */ + unsigned int single_wait:1; + unsigned int single_done:1; enum wb_reason reason; /* why was writeback initiated? */ struct list_head list; /* pending work list */ @@ -165,8 +167,11 @@ static void wb_queue_work(struct bdi_writeback *wb, trace_writeback_queue(wb->bdi, work); spin_lock_bh(&wb->work_lock); - if (!test_bit(WB_registered, &wb->state)) + if (!test_bit(WB_registered, &wb->state)) { + if (work->single_wait) + work->single_done = 1; goto out_unlock; + } if (work->done) atomic_inc(&work->done->cnt); list_add_tail(&work->list, &wb->work_list); @@ -221,6 +226,32 @@ int inode_congested(struct inode *inode, int cong_bits) EXPORT_SYMBOL_GPL(inode_congested); /** + * wb_wait_for_single_work - wait for completion of a single bdi_writeback_work + * @bdi: bdi the work item was issued to + * @work: work item to wait for + * + * Wait for the completion of @work which was issued to one of @bdi's + * bdi_writeback's. The caller must have set @work->single_wait before + * issuing it. This wait operates independently fo + * wb_wait_for_completion() and also disables automatic freeing of @work. + */ +static void wb_wait_for_single_work(struct backing_dev_info *bdi, + struct wb_writeback_work *work) +{ + if (WARN_ON_ONCE(!work->single_wait)) + return; + + wait_event(bdi->wb_waitq, work->single_done); + + /* + * Paired with smp_wmb() in wb_do_writeback() and ensures that all + * modifications to @work prior to assertion of ->single_done is + * visible to the caller once this function returns. + */ + smp_rmb(); +} + +/** * wb_split_bdi_pages - split nr_pages to write according to bandwidth * @wb: target bdi_writeback to split @nr_pages to * @nr_pages: number of pages to write for the whole bdi @@ -1159,14 +1190,26 @@ static long wb_do_writeback(struct bdi_writeback *wb) set_bit(WB_writeback_running, &wb->state); while ((work = get_next_work_item(wb)) != NULL) { struct wb_completion *done = work->done; + bool need_wake_up = false; trace_writeback_exec(wb->bdi, work); wrote += wb_writeback(wb, work); - if (work->auto_free) + if (work->single_wait) { + WARN_ON_ONCE(work->auto_free); + /* paired w/ rmb in wb_wait_for_single_work() */ + smp_wmb(); + work->single_done = 1; + need_wake_up = true; + } else if (work->auto_free) { kfree(work); + } + if (done && atomic_dec_and_test(&done->cnt)) + need_wake_up = true; + + if (need_wake_up) wake_up_all(&wb->bdi->wb_waitq); } -- 2.1.0
WARNING: multiple messages have this Message-ID (diff)
From: Tejun Heo <tj@kernel.org> To: axboe@kernel.dk Cc: linux-kernel@vger.kernel.org, jack@suse.cz, hch@infradead.org, hannes@cmpxchg.org, linux-fsdevel@vger.kernel.org, vgoyal@redhat.com, lizefan@huawei.com, cgroups@vger.kernel.org, linux-mm@kvack.org, mhocko@suse.cz, clm@fb.com, fengguang.wu@intel.com, david@fromorbit.com, gthelen@google.com, Tejun Heo <tj@kernel.org> Subject: [PATCH 43/49] writeback: implement wb_wait_for_single_work() Date: Mon, 6 Apr 2015 15:58:32 -0400 [thread overview] Message-ID: <1428350318-8215-44-git-send-email-tj@kernel.org> (raw) In-Reply-To: <1428350318-8215-1-git-send-email-tj@kernel.org> For cgroup writeback, multiple wb_writeback_work items may need to be issuedto accomplish a single task. The previous patch updated the waiting mechanism such that wb_wait_for_completion() can wait for multiple work items. Issuing mulitple work items involves memory allocation which may fail. As most writeback operations can't fail or blocked on memory allocation, in such cases, we'll fall back to sequential issuing of an on-stack work item, which would need to be waited upon sequentially. This patch implements wb_wait_for_single_work() which waits for a single work item independently from wb_completion waiting so that such fallback mechanism can be used without getting tangled with the usual issuing / completion operation. Signed-off-by: Tejun Heo <tj@kernel.org> Cc: Jens Axboe <axboe@kernel.dk> Cc: Jan Kara <jack@suse.cz> --- fs/fs-writeback.c | 47 +++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 45 insertions(+), 2 deletions(-) diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c index faa9272..2a3cd9c 100644 --- a/fs/fs-writeback.c +++ b/fs/fs-writeback.c @@ -52,6 +52,8 @@ struct wb_writeback_work { unsigned int for_background:1; unsigned int for_sync:1; /* sync(2) WB_SYNC_ALL writeback */ unsigned int auto_free:1; /* free on completion */ + unsigned int single_wait:1; + unsigned int single_done:1; enum wb_reason reason; /* why was writeback initiated? */ struct list_head list; /* pending work list */ @@ -165,8 +167,11 @@ static void wb_queue_work(struct bdi_writeback *wb, trace_writeback_queue(wb->bdi, work); spin_lock_bh(&wb->work_lock); - if (!test_bit(WB_registered, &wb->state)) + if (!test_bit(WB_registered, &wb->state)) { + if (work->single_wait) + work->single_done = 1; goto out_unlock; + } if (work->done) atomic_inc(&work->done->cnt); list_add_tail(&work->list, &wb->work_list); @@ -221,6 +226,32 @@ int inode_congested(struct inode *inode, int cong_bits) EXPORT_SYMBOL_GPL(inode_congested); /** + * wb_wait_for_single_work - wait for completion of a single bdi_writeback_work + * @bdi: bdi the work item was issued to + * @work: work item to wait for + * + * Wait for the completion of @work which was issued to one of @bdi's + * bdi_writeback's. The caller must have set @work->single_wait before + * issuing it. This wait operates independently fo + * wb_wait_for_completion() and also disables automatic freeing of @work. + */ +static void wb_wait_for_single_work(struct backing_dev_info *bdi, + struct wb_writeback_work *work) +{ + if (WARN_ON_ONCE(!work->single_wait)) + return; + + wait_event(bdi->wb_waitq, work->single_done); + + /* + * Paired with smp_wmb() in wb_do_writeback() and ensures that all + * modifications to @work prior to assertion of ->single_done is + * visible to the caller once this function returns. + */ + smp_rmb(); +} + +/** * wb_split_bdi_pages - split nr_pages to write according to bandwidth * @wb: target bdi_writeback to split @nr_pages to * @nr_pages: number of pages to write for the whole bdi @@ -1159,14 +1190,26 @@ static long wb_do_writeback(struct bdi_writeback *wb) set_bit(WB_writeback_running, &wb->state); while ((work = get_next_work_item(wb)) != NULL) { struct wb_completion *done = work->done; + bool need_wake_up = false; trace_writeback_exec(wb->bdi, work); wrote += wb_writeback(wb, work); - if (work->auto_free) + if (work->single_wait) { + WARN_ON_ONCE(work->auto_free); + /* paired w/ rmb in wb_wait_for_single_work() */ + smp_wmb(); + work->single_done = 1; + need_wake_up = true; + } else if (work->auto_free) { kfree(work); + } + if (done && atomic_dec_and_test(&done->cnt)) + need_wake_up = true; + + if (need_wake_up) wake_up_all(&wb->bdi->wb_waitq); } -- 2.1.0 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2015-04-06 20:03 UTC|newest] Thread overview: 144+ messages / expand[flat|nested] mbox.gz Atom feed top 2015-04-06 19:57 [PATCHSET 1/3 v3 block/for-4.1/core] writeback: cgroup writeback support Tejun Heo 2015-04-06 19:57 ` Tejun Heo 2015-04-06 19:57 ` [PATCH 01/49] memcg: add per cgroup dirty page accounting Tejun Heo 2015-04-06 19:57 ` Tejun Heo 2015-04-06 19:57 ` [PATCH 02/49] blkcg: move block/blk-cgroup.h to include/linux/blk-cgroup.h Tejun Heo 2015-04-06 19:57 ` Tejun Heo 2015-04-06 19:57 ` [PATCH 03/49] update !CONFIG_BLK_CGROUP dummies in include/linux/blk-cgroup.h Tejun Heo 2015-04-06 19:57 ` Tejun Heo 2015-04-06 19:57 ` [PATCH 04/49] blkcg: always create the blkcg_gq for the root blkcg Tejun Heo 2015-04-06 19:57 ` Tejun Heo 2015-04-06 19:57 ` [PATCH 05/49] memcg: add mem_cgroup_root_css Tejun Heo 2015-04-06 19:57 ` Tejun Heo 2015-04-06 19:57 ` [PATCH 06/49] blkcg: add blkcg_root_css Tejun Heo 2015-04-06 19:57 ` Tejun Heo 2015-04-06 19:57 ` [PATCH 07/49] cgroup, block: implement task_get_css() and use it in bio_associate_current() Tejun Heo 2015-04-06 19:57 ` Tejun Heo 2015-04-06 19:57 ` [PATCH 08/49] blkcg: implement task_get_blkcg_css() Tejun Heo 2015-04-06 19:57 ` Tejun Heo 2015-04-06 19:57 ` [PATCH 09/49] blkcg: implement bio_associate_blkcg() Tejun Heo 2015-04-06 19:57 ` Tejun Heo 2015-04-06 19:57 ` [PATCH 10/49] memcg: implement mem_cgroup_css_from_page() Tejun Heo 2015-04-06 19:57 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 11/49] writeback: move backing_dev_info->state into bdi_writeback Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-20 15:32 ` Jan Kara 2015-04-20 15:32 ` Jan Kara 2015-04-06 19:58 ` [PATCH 12/49] writeback: move backing_dev_info->bdi_stat[] " Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-20 15:02 ` Jan Kara 2015-04-20 15:02 ` Jan Kara 2015-04-20 17:56 ` Tejun Heo 2015-04-20 17:56 ` Tejun Heo 2015-04-20 17:56 ` Tejun Heo 2015-04-21 8:51 ` Jan Kara 2015-04-21 8:51 ` Jan Kara 2015-04-21 8:51 ` Jan Kara 2015-04-21 15:02 ` Tejun Heo 2015-04-21 15:02 ` Tejun Heo 2015-04-21 15:05 ` Jan Kara 2015-04-21 15:05 ` Jan Kara 2015-04-21 15:05 ` Jan Kara 2015-04-06 19:58 ` [PATCH 13/49] writeback: move bandwidth related fields from backing_dev_info " Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-20 15:09 ` Jan Kara 2015-04-20 15:09 ` Jan Kara 2015-04-20 18:01 ` Tejun Heo 2015-04-20 18:01 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 14/49] writeback: s/bdi/wb/ in mm/page-writeback.c Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-20 15:18 ` Jan Kara 2015-04-20 15:18 ` Jan Kara 2015-04-06 19:58 ` [PATCH 15/49] writeback: move backing_dev_info->wb_lock and ->worklist into bdi_writeback Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-20 15:32 ` Jan Kara 2015-04-20 15:32 ` Jan Kara 2015-04-20 15:32 ` Jan Kara 2015-04-20 18:17 ` Tejun Heo 2015-04-20 18:17 ` Tejun Heo 2015-04-21 8:59 ` Jan Kara 2015-04-21 8:59 ` Jan Kara 2015-04-21 8:59 ` Jan Kara 2015-04-06 19:58 ` [PATCH 16/49] writeback: reorganize mm/backing-dev.c Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-20 15:34 ` Jan Kara 2015-04-20 15:34 ` Jan Kara 2015-04-20 15:34 ` Jan Kara 2015-04-06 19:58 ` [PATCH 17/49] writeback: separate out include/linux/backing-dev-defs.h Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-20 15:37 ` Jan Kara 2015-04-20 15:37 ` Jan Kara 2015-04-06 19:58 ` [PATCH 18/49] bdi: make inode_to_bdi() inline Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-20 15:40 ` Jan Kara 2015-04-20 15:40 ` Jan Kara 2015-04-20 15:40 ` Jan Kara 2015-04-20 18:21 ` Tejun Heo 2015-04-20 18:21 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 19/49] writeback: add @gfp to wb_init() Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-20 15:41 ` Jan Kara 2015-04-20 15:41 ` Jan Kara 2015-04-06 19:58 ` [PATCH 20/49] bdi: separate out congested state into a separate struct Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-21 14:36 ` Jan Kara 2015-04-21 14:36 ` Jan Kara 2015-04-06 19:58 ` [PATCH 21/49] writeback: add {CONFIG|BDI_CAP|FS}_CGROUP_WRITEBACK Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 22/49] writeback: make backing_dev_info host cgroup-specific bdi_writebacks Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 23/49] writeback, blkcg: associate each blkcg_gq with the corresponding bdi_writeback_congested Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 24/49] writeback: attribute stats to the matching per-cgroup bdi_writeback Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 25/49] writeback: let balance_dirty_pages() work on the matching cgroup bdi_writeback Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 26/49] writeback: make congestion functions per bdi_writeback Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 27/49] writeback, blkcg: restructure blk_{set|clear}_queue_congested() Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 28/49] writeback, blkcg: propagate non-root blkcg congestion state Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 29/49] writeback: implement and use inode_congested() Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 30/49] writeback: implement WB_has_dirty_io wb_state flag Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 31/49] writeback: implement backing_dev_info->tot_write_bandwidth Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 32/49] writeback: make bdi_has_dirty_io() take multiple bdi_writeback's into account Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 33/49] writeback: don't issue wb_writeback_work if clean Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 34/49] writeback: make bdi->min/max_ratio handling cgroup writeback aware Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 35/49] writeback: implement bdi_for_each_wb() Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 36/49] writeback: remove bdi_start_writeback() Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 37/49] writeback: make laptop_mode_timer_fn() handle multiple bdi_writeback's Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 38/49] writeback: make writeback_in_progress() take bdi_writeback instead of backing_dev_info Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 39/49] writeback: make bdi_start_background_writeback() " Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 40/49] writeback: make wakeup_flusher_threads() handle multiple bdi_writeback's Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 41/49] writeback: add wb_writeback_work->auto_free Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 42/49] writeback: implement bdi_wait_for_completion() Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` Tejun Heo [this message] 2015-04-06 19:58 ` [PATCH 43/49] writeback: implement wb_wait_for_single_work() Tejun Heo 2015-04-06 19:58 ` [PATCH 44/49] writeback: restructure try_writeback_inodes_sb[_nr]() Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 45/49] writeback: make writeback initiation functions handle multiple bdi_writeback's Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 46/49] writeback: dirty inodes against their matching cgroup bdi_writeback's Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 47/49] buffer, writeback: make __block_write_full_page() honor cgroup writeback Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 48/49] mpage: make __mpage_writepage() " Tejun Heo 2015-04-06 19:58 ` Tejun Heo 2015-04-06 19:58 ` [PATCH 49/49] ext2: enable cgroup writeback support Tejun Heo 2015-04-06 19:58 ` Tejun Heo
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1428350318-8215-44-git-send-email-tj@kernel.org \ --to=tj@kernel.org \ --cc=axboe@kernel.dk \ --cc=cgroups@vger.kernel.org \ --cc=clm@fb.com \ --cc=david@fromorbit.com \ --cc=fengguang.wu@intel.com \ --cc=gthelen@google.com \ --cc=hannes@cmpxchg.org \ --cc=hch@infradead.org \ --cc=jack@suse.cz \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=lizefan@huawei.com \ --cc=mhocko@suse.cz \ --cc=vgoyal@redhat.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.