From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3985FC11F66 for ; Tue, 13 Jul 2021 10:24:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D0B3460FEB for ; Tue, 13 Jul 2021 10:24:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D0B3460FEB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 350878D0002; Tue, 13 Jul 2021 06:24:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1DFA28D0005; Tue, 13 Jul 2021 06:24:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ED8898D0001; Tue, 13 Jul 2021 06:24:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0199.hostedemail.com [216.40.44.199]) by kanga.kvack.org (Postfix) with ESMTP id B9B9B6B0096 for ; Tue, 13 Jul 2021 06:24:51 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id B50D3825D618 for ; Tue, 13 Jul 2021 10:24:50 +0000 (UTC) X-FDA: 78357181140.22.92806B2 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf19.hostedemail.com (Postfix) with ESMTP id 4AE76B0001A5 for ; Tue, 13 Jul 2021 10:24:50 +0000 (UTC) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id 44D512227C; Tue, 13 Jul 2021 10:24:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1626171889; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jVW+Vnlzf7fUAriGIuI3XgyIRl479/5wmGugT/N2XhM=; b=Pzqz7Zk8AkkhF6MAlzVvKixBKDu56t+WU5SFqs+pBiIU1Snn+9k4j950qOpPh9ryZwzhCw frWhTi7rxg568A9NFnWbzOtezV/VUg6Msxed+++WNXOUONxlYs2D6WGThJXq6SsPW3G+NT fX9FybUXZotIcmzmo2ItEuSMdOvzp6c= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1626171889; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jVW+Vnlzf7fUAriGIuI3XgyIRl479/5wmGugT/N2XhM=; b=mw6kChTXXDsQxCOMGQKwvaEWUtWfCXqwOq55WLpuPTAMl6+wdLJUM08K0zbk7JtdSlgaA0 SgqmWlXA4Gqr/0AQ== Received: from quack2.suse.cz (unknown [10.100.224.230]) by relay2.suse.de (Postfix) with ESMTP id 21AE9A3B8B; Tue, 13 Jul 2021 10:24:49 +0000 (UTC) Received: by quack2.suse.cz (Postfix, from userid 1000) id 6B6631E0BC4; Tue, 13 Jul 2021 12:24:48 +0200 (CEST) From: Jan Kara To: Andrew Morton Cc: , Michael Stapelberg , Wu Fengguang , Jan Kara Subject: [PATCH 5/5] writeback: Use READ_ONCE for unlocked reads of writeback stats Date: Tue, 13 Jul 2021 12:24:44 +0200 Message-Id: <20210713102448.30424-5-jack@suse.cz> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210712165811.13163-1-jack@suse.cz> References: <20210712165811.13163-1-jack@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5429; h=from:subject; bh=rT8+qp5Tj5Y+7uYfw1Niw540WYzNBjuJaUBA5gsxUDk=; b=owEBbQGS/pANAwAIAZydqgc/ZEDZAcsmYgBg7WnrUB21vIYwG0vQRFdJiPJ+NPp2FPKwNd4xFq5H u60XBfyJATMEAAEIAB0WIQSrWdEr1p4yirVVKBycnaoHP2RA2QUCYO1p6wAKCRCcnaoHP2RA2cD6CA Cd+7BJUc0xhbtG2zqLk1IttOc5pkDK206BaXUv20ld8EkQMOvs/kv1+viTNFO7SLMXAuKpwJnkwQof /JFkgXJxEK2h3gCBmqGgn1gKHf3zOzR1GKN/wzwWIC4dGlt4ZKhVltW7uaHqnuDix9ycMFJXTkiOOM bzCtdxurcJdqWXKYDNRsTFPiX5LACi5obnxmnI9xS2aIs7U0Y+lH9moYJHCR8+7qMI8anDc/VyDtJz Sp5zfLkkun2SADhocY6WStYhOGuyJiNX+fY3nT2c2AOFXmR6pqlWWBwVkWb/QFAf+DpSndUqLC28wv JzzyhfXfrGhYs5utztzurxNnjxfxj/ X-Developer-Key: i=jack@suse.cz; a=openpgp; fpr=93C6099A142276A28BBE35D815BC833443038D8C Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=Pzqz7Zk8; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=mw6kChTX; spf=pass (imf19.hostedemail.com: domain of jack@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=jack@suse.cz; dmarc=none X-Rspamd-Server: rspam02 X-Stat-Signature: tg3cha9wmdts9jzqdhzr8o76yeft7emm X-Rspamd-Queue-Id: 4AE76B0001A5 X-HE-Tag: 1626171890-287659 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We do some unlocked reads of writeback statistics like avg_write_bandwidth, dirty_ratelimit, or bw_time_stamp. Generally we are fine with getting somewhat out-of-date values but actually getting different values in various parts of the functions because the compiler decided to reload value from original memory location could confuse calculations. Use READ_ONCE for these unlocked accesses and WRITE_ONCE for the updates to be on the safe side. Signed-off-by: Jan Kara --- mm/page-writeback.c | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/mm/page-writeback.c b/mm/page-writeback.c index ee5e43a7453e..8ffd8ed37a23 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -189,7 +189,7 @@ static struct fprop_local_percpu *wb_memcg_completion= s(struct bdi_writeback *wb) static void wb_min_max_ratio(struct bdi_writeback *wb, unsigned long *minp, unsigned long *maxp) { - unsigned long this_bw =3D wb->avg_write_bandwidth; + unsigned long this_bw =3D READ_ONCE(wb->avg_write_bandwidth); unsigned long tot_bw =3D atomic_long_read(&wb->bdi->tot_write_bandwidth= ); unsigned long long min =3D wb->bdi->min_ratio; unsigned long long max =3D wb->bdi->max_ratio; @@ -898,7 +898,7 @@ static long long pos_ratio_polynom(unsigned long setp= oint, static void wb_position_ratio(struct dirty_throttle_control *dtc) { struct bdi_writeback *wb =3D dtc->wb; - unsigned long write_bw =3D wb->avg_write_bandwidth; + unsigned long write_bw =3D READ_ONCE(wb->avg_write_bandwidth); unsigned long freerun =3D dirty_freerun_ceiling(dtc->thresh, dtc->bg_th= resh); unsigned long limit =3D hard_dirty_limit(dtc_dom(dtc), dtc->thresh); unsigned long wb_thresh =3D dtc->wb_thresh; @@ -1121,7 +1121,7 @@ static void wb_update_write_bandwidth(struct bdi_wr= iteback *wb, &wb->bdi->tot_write_bandwidth) <=3D 0); } wb->write_bandwidth =3D bw; - wb->avg_write_bandwidth =3D avg; + wb->avg_write_bandwidth =3D WRITE_ONCE(avg); } =20 static void update_dirty_limit(struct dirty_throttle_control *dtc) @@ -1330,7 +1330,7 @@ static void wb_update_dirty_ratelimit(struct dirty_= throttle_control *dtc, else dirty_ratelimit -=3D step; =20 - wb->dirty_ratelimit =3D max(dirty_ratelimit, 1UL); + wb->dirty_ratelimit =3D WRITE_ONCE(max(dirty_ratelimit, 1UL)); wb->balanced_dirty_ratelimit =3D balanced_dirty_ratelimit; =20 trace_bdi_dirty_ratelimit(wb, dirty_rate, task_ratelimit); @@ -1342,11 +1342,12 @@ static void __wb_update_bandwidth(struct dirty_th= rottle_control *gdtc, { struct bdi_writeback *wb =3D gdtc->wb; unsigned long now =3D jiffies; - unsigned long elapsed =3D now - wb->bw_time_stamp; + unsigned long elapsed; unsigned long dirtied; unsigned long written; =20 spin_lock(&wb->list_lock); + elapsed =3D now - wb->bw_time_stamp; dirtied =3D percpu_counter_read(&wb->stat[WB_DIRTIED]); written =3D percpu_counter_read(&wb->stat[WB_WRITTEN]); =20 @@ -1367,7 +1368,7 @@ static void __wb_update_bandwidth(struct dirty_thro= ttle_control *gdtc, =20 wb->dirtied_stamp =3D dirtied; wb->written_stamp =3D written; - wb->bw_time_stamp =3D now; + wb->bw_time_stamp =3D WRITE_ONCE(now); spin_unlock(&wb->list_lock); } =20 @@ -1391,7 +1392,7 @@ static void wb_bandwidth_estimate_start(struct bdi_= writeback *wb) spin_lock(&wb->list_lock); wb->dirtied_stamp =3D wb_stat(wb, WB_DIRTIED); wb->written_stamp =3D wb_stat(wb, WB_WRITTEN); - wb->bw_time_stamp =3D now; + wb->bw_time_stamp =3D WRITE_ONCE(now); spin_unlock(&wb->list_lock); } } @@ -1416,7 +1417,7 @@ static unsigned long dirty_poll_interval(unsigned l= ong dirty, static unsigned long wb_max_pause(struct bdi_writeback *wb, unsigned long wb_dirty) { - unsigned long bw =3D wb->avg_write_bandwidth; + unsigned long bw =3D READ_ONCE(wb->avg_write_bandwidth); unsigned long t; =20 /* @@ -1438,8 +1439,8 @@ static long wb_min_pause(struct bdi_writeback *wb, unsigned long dirty_ratelimit, int *nr_dirtied_pause) { - long hi =3D ilog2(wb->avg_write_bandwidth); - long lo =3D ilog2(wb->dirty_ratelimit); + long hi =3D ilog2(READ_ONCE(wb->avg_write_bandwidth)); + long lo =3D ilog2(READ_ONCE(wb->dirty_ratelimit)); long t; /* target pause */ long pause; /* estimated next pause */ int pages; /* target nr_dirtied_pause */ @@ -1719,12 +1720,12 @@ static void balance_dirty_pages(struct bdi_writeb= ack *wb, if (dirty_exceeded && !wb->dirty_exceeded) wb->dirty_exceeded =3D 1; =20 - if (time_is_before_jiffies(wb->bw_time_stamp + + if (time_is_before_jiffies(READ_ONCE(wb->bw_time_stamp) + BANDWIDTH_INTERVAL)) __wb_update_bandwidth(gdtc, mdtc, true); =20 /* throttle according to the chosen dtc */ - dirty_ratelimit =3D wb->dirty_ratelimit; + dirty_ratelimit =3D READ_ONCE(wb->dirty_ratelimit); task_ratelimit =3D ((u64)dirty_ratelimit * sdtc->pos_ratio) >> RATELIMIT_CALC_SHIFT; max_pause =3D wb_max_pause(wb, sdtc->wb_dirty); @@ -2365,7 +2366,8 @@ int do_writepages(struct address_space *mapping, st= ruct writeback_control *wbc) * but if there's constant writeback being submitted, this makes sure * writeback bandwidth is updated once in a while. */ - if (time_is_before_jiffies(wb->bw_time_stamp + BANDWIDTH_INTERVAL)) + if (time_is_before_jiffies(READ_ONCE(wb->bw_time_stamp) + + BANDWIDTH_INTERVAL)) wb_update_bandwidth(wb); return ret; } --=20 2.26.2