From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0E8CC77B7E for ; Tue, 2 May 2023 08:29:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233707AbjEBI3R (ORCPT ); Tue, 2 May 2023 04:29:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233276AbjEBI3P (ORCPT ); Tue, 2 May 2023 04:29:15 -0400 Received: from outbound-smtp06.blacknight.com (outbound-smtp06.blacknight.com [81.17.249.39]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5F3734230 for ; Tue, 2 May 2023 01:29:12 -0700 (PDT) Received: from mail.blacknight.com (pemlinmail05.blacknight.ie [81.17.254.26]) by outbound-smtp06.blacknight.com (Postfix) with ESMTPS id C2DA6C2B2A for ; Tue, 2 May 2023 09:29:10 +0100 (IST) Received: (qmail 24126 invoked from network); 2 May 2023 08:29:10 -0000 Received: from unknown (HELO techsingularity.net) (mgorman@techsingularity.net@[84.203.21.103]) by 81.17.254.9 with ESMTPSA (AES256-SHA encrypted, authenticated); 2 May 2023 08:29:10 -0000 Date: Tue, 2 May 2023 09:29:04 +0100 From: Mel Gorman To: Douglas Anderson Cc: Andrew Morton , Vlastimil Babka , Ying , Alexander Viro , Christian Brauner , Hillf Danton , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Gao Xiang , Matthew Wilcox , Yu Zhao Subject: Re: [PATCH v3] migrate_pages: Avoid blocking for IO in MIGRATE_SYNC_LIGHT Message-ID: <20230502082904.je47t3g6dnmrcbz7@techsingularity.net> References: <20230428135414.v3.1.Ia86ccac02a303154a0b8bc60567e7a95d34c96d3@changeid> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline In-Reply-To: <20230428135414.v3.1.Ia86ccac02a303154a0b8bc60567e7a95d34c96d3@changeid> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Apr 28, 2023 at 01:54:38PM -0700, Douglas Anderson wrote: > The MIGRATE_SYNC_LIGHT mode is intended to block for things that will > finish quickly but not for things that will take a long time. Exactly > how long is too long is not well defined, but waits of tens of > milliseconds is likely non-ideal. > > When putting a Chromebook under memory pressure (opening over 90 tabs > on a 4GB machine) it was fairly easy to see delays waiting for some > locks in the kcompactd code path of > 100 ms. While the laptop wasn't > amazingly usable in this state, it was still limping along and this > state isn't something artificial. Sometimes we simply end up with a > lot of memory pressure. > > Putting the same Chromebook under memory pressure while it was running > Android apps (though not stressing them) showed a much worse result > (NOTE: this was on a older kernel but the codepaths here are similar). > Android apps on ChromeOS currently run from a 128K-block, > zlib-compressed, loopback-mounted squashfs disk. If we get a page > fault from something backed by the squashfs filesystem we could end up > holding a folio lock while reading enough from disk to decompress 128K > (and then decompressing it using the somewhat slow zlib algorithms). > That reading goes through the ext4 subsystem (because it's a loopback > mount) before eventually ending up in the block subsystem. This extra > jaunt adds extra overhead. Without much work I could see cases where > we ended up blocked on a folio lock for over a second. With more > extreme memory pressure I could see up to 25 seconds. > > We considered adding a timeout in the case of MIGRATE_SYNC_LIGHT for > the two locks that were seen to be slow [1] and that generated much > discussion. After discussion, it was decided that we should avoid > waiting for the two locks during MIGRATE_SYNC_LIGHT if they were being > held for IO. We'll continue with the unbounded wait for the more full > SYNC modes. > > With this change, I couldn't see any slow waits on these locks with my > previous testcases. > > NOTE: The reason I stated digging into this originally isn't because > some benchmark had gone awry, but because we've received in-the-field > crash reports where we have a hung task waiting on the page lock > (which is the equivalent code path on old kernels). While the root > cause of those crashes is likely unrelated and won't be fixed by this > patch, analyzing those crash reports did point out these very long > waits seemed like something good to fix. With this patch we should no > longer hang waiting on these locks, but presumably the system will > still be in a bad shape and hang somewhere else. > > [1] https://lore.kernel.org/r/20230421151135.v2.1.I2b71e11264c5c214bc59744b9e13e4c353bc5714@changeid > > Suggested-by: Matthew Wilcox > Cc: Mel Gorman > Cc: Hillf Danton > Cc: Gao Xiang > Signed-off-by: Douglas Anderson Acked-by: Mel Gorman -- Mel Gorman SUSE Labs