From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.8 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1, USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BB9EC64E8A for ; Mon, 30 Nov 2020 19:46:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 389802063A for ; Mon, 30 Nov 2020 19:46:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="hTSPjl64" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387769AbgK3Tqq (ORCPT ); Mon, 30 Nov 2020 14:46:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40782 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726769AbgK3Tqq (ORCPT ); Mon, 30 Nov 2020 14:46:46 -0500 Received: from mail-oi1-x242.google.com (mail-oi1-x242.google.com [IPv6:2607:f8b0:4864:20::242]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6E6BC0613D4 for ; Mon, 30 Nov 2020 11:46:05 -0800 (PST) Received: by mail-oi1-x242.google.com with SMTP id k26so15521359oiw.0 for ; Mon, 30 Nov 2020 11:46:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=lzfoUfWajb4p4eNOmNvuQscrN072WhSa9bylfU2yzX0=; b=hTSPjl64XJLko7MKmjQjMrFqQD4dLlsqADqMsqR+j3i/O/jTAASbFKHNMbwdnvdsxG UJZzukjaYm4Ayg5ODgCXo7kBIzIPle5y9CI+Kg0QDNPFnkM+gvFH6a3kk3YxqcjE4PK+ 5Ug7f208kwZgD9lrNpKuOhc3h5tIhD2YRYHrxrNcd5iGfogG4HJ3wUxs7dn6SybwpDDI QoCDuivGuqPcKB0Rpts8le3YCMv1xr3gwRxTqgomXagAehDHvGJaMA2BgjzI29OMV144 Wt2U7BFKpldDf9PtgttIyJVQoDmqVwizngX7QNriPx1R7PxMM84LS0ElqDUgaQD5PwV3 wPtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:in-reply-to:message-id :references:user-agent:mime-version; bh=lzfoUfWajb4p4eNOmNvuQscrN072WhSa9bylfU2yzX0=; b=W3Fyd4lbfMMappEmIB+DQmkcOa5ophty9IM76P9/scHFY7wW33h3k82PzlSi1BZ0Z5 sUhji5EiTpYJijyvNqvP2qOzqmO2LPP1OvEcAPgpzFDCuljzIcvQjQ4neauNfx+CTBVH 9+IGaGaApdHqxGFOn0PwUEVwUp7CdAKewvEe4erl7eHSEnbFcCvvAxOprFC7a0tEKlcV DzlHQezZAQHeALiI4GsWcehqRRPDJrzGx53HNbiHRFCUSPhU/XonbYUEaCM3SLm1Aq8c Aj6s9Ue4YMc/OBLSzi/48Fkz3WYBHwU5/xmqQehouYGTCWjP2cCQegkdxDh62qzUUQzD Gb4g== X-Gm-Message-State: AOAM531ong4AyK6X9CZ5S+IWbntFirY6pl6DTLU2/tat/XKp9bWP/nlu BJweNxmKIg+TL7I6Bh9zsIuuyQ== X-Google-Smtp-Source: ABdhPJzFDs2BTB3DjlhZABlkie1tpUYCKaVrsycFta9dfeVORIayucoUUj+UR4lDvvzwar+7y7I3iQ== X-Received: by 2002:a05:6808:7cd:: with SMTP id f13mr432496oij.38.1606765564711; Mon, 30 Nov 2020 11:46:04 -0800 (PST) Received: from eggly.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id i4sm9994210oos.31.2020.11.30.11.46.03 (version=TLS1 cipher=ECDHE-ECDSA-AES128-SHA bits=128/128); Mon, 30 Nov 2020 11:46:04 -0800 (PST) Date: Mon, 30 Nov 2020 11:45:46 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Matthew Wilcox cc: Hugh Dickins , Andrew Morton , Jan Kara , William Kucharski , Linux-FSDevel , linux-mm , Christoph Hellwig , Johannes Weiner , Yang Shi , dchinner@redhat.com, linux-kernel Subject: Re: [PATCH v4 00/16] Overhaul multi-page lookups for THP In-Reply-To: <20201126200703.GW4327@casper.infradead.org> Message-ID: References: <20201117153947.GL29991@casper.infradead.org> <20201117191513.GV29991@casper.infradead.org> <20201117234302.GC29991@casper.infradead.org> <20201125023234.GH4327@casper.infradead.org> <20201125150859.25adad8ff64db312681184bd@linux-foundation.org> <20201126121546.GN4327@casper.infradead.org> <20201126200703.GW4327@casper.infradead.org> User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 26 Nov 2020, Matthew Wilcox wrote: > On Thu, Nov 26, 2020 at 11:24:59AM -0800, Hugh Dickins wrote: > > > But right now it's the right fix that's important: ack to yours below. > > > > I've not yet worked out the other issues I saw: will report when I have. > > Rebooted this laptop, pretty sure it missed freeing a shmem swap entry, > > not yet reproduced, will study the change there later, but the non-swap > > hang in generic/476 (later seen also in generic/112) more important. It's been a struggle, but I do now have a version of shmem_undo_range() that works reliably, no known bugs, with no changes to your last version outside of shmem_undo_range(). But my testing so far has been with the initial optimization loop (of trylocks in find_lock_entries()) "#if 0"ed out, to give the final loop a harder time. Now I'll bring back that initial loop (maybe cleaning up some start/end variables) and retest - hoping not to regress as I do so. I'll send it late today: I expect I'll just send the body of shmem_undo_range() itself, rather than a diff, since it's too confusing to know what to diff against. Or, maybe you now have your own improved version, and will want me to look at yours rather than sending mine. Andrew, if you're planning another mmotm soon, please remove/comment mm-truncateshmem-handle-truncates-that-split-thps.patch and any of its "fixes" as to-be-updated: all versions to date have been too buggy to keep, and a new version will require its own review, as you noted. I think that means you also have to remove mm-filemap-return-only-head-pages-from-find_get_entries.patch which I think is blameless, but may depend on it logically. > > The good news is that I've sorted out the SCRATCH_DEV issue with > running xfstests. The bad news is that (even on an unmodified kernel), > generic/027 takes 19 hours to run. On xfs, it's 4 minutes. Any idea > why it's so slow on tmpfs? I sent a tarball of four xfstests patches on Thursday, they're valid: but generic/075 and generic/112, very useful for this testing, suffer from an fsx build bug which might or might not affect you: so I'll now reply to that mail with latest tarball. Hugh From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.8 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1, USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51AE4C63777 for ; Mon, 30 Nov 2020 19:46:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B5ACA2076E for ; Mon, 30 Nov 2020 19:46:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="hTSPjl64" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B5ACA2076E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A27AB6B0036; Mon, 30 Nov 2020 14:46:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9DB938D0001; Mon, 30 Nov 2020 14:46:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 879756B0068; Mon, 30 Nov 2020 14:46:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0162.hostedemail.com [216.40.44.162]) by kanga.kvack.org (Postfix) with ESMTP id 5ED636B0036 for ; Mon, 30 Nov 2020 14:46:09 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 816403626 for ; Mon, 30 Nov 2020 19:46:06 +0000 (UTC) X-FDA: 77542115532.24.glass84_160d004273a4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id 59DA11A4A0 for ; Mon, 30 Nov 2020 19:46:06 +0000 (UTC) X-HE-Tag: glass84_160d004273a4 X-Filterd-Recvd-Size: 6067 Received: from mail-oi1-f195.google.com (mail-oi1-f195.google.com [209.85.167.195]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Mon, 30 Nov 2020 19:46:05 +0000 (UTC) Received: by mail-oi1-f195.google.com with SMTP id f11so15491215oij.6 for ; Mon, 30 Nov 2020 11:46:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=lzfoUfWajb4p4eNOmNvuQscrN072WhSa9bylfU2yzX0=; b=hTSPjl64XJLko7MKmjQjMrFqQD4dLlsqADqMsqR+j3i/O/jTAASbFKHNMbwdnvdsxG UJZzukjaYm4Ayg5ODgCXo7kBIzIPle5y9CI+Kg0QDNPFnkM+gvFH6a3kk3YxqcjE4PK+ 5Ug7f208kwZgD9lrNpKuOhc3h5tIhD2YRYHrxrNcd5iGfogG4HJ3wUxs7dn6SybwpDDI QoCDuivGuqPcKB0Rpts8le3YCMv1xr3gwRxTqgomXagAehDHvGJaMA2BgjzI29OMV144 Wt2U7BFKpldDf9PtgttIyJVQoDmqVwizngX7QNriPx1R7PxMM84LS0ElqDUgaQD5PwV3 wPtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:in-reply-to:message-id :references:user-agent:mime-version; bh=lzfoUfWajb4p4eNOmNvuQscrN072WhSa9bylfU2yzX0=; b=ISZteLcwelCvKMZRsKORo3xW9e3pv+Mdkdiu7nuXzZC+SRZQY2TWOZlbx9lhgGwmTX OrfEt0TUN8yoZpFUcOA5AC8sELSinQ4RGm8LojtYgmDeZ+s4yEOjA4hEpjjq+SojcOzF +xSh+itQffSA02gpIRxxiHxr9H8np7CL7xZ4k/l13lSzx2LnzpYN8heM2giW8Bs0vu30 4ubz1/CfsK+a6i0nbsYvjck818YuRDjwPK5qyJiCVNpZmCJ0gB8qTxgAkbj8vnQdp47X 7NP1ny1OG6zQ5KjWihzA+J7Tn8GlY3KruhfA0h9gGRyiFebHFb6co4Iq4+JIqF2Sq06U jPdA== X-Gm-Message-State: AOAM5339FvoqeZWa8WlV8W8tFjwlQVO85jcb3K1QSlLIZ1+iyt1N7R43 Qkru1ksdcYqJUVrvZtgdKjEuhw== X-Google-Smtp-Source: ABdhPJzFDs2BTB3DjlhZABlkie1tpUYCKaVrsycFta9dfeVORIayucoUUj+UR4lDvvzwar+7y7I3iQ== X-Received: by 2002:a05:6808:7cd:: with SMTP id f13mr432496oij.38.1606765564711; Mon, 30 Nov 2020 11:46:04 -0800 (PST) Received: from eggly.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id i4sm9994210oos.31.2020.11.30.11.46.03 (version=TLS1 cipher=ECDHE-ECDSA-AES128-SHA bits=128/128); Mon, 30 Nov 2020 11:46:04 -0800 (PST) Date: Mon, 30 Nov 2020 11:45:46 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Matthew Wilcox cc: Hugh Dickins , Andrew Morton , Jan Kara , William Kucharski , Linux-FSDevel , linux-mm , Christoph Hellwig , Johannes Weiner , Yang Shi , dchinner@redhat.com, linux-kernel Subject: Re: [PATCH v4 00/16] Overhaul multi-page lookups for THP In-Reply-To: <20201126200703.GW4327@casper.infradead.org> Message-ID: References: <20201117153947.GL29991@casper.infradead.org> <20201117191513.GV29991@casper.infradead.org> <20201117234302.GC29991@casper.infradead.org> <20201125023234.GH4327@casper.infradead.org> <20201125150859.25adad8ff64db312681184bd@linux-foundation.org> <20201126121546.GN4327@casper.infradead.org> <20201126200703.GW4327@casper.infradead.org> User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, 26 Nov 2020, Matthew Wilcox wrote: > On Thu, Nov 26, 2020 at 11:24:59AM -0800, Hugh Dickins wrote: > > > But right now it's the right fix that's important: ack to yours below. > > > > I've not yet worked out the other issues I saw: will report when I have. > > Rebooted this laptop, pretty sure it missed freeing a shmem swap entry, > > not yet reproduced, will study the change there later, but the non-swap > > hang in generic/476 (later seen also in generic/112) more important. It's been a struggle, but I do now have a version of shmem_undo_range() that works reliably, no known bugs, with no changes to your last version outside of shmem_undo_range(). But my testing so far has been with the initial optimization loop (of trylocks in find_lock_entries()) "#if 0"ed out, to give the final loop a harder time. Now I'll bring back that initial loop (maybe cleaning up some start/end variables) and retest - hoping not to regress as I do so. I'll send it late today: I expect I'll just send the body of shmem_undo_range() itself, rather than a diff, since it's too confusing to know what to diff against. Or, maybe you now have your own improved version, and will want me to look at yours rather than sending mine. Andrew, if you're planning another mmotm soon, please remove/comment mm-truncateshmem-handle-truncates-that-split-thps.patch and any of its "fixes" as to-be-updated: all versions to date have been too buggy to keep, and a new version will require its own review, as you noted. I think that means you also have to remove mm-filemap-return-only-head-pages-from-find_get_entries.patch which I think is blameless, but may depend on it logically. > > The good news is that I've sorted out the SCRATCH_DEV issue with > running xfstests. The bad news is that (even on an unmodified kernel), > generic/027 takes 19 hours to run. On xfs, it's 4 minutes. Any idea > why it's so slow on tmpfs? I sent a tarball of four xfstests patches on Thursday, they're valid: but generic/075 and generic/112, very useful for this testing, suffer from an fsx build bug which might or might not affect you: so I'll now reply to that mail with latest tarball. Hugh