From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BBEBC433E1 for ; Sun, 2 Aug 2020 18:21:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7B41720792 for ; Sun, 2 Aug 2020 18:21:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="e/2c4X4c" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726821AbgHBSVG (ORCPT ); Sun, 2 Aug 2020 14:21:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45014 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725910AbgHBSVF (ORCPT ); Sun, 2 Aug 2020 14:21:05 -0400 Received: from mail-io1-xd41.google.com (mail-io1-xd41.google.com [IPv6:2607:f8b0:4864:20::d41]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 62BE9C06174A; Sun, 2 Aug 2020 11:21:05 -0700 (PDT) Received: by mail-io1-xd41.google.com with SMTP id a5so21156451ioa.13; Sun, 02 Aug 2020 11:21:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=eymiZWouRu2pskPk6lLMhIYaDCAUHR+AKe1qVabtwRI=; b=e/2c4X4cZyBwPWEhEMz8pWttU6ykLW2oan+3KUk0tfrPc+kfHc6HmskkZsNHKng6iA +t04TL3yFnkRGH7cOqnTguGl3MWnY+JBhEXXZjP0SVZtMgl6kLvXH0v8JazbBmQKjyF/ WDcP7LE91BWEh197Z7ONxFL16Fb2J9qpjPjW47RxlrRSg9gLcZ5M7MDXsiU4T7YWPQG1 CvxrtHVCgjPDa1zINW8d+VPpc9j2dFxLHRelqS5ZKOfeb8kPmwSKTAWEve+wp83W2/Ur HoftnL5dA2b0O+r0FsFFMLZAWBJrsPuYvykQouLc/N6yWH4CUMNpukTsN1J5ZTf59FK7 Etyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=eymiZWouRu2pskPk6lLMhIYaDCAUHR+AKe1qVabtwRI=; b=XlMXv77D/bkwIuHNozEKGENYHbKnLFjWHo713RPBAxM90+qmlqSLZNx0vXk7rU2+p/ QOQq0rNLSk9q2Sa1q4JumLxdhG2ZXcEP66ipJ5MO3Li5uQemF8lxmx2doPAMBBUF1mPa DUsQS9qNcwTtnc6WDFXg4/VoKblMHzeqPrQAdH90kiebJFSA3bk3YwHDVy35c2WWSKyr ePhBrfV/idEPcs0Plsuhfsuu8XWc0WFa1g60Mf/lLptQiFpc+z/N0xVJywNcCHlbX8bE GyXlbLJ6U70fZqWCnM4MamLKuCmY7Vc9thM+1BUc/Miu/tGeqHZcnONUkvqQ/NBOdQHB QGyw== X-Gm-Message-State: AOAM532T2dgosR9KzGSv4SE8+3ZJ/tKno7pubbXExO+f/1CO5xiyxs29 8aQXI7anrx2iKzlLm567iKIYZu4F71gyy4Wz4HA= X-Google-Smtp-Source: ABdhPJyw2q9TKbWYE+IgaM7eYFyWbUfG4XEhgbk4ZkyN+Syfvuim9IQtjE/hc410Zayewn3yPOXZqrz3BFKFqHm/xRs= X-Received: by 2002:a05:6602:2e83:: with SMTP id m3mr2103300iow.38.1596392464610; Sun, 02 Aug 2020 11:21:04 -0700 (PDT) MIME-Version: 1.0 References: <1595681998-19193-19-git-send-email-alex.shi@linux.alibaba.com> <159622999150.2576729.14455020813024958573.stgit@ahduyck-desk1.jf.intel.com> <0c0a415a-52a0-5c06-b4be-80cbd3cb49c2@linux.alibaba.com> In-Reply-To: <0c0a415a-52a0-5c06-b4be-80cbd3cb49c2@linux.alibaba.com> From: Alexander Duyck Date: Sun, 2 Aug 2020 11:20:53 -0700 Message-ID: Subject: Re: [PATCH RFC] mm: Add function for testing if the current lruvec lock is valid To: Alex Shi Cc: "Duyck, Alexander H" , Andrew Morton , Andrey Ryabinin , cgroups@vger.kernel.org, Daniel Jordan , Johannes Weiner , Hugh Dickins , Joonsoo Kim , Konstantin Khlebnikov , "Kirill A. Shutemov" , LKML , linux-mm , kbuild test robot , Mel Gorman , Wei Yang , Rong Chen , Shakeel Butt , Thomas Gleixner , Tejun Heo , Matthew Wilcox , Yang Shi Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Feel free to fold it into your patches if you want. I think Hugh was the one that had submitted a patch that addressed it, and it looks like you folded that into your v17 set. It was probably what he had identified which was the additional LRU checks needing to be removed from the code. Thanks. - Alex On Fri, Jul 31, 2020 at 4:55 PM Alex Shi wrote= : > > It looks much better than mine. and could replace 'mm/lru: introduce the = relock_page_lruvec function' > with your author signed. :) > > BTW, > it's the rcu_read_lock cause the will-it-scale/page_fault3 regression whi= ch you mentained in another > letter? > > Thanks > Alex > > =E5=9C=A8 2020/8/1 =E4=B8=8A=E5=8D=885:14, alexander.h.duyck@intel.com = =E5=86=99=E9=81=93: > > From: Alexander Duyck > > > > When testing for relock we can avoid the need for RCU locking if we sim= ply > > compare the page pgdat and memcg pointers versus those that the lruvec = is > > holding. By doing this we can avoid the extra pointer walks and accesse= s of > > the memory cgroup. > > > > In addition we can avoid the checks entirely if lruvec is currently NUL= L. > > > > Signed-off-by: Alexander Duyck > > --- > > include/linux/memcontrol.h | 52 +++++++++++++++++++++++++++---------= -------- > > 1 file changed, 32 insertions(+), 20 deletions(-) > > > > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h > > index 6e670f991b42..7a02f00bf3de 100644 > > --- a/include/linux/memcontrol.h > > +++ b/include/linux/memcontrol.h > > @@ -405,6 +405,22 @@ static inline struct lruvec *mem_cgroup_lruvec(str= uct mem_cgroup *memcg, > > > > struct lruvec *mem_cgroup_page_lruvec(struct page *, struct pglist_dat= a *); > > > > +static inline bool lruvec_holds_page_lru_lock(struct page *page, > > + struct lruvec *lruvec) > > +{ > > + pg_data_t *pgdat =3D page_pgdat(page); > > + const struct mem_cgroup *memcg; > > + struct mem_cgroup_per_node *mz; > > + > > + if (mem_cgroup_disabled()) > > + return lruvec =3D=3D &pgdat->__lruvec; > > + > > + mz =3D container_of(lruvec, struct mem_cgroup_per_node, lruvec); > > + memcg =3D page->mem_cgroup ? : root_mem_cgroup; > > + > > + return lruvec->pgdat =3D=3D pgdat && mz->memcg =3D=3D memcg; > > +} > > + > > struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p); > > > > struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm); > > @@ -880,6 +896,14 @@ static inline struct lruvec *mem_cgroup_page_lruve= c(struct page *page, > > return &pgdat->__lruvec; > > } > > > > +static inline bool lruvec_holds_page_lru_lock(struct page *page, > > + struct lruvec *lruvec) > > +{ > > + pg_data_t *pgdat =3D page_pgdat(page); > > + > > + return lruvec =3D=3D &pgdat->__lruvec; > > +} > > + > > static inline struct mem_cgroup *parent_mem_cgroup(struct mem_cgroup *= memcg) > > { > > return NULL; > > @@ -1317,18 +1341,12 @@ static inline void unlock_page_lruvec_irqrestor= e(struct lruvec *lruvec, > > static inline struct lruvec *relock_page_lruvec_irq(struct page *page, > > struct lruvec *locked_lruvec) > > { > > - struct pglist_data *pgdat =3D page_pgdat(page); > > - bool locked; > > + if (locked_lruvec) { > > + if (lruvec_holds_page_lru_lock(page, locked_lruvec)) > > + return locked_lruvec; > > > > - rcu_read_lock(); > > - locked =3D mem_cgroup_page_lruvec(page, pgdat) =3D=3D locked_lruv= ec; > > - rcu_read_unlock(); > > - > > - if (locked) > > - return locked_lruvec; > > - > > - if (locked_lruvec) > > unlock_page_lruvec_irq(locked_lruvec); > > + } > > > > return lock_page_lruvec_irq(page); > > } > > @@ -1337,18 +1355,12 @@ static inline struct lruvec *relock_page_lruvec= _irq(struct page *page, > > static inline struct lruvec *relock_page_lruvec_irqsave(struct page *p= age, > > struct lruvec *locked_lruvec, unsigned long *flags) > > { > > - struct pglist_data *pgdat =3D page_pgdat(page); > > - bool locked; > > - > > - rcu_read_lock(); > > - locked =3D mem_cgroup_page_lruvec(page, pgdat) =3D=3D locked_lruv= ec; > > - rcu_read_unlock(); > > - > > - if (locked) > > - return locked_lruvec; > > + if (locked_lruvec) { > > + if (lruvec_holds_page_lru_lock(page, locked_lruvec)) > > + return locked_lruvec; > > > > - if (locked_lruvec) > > unlock_page_lruvec_irqrestore(locked_lruvec, *flags); > > + } > > > > return lock_page_lruvec_irqsave(page, flags); > > } > > > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72334C433DF for ; Sun, 2 Aug 2020 18:21:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 300FC2065E for ; Sun, 2 Aug 2020 18:21:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="e/2c4X4c" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 300FC2065E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A42D68D00CA; Sun, 2 Aug 2020 14:21:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9CAC18D00AA; Sun, 2 Aug 2020 14:21:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 892888D00CA; Sun, 2 Aug 2020 14:21:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0189.hostedemail.com [216.40.44.189]) by kanga.kvack.org (Postfix) with ESMTP id 70AA18D00AA for ; Sun, 2 Aug 2020 14:21:06 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id E50221EE6 for ; Sun, 2 Aug 2020 18:21:05 +0000 (UTC) X-FDA: 77106445290.03.peace67_160699126f97 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id B5BCB28A4E8 for ; Sun, 2 Aug 2020 18:21:05 +0000 (UTC) X-HE-Tag: peace67_160699126f97 X-Filterd-Recvd-Size: 8575 Received: from mail-io1-f66.google.com (mail-io1-f66.google.com [209.85.166.66]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Sun, 2 Aug 2020 18:21:05 +0000 (UTC) Received: by mail-io1-f66.google.com with SMTP id s189so29167734iod.2 for ; Sun, 02 Aug 2020 11:21:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=eymiZWouRu2pskPk6lLMhIYaDCAUHR+AKe1qVabtwRI=; b=e/2c4X4cZyBwPWEhEMz8pWttU6ykLW2oan+3KUk0tfrPc+kfHc6HmskkZsNHKng6iA +t04TL3yFnkRGH7cOqnTguGl3MWnY+JBhEXXZjP0SVZtMgl6kLvXH0v8JazbBmQKjyF/ WDcP7LE91BWEh197Z7ONxFL16Fb2J9qpjPjW47RxlrRSg9gLcZ5M7MDXsiU4T7YWPQG1 CvxrtHVCgjPDa1zINW8d+VPpc9j2dFxLHRelqS5ZKOfeb8kPmwSKTAWEve+wp83W2/Ur HoftnL5dA2b0O+r0FsFFMLZAWBJrsPuYvykQouLc/N6yWH4CUMNpukTsN1J5ZTf59FK7 Etyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=eymiZWouRu2pskPk6lLMhIYaDCAUHR+AKe1qVabtwRI=; b=KQhhLQZFyz6gjsELo3R1XM3KnVWqfdS1fmzF2Ef30EaXqz9+q0GMpeYcqTPY6vYMGU 0FC/qSt+6S1HZXqAuv57GH/8DM2erK8jKoPmXKQJZYNJAgj8OU8kHYgDb/0ERhqF5cs7 PNUd8qviTyRCGItDyEGDi/tgzzDVnU+KjjdBkBP5dXiR6Jg0Dh+qBlm+yLu+aVVOFIn/ fceztEIndDmVnoNyRwtUyaztQt66ObblCV7ZYc0z4Yo4YJJyw9sEPzKiIA7iniv5uibE u2E7FJy5BOw737OzDwbHAXUDe1WPkcLayxiA+Q4M4PF5tspXYZXbSHEnJqec3vxOD1G2 0CQQ== X-Gm-Message-State: AOAM533LWjyMwVa0OQxBJCqRQjkWwbDtN7Wama4r0OPeQgMlM5b8nejn jECi002v6MTtVcTanMoCQQ4ShSXkJraAOIOGZnE= X-Google-Smtp-Source: ABdhPJyw2q9TKbWYE+IgaM7eYFyWbUfG4XEhgbk4ZkyN+Syfvuim9IQtjE/hc410Zayewn3yPOXZqrz3BFKFqHm/xRs= X-Received: by 2002:a05:6602:2e83:: with SMTP id m3mr2103300iow.38.1596392464610; Sun, 02 Aug 2020 11:21:04 -0700 (PDT) MIME-Version: 1.0 References: <1595681998-19193-19-git-send-email-alex.shi@linux.alibaba.com> <159622999150.2576729.14455020813024958573.stgit@ahduyck-desk1.jf.intel.com> <0c0a415a-52a0-5c06-b4be-80cbd3cb49c2@linux.alibaba.com> In-Reply-To: <0c0a415a-52a0-5c06-b4be-80cbd3cb49c2@linux.alibaba.com> From: Alexander Duyck Date: Sun, 2 Aug 2020 11:20:53 -0700 Message-ID: Subject: Re: [PATCH RFC] mm: Add function for testing if the current lruvec lock is valid To: Alex Shi Cc: "Duyck, Alexander H" , Andrew Morton , Andrey Ryabinin , cgroups@vger.kernel.org, Daniel Jordan , Johannes Weiner , Hugh Dickins , Joonsoo Kim , Konstantin Khlebnikov , "Kirill A. Shutemov" , LKML , linux-mm , kbuild test robot , Mel Gorman , Wei Yang , Rong Chen , Shakeel Butt , Thomas Gleixner , Tejun Heo , Matthew Wilcox , Yang Shi Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: B5BCB28A4E8 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam04 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Feel free to fold it into your patches if you want. I think Hugh was the one that had submitted a patch that addressed it, and it looks like you folded that into your v17 set. It was probably what he had identified which was the additional LRU checks needing to be removed from the code. Thanks. - Alex On Fri, Jul 31, 2020 at 4:55 PM Alex Shi wrote= : > > It looks much better than mine. and could replace 'mm/lru: introduce the = relock_page_lruvec function' > with your author signed. :) > > BTW, > it's the rcu_read_lock cause the will-it-scale/page_fault3 regression whi= ch you mentained in another > letter? > > Thanks > Alex > > =E5=9C=A8 2020/8/1 =E4=B8=8A=E5=8D=885:14, alexander.h.duyck@intel.com = =E5=86=99=E9=81=93: > > From: Alexander Duyck > > > > When testing for relock we can avoid the need for RCU locking if we sim= ply > > compare the page pgdat and memcg pointers versus those that the lruvec = is > > holding. By doing this we can avoid the extra pointer walks and accesse= s of > > the memory cgroup. > > > > In addition we can avoid the checks entirely if lruvec is currently NUL= L. > > > > Signed-off-by: Alexander Duyck > > --- > > include/linux/memcontrol.h | 52 +++++++++++++++++++++++++++---------= -------- > > 1 file changed, 32 insertions(+), 20 deletions(-) > > > > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h > > index 6e670f991b42..7a02f00bf3de 100644 > > --- a/include/linux/memcontrol.h > > +++ b/include/linux/memcontrol.h > > @@ -405,6 +405,22 @@ static inline struct lruvec *mem_cgroup_lruvec(str= uct mem_cgroup *memcg, > > > > struct lruvec *mem_cgroup_page_lruvec(struct page *, struct pglist_dat= a *); > > > > +static inline bool lruvec_holds_page_lru_lock(struct page *page, > > + struct lruvec *lruvec) > > +{ > > + pg_data_t *pgdat =3D page_pgdat(page); > > + const struct mem_cgroup *memcg; > > + struct mem_cgroup_per_node *mz; > > + > > + if (mem_cgroup_disabled()) > > + return lruvec =3D=3D &pgdat->__lruvec; > > + > > + mz =3D container_of(lruvec, struct mem_cgroup_per_node, lruvec); > > + memcg =3D page->mem_cgroup ? : root_mem_cgroup; > > + > > + return lruvec->pgdat =3D=3D pgdat && mz->memcg =3D=3D memcg; > > +} > > + > > struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p); > > > > struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm); > > @@ -880,6 +896,14 @@ static inline struct lruvec *mem_cgroup_page_lruve= c(struct page *page, > > return &pgdat->__lruvec; > > } > > > > +static inline bool lruvec_holds_page_lru_lock(struct page *page, > > + struct lruvec *lruvec) > > +{ > > + pg_data_t *pgdat =3D page_pgdat(page); > > + > > + return lruvec =3D=3D &pgdat->__lruvec; > > +} > > + > > static inline struct mem_cgroup *parent_mem_cgroup(struct mem_cgroup *= memcg) > > { > > return NULL; > > @@ -1317,18 +1341,12 @@ static inline void unlock_page_lruvec_irqrestor= e(struct lruvec *lruvec, > > static inline struct lruvec *relock_page_lruvec_irq(struct page *page, > > struct lruvec *locked_lruvec) > > { > > - struct pglist_data *pgdat =3D page_pgdat(page); > > - bool locked; > > + if (locked_lruvec) { > > + if (lruvec_holds_page_lru_lock(page, locked_lruvec)) > > + return locked_lruvec; > > > > - rcu_read_lock(); > > - locked =3D mem_cgroup_page_lruvec(page, pgdat) =3D=3D locked_lruv= ec; > > - rcu_read_unlock(); > > - > > - if (locked) > > - return locked_lruvec; > > - > > - if (locked_lruvec) > > unlock_page_lruvec_irq(locked_lruvec); > > + } > > > > return lock_page_lruvec_irq(page); > > } > > @@ -1337,18 +1355,12 @@ static inline struct lruvec *relock_page_lruvec= _irq(struct page *page, > > static inline struct lruvec *relock_page_lruvec_irqsave(struct page *p= age, > > struct lruvec *locked_lruvec, unsigned long *flags) > > { > > - struct pglist_data *pgdat =3D page_pgdat(page); > > - bool locked; > > - > > - rcu_read_lock(); > > - locked =3D mem_cgroup_page_lruvec(page, pgdat) =3D=3D locked_lruv= ec; > > - rcu_read_unlock(); > > - > > - if (locked) > > - return locked_lruvec; > > + if (locked_lruvec) { > > + if (lruvec_holds_page_lru_lock(page, locked_lruvec)) > > + return locked_lruvec; > > > > - if (locked_lruvec) > > unlock_page_lruvec_irqrestore(locked_lruvec, *flags); > > + } > > > > return lock_page_lruvec_irqsave(page, flags); > > } > > > From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alexander Duyck Subject: Re: [PATCH RFC] mm: Add function for testing if the current lruvec lock is valid Date: Sun, 2 Aug 2020 11:20:53 -0700 Message-ID: References: <1595681998-19193-19-git-send-email-alex.shi@linux.alibaba.com> <159622999150.2576729.14455020813024958573.stgit@ahduyck-desk1.jf.intel.com> <0c0a415a-52a0-5c06-b4be-80cbd3cb49c2@linux.alibaba.com> Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable Return-path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=eymiZWouRu2pskPk6lLMhIYaDCAUHR+AKe1qVabtwRI=; b=e/2c4X4cZyBwPWEhEMz8pWttU6ykLW2oan+3KUk0tfrPc+kfHc6HmskkZsNHKng6iA +t04TL3yFnkRGH7cOqnTguGl3MWnY+JBhEXXZjP0SVZtMgl6kLvXH0v8JazbBmQKjyF/ WDcP7LE91BWEh197Z7ONxFL16Fb2J9qpjPjW47RxlrRSg9gLcZ5M7MDXsiU4T7YWPQG1 CvxrtHVCgjPDa1zINW8d+VPpc9j2dFxLHRelqS5ZKOfeb8kPmwSKTAWEve+wp83W2/Ur HoftnL5dA2b0O+r0FsFFMLZAWBJrsPuYvykQouLc/N6yWH4CUMNpukTsN1J5ZTf59FK7 Etyw== In-Reply-To: <0c0a415a-52a0-5c06-b4be-80cbd3cb49c2@linux.alibaba.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: Content-Type: text/plain; charset="utf-8" To: Alex Shi Cc: "Duyck, Alexander H" , Andrew Morton , Andrey Ryabinin , cgroups@vger.kernel.org, Daniel Jordan , Johannes Weiner , Hugh Dickins , Joonsoo Kim , Konstantin Khlebnikov , "Kirill A. Shutemov" , LKML , linux-mm , kbuild test robot , Mel Gorman , Wei Yang , Rong Chen , Shakeel Butt , Thomas Gleixner , Tejun Heo , Matthew Wilcox Feel free to fold it into your patches if you want. I think Hugh was the one that had submitted a patch that addressed it, and it looks like you folded that into your v17 set. It was probably what he had identified which was the additional LRU checks needing to be removed from the code. Thanks. - Alex On Fri, Jul 31, 2020 at 4:55 PM Alex Shi wrote= : > > It looks much better than mine. and could replace 'mm/lru: introduce the = relock_page_lruvec function' > with your author signed. :) > > BTW, > it's the rcu_read_lock cause the will-it-scale/page_fault3 regression whi= ch you mentained in another > letter? > > Thanks > Alex > > =E5=9C=A8 2020/8/1 =E4=B8=8A=E5=8D=885:14, alexander.h.duyck@intel.com = =E5=86=99=E9=81=93: > > From: Alexander Duyck > > > > When testing for relock we can avoid the need for RCU locking if we sim= ply > > compare the page pgdat and memcg pointers versus those that the lruvec = is > > holding. By doing this we can avoid the extra pointer walks and accesse= s of > > the memory cgroup. > > > > In addition we can avoid the checks entirely if lruvec is currently NUL= L. > > > > Signed-off-by: Alexander Duyck > > --- > > include/linux/memcontrol.h | 52 +++++++++++++++++++++++++++---------= -------- > > 1 file changed, 32 insertions(+), 20 deletions(-) > > > > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h > > index 6e670f991b42..7a02f00bf3de 100644 > > --- a/include/linux/memcontrol.h > > +++ b/include/linux/memcontrol.h > > @@ -405,6 +405,22 @@ static inline struct lruvec *mem_cgroup_lruvec(str= uct mem_cgroup *memcg, > > > > struct lruvec *mem_cgroup_page_lruvec(struct page *, struct pglist_dat= a *); > > > > +static inline bool lruvec_holds_page_lru_lock(struct page *page, > > + struct lruvec *lruvec) > > +{ > > + pg_data_t *pgdat =3D page_pgdat(page); > > + const struct mem_cgroup *memcg; > > + struct mem_cgroup_per_node *mz; > > + > > + if (mem_cgroup_disabled()) > > + return lruvec =3D=3D &pgdat->__lruvec; > > + > > + mz =3D container_of(lruvec, struct mem_cgroup_per_node, lruvec); > > + memcg =3D page->mem_cgroup ? : root_mem_cgroup; > > + > > + return lruvec->pgdat =3D=3D pgdat && mz->memcg =3D=3D memcg; > > +} > > + > > struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p); > > > > struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm); > > @@ -880,6 +896,14 @@ static inline struct lruvec *mem_cgroup_page_lruve= c(struct page *page, > > return &pgdat->__lruvec; > > } > > > > +static inline bool lruvec_holds_page_lru_lock(struct page *page, > > + struct lruvec *lruvec) > > +{ > > + pg_data_t *pgdat =3D page_pgdat(page); > > + > > + return lruvec =3D=3D &pgdat->__lruvec; > > +} > > + > > static inline struct mem_cgroup *parent_mem_cgroup(struct mem_cgroup *= memcg) > > { > > return NULL; > > @@ -1317,18 +1341,12 @@ static inline void unlock_page_lruvec_irqrestor= e(struct lruvec *lruvec, > > static inline struct lruvec *relock_page_lruvec_irq(struct page *page, > > struct lruvec *locked_lruvec) > > { > > - struct pglist_data *pgdat =3D page_pgdat(page); > > - bool locked; > > + if (locked_lruvec) { > > + if (lruvec_holds_page_lru_lock(page, locked_lruvec)) > > + return locked_lruvec; > > > > - rcu_read_lock(); > > - locked =3D mem_cgroup_page_lruvec(page, pgdat) =3D=3D locked_lruv= ec; > > - rcu_read_unlock(); > > - > > - if (locked) > > - return locked_lruvec; > > - > > - if (locked_lruvec) > > unlock_page_lruvec_irq(locked_lruvec); > > + } > > > > return lock_page_lruvec_irq(page); > > } > > @@ -1337,18 +1355,12 @@ static inline struct lruvec *relock_page_lruvec= _irq(struct page *page, > > static inline struct lruvec *relock_page_lruvec_irqsave(struct page *p= age, > > struct lruvec *locked_lruvec, unsigned long *flags) > > { > > - struct pglist_data *pgdat =3D page_pgdat(page); > > - bool locked; > > - > > - rcu_read_lock(); > > - locked =3D mem_cgroup_page_lruvec(page, pgdat) =3D=3D locked_lruv= ec; > > - rcu_read_unlock(); > > - > > - if (locked) > > - return locked_lruvec; > > + if (locked_lruvec) { > > + if (lruvec_holds_page_lru_lock(page, locked_lruvec)) > > + return locked_lruvec; > > > > - if (locked_lruvec) > > unlock_page_lruvec_irqrestore(locked_lruvec, *flags); > > + } > > > > return lock_page_lruvec_irqsave(page, flags); > > } > > >