From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7AD57C433ED for ; Wed, 12 May 2021 12:34:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EDD946112F for ; Wed, 12 May 2021 12:34:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EDD946112F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4BDCE6B0036; Wed, 12 May 2021 08:34:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 447E06B006E; Wed, 12 May 2021 08:34:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 272E76B0070; Wed, 12 May 2021 08:34:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0206.hostedemail.com [216.40.44.206]) by kanga.kvack.org (Postfix) with ESMTP id 066466B0036 for ; Wed, 12 May 2021 08:34:51 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id B8C7A181AF5FA for ; Wed, 12 May 2021 12:34:50 +0000 (UTC) X-FDA: 78132523140.20.5FA1B91 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf19.hostedemail.com (Postfix) with ESMTP id F1DCC90009EF for ; Wed, 12 May 2021 12:34:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1620822889; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=plE+nL+ct6AZIIPoxz1my01jPeDymgnbPjG0iiRllwM=; b=eLkDWng3XltHm+v/4dek193edElz1kauXcwMAe27S25Q4hdnRPsWQrW/OZv+X8jQyE2zwz KnMUSCE0+W8OMC3PiC35kaOO5+EYRCova8GlVJDAzb2srF9PM4isknrOZRP5WpdLcakjwa cXt7/hyVcaJheyG4DXUGj1OHa4eC9B4= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-326-KruoJMLxNhy0_RNgaMAvhA-1; Wed, 12 May 2021 08:34:48 -0400 X-MC-Unique: KruoJMLxNhy0_RNgaMAvhA-1 Received: by mail-qk1-f200.google.com with SMTP id z12-20020a05620a08ccb02902ea1e4a963dso17193198qkz.13 for ; Wed, 12 May 2021 05:34:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=plE+nL+ct6AZIIPoxz1my01jPeDymgnbPjG0iiRllwM=; b=J2BEVyo6O75viipmXekI5BZUpfXK8x8c/G5K2YULr6zOUQc9vl8SAXiu7fMMzSQyte S4l1khdXULFF/Kcl6+/xYGAXdQoV8/WLO1DS8iDB9ac4IXX5uqDYoz3/9Ryk+lWfjcTD kPgArWFvCTxpRSgEIIZ7ufDwzDydrAf77yyFZXExjWjfvsCHz1Ql0QhDRD8QoDB33hT+ 2TlCzwxssj79KvQMbPvOWyxq1vnIvtOwMuJ+lQyG56Ok9Z9HV/FdDneWsGsbywt5scb2 2ohqFJgBmvlwPzBe2XYLOO+qEYrVfBhew4Cf5IGlrFou/A+oZlyDslb56IV3q0HApSSz tGWA== X-Gm-Message-State: AOAM530LOywNqakkK5aybOJJRQU2mu8hvM/KVMYAVZU76IfXmBAJ5z8D 1eadBW6+KMmvljYgh1/8TMMu4IWRDeWHdxK8xfSmizAfNaXzdmaYTGMMWeN63+bOceYVsuI9srH RLmmEJhYQlyg= X-Received: by 2002:a05:622a:341:: with SMTP id r1mr4583011qtw.307.1620822885101; Wed, 12 May 2021 05:34:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx2tlaCkwmNfDKEriLu1NRRZo0A8kzyXZX8X8GFPBiFplx0v0baUwXe1hl/5JNdu3ems/5eWw== X-Received: by 2002:a05:622a:341:: with SMTP id r1mr4582983qtw.307.1620822884723; Wed, 12 May 2021 05:34:44 -0700 (PDT) Received: from t490s (bras-base-toroon474qw-grc-72-184-145-4-219.dsl.bell.ca. [184.145.4.219]) by smtp.gmail.com with ESMTPSA id o189sm15923330qkd.60.2021.05.12.05.34.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 May 2021 05:34:44 -0700 (PDT) Date: Wed, 12 May 2021 08:34:42 -0400 From: Peter Xu To: Geert Uytterhoeven Cc: Andrea Arcangeli , Linux MM , Linux Kernel Mailing List , Jan Kara , John Hubbard , Linus Torvalds , Michal Hocko , Kirill Tkhai , Kirill Shutemov , Oleg Nesterov , Andrew Morton , Jann Horn , Jason Gunthorpe , Matthew Wilcox , Hugh Dickins Subject: Re: [PATCH v2 3/3] mm: gup: pack has_pinned in MMF_HAS_PINNED Message-ID: References: <20210507150553.208763-1-peterx@redhat.com> <20210507150553.208763-4-peterx@redhat.com> MIME-Version: 1.0 In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=eLkDWng3; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf19.hostedemail.com: domain of peterx@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=peterx@redhat.com X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: F1DCC90009EF X-Stat-Signature: 6m1re77dpnz7ghwc3ozgcwx1gxhp834q Received-SPF: none (redhat.com>: No applicable sender policy available) receiver=imf19; identity=mailfrom; envelope-from=""; helo=us-smtp-delivery-124.mimecast.com; client-ip=170.10.133.124 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1620822856-376806 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, May 12, 2021 at 11:49:05AM +0200, Geert Uytterhoeven wrote: > Hi Peter, Andrea, Hi, Geert, Naresh, (Adding Naresh too since Naresh reported the same issue at the meantime) > > On Fri, May 7, 2021 at 7:26 PM Peter Xu wrote: > > From: Andrea Arcangeli > > > > has_pinned 32bit can be packed in the MMF_HAS_PINNED bit as a noop > > cleanup. > > > > Any atomic_inc/dec to the mm cacheline shared by all threads in > > pin-fast would reintroduce a loss of SMP scalability to pin-fast, so > > there's no future potential usefulness to keep an atomic in the mm for > > this. > > > > set_bit(MMF_HAS_PINNED) will be theoretically a bit slower than > > WRITE_ONCE (atomic_set is equivalent to WRITE_ONCE), but the set_bit > > (just like atomic_set after this commit) has to be still issued only > > once per "mm", so the difference between the two will be lost in the > > noise. > > > > will-it-scale "mmap2" shows no change in performance with enterprise > > config as expected. > > > > will-it-scale "pin_fast" retains the > 4000% SMP scalability > > performance improvement against upstream as expected. > > > > This is a noop as far as overall performance and SMP scalability are > > concerned. > > > > Signed-off-by: Andrea Arcangeli > > [peterx: Fix build for task_mmu.c, introduce mm_set_has_pinned_flag, fix > > comment here and there] > > Signed-off-by: Peter Xu > > Thanks for your patch, which is now in linux-next. > > > diff --git a/mm/gup.c b/mm/gup.c > > index 9933bc5c2eff2..bb130723a6717 100644 > > --- a/mm/gup.c > > +++ b/mm/gup.c > > @@ -1270,6 +1270,17 @@ int fixup_user_fault(struct mm_struct *mm, > > } > > EXPORT_SYMBOL_GPL(fixup_user_fault); > > > > +/* > > + * Set the MMF_HAS_PINNED if not set yet; after set it'll be there for the mm's > > + * lifecycle. Avoid setting the bit unless necessary, or it might cause write > > + * cache bouncing on large SMP machines for concurrent pinned gups. > > + */ > > +static inline void mm_set_has_pinned_flag(unsigned long *mm_flags) > > +{ > > + if (!test_bit(MMF_HAS_PINNED, mm_flags)) > > + set_bit(MMF_HAS_PINNED, mm_flags); > > +} > > + > > /* > > * Please note that this function, unlike __get_user_pages will not > > * return 0 for nr_pages > 0 without FOLL_NOWAIT > > @@ -1292,8 +1303,8 @@ static __always_inline long __get_user_pages_locked(struct mm_struct *mm, > > BUG_ON(*locked != 1); > > } > > > > - if ((flags & FOLL_PIN) && !atomic_read(&mm->has_pinned)) > > - atomic_set(&mm->has_pinned, 1); > > + if (flags & FOLL_PIN) > > + mm_set_has_pinned_flag(&mm->flags); > > > > /* > > * FOLL_PIN and FOLL_GET are mutually exclusive. Traditional behavior > > @@ -2617,8 +2628,8 @@ static int internal_get_user_pages_fast(unsigned long start, > > FOLL_FAST_ONLY))) > > return -EINVAL; > > > > - if ((gup_flags & FOLL_PIN) && !atomic_read(¤t->mm->has_pinned)) > > - atomic_set(¤t->mm->has_pinned, 1); > > + if (gup_flags & FOLL_PIN) > > + mm_set_has_pinned_flag(¤t->mm->flags); > > noreply@ellerman.id.au reports: > > FAILED linux-next/m5272c3_defconfig/m68k-gcc8 Wed May 12, 19:30 > http://kisskb.ellerman.id.au/kisskb/buildresult/14543658/ > Commit: Add linux-next specific files for 20210512 > ec85c95b0c90a17413901b018e8ade7b9eae7cad > Compiler: m68k-linux-gcc (GCC) 8.1.0 / GNU ld (GNU Binutils) 2.30 > > mm/gup.c:2698:3: error: implicit declaration of function > 'mm_set_has_pinned_flag'; did you mean 'set_tsk_thread_flag'? > [-Werror=implicit-function-declaration] > > It's definition is inside the #ifdef CONFIG_MMU section, but the last > user isn't. Indeed that's wrong and I replied to the mm-commit email but not here to fix this up yesterday: https://lore.kernel.org/mm-commits/20210511220029.m6tGcxUIw%25akpm@linux-foundation.org/ I'll remember to reply to the thread next time. Sorry for that! -- Peter Xu