From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12726C43460 for ; Wed, 12 May 2021 09:49:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CE10261263 for ; Wed, 12 May 2021 09:49:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230145AbhELJu1 (ORCPT ); Wed, 12 May 2021 05:50:27 -0400 Received: from mail-ua1-f48.google.com ([209.85.222.48]:39548 "EHLO mail-ua1-f48.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230019AbhELJuY (ORCPT ); Wed, 12 May 2021 05:50:24 -0400 Received: by mail-ua1-f48.google.com with SMTP id a12so7278878uak.6 for ; Wed, 12 May 2021 02:49:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=HwuV0oIEoZ+EQ5+Mm5jlhwshZkWr3IhzEV/rE9T6cEw=; b=oWdptxaRLrgXNXV+xxVi7fmHQOUYZve/WEnh8xDzvSzYFzC2WNOG+sfkq5qKDBOmA/ jqA2vmGXcrnEriP9LMo15oAlIDN+cZuYZyPLC5S08/Ww+h74nJJ59ngx4hVkKBHnqYpC p4vxtgxYtJ2v7cCJt7d9d1p8In9ICjpcAc7sJTutEB9ugttIHTgqnOwJ1iysR/xCcXRv 5q2oj4IsoGoP2Ffp0dRU16yQZRTmGn/FJPg2JUppsigNS+qBC+kH4TYugLqhNu3vI5Zz PTSTNH2S8CZ/HRQ8CEFWEUksi6U4t9JLi3Pgh4qc9wL7TKWsg/3k5SH89EDb/dJSGrwf 1lfw== X-Gm-Message-State: AOAM533KmRUTV+5kM3PAtecKw14tdDPLCKeToNv9Qzc4D5QpYaA9x6Gx AsdIG+7ld0aRq5ig+Z39iMqtjFzQkMUqPGZGQXQ= X-Google-Smtp-Source: ABdhPJzQGXZD6dgZ85APfNUlHHznrQzLksJl7I1MmQfQ/FfX780s5vYhVBREXXLKbtUN48frqg1GMbdzUXcfPyHZELY= X-Received: by 2002:a9f:3852:: with SMTP id q18mr30989572uad.58.1620812956684; Wed, 12 May 2021 02:49:16 -0700 (PDT) MIME-Version: 1.0 References: <20210507150553.208763-1-peterx@redhat.com> <20210507150553.208763-4-peterx@redhat.com> In-Reply-To: <20210507150553.208763-4-peterx@redhat.com> From: Geert Uytterhoeven Date: Wed, 12 May 2021 11:49:05 +0200 Message-ID: Subject: Re: [PATCH v2 3/3] mm: gup: pack has_pinned in MMF_HAS_PINNED To: Peter Xu , Andrea Arcangeli Cc: Linux MM , Linux Kernel Mailing List , Jan Kara , John Hubbard , Linus Torvalds , Michal Hocko , Kirill Tkhai , Kirill Shutemov , Oleg Nesterov , Andrew Morton , Jann Horn , Jason Gunthorpe , Matthew Wilcox , Hugh Dickins Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Peter, Andrea, On Fri, May 7, 2021 at 7:26 PM Peter Xu wrote: > From: Andrea Arcangeli > > has_pinned 32bit can be packed in the MMF_HAS_PINNED bit as a noop > cleanup. > > Any atomic_inc/dec to the mm cacheline shared by all threads in > pin-fast would reintroduce a loss of SMP scalability to pin-fast, so > there's no future potential usefulness to keep an atomic in the mm for > this. > > set_bit(MMF_HAS_PINNED) will be theoretically a bit slower than > WRITE_ONCE (atomic_set is equivalent to WRITE_ONCE), but the set_bit > (just like atomic_set after this commit) has to be still issued only > once per "mm", so the difference between the two will be lost in the > noise. > > will-it-scale "mmap2" shows no change in performance with enterprise > config as expected. > > will-it-scale "pin_fast" retains the > 4000% SMP scalability > performance improvement against upstream as expected. > > This is a noop as far as overall performance and SMP scalability are > concerned. > > Signed-off-by: Andrea Arcangeli > [peterx: Fix build for task_mmu.c, introduce mm_set_has_pinned_flag, fix > comment here and there] > Signed-off-by: Peter Xu Thanks for your patch, which is now in linux-next. > diff --git a/mm/gup.c b/mm/gup.c > index 9933bc5c2eff2..bb130723a6717 100644 > --- a/mm/gup.c > +++ b/mm/gup.c > @@ -1270,6 +1270,17 @@ int fixup_user_fault(struct mm_struct *mm, > } > EXPORT_SYMBOL_GPL(fixup_user_fault); > > +/* > + * Set the MMF_HAS_PINNED if not set yet; after set it'll be there for the mm's > + * lifecycle. Avoid setting the bit unless necessary, or it might cause write > + * cache bouncing on large SMP machines for concurrent pinned gups. > + */ > +static inline void mm_set_has_pinned_flag(unsigned long *mm_flags) > +{ > + if (!test_bit(MMF_HAS_PINNED, mm_flags)) > + set_bit(MMF_HAS_PINNED, mm_flags); > +} > + > /* > * Please note that this function, unlike __get_user_pages will not > * return 0 for nr_pages > 0 without FOLL_NOWAIT > @@ -1292,8 +1303,8 @@ static __always_inline long __get_user_pages_locked(struct mm_struct *mm, > BUG_ON(*locked != 1); > } > > - if ((flags & FOLL_PIN) && !atomic_read(&mm->has_pinned)) > - atomic_set(&mm->has_pinned, 1); > + if (flags & FOLL_PIN) > + mm_set_has_pinned_flag(&mm->flags); > > /* > * FOLL_PIN and FOLL_GET are mutually exclusive. Traditional behavior > @@ -2617,8 +2628,8 @@ static int internal_get_user_pages_fast(unsigned long start, > FOLL_FAST_ONLY))) > return -EINVAL; > > - if ((gup_flags & FOLL_PIN) && !atomic_read(¤t->mm->has_pinned)) > - atomic_set(¤t->mm->has_pinned, 1); > + if (gup_flags & FOLL_PIN) > + mm_set_has_pinned_flag(¤t->mm->flags); noreply@ellerman.id.au reports: FAILED linux-next/m5272c3_defconfig/m68k-gcc8 Wed May 12, 19:30 http://kisskb.ellerman.id.au/kisskb/buildresult/14543658/ Commit: Add linux-next specific files for 20210512 ec85c95b0c90a17413901b018e8ade7b9eae7cad Compiler: m68k-linux-gcc (GCC) 8.1.0 / GNU ld (GNU Binutils) 2.30 mm/gup.c:2698:3: error: implicit declaration of function 'mm_set_has_pinned_flag'; did you mean 'set_tsk_thread_flag'? [-Werror=implicit-function-declaration] It's definition is inside the #ifdef CONFIG_MMU section, but the last user isn't. Gr{oetje,eeting}s, Geert -- Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B0CFC433B4 for ; Wed, 12 May 2021 09:49:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6485061263 for ; Wed, 12 May 2021 09:49:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6485061263 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-m68k.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A0B496B0036; Wed, 12 May 2021 05:49:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9BB4B6B006E; Wed, 12 May 2021 05:49:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 85CB76B0070; Wed, 12 May 2021 05:49:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0040.hostedemail.com [216.40.44.40]) by kanga.kvack.org (Postfix) with ESMTP id 66CB96B0036 for ; Wed, 12 May 2021 05:49:18 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1A6D1180AD811 for ; Wed, 12 May 2021 09:49:18 +0000 (UTC) X-FDA: 78132105996.16.65EF20E Received: from mail-ua1-f47.google.com (mail-ua1-f47.google.com [209.85.222.47]) by imf20.hostedemail.com (Postfix) with ESMTP id 63F7AD5 for ; Wed, 12 May 2021 09:49:17 +0000 (UTC) Received: by mail-ua1-f47.google.com with SMTP id g24so7275193uak.11 for ; Wed, 12 May 2021 02:49:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=HwuV0oIEoZ+EQ5+Mm5jlhwshZkWr3IhzEV/rE9T6cEw=; b=aS3Ep7rnPc/Ya6vW08mpXXXol6ETd2wkA4CR1ZFgtjDbxfOTqR90ioOE3pthRmXDFB V77p5p51xNtdnvwZGs60hvmpdEc1wkv/YBOTu7Io+p3J3S4SjQadM+tmcGQnwz+0lQhq snGbbepphyedifx11Wo/OHsUGqquQGaNJIGvZ/USwpYKgVvDTXy+oc9P1z95v85EfTFk qSmnqjEoJ6M0IzHOktOHSFpFSTciUDPfgnbmOo1JpRgeDwO/F3xyYQOz/brQur0/PdpU Hqx5s7LqlEkE2yUjQPW6l7jcmNHXmPIK0IMvUMTS1QD7ZcjOEp6VNaZ+pg+eqArMfuK8 XPQw== X-Gm-Message-State: AOAM531e48rSgeqvHOjhTLceNccVxDZ/xvKBSBozKKJGcQ/n3VA3VIJZ fQ1sIbZ3d3euNKI6k2I4QIvHjJVmL0ChrSyktyM= X-Google-Smtp-Source: ABdhPJzQGXZD6dgZ85APfNUlHHznrQzLksJl7I1MmQfQ/FfX780s5vYhVBREXXLKbtUN48frqg1GMbdzUXcfPyHZELY= X-Received: by 2002:a9f:3852:: with SMTP id q18mr30989572uad.58.1620812956684; Wed, 12 May 2021 02:49:16 -0700 (PDT) MIME-Version: 1.0 References: <20210507150553.208763-1-peterx@redhat.com> <20210507150553.208763-4-peterx@redhat.com> In-Reply-To: <20210507150553.208763-4-peterx@redhat.com> From: Geert Uytterhoeven Date: Wed, 12 May 2021 11:49:05 +0200 Message-ID: Subject: Re: [PATCH v2 3/3] mm: gup: pack has_pinned in MMF_HAS_PINNED To: Peter Xu , Andrea Arcangeli Cc: Linux MM , Linux Kernel Mailing List , Jan Kara , John Hubbard , Linus Torvalds , Michal Hocko , Kirill Tkhai , Kirill Shutemov , Oleg Nesterov , Andrew Morton , Jann Horn , Jason Gunthorpe , Matthew Wilcox , Hugh Dickins Content-Type: text/plain; charset="UTF-8" Authentication-Results: imf20.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf20.hostedemail.com: domain of geertuytterhoeven@gmail.com designates 209.85.222.47 as permitted sender) smtp.mailfrom=geertuytterhoeven@gmail.com X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 63F7AD5 X-Stat-Signature: xehxm7c1qo5ocsyxtar5hpf8tf3i6o78 Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf20; identity=mailfrom; envelope-from=""; helo=mail-ua1-f47.google.com; client-ip=209.85.222.47 X-HE-DKIM-Result: none/none X-HE-Tag: 1620812957-609010 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Peter, Andrea, On Fri, May 7, 2021 at 7:26 PM Peter Xu wrote: > From: Andrea Arcangeli > > has_pinned 32bit can be packed in the MMF_HAS_PINNED bit as a noop > cleanup. > > Any atomic_inc/dec to the mm cacheline shared by all threads in > pin-fast would reintroduce a loss of SMP scalability to pin-fast, so > there's no future potential usefulness to keep an atomic in the mm for > this. > > set_bit(MMF_HAS_PINNED) will be theoretically a bit slower than > WRITE_ONCE (atomic_set is equivalent to WRITE_ONCE), but the set_bit > (just like atomic_set after this commit) has to be still issued only > once per "mm", so the difference between the two will be lost in the > noise. > > will-it-scale "mmap2" shows no change in performance with enterprise > config as expected. > > will-it-scale "pin_fast" retains the > 4000% SMP scalability > performance improvement against upstream as expected. > > This is a noop as far as overall performance and SMP scalability are > concerned. > > Signed-off-by: Andrea Arcangeli > [peterx: Fix build for task_mmu.c, introduce mm_set_has_pinned_flag, fix > comment here and there] > Signed-off-by: Peter Xu Thanks for your patch, which is now in linux-next. > diff --git a/mm/gup.c b/mm/gup.c > index 9933bc5c2eff2..bb130723a6717 100644 > --- a/mm/gup.c > +++ b/mm/gup.c > @@ -1270,6 +1270,17 @@ int fixup_user_fault(struct mm_struct *mm, > } > EXPORT_SYMBOL_GPL(fixup_user_fault); > > +/* > + * Set the MMF_HAS_PINNED if not set yet; after set it'll be there for the mm's > + * lifecycle. Avoid setting the bit unless necessary, or it might cause write > + * cache bouncing on large SMP machines for concurrent pinned gups. > + */ > +static inline void mm_set_has_pinned_flag(unsigned long *mm_flags) > +{ > + if (!test_bit(MMF_HAS_PINNED, mm_flags)) > + set_bit(MMF_HAS_PINNED, mm_flags); > +} > + > /* > * Please note that this function, unlike __get_user_pages will not > * return 0 for nr_pages > 0 without FOLL_NOWAIT > @@ -1292,8 +1303,8 @@ static __always_inline long __get_user_pages_locked(struct mm_struct *mm, > BUG_ON(*locked != 1); > } > > - if ((flags & FOLL_PIN) && !atomic_read(&mm->has_pinned)) > - atomic_set(&mm->has_pinned, 1); > + if (flags & FOLL_PIN) > + mm_set_has_pinned_flag(&mm->flags); > > /* > * FOLL_PIN and FOLL_GET are mutually exclusive. Traditional behavior > @@ -2617,8 +2628,8 @@ static int internal_get_user_pages_fast(unsigned long start, > FOLL_FAST_ONLY))) > return -EINVAL; > > - if ((gup_flags & FOLL_PIN) && !atomic_read(¤t->mm->has_pinned)) > - atomic_set(¤t->mm->has_pinned, 1); > + if (gup_flags & FOLL_PIN) > + mm_set_has_pinned_flag(¤t->mm->flags); noreply@ellerman.id.au reports: FAILED linux-next/m5272c3_defconfig/m68k-gcc8 Wed May 12, 19:30 http://kisskb.ellerman.id.au/kisskb/buildresult/14543658/ Commit: Add linux-next specific files for 20210512 ec85c95b0c90a17413901b018e8ade7b9eae7cad Compiler: m68k-linux-gcc (GCC) 8.1.0 / GNU ld (GNU Binutils) 2.30 mm/gup.c:2698:3: error: implicit declaration of function 'mm_set_has_pinned_flag'; did you mean 'set_tsk_thread_flag'? [-Werror=implicit-function-declaration] It's definition is inside the #ifdef CONFIG_MMU section, but the last user isn't. Gr{oetje,eeting}s, Geert -- Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that. -- Linus Torvalds