From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBC05C433FE for ; Wed, 22 Sep 2021 11:43:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9F44360FA0 for ; Wed, 22 Sep 2021 11:43:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235847AbhIVLor (ORCPT ); Wed, 22 Sep 2021 07:44:47 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:32346 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235730AbhIVLop (ORCPT ); Wed, 22 Sep 2021 07:44:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1632310995; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5n+PJJpfTukBRlRBxGVs9WKo8MC+YhuW/pImDRLgI4I=; b=T5DdL6Y/oLq5TJMROmQn8cGBaq3UgV0yV/ihdYXBfaOq8L+Nol3qC2VayO6Vtl38FXRywr MRFMJxB3/SwOFx+ptfzi6y1wgheyYUwyurHh+ZPsqgmTY0IrizAzv7Ag5aNO/9gs9JVZT2 WEQBzHFqkwU/OQXp9j+Li6zYxJkye2E= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-371--dxr9HukNnWChM0c6VLbgw-1; Wed, 22 Sep 2021 07:43:14 -0400 X-MC-Unique: -dxr9HukNnWChM0c6VLbgw-1 Received: by mail-wr1-f70.google.com with SMTP id j16-20020adfa550000000b0016012acc443so1873058wrb.14 for ; Wed, 22 Sep 2021 04:43:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=5n+PJJpfTukBRlRBxGVs9WKo8MC+YhuW/pImDRLgI4I=; b=cPBmGtxVtz+FMNNTEqslybGV7wowPS5f9N8HhC4Qtcy0m80IeCvnhq9CYZDJGZy5dT ZiW7eQvMxKv8syX7lzWh9muwt2yEN6Le5Z1a/0ZeolwOGqlm2UkFJ0fevNcDhwHxQsFS ChLPjn9fXRopqxrxOM9k8j2E0Znu4hBUowvo0+w3uDgEFAw80nzXQiXlQdH+HOuc6HpP EqIh09EZ0h6YN/7NxRg8gSB7UmbLhM9BTRJ210vmUIGFW3E9pP/hc1mQKjzy92dAdHZS U3jMwspBHp6yvqOfyd+7siXyxdMOGrQtZ8Rt2pvIkTpbN4ZtoLhmBJmHRQo62jEk7WpI 9PrQ== X-Gm-Message-State: AOAM531zC4MQpNGulp2SumHDmM16GVNR0sJ+kFzt7IG70bb6DfXKxkVM hxh6Zs2lBjnh2e69+qLDK8OqmPTbQXr4Uym1skRn5tucGUnE4d/AId6sFwRc8ZInT4ZMt/MySfr ZKHlZC5nnxwQEwxxmUG1efnE7 X-Received: by 2002:a05:6000:7:: with SMTP id h7mr40582780wrx.134.1632310992857; Wed, 22 Sep 2021 04:43:12 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxaLJd1RGo11rnmuA15w9wZ7IbiFzTULiRw8v3kf/DAmp1sfLo/eRLf2Yp2zODBAbsQBhltvw== X-Received: by 2002:a05:6000:7:: with SMTP id h7mr40582760wrx.134.1632310992664; Wed, 22 Sep 2021 04:43:12 -0700 (PDT) Received: from ?IPv6:2a0c:5a80:1d03:b900:c3d1:5974:ce92:3123? ([2a0c:5a80:1d03:b900:c3d1:5974:ce92:3123]) by smtp.gmail.com with ESMTPSA id 135sm5969068wma.32.2021.09.22.04.43.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Sep 2021 04:43:12 -0700 (PDT) Message-ID: Subject: Re: [PATCH 2/6] mm/swap: Introduce alternative per-cpu LRU cache locking From: nsaenzju@redhat.com To: Peter Zijlstra Cc: akpm@linux-foundation.org, frederic@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, tglx@linutronix.de, cl@linux.com, juri.lelli@redhat.com, mingo@redhat.com, mtosatti@redhat.com, nilal@redhat.com, mgorman@suse.de, ppandit@redhat.com, williams@redhat.com, bigeasy@linutronix.de, anna-maria@linutronix.de, linux-rt-users@vger.kernel.org Date: Wed, 22 Sep 2021 13:43:11 +0200 In-Reply-To: <20210922113736.GP4323@worktop.programming.kicks-ass.net> References: <20210921161323.607817-1-nsaenzju@redhat.com> <20210921161323.607817-3-nsaenzju@redhat.com> <20210921220358.GN4323@worktop.programming.kicks-ass.net> <0ec733daf2daaf8a6f5b1fbf56676b9892d5bf73.camel@redhat.com> <20210922113736.GP4323@worktop.programming.kicks-ass.net> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.40.4 (3.40.4-1.fc34) MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 2021-09-22 at 13:37 +0200, Peter Zijlstra wrote: > On Wed, Sep 22, 2021 at 10:47:07AM +0200, nsaenzju@redhat.com wrote: > > On Wed, 2021-09-22 at 00:03 +0200, Peter Zijlstra wrote: > > > On Tue, Sep 21, 2021 at 06:13:20PM +0200, Nicolas Saenz Julienne wrote: > > > > +static inline void lru_cache_lock(struct lru_cache_locks *locks) > > > > +{ > > > > + if (static_branch_unlikely(&remote_pcpu_cache_access)) { > > > > + /* Avoid migration between this_cpu_ptr() and spin_lock() */ > > > > + migrate_disable(); > > > > + spin_lock(this_cpu_ptr(&locks->spin)); > > > > + } else { > > > > + local_lock(&locks->local); > > > > + } > > > > +} > > > > > > > +static inline void lru_cache_unlock(struct lru_cache_locks *locks) > > > > +{ > > > > + if (static_branch_unlikely(&remote_pcpu_cache_access)) { > > > > + spin_unlock(this_cpu_ptr(&locks->spin)); > > > > + migrate_enable(); > > > > + } else { > > > > + local_unlock(&locks->local); > > > > + } > > > > +} > > > > > > *why* use migrate_disable(), that's horrible! > > > > I was trying to be mindful of RT. They don't appreciate people taking spinlocks > > just after having disabled preemption. > > > > I think getting local_lock(&locks->local) is my only option then. But it adds > > an extra redundant spinlock in the RT+NOHZ_FULL case. > > That doesn't make it less horrible. The fundamental problem you seem to > have is that you have to do the this_cpu thing multiple times. > > If instead you make sure to only ever do the per-cpu deref *once* and > then make sure you use the same lru_rotate.pvec as you used > lru_rotate.locks, it all works out much nicer. > > Then you can write things like: > > struct lru_rotate *lr = raw_cpu_ptr(&lru_rotate); > > frob_lock(&lr->locks); > frob_pvec(&lr->pvec); > frob_unlock(&lr->locks); > > and it all no longer matters if you got this or that CPU's instance. > > After all, you no longer rely on per-cpu ness for serialization but the > lock. Thanks, understood. I'll look into it. -- Nicolás Sáenz From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 510BEC433EF for ; Wed, 22 Sep 2021 11:43:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CC96660FA0 for ; Wed, 22 Sep 2021 11:43:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org CC96660FA0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 5EE87900002; Wed, 22 Sep 2021 07:43:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 59E3B6B0072; Wed, 22 Sep 2021 07:43:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 465AA900002; Wed, 22 Sep 2021 07:43:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0254.hostedemail.com [216.40.44.254]) by kanga.kvack.org (Postfix) with ESMTP id 37C926B006C for ; Wed, 22 Sep 2021 07:43:16 -0400 (EDT) Received: from smtpin39.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id EB1FD284AE for ; Wed, 22 Sep 2021 11:43:15 +0000 (UTC) X-FDA: 78615023550.39.BFF6DD2 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf14.hostedemail.com (Postfix) with ESMTP id 9F8FA6001981 for ; Wed, 22 Sep 2021 11:43:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1632310995; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5n+PJJpfTukBRlRBxGVs9WKo8MC+YhuW/pImDRLgI4I=; b=T5DdL6Y/oLq5TJMROmQn8cGBaq3UgV0yV/ihdYXBfaOq8L+Nol3qC2VayO6Vtl38FXRywr MRFMJxB3/SwOFx+ptfzi6y1wgheyYUwyurHh+ZPsqgmTY0IrizAzv7Ag5aNO/9gs9JVZT2 WEQBzHFqkwU/OQXp9j+Li6zYxJkye2E= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-371-SWvCf8ckOs2ANPNGdPRXFg-1; Wed, 22 Sep 2021 07:43:14 -0400 X-MC-Unique: SWvCf8ckOs2ANPNGdPRXFg-1 Received: by mail-wr1-f71.google.com with SMTP id c2-20020adfa302000000b0015e4260febdso1847651wrb.20 for ; Wed, 22 Sep 2021 04:43:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=5n+PJJpfTukBRlRBxGVs9WKo8MC+YhuW/pImDRLgI4I=; b=eO7Xic2KBQRCwt5LWTJBGyvYfG8C4ZoaS3OfroxlbXUl0Z4kfXEvmknM38kS3D6+bd BC3o5plYgOAaHyV1ESa9F9cn4A1//GPMPWqy7Om8vjBrcYI4ZktZxVZvNeVuUNYO7MYa LAEMEpnJHiSyutsGmH3NIi3gd1tKmqPgnV/5SHxnpE/pNAaEu/X9ElNNe1jj4N2v0/R6 bowr74rPywFW1YzSkxPL4AjDm8M7aru+euplijyHPT6xDuFJZC5nzS4YlnqB5D4sGWfW 29DM1P9ULzP9/ZcdU4PIb3Op/tIrX/59JKFHRl2xrmzTHZT2RkU32Ya758uZR2D/Be9C KJ3Q== X-Gm-Message-State: AOAM532Tfipagl/6EKB7utAPNuBwuhmAJqj1BZ8JhMJbkMBD6WnmOozZ z4hEnepOSMdnfAYizNFUEMZaZx1qIlSArzgikj5t2Z6R0n8RM6SiiNc61+JhO7b30wwrfltveZB rYpVOsbjI9uA= X-Received: by 2002:a05:6000:7:: with SMTP id h7mr40582794wrx.134.1632310992859; Wed, 22 Sep 2021 04:43:12 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxaLJd1RGo11rnmuA15w9wZ7IbiFzTULiRw8v3kf/DAmp1sfLo/eRLf2Yp2zODBAbsQBhltvw== X-Received: by 2002:a05:6000:7:: with SMTP id h7mr40582760wrx.134.1632310992664; Wed, 22 Sep 2021 04:43:12 -0700 (PDT) Received: from ?IPv6:2a0c:5a80:1d03:b900:c3d1:5974:ce92:3123? ([2a0c:5a80:1d03:b900:c3d1:5974:ce92:3123]) by smtp.gmail.com with ESMTPSA id 135sm5969068wma.32.2021.09.22.04.43.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Sep 2021 04:43:12 -0700 (PDT) Message-ID: Subject: Re: [PATCH 2/6] mm/swap: Introduce alternative per-cpu LRU cache locking From: nsaenzju@redhat.com To: Peter Zijlstra Cc: akpm@linux-foundation.org, frederic@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, tglx@linutronix.de, cl@linux.com, juri.lelli@redhat.com, mingo@redhat.com, mtosatti@redhat.com, nilal@redhat.com, mgorman@suse.de, ppandit@redhat.com, williams@redhat.com, bigeasy@linutronix.de, anna-maria@linutronix.de, linux-rt-users@vger.kernel.org Date: Wed, 22 Sep 2021 13:43:11 +0200 In-Reply-To: <20210922113736.GP4323@worktop.programming.kicks-ass.net> References: <20210921161323.607817-1-nsaenzju@redhat.com> <20210921161323.607817-3-nsaenzju@redhat.com> <20210921220358.GN4323@worktop.programming.kicks-ass.net> <0ec733daf2daaf8a6f5b1fbf56676b9892d5bf73.camel@redhat.com> <20210922113736.GP4323@worktop.programming.kicks-ass.net> User-Agent: Evolution 3.40.4 (3.40.4-1.fc34) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 9F8FA6001981 Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="T5DdL6Y/"; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf14.hostedemail.com: domain of nsaenzju@redhat.com has no SPF policy when checking 216.205.24.124) smtp.mailfrom=nsaenzju@redhat.com X-Stat-Signature: bf53aqcjyog9ksocwwfd635gkuq998aw X-HE-Tag: 1632310995-977645 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.001184, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, 2021-09-22 at 13:37 +0200, Peter Zijlstra wrote: > On Wed, Sep 22, 2021 at 10:47:07AM +0200, nsaenzju@redhat.com wrote: > > On Wed, 2021-09-22 at 00:03 +0200, Peter Zijlstra wrote: > > > On Tue, Sep 21, 2021 at 06:13:20PM +0200, Nicolas Saenz Julienne wr= ote: > > > > +static inline void lru_cache_lock(struct lru_cache_locks *locks) > > > > +{ > > > > + if (static_branch_unlikely(&remote_pcpu_cache_access)) { > > > > + /* Avoid migration between this_cpu_ptr() and spin_lock() */ > > > > + migrate_disable(); > > > > + spin_lock(this_cpu_ptr(&locks->spin)); > > > > + } else { > > > > + local_lock(&locks->local); > > > > + } > > > > +} > > >=20 > > > > +static inline void lru_cache_unlock(struct lru_cache_locks *lock= s) > > > > +{ > > > > + if (static_branch_unlikely(&remote_pcpu_cache_access)) { > > > > + spin_unlock(this_cpu_ptr(&locks->spin)); > > > > + migrate_enable(); > > > > + } else { > > > > + local_unlock(&locks->local); > > > > + } > > > > +} > > >=20 > > > *why* use migrate_disable(), that's horrible! > >=20 > > I was trying to be mindful of RT. They don't appreciate people taking= spinlocks > > just after having disabled preemption. > >=20 > > I think getting local_lock(&locks->local) is my only option then. But= it adds > > an extra redundant spinlock in the RT+NOHZ_FULL case. >=20 > That doesn't make it less horrible. The fundamental problem you seem to > have is that you have to do the this_cpu thing multiple times. >=20 > If instead you make sure to only ever do the per-cpu deref *once* and > then make sure you use the same lru_rotate.pvec as you used > lru_rotate.locks, it all works out much nicer. >=20 > Then you can write things like: >=20 > struct lru_rotate *lr =3D raw_cpu_ptr(&lru_rotate); >=20 > frob_lock(&lr->locks); > frob_pvec(&lr->pvec); > frob_unlock(&lr->locks); >=20 > and it all no longer matters if you got this or that CPU's instance. >=20 > After all, you no longer rely on per-cpu ness for serialization but the > lock. Thanks, understood. I'll look into it. --=20 Nicol=C3=A1s S=C3=A1enz