From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2923C433EF for ; Sat, 9 Oct 2021 02:02:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2957561038 for ; Sat, 9 Oct 2021 02:02:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2957561038 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 854926B0071; Fri, 8 Oct 2021 22:02:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 804136B0072; Fri, 8 Oct 2021 22:02:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6CBA9900002; Fri, 8 Oct 2021 22:02:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0098.hostedemail.com [216.40.44.98]) by kanga.kvack.org (Postfix) with ESMTP id 5E23A6B0071 for ; Fri, 8 Oct 2021 22:02:23 -0400 (EDT) Received: from smtpin36.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 0746439BB6 for ; Sat, 9 Oct 2021 02:02:23 +0000 (UTC) X-FDA: 78675249366.36.1E67337 Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) by imf04.hostedemail.com (Postfix) with ESMTP id AE1795000C7F for ; Sat, 9 Oct 2021 02:02:22 +0000 (UTC) Received: by mail-pg1-f175.google.com with SMTP id r201so4780524pgr.4 for ; Fri, 08 Oct 2021 19:02:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=tqWHPr6n6uCPqfcd8UA7Ty9EDtwQg2hYwMwBuRivEnA=; b=CUN5WQQrSuG104qIpMwxBWc2UvfOINGHZg5lYOAiQcGW8SnTyCkcpqtKRfrt44XB9A XMdcL6lAccQGgxf003YSZEdp9WFfZzyHByWn+oHY9MEI/kYhNpBo2mjhRkl6Jo2CIKc8 ZpXZLbihQ8KPA/sEPihwdbDws6cwW5yIuMFtcWreeL9ILIu7qFtKWT8e+vP0zBAHBT0K ZH+zUobeWONLEdgjkkVixNZ9h7zBo9aoWwvHlg3UXRUWsUtRDj/lrrpn286YeKIm0y1i PQCTFOFLaWqY8RVWi+C36cKKOHQg0pXtMJsToL6ZDSYaHMlKeys3rNPV5M9dKQj+fspZ CExQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=tqWHPr6n6uCPqfcd8UA7Ty9EDtwQg2hYwMwBuRivEnA=; b=E5QhiPYryKJPuaS2Wvfd3jJjpGfsD460snWBjwkoRVcY24V3IluD9feWASuIWCpHWV Dn/OWIB5sKwOygm1smlhDWfSV2dKhGqJOTf/ajXKe2V8zrbmrz3pT4jXUd16vGZnzCNe NJsuBmHdbbDS5PbObFhtfGG+eZlOQpMjaoCzI+ro5bbfSrm6OB7naLvxcvPP9uVwzl/T nCCXBuEGwiwnoR79CD1NSfzKbyC2SqLJ0wVHxf7aPr4b/beGfTu1NpYRhbKOQChXgUB5 S7GmZiA5reNNA+xatyZDtvfd6g10MSN7jAZ2TfIqe1FW/RH1ZJOMqc2fccRonbq9HhZk i5XQ== X-Gm-Message-State: AOAM530R6QrLgvBCDeexXA5Y2WIDWnlmdrDixvNrOOL1T5mSk0YiNnVO KI078omWLNQ/QN8PeajW8hE+hGkNqpbgMEqdpGw= X-Google-Smtp-Source: ABdhPJxHVuy4liir78YZ7eXSsPUlWT7a9Mh7VlWzPL2eKzg4kW2WQNDqvdIuFrFvg9g9V2m4TpdlqkaVOvhlnAnvoLU= X-Received: by 2002:aa7:991e:0:b0:44c:a172:e670 with SMTP id z30-20020aa7991e000000b0044ca172e670mr13466222pff.10.1633744941496; Fri, 08 Oct 2021 19:02:21 -0700 (PDT) MIME-Version: 1.0 References: <20211009001903.GA3285@kvm.asia-northeast3-a.c.our-ratio-313919.internal> In-Reply-To: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Date: Sat, 9 Oct 2021 11:02:09 +0900 Message-ID: Subject: Re: [RFC] Some questions and an idea on SLUB/SLAB To: Matthew Wilcox Cc: Linux Memory Management List , LKML , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Vlastimil Babka Content-Type: multipart/alternative; boundary="000000000000e43c9a05cde1e2bb" X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: AE1795000C7F X-Stat-Signature: 3mi7cu5r5ns7fs16ibxwxqoz5tcfexzm Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=CUN5WQQr; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf04.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.215.175 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com X-HE-Tag: 1633744942-111589 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --000000000000e43c9a05cde1e2bb Content-Type: text/plain; charset="UTF-8" On Sat, Oct 9, 2021, 9:34 AM Matthew Wilcox wrote: > On Sat, Oct 09, 2021 at 12:19:03AM +0000, Hyeonggon Yoo wrote: > > - Is there a reason that SLUB does not implement cache coloring? > > it will help utilizing hardware cache. Especially in block layer, > > they are literally *squeezing* its performance now. > > Have you tried turning off cache colouring in SLAB and seeing if > performance changes? My impression is that it's useful for caches > with low associativity (direct mapped / 2-way / 4-way), but loses > its effectiveness for caches with higher associativity. For example, > my laptop: > > L1 Data Cache: 48KB, 12-way associative, 64 byte line size > L1 Instruction Cache: 32KB, 8-way associative, 64 byte line size > L2 Unified Cache: 1280KB, 20-way associative, 64 byte line size > L3 Unified Cache: 12288KB, 12-way associative, 64 byte line size > > I very much doubt that cache colouring is still useful for this machine. > And what was result on that benchmark? How many cores on your processor? And is it NUMA or UMA? As I mentioned, color scheme is shared between cpus in same node. I think we need to measure performqnce again after per-cpu coloring. --000000000000e43c9a05cde1e2bb Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Sat, Oct 9, 2021, 9:34 AM Matthew Wilcox <willy@infradead.org> wrote:
On Sat, Oct 09, 2021 at 12:19:03AM +0000, = Hyeonggon Yoo wrote:
>=C2=A0 - Is there a reason that SLUB does not implement cache coloring?=
>=C2=A0 =C2=A0 it will help utilizing hardware cache. Especially in bloc= k layer,
>=C2=A0 =C2=A0 they are literally *squeezing* its performance now.

Have you tried turning off cache colouring in SLAB and seeing if
performance changes?=C2=A0 My impression is that it's useful for caches=
with low associativity (direct mapped / 2-way / 4-way), but loses
its effectiveness for caches with higher associativity.=C2=A0 For example,<= br> my laptop:

=C2=A0L1 Data Cache: 48KB, 12-way associative, 64 byte line size
=C2=A0L1 Instruction Cache: 32KB, 8-way associative, 64 byte line size
=C2=A0L2 Unified Cache: 1280KB, 20-way associative, 64 byte line size
=C2=A0L3 Unified Cache: 12288KB, 12-way associative, 64 byte line size

I very much doubt that cache colouring is still useful for this machine.

And= what was result on that benchmark?

How many cores on your processor?
And is= it NUMA or UMA?

As I me= ntioned, color scheme is shared between cpus in same node.

I think we need to measure performqnce a= gain after per-cpu coloring.=C2=A0

--000000000000e43c9a05cde1e2bb--