From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56E94C6FD1C for ; Wed, 22 Mar 2023 12:15:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7A7036B0071; Wed, 22 Mar 2023 08:15:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 756D16B0072; Wed, 22 Mar 2023 08:15:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 61E7D6B0075; Wed, 22 Mar 2023 08:15:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 534506B0071 for ; Wed, 22 Mar 2023 08:15:43 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 2D5F21C5B1B for ; Wed, 22 Mar 2023 12:15:43 +0000 (UTC) X-FDA: 80596430166.08.CE6A974 Received: from mail-yb1-f174.google.com (mail-yb1-f174.google.com [209.85.219.174]) by imf06.hostedemail.com (Postfix) with ESMTP id 1BFAE180022 for ; Wed, 22 Mar 2023 12:15:40 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=NxEOebc0; spf=pass (imf06.hostedemail.com: domain of merimus@google.com designates 209.85.219.174 as permitted sender) smtp.mailfrom=merimus@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679487341; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5I7pnlJifPsqCiTXjzHOT42nvvhPdwPJNKNl0fr8ppE=; b=s+NA247BtiJoHLJmOxJLPwqr1QU4DJ7ShPPu2G8p58/clG6ODuObFna80z9IgKwq7MHN53 JBr9Bz4zB8F6p41OCyU3piZqJoL9KMQ14C4S6aRZs35osKVS6PjTtoUhg/GH5cVF3Z69nv WQ0+4otkenKx9cGjXWoIPthnx+TVbyk= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=NxEOebc0; spf=pass (imf06.hostedemail.com: domain of merimus@google.com designates 209.85.219.174 as permitted sender) smtp.mailfrom=merimus@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679487341; a=rsa-sha256; cv=none; b=22sjHmyf5z8OzCx//FgKHSjOcZ+Dm8/rWEwm4zd8nSayu7uawQep1yFZtUx2J/dhyG+8pB BiKJ5OE+mjC2cMu5RRhfx28i/Mlv31xcYCmlicXu8aKjklYah+sxyd0u2zuo8pq+vmo4u9 v8enNb/dhzDAGlZdiYcrInwE/8UdQi4= Received: by mail-yb1-f174.google.com with SMTP id s67so3881592ybi.5 for ; Wed, 22 Mar 2023 05:15:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1679487340; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=5I7pnlJifPsqCiTXjzHOT42nvvhPdwPJNKNl0fr8ppE=; b=NxEOebc0X9NtHWRtp2CdGqZg9H3151vJ1LKOh78yJOIMe8AwDaBsDdJDN/wzgyG2yX rTDr5w0kYrlV3uHtxs/cOXWRmvfxPfaf89q1OFjDHPV/R1SkyUPRY/RNceXYoOcxvYwM ZkZXccVKIU344B3uTj4sLCKFamJPE1JANudvi23Tmz5p5sWlpYoi/p6NFFqQnkyVLeYp xIHczLTQBpmfYIa0ZCZhGzRZNckGQ+3elXc5Qpxf+kJmKISFECEJYReKRkAROcViVBVA Hi9lJOZwoD5YTmka8PguKu/NhyR6EtQycCzPwA0ZAyw03/A3W7uzEfhf1730qfXyTggS wKWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679487340; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=5I7pnlJifPsqCiTXjzHOT42nvvhPdwPJNKNl0fr8ppE=; b=jPjvqokEl9AGv0gRGTxwhDdxsgFjL+eXSoiX4ldoSe1IemIGTcGR10iaL3fwX6IkBb cNShL8cFuCgwD7k80LWaeZCnvLYQGAwGQ2dfjrRqMYXbeBoGj+DiR4kEhSK+GWppXVeC 5lS/nBpDT1vMc+Vy/oh7XTO4dyJAB77/36yakeg/a4IAieKAXDMwQcuZvRvCfC3sWxeB gtyu8dZFK3Drl4OUUN/Q6k8cgFko4Tm13O16Fum/n1liKC3QXUYfbY8jpEReetRU57fT qE/OO6JmU3coSGxgrJxBH87wcDdB58REfbf6tgJrLntR92IN4eXThB2J36URTssrZO+v MEVw== X-Gm-Message-State: AAQBX9faEMbToalX3wpEVVgNXVRwk/ChvvaAaOuXSh1DFKSkIHtAvLrC g6Ca8JwckeFDO3SCWVmPXTjtyXLAiOPd4Ny7oqyMXw== X-Google-Smtp-Source: AKy350atZEuNFetFTEN0DtF944I7BlL1/9PWK27Kbw59Er588FYN4S5RPmcerFTDz+K84TJg9pEQMv4cRHoWRo9VAss= X-Received: by 2002:a25:1381:0:b0:b72:d8cf:ab16 with SMTP id 123-20020a251381000000b00b72d8cfab16mr1135866ybt.59.1679487339948; Wed, 22 Mar 2023 05:15:39 -0700 (PDT) MIME-Version: 1.0 References: <4b9fc9c6-b48c-198f-5f80-811a44737e5f@suse.cz> In-Reply-To: <4b9fc9c6-b48c-198f-5f80-811a44737e5f@suse.cz> From: Binder Makin Date: Wed, 22 Mar 2023 08:15:28 -0400 Message-ID: Subject: Re: [LSF/MM/BPF TOPIC] SLOB+SLAB allocators removal and future SLUB improvements To: Vlastimil Babka Cc: lsf-pc@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-block@vger.kernel.org, bpf@vger.kernel.org, linux-xfs@vger.kernel.org, David Rientjes , Christoph Lameter , Pekka Enberg , Joonsoo Kim , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Roman Gushchin Content-Type: multipart/alternative; boundary="0000000000004de11905f77c1eee" X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: zzmbs84p65phr4wbckfbt95jrm6crwgs X-Rspamd-Queue-Id: 1BFAE180022 X-HE-Tag: 1679487340-569111 X-HE-Meta: U2FsdGVkX1+44jSeTq89gtEtIdqa4lG7Xqdx018BF52+Y3hWiVUARjN8tatP+UdrwVzy8yohIo+YHnAzk6ZOVpqbXEmIYjMoZMCDzmn/4ptm4sj9o3aOd9LgzR60B4fli66QQ1FzXSBpYYhwpV88Eha46ul1ObJ60LLb2e2Y6qoAbaZXcu7Cp7p1AFLRZqc8OPa/wjcR70akDKid7390VZmfVFAV0xyd1k3mM+ZzNyLzg0mplvH7hu4N3EKC/3Ra2YWqbT3wPPvEFXlnoGgczBC/3J/9XcNLTEbjJ7rNOYuSl7icseSgleizK7t9LgYTJLTeP+9aGn+45TncY2zbc0zsV3QdQroelooyGZ17Lo8T7Qxrz9rMRIRUKUR8WDp1+wNYQ/B09W2UbDs5+r7z/CNyKLDo9OakUTilSRcXD1WbjNy5c0Ukgrp7lt/7RC3Zl2t0GWylSLzZdtYbvAJ79vOE3iW4kw7+22r5jVuS2e2oH8KroEpahwEnJVYyKGxm7N0laRrp9y41gxv2dqJ297j+wAnQhqIOAc+4P+10shXJiBHDuqjnJcEKTPNpZgSMzN176V6Z837NgDgkVxBjR8oaHlegPvZDmQ6MO4DcwB+D81gG7p326tBd2yEmHQ4v1F4W9jVHi7M9g7un91SMpjWOVhVBkOy0suOPMcM0AuWhKKA/g1ZXho9j2yZXZkyYqVI1N32uFyyxtlxa7pvj79dabIqDCgZDCJ7p5VuOacNb2HFB7uvp/7Sik1dp4cK1Bzlsb8eXJhMMsKQuE9anKeYkL16iUv0NeJb0rTjAI9q/qbSnmF51i+fC2qpQf8rCwTglcHqzSagizHtT6GhfbtGYXhpp4tj9HPZoJu6kQXs2lctMgnWMu1A16LMgW7o2VPj1yWIcvbBW85dXfsOKLQuOWSicShdPOlYkkPWtB0JllYoEKwL0Nwbl6bMmI9a9YqwbuSuAzhOte/EAEu3 U+MBTxXh iY9XTNOxfZxa4ooGTqAzs0z8wL3KjfyDFbPTdPOzA7hSKMNMkkQvG66tnxPwJO+XMegGq7AdwoBa3B5NrzOGSapTVUkeXUCs2eeaigzdoEQyCxGsfyO9A6Iv4cCVMIYu36kPLwYv5qqUd0hReh9zkhrT3EvFa9eeXv3GcLqVZgMz2pR4KQar244P+Pw/x1PCBfGXZwBWCZ1ikD4PudUA+EuKb2agrZlaxrf+WMBos5TSmRgOw0x7zOqsq0xeI3+6+mUA8DGvcsScqRCNMzJQYyIzRq3SuONrA1LG0+yPI/iK9z3cdorXCgG3BKIA1VIC5NWyQl7sBmL34rFcGZXW9rEJdwpC0VDHf0Wc9XsbeNGxo8RdY9Gp467J0idhQk+NR+3YtsIQeIxHL/JfQqhITrT7vDTP4Num4exIqK/nycO0VD8kew7TEK1RJFT6PRFiTLb4LKM+umdyqK9/qkkMu6ye1Acu8ErWemnXIyOT17UFK5solwDi9a9bsLrdZOT7wf1iP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --0000000000004de11905f77c1eee Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Was looking at SLAB removal and started by running A/B tests of SLAB vs SLUB. Please note these are only preliminary results. These were run using 6.1.13 configured for SLAB/SLUB. Machines were standard datacenter servers. Hackbench shows completion time, so smaller is better. On all others larger is better. https://docs.google.com/spreadsheets/d/e/2PACX-1vQ47Mekl8BOp3ekCefwL6wL8SQi= v6Qvp5avkU2ssQSh41gntjivE-aKM4PkwzkC4N_s_MxUdcsokhhz/pubhtml Some notes: SUnreclaim and SReclaimable shows unreclaimable and reclaimable memory. Substantially higher with SLUB, but I believe that is to be expected. Various results showing a 5-10% degradation with SLUB. That feels concerning to me, but I'm not sure what others' tolerance would be. redis results on AMD show some pretty bad degredations. 10-20% range netpipe on Intel also has issues.. 10-17% On Tue, Mar 14, 2023 at 4:05=E2=80=AFAM Vlastimil Babka wr= ote: > As you're probably aware, my plan is to get rid of SLOB and SLAB, leaving > only SLUB going forward. The removal of SLOB seems to be going well, ther= e > were no objections to the deprecation and I've posted v1 of the removal > itself [1] so it could be in -next soon. > > The immediate benefit of that is that we can allow kfree() (and > kfree_rcu()) > to free objects from kmem_cache_alloc() - something that IIRC at least xf= s > people wanted in the past, and SLOB was incompatible with that. > > For SLAB removal I haven't yet heard any objections (but also didn't > deprecate it yet) but if there are any users due to particular workloads > doing better with SLAB than SLUB, we can discuss why those would regress > and > what can be done about that in SLUB. > > Once we have just one slab allocator in the kernel, we can take a closer > look at what the users are missing from it that forces them to create own > allocators (e.g. BPF), and could be considered to be added as a generic > implementation to SLUB. > > Thanks, > Vlastimil > > [1] https://lore.kernel.org/all/20230310103210.22372-1-vbabka@suse.cz/ > > --0000000000004de11905f77c1eee Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Was looking at SLAB removal and started=C2=A0by running A/= B tests of SLAB vs SLUB.=C2=A0 Please note these are only preliminary resul= ts.

These were run using 6.1.13 configured for SLAB/SLUB.
Machine= s were standard datacenter servers.

Hackbench shows completion time,= so smaller is better.
On all others larger is better.
https://docs.google.com/spreadsheets/d/e/2PACX-1vQ47Mekl8BOp3ekCefwL6wL8S= Qiv6Qvp5avkU2ssQSh41gntjivE-aKM4PkwzkC4N_s_MxUdcsokhhz/pubhtml

S= ome notes:
SUnreclaim and SReclaimable shows unreclaimable and reclaimab= le memory.
Substantially higher with SLUB, but I believe that is to be e= xpected.

Various results showing a 5-10% degradation=C2=A0with SLUB.= =C2=A0 That feels concerning to me, but I'm not sure what others' t= olerance would be.

redis results on AMD show some pretty bad degreda= tions.=C2=A0 10-20% range
netpipe on Intel also has issues.. 10-17%
<= /div>
O= n Tue, Mar 14, 2023 at 4:05=E2=80=AFAM Vlastimil Babka <vbabka@suse.cz> wrote:
As you're probably aware, my plan is t= o get rid of SLOB and SLAB, leaving
only SLUB going forward. The removal of SLOB seems to be going well, there<= br> were no objections to the deprecation and I've posted v1 of the removal=
itself [1] so it could be in -next soon.

The immediate benefit of that is that we can allow kfree() (and kfree_rcu()= )
to free objects from kmem_cache_alloc() - something that IIRC at least xfs<= br> people wanted in the past, and SLOB was incompatible with that.

For SLAB removal I haven't yet heard any objections (but also didn'= t
deprecate it yet) but if there are any users due to particular workloads doing better with SLAB than SLUB, we can discuss why those would regress an= d
what can be done about that in SLUB.

Once we have just one slab allocator in the kernel, we can take a closer look at what the users are missing from it that forces them to create own allocators (e.g. BPF), and could be considered to be added as a generic
implementation to SLUB.

Thanks,
Vlastimil

[1] https://lore.kernel.org/all/20= 230310103210.22372-1-vbabka@suse.cz/

--0000000000004de11905f77c1eee--