From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD996C47255 for ; Fri, 8 May 2020 21:35:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A4405218AC for ; Fri, 8 May 2020 21:35:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728032AbgEHVfz (ORCPT ); Fri, 8 May 2020 17:35:55 -0400 Received: from gentwo.org ([3.19.106.255]:33674 "EHLO gentwo.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726811AbgEHVfz (ORCPT ); Fri, 8 May 2020 17:35:55 -0400 Received: by gentwo.org (Postfix, from userid 1002) id 98CBA3FED9; Fri, 8 May 2020 21:35:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by gentwo.org (Postfix) with ESMTP id 9788F3ED24; Fri, 8 May 2020 21:35:54 +0000 (UTC) Date: Fri, 8 May 2020 21:35:54 +0000 (UTC) From: Christopher Lameter X-X-Sender: cl@www.lameter.com To: Roman Gushchin cc: Andrew Morton , Johannes Weiner , Michal Hocko , linux-mm@kvack.org, kernel-team@fb.com, linux-kernel@vger.kernel.org Subject: Re: [PATCH v3 04/19] mm: slub: implement SLUB version of obj_to_index() In-Reply-To: <20200504182922.GA20009@carbon.dhcp.thefacebook.com> Message-ID: References: <20200422204708.2176080-5-guro@fb.com> <20200423000530.GA63356@carbon.lan> <20200425024625.GA107755@carbon.lan> <20200427164638.GC114719@carbon.DHCP.thefacebook.com> <20200430171558.GA339283@carbon.dhcp.thefacebook.com> <20200504182922.GA20009@carbon.dhcp.thefacebook.com> User-Agent: Alpine 2.22 (DEB 394 2020-01-19) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 4 May 2020, Roman Gushchin wrote: > On Sat, May 02, 2020 at 11:54:09PM +0000, Christoph Lameter wrote: > > On Thu, 30 Apr 2020, Roman Gushchin wrote: > > > > > Sorry, but what exactly do you mean? > > > > I think the right approach is to add a pointer to each slab object for > > memcg support. > > > > As I understand, embedding the memcg pointer will hopefully make allocations > cheaper in terms of CPU, but will require more memory. And you think that > it's worth it. Is it a correct understanding? It definitely makes the code less complex. The additional memory is minimal. In many cases you have already some space wasted at the end of the object that could be used for the pointer. > Can you, please, describe a bit more detailed how it should be done > from your point of view? Add it to the metadata at the end of the object. Like the debugging information or the pointer for RCU freeing. > I mean where to store the pointer, should it be SLAB/SLUB-specific code > or a generic code, what do to with kmallocs alignments, should we > merge slabs which had a different size before and now have the same > because of the memcg pointer and aligment, etc. Both SLAB and SLUB have the same capabilities there. Slabs that had different sizes before will now have different sizes as well. So the merging does not change. > I'm happy to follow your advice and perform some tests to get an idea of > how significant the memory overhead is and how big are CPU savings. > I guess with these numbers it will be easy to make a decision. Sure. The main issue are the power of two kmalloc caches and how to add the pointer to these caches in order not to waste memory. SLAB has done this in the past by creating additional structues in a page frame. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 911C1C38A2A for ; Fri, 8 May 2020 21:35:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5BA13218AC for ; Fri, 8 May 2020 21:35:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5BA13218AC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D54F890001B; Fri, 8 May 2020 17:35:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D2B57900004; Fri, 8 May 2020 17:35:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C8EDB90001B; Fri, 8 May 2020 17:35:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0079.hostedemail.com [216.40.44.79]) by kanga.kvack.org (Postfix) with ESMTP id B1B71900004 for ; Fri, 8 May 2020 17:35:55 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 786312496 for ; Fri, 8 May 2020 21:35:55 +0000 (UTC) X-FDA: 76794859470.26.cows18_6de6401676845 X-HE-Tag: cows18_6de6401676845 X-Filterd-Recvd-Size: 3472 Received: from gentwo.org (gentwo.org [3.19.106.255]) by imf28.hostedemail.com (Postfix) with ESMTP for ; Fri, 8 May 2020 21:35:55 +0000 (UTC) Received: by gentwo.org (Postfix, from userid 1002) id 98CBA3FED9; Fri, 8 May 2020 21:35:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by gentwo.org (Postfix) with ESMTP id 9788F3ED24; Fri, 8 May 2020 21:35:54 +0000 (UTC) Date: Fri, 8 May 2020 21:35:54 +0000 (UTC) From: Christopher Lameter X-X-Sender: cl@www.lameter.com To: Roman Gushchin cc: Andrew Morton , Johannes Weiner , Michal Hocko , linux-mm@kvack.org, kernel-team@fb.com, linux-kernel@vger.kernel.org Subject: Re: [PATCH v3 04/19] mm: slub: implement SLUB version of obj_to_index() In-Reply-To: <20200504182922.GA20009@carbon.dhcp.thefacebook.com> Message-ID: References: <20200422204708.2176080-5-guro@fb.com> <20200423000530.GA63356@carbon.lan> <20200425024625.GA107755@carbon.lan> <20200427164638.GC114719@carbon.DHCP.thefacebook.com> <20200430171558.GA339283@carbon.dhcp.thefacebook.com> <20200504182922.GA20009@carbon.dhcp.thefacebook.com> User-Agent: Alpine 2.22 (DEB 394 2020-01-19) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, 4 May 2020, Roman Gushchin wrote: > On Sat, May 02, 2020 at 11:54:09PM +0000, Christoph Lameter wrote: > > On Thu, 30 Apr 2020, Roman Gushchin wrote: > > > > > Sorry, but what exactly do you mean? > > > > I think the right approach is to add a pointer to each slab object for > > memcg support. > > > > As I understand, embedding the memcg pointer will hopefully make allocations > cheaper in terms of CPU, but will require more memory. And you think that > it's worth it. Is it a correct understanding? It definitely makes the code less complex. The additional memory is minimal. In many cases you have already some space wasted at the end of the object that could be used for the pointer. > Can you, please, describe a bit more detailed how it should be done > from your point of view? Add it to the metadata at the end of the object. Like the debugging information or the pointer for RCU freeing. > I mean where to store the pointer, should it be SLAB/SLUB-specific code > or a generic code, what do to with kmallocs alignments, should we > merge slabs which had a different size before and now have the same > because of the memcg pointer and aligment, etc. Both SLAB and SLUB have the same capabilities there. Slabs that had different sizes before will now have different sizes as well. So the merging does not change. > I'm happy to follow your advice and perform some tests to get an idea of > how significant the memory overhead is and how big are CPU savings. > I guess with these numbers it will be easy to make a decision. Sure. The main issue are the power of two kmalloc caches and how to add the pointer to these caches in order not to waste memory. SLAB has done this in the past by creating additional structues in a page frame.