From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13170C388F4 for ; Sat, 28 Sep 2019 01:12:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E341221655 for ; Sat, 28 Sep 2019 01:12:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726408AbfI1BMv (ORCPT ); Fri, 27 Sep 2019 21:12:51 -0400 Received: from gentwo.org ([3.19.106.255]:49786 "EHLO gentwo.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725990AbfI1BMu (ORCPT ); Fri, 27 Sep 2019 21:12:50 -0400 Received: by gentwo.org (Postfix, from userid 1002) id 438FF3EEC9; Sat, 28 Sep 2019 01:12:49 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by gentwo.org (Postfix) with ESMTP id 413B93E8CB; Sat, 28 Sep 2019 01:12:49 +0000 (UTC) Date: Sat, 28 Sep 2019 01:12:49 +0000 (UTC) From: Christopher Lameter X-X-Sender: cl@www.lameter.com To: Vlastimil Babka cc: Andrew Morton , David Sterba , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Pekka Enberg , David Rientjes , Ming Lei , Dave Chinner , Matthew Wilcox , "Darrick J . Wong" , Christoph Hellwig , linux-xfs@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, James Bottomley , linux-btrfs@vger.kernel.org, Roman Gushchin , Johannes Weiner Subject: Re: [PATCH v2 2/2] mm, sl[aou]b: guarantee natural alignment for kmalloc(power-of-two) In-Reply-To: <6a28a096-0e65-c7ea-9ca9-f72d68948e10@suse.cz> Message-ID: References: <20190826111627.7505-1-vbabka@suse.cz> <20190826111627.7505-3-vbabka@suse.cz> <20190923171710.GN2751@twin.jikos.cz> <20190924165425.a79a2dafbaf37828a931df2b@linux-foundation.org> <6a28a096-0e65-c7ea-9ca9-f72d68948e10@suse.cz> User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On Thu, 26 Sep 2019, Vlastimil Babka wrote: > > - It will only work for special cases like the kmalloc array > > without extras like metadata at the end of objects. > > I don't understand what you mean here? The kmalloc caches are special > because they don't have metadata at the end of objects? Others do? Yes. > > - These alignments are only needed in exceptional cases but with the patch > > we will provide the alignment by default even if the allocating subsystem > > does not need it. > > True. This is where we have to make the decision whether to make things > simpler for those that don't realize they need the alignment, and > whether that's worth the cost. We have evidence of those cases, and the > cost is currently zero in the common cases (SLAB, SLUB without debug > runtime-enabled). The cost is zero for a particular layout of the objects in a page using a particular allocator and hardware configuration. However, the layout may be different due to another allocator that prefers to arrange things differently (SLOB puts multiple objects of different types in the same page to save memory), if we need to add data to these objects (debugging info, new metadata about the object, maybe the memcg pointer, maybe other things that may come up), or other innovative approaches (such as putting data of different kmem caches that are commonly used together in the same page to improve locality). The cost is an unnecessary petrification of the data layout of the memory allocators. > > - We have mechanisms to detect alignment problems using debug kernels and > > debug options that have been available for years. These were not used for > > testing in these cases it seems before the patches hit mainline. Once in > > mainly someone ran a debug kernel and found the issue. > > Debugging options are useful if you know there's a bug and you want to > find it. AFAIK the various bots/CIs that do e.g. randconfig, or enable > debug options explicitly, run those kernels in a VM, so I guess that's > why potential breakage due to alignment can lurk in a hw-specific driver. That is not my experience. You need to run debugging to verify that a patch does not cause locking problems, memory corruption etc etc. And upstream code is tested by various people with debugging kernels so they will locate the bugs that others introduce. This is usually not because there was a focus on a particular bug. If you have a hw specific thing that is not generally tested and skip the debugging tests well yes then we have a problem. What I have seen with developers is that they feel the debugging steps are unnecessary for conveniences sake. I have seen build environments that had proper steps for verification with a debug kernel. However, someone disabled them "some months ago" and "nothing happened". Then strange failures in production systems occur. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C63CEC4360C for ; Sat, 28 Sep 2019 01:12:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 850F420872 for ; Sat, 28 Sep 2019 01:12:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 850F420872 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1E5A38E0006; Fri, 27 Sep 2019 21:12:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 195D68E0001; Fri, 27 Sep 2019 21:12:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 084588E0006; Fri, 27 Sep 2019 21:12:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0165.hostedemail.com [216.40.44.165]) by kanga.kvack.org (Postfix) with ESMTP id DBA4C8E0001 for ; Fri, 27 Sep 2019 21:12:50 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 7A4AC180AD803 for ; Sat, 28 Sep 2019 01:12:50 +0000 (UTC) X-FDA: 75982554900.24.cakes15_18c1c7f62a317 X-HE-Tag: cakes15_18c1c7f62a317 X-Filterd-Recvd-Size: 4919 Received: from gentwo.org (gentwo.org [3.19.106.255]) by imf11.hostedemail.com (Postfix) with ESMTP for ; Sat, 28 Sep 2019 01:12:50 +0000 (UTC) Received: by gentwo.org (Postfix, from userid 1002) id 438FF3EEC9; Sat, 28 Sep 2019 01:12:49 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by gentwo.org (Postfix) with ESMTP id 413B93E8CB; Sat, 28 Sep 2019 01:12:49 +0000 (UTC) Date: Sat, 28 Sep 2019 01:12:49 +0000 (UTC) From: Christopher Lameter X-X-Sender: cl@www.lameter.com To: Vlastimil Babka cc: Andrew Morton , David Sterba , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Pekka Enberg , David Rientjes , Ming Lei , Dave Chinner , Matthew Wilcox , "Darrick J . Wong" , Christoph Hellwig , linux-xfs@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, James Bottomley , linux-btrfs@vger.kernel.org, Roman Gushchin , Johannes Weiner Subject: Re: [PATCH v2 2/2] mm, sl[aou]b: guarantee natural alignment for kmalloc(power-of-two) In-Reply-To: <6a28a096-0e65-c7ea-9ca9-f72d68948e10@suse.cz> Message-ID: References: <20190826111627.7505-1-vbabka@suse.cz> <20190826111627.7505-3-vbabka@suse.cz> <20190923171710.GN2751@twin.jikos.cz> <20190924165425.a79a2dafbaf37828a931df2b@linux-foundation.org> <6a28a096-0e65-c7ea-9ca9-f72d68948e10@suse.cz> User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, 26 Sep 2019, Vlastimil Babka wrote: > > - It will only work for special cases like the kmalloc array > > without extras like metadata at the end of objects. > > I don't understand what you mean here? The kmalloc caches are special > because they don't have metadata at the end of objects? Others do? Yes. > > - These alignments are only needed in exceptional cases but with the patch > > we will provide the alignment by default even if the allocating subsystem > > does not need it. > > True. This is where we have to make the decision whether to make things > simpler for those that don't realize they need the alignment, and > whether that's worth the cost. We have evidence of those cases, and the > cost is currently zero in the common cases (SLAB, SLUB without debug > runtime-enabled). The cost is zero for a particular layout of the objects in a page using a particular allocator and hardware configuration. However, the layout may be different due to another allocator that prefers to arrange things differently (SLOB puts multiple objects of different types in the same page to save memory), if we need to add data to these objects (debugging info, new metadata about the object, maybe the memcg pointer, maybe other things that may come up), or other innovative approaches (such as putting data of different kmem caches that are commonly used together in the same page to improve locality). The cost is an unnecessary petrification of the data layout of the memory allocators. > > - We have mechanisms to detect alignment problems using debug kernels and > > debug options that have been available for years. These were not used for > > testing in these cases it seems before the patches hit mainline. Once in > > mainly someone ran a debug kernel and found the issue. > > Debugging options are useful if you know there's a bug and you want to > find it. AFAIK the various bots/CIs that do e.g. randconfig, or enable > debug options explicitly, run those kernels in a VM, so I guess that's > why potential breakage due to alignment can lurk in a hw-specific driver. That is not my experience. You need to run debugging to verify that a patch does not cause locking problems, memory corruption etc etc. And upstream code is tested by various people with debugging kernels so they will locate the bugs that others introduce. This is usually not because there was a focus on a particular bug. If you have a hw specific thing that is not generally tested and skip the debugging tests well yes then we have a problem. What I have seen with developers is that they feel the debugging steps are unnecessary for conveniences sake. I have seen build environments that had proper steps for verification with a debug kernel. However, someone disabled them "some months ago" and "nothing happened". Then strange failures in production systems occur.