From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67F01C7EE26 for ; Sun, 7 May 2023 10:27:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231540AbjEGK1Z (ORCPT ); Sun, 7 May 2023 06:27:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58592 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229525AbjEGK1X (ORCPT ); Sun, 7 May 2023 06:27:23 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [IPv6:2001:67c:2178:6::1c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0DF49100D7; Sun, 7 May 2023 03:27:20 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 1A5DC218D5; Sun, 7 May 2023 10:27:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1683455239; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oav2A28KVKDaotNibkAokXcYPyLTU0ijj1klZnIOiRI=; b=c+QuWE6EU4wWKhietxfwN/3l7yAy0K35os01H7NfRFE82alr8YUVFXQqf5mweH8wlzKf+k lu0AkCEfmwJ2VDmkoLh8SVCJbT4jzXXTf9sUf9reepl+WLoX/HYO+d0q2t7pYqKTvW4ACE 6vuKCAGkjFOK3NT53IiVKVs+9jUJHkc= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E9F17139C3; Sun, 7 May 2023 10:27:18 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id Yy8TOQZ9V2RkOQAAMHmgww (envelope-from ); Sun, 07 May 2023 10:27:18 +0000 Date: Sun, 7 May 2023 12:27:18 +0200 From: Michal Hocko To: Suren Baghdasaryan Cc: akpm@linux-foundation.org, kent.overstreet@linux.dev, vbabka@suse.cz, hannes@cmpxchg.org, roman.gushchin@linux.dev, mgorman@suse.de, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, corbet@lwn.net, void@manifault.com, peterz@infradead.org, juri.lelli@redhat.com, ldufour@linux.ibm.com, catalin.marinas@arm.com, will@kernel.org, arnd@arndb.de, tglx@linutronix.de, mingo@redhat.com, dave.hansen@linux.intel.com, x86@kernel.org, peterx@redhat.com, david@redhat.com, axboe@kernel.dk, mcgrof@kernel.org, masahiroy@kernel.org, nathan@kernel.org, dennis@kernel.org, tj@kernel.org, muchun.song@linux.dev, rppt@kernel.org, paulmck@kernel.org, pasha.tatashin@soleen.com, yosryahmed@google.com, yuzhao@google.com, dhowells@redhat.com, hughd@google.com, andreyknvl@gmail.com, keescook@chromium.org, ndesaulniers@google.com, gregkh@linuxfoundation.org, ebiggers@google.com, ytcoode@gmail.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, bristot@redhat.com, vschneid@redhat.com, cl@linux.com, penberg@kernel.org, iamjoonsoo.kim@lge.com, 42.hyeyoo@gmail.com, glider@google.com, elver@google.com, dvyukov@google.com, shakeelb@google.com, songmuchun@bytedance.com, jbaron@akamai.com, rientjes@google.com, minchan@google.com, kaleshsingh@google.com, kernel-team@android.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, kasan-dev@googlegroups.com, cgroups@vger.kernel.org Subject: Re: [PATCH 00/40] Memory allocation profiling Message-ID: References: <20230501165450.15352-1-surenb@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Precedence: bulk List-ID: On Thu 04-05-23 08:08:13, Suren Baghdasaryan wrote: > On Thu, May 4, 2023 at 2:07 AM Michal Hocko wrote: [...] > > e.g. is it really interesting to know that there is a likely memory > > leak in seq_file proper doing and allocation? No as it is the specific > > implementation using seq_file that is leaking most likely. There are > > other examples like that See? > > Yes, I see that. One level tracking does not provide all the > information needed to track such issues. Something more informative > would cost more. That's why our proposal is to have a light-weight > mechanism to get a high level picture and then be able to zoom into a > specific area using context capture. If you have ideas to improve > this, I'm open to suggestions. Well, I think that a more scalable approach would be to not track in callers but in the allocator itself. The full stack trace might not be all that important or interesting and maybe even increase the overall overhead but a partial one with a configurable depth would sound more interesting to me. A per cache hastable indexed by stack trace reference and extending slab metadata to store the reference for kfree path won't be free but the overhead might be just acceptable. If the stack unwinding is really too expensive for tracking another option would be to add code tags dynamically to the compiled kernel without any actual code changes. I can imagine the tracing infrastructure could be used for that or maybe even consider compiler plugins to inject code for functions marked as allocators. So the kernel could be instrumented even without eny userspace tooling required by users directly. -- Michal Hocko SUSE Labs From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michal Hocko Subject: Re: [PATCH 00/40] Memory allocation profiling Date: Sun, 7 May 2023 12:27:18 +0200 Message-ID: References: <20230501165450.15352-1-surenb@google.com> Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable Return-path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1683455239; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oav2A28KVKDaotNibkAokXcYPyLTU0ijj1klZnIOiRI=; b=c+QuWE6EU4wWKhietxfwN/3l7yAy0K35os01H7NfRFE82alr8YUVFXQqf5mweH8wlzKf+k lu0AkCEfmwJ2VDmkoLh8SVCJbT4jzXXTf9sUf9reepl+WLoX/HYO+d0q2t7pYqKTvW4ACE 6vuKCAGkjFOK3NT53IiVKVs+9jUJHkc= Content-Disposition: inline In-Reply-To: List-ID: Content-Type: text/plain; charset="windows-1252" To: Suren Baghdasaryan Cc: akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org, kent.overstreet-fxUVXftIFDnyG1zEObXtfA@public.gmane.org, vbabka-AlSwsSmVLrQ@public.gmane.org, hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org, roman.gushchin-fxUVXftIFDnyG1zEObXtfA@public.gmane.org, mgorman-l3A5Bk7waGM@public.gmane.org, dave-h16yJtLeMjHk1uMJSBkQmQ@public.gmane.org, willy-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org, liam.howlett-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org, corbet-T1hC0tSOHrs@public.gmane.org, void-gq6j2QGBifHby3iVrkZq2A@public.gmane.org, peterz-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org, juri.lelli-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org, ldufour-tEXmvtCZX7AybS5Ee8rs3A@public.gmane.org, catalin.marinas-5wv7dgnIgG8@public.gmane.org, will-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org, arnd-r2nGTMty4D4@public.gmane.org, tglx-hfZtesqFncYOwBW4kG4KsQ@public.gmane.org, mingo-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org, dave.hansen-VuQAYsv1563Yd54FQh9/CA@public.gmane.org, x86-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org, peterx-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org, david-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org, axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org, mcgrof-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org, masahiroy-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org, nathan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org, dennis-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org, tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org, muchun.song-fxUVXftIFDnyG1zEObXtfA@public.gmane.org, rppt-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org, paulmck-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org, pasha.tatashin-2EmBfe737+LQT0dZR+AlfA@public.gmane.org, yosryahmed-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org, yuzhao-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org, dhowells-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org, hughd-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org, andreyknvl-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org, keescook@chr On Thu 04-05-23 08:08:13, Suren Baghdasaryan wrote: > On Thu, May 4, 2023 at 2:07=E2=80=AFAM Michal Hocko wrote: [...] > > e.g. is it really interesting to know that there is a likely memory > > leak in seq_file proper doing and allocation? No as it is the specific > > implementation using seq_file that is leaking most likely. There are > > other examples like that See? >=20 > Yes, I see that. One level tracking does not provide all the > information needed to track such issues. Something more informative > would cost more. That's why our proposal is to have a light-weight > mechanism to get a high level picture and then be able to zoom into a > specific area using context capture. If you have ideas to improve > this, I'm open to suggestions. Well, I think that a more scalable approach would be to not track in callers but in the allocator itself. The full stack trace might not be all that important or interesting and maybe even increase the overall overhead but a partial one with a configurable depth would sound more interesting to me. A per cache hastable indexed by stack trace reference and extending slab metadata to store the reference for kfree path won't be free but the overhead might be just acceptable. If the stack unwinding is really too expensive for tracking another option would be to add code tags dynamically to the compiled kernel without any actual code changes. I can imagine the tracing infrastructure could be used for that or maybe even consider compiler plugins to inject code for functions marked as allocators. So the kernel could be instrumented even without eny userspace tooling required by users directly. --=20 Michal Hocko SUSE Labs