From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4F63ECAAD5 for ; Mon, 5 Sep 2022 08:58:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238008AbiIEI6u (ORCPT ); Mon, 5 Sep 2022 04:58:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56406 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237957AbiIEI6k (ORCPT ); Mon, 5 Sep 2022 04:58:40 -0400 Received: from mail-yw1-x1130.google.com (mail-yw1-x1130.google.com [IPv6:2607:f8b0:4864:20::1130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC7765006D for ; Mon, 5 Sep 2022 01:58:38 -0700 (PDT) Received: by mail-yw1-x1130.google.com with SMTP id 00721157ae682-3378303138bso64245677b3.9 for ; Mon, 05 Sep 2022 01:58:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date; bh=Mcscb+4yg14w1rBiRtYpdW9I8YuY8tTeWXrlK9wFWrU=; b=Z8sZwX0UrQNHUyE15uUzUljN1G5+XUXWn7ozlWkNQaPS0v+13+PWoolzvMZEH/2/s6 LYS6ZELiyvlMT62CZWUyH+h/p1jXfwfZLcOF83nbGfa1U5bzWw0uJywA1Nq6g4RggQN9 M+azNg9wtAjIuHGBIaj/LBqqtGG4wC1BksRWLrapbeVhca09DWdw9aO+S7lkiOmT419k B7LBrz0EVZ5dFwwSikfSkFiCRf8G7DmN1ptEdy0ZLUUkT5MXom1MAu8JaMJrJbPX6xsQ z/gVg/ZkLNO66Ao9RVZbmyzaUupGI6+7kFaD3U0bHdpOYObc1T0hLgrkbVZFP/rVpApA /zsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date; bh=Mcscb+4yg14w1rBiRtYpdW9I8YuY8tTeWXrlK9wFWrU=; b=jjPYzrmk8hFaoJoFOvQ31KW9SQxkfe+R+Kf+2hXt8nvAshSzBnfi20daX5GKtmJDxA 3gNEvNf0d1o9fihEyPS7l80HwZsgXCyFxmlXMmbFLDonFxsyEUDsjaBxjmqL7rSNkQIl XECtCzTELwepIt0tmRPMPzCIwxyRFeqr+lcQYmYR8gwr71EH2IeTSKTN1Q3QMFAqfwOR o4J6XP+kGg5Cq9gL8WWhjDeixFbyKRGXL4uc1R6Ltuy79q5FYnYyKk1gv1TdQN714p48 1R77DVDdKLYN3bo62DS1HVfrDN4WkCqYy0bGbHGuwc1W2tXHkAdeT+KmKic+t92GrY8E 9R7w== X-Gm-Message-State: ACgBeo0gOdmm252xFem8/wyRX/993BEO4S/wB5Dv2q2bqW096YHMQ9lb 20jLCGXumaY050lxe5HHPYBZexJH+irhNbfq4qz+xg== X-Google-Smtp-Source: AA6agR4vColk4VOoxJ3aA1AZBZ3YZkQTsja6zG5W+nyEUjeTUXHJb0eacFw2AD+E2b0jt8egp4/3QX6cb8SK35bLOpk= X-Received: by 2002:a81:bb41:0:b0:328:fd1b:5713 with SMTP id a1-20020a81bb41000000b00328fd1b5713mr38838381ywl.238.1662368317652; Mon, 05 Sep 2022 01:58:37 -0700 (PDT) MIME-Version: 1.0 References: <20220830214919.53220-1-surenb@google.com> <20220831084230.3ti3vitrzhzsu3fs@moria.home.lan> <20220831101948.f3etturccmp5ovkl@suse.de> <20220831190154.qdlsxfamans3ya5j@moria.home.lan> In-Reply-To: From: Marco Elver Date: Mon, 5 Sep 2022 10:58:01 +0200 Message-ID: Subject: Re: [RFC PATCH 00/30] Code tagging framework and applications To: Michal Hocko Cc: Suren Baghdasaryan , Kent Overstreet , Mel Gorman , Peter Zijlstra , Andrew Morton , Vlastimil Babka , Johannes Weiner , Roman Gushchin , Davidlohr Bueso , Matthew Wilcox , "Liam R. Howlett" , David Vernet , Juri Lelli , Laurent Dufour , Peter Xu , David Hildenbrand , Jens Axboe , mcgrof@kernel.org, masahiroy@kernel.org, nathan@kernel.org, changbin.du@intel.com, ytcoode@gmail.com, Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Benjamin Segall , Daniel Bristot de Oliveira , Valentin Schneider , Christopher Lameter , Pekka Enberg , Joonsoo Kim , 42.hyeyoo@gmail.com, Alexander Potapenko , Dmitry Vyukov , Shakeel Butt , Muchun Song , arnd@arndb.de, jbaron@akamai.com, David Rientjes , Minchan Kim , Kalesh Singh , kernel-team , linux-mm , iommu@lists.linux.dev, kasan-dev@googlegroups.com, io-uring@vger.kernel.org, linux-arch@vger.kernel.org, xen-devel@lists.xenproject.org, linux-bcache@vger.kernel.org, linux-modules@vger.kernel.org, LKML Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 5 Sept 2022 at 10:12, Michal Hocko wrote: > On Sun 04-09-22 18:32:58, Suren Baghdasaryan wrote: > > On Thu, Sep 1, 2022 at 12:15 PM Michal Hocko wrote: > [...] > > > Yes, tracking back the call trace would be really needed. The question > > > is whether this is really prohibitively expensive. How much overhead are > > > we talking about? There is no free lunch here, really. You either have > > > the overhead during runtime when the feature is used or on the source > > > code level for all the future development (with a maze of macros and > > > wrappers). > > > > As promised, I profiled a simple code that repeatedly makes 10 > > allocations/frees in a loop and measured overheads of code tagging, > > call stack capturing and tracing+BPF for page and slab allocations. > > Summary: > > > > Page allocations (overheads are compared to get_free_pages() duration): > > 6.8% Codetag counter manipulations (__lazy_percpu_counter_add + __alloc_tag_add) > > 8.8% lookup_page_ext > > 1237% call stack capture > > 139% tracepoint with attached empty BPF program > > Yes, I am not surprised that the call stack capturing is really > expensive comparing to the allocator fast path (which is really highly > optimized and I suspect that with 10 allocation/free loop you mostly get > your memory from the pcp lists). Is this overhead still _that_ visible > for somehow less microoptimized workloads which have to take slow paths > as well? > > Also what kind of stack unwinder is configured (I guess ORC)? This is > not my area but from what I remember the unwinder overhead varies > between ORC and FP. > > And just to make it clear. I do realize that an overhead from the stack > unwinding is unavoidable. And code tagging would logically have lower > overhead as it performs much less work. But the main point is whether > our existing stack unwiding approach is really prohibitively expensive > to be used for debugging purposes on production systems. I might > misremember but I recall people having bigger concerns with page_owner > memory footprint than the actual stack unwinder overhead. This is just to point out that we've also been looking at cheaper collection of the stack trace (for KASAN and other sanitizers). The cheapest way to unwind the stack would be a system with "shadow call stack" enabled. With compiler support it's available on arm64, see CONFIG_SHADOW_CALL_STACK. For x86 the hope is that at one point the kernel will support CET, which newer Intel and AMD CPUs support. Collecting the call stack would then be a simple memcpy.