From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.6 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66BF9C433E1 for ; Fri, 7 Aug 2020 21:30:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3879A22D03 for ; Fri, 7 Aug 2020 21:30:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="FbP0RxUp" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727092AbgHGVar (ORCPT ); Fri, 7 Aug 2020 17:30:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53486 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727108AbgHGV3l (ORCPT ); Fri, 7 Aug 2020 17:29:41 -0400 Received: from mail-qk1-x749.google.com (mail-qk1-x749.google.com [IPv6:2607:f8b0:4864:20::749]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B61BC061A27 for ; Fri, 7 Aug 2020 14:29:41 -0700 (PDT) Received: by mail-qk1-x749.google.com with SMTP id z1so2377047qkg.23 for ; Fri, 07 Aug 2020 14:29:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Xwhl6oTAS7prpbXCRZdZshg5xz57JrEbZ056Cu9NiLQ=; b=FbP0RxUpuIXc9TPHeycYqgrsWHy+edJZOunO3xzH9xgd3XMesTsydOHKMec+qxUJFA FlXnrJi21QCaDcqgirJGJT0eBkQ0v3Vq6LIzdTyTYf6I4CV8iPofNCGuawph3tfsMnfh rVST9SGjyClH8eIIJqualzLP+dJgXyLWWuF3VRTq6ejnpWajSiynlSZz5Y285bh+tnSW ZxEvaLhRHfrOvoRGHyx1uzFMrik/lFM8Yi0f5EOQG7sM80OuBBVAKF1eI2Nt5evM5J/V ZSfJtPKRCyz4Ary7BmVgPJgTRfuuh+/bHoAJlAFzxb/0O1Zu489t0ofM9yH0JSH6UL05 9KPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Xwhl6oTAS7prpbXCRZdZshg5xz57JrEbZ056Cu9NiLQ=; b=KBpZiK2xUCsWiITid8cXziPdk5OUFVrrlVoTFraILQ+2MT/2tddwliLUOgYdgu/bVT Us+/1FpzyuZMIyJ5hrmrTcc3fcGqGAruu73+DWyxbH/136Ey2/MW9LiQpp7PpSDBAAnp 75y7lmZS95mzkEc3Z8EO46TtRZiBmFPsnhO9GPU04bMxaCAXIL4bzubTMx4JI+oNqOvQ nXO4N7MnUzgtbT9YmiroaYz6Zt50VN9T313Gw1RcDciNn9KUhj8XE/isQmBv7ysCqRSb uNlWDtaNDj+IE/fMFrGgVCtvDpxwmfkriTmIHSUgO+wIDoxpkTtE54uK76aQGYMv/Alg tr0g== X-Gm-Message-State: AOAM531bs51YffaZKM6oIOlxjKyr+WpZH/z2VYHWX1wP0cjZAl+iT2nQ jGAVokvBlbnW9iY6diexh2ezdzejMIsB1uuifrFp9/uFZ6yMQjOhzkiTzMdWQGMUi1NOSTUbUQ2 KrnJ20CmfWzFiNMMtuQQiSsM2SPPfLTAldkONQdww3kXaSBxW/UeL7FGOZT6iDL/tB3d7JtU= X-Google-Smtp-Source: ABdhPJyLLX7R17IOCOR3CY0ZoHgSxM9yzJvgcbYl6HaF+KCt80M13Jn4ki0ErUlYpRFS5LWuyydOwJIuc87a X-Received: by 2002:ad4:5502:: with SMTP id az2mr16402089qvb.148.1596835779896; Fri, 07 Aug 2020 14:29:39 -0700 (PDT) Date: Fri, 7 Aug 2020 14:29:12 -0700 In-Reply-To: <20200807212916.2883031-1-jwadams@google.com> Message-Id: <20200807212916.2883031-4-jwadams@google.com> Mime-Version: 1.0 References: <20200807212916.2883031-1-jwadams@google.com> X-Mailer: git-send-email 2.28.0.236.gb10cc79966-goog Subject: [RFC PATCH 3/7] core/metricfs: metric for kernel warnings From: Jonathan Adams To: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org Cc: netdev@vger.kernel.org, kvm@vger.kernel.org, Paolo Bonzini , Greg KH , Jim Mattson , David Rientjes , Jonathan Adams Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Count kernel warnings by function name of the caller. Each time WARN() is called, which includes WARN_ON(), increment a counter in a 256-entry hash table. The table key is the entry point of the calling function, which is found using kallsyms. We store the name of the function in the table (because it may be a module address); reporting the metric just walks the table and prints the values. The "warnings" metric is cumulative. Signed-off-by: Jonathan Adams --- jwadams@google.com: rebased to 5.8-rc6, removed google-isms, added lockdep_assert_held(), NMI handling, ..._unknown*_counts and locking in warn_tbl_fn(); renamed warn_metric... to warn_tbl... The original work was done in 2012 by an engineer no longer at Google. --- kernel/panic.c | 131 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 131 insertions(+) diff --git a/kernel/panic.c b/kernel/panic.c index e2157ca387c8..c019b41ab387 100644 --- a/kernel/panic.c +++ b/kernel/panic.c @@ -31,6 +31,9 @@ #include #include #include +#include +#include +#include #include #define PANIC_TIMER_STEP 100 @@ -568,6 +571,133 @@ void oops_exit(void) kmsg_dump(KMSG_DUMP_OOPS); } +#ifdef CONFIG_METRICFS + +/* + * Hash table from function address to count of WARNs called within that + * function. + * So far this is an add-only hash table (ie, entries never removed), so some + * simplifying assumptions are made. + */ +#define WARN_TBL_BITS (8) +#define WARN_TBL_SIZE (1<= 0) + warn_tbl[entry].count++; + else + warn_tbl_unknown_count++; + + spin_unlock_irqrestore(&warn_tbl_lock, flags); +} + +/* + * Export the hash table to metricfs. + */ +static void warn_tbl_fn(struct metric_emitter *e) +{ + int i; + unsigned long flags; + int unknown_count = READ_ONCE(warn_tbl_unknown_count) + + atomic_read(&warn_tbl_unknown_nmi_count) + + atomic_read(&warn_tbl_unknown_lookup_count); + + if (unknown_count != 0) + METRIC_EMIT_INT(e, unknown_count, "(unknown)", NULL); + + spin_lock_irqsave(&warn_tbl_lock, flags); + for (i = 0; i < WARN_TBL_SIZE; i++) { + unsigned long fn = (unsigned long)warn_tbl[i].function; + const char *function_name = warn_tbl[i].function_name; + int count = warn_tbl[i].count; + + if (!fn) + continue; + + // function_name[] is constant once function is non-NULL + spin_unlock_irqrestore(&warn_tbl_lock, flags); + METRIC_EMIT_INT(e, count, function_name, NULL); + spin_lock_irqsave(&warn_tbl_lock, flags); + } + spin_unlock_irqrestore(&warn_tbl_lock, flags); +} +METRIC_EXPORT_COUNTER(warnings, "Count of calls to WARN().", + "function", NULL, warn_tbl_fn); + +static int __init metricfs_panic_init(void) +{ + metric_init_warnings(NULL); + return 0; +} +late_initcall(metricfs_panic_init); + +#else /* CONFIG_METRICFS */ +inline void tbl_increment(void *caller) {} +#endif + struct warn_args { const char *fmt; va_list args; @@ -576,6 +706,7 @@ struct warn_args { void __warn(const char *file, int line, void *caller, unsigned taint, struct pt_regs *regs, struct warn_args *args) { + tbl_increment(caller); disable_trace_on_warning(); if (file) -- 2.28.0.236.gb10cc79966-goog