From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BED02C433DF for ; Thu, 28 May 2020 23:53:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6BF7B206E2 for ; Thu, 28 May 2020 23:53:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LaoKiP+c" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6BF7B206E2 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1BF76800B9; Thu, 28 May 2020 19:53:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1957080010; Thu, 28 May 2020 19:53:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 05C00800B9; Thu, 28 May 2020 19:53:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0207.hostedemail.com [216.40.44.207]) by kanga.kvack.org (Postfix) with ESMTP id DFBDD80010 for ; Thu, 28 May 2020 19:53:04 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id ABFE62C8B for ; Thu, 28 May 2020 23:53:04 +0000 (UTC) X-FDA: 76867781088.11.cream27_8f37fd8cd7c2e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin11.hostedemail.com (Postfix) with ESMTP id 87AF6180F8B80 for ; Thu, 28 May 2020 23:53:04 +0000 (UTC) X-HE-Tag: cream27_8f37fd8cd7c2e X-Filterd-Recvd-Size: 7599 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Thu, 28 May 2020 23:53:04 +0000 (UTC) Received: by mail-yb1-f201.google.com with SMTP id d69so884495ybc.22 for ; Thu, 28 May 2020 16:53:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:message-id:mime-version:subject:from:to:cc; bh=2yUKX5hqzt1oHPQ4zNvGMcIYmUYCvNSNOVGMDvCOr3Q=; b=LaoKiP+cg9anMNFNg9Kd4pZYQAtw0FOInUmBUGQu2wKaKQMBIhjiAIspR0+jNfFIUR Wy/vpw0/UipeJjuws5qV8Ox9SMPtVfFnRuwoOi0V8A+Wocp5oYNImICPhZelh6PWwS4/ So2FoMAKYaJb/PO12JSLeHAPj6EbyM/HTreSXbWscfXxhtBb9vvaPGTHXg1Hk7AqxaVp //w3S/CyuQKBwFdks95jIwGGVJLHBtOmyW1cF22iv8nXTF4fUV8kosw00ToCNEDBxtQ7 HQ/W+gonCfXZjojOvbE9fmMDE18lM57VoyhIBV2QG//7DAKMV0qfrn4KKYJ15WbdM4u4 PiiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=2yUKX5hqzt1oHPQ4zNvGMcIYmUYCvNSNOVGMDvCOr3Q=; b=DxRTKZ+0QvTHFRfvDOYh4eUuPh/WDAwRZJo2xIYTCCOd3xNGaUgrTvJXM5/+pvX9H+ EK8FpRFWJhxjnepnHAteG0/YLRqV5M0+55EcRBUOo1yoNat2iRKTb10wMt9TxnpZSTNe w3eRpvMTq2F2WwLhgg8qg0kMtsDfVKkbhitXy2QqI2nSDbGHqufihJeWz8gUNDXQDBPS jbjlX0M+G4F24e4748jgH++4GDY76Kw1cDJSkY/QxcORRh3rJLLVccPau1zOmC48U6Ms 5Y5ft6xQG8r0/xcVkgc2NTfOp50zmtEwbL6gzTvOmiguamVcitfrWH9VR1eo1Z29kQ68 uysA== X-Gm-Message-State: AOAM533LWBRn2KWw0vUIVs+uYYhENI5L75Qi7bfAfuPuWwWvsxBjC46I reXKHISdHEyc/Xm+Ii6TlfLZeCNcbs9nZIvq0wO+ X-Google-Smtp-Source: ABdhPJxTDy0JwP7FWRktZyHchnosXxFw+yM7WVZbQCJl7DvaCtN9Tv/xctTb9glc5ZOrQcq9e7nC5UJyNd/5xZElQ7UW X-Received: by 2002:a25:e7cf:: with SMTP id e198mr820935ybh.188.1590709983105; Thu, 28 May 2020 16:53:03 -0700 (PDT) Date: Thu, 28 May 2020 16:53:00 -0700 Message-Id: <20200528235300.77135-1-axelrasmussen@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652-goog Subject: [PATCH v2 4/7] mmap_lock: allocate histogram (if enabled) in mm_init From: Axel Rasmussen To: Andrew Morton , David Rientjes , Davidlohr Bueso , Ingo Molnar , Ingo Molnar , Jerome Glisse , Laurent Dufour , "Liam R . Howlett" , Matthew Wilcox , Michel Lespinasse , Peter Zijlstra , Vlastimil Babka , Will Deacon Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, AKASHI Takahiro , Aleksa Sarai , Alexander Potapenko , Alexey Dobriyan , Al Viro , Andrei Vagin , Ard Biesheuvel , Brendan Higgins , chenqiwu , Christian Brauner , Christian Kellner , Corentin Labbe , Daniel Jordan , Dan Williams , David Gow , "David S. Miller" , "Dmitry V. Levin" , "Eric W. Biederman" , Eugene Syromiatnikov , Jamie Liu , Jason Gunthorpe , John Garry , John Hubbard , Jonathan Adams , Junaid Shahid , Kees Cook , "Kirill A. Shutemov" , Konstantin Khlebnikov , Krzysztof Kozlowski , Mark Rutland , Masahiro Yamada , Masami Hiramatsu , Mathieu Desnoyers , Michal Hocko , Mikhail Zaslonko , Petr Mladek , Ralph Campbell , Randy Dunlap , Roman Gushchin , Shakeel Butt , Steven Rostedt , Tal Gilboa , Thomas Gleixner , "=?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?=" , Vincenzo Frascino , Yang Shi , Yu Zhao , Axel Rasmussen Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 87AF6180F8B80 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In effect, allocate a histogram for each process. On fork, initialize a new empty histogram (don't keep recorded samples from the parent). Signed-off-by: Axel Rasmussen --- kernel/fork.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/kernel/fork.c b/kernel/fork.c index e702e84897fa..afb036d81a73 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -94,6 +94,7 @@ #include #include #include +#include #include #include @@ -975,6 +976,58 @@ __setup("coredump_filter=", coredump_filter_setup); #include +#ifdef CONFIG_MMAP_LOCK_HISTOGRAMS +static const u64 mmap_lock_contention_buckets[] = { + 1, /* 0ns - uncontended acquisitions */ + 250, /* 250 ns */ + 375, /* 375 ns */ + 500, /* 500 ns */ + 1000, /* 1 us */ + 10000, /* 10 us */ + 100000, /* 100 us */ + 500000, /* 500 us */ + 1000000, /* 1 ms */ + 5000000, /* 5 ms */ + 10000000, /* 10 ms */ + 50000000, /* 50 ms */ + 100000000, /* 100 ms */ + 500000000, /* 500 ms */ + 1000000000, /* 1 s */ + 5000000000UL, /* 5 s */ + 10000000000UL, /* 10 s */ + ~0 /* > 10s */ +}; +#endif + +static void mm_init_histograms(struct mm_struct *mm) +{ +#ifdef CONFIG_MMAP_LOCK_HISTOGRAMS + mm->mmap_lock_contention = + kmalloc(sizeof(struct histogram_rcu), GFP_KERNEL); + if (unlikely(!mm->mmap_lock_contention)) + return; + + if (unlikely(histogram_init_rcu( + mm->mmap_lock_contention, mmap_lock_contention_buckets, + ARRAY_SIZE(mmap_lock_contention_buckets)))) { + kfree(mm->mmap_lock_contention); + mm->mmap_lock_contention = NULL; + } +#endif +} + +static void mm_free_histograms(struct mm_struct *mm) +{ +#ifdef CONFIG_MMAP_LOCK_HISTOGRAMS + if (unlikely(!mm->mmap_lock_contention)) + return; + + histogram_destroy_rcu(mm->mmap_lock_contention); + kfree(mm->mmap_lock_contention); + mm->mmap_lock_contention = NULL; +#endif +} + static void mm_init_aio(struct mm_struct *mm) { #ifdef CONFIG_AIO @@ -1015,6 +1068,7 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p, atomic_set(&mm->mm_users, 1); atomic_set(&mm->mm_count, 1); mmap_init_lock(mm); + mm_init_histograms(mm); INIT_LIST_HEAD(&mm->mmlist); mm->core_state = NULL; mm_pgtables_bytes_init(mm); @@ -1078,6 +1132,7 @@ static inline void __mmput(struct mm_struct *mm) { VM_BUG_ON(atomic_read(&mm->mm_users)); + mm_free_histograms(mm); uprobe_clear_state(mm); exit_aio(mm); ksm_exit(mm); -- 2.27.0.rc0.183.gde8f92d652-goog