From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_2 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABDF2C4338F for ; Fri, 30 Jul 2021 20:03:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2808160F42 for ; Fri, 30 Jul 2021 20:03:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2808160F42 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=goodmis.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 8FE9A8D0001; Fri, 30 Jul 2021 16:03:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8877D6B0036; Fri, 30 Jul 2021 16:03:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 74F4A8D0001; Fri, 30 Jul 2021 16:03:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0215.hostedemail.com [216.40.44.215]) by kanga.kvack.org (Postfix) with ESMTP id 58D626B0033 for ; Fri, 30 Jul 2021 16:03:40 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id B8ABB27754 for ; Fri, 30 Jul 2021 20:03:39 +0000 (UTC) X-FDA: 78420329358.15.572530B Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id 34332F00709A for ; Fri, 30 Jul 2021 20:03:39 +0000 (UTC) Received: from oasis.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D663E60F46; Fri, 30 Jul 2021 20:03:26 +0000 (UTC) Date: Fri, 30 Jul 2021 16:03:19 -0400 From: Steven Rostedt To: Gang Li Cc: Axel Rasmussen , Ingo Molnar , Andrew Morton , Vlastimil Babka , LKML , Linux MM Subject: Re: [PATCH 3/3] mm: mmap_lock: add ip to mmap_lock tracepoints Message-ID: <20210730160319.6dfeaf7a@oasis.local.home> In-Reply-To: <585f936d-9d27-a481-f590-bd07f98f34de@bytedance.com> References: <20210729092853.38242-1-ligang.bdlg@bytedance.com> <585f936d-9d27-a481-f590-bd07f98f34de@bytedance.com> X-Mailer: Claws Mail 3.17.3 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 34332F00709A Authentication-Results: imf11.hostedemail.com; dkim=none; spf=pass (imf11.hostedemail.com: domain of "SRS0=yFb6=MW=goodmis.org=rostedt@kernel.org" designates 198.145.29.99 as permitted sender) smtp.mailfrom="SRS0=yFb6=MW=goodmis.org=rostedt@kernel.org"; dmarc=none X-Stat-Signature: 9daom4j8nyrb7hiz34zfdmjab9kmjnr9 X-HE-Tag: 1627675419-834008 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, 30 Jul 2021 13:32:12 +0800 Gang Li wrote: > Thanks! I have tried your suggestion. They are great, especially > synthetic-events. > > If don't print ip per event, we can only guess which one cause the > contention by "hitcount". > > > > (https://www.kernel.org/doc/html/latest/trace/histogram.html#synthetic-events) > > But it seems that they only support histogram, can I print the > synthetic-events args per event in /sys/kernel/debug/tracing/trace > like other events? I haven't found that in kernel doc. Yes, synthetic events are just like normal events, and have triggers, stack traces, and do pretty much anything that another event can do. I'm just finishing up a libtracfs called tracefs_sql() (hopefully posting it today), that allows you to create a synthetic event via an SQL statement. But I don't think this is what you are looking for. What about using function tracing? Because the tracepoint is called from __mmap_lock* helper functions that function tracer can see, you can just do the following: # trace-cmd start -e mmap_lock -p function -l '__mmap_lock_*' # trace-cmd show [..] trace-cmd-1840 [006] .... 194.576801: __mmap_lock_do_trace_start_locking <-do_user_addr_fault trace-cmd-1840 [006] ...1 194.576805: mmap_lock_start_locking: mm=000000006515cb1f memcg_path=/user.slice/user-0.slice/session-2.scope write=false trace-cmd-1840 [006] .... 194.576806: __mmap_lock_do_trace_acquire_returned <-do_user_addr_fault trace-cmd-1840 [006] ...1 194.576807: mmap_lock_acquire_returned: mm=000000006515cb1f memcg_path=/user.slice/user-0.slice/session-2.scope write=false success=true trace-cmd-1840 [006] .... 194.576811: __mmap_lock_do_trace_released <-do_user_addr_fault trace-cmd-1840 [006] ...1 194.576812: mmap_lock_released: mm=000000006515cb1f memcg_path=/user.slice/user-0.slice/session-2.scope write=false trace-cmd-1840 [006] .... 194.576815: __mmap_lock_do_trace_start_locking <-do_user_addr_fault trace-cmd-1840 [006] ...1 194.576816: mmap_lock_start_locking: mm=000000006515cb1f memcg_path=/user.slice/user-0.slice/session-2.scope write=false trace-cmd-1840 [006] .... 194.576816: __mmap_lock_do_trace_acquire_returned <-do_user_addr_fault trace-cmd-1840 [006] ...1 194.576817: mmap_lock_acquire_returned: mm=000000006515cb1f memcg_path=/user.slice/user-0.slice/session-2.scope write=false success=true trace-cmd-1840 [006] .... 194.576820: __mmap_lock_do_trace_released <-do_user_addr_fault trace-cmd-1840 [006] ...1 194.576821: mmap_lock_released: mm=000000006515cb1f memcg_path=/user.slice/user-0.slice/session-2.scope write=false This looks exactly like the robots you are looking for. -- Steve