From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.4 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C734C43457 for ; Fri, 9 Oct 2020 22:35:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1210D22283 for ; Fri, 9 Oct 2020 22:35:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="eyqAmq06" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391207AbgJIWfO (ORCPT ); Fri, 9 Oct 2020 18:35:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44248 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388159AbgJIWfO (ORCPT ); Fri, 9 Oct 2020 18:35:14 -0400 Received: from mail-yb1-xb41.google.com (mail-yb1-xb41.google.com [IPv6:2607:f8b0:4864:20::b41]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C069C0613D2 for ; Fri, 9 Oct 2020 15:35:14 -0700 (PDT) Received: by mail-yb1-xb41.google.com with SMTP id 67so8386966ybt.6 for ; Fri, 09 Oct 2020 15:35:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=NP7EwU7Wgi3GKoYOtUC4dl2ophm6HoyRW7fEMUmtnzE=; b=eyqAmq06cVJmWm0yKH3E9VMKkdcgK7AyWyd8HtlTQFmYC7LBk5vqu+65udfdzcbc0o xKoChhK97kDIGyS7uDQdmmBmsCwzGM/yUwm2tW/MjiNsMn4vaw3N6hNtqg6NTMLZwhTx gP2aiGWkl4IyhBcuRl4kfsbflrfzSbO+xE+GVdm4EJjb0ZECJfrm7U6F5FU42BT9rlQs 82NrHN2FutgMXRwfD4J9Bnuw33PjQp2Xi3HaQnKYJUOjNbawMeUYOKdHW/almm4UIpNn kXVlVkRW/1pt0cRxB2al7trPI6n+rmfIZ5scYeJM/JMtR+w67Kkp/Xu8WFqq5oX4F1Q8 Wf/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=NP7EwU7Wgi3GKoYOtUC4dl2ophm6HoyRW7fEMUmtnzE=; b=ogexVnOabbVngsEPbeXPk3wCZH/A5bnOvmeJXsyaXBn1kfZWN6TG5QjOOcAyeflESk InmA9CvWb5RfYZrWPVeF94GhSNoxMPIvTBX6b7d9ztc8KNW41M8V+SwY5/jDwnsfMq0B vwsJbEA9i991tBrwOhSICEOp+0MiD76Y5LTyl8q/iLaOElbodfJzHqPCeRxJluiqKQbk I3v21eFV4t/coRyA36kM9I3MdsGRvD0eoN+Zcp6DskvcWbQf2615P7RP4vG9w+JgA9jP BuN+AjgqGsRdnD0Or6IKCTGeXfbEJXF5MO2YaSwzaf+FLDvjWfdltquDiXQqMmbyW9pc P0mw== X-Gm-Message-State: AOAM533p9uRl1d5S3fRtsSt3zFGo3J6391fuquQl8ZOtArIHi+OnY0r8 Ly2RnqBI1B7lEpBFI5fVbtrVNzBN9gxdnXYD18b2fg== X-Google-Smtp-Source: ABdhPJxzMYgtqhmZms+YU3oGvBBYXC5cxhBblMlch2sRtV/6PrjBZdx9b38bkVO0mi4OljexEeEtF7jucz7wXQyAWGc= X-Received: by 2002:a5b:d51:: with SMTP id f17mr18525307ybr.298.1602282913113; Fri, 09 Oct 2020 15:35:13 -0700 (PDT) MIME-Version: 1.0 References: <20201009220524.485102-1-axelrasmussen@google.com> <20201009220524.485102-3-axelrasmussen@google.com> In-Reply-To: <20201009220524.485102-3-axelrasmussen@google.com> From: Michel Lespinasse Date: Fri, 9 Oct 2020 15:35:00 -0700 Message-ID: Subject: Re: [PATCH v3 2/2] mmap_lock: add tracepoints around lock acquisition To: Axel Rasmussen Cc: Steven Rostedt , Ingo Molnar , Andrew Morton , Vlastimil Babka , Daniel Jordan , Laurent Dufour , Jann Horn , Chinwen Chang , Yafang Shao , LKML , linux-mm Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Oct 9, 2020 at 3:05 PM Axel Rasmussen wrote: > The goal of these tracepoints is to be able to debug lock contention > issues. This lock is acquired on most (all?) mmap / munmap / page fault > operations, so a multi-threaded process which does a lot of these can > experience significant contention. > > We trace just before we start acquisition, when the acquisition returns > (whether it succeeded or not), and when the lock is released (or > downgraded). The events are broken out by lock type (read / write). > > The events are also broken out by memcg path. For container-based > workloads, users often think of several processes in a memcg as a single > logical "task", so collecting statistics at this level is useful. > > The end goal is to get latency information. This isn't directly included > in the trace events. Instead, users are expected to compute the time > between "start locking" and "acquire returned", using e.g. synthetic > events or BPF. The benefit we get from this is simpler code. > > Because we use tracepoint_enabled() to decide whether or not to trace, > this patch has effectively no overhead unless tracepoints are enabled at > runtime. If tracepoints are enabled, there is a performance impact, but > how much depends on exactly what e.g. the BPF program does. > > Signed-off-by: Axel Rasmussen Reviewed-by: Michel Lespinasse Looks good to me, thanks! From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.4 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDF46C433DF for ; Fri, 9 Oct 2020 22:35:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 54F43222C4 for ; Fri, 9 Oct 2020 22:35:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="eyqAmq06" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 54F43222C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9B901940014; Fri, 9 Oct 2020 18:35:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 942E394000C; Fri, 9 Oct 2020 18:35:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7E264940014; Fri, 9 Oct 2020 18:35:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0114.hostedemail.com [216.40.44.114]) by kanga.kvack.org (Postfix) with ESMTP id 3FA6694000C for ; Fri, 9 Oct 2020 18:35:15 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id C7862362A for ; Fri, 9 Oct 2020 22:35:14 +0000 (UTC) X-FDA: 77353844148.30.war76_3f055dc271e4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id 96E1C180B3C8B for ; Fri, 9 Oct 2020 22:35:14 +0000 (UTC) X-HE-Tag: war76_3f055dc271e4 X-Filterd-Recvd-Size: 4497 Received: from mail-yb1-f196.google.com (mail-yb1-f196.google.com [209.85.219.196]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Fri, 9 Oct 2020 22:35:13 +0000 (UTC) Received: by mail-yb1-f196.google.com with SMTP id x8so8371327ybe.12 for ; Fri, 09 Oct 2020 15:35:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=NP7EwU7Wgi3GKoYOtUC4dl2ophm6HoyRW7fEMUmtnzE=; b=eyqAmq06cVJmWm0yKH3E9VMKkdcgK7AyWyd8HtlTQFmYC7LBk5vqu+65udfdzcbc0o xKoChhK97kDIGyS7uDQdmmBmsCwzGM/yUwm2tW/MjiNsMn4vaw3N6hNtqg6NTMLZwhTx gP2aiGWkl4IyhBcuRl4kfsbflrfzSbO+xE+GVdm4EJjb0ZECJfrm7U6F5FU42BT9rlQs 82NrHN2FutgMXRwfD4J9Bnuw33PjQp2Xi3HaQnKYJUOjNbawMeUYOKdHW/almm4UIpNn kXVlVkRW/1pt0cRxB2al7trPI6n+rmfIZ5scYeJM/JMtR+w67Kkp/Xu8WFqq5oX4F1Q8 Wf/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=NP7EwU7Wgi3GKoYOtUC4dl2ophm6HoyRW7fEMUmtnzE=; b=uEUUTqutDgXmcTJUzBK07oFpIc6zgzeB4LFgIfXvenkxQcPYNtc74QD+t8AekLcruB pRRoF0IbBwsQP38pOptVM9pwQvlZwnO+VfG95gG6P420WjT/J+azVlvGz+FPONJDRL0S 6hn8Xk2dwDgT4+B1TbX/+q/xBw2F2H7eSkpgBcNdKxbsT99VHe8oLiAisT8TqnPXmmwE LivPN7PZWR4FLhqVvmasfPVwooxBx0beR4KdREp4qTE1wmibQsTnKxoCSJ9LeuK7ng5t UA+wqytFaMeckz87nbiEI9Te9DszbH9GOKKwRzS3uGYOb7IwOY/C4QksUYlAdcsTRbA6 fw1w== X-Gm-Message-State: AOAM531t1bxdjJIAYKvzlICA36ScQKFPcb6QEOM2afjmYfRUpl4dM3+f rrud6X4+DUMbYyGWc2mawvz0Ar3wfmAn3m8jXXkauw== X-Google-Smtp-Source: ABdhPJxzMYgtqhmZms+YU3oGvBBYXC5cxhBblMlch2sRtV/6PrjBZdx9b38bkVO0mi4OljexEeEtF7jucz7wXQyAWGc= X-Received: by 2002:a5b:d51:: with SMTP id f17mr18525307ybr.298.1602282913113; Fri, 09 Oct 2020 15:35:13 -0700 (PDT) MIME-Version: 1.0 References: <20201009220524.485102-1-axelrasmussen@google.com> <20201009220524.485102-3-axelrasmussen@google.com> In-Reply-To: <20201009220524.485102-3-axelrasmussen@google.com> From: Michel Lespinasse Date: Fri, 9 Oct 2020 15:35:00 -0700 Message-ID: Subject: Re: [PATCH v3 2/2] mmap_lock: add tracepoints around lock acquisition To: Axel Rasmussen Cc: Steven Rostedt , Ingo Molnar , Andrew Morton , Vlastimil Babka , Daniel Jordan , Laurent Dufour , Jann Horn , Chinwen Chang , Yafang Shao , LKML , linux-mm Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Oct 9, 2020 at 3:05 PM Axel Rasmussen wrote: > The goal of these tracepoints is to be able to debug lock contention > issues. This lock is acquired on most (all?) mmap / munmap / page fault > operations, so a multi-threaded process which does a lot of these can > experience significant contention. > > We trace just before we start acquisition, when the acquisition returns > (whether it succeeded or not), and when the lock is released (or > downgraded). The events are broken out by lock type (read / write). > > The events are also broken out by memcg path. For container-based > workloads, users often think of several processes in a memcg as a single > logical "task", so collecting statistics at this level is useful. > > The end goal is to get latency information. This isn't directly included > in the trace events. Instead, users are expected to compute the time > between "start locking" and "acquire returned", using e.g. synthetic > events or BPF. The benefit we get from this is simpler code. > > Because we use tracepoint_enabled() to decide whether or not to trace, > this patch has effectively no overhead unless tracepoints are enabled at > runtime. If tracepoints are enabled, there is a performance impact, but > how much depends on exactly what e.g. the BPF program does. > > Signed-off-by: Axel Rasmussen Reviewed-by: Michel Lespinasse Looks good to me, thanks!