From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.1 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55A24C433E4 for ; Wed, 15 Jul 2020 02:05:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2E4CF20663 for ; Wed, 15 Jul 2020 02:05:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="MBfmujPe" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728207AbgGOCFf (ORCPT ); Tue, 14 Jul 2020 22:05:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49640 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726356AbgGOCFd (ORCPT ); Tue, 14 Jul 2020 22:05:33 -0400 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BA235C061755 for ; Tue, 14 Jul 2020 19:05:33 -0700 (PDT) Received: by mail-pj1-x1043.google.com with SMTP id k71so1297701pje.0 for ; Tue, 14 Jul 2020 19:05:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=hMCjz+92Go61XkKZ2KMP8Y2GkwrHTA5+Bwzj3gXnH/0=; b=MBfmujPeyn0IdrSwOHGVo9a3VpMNRn6TeZf7/RFd0B0J6XD9VPV0WJMHnmgGXrF3zJ a5cmGVYTCxM6wTojk8M78+v6KQa+5illqo07dzRaUED0a27GxkDvtLTf1Jy8EV3fPmKI uoD4X/LECFe7OZQbLUC4qeBmYgpOkOoESsQpf4kSLdesuSmOLwvUIhnZzk06XVuajbBZ 4XXE45D+qYtwLmi17ipzwzDkCGBFBWaQkgC88SvkOFSMphMWklW1sXp1o6iKenfVJDpj ciclauAccDb4jB166FCuZSiYxDKciO36aOfGo72e2ZOVhKXyy6ZfQZwMVumPlJXqi4ep nEAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=hMCjz+92Go61XkKZ2KMP8Y2GkwrHTA5+Bwzj3gXnH/0=; b=KdTczizfP2yQWr/SkDszpJ1adATjHjX7NCM6yG9allnVtxQA20nI6RwQDWX6GcVpJE DhGL3gfHVcGbprHwF2aqi2V43ynUVPK/o14+RzpkMygJxjMq20r6ia66bFjufXBS/Exj mfmGSMDN7F3lyaIqC8PQvZQEcM20BIpMJf1/ghCrBkH7Q/q7DenrCzex7+zYDj0i/YxD FBARisCfw42QlVKQ+pdqaYwyAHpJGd+3ZK0s0OtYNkBC+q+hpqR7d/gpl9t7USr+rBmA 4eOvVXSJuqzZMQI5Z65ijMuiYKfAu69pkWD6LdAnLpEm9gJjjfGs0NzyS8iuyK35z0zI e03A== X-Gm-Message-State: AOAM5322ScYP729f9qXw6FY7hjh99Z9swfDVdyCrJmPnPf4M0w6xv99T IIz6P8jA6htbPAdCMp15JA/Cyg== X-Google-Smtp-Source: ABdhPJweKUBha3SLMEvJ8s2uZ+OFHY4WBgN+V4lhlk4NpJLYQnE4We4H9eumEdWXhqvcTPSS4e/8sQ== X-Received: by 2002:a17:90a:bc98:: with SMTP id x24mr6851187pjr.63.1594778733247; Tue, 14 Jul 2020 19:05:33 -0700 (PDT) Received: from localhost ([2400:8904::f03c:91ff:fe8a:bbe4]) by smtp.gmail.com with ESMTPSA id h7sm374726pfq.15.2020.07.14.19.05.31 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 14 Jul 2020 19:05:32 -0700 (PDT) From: Leo Yan To: Will Deacon , Mark Rutland , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Catalin Marinas , Thomas Gleixner , Paul Cercueil , "Ben Dooks (Codethink)" , "Ahmed S. Darwish" , Adrian Hunter , Kan Liang , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 1/6] sched_clock: Expose struct clock_read_data Date: Wed, 15 Jul 2020 10:05:07 +0800 Message-Id: <20200715020512.20991-2-leo.yan@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200715020512.20991-1-leo.yan@linaro.org> References: <20200715020512.20991-1-leo.yan@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Peter Zijlstra In order to support perf_event_mmap_page::cap_time features, an architecture needs, aside from a userspace readable counter register, to expose the exact clock data so that userspace can convert the counter register into a correct timestamp. Provide struct clock_read_data and two (seqcount) helpers so that architectures (arm64 in specific) can expose the numbers to userspace. Signed-off-by: Peter Zijlstra (Intel) --- include/linux/sched_clock.h | 28 +++++++++++++++++++++++++ kernel/time/sched_clock.c | 41 ++++++++++++------------------------- 2 files changed, 41 insertions(+), 28 deletions(-) diff --git a/include/linux/sched_clock.h b/include/linux/sched_clock.h index 0bb04a96a6d4..528718e4ed52 100644 --- a/include/linux/sched_clock.h +++ b/include/linux/sched_clock.h @@ -6,6 +6,34 @@ #define LINUX_SCHED_CLOCK #ifdef CONFIG_GENERIC_SCHED_CLOCK +/** + * struct clock_read_data - data required to read from sched_clock() + * + * @epoch_ns: sched_clock() value at last update + * @epoch_cyc: Clock cycle value at last update. + * @sched_clock_mask: Bitmask for two's complement subtraction of non 64bit + * clocks. + * @read_sched_clock: Current clock source (or dummy source when suspended). + * @mult: Multipler for scaled math conversion. + * @shift: Shift value for scaled math conversion. + * + * Care must be taken when updating this structure; it is read by + * some very hot code paths. It occupies <=40 bytes and, when combined + * with the seqcount used to synchronize access, comfortably fits into + * a 64 byte cache line. + */ +struct clock_read_data { + u64 epoch_ns; + u64 epoch_cyc; + u64 sched_clock_mask; + u64 (*read_sched_clock)(void); + u32 mult; + u32 shift; +}; + +extern struct clock_read_data *sched_clock_read_begin(unsigned int *seq); +extern int sched_clock_read_retry(unsigned int seq); + extern void generic_sched_clock_init(void); extern void sched_clock_register(u64 (*read)(void), int bits, diff --git a/kernel/time/sched_clock.c b/kernel/time/sched_clock.c index fa3f800d7d76..0acaadc3156c 100644 --- a/kernel/time/sched_clock.c +++ b/kernel/time/sched_clock.c @@ -19,31 +19,6 @@ #include "timekeeping.h" -/** - * struct clock_read_data - data required to read from sched_clock() - * - * @epoch_ns: sched_clock() value at last update - * @epoch_cyc: Clock cycle value at last update. - * @sched_clock_mask: Bitmask for two's complement subtraction of non 64bit - * clocks. - * @read_sched_clock: Current clock source (or dummy source when suspended). - * @mult: Multipler for scaled math conversion. - * @shift: Shift value for scaled math conversion. - * - * Care must be taken when updating this structure; it is read by - * some very hot code paths. It occupies <=40 bytes and, when combined - * with the seqcount used to synchronize access, comfortably fits into - * a 64 byte cache line. - */ -struct clock_read_data { - u64 epoch_ns; - u64 epoch_cyc; - u64 sched_clock_mask; - u64 (*read_sched_clock)(void); - u32 mult; - u32 shift; -}; - /** * struct clock_data - all data needed for sched_clock() (including * registration of a new clock source) @@ -93,6 +68,17 @@ static inline u64 notrace cyc_to_ns(u64 cyc, u32 mult, u32 shift) return (cyc * mult) >> shift; } +struct clock_read_data *sched_clock_read_begin(unsigned int *seq) +{ + *seq = raw_read_seqcount(&cd.seq); + return cd.read_data + (*seq & 1); +} + +int sched_clock_read_retry(unsigned int seq) +{ + return read_seqcount_retry(&cd.seq, seq); +} + unsigned long long notrace sched_clock(void) { u64 cyc, res; @@ -100,13 +86,12 @@ unsigned long long notrace sched_clock(void) struct clock_read_data *rd; do { - seq = raw_read_seqcount(&cd.seq); - rd = cd.read_data + (seq & 1); + rd = sched_clock_read_begin(&seq); cyc = (rd->read_sched_clock() - rd->epoch_cyc) & rd->sched_clock_mask; res = rd->epoch_ns + cyc_to_ns(cyc, rd->mult, rd->shift); - } while (read_seqcount_retry(&cd.seq, seq)); + } while (sched_clock_read_retry(seq)); return res; } -- 2.17.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91CBFC433DF for ; Wed, 15 Jul 2020 02:07:14 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5FEEC20663 for ; Wed, 15 Jul 2020 02:07:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="VJVM5zlH"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="MBfmujPe" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5FEEC20663 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:References:In-Reply-To:Message-Id:Date:Subject:To: From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=pgna+R28sviZ5M256dVW2ZpA8H/ZfWfWzudNtKDVVV4=; b=VJVM5zlHKLP3tPsMqoxPQq5xHw 4ONjwJA6iTUOnljuFxoxkRABcNa/S25zjv7Y+FZPLGDPq1EcCyLxQnrRBWhgWVsSMxEe8vrBELfOV YdzxK4p+Be1ZcZi2t58sYAGbg+3vglpGmjta5nonsHVvyXfuNumRlMfXqE0J4WYCb3+kJkil/9Qno Y+SvxPhaWgDf5ausTM68M+f78Tge/HH3nVuKZiCwagmbyKu8qc8R2Fe6RcjUB/nCkpeYGKUZmQZlw K1vQBEWifIch2BDsqFByb0w0EMKMbJK+EX018kw+Zyhm24KHFH2BFHpjIAeDsBj55Vn3QdkISQRvC wF0mCn6g==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jvWni-0007N0-E5; Wed, 15 Jul 2020 02:05:38 +0000 Received: from mail-pj1-x1041.google.com ([2607:f8b0:4864:20::1041]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jvWnf-0007Lu-Ik for linux-arm-kernel@lists.infradead.org; Wed, 15 Jul 2020 02:05:36 +0000 Received: by mail-pj1-x1041.google.com with SMTP id k71so1297712pje.0 for ; Tue, 14 Jul 2020 19:05:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=hMCjz+92Go61XkKZ2KMP8Y2GkwrHTA5+Bwzj3gXnH/0=; b=MBfmujPeyn0IdrSwOHGVo9a3VpMNRn6TeZf7/RFd0B0J6XD9VPV0WJMHnmgGXrF3zJ a5cmGVYTCxM6wTojk8M78+v6KQa+5illqo07dzRaUED0a27GxkDvtLTf1Jy8EV3fPmKI uoD4X/LECFe7OZQbLUC4qeBmYgpOkOoESsQpf4kSLdesuSmOLwvUIhnZzk06XVuajbBZ 4XXE45D+qYtwLmi17ipzwzDkCGBFBWaQkgC88SvkOFSMphMWklW1sXp1o6iKenfVJDpj ciclauAccDb4jB166FCuZSiYxDKciO36aOfGo72e2ZOVhKXyy6ZfQZwMVumPlJXqi4ep nEAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=hMCjz+92Go61XkKZ2KMP8Y2GkwrHTA5+Bwzj3gXnH/0=; b=QzqKXx64zI5CwP5lfBWQ8/OVj7UajxdcfvMqxZ8PVdcSU09sHH1Fnkw9JV53aP7ZB/ 7k8UBMVo60EJGPBkw5hTk6MIq/MJhtlsWAqnkCGmYtrw6X9oQ4fzhqpJXbY+Pk3W0XD6 Kay+QDw62weALD2Me5if75718KrJ7wOdw0ICGY2LGFIFnO7IafNA/KWxnmVTHsFAunS4 sj+I/qEtQ7gG8kRaR0ONg1Iyz2BO9Ai6V/O1nrhh08pHUuUruPYnfuFnTeG/r7TXbshX cYuv1iMKa9lMmxpUrfAPd/qLGd0gX+oCOdcAntoAe1bTt+Y5m0VcKQHGPQs6wYvNGsqE oj4g== X-Gm-Message-State: AOAM532VJr68Vvokxflp+qzfzLkAbnKqj1XYkKIC9IHkf3iQRfgAYNYd H4c9A7ivJCgEiIijmZsKbgAsnA== X-Google-Smtp-Source: ABdhPJweKUBha3SLMEvJ8s2uZ+OFHY4WBgN+V4lhlk4NpJLYQnE4We4H9eumEdWXhqvcTPSS4e/8sQ== X-Received: by 2002:a17:90a:bc98:: with SMTP id x24mr6851187pjr.63.1594778733247; Tue, 14 Jul 2020 19:05:33 -0700 (PDT) Received: from localhost ([2400:8904::f03c:91ff:fe8a:bbe4]) by smtp.gmail.com with ESMTPSA id h7sm374726pfq.15.2020.07.14.19.05.31 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 14 Jul 2020 19:05:32 -0700 (PDT) From: Leo Yan To: Will Deacon , Mark Rutland , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Catalin Marinas , Thomas Gleixner , Paul Cercueil , "Ben Dooks (Codethink)" , "Ahmed S. Darwish" , Adrian Hunter , Kan Liang , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 1/6] sched_clock: Expose struct clock_read_data Date: Wed, 15 Jul 2020 10:05:07 +0800 Message-Id: <20200715020512.20991-2-leo.yan@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200715020512.20991-1-leo.yan@linaro.org> References: <20200715020512.20991-1-leo.yan@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200714_220535_688322_68672852 X-CRM114-Status: GOOD ( 15.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Peter Zijlstra In order to support perf_event_mmap_page::cap_time features, an architecture needs, aside from a userspace readable counter register, to expose the exact clock data so that userspace can convert the counter register into a correct timestamp. Provide struct clock_read_data and two (seqcount) helpers so that architectures (arm64 in specific) can expose the numbers to userspace. Signed-off-by: Peter Zijlstra (Intel) --- include/linux/sched_clock.h | 28 +++++++++++++++++++++++++ kernel/time/sched_clock.c | 41 ++++++++++++------------------------- 2 files changed, 41 insertions(+), 28 deletions(-) diff --git a/include/linux/sched_clock.h b/include/linux/sched_clock.h index 0bb04a96a6d4..528718e4ed52 100644 --- a/include/linux/sched_clock.h +++ b/include/linux/sched_clock.h @@ -6,6 +6,34 @@ #define LINUX_SCHED_CLOCK #ifdef CONFIG_GENERIC_SCHED_CLOCK +/** + * struct clock_read_data - data required to read from sched_clock() + * + * @epoch_ns: sched_clock() value at last update + * @epoch_cyc: Clock cycle value at last update. + * @sched_clock_mask: Bitmask for two's complement subtraction of non 64bit + * clocks. + * @read_sched_clock: Current clock source (or dummy source when suspended). + * @mult: Multipler for scaled math conversion. + * @shift: Shift value for scaled math conversion. + * + * Care must be taken when updating this structure; it is read by + * some very hot code paths. It occupies <=40 bytes and, when combined + * with the seqcount used to synchronize access, comfortably fits into + * a 64 byte cache line. + */ +struct clock_read_data { + u64 epoch_ns; + u64 epoch_cyc; + u64 sched_clock_mask; + u64 (*read_sched_clock)(void); + u32 mult; + u32 shift; +}; + +extern struct clock_read_data *sched_clock_read_begin(unsigned int *seq); +extern int sched_clock_read_retry(unsigned int seq); + extern void generic_sched_clock_init(void); extern void sched_clock_register(u64 (*read)(void), int bits, diff --git a/kernel/time/sched_clock.c b/kernel/time/sched_clock.c index fa3f800d7d76..0acaadc3156c 100644 --- a/kernel/time/sched_clock.c +++ b/kernel/time/sched_clock.c @@ -19,31 +19,6 @@ #include "timekeeping.h" -/** - * struct clock_read_data - data required to read from sched_clock() - * - * @epoch_ns: sched_clock() value at last update - * @epoch_cyc: Clock cycle value at last update. - * @sched_clock_mask: Bitmask for two's complement subtraction of non 64bit - * clocks. - * @read_sched_clock: Current clock source (or dummy source when suspended). - * @mult: Multipler for scaled math conversion. - * @shift: Shift value for scaled math conversion. - * - * Care must be taken when updating this structure; it is read by - * some very hot code paths. It occupies <=40 bytes and, when combined - * with the seqcount used to synchronize access, comfortably fits into - * a 64 byte cache line. - */ -struct clock_read_data { - u64 epoch_ns; - u64 epoch_cyc; - u64 sched_clock_mask; - u64 (*read_sched_clock)(void); - u32 mult; - u32 shift; -}; - /** * struct clock_data - all data needed for sched_clock() (including * registration of a new clock source) @@ -93,6 +68,17 @@ static inline u64 notrace cyc_to_ns(u64 cyc, u32 mult, u32 shift) return (cyc * mult) >> shift; } +struct clock_read_data *sched_clock_read_begin(unsigned int *seq) +{ + *seq = raw_read_seqcount(&cd.seq); + return cd.read_data + (*seq & 1); +} + +int sched_clock_read_retry(unsigned int seq) +{ + return read_seqcount_retry(&cd.seq, seq); +} + unsigned long long notrace sched_clock(void) { u64 cyc, res; @@ -100,13 +86,12 @@ unsigned long long notrace sched_clock(void) struct clock_read_data *rd; do { - seq = raw_read_seqcount(&cd.seq); - rd = cd.read_data + (seq & 1); + rd = sched_clock_read_begin(&seq); cyc = (rd->read_sched_clock() - rd->epoch_cyc) & rd->sched_clock_mask; res = rd->epoch_ns + cyc_to_ns(cyc, rd->mult, rd->shift); - } while (read_seqcount_retry(&cd.seq, seq)); + } while (sched_clock_read_retry(seq)); return res; } -- 2.17.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel