From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B5D1C43334 for ; Fri, 24 Jun 2022 17:52:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230415AbiFXRwp (ORCPT ); Fri, 24 Jun 2022 13:52:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50962 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232375AbiFXRwF (ORCPT ); Fri, 24 Jun 2022 13:52:05 -0400 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F357E7E000; Fri, 24 Jun 2022 10:51:18 -0700 (PDT) Received: from pps.filterd (m0109333.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 25OBn3nl007547; Fri, 24 Jun 2022 10:51:04 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=message-id : date : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding : mime-version; s=facebook; bh=LqOuJCnyPweAl+PHWwaNP+34b+JLUz2isha0pbS+U6o=; b=JO2hMB7NR3k58XmgqrSPDXTnqf+iZDDpWL/07J74fHTTO9pCX/uvj+zziZfColRF8ZRN kWMjPXjziyZK4yoQwulkAMmDwh0Ml/DE+7UQ48UsAGLDUNCRw7Gz4sEpPNjQy5uB4Eqs uT8UX4fJpndrgM2DSj1XCgNLXsAG18HU/M8= Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2169.outbound.protection.outlook.com [104.47.58.169]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3gvqnc1cu0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 24 Jun 2022 10:51:04 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FoUW6MBp4dhZFVN3kh7QGuzSOGxUspYCtnzAMLLzlE+3ZByRwwyWqBmsxnCgdcODco3UF2/XixvIg4LA8fVDvJrqk9+7KQjIot/sftI+fZzScoh0dBwDQR+Syc3637OrICzRwHapx713LP/cjXSn8+C/eR10nX/vHYWYUrEttiMS8WHEowMShEoN3+XMb9M2nvHXAEFvrnCkXZnopTJDTIOf0LUk4NkrjPh22gE1OdnRaqfi+GFLk9GXt5odL+98FIwD69jYh0XEQW7OHphZTOgpd2swruOxZWIt2NohKWK6SPSYpwe2rrc1TENvlQbDFfqQYxXscQ/4bGedYkRtdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LqOuJCnyPweAl+PHWwaNP+34b+JLUz2isha0pbS+U6o=; b=Jn4Zl5MTgfEn0BLp1Cx/N/iQQjn7jNcdbi2oUCcEfXNM7vTGxEKdTz8j/vSDnMP7+IMvsc3h049VTCnMUuvx4/B74Di5Nm8c/pI5dvF40KIU6jiB+p+qDLcoFxWhnPhg3lXkfTAiFzK8RmF89+T9gxqXVKCvNf6kz2/Kj3+pQbvq+XHVVsoCgRFdqGUrDGYvpAJWFkHoCqfvLBxRT/8hhELDHPXBoqJxjAo/pzqiG97aalAgdVEEfjPkOSXDh9q8ICxiIspZBMzPf8Q5YXrvvbTm1LNOdzoWgTiy5UQcJ+8DqCtqp+yLBxo6CSbqokwFLxO/RJ0R3qP6GEw9GkhE5A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=fb.com; dmarc=pass action=none header.from=fb.com; dkim=pass header.d=fb.com; arc=none Received: from DM6PR15MB4039.namprd15.prod.outlook.com (2603:10b6:5:2b2::20) by SA1PR15MB4675.namprd15.prod.outlook.com (2603:10b6:806:19e::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5353.22; Fri, 24 Jun 2022 17:51:02 +0000 Received: from DM6PR15MB4039.namprd15.prod.outlook.com ([fe80::2197:2f04:3527:d764]) by DM6PR15MB4039.namprd15.prod.outlook.com ([fe80::2197:2f04:3527:d764%8]) with mapi id 15.20.5373.015; Fri, 24 Jun 2022 17:51:02 +0000 Message-ID: <5a9a3256-d079-75f3-4b7d-84eef0862af3@fb.com> Date: Fri, 24 Jun 2022 13:50:59 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.10.0 Subject: Re: [PATCH bpf-next] selftests/bpf: Add benchmark for local_storage RCU Tasks Trace usage Content-Language: en-US To: Martin KaFai Lau Cc: bpf@vger.kernel.org, rcu@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Paul E . McKenney" , Eric Dumazet , Kernel Team References: <20220623234609.543263-1-davemarchevsky@fb.com> <20220624172238.wpioajigxywd4hxv@kafai-mbp> From: Dave Marchevsky In-Reply-To: <20220624172238.wpioajigxywd4hxv@kafai-mbp> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-ClientProxiedBy: BL0PR05CA0021.namprd05.prod.outlook.com (2603:10b6:208:91::31) To DM6PR15MB4039.namprd15.prod.outlook.com (2603:10b6:5:2b2::20) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5efa0ef4-627e-4939-d1e7-08da560a1a6a X-MS-TrafficTypeDiagnostic: SA1PR15MB4675:EE_ X-FB-Source: Internal X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KNmbAKYSbqJuFt7RXv9gJ7o2HTBdYVK+TQ33y4GaC+FGzAlPJXcfqxjCRpDlpL+hhTLH8phspwdcnnpj7VbuG1Ct1uZKqmyqX4wexWENEUSxz7kU2NoP4EBn4USmw9/CBby6MqbALP685m9jFg1MtY1gPzVXP1ortsl6C03okm83fJnii//DIzeNp9EjfcLhK4r1pHjITWgG+naw2ylE7d6+1nK+Ru/XeI3qT5GYfo7fo9pwQKUEOJH/I0jPGzfDhbHmGN/YY2nr9/ZkMWS46jPNwO/7rNZUmskHAtmpTWgmiFXgVO7c4bRKXmf3zQEz3xSgASGdQEQ4z6HweVsItkyC9DGG7YZCgyK3T7kes6sxxduWmWmWtDRBMhfQj+TMUfzefPls/Y1SO+3gpdhRn3viPwiWrkIrWYJjnUxpjW8RA8e26C4ECSgGqk4F3f86RR/z67kPDMATfDy/txpPFGvNVksBZHcM05omLyQdlzyEXSK5GTc0qeGjrot2cYVaciaNRd3nuDTR8jjZy837nIWLdbvVQCTklbYhZ1vs6E1MRqTA1qTon8xRnRvwO3mPMGgWhVldMq96EANAz4cHbdq6JXYKfbzaW54GL3HgjjSATMQjF0Kr0TS/ejoTs2AOodkZAnWY96FV0Pk4W49+k1eWyDCUfV2a/h91fxLO3VYXb7VeQB8XoBJ2tsDIXrymoeLrQE2hgi/i9mTZwY6Scyg2Y+I1CODZacLUszw0SfGUrmzU7A7iYM+hhZl/VBNzXKd9NOfmJ65YSwqQvpJIpqtXgGv6oqHitDDasdaxm9LetH5CW9GezzLRP/0XgCV8 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR15MB4039.namprd15.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(376002)(346002)(136003)(366004)(396003)(39860400002)(31686004)(36756003)(8936002)(478600001)(5660300002)(2906002)(6862004)(54906003)(37006003)(316002)(6636002)(6486002)(8676002)(66556008)(4326008)(66476007)(66946007)(41300700001)(31696002)(86362001)(53546011)(6512007)(6506007)(2616005)(186003)(6666004)(38100700002)(83380400001)(43740500002)(45980500001);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?djhDWDY0ZUpBNnE1em1HYzNHNjdsVkMxcEw0VGhJY3pLRUhiVWJoTjhpUWRo?= =?utf-8?B?dVdUN3dxSjRFWVRTQURmSlc0a2JJMkxueUk3TjFGRGlBWkZVSHRXbndVT2pv?= =?utf-8?B?Ri82VVVVSDBHcmxIaFpqMDA5OHNlVGtVZUdQRnc5a0ovZDFwejh2QzMxYTVF?= =?utf-8?B?cWlKUy84dnoxZ1lPTVpUWTRNYTdkUzc1M1g1QUk4VmZ3M0VPdzlFWGd2WnJJ?= =?utf-8?B?R2xuUGFybWdIY2FoOFMxRjVZb05sazd5TU8vOFFkRUdCVHJWNWJMQldaNmMr?= =?utf-8?B?TldUbytGc0FRWGdBbjdsSDh4VUt5aXJvTzRsbkJ6QUJhVFZ2WG9rV1I1dDNX?= =?utf-8?B?NTQrU2wyVFAxckJobEVnMlBKbnZZWi85SkhRTzFJR3NLTjVMSDNJaUMzdFFF?= =?utf-8?B?eVZ4MnBlOWxiZ0lmU25Ca3BDc0hjNWplckVsUWhJYnpTRzFQdURMRjZUbXZP?= =?utf-8?B?VXF0eWxYQ3dmS09YVVYrRE45ZVBnbnNWQWpZbHdJN2Nhb1V3cWN3aUdpRmJN?= =?utf-8?B?SS8wRXArT0JvY2ZPbTFYdDlhNVQvMkszNDFCQVFTRFhzWU5SdzlaYVJZcWRR?= =?utf-8?B?bjZKYStpMnJ1Q0dvTjRURnRkVjF1eXZ2NGlRVkMwbTdaczVEWHJnZ0JCbCtD?= =?utf-8?B?Yk42ck5LN1IzbUxFMU82NUduNDd0RkZFT05SK1R4UHBaZ0NUKzlaM2czRXM3?= =?utf-8?B?ZWZ5RjBMSTBjaWRwUWJwckNhK2VqMXpwSVg3OVNQNGxtVWEvVHN1VkxmRWdk?= =?utf-8?B?UDVmV0JzOWdkL09uZUEvaWsvMVJmT1FROEpOREY4M1VoK2Z4SXp2akVJUDYy?= =?utf-8?B?NmFkN0x2TjJJbzV1VmJZalNPTnBhYjRaVHJsYzAzLzFRQzcwdGtrc1dOcXIw?= =?utf-8?B?cGttRnZyOXo5Qk11VkRuYkVqa24rUEt6S2MzYk10eGpCNGcrZThCMlcva0dT?= =?utf-8?B?Vm1vTzByZE5HSmtCdk5QdmFCK2xjaW1LNi9TTzdhYkdaOVhtbEUydkV3T0lq?= =?utf-8?B?TDNMMm1Bbmpaby9GWlBFbEo0YS9NVnhPMHpJZG11RjFxY1dGMWlSK21NNjgw?= =?utf-8?B?K0pEYTdFWEV3bFN5NzUwYXJaeGlyV01vQzBVblVqcnp3UDluVnBQa3lMSDBV?= =?utf-8?B?UjVSNENYSEhLcGFsQzI4VUxjcjhaSittWVlzRTJ6Qzk2S25XaUpsWE1sQTJX?= =?utf-8?B?cEZQRHowa3JoZ3lDSjlMWVN4UDlDa1Q0SnlwUzBvNTYxWTNFbjhjMTV3emp5?= =?utf-8?B?U0RVODJ4d2dpOGZMTlRxYjBxSlhYOGZyYlFRbWgvRzFNVkc5ZG1nSnFuMkJH?= =?utf-8?B?K3IxRUEzbWluYlNqMWFoWnJxTjJLVFhxT3N3bDhvRmxiOEU5OGhyUWlORm5Q?= =?utf-8?B?YXpyKzJaa2NrbzRIT0hnaWlxM2dhOWg3ZUZ5L0g1cUtYTGcxNkdSNlVEZS9l?= =?utf-8?B?eThWZTIwbEpBdmI0VUlJc1d1L1JMSzBoTTZqUGZLclNjSGV0Q1BucGZiNEQv?= =?utf-8?B?K3NTVllQNnRBTUZFNE5qcnY2OUVFSk5lK1JTZGo1SFpVZktnbi9vYTUvSVJn?= =?utf-8?B?VEcrdFk3YTlZTDAzT3Z3MGo3VjJSNFpraDNSYUoveUp4Z1NzWDcvVGFoSDYr?= =?utf-8?B?VDhPUVNBb2gvS1FMSGtuQmdBN1VNdndvc0xRR0Z5YjNtLzVReWJOT2d1MEtQ?= =?utf-8?B?R3RkUmdlUzZjVXBSNFpod1c5Qnk1UThNclJsRVNtc1pZZG1zSHp1eFk2YnVZ?= =?utf-8?B?NHcvOThUb2Y4bUlJS2pKK1VvWXVSWkN1OW9wdnpoUk1uQWdFSXk2Nk9oNlI1?= =?utf-8?B?VGhGUlIyVWNFWFFUTWZrSzdjYUJIekxCV1FkVEliNzA2U084Yzh0T1ZsOWhK?= =?utf-8?B?NCszamZxa3B4VTlrWWkyYWIyNDRGSVk5VGdFRWdES3pKWDNqNnhtTnZDQWkz?= =?utf-8?B?TGduUzNXQkIzUlV1NHZYM29MTHA5ZDFBbUNWSXFpNzdkVzVHVXpxVUtwdkpa?= =?utf-8?B?L2o4dFQwdGI5VzlySS8ycWdLTVdkdUErQ1lONFVSZmZ6SlJLL2g5dllhOU1O?= =?utf-8?B?UWJZUWFGTFRaY2xJMkpOd3RwUDJWaVhmaFlCZU1Pd0ROUUk2Tlk2eE9pRktE?= =?utf-8?B?aXoySjM0c3JBcXJScXU2dE9HZi94VlpEUTVVcVRST3Z0U0VJL1FtT2ovREpC?= =?utf-8?Q?UisRyrGzJCWu2sTv4fPh/FE=3D?= X-OriginatorOrg: fb.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5efa0ef4-627e-4939-d1e7-08da560a1a6a X-MS-Exchange-CrossTenant-AuthSource: DM6PR15MB4039.namprd15.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jun 2022 17:51:02.0622 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IBYw8rh22uKDqRiRJF/z0WL9neM2WNEaCN2iMjTgrgCEqVAbx61aCFv2L+oHlqdJh68+PxpKVgdG7Qs3qZRjxg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR15MB4675 X-Proofpoint-ORIG-GUID: nltZXTH-4QLeQ77p3AhXt2YAvxDzRUkx X-Proofpoint-GUID: nltZXTH-4QLeQ77p3AhXt2YAvxDzRUkx X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-06-24_08,2022-06-23_01,2022-06-22_01 Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org On 6/24/22 1:22 PM, Martin KaFai Lau wrote: > On Thu, Jun 23, 2022 at 04:46:09PM -0700, Dave Marchevsky wrote: >> +static void report_progress(int iter, struct bench_res *res, long delta_ns) >> +{ >> + if (ctx.skel->bss->unexpected) { >> + fprintf(stderr, "Error: Unexpected order of bpf prog calls (postgp after pregp)."); >> + fprintf(stderr, "Data can't be trusted, exiting\n"); >> + exit(1); >> + } >> + >> + if (args.quiet) >> + return; >> + >> + printf("Iter %d\t avg tasks_trace grace period latency\t%lf ns\n", >> + iter, res->gp_ns / (double)res->gp_ct); >> + printf("Iter %d\t avg ticks per tasks_trace grace period\t%lf\n", >> + iter, res->stime / (double)res->gp_ct); >> +} >> + > > [ ... ] > >> diff --git a/tools/testing/selftests/bpf/benchs/run_bench_local_storage_rcu_tasks_trace.sh b/tools/testing/selftests/bpf/benchs/run_bench_local_storage_rcu_tasks_trace.sh >> new file mode 100755 >> index 000000000000..5dac1f02892c >> --- /dev/null >> +++ b/tools/testing/selftests/bpf/benchs/run_bench_local_storage_rcu_tasks_trace.sh >> @@ -0,0 +1,11 @@ >> +#!/bin/bash >> +# SPDX-License-Identifier: GPL-2.0 >> + >> +kthread_pid=`pgrep rcu_tasks_trace_kthread` >> + >> +if [ -z $kthread_pid ]; then >> + echo "error: Couldn't find rcu_tasks_trace_kthread" >> + exit 1 >> +fi >> + >> +./bench --nr_procs 15000 --kthread_pid $kthread_pid -d 600 --quiet 1 local-storage-tasks-trace >> diff --git a/tools/testing/selftests/bpf/progs/local_storage_rcu_tasks_trace_bench.c b/tools/testing/selftests/bpf/progs/local_storage_rcu_tasks_trace_bench.c >> new file mode 100644 >> index 000000000000..9b11342b19a0 >> --- /dev/null >> +++ b/tools/testing/selftests/bpf/progs/local_storage_rcu_tasks_trace_bench.c >> @@ -0,0 +1,65 @@ >> +// SPDX-License-Identifier: GPL-2.0 >> +/* Copyright (c) 2022 Meta Platforms, Inc. and affiliates. */ >> + >> +#include "vmlinux.h" >> +#include >> +#include "bpf_misc.h" >> + >> +struct { >> + __uint(type, BPF_MAP_TYPE_TASK_STORAGE); >> + __uint(map_flags, BPF_F_NO_PREALLOC); >> + __type(key, int); >> + __type(value, int); >> +} task_storage SEC(".maps"); >> + >> +long hits; >> +long gp_hits; >> +long gp_times; >> +long current_gp_start; >> +long unexpected; >> + >> +SEC("fentry/" SYS_PREFIX "sys_getpgid") >> +int get_local(void *ctx) >> +{ >> + struct task_struct *task; >> + int idx; >> + int *s; >> + >> + idx = 0; >> + task = bpf_get_current_task_btf(); >> + s = bpf_task_storage_get(&task_storage, task, &idx, >> + BPF_LOCAL_STORAGE_GET_F_CREATE); >> + if (!s) >> + return 0; >> + >> + *s = 3; >> + bpf_task_storage_delete(&task_storage, task); >> + __sync_add_and_fetch(&hits, 1); >> + return 0; >> +} >> + >> +SEC("kprobe/rcu_tasks_trace_pregp_step") > nit. Similar to the fentry sys_getpgid above. > may as well use fentry for everything. > Will do. >> +int pregp_step(struct pt_regs *ctx) >> +{ >> + current_gp_start = bpf_ktime_get_ns(); >> + return 0; >> +} >> + >> +SEC("kprobe/rcu_tasks_trace_postgp") >> +int postgp(struct pt_regs *ctx) >> +{ >> + if (!current_gp_start) { >> + /* Will only happen if prog tracing rcu_tasks_trace_pregp_step doesn't >> + * execute before this prog >> + */ >> + __sync_add_and_fetch(&unexpected, 1); > I consistently hit this: > ./bench --nr_procs 1500 --kthread_pid ... -d 60 --quiet 1 local-storage-tasks-trace > Setting up benchmark 'local-storage-tasks-trace'... > Spun up 1500 procs (our pid 28351) > Benchmark 'local-storage-tasks-trace' started. > Error: Unexpected order of bpf prog calls (postgp after pregp).Data can't be trusted, exiting > > May be there is a chance for the very first postgp being called > before the pregp_step? > I will change the logic to only consider postgp w/o pregp "unexpected" after the postgp probe has been triggered once. > Thanks for working on this!