From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 531E5C433DF for ; Mon, 19 Oct 2020 15:45:20 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C0B3822282 for ; Mon, 19 Oct 2020 15:45:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="awdDttR8"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="moaW1E0v" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C0B3822282 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=CtMPELTDjLaHAvy5VpRtEs9XjGW0aJfK9YY+5pHrQXE=; b=awdDttR8wrWxHBqRyFSxJzV2k VMKBRd8HTvVJFyfdO2jl0vXGQMoQnxAHky6l+CZ+vFKkF5rAREUQFlqaGLxh5SLO0JP0oxaOVitph ajV8HsTRcNMPbc0Pj6LyNaOZ3f5xe5P4ywGY+pogk/kBSy4GJCjbIflpU/xuWCFeTPLNltet1zS27 r0/uLL1OYEraY6wHzIbxtI2cEFAZoFR/deJmjoXpqBlrM/iz0LCVGhMms+rF4ujWJa0mEPbyzUsJS 1Q/CNz/ZIsxHLeU1jh5YuA6u7nuxOsypgLWIfEzstW4koW03BwhtZbi5mDSR+SaVH0erGQldntRHS Jn43HtXnw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kUXJt-0005Ra-6N; Mon, 19 Oct 2020 15:43:33 +0000 Received: from mail-lf1-x141.google.com ([2a00:1450:4864:20::141]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kUXJq-0005Qy-1l for linux-arm-kernel@lists.infradead.org; Mon, 19 Oct 2020 15:43:30 +0000 Received: by mail-lf1-x141.google.com with SMTP id c141so19085lfg.5 for ; Mon, 19 Oct 2020 08:43:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=XExGzAcdQVZ+Wsl0Ih6yKEGYRJsLil4v2iKvYQXD8O8=; b=moaW1E0vHgrFCU56qBl5pxhY6LYq9YuCPprIFZ3cObBUncUN1wPU5tXf1fT5w/2xsI ZIO1Y9wYHsLGviMnPKj+i6dniHjtZPUXu7AKr/pPdhUwUpDZ3LZCIelMTE1DXR3TkY55 aMERY0OrhpJ4zI1lU9QVAK+f8Oa/Djutgs7oEi7b+Mj5I+hCbUxtuqoCWPTAZxApezDb 9gHCXRH4/GURRHkvYDTKVNMyeY2dO8qKlBqDRIilr2eF84/5aUqqi5Au3MvQ4Zp/losz hbtKdJtT6OOFQo+Blh3ivz7Jm/mLH+rufFHzU+KpCr4S1MBqoAVcIpJ6lxt1G9FD4Pky W1Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=XExGzAcdQVZ+Wsl0Ih6yKEGYRJsLil4v2iKvYQXD8O8=; b=KgNieRZ5hc9LQCulxckzXzN84nE/ijtzkYePlTEb92sbRFFdtu6IIAnlrWMqC3vql5 l0wzRf4tc9LpT+GlsHAvzuGyYFUhskieqZdz8Yje6RdQezJj4UnfKMIyTJtdm6vP0EOh XYxbl4vPEaGFZHpMuuxfW95RtZlFJwnWrZBt8FoVwNZdVpEKWLHdqsk4MsyFp/RQSAQl P65joWQ+8/eHPCNHkCBKtXZQ13F9pxhcv6ZH6BykdsoiB5TOIG0Bny/U2erArHKa1Bea Q9OBorMvnlCcV7mFnY7011Rclq8+0OWj3KkUcs0ZaCaR90DJFapgvLRzgIH1kucwkl0G 9Hlw== X-Gm-Message-State: AOAM531T8F/v1w6eHtFMJfVsVnsM+U2j/9aRokKrt6h3FYYCslHk+nrh Em7P0SzY57odYUrNQM6Fx2DP+jMPsT+iu00CZ8r22w== X-Google-Smtp-Source: ABdhPJyyEHCGeedlETa86EdcrbjLr0FuCnBU+As+mylycg6aD5k+hzMdjO1OPT2oZ/S4w0/q9F8B5ncFxf2jTx3yLo8= X-Received: by 2002:ac2:5a4f:: with SMTP id r15mr113032lfn.258.1603122208344; Mon, 19 Oct 2020 08:43:28 -0700 (PDT) MIME-Version: 1.0 References: <20200901144324.1071694-1-maz@kernel.org> <20200901144324.1071694-4-maz@kernel.org> <353f13b0dcc6c7ea1b44012d9632a0cc@kernel.org> In-Reply-To: <353f13b0dcc6c7ea1b44012d9632a0cc@kernel.org> From: Vincent Guittot Date: Mon, 19 Oct 2020 17:43:16 +0200 Message-ID: Subject: Re: [PATCH v3 03/16] arm64: Allow IPIs to be handled as normal interrupts To: Marc Zyngier X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201019_114330_181572_95DC15D1 X-CRM114-Status: GOOD ( 29.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Sumit Garg , Android Kernel Team , Florian Fainelli , Russell King , Jason Cooper , Saravana Kannan , Andrew Lunn , Catalin Marinas , Gregory Clement , linux-kernel , Thomas Gleixner , Will Deacon , Valentin Schneider , LAK Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, 19 Oct 2020 at 15:04, Marc Zyngier wrote: > > Hi Vincent, > > On 2020-10-19 13:42, Vincent Guittot wrote: > > Hi Marc, > > > > On Tue, 1 Sep 2020 at 16:44, Marc Zyngier wrote: > >> > >> In order to deal with IPIs as normal interrupts, let's add > >> a new way to register them with the architecture code. > >> > >> set_smp_ipi_range() takes a range of interrupts, and allows > >> the arch code to request them as if the were normal interrupts. > >> A standard handler is then called by the core IRQ code to deal > >> with the IPI. > >> > >> This means that we don't need to call irq_enter/irq_exit, and > >> that we don't need to deal with set_irq_regs either. So let's > >> move the dispatcher into its own function, and leave handle_IPI() > >> as a compatibility function. > >> > >> On the sending side, let's make use of ipi_send_mask, which > >> already exists for this purpose. > >> > >> One of the major difference is that we end up, in some cases > >> (such as when performing IRQ time accounting on the scheduler > >> IPI), end up with nested irq_enter()/irq_exit() pairs. > >> Other than the (relatively small) overhead, there should be > >> no consequences to it (these pairs are designed to nest > >> correctly, and the accounting shouldn't be off). > > > > While rebasing on mainline, I have faced a performance regression for > > the benchmark: > > perf bench sched pipe > > on my arm64 dual quad core (hikey) and my 2 nodes x 112 CPUS (thx2) > > > > The regression comes from: > > commit: d3afc7f12987 ("arm64: Allow IPIs to be handled as normal > > interrupts") > > That's interesting, as this patch doesn't really change anything (most > of the potential overhead comes in later). The only potential overhead > I can see is that the scheduler_ipi() call is now wrapped around > irq_enter()/irq_exit(). > > > > > v5.9 + this patch > > hikey : 48818(+/- 0.31) 37503(+/- 0.15%) -23.2% > > thx2 : 132410(+/- 1.72) 122646(+/- 1.92%) -7.4% > > > > By + this patch, I mean merging branch from this patch. Whereas > > merging the previous: > > commit: 83cfac95c018 ("genirq: Allow interrupts to be excluded from > > /proc/interrupts") > > It doesn't show any regression > > Since you are running perf, can you spot where the overhead occurs? hmm... Difficult to say because tracing the bench decreases a lot the result. I have pasted the perf reports. With this patch : # Samples: 634 of event 'cpu-clock' # Event count (approx.): 158500000 # # Overhead Command Shared Object Symbol # ........ .......... .................. .................................. # 31.86% sched-pipe [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore 8.68% sched-pipe [kernel.kallsyms] [k] _raw_spin_unlock_irq 6.31% sched-pipe [kernel.kallsyms] [k] __schedule 5.21% sched-pipe [kernel.kallsyms] [k] schedule 4.73% sched-pipe [kernel.kallsyms] [k] pipe_read 3.31% sched-pipe [kernel.kallsyms] [k] el0_svc_common.constprop.3 2.84% sched-pipe [kernel.kallsyms] [k] ww_mutex_lock_interruptible 2.52% sched-pipe [kernel.kallsyms] [k] init_wait_entry 2.37% sched-pipe [kernel.kallsyms] [k] mutex_unlock 2.21% sched-pipe [kernel.kallsyms] [k] new_sync_read 1.89% sched-pipe [kernel.kallsyms] [k] new_sync_write 1.74% sched-pipe [kernel.kallsyms] [k] security_file_permission 1.74% sched-pipe [kernel.kallsyms] [k] vfs_read 1.58% sched-pipe [kernel.kallsyms] [k] __my_cpu_offset 1.26% sched-pipe libpthread-2.24.so [.] 0x0000000000010a2c 1.10% sched-pipe [kernel.kallsyms] [k] mutex_lock 1.10% sched-pipe [kernel.kallsyms] [k] vfs_write After reverting this patch which gives a result similar to v5.9: # Samples: 659 of event 'cpu-clock' # Event count (approx.): 164750000 # # Overhead Command Shared Object Symbol # ........ .......... .................. ............................... # 29.29% sched-pipe [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore 21.40% sched-pipe [kernel.kallsyms] [k] _raw_spin_unlock_irq 4.86% sched-pipe [kernel.kallsyms] [k] pipe_read 4.55% sched-pipe [kernel.kallsyms] [k] ww_mutex_lock_interruptible 2.88% sched-pipe [kernel.kallsyms] [k] __schedule 2.88% sched-pipe [kernel.kallsyms] [k] _raw_spin_lock_irqsave 2.88% sched-pipe [kernel.kallsyms] [k] schedule 2.12% sched-pipe [kernel.kallsyms] [k] new_sync_read 1.82% sched-pipe [kernel.kallsyms] [k] mutex_lock 1.67% sched-pipe [kernel.kallsyms] [k] el0_svc_common.constprop.3 1.67% sched-pipe [kernel.kallsyms] [k] pipe_write 1.21% sched-pipe [kernel.kallsyms] [k] rw_verify_area 1.21% sched-pipe [kernel.kallsyms] [k] security_file_permission 1.06% sched-pipe [kernel.kallsyms] [k] fsnotify I have only put symbol with overhead above 1% so _raw_spin_unlock_irq, schedule and __schedule seem the most impacted but i can't get any conclusion I can sent you perf.data files if you want > > Thanks, > > M. > -- > Jazz is not dead. It just smells funny... _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel