From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 545BDC433B4 for ; Fri, 7 May 2021 08:56:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1546F61078 for ; Fri, 7 May 2021 08:56:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235759AbhEGI5e convert rfc822-to-8bit (ORCPT ); Fri, 7 May 2021 04:57:34 -0400 Received: from mail.kernel.org ([198.145.29.99]:53404 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229621AbhEGI5d (ORCPT ); Fri, 7 May 2021 04:57:33 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AF52861057; Fri, 7 May 2021 08:56:33 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1lewHf-00BQbF-LB; Fri, 07 May 2021 09:56:31 +0100 Date: Fri, 07 May 2021 09:56:24 +0100 Message-ID: <87lf8qq5vr.wl-maz@kernel.org> From: Marc Zyngier To: He Ying Cc: , , , , , , , , , , , , , Subject: Re: [PATCH v3 03/16] arm64: Allow IPIs to be handled as normal interrupts In-Reply-To: References: <87pmy4qe7e.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: heying24@huawei.com, vincent.guittot@linaro.org, Valentin.Schneider@arm.com, andrew@lunn.ch, catalin.marinas@arm.com, f.fainelli@gmail.com, gregory.clement@bootlin.com, kernel-team@android.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux@arm.linux.org.uk, saravanak@google.com, sumit.garg@linaro.org, tglx@linutronix.de, will@kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, 07 May 2021 08:30:06 +0100, He Ying wrote: > > > 在 2021/5/6 19:44, Marc Zyngier 写道: > > On Thu, 06 May 2021 08:50:42 +0100, > > He Ying wrote: > >> Hello Marc, > >> > >> We have faced a performance regression for handling ipis since this > >> commit. I think it's the same issue reported by Vincent. > > Can you share more details on what regression you have observed? > > What's the workload, the system, the performance drop? > > OK. We have just calculated the pmu cycles from the entry of gic_handle_irq > to the entry of do_handle_ipi. Here is some more information about our test: > > CPU: Hisilicon hip05-d02 > > Applying the patch series: 1115 cycles > Reverting the patch series: 599 cycles And? How is that meaningful? Interrupts are pretty rare compared to everything that happens in the system. How does it affect the behaviour of the system as a whole? > > > > >> I found you pointed out the possible two causes: > >> > >> (1) irq_enter/exit on the rescheduling IPI means we reschedule much > >> more often. > > It turned out to be a red herring. We don't reschedule more often, but > > we instead suffer from the overhead of irq_enter()/irq_exit(). > > However, this only matters for silly benchmarks, and no real-life > > workload showed any significant regression. Have you identified such > > realistic workload? > > I'm afraid not. We just run some benchmarks and calculated pmu cycle > counters. But we have observed running time from the entry of > gic_handle_irq to the entry of do_handle_ipi almost doubles. Doesn't > it affect realistic workload? Then I'm not that interested. Show me an actual regression in a real workload that affects people, and I'll be a bit more sympathetic to your complain. But quoting raw numbers do not help. There is a number of advantages to having IPI as IRQs, as it allows us to deal with proper allocation (other subsystem want to use IPIs), and eventually NMIs. There is a trade-off, and if that means wasting a few cycles, so be it. > >> (2) irq_domain lookups add some overhead. > > While this is also a potential source of overhead, it turned out not > > to be the case. > OK. > > > >> But I don't see any following patches in mainline. So, are you still > >> working on this issue? Looking forward to your reply. > > See [1]. However, there is probably better things to do than this > > low-level specialisation of IPIs, and Thomas outlined what needs to be > > done (see v1 of the patch series). > > OK. I see the patch series. Would it be applied to the mainline > someday? I notice that more than 5 months have passed since you sent > the patch series. I have no plan to merge these patches any time soon, given that nobody has shown a measurable regression using something other than a trivial benchmark. If you come up with such an example, I will of course reconsider this position. Thanks, M. -- Without deviation from the norm, progress is not possible. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEA62C433ED for ; Fri, 7 May 2021 08:58:30 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EE8E8613C5 for ; Fri, 7 May 2021 08:58:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EE8E8613C5 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Subject:Cc:To: From:Message-ID:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=NC6bd7KXHKgmnUJi+Kt+9yOKEA22mra1/eh37JsEJwU=; b=rbw6iwfH8vE8rBFphOkAPUIrv ycP5p/J5HAcAJsqoFSOSFFRMp8d05tAsGKFSMEWiq6Wuneq2su4tgyxbAYTWUr4S0iqp5uwrekKPb +qwmzpAmGoJnLZpCGw2RWRJqS2k4Cfo+L8lL7/pr1X5tK4afYMR/dR49QjTgaJ3VG6D/8/yDSvXdk ZA0yMqGD8RWJ1eNSxTU37L0hwInH89Jcrn+QlGNGPPGBTIWn5clWnemdw6Hdj06iezlaxjxM+ZHja vbX1xoNhD3CKZu+INF2iaPBGk9SeaNQ1tnry5LhdcwXgdy6YmdxtZnXUJEhINVDAO4AWTsR38jXDK 0rlbr4Zpw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lewHp-006Xv6-Nc; Fri, 07 May 2021 08:56:41 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lewHk-006Xue-Tj for linux-arm-kernel@desiato.infradead.org; Fri, 07 May 2021 08:56:37 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: Content-Type:MIME-Version:References:In-Reply-To:Subject:Cc:To:From: Message-ID:Date:Sender:Reply-To:Content-ID:Content-Description; bh=lN0ZH0TxK/NOY3wwWYPtUar40FT1plNf0LEUnVS7iKc=; b=mjMWuDV0153axhUPcS9XxNyQaw lhy9iZVa2P++Rs1Q+CckOsqszt8YeJ/29bcgfGuCvTiZ2FMOg+1jbtQoBkbSOizJyCyQv6SnVNIc1 yewf43pYQB1tb6WTKTDPeW/mSzEkAC1N95fD24nDFMXj3Iqyu2//Ud9gT0wdWzmmSZfB47DyiAzYD 5zUAY7TuNWr0dcmZRtctTs8zbT+v9w2k3yiR6fDNhPBSRceNGanU0W8pbgUsm+Zx8IAAvX5WNEf2f mThzM/ccFqYRktV8F6xdi99UeGosASZNZohGwhv4vIv8Rq52lHLPCnPlf9n6TXwxSGwi1i2OdW6cB vXlXAS3w==; Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lewHi-006iFe-8w for linux-arm-kernel@lists.infradead.org; Fri, 07 May 2021 08:56:35 +0000 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AF52861057; Fri, 7 May 2021 08:56:33 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1lewHf-00BQbF-LB; Fri, 07 May 2021 09:56:31 +0100 Date: Fri, 07 May 2021 09:56:24 +0100 Message-ID: <87lf8qq5vr.wl-maz@kernel.org> From: Marc Zyngier To: He Ying Cc: , , , , , , , , , , , , , Subject: Re: [PATCH v3 03/16] arm64: Allow IPIs to be handled as normal interrupts In-Reply-To: References: <87pmy4qe7e.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: heying24@huawei.com, vincent.guittot@linaro.org, Valentin.Schneider@arm.com, andrew@lunn.ch, catalin.marinas@arm.com, f.fainelli@gmail.com, gregory.clement@bootlin.com, kernel-team@android.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux@arm.linux.org.uk, saravanak@google.com, sumit.garg@linaro.org, tglx@linutronix.de, will@kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210507_015634_433678_91A8F77C X-CRM114-Status: GOOD ( 36.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gRnJpLCAwNyBNYXkgMjAyMSAwODozMDowNiArMDEwMCwKSGUgWWluZyA8aGV5aW5nMjRAaHVh d2VpLmNvbT4gd3JvdGU6Cj4gCj4gCj4g5ZyoIDIwMjEvNS82IDE5OjQ0LCBNYXJjIFp5bmdpZXIg 5YaZ6YGTOgo+ID4gT24gVGh1LCAwNiBNYXkgMjAyMSAwODo1MDo0MiArMDEwMCwKPiA+IEhlIFlp bmcgPGhleWluZzI0QGh1YXdlaS5jb20+IHdyb3RlOgo+ID4+IEhlbGxvIE1hcmMsCj4gPj4gCj4g Pj4gV2UgaGF2ZSBmYWNlZCBhIHBlcmZvcm1hbmNlIHJlZ3Jlc3Npb24gZm9yIGhhbmRsaW5nIGlw aXMgc2luY2UgdGhpcwo+ID4+IGNvbW1pdC4gSSB0aGluayBpdCdzIHRoZSBzYW1lIGlzc3VlIHJl cG9ydGVkIGJ5IFZpbmNlbnQuCj4gPiBDYW4geW91IHNoYXJlIG1vcmUgZGV0YWlscyBvbiB3aGF0 IHJlZ3Jlc3Npb24geW91IGhhdmUgb2JzZXJ2ZWQ/Cj4gPiBXaGF0J3MgdGhlIHdvcmtsb2FkLCB0 aGUgc3lzdGVtLCB0aGUgcGVyZm9ybWFuY2UgZHJvcD8KPiAKPiBPSy4gV2UgaGF2ZSBqdXN0IGNh bGN1bGF0ZWQgdGhlIHBtdSBjeWNsZXMgZnJvbSB0aGUgZW50cnkgb2YgZ2ljX2hhbmRsZV9pcnEK PiB0byB0aGUgZW50cnkgb2YgZG9faGFuZGxlX2lwaS4gSGVyZSBpcyBzb21lIG1vcmUgaW5mb3Jt YXRpb24gYWJvdXQgb3VyIHRlc3Q6Cj4gCj4gQ1BVOiBIaXNpbGljb24gaGlwMDUtZDAyCj4gCj4g QXBwbHlpbmcgdGhlIHBhdGNoIHNlcmllczogMTExNSBjeWNsZXMKPiBSZXZlcnRpbmcgdGhlIHBh dGNoIHNlcmllczogNTk5IGN5Y2xlcwoKQW5kPyBIb3cgaXMgdGhhdCBtZWFuaW5nZnVsPyBJbnRl cnJ1cHRzIGFyZSBwcmV0dHkgcmFyZSBjb21wYXJlZCB0bwpldmVyeXRoaW5nIHRoYXQgaGFwcGVu cyBpbiB0aGUgc3lzdGVtLiBIb3cgZG9lcyBpdCBhZmZlY3QgdGhlCmJlaGF2aW91ciBvZiB0aGUg c3lzdGVtIGFzIGEgd2hvbGU/Cgo+IAo+ID4gCj4gPj4gSSBmb3VuZCB5b3UgcG9pbnRlZCBvdXQg dGhlIHBvc3NpYmxlIHR3byBjYXVzZXM6Cj4gPj4gCj4gPj4gKDEpIGlycV9lbnRlci9leGl0IG9u IHRoZSByZXNjaGVkdWxpbmcgSVBJIG1lYW5zIHdlIHJlc2NoZWR1bGUgbXVjaAo+ID4+IG1vcmUg b2Z0ZW4uCj4gPiBJdCB0dXJuZWQgb3V0IHRvIGJlIGEgcmVkIGhlcnJpbmcuIFdlIGRvbid0IHJl c2NoZWR1bGUgbW9yZSBvZnRlbiwgYnV0Cj4gPiB3ZSBpbnN0ZWFkIHN1ZmZlciBmcm9tIHRoZSBv dmVyaGVhZCBvZiBpcnFfZW50ZXIoKS9pcnFfZXhpdCgpLgo+ID4gSG93ZXZlciwgdGhpcyBvbmx5 IG1hdHRlcnMgZm9yIHNpbGx5IGJlbmNobWFya3MsIGFuZCBubyByZWFsLWxpZmUKPiA+IHdvcmts b2FkIHNob3dlZCBhbnkgc2lnbmlmaWNhbnQgcmVncmVzc2lvbi4gSGF2ZSB5b3UgaWRlbnRpZmll ZCBzdWNoCj4gPiByZWFsaXN0aWMgd29ya2xvYWQ/Cj4gCj4gSSdtIGFmcmFpZCBub3QuIFdlIGp1 c3QgcnVuIHNvbWUgYmVuY2htYXJrcyBhbmQgY2FsY3VsYXRlZCBwbXUgY3ljbGUKPiBjb3VudGVy cy4gIEJ1dCB3ZSBoYXZlIG9ic2VydmVkIHJ1bm5pbmcgdGltZSBmcm9tIHRoZSBlbnRyeSBvZgo+ IGdpY19oYW5kbGVfaXJxIHRvIHRoZSBlbnRyeSBvZiBkb19oYW5kbGVfaXBpIGFsbW9zdCBkb3Vi bGVzLiBEb2Vzbid0Cj4gaXQgYWZmZWN0IHJlYWxpc3RpYyB3b3JrbG9hZD8KClRoZW4gSSdtIG5v dCB0aGF0IGludGVyZXN0ZWQuIFNob3cgbWUgYW4gYWN0dWFsIHJlZ3Jlc3Npb24gaW4gYSByZWFs Cndvcmtsb2FkIHRoYXQgYWZmZWN0cyBwZW9wbGUsIGFuZCBJJ2xsIGJlIGEgYml0IG1vcmUgc3lt cGF0aGV0aWMgdG8KeW91ciBjb21wbGFpbi4gQnV0IHF1b3RpbmcgcmF3IG51bWJlcnMgZG8gbm90 IGhlbHAuCgpUaGVyZSBpcyBhIG51bWJlciBvZiBhZHZhbnRhZ2VzIHRvIGhhdmluZyBJUEkgYXMg SVJRcywgYXMgaXQgYWxsb3dzIHVzCnRvIGRlYWwgd2l0aCBwcm9wZXIgYWxsb2NhdGlvbiAob3Ro ZXIgc3Vic3lzdGVtIHdhbnQgdG8gdXNlIElQSXMpLCBhbmQKZXZlbnR1YWxseSBOTUlzLiBUaGVy ZSBpcyBhIHRyYWRlLW9mZiwgYW5kIGlmIHRoYXQgbWVhbnMgd2FzdGluZyBhIGZldwpjeWNsZXMs IHNvIGJlIGl0LgoKPiA+PiAoMikgaXJxX2RvbWFpbiBsb29rdXBzIGFkZCBzb21lIG92ZXJoZWFk Lgo+ID4gV2hpbGUgdGhpcyBpcyBhbHNvIGEgcG90ZW50aWFsIHNvdXJjZSBvZiBvdmVyaGVhZCwg aXQgdHVybmVkIG91dCBub3QKPiA+IHRvIGJlIHRoZSBjYXNlLgo+IE9LLgo+ID4gCj4gPj4gQnV0 IEkgZG9uJ3Qgc2VlIGFueSBmb2xsb3dpbmcgcGF0Y2hlcyBpbiBtYWlubGluZS4gU28sIGFyZSB5 b3Ugc3RpbGwKPiA+PiB3b3JraW5nIG9uIHRoaXMgaXNzdWU/ICBMb29raW5nIGZvcndhcmQgdG8g eW91ciByZXBseS4KPiA+IFNlZSBbMV0uIEhvd2V2ZXIsIHRoZXJlIGlzIHByb2JhYmx5IGJldHRl ciB0aGluZ3MgdG8gZG8gdGhhbiB0aGlzCj4gPiBsb3ctbGV2ZWwgc3BlY2lhbGlzYXRpb24gb2Yg SVBJcywgYW5kIFRob21hcyBvdXRsaW5lZCB3aGF0IG5lZWRzIHRvIGJlCj4gPiBkb25lIChzZWUg djEgb2YgdGhlIHBhdGNoIHNlcmllcykuCj4gCj4gT0suIEkgc2VlIHRoZSBwYXRjaCBzZXJpZXMu IFdvdWxkIGl0IGJlIGFwcGxpZWQgdG8gdGhlIG1haW5saW5lCj4gc29tZWRheT8gSSBub3RpY2Ug dGhhdCBtb3JlIHRoYW4gNSBtb250aHMgaGF2ZSBwYXNzZWQgc2luY2UgeW91IHNlbnQKPiB0aGUg cGF0Y2ggc2VyaWVzLgoKSSBoYXZlIG5vIHBsYW4gdG8gbWVyZ2UgdGhlc2UgcGF0Y2hlcyBhbnkg dGltZSBzb29uLCBnaXZlbiB0aGF0IG5vYm9keQpoYXMgc2hvd24gYSBtZWFzdXJhYmxlIHJlZ3Jl c3Npb24gdXNpbmcgc29tZXRoaW5nIG90aGVyIHRoYW4gYSB0cml2aWFsCmJlbmNobWFyay4gSWYg eW91IGNvbWUgdXAgd2l0aCBzdWNoIGFuIGV4YW1wbGUsIEkgd2lsbCBvZiBjb3Vyc2UKcmVjb25z aWRlciB0aGlzIHBvc2l0aW9uLgoKVGhhbmtzLAoKCU0uCgotLSAKV2l0aG91dCBkZXZpYXRpb24g ZnJvbSB0aGUgbm9ybSwgcHJvZ3Jlc3MgaXMgbm90IHBvc3NpYmxlLgoKX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtYXJtLWtlcm5lbCBtYWlsaW5n IGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5p bmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtYXJtLWtlcm5lbAo=