From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74A90C07E96 for ; Thu, 15 Jul 2021 10:07:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 538AB61178 for ; Thu, 15 Jul 2021 10:07:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232558AbhGOKJ7 (ORCPT ); Thu, 15 Jul 2021 06:09:59 -0400 Received: from foss.arm.com ([217.140.110.172]:50472 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232495AbhGOKJ7 (ORCPT ); Thu, 15 Jul 2021 06:09:59 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C29E531B; Thu, 15 Jul 2021 03:07:05 -0700 (PDT) Received: from C02TD0UTHF1T.local (unknown [10.57.0.163]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5EF843F694; Thu, 15 Jul 2021 03:07:04 -0700 (PDT) Date: Thu, 15 Jul 2021 11:06:54 +0100 From: Mark Rutland To: Robin Murphy Cc: Qu Wenruo , kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, Linux ARM Subject: Re: Any way to disable KVM VHE extension? Message-ID: <20210715100643.GA53188@C02TD0UTHF1T.local> References: <37f873cf-1b39-ea7f-a5e7-6feb0200dd4c@suse.com> <0e992d47-1f17-d49f-8341-670770ac49ef@suse.com> <6331dfe5-a028-4a71-6cc1-479003a58f47@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <6331dfe5-a028-4a71-6cc1-479003a58f47@arm.com> Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Thu, Jul 15, 2021 at 11:00:42AM +0100, Robin Murphy wrote: > On 2021-07-15 10:44, Qu Wenruo wrote: > > > > > > On 2021/7/15 下午5:28, Robin Murphy wrote: > > > On 2021-07-15 09:55, Qu Wenruo wrote: > > > > Hi, > > > > > > > > Recently I'm playing around the Nvidia Xavier AGX board, which > > > > has VHE extension support. > > > > > > > > In theory, considering the CPU and memory, it should be pretty > > > > powerful compared to boards like RPI CM4. > > > > > > > > But to my surprise, KVM runs pretty poor on Xavier. > > > > > > > > Just booting the edk2 firmware could take over 10s, and 20s to > > > > fully boot the kernel. > > > > Even my VM on RPI CM4 has way faster boot time, even just > > > > running on PCIE2.0 x1 lane NVME, and just 4 2.1Ghz A72 core. > > > > > > > > This is definitely out of my expectation, I double checked to be > > > > sure that it's running in KVM mode. > > > > > > > > But further digging shows that, since Xavier AGX CPU supports > > > > VHE, kvm is running in VHE mode other than HYP mode on CM4. > > > > > > > > Is there anyway to manually disable VHE mode to test the more > > > > common HYP mode on Xavier? > > > > > > According to kernel-parameters.txt, "kvm-arm.mode=nvhe" (or its > > > low-level equivalent "id_aa64mmfr1.vh=0") on the command line should > > > do that. > > > > Thanks for this one, I stupidly only searched modinfo of kvm, and didn't > > even bother to search arch/arm64/kvm... > > > > > > > > However I'd imagine the discrepancy is likely to be something more > > > fundamental to the wildly different microarchitectures. There's > > > certainly no harm in giving non-VHE a go for comparison, but I > > > wouldn't be surprised if it turns out even slower... > > > > You're totally right, with nvhe mode, it's still the same slow speed. > > > > BTW, what did you mean by the "wildly different microarch"? > > Is ARMv8.2 arch that different from ARMv8 of RPI4? > > I don't mean Armv8.x architectural features, I mean the actual > implementation of NVIDIA's Carmel core is very, very different from > Cortex-A72 or indeed our newer v8.2 Cortex-A designs. > > > And any extra methods I could try to explore the reason of the slowness? > > I guess the first check would be whether you're trapping and exiting the VM > significantly more. I believe there are stats somewhere, but I don't know > exactly where, sorry - I know very little about actually *using* KVM :) > > If it's not that, then it might just be that EDK2 is doing a lot of cache > maintenance or system register modification or some other operation that > happens to be slower on Carmel compared to Cortex-A72. It would also be worthchecking tha the CPUs are running at the speed you expect, in e.g. case the lack of a DVFS driver means they're running slow, and this just happens to be more noticeable in a VM. You can estimate that by using `perf stat` on the host on a busy loop, and looking what the cpu-cycles count implies. Thanks, Mark. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED08CC07E96 for ; Thu, 15 Jul 2021 10:07:10 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 60F6B61178 for ; Thu, 15 Jul 2021 10:07:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 60F6B61178 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id E20EC40068; Thu, 15 Jul 2021 06:07:09 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DsEsvxvvrjTk; Thu, 15 Jul 2021 06:07:08 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 9AEBE4A19F; Thu, 15 Jul 2021 06:07:08 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 8F6E04B081 for ; Thu, 15 Jul 2021 06:07:07 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IyN++5K8aFzK for ; Thu, 15 Jul 2021 06:07:06 -0400 (EDT) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 3CC3649FB0 for ; Thu, 15 Jul 2021 06:07:06 -0400 (EDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C29E531B; Thu, 15 Jul 2021 03:07:05 -0700 (PDT) Received: from C02TD0UTHF1T.local (unknown [10.57.0.163]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5EF843F694; Thu, 15 Jul 2021 03:07:04 -0700 (PDT) Date: Thu, 15 Jul 2021 11:06:54 +0100 From: Mark Rutland To: Robin Murphy Subject: Re: Any way to disable KVM VHE extension? Message-ID: <20210715100643.GA53188@C02TD0UTHF1T.local> References: <37f873cf-1b39-ea7f-a5e7-6feb0200dd4c@suse.com> <0e992d47-1f17-d49f-8341-670770ac49ef@suse.com> <6331dfe5-a028-4a71-6cc1-479003a58f47@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <6331dfe5-a028-4a71-6cc1-479003a58f47@arm.com> Cc: Linux ARM , kvmarm@lists.cs.columbia.edu, Qu Wenruo , kvm@vger.kernel.org X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu T24gVGh1LCBKdWwgMTUsIDIwMjEgYXQgMTE6MDA6NDJBTSArMDEwMCwgUm9iaW4gTXVycGh5IHdy b3RlOgo+IE9uIDIwMjEtMDctMTUgMTA6NDQsIFF1IFdlbnJ1byB3cm90ZToKPiA+IAo+ID4gCj4g PiBPbiAyMDIxLzcvMTUg5LiL5Y2INToyOCwgUm9iaW4gTXVycGh5IHdyb3RlOgo+ID4gPiBPbiAy MDIxLTA3LTE1IDA5OjU1LCBRdSBXZW5ydW8gd3JvdGU6Cj4gPiA+ID4gSGksCj4gPiA+ID4gCj4g PiA+ID4gUmVjZW50bHkgSSdtIHBsYXlpbmcgYXJvdW5kIHRoZSBOdmlkaWEgWGF2aWVyIEFHWCBi b2FyZCwgd2hpY2gKPiA+ID4gPiBoYXMgVkhFIGV4dGVuc2lvbiBzdXBwb3J0Lgo+ID4gPiA+IAo+ ID4gPiA+IEluIHRoZW9yeSwgY29uc2lkZXJpbmcgdGhlIENQVSBhbmQgbWVtb3J5LCBpdCBzaG91 bGQgYmUgcHJldHR5Cj4gPiA+ID4gcG93ZXJmdWwgY29tcGFyZWQgdG8gYm9hcmRzIGxpa2UgUlBJ IENNNC4KPiA+ID4gPiAKPiA+ID4gPiBCdXQgdG8gbXkgc3VycHJpc2UsIEtWTSBydW5zIHByZXR0 eSBwb29yIG9uIFhhdmllci4KPiA+ID4gPiAKPiA+ID4gPiBKdXN0IGJvb3RpbmcgdGhlIGVkazIg ZmlybXdhcmUgY291bGQgdGFrZSBvdmVyIDEwcywgYW5kIDIwcyB0bwo+ID4gPiA+IGZ1bGx5IGJv b3QgdGhlIGtlcm5lbC4KPiA+ID4gPiBFdmVuIG15IFZNIG9uIFJQSSBDTTQgaGFzIHdheSBmYXN0 ZXIgYm9vdCB0aW1lLCBldmVuIGp1c3QKPiA+ID4gPiBydW5uaW5nIG9uIFBDSUUyLjAgeDEgbGFu ZSBOVk1FLCBhbmQganVzdCA0IDIuMUdoeiBBNzIgY29yZS4KPiA+ID4gPiAKPiA+ID4gPiBUaGlz IGlzIGRlZmluaXRlbHkgb3V0IG9mIG15IGV4cGVjdGF0aW9uLCBJIGRvdWJsZSBjaGVja2VkIHRv IGJlCj4gPiA+ID4gc3VyZSB0aGF0IGl0J3MgcnVubmluZyBpbiBLVk0gbW9kZS4KPiA+ID4gPiAK PiA+ID4gPiBCdXQgZnVydGhlciBkaWdnaW5nIHNob3dzIHRoYXQsIHNpbmNlIFhhdmllciBBR1gg Q1BVIHN1cHBvcnRzCj4gPiA+ID4gVkhFLCBrdm0gaXMgcnVubmluZyBpbiBWSEUgbW9kZSBvdGhl ciB0aGFuIEhZUCBtb2RlIG9uIENNNC4KPiA+ID4gPiAKPiA+ID4gPiBJcyB0aGVyZSBhbnl3YXkg dG8gbWFudWFsbHkgZGlzYWJsZSBWSEUgbW9kZSB0byB0ZXN0IHRoZSBtb3JlCj4gPiA+ID4gY29t bW9uIEhZUCBtb2RlIG9uIFhhdmllcj8KPiA+ID4gCj4gPiA+IEFjY29yZGluZyB0byBrZXJuZWwt cGFyYW1ldGVycy50eHQsICJrdm0tYXJtLm1vZGU9bnZoZSIgKG9yIGl0cwo+ID4gPiBsb3ctbGV2 ZWwgZXF1aXZhbGVudCAiaWRfYWE2NG1tZnIxLnZoPTAiKSBvbiB0aGUgY29tbWFuZCBsaW5lIHNo b3VsZAo+ID4gPiBkbyB0aGF0Lgo+ID4gCj4gPiBUaGFua3MgZm9yIHRoaXMgb25lLCBJIHN0dXBp ZGx5IG9ubHkgc2VhcmNoZWQgbW9kaW5mbyBvZiBrdm0sIGFuZCBkaWRuJ3QKPiA+IGV2ZW4gYm90 aGVyIHRvIHNlYXJjaCBhcmNoL2FybTY0L2t2bS4uLgo+ID4gCj4gPiA+IAo+ID4gPiBIb3dldmVy IEknZCBpbWFnaW5lIHRoZSBkaXNjcmVwYW5jeSBpcyBsaWtlbHkgdG8gYmUgc29tZXRoaW5nIG1v cmUKPiA+ID4gZnVuZGFtZW50YWwgdG8gdGhlIHdpbGRseSBkaWZmZXJlbnQgbWljcm9hcmNoaXRl Y3R1cmVzLiBUaGVyZSdzCj4gPiA+IGNlcnRhaW5seSBubyBoYXJtIGluIGdpdmluZyBub24tVkhF IGEgZ28gZm9yIGNvbXBhcmlzb24sIGJ1dCBJCj4gPiA+IHdvdWxkbid0IGJlIHN1cnByaXNlZCBp ZiBpdCB0dXJucyBvdXQgZXZlbiBzbG93ZXIuLi4KPiA+IAo+ID4gWW91J3JlIHRvdGFsbHkgcmln aHQsIHdpdGggbnZoZSBtb2RlLCBpdCdzIHN0aWxsIHRoZSBzYW1lIHNsb3cgc3BlZWQuCj4gPiAK PiA+IEJUVywgd2hhdCBkaWQgeW91IG1lYW4gYnkgdGhlICJ3aWxkbHkgZGlmZmVyZW50IG1pY3Jv YXJjaCI/Cj4gPiBJcyBBUk12OC4yIGFyY2ggdGhhdCBkaWZmZXJlbnQgZnJvbSBBUk12OCBvZiBS UEk0Pwo+IAo+IEkgZG9uJ3QgbWVhbiBBcm12OC54IGFyY2hpdGVjdHVyYWwgZmVhdHVyZXMsIEkg bWVhbiB0aGUgYWN0dWFsCj4gaW1wbGVtZW50YXRpb24gb2YgTlZJRElBJ3MgQ2FybWVsIGNvcmUg aXMgdmVyeSwgdmVyeSBkaWZmZXJlbnQgZnJvbQo+IENvcnRleC1BNzIgb3IgaW5kZWVkIG91ciBu ZXdlciB2OC4yIENvcnRleC1BIGRlc2lnbnMuCj4gCj4gPiBBbmQgYW55IGV4dHJhIG1ldGhvZHMg SSBjb3VsZCB0cnkgdG8gZXhwbG9yZSB0aGUgcmVhc29uIG9mIHRoZSBzbG93bmVzcz8KPiAKPiBJ IGd1ZXNzIHRoZSBmaXJzdCBjaGVjayB3b3VsZCBiZSB3aGV0aGVyIHlvdSdyZSB0cmFwcGluZyBh bmQgZXhpdGluZyB0aGUgVk0KPiBzaWduaWZpY2FudGx5IG1vcmUuIEkgYmVsaWV2ZSB0aGVyZSBh cmUgc3RhdHMgc29tZXdoZXJlLCBidXQgSSBkb24ndCBrbm93Cj4gZXhhY3RseSB3aGVyZSwgc29y cnkgLSBJIGtub3cgdmVyeSBsaXR0bGUgYWJvdXQgYWN0dWFsbHkgKnVzaW5nKiBLVk0gOikKPiAK PiBJZiBpdCdzIG5vdCB0aGF0LCB0aGVuIGl0IG1pZ2h0IGp1c3QgYmUgdGhhdCBFREsyIGlzIGRv aW5nIGEgbG90IG9mIGNhY2hlCj4gbWFpbnRlbmFuY2Ugb3Igc3lzdGVtIHJlZ2lzdGVyIG1vZGlm aWNhdGlvbiBvciBzb21lIG90aGVyIG9wZXJhdGlvbiB0aGF0Cj4gaGFwcGVucyB0byBiZSBzbG93 ZXIgb24gQ2FybWVsIGNvbXBhcmVkIHRvIENvcnRleC1BNzIuCgpJdCB3b3VsZCBhbHNvIGJlIHdv cnRoY2hlY2tpbmcgdGhhIHRoZSBDUFVzIGFyZSBydW5uaW5nIGF0IHRoZSBzcGVlZCB5b3UKZXhw ZWN0LCBpbiBlLmcuIGNhc2UgdGhlIGxhY2sgb2YgYSBEVkZTIGRyaXZlciBtZWFucyB0aGV5J3Jl IHJ1bm5pbmcKc2xvdywgYW5kIHRoaXMganVzdCBoYXBwZW5zIHRvIGJlIG1vcmUgbm90aWNlYWJs ZSBpbiBhIFZNLgoKWW91IGNhbiBlc3RpbWF0ZSB0aGF0IGJ5IHVzaW5nIGBwZXJmIHN0YXRgIG9u IHRoZSBob3N0IG9uIGEgYnVzeSBsb29wLAphbmQgbG9va2luZyB3aGF0IHRoZSBjcHUtY3ljbGVz IGNvdW50IGltcGxpZXMuCgpUaGFua3MsCk1hcmsuCl9fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fCmt2bWFybSBtYWlsaW5nIGxpc3QKa3ZtYXJtQGxpc3RzLmNz LmNvbHVtYmlhLmVkdQpodHRwczovL2xpc3RzLmNzLmNvbHVtYmlhLmVkdS9tYWlsbWFuL2xpc3Rp bmZvL2t2bWFybQo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A03E3C07E96 for ; Thu, 15 Jul 2021 10:08:56 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 69EFA61178 for ; Thu, 15 Jul 2021 10:08:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 69EFA61178 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=E0JWgkdKrslYoXi0ewE0TpNknmh3pkFsMiMA8VyvBdE=; b=05kFyCvdiYCvQX 5f5lrcTwC13uC0DVZqeDCfX0/WKSK2Ro6AXAh7qFNpghatG4rwSfSahpo6yTp715tPK6yRSJqkxcT F8cKwZ2nxnA3LgJ3AMaKbTSnUoch7ZEZ9I4FZv3DN/Pdl1aFjyuTvpm4jy08cT8l3b6HZZFA0Y2DS zdnmzHJw1ryHvx55PsTV7pmqNP8qYgA9yHOEdCe3d/9Ad8UrsgvJsptujYw414S6bncxJgLd9qybE pxQKGrNnJAl1nLd8S+TkxFDCsgM6HNZ/8P3VdfZHYCb91WzRsmRP4cK39PUkIneyU3Ua5ZHwFA9dK hmV0VqKpdnjAb7gY3S6Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m3yGt-000Orl-Hp; Thu, 15 Jul 2021 10:07:11 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m3yGp-000Oqw-M2 for linux-arm-kernel@lists.infradead.org; Thu, 15 Jul 2021 10:07:09 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C29E531B; Thu, 15 Jul 2021 03:07:05 -0700 (PDT) Received: from C02TD0UTHF1T.local (unknown [10.57.0.163]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5EF843F694; Thu, 15 Jul 2021 03:07:04 -0700 (PDT) Date: Thu, 15 Jul 2021 11:06:54 +0100 From: Mark Rutland To: Robin Murphy Cc: Qu Wenruo , kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, Linux ARM Subject: Re: Any way to disable KVM VHE extension? Message-ID: <20210715100643.GA53188@C02TD0UTHF1T.local> References: <37f873cf-1b39-ea7f-a5e7-6feb0200dd4c@suse.com> <0e992d47-1f17-d49f-8341-670770ac49ef@suse.com> <6331dfe5-a028-4a71-6cc1-479003a58f47@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <6331dfe5-a028-4a71-6cc1-479003a58f47@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210715_030707_857341_2E601259 X-CRM114-Status: GOOD ( 35.38 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gVGh1LCBKdWwgMTUsIDIwMjEgYXQgMTE6MDA6NDJBTSArMDEwMCwgUm9iaW4gTXVycGh5IHdy b3RlOgo+IE9uIDIwMjEtMDctMTUgMTA6NDQsIFF1IFdlbnJ1byB3cm90ZToKPiA+IAo+ID4gCj4g PiBPbiAyMDIxLzcvMTUg5LiL5Y2INToyOCwgUm9iaW4gTXVycGh5IHdyb3RlOgo+ID4gPiBPbiAy MDIxLTA3LTE1IDA5OjU1LCBRdSBXZW5ydW8gd3JvdGU6Cj4gPiA+ID4gSGksCj4gPiA+ID4gCj4g PiA+ID4gUmVjZW50bHkgSSdtIHBsYXlpbmcgYXJvdW5kIHRoZSBOdmlkaWEgWGF2aWVyIEFHWCBi b2FyZCwgd2hpY2gKPiA+ID4gPiBoYXMgVkhFIGV4dGVuc2lvbiBzdXBwb3J0Lgo+ID4gPiA+IAo+ ID4gPiA+IEluIHRoZW9yeSwgY29uc2lkZXJpbmcgdGhlIENQVSBhbmQgbWVtb3J5LCBpdCBzaG91 bGQgYmUgcHJldHR5Cj4gPiA+ID4gcG93ZXJmdWwgY29tcGFyZWQgdG8gYm9hcmRzIGxpa2UgUlBJ IENNNC4KPiA+ID4gPiAKPiA+ID4gPiBCdXQgdG8gbXkgc3VycHJpc2UsIEtWTSBydW5zIHByZXR0 eSBwb29yIG9uIFhhdmllci4KPiA+ID4gPiAKPiA+ID4gPiBKdXN0IGJvb3RpbmcgdGhlIGVkazIg ZmlybXdhcmUgY291bGQgdGFrZSBvdmVyIDEwcywgYW5kIDIwcyB0bwo+ID4gPiA+IGZ1bGx5IGJv b3QgdGhlIGtlcm5lbC4KPiA+ID4gPiBFdmVuIG15IFZNIG9uIFJQSSBDTTQgaGFzIHdheSBmYXN0 ZXIgYm9vdCB0aW1lLCBldmVuIGp1c3QKPiA+ID4gPiBydW5uaW5nIG9uIFBDSUUyLjAgeDEgbGFu ZSBOVk1FLCBhbmQganVzdCA0IDIuMUdoeiBBNzIgY29yZS4KPiA+ID4gPiAKPiA+ID4gPiBUaGlz IGlzIGRlZmluaXRlbHkgb3V0IG9mIG15IGV4cGVjdGF0aW9uLCBJIGRvdWJsZSBjaGVja2VkIHRv IGJlCj4gPiA+ID4gc3VyZSB0aGF0IGl0J3MgcnVubmluZyBpbiBLVk0gbW9kZS4KPiA+ID4gPiAK PiA+ID4gPiBCdXQgZnVydGhlciBkaWdnaW5nIHNob3dzIHRoYXQsIHNpbmNlIFhhdmllciBBR1gg Q1BVIHN1cHBvcnRzCj4gPiA+ID4gVkhFLCBrdm0gaXMgcnVubmluZyBpbiBWSEUgbW9kZSBvdGhl ciB0aGFuIEhZUCBtb2RlIG9uIENNNC4KPiA+ID4gPiAKPiA+ID4gPiBJcyB0aGVyZSBhbnl3YXkg dG8gbWFudWFsbHkgZGlzYWJsZSBWSEUgbW9kZSB0byB0ZXN0IHRoZSBtb3JlCj4gPiA+ID4gY29t bW9uIEhZUCBtb2RlIG9uIFhhdmllcj8KPiA+ID4gCj4gPiA+IEFjY29yZGluZyB0byBrZXJuZWwt cGFyYW1ldGVycy50eHQsICJrdm0tYXJtLm1vZGU9bnZoZSIgKG9yIGl0cwo+ID4gPiBsb3ctbGV2 ZWwgZXF1aXZhbGVudCAiaWRfYWE2NG1tZnIxLnZoPTAiKSBvbiB0aGUgY29tbWFuZCBsaW5lIHNo b3VsZAo+ID4gPiBkbyB0aGF0Lgo+ID4gCj4gPiBUaGFua3MgZm9yIHRoaXMgb25lLCBJIHN0dXBp ZGx5IG9ubHkgc2VhcmNoZWQgbW9kaW5mbyBvZiBrdm0sIGFuZCBkaWRuJ3QKPiA+IGV2ZW4gYm90 aGVyIHRvIHNlYXJjaCBhcmNoL2FybTY0L2t2bS4uLgo+ID4gCj4gPiA+IAo+ID4gPiBIb3dldmVy IEknZCBpbWFnaW5lIHRoZSBkaXNjcmVwYW5jeSBpcyBsaWtlbHkgdG8gYmUgc29tZXRoaW5nIG1v cmUKPiA+ID4gZnVuZGFtZW50YWwgdG8gdGhlIHdpbGRseSBkaWZmZXJlbnQgbWljcm9hcmNoaXRl Y3R1cmVzLiBUaGVyZSdzCj4gPiA+IGNlcnRhaW5seSBubyBoYXJtIGluIGdpdmluZyBub24tVkhF IGEgZ28gZm9yIGNvbXBhcmlzb24sIGJ1dCBJCj4gPiA+IHdvdWxkbid0IGJlIHN1cnByaXNlZCBp ZiBpdCB0dXJucyBvdXQgZXZlbiBzbG93ZXIuLi4KPiA+IAo+ID4gWW91J3JlIHRvdGFsbHkgcmln aHQsIHdpdGggbnZoZSBtb2RlLCBpdCdzIHN0aWxsIHRoZSBzYW1lIHNsb3cgc3BlZWQuCj4gPiAK PiA+IEJUVywgd2hhdCBkaWQgeW91IG1lYW4gYnkgdGhlICJ3aWxkbHkgZGlmZmVyZW50IG1pY3Jv YXJjaCI/Cj4gPiBJcyBBUk12OC4yIGFyY2ggdGhhdCBkaWZmZXJlbnQgZnJvbSBBUk12OCBvZiBS UEk0Pwo+IAo+IEkgZG9uJ3QgbWVhbiBBcm12OC54IGFyY2hpdGVjdHVyYWwgZmVhdHVyZXMsIEkg bWVhbiB0aGUgYWN0dWFsCj4gaW1wbGVtZW50YXRpb24gb2YgTlZJRElBJ3MgQ2FybWVsIGNvcmUg aXMgdmVyeSwgdmVyeSBkaWZmZXJlbnQgZnJvbQo+IENvcnRleC1BNzIgb3IgaW5kZWVkIG91ciBu ZXdlciB2OC4yIENvcnRleC1BIGRlc2lnbnMuCj4gCj4gPiBBbmQgYW55IGV4dHJhIG1ldGhvZHMg SSBjb3VsZCB0cnkgdG8gZXhwbG9yZSB0aGUgcmVhc29uIG9mIHRoZSBzbG93bmVzcz8KPiAKPiBJ IGd1ZXNzIHRoZSBmaXJzdCBjaGVjayB3b3VsZCBiZSB3aGV0aGVyIHlvdSdyZSB0cmFwcGluZyBh bmQgZXhpdGluZyB0aGUgVk0KPiBzaWduaWZpY2FudGx5IG1vcmUuIEkgYmVsaWV2ZSB0aGVyZSBh cmUgc3RhdHMgc29tZXdoZXJlLCBidXQgSSBkb24ndCBrbm93Cj4gZXhhY3RseSB3aGVyZSwgc29y cnkgLSBJIGtub3cgdmVyeSBsaXR0bGUgYWJvdXQgYWN0dWFsbHkgKnVzaW5nKiBLVk0gOikKPiAK PiBJZiBpdCdzIG5vdCB0aGF0LCB0aGVuIGl0IG1pZ2h0IGp1c3QgYmUgdGhhdCBFREsyIGlzIGRv aW5nIGEgbG90IG9mIGNhY2hlCj4gbWFpbnRlbmFuY2Ugb3Igc3lzdGVtIHJlZ2lzdGVyIG1vZGlm aWNhdGlvbiBvciBzb21lIG90aGVyIG9wZXJhdGlvbiB0aGF0Cj4gaGFwcGVucyB0byBiZSBzbG93 ZXIgb24gQ2FybWVsIGNvbXBhcmVkIHRvIENvcnRleC1BNzIuCgpJdCB3b3VsZCBhbHNvIGJlIHdv cnRoY2hlY2tpbmcgdGhhIHRoZSBDUFVzIGFyZSBydW5uaW5nIGF0IHRoZSBzcGVlZCB5b3UKZXhw ZWN0LCBpbiBlLmcuIGNhc2UgdGhlIGxhY2sgb2YgYSBEVkZTIGRyaXZlciBtZWFucyB0aGV5J3Jl IHJ1bm5pbmcKc2xvdywgYW5kIHRoaXMganVzdCBoYXBwZW5zIHRvIGJlIG1vcmUgbm90aWNlYWJs ZSBpbiBhIFZNLgoKWW91IGNhbiBlc3RpbWF0ZSB0aGF0IGJ5IHVzaW5nIGBwZXJmIHN0YXRgIG9u IHRoZSBob3N0IG9uIGEgYnVzeSBsb29wLAphbmQgbG9va2luZyB3aGF0IHRoZSBjcHUtY3ljbGVz IGNvdW50IGltcGxpZXMuCgpUaGFua3MsCk1hcmsuCgpfX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fXwpsaW51eC1hcm0ta2VybmVsIG1haWxpbmcgbGlzdApsaW51 eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5v cmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1hcm0ta2VybmVsCg==