All of lore.kernel.org
 help / color / mirror / Atom feed
From: Qais Yousef <qais.yousef@arm.com>
To: Will Deacon <will@kernel.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>,
	"linux-arm-kernel@lists.infradead.org" 
	<linux-arm-kernel@lists.infradead.org>,
	"linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	Marc Zyngier <maz@kernel.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Morten Rasmussen <Morten.Rasmussen@arm.com>,
	Suren Baghdasaryan <surenb@google.com>,
	"kernel-team@android.com" <kernel-team@android.com>
Subject: Re: [PATCH 2/6] arm64: Allow mismatched 32-bit EL0 support
Date: Thu, 12 Nov 2020 11:55:55 +0000	[thread overview]
Message-ID: <20201112115555.65sfsod6uf6xm5gy@e107158-lin.cambridge.arm.com> (raw)
In-Reply-To: <20201112102424.GB19506@willie-the-truck>

On 11/12/20 10:24, Will Deacon wrote:
> On Wed, Nov 11, 2020 at 04:27:00PM +0000, Qais Yousef wrote:
> > On 11/09/20 13:52, Will Deacon wrote:
> > > On Fri, Nov 06, 2020 at 02:48:35PM +0000, Qais Yousef wrote:
> > > > On 11/06/20 13:00, Will Deacon wrote:
> > > > > On Fri, Nov 06, 2020 at 12:54:25PM +0000, Qais Yousef wrote:
> > > > > > FWIW I have my v3 over here in case it's of any help. It solves the problem of
> > > > > > HWCAP discovery when late AArch32 CPU is booted by populating boot_cpu_date
> > > > > > with 32bit features then.
> > > > > > 
> > > > > > 	git clone https://git.gitlab.arm.com/linux-arm/linux-qy.git -b asym-aarch32-upstream-v3 origin/asym-aarch32-upstream-v3
> > > > > 
> > > > > Cheers, I've done something similar. I was hoping to post it today, but I've
> > > > > been side-tracked with bug fixing this morning. The main headache I ended up
> > > > > with was allowing late-onlining of 64-bit-only CPUs if all the boot CPUs
> > > > > are 32-bit capable. What do you do in that case?
> > > > 
> > > > Do you mean if CPUs 0-3 were 32bit capable and we boot with maxcpus=4 then
> > > > attempt to bring the remaining 64bit-only cpus online later?
> > > 
> > > Right. I think we will refuse to online them. I'll post my attempt at
> > > handling that shortly.
> > 
> > Sorry for the delayed response.
> > 
> > You're right, I tried that and they refuse to come online. We missed that tbh.
> > 
> > Haven't thought what we should do yet. I tried your v2 and it failed similarly.
> 
> Hmm, it shouldn't do. Please could you provide the log? My hunch is that you
> are blatting 32-bit EL1 support as well, and we can't handle a mismatch for
> that with a late CPU. Do you know if the CPUs being integrated into these
> broken designs have a mismatch at EL1 as well?

Hmm my test could have been invalid then. We shouldn't have mismatch at EL1,
for ease of testing I used a hacked up patch to fake asymmetry on Juno. Testing
on FVP now, it takes time to boot up though..

Let me re-run this and get you the log from proper environment. Assuming it
still fails.

> > I usually have a similar hunk in my testing to check how the kernel perceives
> > the 32bit support when I execute a binary:
> > 
> > 	diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> > 	index f447d313a9c5..a9549e55a6c8 100644
> > 	--- a/arch/arm64/include/asm/cpufeature.h
> > 	+++ b/arch/arm64/include/asm/cpufeature.h
> > 	@@ -611,6 +611,9 @@ static inline bool system_supports_32bit_el0(void)
> > 	 {
> > 		u64 pfr0 = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
> > 
> > 	+       pr_err("System supports symmetric 32bit el0: %d\n", id_aa64pfr0_32bit_el0(pfr0));
> > 	+       pr_err("System supports Asymmetric 32bit el0: %ld\n", static_branch_unlikely(&arm64_mismatched_32bit_el0));
> > 	+
> > 		return id_aa64pfr0_32bit_el0(pfr0) ||
> > 		       static_branch_unlikely(&arm64_mismatched_32bit_el0);
> > 	 }
> > 
> > In your v2 both conditions are true. In my series we see the system as
> > symmetric if we boot the 32bit capable cpus _only_.
> 
> The "arm64_mismatched_32bit_el0" key drives both the creation of the sysfs
> file and the allocation of the cpu mask. See the comment in cpufeature.c
> That file should be created whenever the command-line is passed to enable
> this feature, because a late CPU could come up and set bits in there. The
> presence of the file can therefore inform userspace that this can happen.

Okay. I just didn't expect both to return true here. It's not a bug per se.
It's just slightly misleading for arm64_mismatched_32bit_el0 to be true when
the system is symmetric.

Thanks

--
Qais Yousef

WARNING: multiple messages have this Message-ID (diff)
From: Qais Yousef <qais.yousef@arm.com>
To: Will Deacon <will@kernel.org>
Cc: "linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
	Marc Zyngier <maz@kernel.org>,
	"kernel-team@android.com" <kernel-team@android.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Suren Baghdasaryan <surenb@google.com>,
	Morten Rasmussen <Morten.Rasmussen@arm.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>
Subject: Re: [PATCH 2/6] arm64: Allow mismatched 32-bit EL0 support
Date: Thu, 12 Nov 2020 11:55:55 +0000	[thread overview]
Message-ID: <20201112115555.65sfsod6uf6xm5gy@e107158-lin.cambridge.arm.com> (raw)
In-Reply-To: <20201112102424.GB19506@willie-the-truck>

On 11/12/20 10:24, Will Deacon wrote:
> On Wed, Nov 11, 2020 at 04:27:00PM +0000, Qais Yousef wrote:
> > On 11/09/20 13:52, Will Deacon wrote:
> > > On Fri, Nov 06, 2020 at 02:48:35PM +0000, Qais Yousef wrote:
> > > > On 11/06/20 13:00, Will Deacon wrote:
> > > > > On Fri, Nov 06, 2020 at 12:54:25PM +0000, Qais Yousef wrote:
> > > > > > FWIW I have my v3 over here in case it's of any help. It solves the problem of
> > > > > > HWCAP discovery when late AArch32 CPU is booted by populating boot_cpu_date
> > > > > > with 32bit features then.
> > > > > > 
> > > > > > 	git clone https://git.gitlab.arm.com/linux-arm/linux-qy.git -b asym-aarch32-upstream-v3 origin/asym-aarch32-upstream-v3
> > > > > 
> > > > > Cheers, I've done something similar. I was hoping to post it today, but I've
> > > > > been side-tracked with bug fixing this morning. The main headache I ended up
> > > > > with was allowing late-onlining of 64-bit-only CPUs if all the boot CPUs
> > > > > are 32-bit capable. What do you do in that case?
> > > > 
> > > > Do you mean if CPUs 0-3 were 32bit capable and we boot with maxcpus=4 then
> > > > attempt to bring the remaining 64bit-only cpus online later?
> > > 
> > > Right. I think we will refuse to online them. I'll post my attempt at
> > > handling that shortly.
> > 
> > Sorry for the delayed response.
> > 
> > You're right, I tried that and they refuse to come online. We missed that tbh.
> > 
> > Haven't thought what we should do yet. I tried your v2 and it failed similarly.
> 
> Hmm, it shouldn't do. Please could you provide the log? My hunch is that you
> are blatting 32-bit EL1 support as well, and we can't handle a mismatch for
> that with a late CPU. Do you know if the CPUs being integrated into these
> broken designs have a mismatch at EL1 as well?

Hmm my test could have been invalid then. We shouldn't have mismatch at EL1,
for ease of testing I used a hacked up patch to fake asymmetry on Juno. Testing
on FVP now, it takes time to boot up though..

Let me re-run this and get you the log from proper environment. Assuming it
still fails.

> > I usually have a similar hunk in my testing to check how the kernel perceives
> > the 32bit support when I execute a binary:
> > 
> > 	diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> > 	index f447d313a9c5..a9549e55a6c8 100644
> > 	--- a/arch/arm64/include/asm/cpufeature.h
> > 	+++ b/arch/arm64/include/asm/cpufeature.h
> > 	@@ -611,6 +611,9 @@ static inline bool system_supports_32bit_el0(void)
> > 	 {
> > 		u64 pfr0 = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
> > 
> > 	+       pr_err("System supports symmetric 32bit el0: %d\n", id_aa64pfr0_32bit_el0(pfr0));
> > 	+       pr_err("System supports Asymmetric 32bit el0: %ld\n", static_branch_unlikely(&arm64_mismatched_32bit_el0));
> > 	+
> > 		return id_aa64pfr0_32bit_el0(pfr0) ||
> > 		       static_branch_unlikely(&arm64_mismatched_32bit_el0);
> > 	 }
> > 
> > In your v2 both conditions are true. In my series we see the system as
> > symmetric if we boot the 32bit capable cpus _only_.
> 
> The "arm64_mismatched_32bit_el0" key drives both the creation of the sysfs
> file and the allocation of the cpu mask. See the comment in cpufeature.c
> That file should be created whenever the command-line is passed to enable
> this feature, because a late CPU could come up and set bits in there. The
> presence of the file can therefore inform userspace that this can happen.

Okay. I just didn't expect both to return true here. It's not a bug per se.
It's just slightly misleading for arm64_mismatched_32bit_el0 to be true when
the system is symmetric.

Thanks

--
Qais Yousef

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

  reply	other threads:[~2020-11-12 11:56 UTC|newest]

Thread overview: 96+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-27 21:51 [PATCH 0/6] An alternative series for asymmetric AArch32 systems Will Deacon
2020-10-27 21:51 ` Will Deacon
2020-10-27 21:51 ` [PATCH 1/6] KVM: arm64: Handle Asymmetric " Will Deacon
2020-10-27 21:51   ` Will Deacon
2020-10-27 21:51 ` [PATCH 2/6] arm64: Allow mismatched 32-bit EL0 support Will Deacon
2020-10-27 21:51   ` Will Deacon
2020-10-28 11:12   ` Catalin Marinas
2020-10-28 11:12     ` Catalin Marinas
2020-10-28 11:17     ` Will Deacon
2020-10-28 11:17       ` Will Deacon
2020-10-28 11:22       ` Catalin Marinas
2020-10-28 11:22         ` Catalin Marinas
2020-10-28 11:23         ` Will Deacon
2020-10-28 11:23           ` Will Deacon
2020-10-28 11:49           ` Catalin Marinas
2020-10-28 11:49             ` Catalin Marinas
2020-10-28 12:40             ` Will Deacon
2020-10-28 12:40               ` Will Deacon
2020-10-28 18:56               ` Catalin Marinas
2020-10-28 18:56                 ` Catalin Marinas
2020-10-29 22:20                 ` Will Deacon
2020-10-29 22:20                   ` Will Deacon
2020-10-30 11:18                   ` Catalin Marinas
2020-10-30 11:18                     ` Catalin Marinas
2020-10-30 16:13                     ` Will Deacon
2020-10-30 16:13                       ` Will Deacon
2020-11-02 11:44                       ` Catalin Marinas
2020-11-02 11:44                         ` Catalin Marinas
2020-11-05 21:38                         ` Will Deacon
2020-11-05 21:38                           ` Will Deacon
2020-11-06 12:54                           ` Qais Yousef
2020-11-06 12:54                             ` Qais Yousef
2020-11-06 13:00                             ` Will Deacon
2020-11-06 13:00                               ` Will Deacon
2020-11-06 14:48                               ` Qais Yousef
2020-11-06 14:48                                 ` Qais Yousef
2020-11-09 13:52                                 ` Will Deacon
2020-11-09 13:52                                   ` Will Deacon
2020-11-11 16:27                                   ` Qais Yousef
2020-11-11 16:27                                     ` Qais Yousef
2020-11-12 10:24                                     ` Will Deacon
2020-11-12 10:24                                       ` Will Deacon
2020-11-12 11:55                                       ` Qais Yousef [this message]
2020-11-12 11:55                                         ` Qais Yousef
2020-11-12 16:49                                         ` Qais Yousef
2020-11-12 16:49                                           ` Qais Yousef
2020-11-12 17:06                                           ` Marc Zyngier
2020-11-12 17:06                                             ` Marc Zyngier
2020-11-12 17:36                                             ` Qais Yousef
2020-11-12 17:36                                               ` Qais Yousef
2020-11-12 17:44                                               ` Will Deacon
2020-11-12 17:44                                                 ` Will Deacon
2020-11-12 17:36                                           ` Will Deacon
2020-11-12 17:36                                             ` Will Deacon
2020-11-13 10:45                                             ` Qais Yousef
2020-11-13 10:45                                               ` Qais Yousef
2020-11-06 14:30                           ` Catalin Marinas
2020-11-06 14:30                             ` Catalin Marinas
2020-10-28 11:18   ` Catalin Marinas
2020-10-28 11:18     ` Catalin Marinas
2020-10-28 11:21     ` Will Deacon
2020-10-28 11:21       ` Will Deacon
2020-10-27 21:51 ` [PATCH 3/6] KVM: arm64: Kill 32-bit vCPUs on systems with mismatched " Will Deacon
2020-10-27 21:51   ` Will Deacon
2020-10-27 21:51 ` [PATCH 4/6] arm64: Kill 32-bit applications scheduled on 64-bit-only CPUs Will Deacon
2020-10-27 21:51   ` Will Deacon
2020-10-28 12:10   ` Catalin Marinas
2020-10-28 12:10     ` Catalin Marinas
2020-10-28 12:36     ` Will Deacon
2020-10-28 12:36       ` Will Deacon
2020-10-27 21:51 ` [PATCH 5/6] arm64: Advertise CPUs capable of running 32-bit applcations in sysfs Will Deacon
2020-10-27 21:51   ` Will Deacon
2020-10-28  8:37   ` Greg Kroah-Hartman
2020-10-28  8:37     ` Greg Kroah-Hartman
2020-10-28  9:51     ` Will Deacon
2020-10-28  9:51       ` Will Deacon
2020-10-28 12:15   ` Catalin Marinas
2020-10-28 12:15     ` Catalin Marinas
2020-10-28 12:27     ` Will Deacon
2020-10-28 12:27       ` Will Deacon
2020-10-28 15:14       ` Catalin Marinas
2020-10-28 15:14         ` Catalin Marinas
2020-10-28 15:35         ` Will Deacon
2020-10-28 15:35           ` Will Deacon
2020-10-27 21:51 ` [PATCH 6/6] arm64: Hook up cmdline parameter to allow mismatched 32-bit EL0 Will Deacon
2020-10-27 21:51   ` Will Deacon
2020-10-29 18:42 ` [PATCH 0/6] An alternative series for asymmetric AArch32 systems Suren Baghdasaryan
2020-10-29 18:42   ` Suren Baghdasaryan
2020-10-29 22:17   ` Will Deacon
2020-10-29 22:17     ` Will Deacon
2020-10-30 16:16 ` Marc Zyngier
2020-10-30 16:16   ` Marc Zyngier
2020-10-30 16:24   ` Will Deacon
2020-10-30 16:24     ` Will Deacon
2020-10-30 17:04     ` Marc Zyngier
2020-10-30 17:04       ` Marc Zyngier

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201112115555.65sfsod6uf6xm5gy@e107158-lin.cambridge.arm.com \
    --to=qais.yousef@arm.com \
    --cc=Morten.Rasmussen@arm.com \
    --cc=catalin.marinas@arm.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=kernel-team@android.com \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=maz@kernel.org \
    --cc=peterz@infradead.org \
    --cc=surenb@google.com \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.