archive mirror
 help / color / mirror / Atom feed
From: Akihiko Odaki <>
To: Marc Zyngier <>
	Mathieu Poirier <>,
	Oliver Upton <>,
	Suzuki K Poulose <>,
	Alexandru Elisei <>,
	James Morse <>, Will Deacon <>,
	Catalin Marinas <>,, Alyssa Rosenzweig <>,
	Sven Peter <>, Hector Martin <>
Subject: Re: [PATCH 0/3] KVM: arm64: Handle CCSIDR associativity mismatches
Date: Fri, 2 Dec 2022 02:26:08 +0900	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

On 2022/12/01 20:06, Marc Zyngier wrote:
> On Thu, 01 Dec 2022 10:49:11 +0000,
> Akihiko Odaki <> wrote:
> Thanks for looking into this.
>> M2 MacBook Air has mismatched CCSIDR associativity bits, which makes the
>> bits a KVM vCPU sees inconsistent when migrating.
> Can you describe the actual discrepancy? Is that an issue between the
> two core types? In which case, nothing says that these two cluster
> should have the same cache topology.

Yes, the processor has big.LITTLE configuration.

On the processor, the valid CSSELR values are 0 (L1D), 1 (L1I), 3 (L2D). 
For each CSSELR values, each cluster has:
- 0x700FE03A, 0x203FE01A, 0x70FFE07B
- 0x701FE03A, 0x203FE02A, 0x73FFE07B

>> It also makes QEMU fail restoring the vCPU registers because QEMU saves
>> and restores all of the registers including CCSIDRs, and if the vCPU
>> migrated among physical CPUs between saving and restoring, it tries to
>> restore CCSIDR values that mismatch with the current physical CPU, which
>> causes EFAULT.
> Well, QEMU will have plenty of other problems, starting with MIDRs,
> which always reflect the physical one. In general, KVM isn't well
> geared for VMs spanning multiple CPU types. It is improving, but there
> is a long way to go.

On M2 MacBook Air, I have seen no other difference in standard ID 
registers and CCSIDRs are exceptions. Perhaps Apple designed this way so 
that macOS's Hypervisor can freely migrate vCPU, but I can't assure that 
without more analysis. This is still enough to migrate vCPU running 
Linux at least.

>> Trap CCSIDRs if there are CCSIDR value msimatches, and override the
>> associativity bits when handling the trap.
> TBH, I'd rather we stop reporting this stuff altogether.
> There is nothing a correctly written arm64 guest should do with any of
> this (this is only useful for set/way CMOs, which non-secure SW should
> never issue). It would be a lot better to expose a virtual topology
> (one set, one way, one level). It would also save us from the CCSIDRX
> silliness.
> The only complexity would be to still accept different topologies from
> userspace so that we can restore a VM saved before this virtual
> topology.

Another (minor) concern is that trapping relevant registers may cost too 
much. Currently KVM traps CSSELR and CCSIDR accesses with HCR_TID2, but 
HCR_TID2 also affects CTR_EL0. Although I'm not sure if the register is 
referred frequently, Arm introduced FEAT_EVT to trap CSSELR and CSSIDR 
but not CTR_EL0 so there may be some case where trapping CTR_EL0 is not 
tolerated. Perhaps Arm worried that a userspace application may read 
CTR_EL0 frequently.

If you think the concern on VM restoration you mentioned and the 
trapping overhead is tolerable, I'll write a new, much smaller patch 

Akihiko Odaki

> Do you mind having a look at this?
> Thanks,
> 	M.

  reply	other threads:[~2022-12-01 17:26 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-12-01 10:49 [PATCH 0/3] KVM: arm64: Handle CCSIDR associativity mismatches Akihiko Odaki
2022-12-01 10:49 ` [PATCH 1/3] KVM: arm64: Make CCSIDRs consistent Akihiko Odaki
2022-12-01 10:49 ` [PATCH 2/3] arm64: errata: Check for mismatched cache associativity Akihiko Odaki
2022-12-01 10:49 ` [PATCH 3/3] KVM: arm64: Handle CCSIDR associativity mismatches Akihiko Odaki
2022-12-01 11:06 ` [PATCH 0/3] " Marc Zyngier
2022-12-01 17:26   ` Akihiko Odaki [this message]
2022-12-01 23:13     ` Marc Zyngier
2022-12-02  5:17       ` Akihiko Odaki
2022-12-02  9:40         ` Marc Zyngier
2022-12-02  9:55           ` Akihiko Odaki
2022-12-04 14:57             ` Marc Zyngier
2022-12-11  5:25               ` Akihiko Odaki
2022-12-11 10:21                 ` Marc Zyngier
2022-12-11 10:44                   ` Akihiko Odaki
2022-12-01 18:29   ` Oliver Upton
2022-12-01 23:14     ` Marc Zyngier
2022-12-02 18:54       ` Oliver Upton

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).