From: Vikram Sethi <vsethi@nvidia.com>
To: Marc Zyngier <maz@kernel.org>
Cc: Shanker Donthineni <sdonthineni@nvidia.com>,
Alex Williamson <alex.williamson@redhat.com>,
Will Deacon <will@kernel.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Christoffer Dall <christoffer.dall@arm.com>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>,
"kvmarm@lists.cs.columbia.edu" <kvmarm@lists.cs.columbia.edu>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
Jason Sequeira <jsequeira@nvidia.com>
Subject: RE: [RFC 1/2] vfio/pci: keep the prefetchable attribute of a BAR region in VMA
Date: Sun, 2 May 2021 17:56:31 +0000 [thread overview]
Message-ID: <BL0PR12MB25329EF5DFA7BBAA732064A7BD5C9@BL0PR12MB2532.namprd12.prod.outlook.com> (raw)
In-Reply-To: <87eeeqvm1d.wl-maz@kernel.org>
Hi Marc,
> From: Marc Zyngier <maz@kernel.org>
> Hi Vikram,
>
> The problem I see is that we have VM and userspace being written in terms
> of Write-Combine, which is:
>
> - loosely defined even on x86
>
> - subject to interpretations in the way it maps to PCI
>
> - has no direct equivalent in the ARMv8 collection of memory
> attributes (and Normal_NC comes with speculation capabilities which
> strikes me as extremely undesirable on arbitrary devices)
If speculation with Normal NC to prefetchable BARs in devices was a problem,
those devices would already be broken in baremetal with ioremap_wc on arm64,
and we would need quirks there to not do Normal NC for them but Device GRE,
and if such a quirk was needed on baremetal, it could be picked up by vfio/KVM
as well. But we haven't seen any broken devices doing wc on baremetal on ARM64, have we?
I know we have tested NICs write combining on arm64 in baremetal, as well as GPU
and NVMe CMB without issues.
Further, I don't see why speculation to non cacheble would be an issue if prefetch
without side effects is allowed by the device, which is what a prefetchable BAR is.
If it is an issue for a device I would consider that a bug already needing a quirk in
Baremetal/host kernel already.
From PCI spec " A prefetchable address range may have write side effects,
but it may not have read side effects."
>
> How do we translate this into something consistent? I'd like to see an actual
> description of what we *really* expect from WC on prefetchable PCI regions,
> turn that into a documented definition agreed across architectures, and then
> we can look at implementing it with one memory type or another on arm64.
>
> Because once we expose that memory type at S2 for KVM guests, it
> becomes ABI and there is no turning back. So I want to get it right once and
> for all.
>
I agree that we need a precise definition for the Linux ioremap_wc API wrt what
drivers (kernel and userspace) can expect and whether memset/memcpy is expected
to work or not and whether aligned accesses are a requirement.
To the extent ABI is set, I would think that the ABI is also already set in the host kernel
for arm64 WC = Normal NC, so why should that not also be the ABI for same driver in VMs.
> Thanks,
>
> M.
>
> --
> Without deviation from the norm, progress is not possible.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2021-05-02 17:58 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-29 16:29 [RFC 0/2] [RFC] Honor PCI prefetchable attributes for a virtual machine on ARM64 Shanker Donthineni
2021-04-29 16:29 ` [RFC 1/2] vfio/pci: keep the prefetchable attribute of a BAR region in VMA Shanker Donthineni
2021-04-29 18:28 ` Alex Williamson
2021-04-29 19:14 ` Shanker R Donthineni
2021-04-29 19:46 ` Alex Williamson
2021-04-29 22:08 ` Vikram Sethi
2021-04-30 11:25 ` Shanker R Donthineni
2021-04-30 11:47 ` Marc Zyngier
2021-04-30 13:07 ` Shanker R Donthineni
2021-04-30 15:07 ` Marc Zyngier
2021-04-30 14:58 ` Shanker R Donthineni
2021-04-30 15:31 ` Marc Zyngier
2021-04-30 16:57 ` Vikram Sethi
2021-05-01 9:30 ` Marc Zyngier
2021-05-01 11:36 ` Shanker R Donthineni
2021-05-03 9:50 ` Marc Zyngier
2021-05-03 12:08 ` Shanker R Donthineni
2021-05-02 17:56 ` Vikram Sethi [this message]
2021-05-03 10:17 ` Marc Zyngier
2021-05-03 13:35 ` Mark Kettenis
2021-05-03 13:59 ` Vikram Sethi
2021-05-03 14:44 ` Alex Williamson
2021-05-03 22:03 ` Vikram Sethi
2021-05-04 8:30 ` Will Deacon
2021-05-05 18:02 ` Catalin Marinas
2021-05-06 7:22 ` Christoph Hellwig
2021-05-08 16:33 ` Shanker R Donthineni
2021-06-02 9:37 ` Marc Zyngier
2021-05-04 18:03 ` Alex Williamson
2021-06-02 9:11 ` Marc Zyngier
2021-04-30 9:54 ` Lorenzo Pieralisi
2021-04-30 12:38 ` Jason Gunthorpe
2021-04-29 16:29 ` [RFC 2/2] KVM: arm64: Add write-combine support for stage-2 entries Shanker Donthineni
2021-05-03 7:01 ` [RFC 0/2] [RFC] Honor PCI prefetchable attributes for a virtual machine on ARM64 Christoph Hellwig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=BL0PR12MB25329EF5DFA7BBAA732064A7BD5C9@BL0PR12MB2532.namprd12.prod.outlook.com \
--to=vsethi@nvidia.com \
--cc=alex.williamson@redhat.com \
--cc=catalin.marinas@arm.com \
--cc=christoffer.dall@arm.com \
--cc=jsequeira@nvidia.com \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=maz@kernel.org \
--cc=sdonthineni@nvidia.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).