From: Shanker Donthineni <sdonthineni@nvidia.com>
To: Marc Zyngier <maz@kernel.org>,
Alex Williamson <alex.williamson@redhat.com>
Cc: Will Deacon <will@kernel.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Christoffer Dall <christoffer.dall@arm.com>,
<linux-arm-kernel@lists.infradead.org>,
<kvmarm@lists.cs.columbia.edu>, <linux-kernel@vger.kernel.org>,
<kvm@vger.kernel.org>, Vikram Sethi <vsethi@nvidia.com>,
Shanker Donthineni <sdonthineni@nvidia.com>,
"Jason Sequeira" <jsequeira@nvidia.com>
Subject: [RFC 0/2] [RFC] Honor PCI prefetchable attributes for a virtual machine on ARM64
Date: Thu, 29 Apr 2021 11:29:04 -0500 [thread overview]
Message-ID: <20210429162906.32742-1-sdonthineni@nvidia.com> (raw)
Problem statement: Virtual machine crashes when NVIDIA GPU driver access a prefetchable BAR space due to the unaligned reads/writes for pass-through devices. The same binary works fine as expected in the host kernel. Only one BAR has control & status registers (CSR) and other PCI BARs are marked as prefetchable. NVIDIA GPU driver uses the write-combine feature for mapping the prefetchable BARs to improve performance. This problem applies to all other drivers which want to enable WC.
Solution: Honor PCI prefetchable attributes for the guest operating systems.
Proposal: ARM64-KVM uses VMA struct for the needed information e.g. region physical address, size, and memory-type (struct page backed mapping or anonymous memory) for setting up a stage-2 page table. Right now memory region either can be mapped as DEVICE (strongly ordered) or NORMAL (write-back cache) depends on the flag VM_PFNMAP in VMA. VFIO-PCI will keep the prefetchable (write-combine) information in vma->vm_page_prot similar to other fields, and KVM will prepare stage-2 entries based on the memory-type attribute that was set in VMA.
Shanker Donthineni (2):
vfio/pci: keep the prefetchable attribute of a BAR region in VMA
KVM: arm64: Add write-combine support for stage-2 entries
arch/arm64/include/asm/kvm_mmu.h | 3 ++-
arch/arm64/include/asm/kvm_pgtable.h | 2 ++
arch/arm64/include/asm/memory.h | 4 +++-
arch/arm64/kvm/hyp/pgtable.c | 9 +++++++--
arch/arm64/kvm/mmu.c | 22 +++++++++++++++++++---
arch/arm64/kvm/vgic/vgic-v2.c | 2 +-
drivers/vfio/pci/vfio_pci.c | 6 +++++-
7 files changed, 39 insertions(+), 9 deletions(-)
--
2.17.1
WARNING: multiple messages have this Message-ID (diff)
From: Shanker Donthineni <sdonthineni@nvidia.com>
To: Marc Zyngier <maz@kernel.org>,
Alex Williamson <alex.williamson@redhat.com>
Cc: Jason Sequeira <jsequeira@nvidia.com>,
kvm@vger.kernel.org, Catalin Marinas <catalin.marinas@arm.com>,
linux-kernel@vger.kernel.org, Vikram Sethi <vsethi@nvidia.com>,
Shanker Donthineni <sdonthineni@nvidia.com>,
Will Deacon <will@kernel.org>,
kvmarm@lists.cs.columbia.edu,
linux-arm-kernel@lists.infradead.org
Subject: [RFC 0/2] [RFC] Honor PCI prefetchable attributes for a virtual machine on ARM64
Date: Thu, 29 Apr 2021 11:29:04 -0500 [thread overview]
Message-ID: <20210429162906.32742-1-sdonthineni@nvidia.com> (raw)
Problem statement: Virtual machine crashes when NVIDIA GPU driver access a prefetchable BAR space due to the unaligned reads/writes for pass-through devices. The same binary works fine as expected in the host kernel. Only one BAR has control & status registers (CSR) and other PCI BARs are marked as prefetchable. NVIDIA GPU driver uses the write-combine feature for mapping the prefetchable BARs to improve performance. This problem applies to all other drivers which want to enable WC.
Solution: Honor PCI prefetchable attributes for the guest operating systems.
Proposal: ARM64-KVM uses VMA struct for the needed information e.g. region physical address, size, and memory-type (struct page backed mapping or anonymous memory) for setting up a stage-2 page table. Right now memory region either can be mapped as DEVICE (strongly ordered) or NORMAL (write-back cache) depends on the flag VM_PFNMAP in VMA. VFIO-PCI will keep the prefetchable (write-combine) information in vma->vm_page_prot similar to other fields, and KVM will prepare stage-2 entries based on the memory-type attribute that was set in VMA.
Shanker Donthineni (2):
vfio/pci: keep the prefetchable attribute of a BAR region in VMA
KVM: arm64: Add write-combine support for stage-2 entries
arch/arm64/include/asm/kvm_mmu.h | 3 ++-
arch/arm64/include/asm/kvm_pgtable.h | 2 ++
arch/arm64/include/asm/memory.h | 4 +++-
arch/arm64/kvm/hyp/pgtable.c | 9 +++++++--
arch/arm64/kvm/mmu.c | 22 +++++++++++++++++++---
arch/arm64/kvm/vgic/vgic-v2.c | 2 +-
drivers/vfio/pci/vfio_pci.c | 6 +++++-
7 files changed, 39 insertions(+), 9 deletions(-)
--
2.17.1
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
WARNING: multiple messages have this Message-ID (diff)
From: Shanker Donthineni <sdonthineni@nvidia.com>
To: Marc Zyngier <maz@kernel.org>,
Alex Williamson <alex.williamson@redhat.com>
Cc: Will Deacon <will@kernel.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Christoffer Dall <christoffer.dall@arm.com>,
<linux-arm-kernel@lists.infradead.org>,
<kvmarm@lists.cs.columbia.edu>, <linux-kernel@vger.kernel.org>,
<kvm@vger.kernel.org>, Vikram Sethi <vsethi@nvidia.com>,
Shanker Donthineni <sdonthineni@nvidia.com>,
"Jason Sequeira" <jsequeira@nvidia.com>
Subject: [RFC 0/2] [RFC] Honor PCI prefetchable attributes for a virtual machine on ARM64
Date: Thu, 29 Apr 2021 11:29:04 -0500 [thread overview]
Message-ID: <20210429162906.32742-1-sdonthineni@nvidia.com> (raw)
Problem statement: Virtual machine crashes when NVIDIA GPU driver access a prefetchable BAR space due to the unaligned reads/writes for pass-through devices. The same binary works fine as expected in the host kernel. Only one BAR has control & status registers (CSR) and other PCI BARs are marked as prefetchable. NVIDIA GPU driver uses the write-combine feature for mapping the prefetchable BARs to improve performance. This problem applies to all other drivers which want to enable WC.
Solution: Honor PCI prefetchable attributes for the guest operating systems.
Proposal: ARM64-KVM uses VMA struct for the needed information e.g. region physical address, size, and memory-type (struct page backed mapping or anonymous memory) for setting up a stage-2 page table. Right now memory region either can be mapped as DEVICE (strongly ordered) or NORMAL (write-back cache) depends on the flag VM_PFNMAP in VMA. VFIO-PCI will keep the prefetchable (write-combine) information in vma->vm_page_prot similar to other fields, and KVM will prepare stage-2 entries based on the memory-type attribute that was set in VMA.
Shanker Donthineni (2):
vfio/pci: keep the prefetchable attribute of a BAR region in VMA
KVM: arm64: Add write-combine support for stage-2 entries
arch/arm64/include/asm/kvm_mmu.h | 3 ++-
arch/arm64/include/asm/kvm_pgtable.h | 2 ++
arch/arm64/include/asm/memory.h | 4 +++-
arch/arm64/kvm/hyp/pgtable.c | 9 +++++++--
arch/arm64/kvm/mmu.c | 22 +++++++++++++++++++---
arch/arm64/kvm/vgic/vgic-v2.c | 2 +-
drivers/vfio/pci/vfio_pci.c | 6 +++++-
7 files changed, 39 insertions(+), 9 deletions(-)
--
2.17.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next reply other threads:[~2021-04-29 16:29 UTC|newest]
Thread overview: 99+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-29 16:29 Shanker Donthineni [this message]
2021-04-29 16:29 ` [RFC 0/2] [RFC] Honor PCI prefetchable attributes for a virtual machine on ARM64 Shanker Donthineni
2021-04-29 16:29 ` Shanker Donthineni
2021-04-29 16:29 ` [RFC 1/2] vfio/pci: keep the prefetchable attribute of a BAR region in VMA Shanker Donthineni
2021-04-29 16:29 ` Shanker Donthineni
2021-04-29 16:29 ` Shanker Donthineni
2021-04-29 18:28 ` Alex Williamson
2021-04-29 18:28 ` Alex Williamson
2021-04-29 18:28 ` Alex Williamson
2021-04-29 19:14 ` Shanker R Donthineni
2021-04-29 19:14 ` Shanker R Donthineni
2021-04-29 19:14 ` Shanker R Donthineni
2021-04-29 19:46 ` Alex Williamson
2021-04-29 19:46 ` Alex Williamson
2021-04-29 19:46 ` Alex Williamson
2021-04-29 22:08 ` Vikram Sethi
2021-04-29 22:08 ` Vikram Sethi
2021-04-29 22:08 ` Vikram Sethi
2021-04-30 11:25 ` Shanker R Donthineni
2021-04-30 11:25 ` Shanker R Donthineni
2021-04-30 11:25 ` Shanker R Donthineni
2021-04-30 11:47 ` Marc Zyngier
2021-04-30 11:47 ` Marc Zyngier
2021-04-30 13:07 ` Shanker R Donthineni
2021-04-30 13:07 ` Shanker R Donthineni
2021-04-30 13:07 ` Shanker R Donthineni
2021-04-30 15:07 ` Marc Zyngier
2021-04-30 15:07 ` Marc Zyngier
2021-04-30 14:58 ` Shanker R Donthineni
2021-04-30 14:58 ` Shanker R Donthineni
2021-04-30 14:58 ` Shanker R Donthineni
2021-04-30 15:31 ` Marc Zyngier
2021-04-30 15:31 ` Marc Zyngier
2021-04-30 16:57 ` Vikram Sethi
2021-04-30 16:57 ` Vikram Sethi
2021-04-30 16:57 ` Vikram Sethi
2021-05-01 9:30 ` Marc Zyngier
2021-05-01 9:30 ` Marc Zyngier
2021-05-01 9:30 ` Marc Zyngier
2021-05-01 11:36 ` Shanker R Donthineni
2021-05-01 11:36 ` Shanker R Donthineni
2021-05-01 11:36 ` Shanker R Donthineni
2021-05-03 9:50 ` Marc Zyngier
2021-05-03 9:50 ` Marc Zyngier
2021-05-03 12:08 ` Shanker R Donthineni
2021-05-03 12:08 ` Shanker R Donthineni
2021-05-03 12:08 ` Shanker R Donthineni
2021-05-02 17:56 ` Vikram Sethi
2021-05-02 17:56 ` Vikram Sethi
2021-05-02 17:56 ` Vikram Sethi
2021-05-03 10:17 ` Marc Zyngier
2021-05-03 10:17 ` Marc Zyngier
2021-05-03 10:17 ` Marc Zyngier
2021-05-03 13:35 ` Mark Kettenis
2021-05-03 13:35 ` Mark Kettenis
2021-05-03 13:35 ` Mark Kettenis
2021-05-03 13:59 ` Vikram Sethi
2021-05-03 13:59 ` Vikram Sethi
2021-05-03 13:59 ` Vikram Sethi
2021-05-03 14:44 ` Alex Williamson
2021-05-03 14:44 ` Alex Williamson
2021-05-03 14:44 ` Alex Williamson
2021-05-03 22:03 ` Vikram Sethi
2021-05-03 22:03 ` Vikram Sethi
2021-05-03 22:03 ` Vikram Sethi
2021-05-04 8:30 ` Will Deacon
2021-05-04 8:30 ` Will Deacon
2021-05-04 8:30 ` Will Deacon
2021-05-05 18:02 ` Catalin Marinas
2021-05-05 18:02 ` Catalin Marinas
2021-05-05 18:02 ` Catalin Marinas
2021-05-06 7:22 ` Christoph Hellwig
2021-05-06 7:22 ` Christoph Hellwig
2021-05-06 7:22 ` Christoph Hellwig
2021-05-08 16:33 ` Shanker R Donthineni
2021-05-08 16:33 ` Shanker R Donthineni
2021-05-08 16:33 ` Shanker R Donthineni
2021-06-02 9:37 ` Marc Zyngier
2021-06-02 9:37 ` Marc Zyngier
2021-06-02 9:37 ` Marc Zyngier
2021-05-04 18:03 ` Alex Williamson
2021-05-04 18:03 ` Alex Williamson
2021-05-04 18:03 ` Alex Williamson
2021-06-02 9:11 ` Marc Zyngier
2021-06-02 9:11 ` Marc Zyngier
2021-06-02 9:11 ` Marc Zyngier
2021-04-30 9:54 ` Lorenzo Pieralisi
2021-04-30 9:54 ` Lorenzo Pieralisi
2021-04-30 9:54 ` Lorenzo Pieralisi
2021-04-30 12:38 ` Jason Gunthorpe
2021-04-30 12:38 ` Jason Gunthorpe
2021-04-30 12:38 ` Jason Gunthorpe
2021-04-29 16:29 ` [RFC 2/2] KVM: arm64: Add write-combine support for stage-2 entries Shanker Donthineni
2021-04-29 16:29 ` Shanker Donthineni
2021-04-29 16:29 ` Shanker Donthineni
2021-04-29 19:34 ` kernel test robot
2021-05-03 7:01 ` [RFC 0/2] [RFC] Honor PCI prefetchable attributes for a virtual machine on ARM64 Christoph Hellwig
2021-05-03 7:01 ` Christoph Hellwig
2021-05-03 7:01 ` Christoph Hellwig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210429162906.32742-1-sdonthineni@nvidia.com \
--to=sdonthineni@nvidia.com \
--cc=alex.williamson@redhat.com \
--cc=catalin.marinas@arm.com \
--cc=christoffer.dall@arm.com \
--cc=jsequeira@nvidia.com \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=maz@kernel.org \
--cc=vsethi@nvidia.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.