From: <ankita@nvidia.com> To: <ankita@nvidia.com>, <jgg@nvidia.com>, <maz@kernel.org>, <oliver.upton@linux.dev>, <james.morse@arm.com>, <suzuki.poulose@arm.com>, <yuzenghui@huawei.com>, <reinette.chatre@intel.com>, <surenb@google.com>, <stefanha@redhat.com>, <brauner@kernel.org>, <catalin.marinas@arm.com>, <will@kernel.org>, <mark.rutland@arm.com>, <alex.williamson@redhat.com>, <kevin.tian@intel.com>, <yi.l.liu@intel.com>, <ardb@kernel.org>, <akpm@linux-foundation.org>, <andreyknvl@gmail.com>, <wangjinchao@xfusion.com>, <gshan@redhat.com>, <ricarkol@google.com>, <linux-mm@kvack.org>, <lpieralisi@kernel.org>, <rananta@google.com>, <ryan.roberts@arm.com> Cc: <aniketa@nvidia.com>, <cjia@nvidia.com>, <kwankhede@nvidia.com>, <targupta@nvidia.com>, <vsethi@nvidia.com>, <acurrid@nvidia.com>, <apopple@nvidia.com>, <jhubbard@nvidia.com>, <danw@nvidia.com>, <kvmarm@lists.linux.dev>, <mochs@nvidia.com>, <zhiw@nvidia.com>, <kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>, <linux-arm-kernel@lists.infradead.org> Subject: [PATCH v6 3/4] kvm: arm64: set io memory s2 pte as normalnc for vfio pci device Date: Thu, 8 Feb 2024 02:16:51 +0530 [thread overview] Message-ID: <20240207204652.22954-4-ankita@nvidia.com> (raw) In-Reply-To: <20240207204652.22954-1-ankita@nvidia.com> From: Ankit Agrawal <ankita@nvidia.com> To provide VM with the ability to get device IO memory with NormalNC property, map device MMIO in KVM for ARM64 at stage2 as NormalNC. Having NormalNC S2 default puts guests in control (based on [1], "Combining stage 1 and stage 2 memory type attributes") of device MMIO regions memory mappings. The rules are summarized below: ([(S1) - stage1], [(S2) - stage 2]) S1 | S2 | Result NORMAL-WB | NORMAL-NC | NORMAL-NC NORMAL-WT | NORMAL-NC | NORMAL-NC NORMAL-NC | NORMAL-NC | NORMAL-NC DEVICE<attr> | NORMAL-NC | DEVICE<attr> Still this cannot be generalized to non PCI devices such as GICv2. There is insufficient information and uncertainity in the behavior of non PCI driver. A driver must indicate support using the new flag VM_VFIO_ALLOW_WC. Adapt KVM to make use of the flag VM_VFIO_ALLOW_WC as indicator to activate the S2 setting to NormalNc. [1] section D8.5.5 of DDI0487J_a_a-profile_architecture_reference_manual.pdf Signed-off-by: Ankit Agrawal <ankita@nvidia.com> Suggested-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Jason Gunthorpe <jgg@nvidia.com> --- arch/arm64/kvm/mmu.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index d14504821b79..e1e6847a793b 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1381,7 +1381,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, int ret = 0; bool write_fault, writable, force_pte = false; bool exec_fault, mte_allowed; - bool device = false; + bool device = false, vfio_allow_wc = false; unsigned long mmu_seq; struct kvm *kvm = vcpu->kvm; struct kvm_mmu_memory_cache *memcache = &vcpu->arch.mmu_page_cache; @@ -1472,6 +1472,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, gfn = fault_ipa >> PAGE_SHIFT; mte_allowed = kvm_vma_mte_allowed(vma); + vfio_allow_wc = (vma->vm_flags & VM_VFIO_ALLOW_WC); + /* Don't use the VMA after the unlock -- it may have vanished */ vma = NULL; @@ -1557,10 +1559,18 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, if (exec_fault) prot |= KVM_PGTABLE_PROT_X; - if (device) - prot |= KVM_PGTABLE_PROT_DEVICE; - else if (cpus_have_final_cap(ARM64_HAS_CACHE_DIC)) + if (device) { + /* + * To provide VM with the ability to get device IO memory + * with NormalNC property, map device MMIO as NormalNC in S2. + */ + if (vfio_allow_wc) + prot |= KVM_PGTABLE_PROT_NORMAL_NC; + else + prot |= KVM_PGTABLE_PROT_DEVICE; + } else if (cpus_have_final_cap(ARM64_HAS_CACHE_DIC)) { prot |= KVM_PGTABLE_PROT_X; + } /* * Under the premise of getting a FSC_PERM fault, we just need to relax -- 2.34.1
WARNING: multiple messages have this Message-ID (diff)
From: <ankita@nvidia.com> To: <ankita@nvidia.com>, <jgg@nvidia.com>, <maz@kernel.org>, <oliver.upton@linux.dev>, <james.morse@arm.com>, <suzuki.poulose@arm.com>, <yuzenghui@huawei.com>, <reinette.chatre@intel.com>, <surenb@google.com>, <stefanha@redhat.com>, <brauner@kernel.org>, <catalin.marinas@arm.com>, <will@kernel.org>, <mark.rutland@arm.com>, <alex.williamson@redhat.com>, <kevin.tian@intel.com>, <yi.l.liu@intel.com>, <ardb@kernel.org>, <akpm@linux-foundation.org>, <andreyknvl@gmail.com>, <wangjinchao@xfusion.com>, <gshan@redhat.com>, <ricarkol@google.com>, <linux-mm@kvack.org>, <lpieralisi@kernel.org>, <rananta@google.com>, <ryan.roberts@arm.com> Cc: <aniketa@nvidia.com>, <cjia@nvidia.com>, <kwankhede@nvidia.com>, <targupta@nvidia.com>, <vsethi@nvidia.com>, <acurrid@nvidia.com>, <apopple@nvidia.com>, <jhubbard@nvidia.com>, <danw@nvidia.com>, <kvmarm@lists.linux.dev>, <mochs@nvidia.com>, <zhiw@nvidia.com>, <kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>, <linux-arm-kernel@lists.infradead.org> Subject: [PATCH v6 3/4] kvm: arm64: set io memory s2 pte as normalnc for vfio pci device Date: Thu, 8 Feb 2024 02:16:51 +0530 [thread overview] Message-ID: <20240207204652.22954-4-ankita@nvidia.com> (raw) In-Reply-To: <20240207204652.22954-1-ankita@nvidia.com> From: Ankit Agrawal <ankita@nvidia.com> To provide VM with the ability to get device IO memory with NormalNC property, map device MMIO in KVM for ARM64 at stage2 as NormalNC. Having NormalNC S2 default puts guests in control (based on [1], "Combining stage 1 and stage 2 memory type attributes") of device MMIO regions memory mappings. The rules are summarized below: ([(S1) - stage1], [(S2) - stage 2]) S1 | S2 | Result NORMAL-WB | NORMAL-NC | NORMAL-NC NORMAL-WT | NORMAL-NC | NORMAL-NC NORMAL-NC | NORMAL-NC | NORMAL-NC DEVICE<attr> | NORMAL-NC | DEVICE<attr> Still this cannot be generalized to non PCI devices such as GICv2. There is insufficient information and uncertainity in the behavior of non PCI driver. A driver must indicate support using the new flag VM_VFIO_ALLOW_WC. Adapt KVM to make use of the flag VM_VFIO_ALLOW_WC as indicator to activate the S2 setting to NormalNc. [1] section D8.5.5 of DDI0487J_a_a-profile_architecture_reference_manual.pdf Signed-off-by: Ankit Agrawal <ankita@nvidia.com> Suggested-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Jason Gunthorpe <jgg@nvidia.com> --- arch/arm64/kvm/mmu.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index d14504821b79..e1e6847a793b 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1381,7 +1381,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, int ret = 0; bool write_fault, writable, force_pte = false; bool exec_fault, mte_allowed; - bool device = false; + bool device = false, vfio_allow_wc = false; unsigned long mmu_seq; struct kvm *kvm = vcpu->kvm; struct kvm_mmu_memory_cache *memcache = &vcpu->arch.mmu_page_cache; @@ -1472,6 +1472,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, gfn = fault_ipa >> PAGE_SHIFT; mte_allowed = kvm_vma_mte_allowed(vma); + vfio_allow_wc = (vma->vm_flags & VM_VFIO_ALLOW_WC); + /* Don't use the VMA after the unlock -- it may have vanished */ vma = NULL; @@ -1557,10 +1559,18 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, if (exec_fault) prot |= KVM_PGTABLE_PROT_X; - if (device) - prot |= KVM_PGTABLE_PROT_DEVICE; - else if (cpus_have_final_cap(ARM64_HAS_CACHE_DIC)) + if (device) { + /* + * To provide VM with the ability to get device IO memory + * with NormalNC property, map device MMIO as NormalNC in S2. + */ + if (vfio_allow_wc) + prot |= KVM_PGTABLE_PROT_NORMAL_NC; + else + prot |= KVM_PGTABLE_PROT_DEVICE; + } else if (cpus_have_final_cap(ARM64_HAS_CACHE_DIC)) { prot |= KVM_PGTABLE_PROT_X; + } /* * Under the premise of getting a FSC_PERM fault, we just need to relax -- 2.34.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2024-02-07 20:47 UTC|newest] Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top 2024-02-07 20:46 [PATCH v6 0/4] kvm: arm64: allow the VM to select DEVICE_* and NORMAL_NC for IO memory ankita 2024-02-07 20:46 ` ankita 2024-02-07 20:46 ` [PATCH v6 1/4] kvm: arm64: introduce new flag for non-cacheable " ankita 2024-02-07 20:46 ` ankita 2024-02-08 13:00 ` Catalin Marinas 2024-02-08 13:00 ` Catalin Marinas 2024-02-08 13:24 ` Oliver Upton 2024-02-08 13:24 ` Oliver Upton 2024-02-09 14:10 ` Ankit Agrawal 2024-02-09 14:10 ` Ankit Agrawal 2024-02-08 13:19 ` Will Deacon 2024-02-08 13:19 ` Will Deacon 2024-02-09 14:12 ` Ankit Agrawal 2024-02-09 14:12 ` Ankit Agrawal 2024-02-07 20:46 ` [PATCH v6 2/4] mm: introduce new flag to indicate wc safe ankita 2024-02-07 20:46 ` ankita 2024-02-08 13:03 ` Catalin Marinas 2024-02-08 13:03 ` Catalin Marinas 2024-02-08 14:34 ` Jason Gunthorpe 2024-02-08 14:34 ` Jason Gunthorpe 2024-02-07 20:46 ` ankita [this message] 2024-02-07 20:46 ` [PATCH v6 3/4] kvm: arm64: set io memory s2 pte as normalnc for vfio pci device ankita 2024-02-08 13:26 ` Oliver Upton 2024-02-08 13:26 ` Oliver Upton 2024-02-08 14:51 ` Catalin Marinas 2024-02-08 14:51 ` Catalin Marinas 2024-02-09 14:05 ` Ankit Agrawal 2024-02-09 14:05 ` Ankit Agrawal 2024-02-07 20:46 ` [PATCH v6 4/4] vfio: convey kvm that the vfio-pci device is wc safe ankita 2024-02-07 20:46 ` ankita 2024-02-08 14:53 ` Catalin Marinas 2024-02-08 14:53 ` Catalin Marinas 2024-02-08 17:30 ` Alex Williamson 2024-02-08 17:30 ` Alex Williamson 2024-02-08 17:54 ` Jason Gunthorpe 2024-02-08 17:54 ` Jason Gunthorpe 2024-02-09 14:02 ` Ankit Agrawal 2024-02-09 14:02 ` Ankit Agrawal
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20240207204652.22954-4-ankita@nvidia.com \ --to=ankita@nvidia.com \ --cc=acurrid@nvidia.com \ --cc=akpm@linux-foundation.org \ --cc=alex.williamson@redhat.com \ --cc=andreyknvl@gmail.com \ --cc=aniketa@nvidia.com \ --cc=apopple@nvidia.com \ --cc=ardb@kernel.org \ --cc=brauner@kernel.org \ --cc=catalin.marinas@arm.com \ --cc=cjia@nvidia.com \ --cc=danw@nvidia.com \ --cc=gshan@redhat.com \ --cc=james.morse@arm.com \ --cc=jgg@nvidia.com \ --cc=jhubbard@nvidia.com \ --cc=kevin.tian@intel.com \ --cc=kvm@vger.kernel.org \ --cc=kvmarm@lists.linux.dev \ --cc=kwankhede@nvidia.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=lpieralisi@kernel.org \ --cc=mark.rutland@arm.com \ --cc=maz@kernel.org \ --cc=mochs@nvidia.com \ --cc=oliver.upton@linux.dev \ --cc=rananta@google.com \ --cc=reinette.chatre@intel.com \ --cc=ricarkol@google.com \ --cc=ryan.roberts@arm.com \ --cc=stefanha@redhat.com \ --cc=surenb@google.com \ --cc=suzuki.poulose@arm.com \ --cc=targupta@nvidia.com \ --cc=vsethi@nvidia.com \ --cc=wangjinchao@xfusion.com \ --cc=will@kernel.org \ --cc=yi.l.liu@intel.com \ --cc=yuzenghui@huawei.com \ --cc=zhiw@nvidia.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.