From: James Hogan <james.hogan@imgtec.com> To: <linux-mips@linux-mips.org> Cc: "James Hogan" <james.hogan@imgtec.com>, "Paolo Bonzini" <pbonzini@redhat.com>, "Radim Krčmář" <rkrcmar@redhat.com>, "Ralf Baechle" <ralf@linux-mips.org>, kvm@vger.kernel.org Subject: [PATCH 17/30] KVM: MIPS: Add fast path TLB refill handler Date: Fri, 6 Jan 2017 01:32:49 +0000 [thread overview] Message-ID: <a4d6b3c495b44ad34e3e2b507a6a09f7a9f109f6.1483665879.git-series.james.hogan@imgtec.com> (raw) In-Reply-To: <cover.d6d201de414322ed2c1372e164254e6055ef7db9.1483665879.git-series.james.hogan@imgtec.com> Use functions from the general MIPS TLB exception vector generation code (tlbex.c) to construct a fast path TLB refill handler similar to the general one, but cut down and capable of preserving K0 and K1. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org --- arch/mips/include/asm/kvm_host.h | 1 +- arch/mips/kvm/entry.c | 78 +++++++++++++++++++++++++++++++++- arch/mips/kvm/mips.c | 8 +-- 3 files changed, 84 insertions(+), 3 deletions(-) diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h index 8c4a33e9687d..7bf8ee8bc01d 100644 --- a/arch/mips/include/asm/kvm_host.h +++ b/arch/mips/include/asm/kvm_host.h @@ -554,6 +554,7 @@ extern int kvm_mips_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu); /* Building of entry/exception code */ int kvm_mips_entry_setup(void); void *kvm_mips_build_vcpu_run(void *addr); +void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler); void *kvm_mips_build_exception(void *addr, void *handler); void *kvm_mips_build_exit(void *addr); diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c index 7424d3d566ff..1ae33e0e675c 100644 --- a/arch/mips/kvm/entry.c +++ b/arch/mips/kvm/entry.c @@ -16,6 +16,7 @@ #include <asm/mmu_context.h> #include <asm/msa.h> #include <asm/setup.h> +#include <asm/tlbex.h> #include <asm/uasm.h> /* Register names */ @@ -122,6 +123,9 @@ int kvm_mips_entry_setup(void) */ unsigned int kscratch_mask = cpu_data[0].kscratch_mask; + if (pgd_reg != -1) + kscratch_mask &= ~BIT(pgd_reg); + /* Pick a scratch register for storing VCPU */ if (kscratch_mask) { scratch_vcpu[0] = c0_kscratch(); @@ -381,6 +385,80 @@ static void *kvm_mips_build_enter_guest(void *addr) } /** + * kvm_mips_build_tlb_refill_exception() - Assemble TLB refill handler. + * @addr: Address to start writing code. + * @handler: Address of common handler (within range of @addr). + * + * Assemble TLB refill exception fast path handler for guest execution. + * + * Returns: Next address after end of written function. + */ +void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler) +{ + u32 *p = addr; + struct uasm_label labels[2]; + struct uasm_reloc relocs[2]; + struct uasm_label *l = labels; + struct uasm_reloc *r = relocs; + + memset(labels, 0, sizeof(labels)); + memset(relocs, 0, sizeof(relocs)); + + /* Save guest k1 into scratch register */ + UASM_i_MTC0(&p, K1, scratch_tmp[0], scratch_tmp[1]); + + /* Get the VCPU pointer from the VCPU scratch register */ + UASM_i_MFC0(&p, K1, scratch_vcpu[0], scratch_vcpu[1]); + + /* Save guest k0 into VCPU structure */ + UASM_i_SW(&p, K0, offsetof(struct kvm_vcpu, arch.gprs[K0]), K1); + + /* + * Some of the common tlbex code uses current_cpu_type(). For KVM we + * assume symmetry and just disable preemption to silence the warning. + */ + preempt_disable(); + + /* + * Now for the actual refill bit. A lot of this can be common with the + * Linux TLB refill handler, however we don't need to handle so many + * cases. We only need to handle user mode refills, and user mode runs + * with 32-bit addressing. + * + * Therefore the branch to label_vmalloc generated by build_get_pmde64() + * that isn't resolved should never actually get taken and is harmless + * to leave in place for now. + */ + +#ifdef CONFIG_64BIT + build_get_pmde64(&p, &l, &r, K0, K1); /* get pmd in K1 */ +#else + build_get_pgde32(&p, K0, K1); /* get pgd in K1 */ +#endif + + /* we don't support huge pages yet */ + + build_get_ptep(&p, K0, K1); + build_update_entries(&p, K0, K1); + build_tlb_write_entry(&p, &l, &r, tlb_random); + + preempt_enable(); + + /* Get the VCPU pointer from the VCPU scratch register again */ + UASM_i_MFC0(&p, K1, scratch_vcpu[0], scratch_vcpu[1]); + + /* Restore the guest's k0/k1 registers */ + UASM_i_LW(&p, K0, offsetof(struct kvm_vcpu, arch.gprs[K0]), K1); + uasm_i_ehb(&p); + UASM_i_MFC0(&p, K1, scratch_tmp[0], scratch_tmp[1]); + + /* Jump to guest */ + uasm_i_eret(&p); + + return p; +} + +/** * kvm_mips_build_exception() - Assemble first level guest exception handler. * @addr: Address to start writing code. * @handler: Address of common handler (within range of @addr). diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c index 39792ec73a6d..3cf720790ce6 100644 --- a/arch/mips/kvm/mips.c +++ b/arch/mips/kvm/mips.c @@ -264,7 +264,7 @@ static inline void dump_handler(const char *symbol, void *start, void *end) struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id) { int err, size; - void *gebase, *p, *handler; + void *gebase, *p, *handler, *refill_start, *refill_end; int i; struct kvm_vcpu *vcpu = kzalloc(sizeof(struct kvm_vcpu), GFP_KERNEL); @@ -317,8 +317,9 @@ struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id) /* Build guest exception vectors dynamically in unmapped memory */ handler = gebase + 0x2000; - /* TLB Refill, EXL = 0 */ - kvm_mips_build_exception(gebase, handler); + /* TLB refill */ + refill_start = gebase; + refill_end = kvm_mips_build_tlb_refill_exception(refill_start, handler); /* General Exception Entry point */ kvm_mips_build_exception(gebase + 0x180, handler); @@ -344,6 +345,7 @@ struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id) pr_debug("#include <asm/regdef.h>\n"); pr_debug("\n"); dump_handler("kvm_vcpu_run", vcpu->arch.vcpu_run, p); + dump_handler("kvm_tlb_refill", refill_start, refill_end); dump_handler("kvm_gen_exc", gebase + 0x180, gebase + 0x200); dump_handler("kvm_exit", gebase + 0x2000, vcpu->arch.vcpu_run); -- git-series 0.8.10
WARNING: multiple messages have this Message-ID (diff)
From: James Hogan <james.hogan@imgtec.com> To: linux-mips@linux-mips.org Cc: "James Hogan" <james.hogan@imgtec.com>, "Paolo Bonzini" <pbonzini@redhat.com>, "Radim Krčmář" <rkrcmar@redhat.com>, "Ralf Baechle" <ralf@linux-mips.org>, kvm@vger.kernel.org Subject: [PATCH 17/30] KVM: MIPS: Add fast path TLB refill handler Date: Fri, 6 Jan 2017 01:32:49 +0000 [thread overview] Message-ID: <a4d6b3c495b44ad34e3e2b507a6a09f7a9f109f6.1483665879.git-series.james.hogan@imgtec.com> (raw) Message-ID: <20170106013249.12KoYQbYoAzfVo0RvkmlEzpDF6R6O002Sk1fyUGAH74@z> (raw) In-Reply-To: <cover.d6d201de414322ed2c1372e164254e6055ef7db9.1483665879.git-series.james.hogan@imgtec.com> Use functions from the general MIPS TLB exception vector generation code (tlbex.c) to construct a fast path TLB refill handler similar to the general one, but cut down and capable of preserving K0 and K1. Signed-off-by: James Hogan <james.hogan@imgtec.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: "Radim Krčmář" <rkrcmar@redhat.com> Cc: Ralf Baechle <ralf@linux-mips.org> Cc: linux-mips@linux-mips.org Cc: kvm@vger.kernel.org --- arch/mips/include/asm/kvm_host.h | 1 +- arch/mips/kvm/entry.c | 78 +++++++++++++++++++++++++++++++++- arch/mips/kvm/mips.c | 8 +-- 3 files changed, 84 insertions(+), 3 deletions(-) diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h index 8c4a33e9687d..7bf8ee8bc01d 100644 --- a/arch/mips/include/asm/kvm_host.h +++ b/arch/mips/include/asm/kvm_host.h @@ -554,6 +554,7 @@ extern int kvm_mips_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu); /* Building of entry/exception code */ int kvm_mips_entry_setup(void); void *kvm_mips_build_vcpu_run(void *addr); +void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler); void *kvm_mips_build_exception(void *addr, void *handler); void *kvm_mips_build_exit(void *addr); diff --git a/arch/mips/kvm/entry.c b/arch/mips/kvm/entry.c index 7424d3d566ff..1ae33e0e675c 100644 --- a/arch/mips/kvm/entry.c +++ b/arch/mips/kvm/entry.c @@ -16,6 +16,7 @@ #include <asm/mmu_context.h> #include <asm/msa.h> #include <asm/setup.h> +#include <asm/tlbex.h> #include <asm/uasm.h> /* Register names */ @@ -122,6 +123,9 @@ int kvm_mips_entry_setup(void) */ unsigned int kscratch_mask = cpu_data[0].kscratch_mask; + if (pgd_reg != -1) + kscratch_mask &= ~BIT(pgd_reg); + /* Pick a scratch register for storing VCPU */ if (kscratch_mask) { scratch_vcpu[0] = c0_kscratch(); @@ -381,6 +385,80 @@ static void *kvm_mips_build_enter_guest(void *addr) } /** + * kvm_mips_build_tlb_refill_exception() - Assemble TLB refill handler. + * @addr: Address to start writing code. + * @handler: Address of common handler (within range of @addr). + * + * Assemble TLB refill exception fast path handler for guest execution. + * + * Returns: Next address after end of written function. + */ +void *kvm_mips_build_tlb_refill_exception(void *addr, void *handler) +{ + u32 *p = addr; + struct uasm_label labels[2]; + struct uasm_reloc relocs[2]; + struct uasm_label *l = labels; + struct uasm_reloc *r = relocs; + + memset(labels, 0, sizeof(labels)); + memset(relocs, 0, sizeof(relocs)); + + /* Save guest k1 into scratch register */ + UASM_i_MTC0(&p, K1, scratch_tmp[0], scratch_tmp[1]); + + /* Get the VCPU pointer from the VCPU scratch register */ + UASM_i_MFC0(&p, K1, scratch_vcpu[0], scratch_vcpu[1]); + + /* Save guest k0 into VCPU structure */ + UASM_i_SW(&p, K0, offsetof(struct kvm_vcpu, arch.gprs[K0]), K1); + + /* + * Some of the common tlbex code uses current_cpu_type(). For KVM we + * assume symmetry and just disable preemption to silence the warning. + */ + preempt_disable(); + + /* + * Now for the actual refill bit. A lot of this can be common with the + * Linux TLB refill handler, however we don't need to handle so many + * cases. We only need to handle user mode refills, and user mode runs + * with 32-bit addressing. + * + * Therefore the branch to label_vmalloc generated by build_get_pmde64() + * that isn't resolved should never actually get taken and is harmless + * to leave in place for now. + */ + +#ifdef CONFIG_64BIT + build_get_pmde64(&p, &l, &r, K0, K1); /* get pmd in K1 */ +#else + build_get_pgde32(&p, K0, K1); /* get pgd in K1 */ +#endif + + /* we don't support huge pages yet */ + + build_get_ptep(&p, K0, K1); + build_update_entries(&p, K0, K1); + build_tlb_write_entry(&p, &l, &r, tlb_random); + + preempt_enable(); + + /* Get the VCPU pointer from the VCPU scratch register again */ + UASM_i_MFC0(&p, K1, scratch_vcpu[0], scratch_vcpu[1]); + + /* Restore the guest's k0/k1 registers */ + UASM_i_LW(&p, K0, offsetof(struct kvm_vcpu, arch.gprs[K0]), K1); + uasm_i_ehb(&p); + UASM_i_MFC0(&p, K1, scratch_tmp[0], scratch_tmp[1]); + + /* Jump to guest */ + uasm_i_eret(&p); + + return p; +} + +/** * kvm_mips_build_exception() - Assemble first level guest exception handler. * @addr: Address to start writing code. * @handler: Address of common handler (within range of @addr). diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c index 39792ec73a6d..3cf720790ce6 100644 --- a/arch/mips/kvm/mips.c +++ b/arch/mips/kvm/mips.c @@ -264,7 +264,7 @@ static inline void dump_handler(const char *symbol, void *start, void *end) struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id) { int err, size; - void *gebase, *p, *handler; + void *gebase, *p, *handler, *refill_start, *refill_end; int i; struct kvm_vcpu *vcpu = kzalloc(sizeof(struct kvm_vcpu), GFP_KERNEL); @@ -317,8 +317,9 @@ struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id) /* Build guest exception vectors dynamically in unmapped memory */ handler = gebase + 0x2000; - /* TLB Refill, EXL = 0 */ - kvm_mips_build_exception(gebase, handler); + /* TLB refill */ + refill_start = gebase; + refill_end = kvm_mips_build_tlb_refill_exception(refill_start, handler); /* General Exception Entry point */ kvm_mips_build_exception(gebase + 0x180, handler); @@ -344,6 +345,7 @@ struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, unsigned int id) pr_debug("#include <asm/regdef.h>\n"); pr_debug("\n"); dump_handler("kvm_vcpu_run", vcpu->arch.vcpu_run, p); + dump_handler("kvm_tlb_refill", refill_start, refill_end); dump_handler("kvm_gen_exc", gebase + 0x180, gebase + 0x200); dump_handler("kvm_exit", gebase + 0x2000, vcpu->arch.vcpu_run); -- git-series 0.8.10
next prev parent reply other threads:[~2017-01-06 1:40 UTC|newest] Thread overview: 67+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-01-06 1:32 [PATCH 0/30] KVM: MIPS: Implement GVA page tables James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 1/30] mm: Export init_mm for MIPS KVM use of pgd_alloc() James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-17 16:23 ` Ralf Baechle 2017-01-17 16:27 ` Ralf Baechle 2017-01-06 1:32 ` [PATCH 2/30] MIPS: Export pgd/pmd symbols for KVM James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-17 16:24 ` Ralf Baechle 2017-01-06 1:32 ` [PATCH 3/30] MIPS: uasm: Add include guards in asm/uasm.h James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-17 16:25 ` Ralf Baechle 2017-01-06 1:32 ` [PATCH 4/30] MIPS: Export some tlbex internals for KVM to use James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-17 16:26 ` Ralf Baechle 2017-01-06 1:32 ` [PATCH 5/30] KVM: MIPS: Drop partial KVM_NMI implementation James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 6/30] KVM: MIPS/MMU: Simplify ASID restoration James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 7/30] KVM: MIPS: Convert get/set_regs -> vcpu_load/put James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 8/30] KVM: MIPS/MMU: Move preempt/ASID handling to implementation James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 9/30] KVM: MIPS: Remove duplicated ASIDs from vcpu James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 10/30] KVM: MIPS: Add vcpu_run() & vcpu_reenter() callbacks James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 11/30] KVM: MIPS/T&E: Restore host asid on return to host James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 12/30] KVM: MIPS/T&E: active_mm = init_mm in guest context James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 13/30] KVM: MIPS: Wire up vcpu uninit James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 14/30] KVM: MIPS/T&E: Allocate GVA -> HPA page tables James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 15/30] KVM: MIPS/T&E: Activate GVA page tables in guest context James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 16/30] KVM: MIPS: Support NetLogic KScratch registers James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` James Hogan [this message] 2017-01-06 1:32 ` [PATCH 17/30] KVM: MIPS: Add fast path TLB refill handler James Hogan 2017-01-06 1:32 ` [PATCH 18/30] KVM: MIPS/TLB: Fix off-by-one in TLB invalidate James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 19/30] KVM: MIPS/TLB: Generalise host TLB invalidate to kernel ASID James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 20/30] KVM: MIPS/MMU: Invalidate GVA PTs on ASID changes James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 21/30] KVM: MIPS/MMU: Invalidate stale GVA PTEs on TLBW James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 22/30] KVM: MIPS/MMU: Convert KSeg0 faults to page tables James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 23/30] KVM: MIPS/MMU: Convert TLB mapped " James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 24/30] KVM: MIPS/MMU: Convert commpage fault handling " James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 25/30] KVM: MIPS: Drop vm_init() callback James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 26/30] KVM: MIPS: Use uaccess to read/modify guest instructions James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:32 ` [PATCH 27/30] KVM: MIPS/Emulate: Fix CACHE emulation for EVA hosts James Hogan 2017-01-06 1:32 ` James Hogan 2017-01-06 1:33 ` [PATCH 28/30] KVM: MIPS/TLB: Drop kvm_local_flush_tlb_all() James Hogan 2017-01-06 1:33 ` James Hogan 2017-01-06 1:33 ` [PATCH 29/30] KVM: MIPS/Emulate: Drop redundant TLB flushes on exceptions James Hogan 2017-01-06 1:33 ` James Hogan 2017-01-06 1:33 ` [PATCH 30/30] KVM: MIPS/MMU: Drop kvm_get_new_mmu_context() James Hogan 2017-01-06 1:33 ` James Hogan
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=a4d6b3c495b44ad34e3e2b507a6a09f7a9f109f6.1483665879.git-series.james.hogan@imgtec.com \ --to=james.hogan@imgtec.com \ --cc=kvm@vger.kernel.org \ --cc=linux-mips@linux-mips.org \ --cc=pbonzini@redhat.com \ --cc=ralf@linux-mips.org \ --cc=rkrcmar@redhat.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).