* [PATCH RESEND v4] powerpc: kvm: fix rare but potential deadlock scene
@ 2013-11-19 6:12 Liu Ping Fan
2013-11-19 6:12 ` [PATCH v3] powerpc: kvm: optimize "sc 1" as fast return Liu Ping Fan
2013-11-19 10:39 ` [PATCH RESEND v4] powerpc: kvm: fix rare but potential deadlock scene Alexander Graf
0 siblings, 2 replies; 6+ messages in thread
From: Liu Ping Fan @ 2013-11-19 6:12 UTC (permalink / raw)
To: linuxppc-dev, kvm-ppc
Cc: kvm, Gleb Natapov, Alexander Graf, Paul Mackerras, Paolo Bonzini
Since kvmppc_hv_find_lock_hpte() is called from both virtmode and
realmode, so it can trigger the deadlock.
Suppose the following scene:
Two physical cpuM, cpuN, two VM instances A, B, each VM has a group of
vcpus.
If on cpuM, vcpu_A_1 holds bitlock X (HPTE_V_HVLOCK), then is switched
out, and on cpuN, vcpu_A_2 try to lock X in realmode, then cpuN will be
caught in realmode for a long time.
What makes things even worse if the following happens,
On cpuM, bitlockX is hold, on cpuN, Y is hold.
vcpu_B_2 try to lock Y on cpuM in realmode
vcpu_A_2 try to lock X on cpuN in realmode
Oops! deadlock happens
Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
---
arch/powerpc/kvm/book3s_64_mmu_hv.c | 6 +++++-
arch/powerpc/kvm/book3s_hv_rm_mmu.c | 4 ++++
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c
index 842f081..abf81fe 100644
--- a/arch/powerpc/kvm/book3s_64_mmu_hv.c
+++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c
@@ -473,11 +473,14 @@ static int kvmppc_mmu_book3s_64_hv_xlate(struct kvm_vcpu *vcpu, gva_t eaddr,
slb_v = vcpu->kvm->arch.vrma_slb_v;
}
+ preempt_disable();
/* Find the HPTE in the hash table */
index = kvmppc_hv_find_lock_hpte(kvm, eaddr, slb_v,
HPTE_V_VALID | HPTE_V_ABSENT);
- if (index < 0)
+ if (index < 0) {
+ preempt_enable();
return -ENOENT;
+ }
hptep = (unsigned long *)(kvm->arch.hpt_virt + (index << 4));
v = hptep[0] & ~HPTE_V_HVLOCK;
gr = kvm->arch.revmap[index].guest_rpte;
@@ -485,6 +488,7 @@ static int kvmppc_mmu_book3s_64_hv_xlate(struct kvm_vcpu *vcpu, gva_t eaddr,
/* Unlock the HPTE */
asm volatile("lwsync" : : : "memory");
hptep[0] = v;
+ preempt_enable();
gpte->eaddr = eaddr;
gpte->vpage = ((v & HPTE_V_AVPN) << 4) | ((eaddr >> 12) & 0xfff);
diff --git a/arch/powerpc/kvm/book3s_hv_rm_mmu.c b/arch/powerpc/kvm/book3s_hv_rm_mmu.c
index 9c51544..ea17b30 100644
--- a/arch/powerpc/kvm/book3s_hv_rm_mmu.c
+++ b/arch/powerpc/kvm/book3s_hv_rm_mmu.c
@@ -749,6 +749,10 @@ static int slb_base_page_shift[4] = {
20, /* 1M, unsupported */
};
+/* When called from virtmode, this func should be protected by
+ * preempt_disable(), otherwise, the holding of HPTE_V_HVLOCK
+ * can trigger deadlock issue.
+ */
long kvmppc_hv_find_lock_hpte(struct kvm *kvm, gva_t eaddr, unsigned long slb_v,
unsigned long valid)
{
--
1.8.1.4
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v3] powerpc: kvm: optimize "sc 1" as fast return
2013-11-19 6:12 [PATCH RESEND v4] powerpc: kvm: fix rare but potential deadlock scene Liu Ping Fan
@ 2013-11-19 6:12 ` Liu Ping Fan
2013-11-19 10:40 ` Alexander Graf
2013-11-19 10:39 ` [PATCH RESEND v4] powerpc: kvm: fix rare but potential deadlock scene Alexander Graf
1 sibling, 1 reply; 6+ messages in thread
From: Liu Ping Fan @ 2013-11-19 6:12 UTC (permalink / raw)
To: linuxppc-dev, kvm-ppc
Cc: kvm, Gleb Natapov, Alexander Graf, Paul Mackerras, Paolo Bonzini
In some scene, e.g openstack CI, PR guest can trigger "sc 1" frequently,
this patch optimizes the path by directly delivering BOOK3S_INTERRUPT_SYSCALL
to HV guest, so powernv can return to HV guest without heavy exit, i.e,
no need to swap TLB, HTAB,.. etc
Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
---
v3: add some document
---
arch/powerpc/kvm/book3s_hv.c | 10 ++++------
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 19 ++++++++++++++++++-
2 files changed, 22 insertions(+), 7 deletions(-)
diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c
index 62a2b5a..1addb1a 100644
--- a/arch/powerpc/kvm/book3s_hv.c
+++ b/arch/powerpc/kvm/book3s_hv.c
@@ -628,12 +628,10 @@ static int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu,
/* hcall - punt to userspace */
int i;
- if (vcpu->arch.shregs.msr & MSR_PR) {
- /* sc 1 from userspace - reflect to guest syscall */
- kvmppc_book3s_queue_irqprio(vcpu, BOOK3S_INTERRUPT_SYSCALL);
- r = RESUME_GUEST;
- break;
- }
+ /* hypercall with MSR_PR has already been handled in rmode,
+ * and never reaches here.
+ */
+
run->papr_hcall.nr = kvmppc_get_gpr(vcpu, 3);
for (i = 0; i < 9; ++i)
run->papr_hcall.args[i] = kvmppc_get_gpr(vcpu, 4 + i);
diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
index c71103b..eea7ca7 100644
--- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S
+++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
@@ -534,6 +534,13 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_206)
5: mtspr SPRN_SRR0, r6
mtspr SPRN_SRR1, r7
+/*
+ * Required state:
+ * R4 = vcpu
+ * R10: value for HSRR0
+ * R11: value for HSRR1
+ * R13 = PACA
+ */
fast_guest_return:
li r0,0
stb r0,VCPU_CEDED(r4) /* cancel cede */
@@ -1388,7 +1395,8 @@ kvmppc_hisi:
hcall_try_real_mode:
ld r3,VCPU_GPR(R3)(r9)
andi. r0,r11,MSR_PR
- bne guest_exit_cont
+ /* sc 1 from userspace - reflect to guest syscall */
+ bne sc_1_fast_return
clrrdi r3,r3,2
cmpldi r3,hcall_real_table_end - hcall_real_table
bge guest_exit_cont
@@ -1409,6 +1417,15 @@ hcall_try_real_mode:
ld r11,VCPU_MSR(r4)
b fast_guest_return
+sc_1_fast_return:
+ mtspr SPRN_SRR0,r10
+ mtspr SPRN_SRR1,r11
+ li r10, BOOK3S_INTERRUPT_SYSCALL
+ li r11, (MSR_ME << 1) | 1 /* synthesize MSR_SF | MSR_ME */
+ rotldi r11, r11, 63
+ mr r4,r9
+ b fast_guest_return
+
/* We've attempted a real mode hcall, but it's punted it back
* to userspace. We need to restore some clobbered volatiles
* before resuming the pass-it-to-qemu path */
--
1.8.1.4
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH RESEND v4] powerpc: kvm: fix rare but potential deadlock scene
2013-11-19 6:12 [PATCH RESEND v4] powerpc: kvm: fix rare but potential deadlock scene Liu Ping Fan
2013-11-19 6:12 ` [PATCH v3] powerpc: kvm: optimize "sc 1" as fast return Liu Ping Fan
@ 2013-11-19 10:39 ` Alexander Graf
2013-11-20 2:42 ` Liu ping fan
1 sibling, 1 reply; 6+ messages in thread
From: Alexander Graf @ 2013-11-19 10:39 UTC (permalink / raw)
To: Liu Ping Fan
Cc: Gleb Natapov, kvm@vger.kernel.org mailing list, kvm-ppc,
Paul Mackerras, Paolo Bonzini, linuxppc-dev
On 19.11.2013, at 07:12, Liu Ping Fan <kernelfans@gmail.com> wrote:
> Since kvmppc_hv_find_lock_hpte() is called from both virtmode and
> realmode, so it can trigger the deadlock.
>
> Suppose the following scene:
>
> Two physical cpuM, cpuN, two VM instances A, B, each VM has a group of
> vcpus.
>
> If on cpuM, vcpu_A_1 holds bitlock X (HPTE_V_HVLOCK), then is switched
> out, and on cpuN, vcpu_A_2 try to lock X in realmode, then cpuN will be
> caught in realmode for a long time.
>
> What makes things even worse if the following happens,
> On cpuM, bitlockX is hold, on cpuN, Y is hold.
> vcpu_B_2 try to lock Y on cpuM in realmode
> vcpu_A_2 try to lock X on cpuN in realmode
>
> Oops! deadlock happens
>
> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
Any particular reason for the resend? The patch is already applied, no?
Alex
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v3] powerpc: kvm: optimize "sc 1" as fast return
2013-11-19 6:12 ` [PATCH v3] powerpc: kvm: optimize "sc 1" as fast return Liu Ping Fan
@ 2013-11-19 10:40 ` Alexander Graf
0 siblings, 0 replies; 6+ messages in thread
From: Alexander Graf @ 2013-11-19 10:40 UTC (permalink / raw)
To: Liu Ping Fan
Cc: Gleb Natapov, kvm@vger.kernel.org mailing list, kvm-ppc,
Paul Mackerras, Paolo Bonzini, linuxppc-dev
On 19.11.2013, at 07:12, Liu Ping Fan <kernelfans@gmail.com> wrote:
> In some scene, e.g openstack CI, PR guest can trigger "sc 1" =
frequently,
> this patch optimizes the path by directly delivering =
BOOK3S_INTERRUPT_SYSCALL
> to HV guest, so powernv can return to HV guest without heavy exit, =
i.e,
> no need to swap TLB, HTAB,.. etc
>=20
> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
> ---
> v3: add some document
This is v4 I think
> ---
> arch/powerpc/kvm/book3s_hv.c | 10 ++++------
> arch/powerpc/kvm/book3s_hv_rmhandlers.S | 19 ++++++++++++++++++-
> 2 files changed, 22 insertions(+), 7 deletions(-)
>=20
> diff --git a/arch/powerpc/kvm/book3s_hv.c =
b/arch/powerpc/kvm/book3s_hv.c
> index 62a2b5a..1addb1a 100644
> --- a/arch/powerpc/kvm/book3s_hv.c
> +++ b/arch/powerpc/kvm/book3s_hv.c
> @@ -628,12 +628,10 @@ static int kvmppc_handle_exit(struct kvm_run =
*run, struct kvm_vcpu *vcpu,
> /* hcall - punt to userspace */
> int i;
>=20
> - if (vcpu->arch.shregs.msr & MSR_PR) {
> - /* sc 1 from userspace - reflect to guest =
syscall */
> - kvmppc_book3s_queue_irqprio(vcpu, =
BOOK3S_INTERRUPT_SYSCALL);
> - r =3D RESUME_GUEST;
> - break;
> - }
> + /* hypercall with MSR_PR has already been handled in =
rmode,
> + * and never reaches here.
> + */
It would've been nice to also mention the real mode hypercall handling, =
but I can just post a follow-up patch for that one.
Thanks, applied to kvm-ppc-queue.
Alex
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH RESEND v4] powerpc: kvm: fix rare but potential deadlock scene
2013-11-19 10:39 ` [PATCH RESEND v4] powerpc: kvm: fix rare but potential deadlock scene Alexander Graf
@ 2013-11-20 2:42 ` Liu ping fan
2013-11-20 10:53 ` Alexander Graf
0 siblings, 1 reply; 6+ messages in thread
From: Liu ping fan @ 2013-11-20 2:42 UTC (permalink / raw)
To: Alexander Graf
Cc: Gleb Natapov, kvm@vger.kernel.org mailing list, kvm-ppc,
Paul Mackerras, Paolo Bonzini, linuxppc-dev
On Tue, Nov 19, 2013 at 6:39 PM, Alexander Graf <agraf@suse.de> wrote:
>
> On 19.11.2013, at 07:12, Liu Ping Fan <kernelfans@gmail.com> wrote:
>
>> Since kvmppc_hv_find_lock_hpte() is called from both virtmode and
>> realmode, so it can trigger the deadlock.
>>
>> Suppose the following scene:
>>
>> Two physical cpuM, cpuN, two VM instances A, B, each VM has a group of
>> vcpus.
>>
>> If on cpuM, vcpu_A_1 holds bitlock X (HPTE_V_HVLOCK), then is switched
>> out, and on cpuN, vcpu_A_2 try to lock X in realmode, then cpuN will be
>> caught in realmode for a long time.
>>
>> What makes things even worse if the following happens,
>> On cpuM, bitlockX is hold, on cpuN, Y is hold.
>> vcpu_B_2 try to lock Y on cpuM in realmode
>> vcpu_A_2 try to lock X on cpuN in realmode
>>
>> Oops! deadlock happens
>>
>> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
>
> Any particular reason for the resend? The patch is already applied, no?
>
Oh, seems that I misunderstood your meaning. You said "Actually, I've
changed my mind and moved the patch to the for-3.13 branch instead.
Please make sure to CC kvm@vger on all patches you submit though". So
I think it is necessary to resend with cc kvm@vger
Regards,
Pingfan
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH RESEND v4] powerpc: kvm: fix rare but potential deadlock scene
2013-11-20 2:42 ` Liu ping fan
@ 2013-11-20 10:53 ` Alexander Graf
0 siblings, 0 replies; 6+ messages in thread
From: Alexander Graf @ 2013-11-20 10:53 UTC (permalink / raw)
To: Liu ping fan
Cc: Gleb Natapov, kvm@vger.kernel.org mailing list, kvm-ppc,
Paul Mackerras, Paolo Bonzini, linuxppc-dev
> Am 20.11.2013 um 03:42 schrieb Liu ping fan <kernelfans@gmail.com>:
>=20
>> On Tue, Nov 19, 2013 at 6:39 PM, Alexander Graf <agraf@suse.de> wrote:
>>=20
>>> On 19.11.2013, at 07:12, Liu Ping Fan <kernelfans@gmail.com> wrote:
>>>=20
>>> Since kvmppc_hv_find_lock_hpte() is called from both virtmode and
>>> realmode, so it can trigger the deadlock.
>>>=20
>>> Suppose the following scene:
>>>=20
>>> Two physical cpuM, cpuN, two VM instances A, B, each VM has a group of
>>> vcpus.
>>>=20
>>> If on cpuM, vcpu_A_1 holds bitlock X (HPTE_V_HVLOCK), then is switched
>>> out, and on cpuN, vcpu_A_2 try to lock X in realmode, then cpuN will be
>>> caught in realmode for a long time.
>>>=20
>>> What makes things even worse if the following happens,
>>> On cpuM, bitlockX is hold, on cpuN, Y is hold.
>>> vcpu_B_2 try to lock Y on cpuM in realmode
>>> vcpu_A_2 try to lock X on cpuN in realmode
>>>=20
>>> Oops! deadlock happens
>>>=20
>>> Signed-off-by: Liu Ping Fan <pingfank@linux.vnet.ibm.com>
>>=20
>> Any particular reason for the resend? The patch is already applied, no?
> Oh, seems that I misunderstood your meaning. You said "Actually, I've
> changed my mind and moved the patch to the for-3.13 branch instead.
> Please make sure to CC kvm@vger on all patches you submit though". So
> I think it is necessary to resend with cc kvm@vger
Oh. I meant "for next time" :). But now that it happened for this patch alre=
ady it's even better.
Thanks a lot!
Alex
>=20
> Regards,
> Pingfan
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-11-20 10:54 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-11-19 6:12 [PATCH RESEND v4] powerpc: kvm: fix rare but potential deadlock scene Liu Ping Fan
2013-11-19 6:12 ` [PATCH v3] powerpc: kvm: optimize "sc 1" as fast return Liu Ping Fan
2013-11-19 10:40 ` Alexander Graf
2013-11-19 10:39 ` [PATCH RESEND v4] powerpc: kvm: fix rare but potential deadlock scene Alexander Graf
2013-11-20 2:42 ` Liu ping fan
2013-11-20 10:53 ` Alexander Graf
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).