* [PATCH 0/1] KVM: s390: backport for stable of "KVM: s390: index
@ 2021-09-20 15:06 Christian Borntraeger
2021-09-20 15:06 ` [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx Christian Borntraeger
` (2 more replies)
0 siblings, 3 replies; 20+ messages in thread
From: Christian Borntraeger @ 2021-09-20 15:06 UTC (permalink / raw)
To: stable, Paolo Bonzini
Cc: gregkh, KVM, Cornelia Huck, Christian Borntraeger, Janosch Frank,
David Hildenbrand, linux-s390, Thomas Huth, Claudio Imbrenda,
Radim Krčmář,
Nitesh Narayan Lal, Halil Pasic
Stable team,
here is a backport for 4.19 of
commit a3e03bc1368 ("KVM: s390: index kvm->arch.idle_mask by vcpu_idx")
This basically removes the kick_mask parts that were introduced with
kernel 5.0 and fixes up the location of the idle_mask to the older
place.
FWIW, it might be a good idea to also backport
8750e72a79dd ("KVM: remember position in kvm->vcpus array") to avoid
a performance regression for large guests (many vCPUs) when this patch
is applied.
@Paolo Bonzini, would you be ok with 8750e72a79dd in older stable releases?
Halil Pasic (1):
KVM: s390: index kvm->arch.idle_mask by vcpu_idx
arch/s390/kvm/interrupt.c | 4 ++--
arch/s390/kvm/kvm-s390.h | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
--
2.31.1
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx
2021-09-20 15:06 [PATCH 0/1] KVM: s390: backport for stable of "KVM: s390: index Christian Borntraeger
@ 2021-09-20 15:06 ` Christian Borntraeger
2021-09-21 2:43 ` Halil Pasic
2021-09-20 15:24 ` [PATCH 0/1] KVM: s390: backport for stable of "KVM: s390: index Greg KH
2021-09-20 18:05 ` Paolo Bonzini
2 siblings, 1 reply; 20+ messages in thread
From: Christian Borntraeger @ 2021-09-20 15:06 UTC (permalink / raw)
To: stable
Cc: gregkh, KVM, Cornelia Huck, Christian Borntraeger, Janosch Frank,
David Hildenbrand, linux-s390, Thomas Huth, Claudio Imbrenda,
Paolo Bonzini, Halil Pasic, Radim Krčmář,
Nitesh Narayan Lal
From: Halil Pasic <pasic@linux.ibm.com>
While in practice vcpu->vcpu_idx == vcpu->vcp_id is often true, it may
not always be, and we must not rely on this. Reason is that KVM decides
the vcpu_idx, userspace decides the vcpu_id, thus the two might not
match.
Currently kvm->arch.idle_mask is indexed by vcpu_id, which implies
that code like
for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
vcpu = kvm_get_vcpu(kvm, vcpu_id);
do_stuff(vcpu);
}
is not legit. Reason is that kvm_get_vcpu expects an vcpu_idx, not an
vcpu_id. The trouble is, we do actually use kvm->arch.idle_mask like
this. To fix this problem we have two options. Either use
kvm_get_vcpu_by_id(vcpu_id), which would loop to find the right vcpu_id,
or switch to indexing via vcpu_idx. The latter is preferable for obvious
reasons.
Let us make switch from indexing kvm->arch.idle_mask by vcpu_id to
indexing it by vcpu_idx. To keep gisa_int.kicked_mask indexed by the
same index as idle_mask lets make the same change for it as well.
Fixes: 1ee0bc559dc3 ("KVM: s390: get rid of local_int array")
Signed-off-by: Halil Pasic <pasic@linux.ibm.com>
Reviewed-by: Christian Bornträger <borntraeger@de.ibm.com>
Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Cc: <stable@vger.kernel.org> # 3.15+
Link: https://lore.kernel.org/r/20210827125429.1912577-1-pasic@linux.ibm.com
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
[borntraeger@de.ibm.com]: change idle mask, remove kicked_mask
---
arch/s390/kvm/interrupt.c | 4 ++--
arch/s390/kvm/kvm-s390.h | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
index 3515f2b55eb9..dc5ecaea30d7 100644
--- a/arch/s390/kvm/interrupt.c
+++ b/arch/s390/kvm/interrupt.c
@@ -318,13 +318,13 @@ static unsigned long deliverable_irqs(struct kvm_vcpu *vcpu)
static void __set_cpu_idle(struct kvm_vcpu *vcpu)
{
kvm_s390_set_cpuflags(vcpu, CPUSTAT_WAIT);
- set_bit(vcpu->vcpu_id, vcpu->kvm->arch.float_int.idle_mask);
+ set_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.float_int.idle_mask);
}
static void __unset_cpu_idle(struct kvm_vcpu *vcpu)
{
kvm_s390_clear_cpuflags(vcpu, CPUSTAT_WAIT);
- clear_bit(vcpu->vcpu_id, vcpu->kvm->arch.float_int.idle_mask);
+ clear_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.float_int.idle_mask);
}
static void __reset_intercept_indicators(struct kvm_vcpu *vcpu)
diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h
index 981e3ba97461..0a2ffd5378be 100644
--- a/arch/s390/kvm/kvm-s390.h
+++ b/arch/s390/kvm/kvm-s390.h
@@ -67,7 +67,7 @@ static inline int is_vcpu_stopped(struct kvm_vcpu *vcpu)
static inline int is_vcpu_idle(struct kvm_vcpu *vcpu)
{
- return test_bit(vcpu->vcpu_id, vcpu->kvm->arch.float_int.idle_mask);
+ return test_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.float_int.idle_mask);
}
static inline int kvm_is_ucontrol(struct kvm *kvm)
--
2.31.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH 0/1] KVM: s390: backport for stable of "KVM: s390: index
2021-09-20 15:06 [PATCH 0/1] KVM: s390: backport for stable of "KVM: s390: index Christian Borntraeger
2021-09-20 15:06 ` [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx Christian Borntraeger
@ 2021-09-20 15:24 ` Greg KH
2021-09-20 15:30 ` Christian Borntraeger
2021-09-20 18:05 ` Paolo Bonzini
2 siblings, 1 reply; 20+ messages in thread
From: Greg KH @ 2021-09-20 15:24 UTC (permalink / raw)
To: Christian Borntraeger
Cc: stable, Paolo Bonzini, KVM, Cornelia Huck, Janosch Frank,
David Hildenbrand, linux-s390, Thomas Huth, Claudio Imbrenda,
Radim Krčmář,
Nitesh Narayan Lal, Halil Pasic
On Mon, Sep 20, 2021 at 05:06:15PM +0200, Christian Borntraeger wrote:
> Stable team,
>
> here is a backport for 4.19 of
> commit a3e03bc1368 ("KVM: s390: index kvm->arch.idle_mask by vcpu_idx")
> This basically removes the kick_mask parts that were introduced with
> kernel 5.0 and fixes up the location of the idle_mask to the older
> place.
Now queued up, thanks.
> FWIW, it might be a good idea to also backport
> 8750e72a79dd ("KVM: remember position in kvm->vcpus array") to avoid
> a performance regression for large guests (many vCPUs) when this patch
> is applied.
> @Paolo Bonzini, would you be ok with 8750e72a79dd in older stable releases?
That would also have to go into 5.4.y, right?
thanks,
greg k-h
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 0/1] KVM: s390: backport for stable of "KVM: s390: index
2021-09-20 15:24 ` [PATCH 0/1] KVM: s390: backport for stable of "KVM: s390: index Greg KH
@ 2021-09-20 15:30 ` Christian Borntraeger
0 siblings, 0 replies; 20+ messages in thread
From: Christian Borntraeger @ 2021-09-20 15:30 UTC (permalink / raw)
To: Greg KH
Cc: stable, Paolo Bonzini, KVM, Cornelia Huck, Janosch Frank,
David Hildenbrand, linux-s390, Thomas Huth, Claudio Imbrenda,
Radim Krčmář,
Nitesh Narayan Lal, Halil Pasic
Am 20.09.21 um 17:24 schrieb Greg KH:
> On Mon, Sep 20, 2021 at 05:06:15PM +0200, Christian Borntraeger wrote:
>> Stable team,
>>
>> here is a backport for 4.19 of
>> commit a3e03bc1368 ("KVM: s390: index kvm->arch.idle_mask by vcpu_idx")
>> This basically removes the kick_mask parts that were introduced with
>> kernel 5.0 and fixes up the location of the idle_mask to the older
>> place.
>
> Now queued up, thanks.
>
>> FWIW, it might be a good idea to also backport
>> 8750e72a79dd ("KVM: remember position in kvm->vcpus array") to avoid
>> a performance regression for large guests (many vCPUs) when this patch
>> is applied.
>> @Paolo Bonzini, would you be ok with 8750e72a79dd in older stable releases?
>
> That would also have to go into 5.4.y, right?
Ideally yes.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 0/1] KVM: s390: backport for stable of "KVM: s390: index
2021-09-20 15:06 [PATCH 0/1] KVM: s390: backport for stable of "KVM: s390: index Christian Borntraeger
2021-09-20 15:06 ` [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx Christian Borntraeger
2021-09-20 15:24 ` [PATCH 0/1] KVM: s390: backport for stable of "KVM: s390: index Greg KH
@ 2021-09-20 18:05 ` Paolo Bonzini
2021-09-21 8:46 ` Christian Borntraeger
2 siblings, 1 reply; 20+ messages in thread
From: Paolo Bonzini @ 2021-09-20 18:05 UTC (permalink / raw)
To: Christian Borntraeger, stable
Cc: gregkh, KVM, Cornelia Huck, Janosch Frank, David Hildenbrand,
linux-s390, Thomas Huth, Claudio Imbrenda,
Radim Krčmář,
Nitesh Narayan Lal, Halil Pasic
On 20/09/21 17:06, Christian Borntraeger wrote:
> here is a backport for 4.19 of
> commit a3e03bc1368 ("KVM: s390: index kvm->arch.idle_mask by vcpu_idx")
> This basically removes the kick_mask parts that were introduced with
> kernel 5.0 and fixes up the location of the idle_mask to the older
> place.
>
> FWIW, it might be a good idea to also backport
> 8750e72a79dd ("KVM: remember position in kvm->vcpus array") to avoid
> a performance regression for large guests (many vCPUs) when this patch
> is applied.
> @Paolo Bonzini, would you be ok with 8750e72a79dd in older stable releases?
Sure, I suppose you're going to send a separate backport that I can ack.
Paolo
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx
2021-09-20 15:06 ` [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx Christian Borntraeger
@ 2021-09-21 2:43 ` Halil Pasic
0 siblings, 0 replies; 20+ messages in thread
From: Halil Pasic @ 2021-09-21 2:43 UTC (permalink / raw)
To: Christian Borntraeger
Cc: stable, gregkh, KVM, Cornelia Huck, Janosch Frank,
David Hildenbrand, linux-s390, Thomas Huth, Claudio Imbrenda,
Paolo Bonzini, Radim Krčmář,
Nitesh Narayan Lal, Halil Pasic
On Mon, 20 Sep 2021 17:06:16 +0200
Christian Borntraeger <borntraeger@de.ibm.com> wrote:
> Fixes: 1ee0bc559dc3 ("KVM: s390: get rid of local_int array")
> Signed-off-by: Halil Pasic <pasic@linux.ibm.com>
> Reviewed-by: Christian Bornträger <borntraeger@de.ibm.com>
> Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
> Cc: <stable@vger.kernel.org> # 3.15+
> Link: https://lore.kernel.org/r/20210827125429.1912577-1-pasic@linux.ibm.com
> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
> [borntraeger@de.ibm.com]: change idle mask, remove kicked_mask
LGTM. Thanks for taking care of this!
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 0/1] KVM: s390: backport for stable of "KVM: s390: index
2021-09-20 18:05 ` Paolo Bonzini
@ 2021-09-21 8:46 ` Christian Borntraeger
2021-09-21 13:04 ` Paolo Bonzini
0 siblings, 1 reply; 20+ messages in thread
From: Christian Borntraeger @ 2021-09-21 8:46 UTC (permalink / raw)
To: Paolo Bonzini, stable
Cc: gregkh, KVM, Cornelia Huck, Janosch Frank, David Hildenbrand,
linux-s390, Thomas Huth, Claudio Imbrenda,
Radim Krčmář,
Nitesh Narayan Lal, Halil Pasic
Am 20.09.21 um 20:05 schrieb Paolo Bonzini:
> On 20/09/21 17:06, Christian Borntraeger wrote:
>> here is a backport for 4.19 of
>> commit a3e03bc1368 ("KVM: s390: index kvm->arch.idle_mask by vcpu_idx")
>> This basically removes the kick_mask parts that were introduced with
>> kernel 5.0 and fixes up the location of the idle_mask to the older
>> place.
>>
>> FWIW, it might be a good idea to also backport
>> 8750e72a79dd ("KVM: remember position in kvm->vcpus array") to avoid
>> a performance regression for large guests (many vCPUs) when this patch
>> is applied.
>> @Paolo Bonzini, would you be ok with 8750e72a79dd in older stable releases?
>
> Sure, I suppose you're going to send a separate backport that I can ack.
It does seem to apply when cherry-picked, but I can send it as a patch if you and Greg
prefer it that way.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 0/1] KVM: s390: backport for stable of "KVM: s390: index
2021-09-21 8:46 ` Christian Borntraeger
@ 2021-09-21 13:04 ` Paolo Bonzini
0 siblings, 0 replies; 20+ messages in thread
From: Paolo Bonzini @ 2021-09-21 13:04 UTC (permalink / raw)
To: Christian Borntraeger, stable
Cc: gregkh, KVM, Cornelia Huck, Janosch Frank, David Hildenbrand,
linux-s390, Thomas Huth, Claudio Imbrenda,
Radim Krčmář,
Nitesh Narayan Lal, Halil Pasic
On 21/09/21 10:46, Christian Borntraeger wrote:
>>>
>>
>> Sure, I suppose you're going to send a separate backport that I can
>> ack.
>
> It does seem to apply when cherry-picked, but I can send it as a
> patch if you and Greg prefer it that way.
Yes, please do! Thanks,
Paolo
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx
2022-01-31 12:09 ` Petr Tesařík
@ 2022-01-31 12:24 ` Halil Pasic
0 siblings, 0 replies; 20+ messages in thread
From: Halil Pasic @ 2022-01-31 12:24 UTC (permalink / raw)
To: Petr Tesařík
Cc: Christian Borntraeger, Janosch Frank, David Hildenbrand,
Cornelia Huck, Claudio Imbrenda, Heiko Carstens, Vasily Gorbik,
kvm, linux-s390, linux-kernel, stable, Michael Mueller,
Halil Pasic
On Mon, 31 Jan 2022 13:09:26 +0100
Petr Tesařík <ptesarik@suse.cz> wrote:
> Hi Halil,
>
> Dne 31. 01. 22 v 12:53 Halil Pasic napsal(a):
> > On Mon, 31 Jan 2022 11:13:18 +0100
> > Petr Tesařík <ptesarik@suse.cz> wrote:
> >
> >> Hi Halil,
> >>
> >> Dne 27. 08. 21 v 14:54 Halil Pasic napsal(a):
> >>> While in practice vcpu->vcpu_idx == vcpu->vcp_id is often true,
> >>> it may not always be, and we must not rely on this.
> >>>
> >>> Currently kvm->arch.idle_mask is indexed by vcpu_id, which implies
> >>> that code like
> >>> for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
> >>> vcpu = kvm_get_vcpu(kvm, vcpu_id);
> >>> do_stuff(vcpu);
> >>> }
> >>> is not legit. The trouble is, we do actually use kvm->arch.idle_mask
> >>> like this. To fix this problem we have two options. Either use
> >>> kvm_get_vcpu_by_id(vcpu_id), which would loop to find the right vcpu_id,
> >>> or switch to indexing via vcpu_idx. The latter is preferable for obvious
> >>> reasons.
> >>
> >> I'm just backporting this fix to SLES 12 SP5, and I've noticed that
> >> there is still this code in __floating_irq_kick():
> >>
> >> /* find idle VCPUs first, then round robin */
> >> sigcpu = find_first_bit(fi->idle_mask, online_vcpus);
> >> /* ... round robin loop removed ...
> >> dst_vcpu = kvm_get_vcpu(kvm, sigcpu);
> >>
> >> It seems to me that this does exactly the thing that is not legit, but
> >> I'm no expert. Did I miss something?
> >>
> >
> > We made that legit by making the N-th bit in idle_mask correspond to the
> > vcpu whose vcpu_idx == N. The second argument of kvm_get_vcpu() is the
> > vcpu_idx. IMHO that ain't super-intuitive because it ain't spelled out.
> >
> > So before this was a mismatch (with a vcpu_id based bitmap we would have
> > to use kvm_get_vcpu_by_id()), and with this patch applied this code
> > becomes legit because both idle_mask and kvm_get_vcpu() operate with
> > vcpu_idx.
> >
> > Does that make sense?
>
> Yes!
>
> > I'm sorry the commit message did not convey this clearly enough...
>
> No, it's not your fault. I didn't pay enough attention to the change,
> and with vcpu_id and vcpu_idx being so similar I got confused.
No problem at all!
>
> In short, there's no bug now, indeed. Thanks for your patience.
>
Thank you for being mindful when backporting!
Regards,
Halil
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx
2022-01-31 11:53 ` Halil Pasic
@ 2022-01-31 12:09 ` Petr Tesařík
2022-01-31 12:24 ` Halil Pasic
0 siblings, 1 reply; 20+ messages in thread
From: Petr Tesařík @ 2022-01-31 12:09 UTC (permalink / raw)
To: Halil Pasic
Cc: Christian Borntraeger, Janosch Frank, David Hildenbrand,
Cornelia Huck, Claudio Imbrenda, Heiko Carstens, Vasily Gorbik,
kvm, linux-s390, linux-kernel, stable, Michael Mueller
Hi Halil,
Dne 31. 01. 22 v 12:53 Halil Pasic napsal(a):
> On Mon, 31 Jan 2022 11:13:18 +0100
> Petr Tesařík <ptesarik@suse.cz> wrote:
>
>> Hi Halil,
>>
>> Dne 27. 08. 21 v 14:54 Halil Pasic napsal(a):
>>> While in practice vcpu->vcpu_idx == vcpu->vcp_id is often true,
>>> it may not always be, and we must not rely on this.
>>>
>>> Currently kvm->arch.idle_mask is indexed by vcpu_id, which implies
>>> that code like
>>> for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
>>> vcpu = kvm_get_vcpu(kvm, vcpu_id);
>>> do_stuff(vcpu);
>>> }
>>> is not legit. The trouble is, we do actually use kvm->arch.idle_mask
>>> like this. To fix this problem we have two options. Either use
>>> kvm_get_vcpu_by_id(vcpu_id), which would loop to find the right vcpu_id,
>>> or switch to indexing via vcpu_idx. The latter is preferable for obvious
>>> reasons.
>>
>> I'm just backporting this fix to SLES 12 SP5, and I've noticed that
>> there is still this code in __floating_irq_kick():
>>
>> /* find idle VCPUs first, then round robin */
>> sigcpu = find_first_bit(fi->idle_mask, online_vcpus);
>> /* ... round robin loop removed ...
>> dst_vcpu = kvm_get_vcpu(kvm, sigcpu);
>>
>> It seems to me that this does exactly the thing that is not legit, but
>> I'm no expert. Did I miss something?
>>
>
> We made that legit by making the N-th bit in idle_mask correspond to the
> vcpu whose vcpu_idx == N. The second argument of kvm_get_vcpu() is the
> vcpu_idx. IMHO that ain't super-intuitive because it ain't spelled out.
>
> So before this was a mismatch (with a vcpu_id based bitmap we would have
> to use kvm_get_vcpu_by_id()), and with this patch applied this code
> becomes legit because both idle_mask and kvm_get_vcpu() operate with
> vcpu_idx.
>
> Does that make sense?
Yes!
> I'm sorry the commit message did not convey this clearly enough...
No, it's not your fault. I didn't pay enough attention to the change,
and with vcpu_id and vcpu_idx being so similar I got confused.
In short, there's no bug now, indeed. Thanks for your patience.
Petr T
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx
2022-01-31 10:13 ` Petr Tesařík
@ 2022-01-31 11:53 ` Halil Pasic
2022-01-31 12:09 ` Petr Tesařík
0 siblings, 1 reply; 20+ messages in thread
From: Halil Pasic @ 2022-01-31 11:53 UTC (permalink / raw)
To: Petr Tesařík
Cc: Christian Borntraeger, Janosch Frank, David Hildenbrand,
Cornelia Huck, Claudio Imbrenda, Heiko Carstens, Vasily Gorbik,
kvm, linux-s390, linux-kernel, stable, Michael Mueller,
Halil Pasic
On Mon, 31 Jan 2022 11:13:18 +0100
Petr Tesařík <ptesarik@suse.cz> wrote:
> Hi Halil,
>
> Dne 27. 08. 21 v 14:54 Halil Pasic napsal(a):
> > While in practice vcpu->vcpu_idx == vcpu->vcp_id is often true,
> > it may not always be, and we must not rely on this.
> >
> > Currently kvm->arch.idle_mask is indexed by vcpu_id, which implies
> > that code like
> > for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
> > vcpu = kvm_get_vcpu(kvm, vcpu_id);
> > do_stuff(vcpu);
> > }
> > is not legit. The trouble is, we do actually use kvm->arch.idle_mask
> > like this. To fix this problem we have two options. Either use
> > kvm_get_vcpu_by_id(vcpu_id), which would loop to find the right vcpu_id,
> > or switch to indexing via vcpu_idx. The latter is preferable for obvious
> > reasons.
>
> I'm just backporting this fix to SLES 12 SP5, and I've noticed that
> there is still this code in __floating_irq_kick():
>
> /* find idle VCPUs first, then round robin */
> sigcpu = find_first_bit(fi->idle_mask, online_vcpus);
> /* ... round robin loop removed ...
> dst_vcpu = kvm_get_vcpu(kvm, sigcpu);
>
> It seems to me that this does exactly the thing that is not legit, but
> I'm no expert. Did I miss something?
>
We made that legit by making the N-th bit in idle_mask correspond to the
vcpu whose vcpu_idx == N. The second argument of kvm_get_vcpu() is the
vcpu_idx. IMHO that ain't super-intuitive because it ain't spelled out.
So before this was a mismatch (with a vcpu_id based bitmap we would have
to use kvm_get_vcpu_by_id()), and with this patch applied this code
becomes legit because both idle_mask and kvm_get_vcpu() operate with
vcpu_idx.
Does that make sense?
I'm sorry the commit message did not convey this clearly enough...
Regards,
Halil
> Petr T
>
> > Let us make switch from indexing kvm->arch.idle_mask by vcpu_id to
> > indexing it by vcpu_idx. To keep gisa_int.kicked_mask indexed by the
> > same index as idle_mask lets make the same change for it as well.
> >
> > Signed-off-by: Halil Pasic <pasic@linux.ibm.com>
> > Fixes: 1ee0bc559dc3 ("KVM: s390: get rid of local_int array")
> > Cc: <stable@vger.kernel.org> # 3.15+
> > ---
> > arch/s390/include/asm/kvm_host.h | 1 +
> > arch/s390/kvm/interrupt.c | 12 ++++++------
> > arch/s390/kvm/kvm-s390.c | 2 +-
> > arch/s390/kvm/kvm-s390.h | 2 +-
> > 4 files changed, 9 insertions(+), 8 deletions(-)
> >
> > diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
> > index 161a9e12bfb8..630eab0fa176 100644
> > --- a/arch/s390/include/asm/kvm_host.h
> > +++ b/arch/s390/include/asm/kvm_host.h
> > @@ -957,6 +957,7 @@ struct kvm_arch{
> > atomic64_t cmma_dirty_pages;
> > /* subset of available cpu features enabled by user space */
> > DECLARE_BITMAP(cpu_feat, KVM_S390_VM_CPU_FEAT_NR_BITS);
> > + /* indexed by vcpu_idx */
> > DECLARE_BITMAP(idle_mask, KVM_MAX_VCPUS);
> > struct kvm_s390_gisa_interrupt gisa_int;
> > struct kvm_s390_pv pv;
> > diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
> > index d548d60caed2..16256e17a544 100644
> > --- a/arch/s390/kvm/interrupt.c
> > +++ b/arch/s390/kvm/interrupt.c
> > @@ -419,13 +419,13 @@ static unsigned long deliverable_irqs(struct kvm_vcpu *vcpu)
> > static void __set_cpu_idle(struct kvm_vcpu *vcpu)
> > {
> > kvm_s390_set_cpuflags(vcpu, CPUSTAT_WAIT);
> > - set_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
> > + set_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
> > }
> >
> > static void __unset_cpu_idle(struct kvm_vcpu *vcpu)
> > {
> > kvm_s390_clear_cpuflags(vcpu, CPUSTAT_WAIT);
> > - clear_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
> > + clear_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
> > }
> >
> > static void __reset_intercept_indicators(struct kvm_vcpu *vcpu)
> > @@ -3050,18 +3050,18 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len)
> >
> > static void __airqs_kick_single_vcpu(struct kvm *kvm, u8 deliverable_mask)
> > {
> > - int vcpu_id, online_vcpus = atomic_read(&kvm->online_vcpus);
> > + int vcpu_idx, online_vcpus = atomic_read(&kvm->online_vcpus);
> > struct kvm_s390_gisa_interrupt *gi = &kvm->arch.gisa_int;
> > struct kvm_vcpu *vcpu;
> >
> > - for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
> > - vcpu = kvm_get_vcpu(kvm, vcpu_id);
> > + for_each_set_bit(vcpu_idx, kvm->arch.idle_mask, online_vcpus) {
> > + vcpu = kvm_get_vcpu(kvm, vcpu_idx);
> > if (psw_ioint_disabled(vcpu))
> > continue;
> > deliverable_mask &= (u8)(vcpu->arch.sie_block->gcr[6] >> 24);
> > if (deliverable_mask) {
> > /* lately kicked but not yet running */
> > - if (test_and_set_bit(vcpu_id, gi->kicked_mask))
> > + if (test_and_set_bit(vcpu_idx, gi->kicked_mask))
> > return;
> > kvm_s390_vcpu_wakeup(vcpu);
> > return;
> > diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
> > index 4527ac7b5961..8580543c5bc3 100644
> > --- a/arch/s390/kvm/kvm-s390.c
> > +++ b/arch/s390/kvm/kvm-s390.c
> > @@ -4044,7 +4044,7 @@ static int vcpu_pre_run(struct kvm_vcpu *vcpu)
> > kvm_s390_patch_guest_per_regs(vcpu);
> > }
> >
> > - clear_bit(vcpu->vcpu_id, vcpu->kvm->arch.gisa_int.kicked_mask);
> > + clear_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.gisa_int.kicked_mask);
> >
> > vcpu->arch.sie_block->icptcode = 0;
> > cpuflags = atomic_read(&vcpu->arch.sie_block->cpuflags);
> > diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h
> > index 9fad25109b0d..ecd741ee3276 100644
> > --- a/arch/s390/kvm/kvm-s390.h
> > +++ b/arch/s390/kvm/kvm-s390.h
> > @@ -79,7 +79,7 @@ static inline int is_vcpu_stopped(struct kvm_vcpu *vcpu)
> >
> > static inline int is_vcpu_idle(struct kvm_vcpu *vcpu)
> > {
> > - return test_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
> > + return test_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
> > }
> >
> > static inline int kvm_is_ucontrol(struct kvm *kvm)
> >
> > base-commit: 77dd11439b86e3f7990e4c0c9e0b67dca82750ba
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx
2021-08-27 12:54 [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx Halil Pasic
` (2 preceding siblings ...)
2021-08-28 11:04 ` Michael Mueller
@ 2022-01-31 10:13 ` Petr Tesařík
2022-01-31 11:53 ` Halil Pasic
3 siblings, 1 reply; 20+ messages in thread
From: Petr Tesařík @ 2022-01-31 10:13 UTC (permalink / raw)
To: Halil Pasic, Christian Borntraeger, Janosch Frank,
David Hildenbrand, Cornelia Huck, Claudio Imbrenda,
Heiko Carstens, Vasily Gorbik, kvm, linux-s390, linux-kernel
Cc: stable, Michael Mueller
Hi Halil,
Dne 27. 08. 21 v 14:54 Halil Pasic napsal(a):
> While in practice vcpu->vcpu_idx == vcpu->vcp_id is often true,
> it may not always be, and we must not rely on this.
>
> Currently kvm->arch.idle_mask is indexed by vcpu_id, which implies
> that code like
> for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
> vcpu = kvm_get_vcpu(kvm, vcpu_id);
> do_stuff(vcpu);
> }
> is not legit. The trouble is, we do actually use kvm->arch.idle_mask
> like this. To fix this problem we have two options. Either use
> kvm_get_vcpu_by_id(vcpu_id), which would loop to find the right vcpu_id,
> or switch to indexing via vcpu_idx. The latter is preferable for obvious
> reasons.
I'm just backporting this fix to SLES 12 SP5, and I've noticed that
there is still this code in __floating_irq_kick():
/* find idle VCPUs first, then round robin */
sigcpu = find_first_bit(fi->idle_mask, online_vcpus);
/* ... round robin loop removed ...
dst_vcpu = kvm_get_vcpu(kvm, sigcpu);
It seems to me that this does exactly the thing that is not legit, but
I'm no expert. Did I miss something?
Petr T
> Let us make switch from indexing kvm->arch.idle_mask by vcpu_id to
> indexing it by vcpu_idx. To keep gisa_int.kicked_mask indexed by the
> same index as idle_mask lets make the same change for it as well.
>
> Signed-off-by: Halil Pasic <pasic@linux.ibm.com>
> Fixes: 1ee0bc559dc3 ("KVM: s390: get rid of local_int array")
> Cc: <stable@vger.kernel.org> # 3.15+
> ---
> arch/s390/include/asm/kvm_host.h | 1 +
> arch/s390/kvm/interrupt.c | 12 ++++++------
> arch/s390/kvm/kvm-s390.c | 2 +-
> arch/s390/kvm/kvm-s390.h | 2 +-
> 4 files changed, 9 insertions(+), 8 deletions(-)
>
> diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
> index 161a9e12bfb8..630eab0fa176 100644
> --- a/arch/s390/include/asm/kvm_host.h
> +++ b/arch/s390/include/asm/kvm_host.h
> @@ -957,6 +957,7 @@ struct kvm_arch{
> atomic64_t cmma_dirty_pages;
> /* subset of available cpu features enabled by user space */
> DECLARE_BITMAP(cpu_feat, KVM_S390_VM_CPU_FEAT_NR_BITS);
> + /* indexed by vcpu_idx */
> DECLARE_BITMAP(idle_mask, KVM_MAX_VCPUS);
> struct kvm_s390_gisa_interrupt gisa_int;
> struct kvm_s390_pv pv;
> diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
> index d548d60caed2..16256e17a544 100644
> --- a/arch/s390/kvm/interrupt.c
> +++ b/arch/s390/kvm/interrupt.c
> @@ -419,13 +419,13 @@ static unsigned long deliverable_irqs(struct kvm_vcpu *vcpu)
> static void __set_cpu_idle(struct kvm_vcpu *vcpu)
> {
> kvm_s390_set_cpuflags(vcpu, CPUSTAT_WAIT);
> - set_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
> + set_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
> }
>
> static void __unset_cpu_idle(struct kvm_vcpu *vcpu)
> {
> kvm_s390_clear_cpuflags(vcpu, CPUSTAT_WAIT);
> - clear_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
> + clear_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
> }
>
> static void __reset_intercept_indicators(struct kvm_vcpu *vcpu)
> @@ -3050,18 +3050,18 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len)
>
> static void __airqs_kick_single_vcpu(struct kvm *kvm, u8 deliverable_mask)
> {
> - int vcpu_id, online_vcpus = atomic_read(&kvm->online_vcpus);
> + int vcpu_idx, online_vcpus = atomic_read(&kvm->online_vcpus);
> struct kvm_s390_gisa_interrupt *gi = &kvm->arch.gisa_int;
> struct kvm_vcpu *vcpu;
>
> - for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
> - vcpu = kvm_get_vcpu(kvm, vcpu_id);
> + for_each_set_bit(vcpu_idx, kvm->arch.idle_mask, online_vcpus) {
> + vcpu = kvm_get_vcpu(kvm, vcpu_idx);
> if (psw_ioint_disabled(vcpu))
> continue;
> deliverable_mask &= (u8)(vcpu->arch.sie_block->gcr[6] >> 24);
> if (deliverable_mask) {
> /* lately kicked but not yet running */
> - if (test_and_set_bit(vcpu_id, gi->kicked_mask))
> + if (test_and_set_bit(vcpu_idx, gi->kicked_mask))
> return;
> kvm_s390_vcpu_wakeup(vcpu);
> return;
> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
> index 4527ac7b5961..8580543c5bc3 100644
> --- a/arch/s390/kvm/kvm-s390.c
> +++ b/arch/s390/kvm/kvm-s390.c
> @@ -4044,7 +4044,7 @@ static int vcpu_pre_run(struct kvm_vcpu *vcpu)
> kvm_s390_patch_guest_per_regs(vcpu);
> }
>
> - clear_bit(vcpu->vcpu_id, vcpu->kvm->arch.gisa_int.kicked_mask);
> + clear_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.gisa_int.kicked_mask);
>
> vcpu->arch.sie_block->icptcode = 0;
> cpuflags = atomic_read(&vcpu->arch.sie_block->cpuflags);
> diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h
> index 9fad25109b0d..ecd741ee3276 100644
> --- a/arch/s390/kvm/kvm-s390.h
> +++ b/arch/s390/kvm/kvm-s390.h
> @@ -79,7 +79,7 @@ static inline int is_vcpu_stopped(struct kvm_vcpu *vcpu)
>
> static inline int is_vcpu_idle(struct kvm_vcpu *vcpu)
> {
> - return test_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
> + return test_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
> }
>
> static inline int kvm_is_ucontrol(struct kvm *kvm)
>
> base-commit: 77dd11439b86e3f7990e4c0c9e0b67dca82750ba
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx
2021-08-27 21:23 ` Halil Pasic
@ 2021-08-29 6:25 ` Christian Borntraeger
0 siblings, 0 replies; 20+ messages in thread
From: Christian Borntraeger @ 2021-08-29 6:25 UTC (permalink / raw)
To: Halil Pasic
Cc: Claudio Imbrenda, Janosch Frank, David Hildenbrand,
Cornelia Huck, Heiko Carstens, Vasily Gorbik, kvm, linux-s390,
linux-kernel, Michael Mueller
On 27.08.21 23:23, Halil Pasic wrote:
> On Fri, 27 Aug 2021 18:36:48 +0200
> Christian Borntraeger <borntraeger@de.ibm.com> wrote:
>
>> On 27.08.21 16:06, Claudio Imbrenda wrote:
>>> On Fri, 27 Aug 2021 14:54:29 +0200
>>> Halil Pasic <pasic@linux.ibm.com> wrote:
>>>
>>>> While in practice vcpu->vcpu_idx == vcpu->vcp_id is often true,
>
> s/vcp_id/vcpu_id/
>
>>>> it may not always be, and we must not rely on this.
>>>
>>> why?
>>>
>>> maybe add a simple explanation of why vcpu_idx and vcpu_id can be
>>> different, namely:
>>> KVM decides the vcpu_idx, userspace decides the vcpu_id, thus the two
>>> might not match
>>>
>>>>
>>>> Currently kvm->arch.idle_mask is indexed by vcpu_id, which implies
>>>> that code like
>>>> for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
>>>> vcpu = kvm_get_vcpu(kvm, vcpu_id);
>>>
>>> you can also add a sentence to clarify that kvm_get_vcpu expects an
>>> vcpu_idx, not an vcpu_id.
>>>
>>>> do_stuff(vcpu);
>>
>> I will modify the patch description accordingly before sending to Paolo.
>> Thanks for noticing.
>
> Can you also please fix the typo I pointed out above (in the first line
> of the long description).
I already queued, but I think this is not a big deal.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx
2021-08-27 12:54 [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx Halil Pasic
2021-08-27 13:42 ` Christian Borntraeger
2021-08-27 14:06 ` Claudio Imbrenda
@ 2021-08-28 11:04 ` Michael Mueller
2022-01-31 10:13 ` Petr Tesařík
3 siblings, 0 replies; 20+ messages in thread
From: Michael Mueller @ 2021-08-28 11:04 UTC (permalink / raw)
To: Halil Pasic, Christian Borntraeger, Janosch Frank,
David Hildenbrand, Cornelia Huck, Claudio Imbrenda,
Heiko Carstens, Vasily Gorbik, kvm, linux-s390, linux-kernel
Cc: stable
On 27.08.21 14:54, Halil Pasic wrote:
> While in practice vcpu->vcpu_idx == vcpu->vcp_id is often true,
> it may not always be, and we must not rely on this.
>
> Currently kvm->arch.idle_mask is indexed by vcpu_id, which implies
> that code like
> for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
> vcpu = kvm_get_vcpu(kvm, vcpu_id);
> do_stuff(vcpu);
> }
> is not legit. The trouble is, we do actually use kvm->arch.idle_mask
> like this. To fix this problem we have two options. Either use
> kvm_get_vcpu_by_id(vcpu_id), which would loop to find the right vcpu_id,
> or switch to indexing via vcpu_idx. The latter is preferable for obvious
> reasons.
>
> Let us make switch from indexing kvm->arch.idle_mask by vcpu_id to
> indexing it by vcpu_idx. To keep gisa_int.kicked_mask indexed by the
> same index as idle_mask lets make the same change for it as well.
>
> Signed-off-by: Halil Pasic <pasic@linux.ibm.com>
> Fixes: 1ee0bc559dc3 ("KVM: s390: get rid of local_int array")
> Cc: <stable@vger.kernel.org> # 3.15+
> ---
> arch/s390/include/asm/kvm_host.h | 1 +
> arch/s390/kvm/interrupt.c | 12 ++++++------
> arch/s390/kvm/kvm-s390.c | 2 +-
> arch/s390/kvm/kvm-s390.h | 2 +-
> 4 files changed, 9 insertions(+), 8 deletions(-)
>
> diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
> index 161a9e12bfb8..630eab0fa176 100644
> --- a/arch/s390/include/asm/kvm_host.h
> +++ b/arch/s390/include/asm/kvm_host.h
> @@ -957,6 +957,7 @@ struct kvm_arch{
> atomic64_t cmma_dirty_pages;
> /* subset of available cpu features enabled by user space */
> DECLARE_BITMAP(cpu_feat, KVM_S390_VM_CPU_FEAT_NR_BITS);
> + /* indexed by vcpu_idx */
> DECLARE_BITMAP(idle_mask, KVM_MAX_VCPUS);
> struct kvm_s390_gisa_interrupt gisa_int;
> struct kvm_s390_pv pv;
> diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
> index d548d60caed2..16256e17a544 100644
> --- a/arch/s390/kvm/interrupt.c
> +++ b/arch/s390/kvm/interrupt.c
> @@ -419,13 +419,13 @@ static unsigned long deliverable_irqs(struct kvm_vcpu *vcpu)
> static void __set_cpu_idle(struct kvm_vcpu *vcpu)
> {
> kvm_s390_set_cpuflags(vcpu, CPUSTAT_WAIT);
> - set_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
> + set_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
> }
>
> static void __unset_cpu_idle(struct kvm_vcpu *vcpu)
> {
> kvm_s390_clear_cpuflags(vcpu, CPUSTAT_WAIT);
> - clear_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
> + clear_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
> }
>
> static void __reset_intercept_indicators(struct kvm_vcpu *vcpu)
> @@ -3050,18 +3050,18 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len)
>
> static void __airqs_kick_single_vcpu(struct kvm *kvm, u8 deliverable_mask)
> {
> - int vcpu_id, online_vcpus = atomic_read(&kvm->online_vcpus);
> + int vcpu_idx, online_vcpus = atomic_read(&kvm->online_vcpus);
> struct kvm_s390_gisa_interrupt *gi = &kvm->arch.gisa_int;
> struct kvm_vcpu *vcpu;
>
> - for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
> - vcpu = kvm_get_vcpu(kvm, vcpu_id);
> + for_each_set_bit(vcpu_idx, kvm->arch.idle_mask, online_vcpus) {
> + vcpu = kvm_get_vcpu(kvm, vcpu_idx);
> if (psw_ioint_disabled(vcpu))
> continue;
> deliverable_mask &= (u8)(vcpu->arch.sie_block->gcr[6] >> 24);
> if (deliverable_mask) {
> /* lately kicked but not yet running */
> - if (test_and_set_bit(vcpu_id, gi->kicked_mask))
> + if (test_and_set_bit(vcpu_idx, gi->kicked_mask))
> return;
> kvm_s390_vcpu_wakeup(vcpu);
> return;
> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
> index 4527ac7b5961..8580543c5bc3 100644
> --- a/arch/s390/kvm/kvm-s390.c
> +++ b/arch/s390/kvm/kvm-s390.c
> @@ -4044,7 +4044,7 @@ static int vcpu_pre_run(struct kvm_vcpu *vcpu)
> kvm_s390_patch_guest_per_regs(vcpu);
> }
>
> - clear_bit(vcpu->vcpu_id, vcpu->kvm->arch.gisa_int.kicked_mask);
> + clear_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.gisa_int.kicked_mask);
>
> vcpu->arch.sie_block->icptcode = 0;
> cpuflags = atomic_read(&vcpu->arch.sie_block->cpuflags);
> diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h
> index 9fad25109b0d..ecd741ee3276 100644
> --- a/arch/s390/kvm/kvm-s390.h
> +++ b/arch/s390/kvm/kvm-s390.h
> @@ -79,7 +79,7 @@ static inline int is_vcpu_stopped(struct kvm_vcpu *vcpu)
>
> static inline int is_vcpu_idle(struct kvm_vcpu *vcpu)
> {
> - return test_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
> + return test_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
> }
>
> static inline int kvm_is_ucontrol(struct kvm *kvm)
>
> base-commit: 77dd11439b86e3f7990e4c0c9e0b67dca82750ba
Reviewed-by: Michael Mueller <mimu@linux.ibm.com>
I did also run some tests and have prepared a qemu that will choose
vcpu_id's that are not equal to the index in the kernel.
Michael
>
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx
2021-08-27 16:36 ` Christian Borntraeger
@ 2021-08-27 21:23 ` Halil Pasic
2021-08-29 6:25 ` Christian Borntraeger
0 siblings, 1 reply; 20+ messages in thread
From: Halil Pasic @ 2021-08-27 21:23 UTC (permalink / raw)
To: Christian Borntraeger
Cc: Claudio Imbrenda, Janosch Frank, David Hildenbrand,
Cornelia Huck, Heiko Carstens, Vasily Gorbik, kvm, linux-s390,
linux-kernel, stable, Michael Mueller, Halil Pasic
On Fri, 27 Aug 2021 18:36:48 +0200
Christian Borntraeger <borntraeger@de.ibm.com> wrote:
> On 27.08.21 16:06, Claudio Imbrenda wrote:
> > On Fri, 27 Aug 2021 14:54:29 +0200
> > Halil Pasic <pasic@linux.ibm.com> wrote:
> >
> >> While in practice vcpu->vcpu_idx == vcpu->vcp_id is often true,
s/vcp_id/vcpu_id/
> >> it may not always be, and we must not rely on this.
> >
> > why?
> >
> > maybe add a simple explanation of why vcpu_idx and vcpu_id can be
> > different, namely:
> > KVM decides the vcpu_idx, userspace decides the vcpu_id, thus the two
> > might not match
> >
> >>
> >> Currently kvm->arch.idle_mask is indexed by vcpu_id, which implies
> >> that code like
> >> for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
> >> vcpu = kvm_get_vcpu(kvm, vcpu_id);
> >
> > you can also add a sentence to clarify that kvm_get_vcpu expects an
> > vcpu_idx, not an vcpu_id.
> >
> >> do_stuff(vcpu);
>
> I will modify the patch description accordingly before sending to Paolo.
> Thanks for noticing.
Can you also please fix the typo I pointed out above (in the first line
of the long description).
Thanks!
Halil
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx
2021-08-27 14:06 ` Claudio Imbrenda
2021-08-27 16:36 ` Christian Borntraeger
@ 2021-08-27 21:19 ` Halil Pasic
1 sibling, 0 replies; 20+ messages in thread
From: Halil Pasic @ 2021-08-27 21:19 UTC (permalink / raw)
To: Claudio Imbrenda
Cc: Christian Borntraeger, Janosch Frank, David Hildenbrand,
Cornelia Huck, Heiko Carstens, Vasily Gorbik, kvm, linux-s390,
linux-kernel, stable, Michael Mueller, Halil Pasic
On Fri, 27 Aug 2021 16:06:16 +0200
Claudio Imbrenda <imbrenda@linux.ibm.com> wrote:
> On Fri, 27 Aug 2021 14:54:29 +0200
> Halil Pasic <pasic@linux.ibm.com> wrote:
>
> > While in practice vcpu->vcpu_idx == vcpu->vcp_id is often true,
s/vcp_id/vcpu_id/
> > it may not always be, and we must not rely on this.
>
> why?
>
> maybe add a simple explanation of why vcpu_idx and vcpu_id can be
> different, namely:
> KVM decides the vcpu_idx, userspace decides the vcpu_id, thus the two
> might not match
Not sure that is a good explanation. A quote from
Documentation/virt/kvm/api.rst:
"""
4.7 KVM_CREATE_VCPU
-------------------
:Capability: basic
:Architectures: all
:Type: vm ioctl
:Parameters: vcpu id (apic id on x86)
:Returns: vcpu fd on success, -1 on error
This API adds a vcpu to a virtual machine. No more than max_vcpus may be added.
The vcpu id is an integer in the range [0, max_vcpu_id).
The recommended max_vcpus value can be retrieved using the KVM_CAP_NR_VCPUS of
the KVM_CHECK_EXTENSION ioctl() at run-time.
The maximum possible value for max_vcpus can be retrieved using the
KVM_CAP_MAX_VCPUS of the KVM_CHECK_EXTENSION ioctl() at run-time.
"""
Based on that and a quick look at the code, it looks to me like the
set of valid vcpu_id values are a subset of the range of vcpu_idx-es,
i.e. that kvm could decide to choose vcpu_id for the value of vcpu_idx.
I don't think it should, but it could. Were the set of valid vcpu_id
values not a subset of the set of supported vcpu_idx values, then one
could argue that this is why.
I didn't want to get into explaining the why, I just wanted to state the
fact.
>
> >
> > Currently kvm->arch.idle_mask is indexed by vcpu_id, which implies
> > that code like
> > for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
> > vcpu = kvm_get_vcpu(kvm, vcpu_id);
>
> you can also add a sentence to clarify that kvm_get_vcpu expects an
> vcpu_idx, not an vcpu_id.
maybe ...
>
> > do_stuff(vcpu);
> > }
> > is not legit. The trouble is, we do actually use kvm->arch.idle_mask
... s/legit\./legit, because kvm_get_vcpu() expects a vcpu_idx and not a
vcpu_id.
But I agree kvm_get_vcpu(kvm, vcpu_id); does not scream BUG at me while
kvm_get_vcpu_by_idx(kvm, vcpu_id) would look much more suspicious.
[..]
>
> otherwise looks good to me.
>
> Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Thanks for your reveiew!
Halil
[..]
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx
2021-08-27 14:06 ` Claudio Imbrenda
@ 2021-08-27 16:36 ` Christian Borntraeger
2021-08-27 21:23 ` Halil Pasic
2021-08-27 21:19 ` Halil Pasic
1 sibling, 1 reply; 20+ messages in thread
From: Christian Borntraeger @ 2021-08-27 16:36 UTC (permalink / raw)
To: Claudio Imbrenda, Halil Pasic
Cc: Janosch Frank, David Hildenbrand, Cornelia Huck, Heiko Carstens,
Vasily Gorbik, kvm, linux-s390, linux-kernel, stable,
Michael Mueller
On 27.08.21 16:06, Claudio Imbrenda wrote:
> On Fri, 27 Aug 2021 14:54:29 +0200
> Halil Pasic <pasic@linux.ibm.com> wrote:
>
>> While in practice vcpu->vcpu_idx == vcpu->vcp_id is often true,
>> it may not always be, and we must not rely on this.
>
> why?
>
> maybe add a simple explanation of why vcpu_idx and vcpu_id can be
> different, namely:
> KVM decides the vcpu_idx, userspace decides the vcpu_id, thus the two
> might not match
>
>>
>> Currently kvm->arch.idle_mask is indexed by vcpu_id, which implies
>> that code like
>> for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
>> vcpu = kvm_get_vcpu(kvm, vcpu_id);
>
> you can also add a sentence to clarify that kvm_get_vcpu expects an
> vcpu_idx, not an vcpu_id.
>
>> do_stuff(vcpu);
I will modify the patch description accordingly before sending to Paolo.
Thanks for noticing.
>> }
>> is not legit. The trouble is, we do actually use kvm->arch.idle_mask
>> like this. To fix this problem we have two options. Either use
>> kvm_get_vcpu_by_id(vcpu_id), which would loop to find the right vcpu_id,
>> or switch to indexing via vcpu_idx. The latter is preferable for obvious
>> reasons.
>>
>> Let us make switch from indexing kvm->arch.idle_mask by vcpu_id to
>> indexing it by vcpu_idx. To keep gisa_int.kicked_mask indexed by the
>> same index as idle_mask lets make the same change for it as well.
>>
>> Signed-off-by: Halil Pasic <pasic@linux.ibm.com>
>> Fixes: 1ee0bc559dc3 ("KVM: s390: get rid of local_int array")
>
> otherwise looks good to me.
>
> Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
>
>> Cc: <stable@vger.kernel.org> # 3.15+
>> ---
>> arch/s390/include/asm/kvm_host.h | 1 +
>> arch/s390/kvm/interrupt.c | 12 ++++++------
>> arch/s390/kvm/kvm-s390.c | 2 +-
>> arch/s390/kvm/kvm-s390.h | 2 +-
>> 4 files changed, 9 insertions(+), 8 deletions(-)
>>
>> diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
>> index 161a9e12bfb8..630eab0fa176 100644
>> --- a/arch/s390/include/asm/kvm_host.h
>> +++ b/arch/s390/include/asm/kvm_host.h
>> @@ -957,6 +957,7 @@ struct kvm_arch{
>> atomic64_t cmma_dirty_pages;
>> /* subset of available cpu features enabled by user space */
>> DECLARE_BITMAP(cpu_feat, KVM_S390_VM_CPU_FEAT_NR_BITS);
>> + /* indexed by vcpu_idx */
>> DECLARE_BITMAP(idle_mask, KVM_MAX_VCPUS);
>> struct kvm_s390_gisa_interrupt gisa_int;
>> struct kvm_s390_pv pv;
>> diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
>> index d548d60caed2..16256e17a544 100644
>> --- a/arch/s390/kvm/interrupt.c
>> +++ b/arch/s390/kvm/interrupt.c
>> @@ -419,13 +419,13 @@ static unsigned long deliverable_irqs(struct kvm_vcpu *vcpu)
>> static void __set_cpu_idle(struct kvm_vcpu *vcpu)
>> {
>> kvm_s390_set_cpuflags(vcpu, CPUSTAT_WAIT);
>> - set_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
>> + set_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
>> }
>>
>> static void __unset_cpu_idle(struct kvm_vcpu *vcpu)
>> {
>> kvm_s390_clear_cpuflags(vcpu, CPUSTAT_WAIT);
>> - clear_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
>> + clear_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
>> }
>>
>> static void __reset_intercept_indicators(struct kvm_vcpu *vcpu)
>> @@ -3050,18 +3050,18 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len)
>>
>> static void __airqs_kick_single_vcpu(struct kvm *kvm, u8 deliverable_mask)
>> {
>> - int vcpu_id, online_vcpus = atomic_read(&kvm->online_vcpus);
>> + int vcpu_idx, online_vcpus = atomic_read(&kvm->online_vcpus);
>> struct kvm_s390_gisa_interrupt *gi = &kvm->arch.gisa_int;
>> struct kvm_vcpu *vcpu;
>>
>> - for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
>> - vcpu = kvm_get_vcpu(kvm, vcpu_id);
>> + for_each_set_bit(vcpu_idx, kvm->arch.idle_mask, online_vcpus) {
>> + vcpu = kvm_get_vcpu(kvm, vcpu_idx);
>> if (psw_ioint_disabled(vcpu))
>> continue;
>> deliverable_mask &= (u8)(vcpu->arch.sie_block->gcr[6] >> 24);
>> if (deliverable_mask) {
>> /* lately kicked but not yet running */
>> - if (test_and_set_bit(vcpu_id, gi->kicked_mask))
>> + if (test_and_set_bit(vcpu_idx, gi->kicked_mask))
>> return;
>> kvm_s390_vcpu_wakeup(vcpu);
>> return;
>> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
>> index 4527ac7b5961..8580543c5bc3 100644
>> --- a/arch/s390/kvm/kvm-s390.c
>> +++ b/arch/s390/kvm/kvm-s390.c
>> @@ -4044,7 +4044,7 @@ static int vcpu_pre_run(struct kvm_vcpu *vcpu)
>> kvm_s390_patch_guest_per_regs(vcpu);
>> }
>>
>> - clear_bit(vcpu->vcpu_id, vcpu->kvm->arch.gisa_int.kicked_mask);
>> + clear_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.gisa_int.kicked_mask);
>>
>> vcpu->arch.sie_block->icptcode = 0;
>> cpuflags = atomic_read(&vcpu->arch.sie_block->cpuflags);
>> diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h
>> index 9fad25109b0d..ecd741ee3276 100644
>> --- a/arch/s390/kvm/kvm-s390.h
>> +++ b/arch/s390/kvm/kvm-s390.h
>> @@ -79,7 +79,7 @@ static inline int is_vcpu_stopped(struct kvm_vcpu *vcpu)
>>
>> static inline int is_vcpu_idle(struct kvm_vcpu *vcpu)
>> {
>> - return test_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
>> + return test_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
>> }
>>
>> static inline int kvm_is_ucontrol(struct kvm *kvm)
>>
>> base-commit: 77dd11439b86e3f7990e4c0c9e0b67dca82750ba
>
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx
2021-08-27 12:54 [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx Halil Pasic
2021-08-27 13:42 ` Christian Borntraeger
@ 2021-08-27 14:06 ` Claudio Imbrenda
2021-08-27 16:36 ` Christian Borntraeger
2021-08-27 21:19 ` Halil Pasic
2021-08-28 11:04 ` Michael Mueller
2022-01-31 10:13 ` Petr Tesařík
3 siblings, 2 replies; 20+ messages in thread
From: Claudio Imbrenda @ 2021-08-27 14:06 UTC (permalink / raw)
To: Halil Pasic
Cc: Christian Borntraeger, Janosch Frank, David Hildenbrand,
Cornelia Huck, Heiko Carstens, Vasily Gorbik, kvm, linux-s390,
linux-kernel, stable, Michael Mueller
On Fri, 27 Aug 2021 14:54:29 +0200
Halil Pasic <pasic@linux.ibm.com> wrote:
> While in practice vcpu->vcpu_idx == vcpu->vcp_id is often true,
> it may not always be, and we must not rely on this.
why?
maybe add a simple explanation of why vcpu_idx and vcpu_id can be
different, namely:
KVM decides the vcpu_idx, userspace decides the vcpu_id, thus the two
might not match
>
> Currently kvm->arch.idle_mask is indexed by vcpu_id, which implies
> that code like
> for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
> vcpu = kvm_get_vcpu(kvm, vcpu_id);
you can also add a sentence to clarify that kvm_get_vcpu expects an
vcpu_idx, not an vcpu_id.
> do_stuff(vcpu);
> }
> is not legit. The trouble is, we do actually use kvm->arch.idle_mask
> like this. To fix this problem we have two options. Either use
> kvm_get_vcpu_by_id(vcpu_id), which would loop to find the right vcpu_id,
> or switch to indexing via vcpu_idx. The latter is preferable for obvious
> reasons.
>
> Let us make switch from indexing kvm->arch.idle_mask by vcpu_id to
> indexing it by vcpu_idx. To keep gisa_int.kicked_mask indexed by the
> same index as idle_mask lets make the same change for it as well.
>
> Signed-off-by: Halil Pasic <pasic@linux.ibm.com>
> Fixes: 1ee0bc559dc3 ("KVM: s390: get rid of local_int array")
otherwise looks good to me.
Reviewed-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
> Cc: <stable@vger.kernel.org> # 3.15+
> ---
> arch/s390/include/asm/kvm_host.h | 1 +
> arch/s390/kvm/interrupt.c | 12 ++++++------
> arch/s390/kvm/kvm-s390.c | 2 +-
> arch/s390/kvm/kvm-s390.h | 2 +-
> 4 files changed, 9 insertions(+), 8 deletions(-)
>
> diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
> index 161a9e12bfb8..630eab0fa176 100644
> --- a/arch/s390/include/asm/kvm_host.h
> +++ b/arch/s390/include/asm/kvm_host.h
> @@ -957,6 +957,7 @@ struct kvm_arch{
> atomic64_t cmma_dirty_pages;
> /* subset of available cpu features enabled by user space */
> DECLARE_BITMAP(cpu_feat, KVM_S390_VM_CPU_FEAT_NR_BITS);
> + /* indexed by vcpu_idx */
> DECLARE_BITMAP(idle_mask, KVM_MAX_VCPUS);
> struct kvm_s390_gisa_interrupt gisa_int;
> struct kvm_s390_pv pv;
> diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
> index d548d60caed2..16256e17a544 100644
> --- a/arch/s390/kvm/interrupt.c
> +++ b/arch/s390/kvm/interrupt.c
> @@ -419,13 +419,13 @@ static unsigned long deliverable_irqs(struct kvm_vcpu *vcpu)
> static void __set_cpu_idle(struct kvm_vcpu *vcpu)
> {
> kvm_s390_set_cpuflags(vcpu, CPUSTAT_WAIT);
> - set_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
> + set_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
> }
>
> static void __unset_cpu_idle(struct kvm_vcpu *vcpu)
> {
> kvm_s390_clear_cpuflags(vcpu, CPUSTAT_WAIT);
> - clear_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
> + clear_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
> }
>
> static void __reset_intercept_indicators(struct kvm_vcpu *vcpu)
> @@ -3050,18 +3050,18 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len)
>
> static void __airqs_kick_single_vcpu(struct kvm *kvm, u8 deliverable_mask)
> {
> - int vcpu_id, online_vcpus = atomic_read(&kvm->online_vcpus);
> + int vcpu_idx, online_vcpus = atomic_read(&kvm->online_vcpus);
> struct kvm_s390_gisa_interrupt *gi = &kvm->arch.gisa_int;
> struct kvm_vcpu *vcpu;
>
> - for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
> - vcpu = kvm_get_vcpu(kvm, vcpu_id);
> + for_each_set_bit(vcpu_idx, kvm->arch.idle_mask, online_vcpus) {
> + vcpu = kvm_get_vcpu(kvm, vcpu_idx);
> if (psw_ioint_disabled(vcpu))
> continue;
> deliverable_mask &= (u8)(vcpu->arch.sie_block->gcr[6] >> 24);
> if (deliverable_mask) {
> /* lately kicked but not yet running */
> - if (test_and_set_bit(vcpu_id, gi->kicked_mask))
> + if (test_and_set_bit(vcpu_idx, gi->kicked_mask))
> return;
> kvm_s390_vcpu_wakeup(vcpu);
> return;
> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
> index 4527ac7b5961..8580543c5bc3 100644
> --- a/arch/s390/kvm/kvm-s390.c
> +++ b/arch/s390/kvm/kvm-s390.c
> @@ -4044,7 +4044,7 @@ static int vcpu_pre_run(struct kvm_vcpu *vcpu)
> kvm_s390_patch_guest_per_regs(vcpu);
> }
>
> - clear_bit(vcpu->vcpu_id, vcpu->kvm->arch.gisa_int.kicked_mask);
> + clear_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.gisa_int.kicked_mask);
>
> vcpu->arch.sie_block->icptcode = 0;
> cpuflags = atomic_read(&vcpu->arch.sie_block->cpuflags);
> diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h
> index 9fad25109b0d..ecd741ee3276 100644
> --- a/arch/s390/kvm/kvm-s390.h
> +++ b/arch/s390/kvm/kvm-s390.h
> @@ -79,7 +79,7 @@ static inline int is_vcpu_stopped(struct kvm_vcpu *vcpu)
>
> static inline int is_vcpu_idle(struct kvm_vcpu *vcpu)
> {
> - return test_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
> + return test_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
> }
>
> static inline int kvm_is_ucontrol(struct kvm *kvm)
>
> base-commit: 77dd11439b86e3f7990e4c0c9e0b67dca82750ba
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx
2021-08-27 12:54 [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx Halil Pasic
@ 2021-08-27 13:42 ` Christian Borntraeger
2021-08-27 14:06 ` Claudio Imbrenda
` (2 subsequent siblings)
3 siblings, 0 replies; 20+ messages in thread
From: Christian Borntraeger @ 2021-08-27 13:42 UTC (permalink / raw)
To: Halil Pasic, Janosch Frank, David Hildenbrand, Cornelia Huck,
Claudio Imbrenda, Heiko Carstens, Vasily Gorbik, kvm, linux-s390,
linux-kernel
Cc: stable, Michael Mueller
On 27.08.21 14:54, Halil Pasic wrote:
> While in practice vcpu->vcpu_idx == vcpu->vcp_id is often true,
> it may not always be, and we must not rely on this.
>
> Currently kvm->arch.idle_mask is indexed by vcpu_id, which implies
> that code like
> for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
> vcpu = kvm_get_vcpu(kvm, vcpu_id);
> do_stuff(vcpu);
> }
> is not legit. The trouble is, we do actually use kvm->arch.idle_mask
> like this. To fix this problem we have two options. Either use
> kvm_get_vcpu_by_id(vcpu_id), which would loop to find the right vcpu_id,
> or switch to indexing via vcpu_idx. The latter is preferable for obvious
> reasons.
>
> Let us make switch from indexing kvm->arch.idle_mask by vcpu_id to
> indexing it by vcpu_idx. To keep gisa_int.kicked_mask indexed by the
> same index as idle_mask lets make the same change for it as well.
>
> Signed-off-by: Halil Pasic <pasic@linux.ibm.com>
> Fixes: 1ee0bc559dc3 ("KVM: s390: get rid of local_int array")
> Cc: <stable@vger.kernel.org> # 3.15+
Reviewed-by: Christian Bornträger <borntraeger@de.ibm.com>
Will apply and wait for the nightly run.
> ---
> arch/s390/include/asm/kvm_host.h | 1 +
> arch/s390/kvm/interrupt.c | 12 ++++++------
> arch/s390/kvm/kvm-s390.c | 2 +-
> arch/s390/kvm/kvm-s390.h | 2 +-
> 4 files changed, 9 insertions(+), 8 deletions(-)
>
> diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
> index 161a9e12bfb8..630eab0fa176 100644
> --- a/arch/s390/include/asm/kvm_host.h
> +++ b/arch/s390/include/asm/kvm_host.h
> @@ -957,6 +957,7 @@ struct kvm_arch{
> atomic64_t cmma_dirty_pages;
> /* subset of available cpu features enabled by user space */
> DECLARE_BITMAP(cpu_feat, KVM_S390_VM_CPU_FEAT_NR_BITS);
> + /* indexed by vcpu_idx */
> DECLARE_BITMAP(idle_mask, KVM_MAX_VCPUS);
> struct kvm_s390_gisa_interrupt gisa_int;
> struct kvm_s390_pv pv;
> diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
> index d548d60caed2..16256e17a544 100644
> --- a/arch/s390/kvm/interrupt.c
> +++ b/arch/s390/kvm/interrupt.c
> @@ -419,13 +419,13 @@ static unsigned long deliverable_irqs(struct kvm_vcpu *vcpu)
> static void __set_cpu_idle(struct kvm_vcpu *vcpu)
> {
> kvm_s390_set_cpuflags(vcpu, CPUSTAT_WAIT);
> - set_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
> + set_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
> }
>
> static void __unset_cpu_idle(struct kvm_vcpu *vcpu)
> {
> kvm_s390_clear_cpuflags(vcpu, CPUSTAT_WAIT);
> - clear_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
> + clear_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
> }
>
> static void __reset_intercept_indicators(struct kvm_vcpu *vcpu)
> @@ -3050,18 +3050,18 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len)
>
> static void __airqs_kick_single_vcpu(struct kvm *kvm, u8 deliverable_mask)
> {
> - int vcpu_id, online_vcpus = atomic_read(&kvm->online_vcpus);
> + int vcpu_idx, online_vcpus = atomic_read(&kvm->online_vcpus);
> struct kvm_s390_gisa_interrupt *gi = &kvm->arch.gisa_int;
> struct kvm_vcpu *vcpu;
>
> - for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
> - vcpu = kvm_get_vcpu(kvm, vcpu_id);
> + for_each_set_bit(vcpu_idx, kvm->arch.idle_mask, online_vcpus) {
> + vcpu = kvm_get_vcpu(kvm, vcpu_idx);
> if (psw_ioint_disabled(vcpu))
> continue;
> deliverable_mask &= (u8)(vcpu->arch.sie_block->gcr[6] >> 24);
> if (deliverable_mask) {
> /* lately kicked but not yet running */
> - if (test_and_set_bit(vcpu_id, gi->kicked_mask))
> + if (test_and_set_bit(vcpu_idx, gi->kicked_mask))
> return;
> kvm_s390_vcpu_wakeup(vcpu);
> return;
> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
> index 4527ac7b5961..8580543c5bc3 100644
> --- a/arch/s390/kvm/kvm-s390.c
> +++ b/arch/s390/kvm/kvm-s390.c
> @@ -4044,7 +4044,7 @@ static int vcpu_pre_run(struct kvm_vcpu *vcpu)
> kvm_s390_patch_guest_per_regs(vcpu);
> }
>
> - clear_bit(vcpu->vcpu_id, vcpu->kvm->arch.gisa_int.kicked_mask);
> + clear_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.gisa_int.kicked_mask);
>
> vcpu->arch.sie_block->icptcode = 0;
> cpuflags = atomic_read(&vcpu->arch.sie_block->cpuflags);
> diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h
> index 9fad25109b0d..ecd741ee3276 100644
> --- a/arch/s390/kvm/kvm-s390.h
> +++ b/arch/s390/kvm/kvm-s390.h
> @@ -79,7 +79,7 @@ static inline int is_vcpu_stopped(struct kvm_vcpu *vcpu)
>
> static inline int is_vcpu_idle(struct kvm_vcpu *vcpu)
> {
> - return test_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
> + return test_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
> }
>
> static inline int kvm_is_ucontrol(struct kvm *kvm)
>
> base-commit: 77dd11439b86e3f7990e4c0c9e0b67dca82750ba
>
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx
@ 2021-08-27 12:54 Halil Pasic
2021-08-27 13:42 ` Christian Borntraeger
` (3 more replies)
0 siblings, 4 replies; 20+ messages in thread
From: Halil Pasic @ 2021-08-27 12:54 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank, David Hildenbrand,
Cornelia Huck, Claudio Imbrenda, Heiko Carstens, Vasily Gorbik,
kvm, linux-s390, linux-kernel
Cc: Halil Pasic, stable, Michael Mueller
While in practice vcpu->vcpu_idx == vcpu->vcp_id is often true,
it may not always be, and we must not rely on this.
Currently kvm->arch.idle_mask is indexed by vcpu_id, which implies
that code like
for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
vcpu = kvm_get_vcpu(kvm, vcpu_id);
do_stuff(vcpu);
}
is not legit. The trouble is, we do actually use kvm->arch.idle_mask
like this. To fix this problem we have two options. Either use
kvm_get_vcpu_by_id(vcpu_id), which would loop to find the right vcpu_id,
or switch to indexing via vcpu_idx. The latter is preferable for obvious
reasons.
Let us make switch from indexing kvm->arch.idle_mask by vcpu_id to
indexing it by vcpu_idx. To keep gisa_int.kicked_mask indexed by the
same index as idle_mask lets make the same change for it as well.
Signed-off-by: Halil Pasic <pasic@linux.ibm.com>
Fixes: 1ee0bc559dc3 ("KVM: s390: get rid of local_int array")
Cc: <stable@vger.kernel.org> # 3.15+
---
arch/s390/include/asm/kvm_host.h | 1 +
arch/s390/kvm/interrupt.c | 12 ++++++------
arch/s390/kvm/kvm-s390.c | 2 +-
arch/s390/kvm/kvm-s390.h | 2 +-
4 files changed, 9 insertions(+), 8 deletions(-)
diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
index 161a9e12bfb8..630eab0fa176 100644
--- a/arch/s390/include/asm/kvm_host.h
+++ b/arch/s390/include/asm/kvm_host.h
@@ -957,6 +957,7 @@ struct kvm_arch{
atomic64_t cmma_dirty_pages;
/* subset of available cpu features enabled by user space */
DECLARE_BITMAP(cpu_feat, KVM_S390_VM_CPU_FEAT_NR_BITS);
+ /* indexed by vcpu_idx */
DECLARE_BITMAP(idle_mask, KVM_MAX_VCPUS);
struct kvm_s390_gisa_interrupt gisa_int;
struct kvm_s390_pv pv;
diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
index d548d60caed2..16256e17a544 100644
--- a/arch/s390/kvm/interrupt.c
+++ b/arch/s390/kvm/interrupt.c
@@ -419,13 +419,13 @@ static unsigned long deliverable_irqs(struct kvm_vcpu *vcpu)
static void __set_cpu_idle(struct kvm_vcpu *vcpu)
{
kvm_s390_set_cpuflags(vcpu, CPUSTAT_WAIT);
- set_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
+ set_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
}
static void __unset_cpu_idle(struct kvm_vcpu *vcpu)
{
kvm_s390_clear_cpuflags(vcpu, CPUSTAT_WAIT);
- clear_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
+ clear_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
}
static void __reset_intercept_indicators(struct kvm_vcpu *vcpu)
@@ -3050,18 +3050,18 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len)
static void __airqs_kick_single_vcpu(struct kvm *kvm, u8 deliverable_mask)
{
- int vcpu_id, online_vcpus = atomic_read(&kvm->online_vcpus);
+ int vcpu_idx, online_vcpus = atomic_read(&kvm->online_vcpus);
struct kvm_s390_gisa_interrupt *gi = &kvm->arch.gisa_int;
struct kvm_vcpu *vcpu;
- for_each_set_bit(vcpu_id, kvm->arch.idle_mask, online_vcpus) {
- vcpu = kvm_get_vcpu(kvm, vcpu_id);
+ for_each_set_bit(vcpu_idx, kvm->arch.idle_mask, online_vcpus) {
+ vcpu = kvm_get_vcpu(kvm, vcpu_idx);
if (psw_ioint_disabled(vcpu))
continue;
deliverable_mask &= (u8)(vcpu->arch.sie_block->gcr[6] >> 24);
if (deliverable_mask) {
/* lately kicked but not yet running */
- if (test_and_set_bit(vcpu_id, gi->kicked_mask))
+ if (test_and_set_bit(vcpu_idx, gi->kicked_mask))
return;
kvm_s390_vcpu_wakeup(vcpu);
return;
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index 4527ac7b5961..8580543c5bc3 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -4044,7 +4044,7 @@ static int vcpu_pre_run(struct kvm_vcpu *vcpu)
kvm_s390_patch_guest_per_regs(vcpu);
}
- clear_bit(vcpu->vcpu_id, vcpu->kvm->arch.gisa_int.kicked_mask);
+ clear_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.gisa_int.kicked_mask);
vcpu->arch.sie_block->icptcode = 0;
cpuflags = atomic_read(&vcpu->arch.sie_block->cpuflags);
diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h
index 9fad25109b0d..ecd741ee3276 100644
--- a/arch/s390/kvm/kvm-s390.h
+++ b/arch/s390/kvm/kvm-s390.h
@@ -79,7 +79,7 @@ static inline int is_vcpu_stopped(struct kvm_vcpu *vcpu)
static inline int is_vcpu_idle(struct kvm_vcpu *vcpu)
{
- return test_bit(vcpu->vcpu_id, vcpu->kvm->arch.idle_mask);
+ return test_bit(kvm_vcpu_get_idx(vcpu), vcpu->kvm->arch.idle_mask);
}
static inline int kvm_is_ucontrol(struct kvm *kvm)
base-commit: 77dd11439b86e3f7990e4c0c9e0b67dca82750ba
--
2.25.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
end of thread, other threads:[~2022-01-31 12:25 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-20 15:06 [PATCH 0/1] KVM: s390: backport for stable of "KVM: s390: index Christian Borntraeger
2021-09-20 15:06 ` [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx Christian Borntraeger
2021-09-21 2:43 ` Halil Pasic
2021-09-20 15:24 ` [PATCH 0/1] KVM: s390: backport for stable of "KVM: s390: index Greg KH
2021-09-20 15:30 ` Christian Borntraeger
2021-09-20 18:05 ` Paolo Bonzini
2021-09-21 8:46 ` Christian Borntraeger
2021-09-21 13:04 ` Paolo Bonzini
-- strict thread matches above, loose matches on Subject: below --
2021-08-27 12:54 [PATCH 1/1] KVM: s390: index kvm->arch.idle_mask by vcpu_idx Halil Pasic
2021-08-27 13:42 ` Christian Borntraeger
2021-08-27 14:06 ` Claudio Imbrenda
2021-08-27 16:36 ` Christian Borntraeger
2021-08-27 21:23 ` Halil Pasic
2021-08-29 6:25 ` Christian Borntraeger
2021-08-27 21:19 ` Halil Pasic
2021-08-28 11:04 ` Michael Mueller
2022-01-31 10:13 ` Petr Tesařík
2022-01-31 11:53 ` Halil Pasic
2022-01-31 12:09 ` Petr Tesařík
2022-01-31 12:24 ` Halil Pasic
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.