[PATCH] KVM: s390x: add debug statement for diag 318 CPNC data

[PATCH] KVM: s390x: add debug statement for diag 318 CPNC data

[Collin Walling]

The diag 318 data contains values that denote information regarding the
guest's environment. Currently, it is unecessarily difficult to observe
this value (either manually-inserted debug statements, gdb stepping, mem
dumping etc). It's useful to observe this information to obtain an
at-a-glance view of the guest's environment, so lets add a simple VCPU
event that prints the CPNC to the s390dbf logs.

Signed-off-by: Collin Walling <walling@linux.ibm.com>
---
 arch/s390/kvm/kvm-s390.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index 6a6dd5e1daf6..da3ff24eabd0 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -4254,6 +4254,7 @@ static void sync_regs_fmt2(struct kvm_vcpu *vcpu)
 	if (kvm_run->kvm_dirty_regs & KVM_SYNC_DIAG318) {
 		vcpu->arch.diag318_info.val = kvm_run->s.regs.diag318;
 		vcpu->arch.sie_block->cpnc = vcpu->arch.diag318_info.cpnc;
+		VCPU_EVENT(vcpu, 2, "setting cpnc to %d", vcpu->arch.diag318_info.cpnc);
 	}
 	/*
 	 * If userspace sets the riccb (e.g. after migration) to a valid state,
-- 
2.31.1

Re: [PATCH] KVM: s390x: add debug statement for diag 318 CPNC data

[Christian Borntraeger]

Am 27.10.21 um 04:54 schrieb Collin Walling:
> The diag 318 data contains values that denote information regarding the
> guest's environment. Currently, it is unecessarily difficult to observe
> this value (either manually-inserted debug statements, gdb stepping, mem
> dumping etc). It's useful to observe this information to obtain an
> at-a-glance view of the guest's environment, so lets add a simple VCPU
> event that prints the CPNC to the s390dbf logs.
> 
> Signed-off-by: Collin Walling <walling@linux.ibm.com>

Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>

And it even finds a bug in QEMU. We clear the CPNC on local CPU resets.
Can you have a look? I think we just have to move the cpnc in the env
field from the normal/initial reset range to the full reset range.
> ---
>   arch/s390/kvm/kvm-s390.c | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
> index 6a6dd5e1daf6..da3ff24eabd0 100644
> --- a/arch/s390/kvm/kvm-s390.c
> +++ b/arch/s390/kvm/kvm-s390.c
> @@ -4254,6 +4254,7 @@ static void sync_regs_fmt2(struct kvm_vcpu *vcpu)
>   	if (kvm_run->kvm_dirty_regs & KVM_SYNC_DIAG318) {
>   		vcpu->arch.diag318_info.val = kvm_run->s.regs.diag318;
>   		vcpu->arch.sie_block->cpnc = vcpu->arch.diag318_info.cpnc;
> +		VCPU_EVENT(vcpu, 2, "setting cpnc to %d", vcpu->arch.diag318_info.cpnc);
>   	}
>   	/*
>   	 * If userspace sets the riccb (e.g. after migration) to a valid state,
> 

Re: [PATCH] KVM: s390x: add debug statement for diag 318 CPNC data

[Christian Borntraeger]

Am 27.10.21 um 04:54 schrieb Collin Walling:
> The diag 318 data contains values that denote information regarding the
> guest's environment. Currently, it is unecessarily difficult to observe
> this value (either manually-inserted debug statements, gdb stepping, mem
> dumping etc). It's useful to observe this information to obtain an
> at-a-glance view of the guest's environment, so lets add a simple VCPU
> event that prints the CPNC to the s390dbf logs.
> 
> Signed-off-by: Collin Walling <walling@linux.ibm.com>

applied and queued
> ---
>   arch/s390/kvm/kvm-s390.c | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
> index 6a6dd5e1daf6..da3ff24eabd0 100644
> --- a/arch/s390/kvm/kvm-s390.c
> +++ b/arch/s390/kvm/kvm-s390.c
> @@ -4254,6 +4254,7 @@ static void sync_regs_fmt2(struct kvm_vcpu *vcpu)
>   	if (kvm_run->kvm_dirty_regs & KVM_SYNC_DIAG318) {
>   		vcpu->arch.diag318_info.val = kvm_run->s.regs.diag318;
>   		vcpu->arch.sie_block->cpnc = vcpu->arch.diag318_info.cpnc;
> +		VCPU_EVENT(vcpu, 2, "setting cpnc to %d", vcpu->arch.diag318_info.cpnc);

After comparing this with the other events I think level==3 is better. Changed when applying.
>   	}
>   	/*
>   	 * If userspace sets the riccb (e.g. after migration) to a valid state,
> 

Re: [PATCH] KVM: s390x: add debug statement for diag 318 CPNC data

[Collin Walling]

On 10/27/21 01:37, Christian Borntraeger wrote:
> Am 27.10.21 um 04:54 schrieb Collin Walling:
>> The diag 318 data contains values that denote information regarding the
>> guest's environment. Currently, it is unecessarily difficult to observe
>> this value (either manually-inserted debug statements, gdb stepping, mem
>> dumping etc). It's useful to observe this information to obtain an
>> at-a-glance view of the guest's environment, so lets add a simple VCPU
>> event that prints the CPNC to the s390dbf logs.
>>
>> Signed-off-by: Collin Walling <walling@linux.ibm.com>
> 
> Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
> 
> And it even finds a bug in QEMU. We clear the CPNC on local CPU resets.
> Can you have a look? I think we just have to move the cpnc in the env
> field from the normal/initial reset range to the full reset range.

I'll take a look at this right away.

>> ---
>>   arch/s390/kvm/kvm-s390.c | 1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
>> index 6a6dd5e1daf6..da3ff24eabd0 100644
>> --- a/arch/s390/kvm/kvm-s390.c
>> +++ b/arch/s390/kvm/kvm-s390.c
>> @@ -4254,6 +4254,7 @@ static void sync_regs_fmt2(struct kvm_vcpu *vcpu)
>>       if (kvm_run->kvm_dirty_regs & KVM_SYNC_DIAG318) {
>>           vcpu->arch.diag318_info.val = kvm_run->s.regs.diag318;
>>           vcpu->arch.sie_block->cpnc = vcpu->arch.diag318_info.cpnc;
>> +        VCPU_EVENT(vcpu, 2, "setting cpnc to %d",
>> vcpu->arch.diag318_info.cpnc);
>>       }
>>       /*
>>        * If userspace sets the riccb (e.g. after migration) to a valid
>> state,
>>


-- 
Regards,
Collin

Stay safe and stay healthy

Re: [PATCH] KVM: s390x: add debug statement for diag 318 CPNC data

[Collin Walling]

On 10/27/21 01:41, Christian Borntraeger wrote:
> Am 27.10.21 um 04:54 schrieb Collin Walling:
>> The diag 318 data contains values that denote information regarding the
>> guest's environment. Currently, it is unecessarily difficult to observe
>> this value (either manually-inserted debug statements, gdb stepping, mem
>> dumping etc). It's useful to observe this information to obtain an
>> at-a-glance view of the guest's environment, so lets add a simple VCPU
>> event that prints the CPNC to the s390dbf logs.
>>
>> Signed-off-by: Collin Walling <walling@linux.ibm.com>
> 
> applied and queued
>> ---
>>   arch/s390/kvm/kvm-s390.c | 1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
>> index 6a6dd5e1daf6..da3ff24eabd0 100644
>> --- a/arch/s390/kvm/kvm-s390.c
>> +++ b/arch/s390/kvm/kvm-s390.c
>> @@ -4254,6 +4254,7 @@ static void sync_regs_fmt2(struct kvm_vcpu *vcpu)
>>       if (kvm_run->kvm_dirty_regs & KVM_SYNC_DIAG318) {
>>           vcpu->arch.diag318_info.val = kvm_run->s.regs.diag318;
>>           vcpu->arch.sie_block->cpnc = vcpu->arch.diag318_info.cpnc;
>> +        VCPU_EVENT(vcpu, 2, "setting cpnc to %d",
>> vcpu->arch.diag318_info.cpnc);
> 
> After comparing this with the other events I think level==3 is better.
> Changed when applying.
>>       }
>>       /*
>>        * If userspace sets the riccb (e.g. after migration) to a valid
>> state,
>>

Thanks!


-- 
Regards,
Collin

Stay safe and stay healthy

Re: [PATCH] KVM: s390x: add debug statement for diag 318 CPNC data

[Janosch Frank]

On 10/27/21 04:54, Collin Walling wrote:
> The diag 318 data contains values that denote information regarding the
> guest's environment. Currently, it is unecessarily difficult to observe
> this value (either manually-inserted debug statements, gdb stepping, mem
> dumping etc). It's useful to observe this information to obtain an
> at-a-glance view of the guest's environment, so lets add a simple VCPU
> event that prints the CPNC to the s390dbf logs.
> 
> Signed-off-by: Collin Walling <walling@linux.ibm.com>
> ---
>   arch/s390/kvm/kvm-s390.c | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
> index 6a6dd5e1daf6..da3ff24eabd0 100644
> --- a/arch/s390/kvm/kvm-s390.c
> +++ b/arch/s390/kvm/kvm-s390.c
> @@ -4254,6 +4254,7 @@ static void sync_regs_fmt2(struct kvm_vcpu *vcpu)
>   	if (kvm_run->kvm_dirty_regs & KVM_SYNC_DIAG318) {
>   		vcpu->arch.diag318_info.val = kvm_run->s.regs.diag318;
>   		vcpu->arch.sie_block->cpnc = vcpu->arch.diag318_info.cpnc;
> +		VCPU_EVENT(vcpu, 2, "setting cpnc to %d", vcpu->arch.diag318_info.cpnc);
>   	}
>   	/*
>   	 * If userspace sets the riccb (e.g. after migration) to a valid state,
> 

Won't that turn up for every vcpu and spam the log?

Re: [PATCH] KVM: s390x: add debug statement for diag 318 CPNC data

[Christian Borntraeger]

Am 08.11.21 um 12:12 schrieb Janosch Frank:
> On 10/27/21 04:54, Collin Walling wrote:
>> The diag 318 data contains values that denote information regarding the
>> guest's environment. Currently, it is unecessarily difficult to observe
>> this value (either manually-inserted debug statements, gdb stepping, mem
>> dumping etc). It's useful to observe this information to obtain an
>> at-a-glance view of the guest's environment, so lets add a simple VCPU
>> event that prints the CPNC to the s390dbf logs.
>>
>> Signed-off-by: Collin Walling <walling@linux.ibm.com>
>> ---
>>   arch/s390/kvm/kvm-s390.c | 1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
>> index 6a6dd5e1daf6..da3ff24eabd0 100644
>> --- a/arch/s390/kvm/kvm-s390.c
>> +++ b/arch/s390/kvm/kvm-s390.c
>> @@ -4254,6 +4254,7 @@ static void sync_regs_fmt2(struct kvm_vcpu *vcpu)
>>       if (kvm_run->kvm_dirty_regs & KVM_SYNC_DIAG318) {
>>           vcpu->arch.diag318_info.val = kvm_run->s.regs.diag318;
>>           vcpu->arch.sie_block->cpnc = vcpu->arch.diag318_info.cpnc;
>> +        VCPU_EVENT(vcpu, 2, "setting cpnc to %d", vcpu->arch.diag318_info.cpnc);
>>       }
>>       /*
>>        * If userspace sets the riccb (e.g. after migration) to a valid state,
>>
> 
> Won't that turn up for every vcpu and spam the log?

only if the userspace always sets the dirty bit (which it should not).

Re: [PATCH] KVM: s390x: add debug statement for diag 318 CPNC data

[Janosch Frank]

On 11/8/21 13:04, Christian Borntraeger wrote:
> 
> 
> Am 08.11.21 um 12:12 schrieb Janosch Frank:
>> On 10/27/21 04:54, Collin Walling wrote:
>>> The diag 318 data contains values that denote information regarding the
>>> guest's environment. Currently, it is unecessarily difficult to observe
>>> this value (either manually-inserted debug statements, gdb stepping, mem
>>> dumping etc). It's useful to observe this information to obtain an
>>> at-a-glance view of the guest's environment, so lets add a simple VCPU
>>> event that prints the CPNC to the s390dbf logs.
>>>
>>> Signed-off-by: Collin Walling <walling@linux.ibm.com>
>>> ---
>>>    arch/s390/kvm/kvm-s390.c | 1 +
>>>    1 file changed, 1 insertion(+)
>>>
>>> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
>>> index 6a6dd5e1daf6..da3ff24eabd0 100644
>>> --- a/arch/s390/kvm/kvm-s390.c
>>> +++ b/arch/s390/kvm/kvm-s390.c
>>> @@ -4254,6 +4254,7 @@ static void sync_regs_fmt2(struct kvm_vcpu *vcpu)
>>>        if (kvm_run->kvm_dirty_regs & KVM_SYNC_DIAG318) {
>>>            vcpu->arch.diag318_info.val = kvm_run->s.regs.diag318;
>>>            vcpu->arch.sie_block->cpnc = vcpu->arch.diag318_info.cpnc;
>>> +        VCPU_EVENT(vcpu, 2, "setting cpnc to %d", vcpu->arch.diag318_info.cpnc);
>>>        }
>>>        /*
>>>         * If userspace sets the riccb (e.g. after migration) to a valid state,
>>>
>>
>> Won't that turn up for every vcpu and spam the log?
> 
> only if the userspace always sets the dirty bit (which it should not).
> 

But that's exactly what it does, no?
We do a loop over all vcpus and call kvm_s390_set_diag318() which sets 
the info in kvm_run and sets the diag318 bit in the kvm_dirty_regs.

@Collin: Could you check that please?

Re: [PATCH] KVM: s390x: add debug statement for diag 318 CPNC data

[Christian Borntraeger]

Am 08.11.21 um 13:48 schrieb Janosch Frank:
> On 11/8/21 13:04, Christian Borntraeger wrote:
>>
>>
>> Am 08.11.21 um 12:12 schrieb Janosch Frank:
>>> On 10/27/21 04:54, Collin Walling wrote:
>>>> The diag 318 data contains values that denote information regarding the
>>>> guest's environment. Currently, it is unecessarily difficult to observe
>>>> this value (either manually-inserted debug statements, gdb stepping, mem
>>>> dumping etc). It's useful to observe this information to obtain an
>>>> at-a-glance view of the guest's environment, so lets add a simple VCPU
>>>> event that prints the CPNC to the s390dbf logs.
>>>>
>>>> Signed-off-by: Collin Walling <walling@linux.ibm.com>
>>>> ---
>>>>    arch/s390/kvm/kvm-s390.c | 1 +
>>>>    1 file changed, 1 insertion(+)
>>>>
>>>> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
>>>> index 6a6dd5e1daf6..da3ff24eabd0 100644
>>>> --- a/arch/s390/kvm/kvm-s390.c
>>>> +++ b/arch/s390/kvm/kvm-s390.c
>>>> @@ -4254,6 +4254,7 @@ static void sync_regs_fmt2(struct kvm_vcpu *vcpu)
>>>>        if (kvm_run->kvm_dirty_regs & KVM_SYNC_DIAG318) {
>>>>            vcpu->arch.diag318_info.val = kvm_run->s.regs.diag318;
>>>>            vcpu->arch.sie_block->cpnc = vcpu->arch.diag318_info.cpnc;
>>>> +        VCPU_EVENT(vcpu, 2, "setting cpnc to %d", vcpu->arch.diag318_info.cpnc);
>>>>        }
>>>>        /*
>>>>         * If userspace sets the riccb (e.g. after migration) to a valid state,
>>>>
>>>
>>> Won't that turn up for every vcpu and spam the log?
>>
>> only if the userspace always sets the dirty bit (which it should not).
>>
> 
> But that's exactly what it does, no?
> We do a loop over all vcpus and call kvm_s390_set_diag318() which sets the info in kvm_run and sets the diag318 bit in the kvm_dirty_regs.

Yes, ONCE per CPU. And this is exactly what I want to see. (and it did show a bug in qemu that we only set it for one cpu to the correct value).

> 
> @Collin: Could you check that please?

Re: [PATCH] KVM: s390x: add debug statement for diag 318 CPNC data

[Janosch Frank]

On 11/8/21 13:50, Christian Borntraeger wrote:
> 
> 
> Am 08.11.21 um 13:48 schrieb Janosch Frank:
>> On 11/8/21 13:04, Christian Borntraeger wrote:
>>>
>>>
>>> Am 08.11.21 um 12:12 schrieb Janosch Frank:
>>>> On 10/27/21 04:54, Collin Walling wrote:
>>>>> The diag 318 data contains values that denote information regarding the
>>>>> guest's environment. Currently, it is unecessarily difficult to observe
>>>>> this value (either manually-inserted debug statements, gdb stepping, mem
>>>>> dumping etc). It's useful to observe this information to obtain an
>>>>> at-a-glance view of the guest's environment, so lets add a simple VCPU
>>>>> event that prints the CPNC to the s390dbf logs.
>>>>>
>>>>> Signed-off-by: Collin Walling <walling@linux.ibm.com>
>>>>> ---
>>>>>     arch/s390/kvm/kvm-s390.c | 1 +
>>>>>     1 file changed, 1 insertion(+)
>>>>>
>>>>> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
>>>>> index 6a6dd5e1daf6..da3ff24eabd0 100644
>>>>> --- a/arch/s390/kvm/kvm-s390.c
>>>>> +++ b/arch/s390/kvm/kvm-s390.c
>>>>> @@ -4254,6 +4254,7 @@ static void sync_regs_fmt2(struct kvm_vcpu *vcpu)
>>>>>         if (kvm_run->kvm_dirty_regs & KVM_SYNC_DIAG318) {
>>>>>             vcpu->arch.diag318_info.val = kvm_run->s.regs.diag318;
>>>>>             vcpu->arch.sie_block->cpnc = vcpu->arch.diag318_info.cpnc;
>>>>> +        VCPU_EVENT(vcpu, 2, "setting cpnc to %d", vcpu->arch.diag318_info.cpnc);
>>>>>         }
>>>>>         /*
>>>>>          * If userspace sets the riccb (e.g. after migration) to a valid state,
>>>>>
>>>>
>>>> Won't that turn up for every vcpu and spam the log?
>>>
>>> only if the userspace always sets the dirty bit (which it should not).
>>>
>>
>> But that's exactly what it does, no?
>> We do a loop over all vcpus and call kvm_s390_set_diag318() which sets the info in kvm_run and sets the diag318 bit in the kvm_dirty_regs.
> 
> Yes, ONCE per CPU. And this is exactly what I want to see. (and it did show a bug in qemu that we only set it for one cpu to the correct value).

Ok.
I didn't really want to have n entries in the log hence I asked.

318 is a bit weird as it's a per VM value we need to put into all sie 
blocks.

> 
>>
>> @Collin: Could you check that please?