[Qemu-devel] [PATCH v3 0/9] s390x: TOD refactoring + TCG CPU hotplug support

[Qemu-devel] [PATCH v3 0/9] s390x: TOD refactoring + TCG CPU hotplug support

[David Hildenbrand]

The TOD in TCG is not handled correctly:
- each CPU has its own TOD based on CPU creation time vs. a system TOD
- TOD is not migrated
- TOD timer is not restarted during migration
- CKC interrupts/TOD timer is not cleared when resetting the CKC

This (and a cpu creation problem for single threaded TCG) currently made
CPU hotplug under TCG not work. Now it's working

The third patch also refactors TOD handling for KVM (moved into a new
TOD device). The second patch makes sure that we can use a const pointer
for the new TOD clock struct.

v2 -> v3:
- added "s390x/tcg: avoid overflows in time2tod/tod2time"
- added "s390x/kvm: pass values instead of pointers to kvm_s390_set_clock_*()"
- "s390x/tod: factor out TOD into separate device"
-- Use "const" for parameters in function definitions
-- Use "s390-tod-qemu" and "s390-tod-kvm" as type names
-- Drop hotpluggable=false
- "s390x/tcg: properly implement the TOD"
-- s390_get_tod() -> s390_get_todstate()
-- use error_abort
- "s390x/tcg: SET CLOCK COMPARATOR can clear CKC interrupts"
-- Add a comment for the -1ULL check
- "s390x/tcg: implement SET CLOCK"
-- use error_abort
- "s390x/tcg: fix CPU hotplug with single-threaded TCG"
-- add a comment describing the run_on_cpu() difference

v1 -> v2:
- "s390x/tcg: properly implement the TOD"
-- introduce "tcg_s390x.h" similar to "kvm_s390x.h"
-- dropped one unnecessary include introduction
- "s390x/tcg: rearm the CKC timer during migration"
-- introduce "tcg-stub.c" similar to "kvm-stub.c"

David Hildenbrand (9):
  s390x/tcg: avoid overflows in time2tod/tod2time
  s390x/kvm: pass values instead of pointers to kvm_s390_set_clock_*()
  s390x/tod: factor out TOD into separate device
  s390x/tcg: drop tod_basetime
  s390x/tcg: properly implement the TOD
  s390x/tcg: SET CLOCK COMPARATOR can clear CKC interrupts
  s390x/tcg: implement SET CLOCK
  s390x/tcg: rearm the CKC timer during migration
  s390x/tcg: fix CPU hotplug with single-threaded TCG

 hw/s390x/Makefile.objs     |   3 +
 hw/s390x/s390-virtio-ccw.c |  57 +---------------
 hw/s390x/tod-kvm.c         |  64 ++++++++++++++++++
 hw/s390x/tod-qemu.c        |  87 +++++++++++++++++++++++++
 hw/s390x/tod.c             | 130 +++++++++++++++++++++++++++++++++++++
 include/hw/s390x/tod.h     |  65 +++++++++++++++++++
 target/s390x/Makefile.objs |   1 +
 target/s390x/cpu.c         |  57 ++++------------
 target/s390x/cpu.h         |   4 --
 target/s390x/helper.h      |   1 +
 target/s390x/insn-data.def |   3 +-
 target/s390x/internal.h    |  15 -----
 target/s390x/kvm-stub.c    |   4 +-
 target/s390x/kvm.c         |  12 ++--
 target/s390x/kvm_s390x.h   |   6 +-
 target/s390x/machine.c     |   6 ++
 target/s390x/misc_helper.c |  55 ++++++++++++++--
 target/s390x/tcg-stub.c    |  20 ++++++
 target/s390x/tcg_s390x.h   |  18 +++++
 target/s390x/translate.c   |   9 +++
 20 files changed, 480 insertions(+), 137 deletions(-)
 create mode 100644 hw/s390x/tod-kvm.c
 create mode 100644 hw/s390x/tod-qemu.c
 create mode 100644 hw/s390x/tod.c
 create mode 100644 include/hw/s390x/tod.h
 create mode 100644 target/s390x/tcg-stub.c
 create mode 100644 target/s390x/tcg_s390x.h

-- 
2.17.1

[Qemu-devel] [PATCH v3 1/9] s390x/tcg: avoid overflows in time2tod/tod2time

[David Hildenbrand]

Big values for the TOD/ns clock can result in some overflows that can be
avoided. Not all overflows can be handled however, as the conversion either
multiplies by 4.096 or divided by 4.096.

Apply the trick used in the Linux kernel in arch/s390/include/asm/timex.h
for tod_to_ns() and use the same trick also for the conversion in the
other direction.

Signed-off-by: David Hildenbrand <david@redhat.com>
---
 target/s390x/internal.h | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/target/s390x/internal.h b/target/s390x/internal.h
index e392a02d12..6cf63340bf 100644
--- a/target/s390x/internal.h
+++ b/target/s390x/internal.h
@@ -243,13 +243,14 @@ enum cc_op {
 /* Converts ns to s390's clock format */
 static inline uint64_t time2tod(uint64_t ns)
 {
-    return (ns << 9) / 125;
+    return (ns << 9) / 125 + (((ns & 0xff10000000000000ull) / 125) << 9);
+
 }
 
 /* Converts s390's clock format to ns */
 static inline uint64_t tod2time(uint64_t t)
 {
-    return (t * 125) >> 9;
+    return ((t >> 9) * 125) + (((t & 0x1ff) * 125) >> 9);
 }
 
 static inline hwaddr decode_basedisp_s(CPUS390XState *env, uint32_t ipb,
-- 
2.17.1

Re: [Qemu-devel] [PATCH v3 1/9] s390x/tcg: avoid overflows in time2tod/tod2time

[Thomas Huth]

On 25.06.2018 13:53, David Hildenbrand wrote:
> Big values for the TOD/ns clock can result in some overflows that can be
> avoided. Not all overflows can be handled however, as the conversion either
> multiplies by 4.096 or divided by 4.096.
> 
> Apply the trick used in the Linux kernel in arch/s390/include/asm/timex.h
> for tod_to_ns() and use the same trick also for the conversion in the
> other direction.
> 
> Signed-off-by: David Hildenbrand <david@redhat.com>
> ---
>  target/s390x/internal.h | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/target/s390x/internal.h b/target/s390x/internal.h
> index e392a02d12..6cf63340bf 100644
> --- a/target/s390x/internal.h
> +++ b/target/s390x/internal.h
> @@ -243,13 +243,14 @@ enum cc_op {
>  /* Converts ns to s390's clock format */
>  static inline uint64_t time2tod(uint64_t ns)
>  {
> -    return (ns << 9) / 125;
> +    return (ns << 9) / 125 + (((ns & 0xff10000000000000ull) / 125) << 9);
> +
>  }
>  
>  /* Converts s390's clock format to ns */
>  static inline uint64_t tod2time(uint64_t t)
>  {
> -    return (t * 125) >> 9;
> +    return ((t >> 9) * 125) + (((t & 0x1ff) * 125) >> 9);
>  }
>  
>  static inline hwaddr decode_basedisp_s(CPUS390XState *env, uint32_t ipb,
> 

Looks reasonable.

Reviewed-by: Thomas Huth <thuth@redhat.com>

[Qemu-devel] [PATCH v3 2/9] s390x/kvm: pass values instead of pointers to kvm_s390_set_clock_*()

[David Hildenbrand]

We are going to factor out the TOD into a separate device and use const
pointers for device class functions where possible. We are passing right
now ordinary pointers that should never be touched when setting the TOD.
Let's just pass the values directly.

Signed-off-by: David Hildenbrand <david@redhat.com>
---
 target/s390x/cpu.c       |  4 ++--
 target/s390x/kvm-stub.c  |  4 ++--
 target/s390x/kvm.c       | 12 ++++++------
 target/s390x/kvm_s390x.h |  4 ++--
 4 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
index c268065887..68512e3e54 100644
--- a/target/s390x/cpu.c
+++ b/target/s390x/cpu.c
@@ -413,9 +413,9 @@ int s390_set_clock(uint8_t *tod_high, uint64_t *tod_low)
     int r = 0;
 
     if (kvm_enabled()) {
-        r = kvm_s390_set_clock_ext(tod_high, tod_low);
+        r = kvm_s390_set_clock_ext(*tod_high, *tod_low);
         if (r == -ENXIO) {
-            return kvm_s390_set_clock(tod_high, tod_low);
+            return kvm_s390_set_clock(*tod_high, *tod_low);
         }
     }
     /* Fixme TCG */
diff --git a/target/s390x/kvm-stub.c b/target/s390x/kvm-stub.c
index 29b10542cc..bf7795e47a 100644
--- a/target/s390x/kvm-stub.c
+++ b/target/s390x/kvm-stub.c
@@ -60,12 +60,12 @@ int kvm_s390_get_clock_ext(uint8_t *tod_high, uint64_t *tod_low)
     return -ENOSYS;
 }
 
-int kvm_s390_set_clock(uint8_t *tod_high, uint64_t *tod_low)
+int kvm_s390_set_clock(uint8_t tod_high, uint64_t tod_low)
 {
     return -ENOSYS;
 }
 
-int kvm_s390_set_clock_ext(uint8_t *tod_high, uint64_t *tod_low)
+int kvm_s390_set_clock_ext(uint8_t tod_high, uint64_t tod_low)
 {
     return -ENOSYS;
 }
diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c
index ac370da281..8bcd832123 100644
--- a/target/s390x/kvm.c
+++ b/target/s390x/kvm.c
@@ -666,13 +666,13 @@ int kvm_s390_get_clock_ext(uint8_t *tod_high, uint64_t *tod_low)
     return r;
 }
 
-int kvm_s390_set_clock(uint8_t *tod_high, uint64_t *tod_low)
+int kvm_s390_set_clock(uint8_t tod_high, uint64_t tod_low)
 {
     int r;
     struct kvm_device_attr attr = {
         .group = KVM_S390_VM_TOD,
         .attr = KVM_S390_VM_TOD_LOW,
-        .addr = (uint64_t)tod_low,
+        .addr = (uint64_t)&tod_low,
     };
 
     r = kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, &attr);
@@ -681,15 +681,15 @@ int kvm_s390_set_clock(uint8_t *tod_high, uint64_t *tod_low)
     }
 
     attr.attr = KVM_S390_VM_TOD_HIGH;
-    attr.addr = (uint64_t)tod_high;
+    attr.addr = (uint64_t)&tod_high;
     return kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, &attr);
 }
 
-int kvm_s390_set_clock_ext(uint8_t *tod_high, uint64_t *tod_low)
+int kvm_s390_set_clock_ext(uint8_t tod_high, uint64_t tod_low)
 {
     struct kvm_s390_vm_tod_clock gtod = {
-        .epoch_idx = *tod_high,
-        .tod  = *tod_low,
+        .epoch_idx = tod_high,
+        .tod  = tod_low,
     };
     struct kvm_device_attr attr = {
         .group = KVM_S390_VM_TOD,
diff --git a/target/s390x/kvm_s390x.h b/target/s390x/kvm_s390x.h
index c383bf4ee9..36eb34bf99 100644
--- a/target/s390x/kvm_s390x.h
+++ b/target/s390x/kvm_s390x.h
@@ -25,8 +25,8 @@ int kvm_s390_get_ri(void);
 int kvm_s390_get_gs(void);
 int kvm_s390_get_clock(uint8_t *tod_high, uint64_t *tod_clock);
 int kvm_s390_get_clock_ext(uint8_t *tod_high, uint64_t *tod_clock);
-int kvm_s390_set_clock(uint8_t *tod_high, uint64_t *tod_clock);
-int kvm_s390_set_clock_ext(uint8_t *tod_high, uint64_t *tod_clock);
+int kvm_s390_set_clock(uint8_t tod_high, uint64_t tod_clock);
+int kvm_s390_set_clock_ext(uint8_t tod_high, uint64_t tod_clock);
 void kvm_s390_enable_css_support(S390CPU *cpu);
 int kvm_s390_assign_subch_ioeventfd(EventNotifier *notifier, uint32_t sch,
                                     int vq, bool assign);
-- 
2.17.1

Re: [Qemu-devel] [PATCH v3 2/9] s390x/kvm: pass values instead of pointers to kvm_s390_set_clock_*()

[Cornelia Huck]

On Mon, 25 Jun 2018 13:53:45 +0200
David Hildenbrand <david@redhat.com> wrote:

> We are going to factor out the TOD into a separate device and use const
> pointers for device class functions where possible. We are passing right
> now ordinary pointers that should never be touched when setting the TOD.
> Let's just pass the values directly.
> 
> Signed-off-by: David Hildenbrand <david@redhat.com>
> ---
>  target/s390x/cpu.c       |  4 ++--
>  target/s390x/kvm-stub.c  |  4 ++--
>  target/s390x/kvm.c       | 12 ++++++------
>  target/s390x/kvm_s390x.h |  4 ++--
>  4 files changed, 12 insertions(+), 12 deletions(-)
> 
> diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
> index c268065887..68512e3e54 100644
> --- a/target/s390x/cpu.c
> +++ b/target/s390x/cpu.c
> @@ -413,9 +413,9 @@ int s390_set_clock(uint8_t *tod_high, uint64_t *tod_low)

Any reason why you keep the pointers here?

>      int r = 0;
>  
>      if (kvm_enabled()) {
> -        r = kvm_s390_set_clock_ext(tod_high, tod_low);
> +        r = kvm_s390_set_clock_ext(*tod_high, *tod_low);
>          if (r == -ENXIO) {
> -            return kvm_s390_set_clock(tod_high, tod_low);
> +            return kvm_s390_set_clock(*tod_high, *tod_low);

Especially as it would be more clean to check for !NULL before
dereferencing...

>          }
>      }
>      /* Fixme TCG */

Re: [Qemu-devel] [PATCH v3 2/9] s390x/kvm: pass values instead of pointers to kvm_s390_set_clock_*()

[David Hildenbrand]

On 25.06.2018 17:50, Cornelia Huck wrote:
> On Mon, 25 Jun 2018 13:53:45 +0200
> David Hildenbrand <david@redhat.com> wrote:
> 
>> We are going to factor out the TOD into a separate device and use const
>> pointers for device class functions where possible. We are passing right
>> now ordinary pointers that should never be touched when setting the TOD.
>> Let's just pass the values directly.
>>
>> Signed-off-by: David Hildenbrand <david@redhat.com>
>> ---
>>  target/s390x/cpu.c       |  4 ++--
>>  target/s390x/kvm-stub.c  |  4 ++--
>>  target/s390x/kvm.c       | 12 ++++++------
>>  target/s390x/kvm_s390x.h |  4 ++--
>>  4 files changed, 12 insertions(+), 12 deletions(-)
>>
>> diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
>> index c268065887..68512e3e54 100644
>> --- a/target/s390x/cpu.c
>> +++ b/target/s390x/cpu.c
>> @@ -413,9 +413,9 @@ int s390_set_clock(uint8_t *tod_high, uint64_t *tod_low)
> 
> Any reason why you keep the pointers here?
> 
>>      int r = 0;
>>  
>>      if (kvm_enabled()) {
>> -        r = kvm_s390_set_clock_ext(tod_high, tod_low);
>> +        r = kvm_s390_set_clock_ext(*tod_high, *tod_low);
>>          if (r == -ENXIO) {
>> -            return kvm_s390_set_clock(tod_high, tod_low);
>> +            return kvm_s390_set_clock(*tod_high, *tod_low);
> 
> Especially as it would be more clean to check for !NULL before
> dereferencing...

See the next patch :)

(I assume that refactoring code in order to rip it out does not make sense)

> 
>>          }
>>      }
>>      /* Fixme TCG */


-- 

Thanks,

David / dhildenb

Re: [Qemu-devel] [PATCH v3 2/9] s390x/kvm: pass values instead of pointers to kvm_s390_set_clock_*()

[Cornelia Huck]

On Mon, 25 Jun 2018 17:54:42 +0200
David Hildenbrand <david@redhat.com> wrote:

> On 25.06.2018 17:50, Cornelia Huck wrote:
> > On Mon, 25 Jun 2018 13:53:45 +0200
> > David Hildenbrand <david@redhat.com> wrote:
> >   
> >> We are going to factor out the TOD into a separate device and use const
> >> pointers for device class functions where possible. We are passing right
> >> now ordinary pointers that should never be touched when setting the TOD.
> >> Let's just pass the values directly.
> >>
> >> Signed-off-by: David Hildenbrand <david@redhat.com>
> >> ---
> >>  target/s390x/cpu.c       |  4 ++--
> >>  target/s390x/kvm-stub.c  |  4 ++--
> >>  target/s390x/kvm.c       | 12 ++++++------
> >>  target/s390x/kvm_s390x.h |  4 ++--
> >>  4 files changed, 12 insertions(+), 12 deletions(-)
> >>
> >> diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
> >> index c268065887..68512e3e54 100644
> >> --- a/target/s390x/cpu.c
> >> +++ b/target/s390x/cpu.c
> >> @@ -413,9 +413,9 @@ int s390_set_clock(uint8_t *tod_high, uint64_t *tod_low)  
> > 
> > Any reason why you keep the pointers here?
> >   
> >>      int r = 0;
> >>  
> >>      if (kvm_enabled()) {
> >> -        r = kvm_s390_set_clock_ext(tod_high, tod_low);
> >> +        r = kvm_s390_set_clock_ext(*tod_high, *tod_low);
> >>          if (r == -ENXIO) {
> >> -            return kvm_s390_set_clock(tod_high, tod_low);
> >> +            return kvm_s390_set_clock(*tod_high, *tod_low);  
> > 
> > Especially as it would be more clean to check for !NULL before
> > dereferencing...  
> 
> See the next patch :)
> 
> (I assume that refactoring code in order to rip it out does not make sense)

Add a comment in the commit message?

"Note that s390_set_clock() will be removed in a follow-on patch and
therefore its calling convention is not changed."

Re: [Qemu-devel] [qemu-s390x] [PATCH v3 2/9] s390x/kvm: pass values instead of pointers to kvm_s390_set_clock_*()

[David Hildenbrand]

On 25.06.2018 18:03, Cornelia Huck wrote:
> On Mon, 25 Jun 2018 17:54:42 +0200
> David Hildenbrand <david@redhat.com> wrote:
> 
>> On 25.06.2018 17:50, Cornelia Huck wrote:
>>> On Mon, 25 Jun 2018 13:53:45 +0200
>>> David Hildenbrand <david@redhat.com> wrote:
>>>   
>>>> We are going to factor out the TOD into a separate device and use const
>>>> pointers for device class functions where possible. We are passing right
>>>> now ordinary pointers that should never be touched when setting the TOD.
>>>> Let's just pass the values directly.
>>>>
>>>> Signed-off-by: David Hildenbrand <david@redhat.com>
>>>> ---
>>>>  target/s390x/cpu.c       |  4 ++--
>>>>  target/s390x/kvm-stub.c  |  4 ++--
>>>>  target/s390x/kvm.c       | 12 ++++++------
>>>>  target/s390x/kvm_s390x.h |  4 ++--
>>>>  4 files changed, 12 insertions(+), 12 deletions(-)
>>>>
>>>> diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
>>>> index c268065887..68512e3e54 100644
>>>> --- a/target/s390x/cpu.c
>>>> +++ b/target/s390x/cpu.c
>>>> @@ -413,9 +413,9 @@ int s390_set_clock(uint8_t *tod_high, uint64_t *tod_low)  
>>>
>>> Any reason why you keep the pointers here?
>>>   
>>>>      int r = 0;
>>>>  
>>>>      if (kvm_enabled()) {
>>>> -        r = kvm_s390_set_clock_ext(tod_high, tod_low);
>>>> +        r = kvm_s390_set_clock_ext(*tod_high, *tod_low);
>>>>          if (r == -ENXIO) {
>>>> -            return kvm_s390_set_clock(tod_high, tod_low);
>>>> +            return kvm_s390_set_clock(*tod_high, *tod_low);  
>>>
>>> Especially as it would be more clean to check for !NULL before
>>> dereferencing...  
>>
>> See the next patch :)
>>
>> (I assume that refactoring code in order to rip it out does not make sense)
> 
> Add a comment in the commit message?
> 
> "Note that s390_set_clock() will be removed in a follow-on patch and
> therefore its calling convention is not changed."
> 

Sure I can do that. Thanks!

-- 

Thanks,

David / dhildenb

Re: [Qemu-devel] [qemu-s390x] [PATCH v3 2/9] s390x/kvm: pass values instead of pointers to kvm_s390_set_clock_*()

[Cornelia Huck]

On Tue, 26 Jun 2018 11:54:32 +0200
David Hildenbrand <david@redhat.com> wrote:

> On 25.06.2018 18:03, Cornelia Huck wrote:
> > On Mon, 25 Jun 2018 17:54:42 +0200
> > David Hildenbrand <david@redhat.com> wrote:
> >   
> >> On 25.06.2018 17:50, Cornelia Huck wrote:  
> >>> On Mon, 25 Jun 2018 13:53:45 +0200
> >>> David Hildenbrand <david@redhat.com> wrote:
> >>>     
> >>>> We are going to factor out the TOD into a separate device and use const
> >>>> pointers for device class functions where possible. We are passing right
> >>>> now ordinary pointers that should never be touched when setting the TOD.
> >>>> Let's just pass the values directly.
> >>>>
> >>>> Signed-off-by: David Hildenbrand <david@redhat.com>
> >>>> ---
> >>>>  target/s390x/cpu.c       |  4 ++--
> >>>>  target/s390x/kvm-stub.c  |  4 ++--
> >>>>  target/s390x/kvm.c       | 12 ++++++------
> >>>>  target/s390x/kvm_s390x.h |  4 ++--
> >>>>  4 files changed, 12 insertions(+), 12 deletions(-)
> >>>>
> >>>> diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
> >>>> index c268065887..68512e3e54 100644
> >>>> --- a/target/s390x/cpu.c
> >>>> +++ b/target/s390x/cpu.c
> >>>> @@ -413,9 +413,9 @@ int s390_set_clock(uint8_t *tod_high, uint64_t *tod_low)    
> >>>
> >>> Any reason why you keep the pointers here?
> >>>     
> >>>>      int r = 0;
> >>>>  
> >>>>      if (kvm_enabled()) {
> >>>> -        r = kvm_s390_set_clock_ext(tod_high, tod_low);
> >>>> +        r = kvm_s390_set_clock_ext(*tod_high, *tod_low);
> >>>>          if (r == -ENXIO) {
> >>>> -            return kvm_s390_set_clock(tod_high, tod_low);
> >>>> +            return kvm_s390_set_clock(*tod_high, *tod_low);    
> >>>
> >>> Especially as it would be more clean to check for !NULL before
> >>> dereferencing...    
> >>
> >> See the next patch :)
> >>
> >> (I assume that refactoring code in order to rip it out does not make sense)  
> > 
> > Add a comment in the commit message?
> > 
> > "Note that s390_set_clock() will be removed in a follow-on patch and
> > therefore its calling convention is not changed."
> >   
> 
> Sure I can do that. Thanks!
> 

Well, no need to respin if nothing else comes up, I can add that myself.

[Qemu-devel] [PATCH v3 3/9] s390x/tod: factor out TOD into separate device

[David Hildenbrand]

Let's treat this like a separate device. TCG will have to store the
actual state/time later on.

Include cpu-qom.h in kvm_s390x.h (due to S390CPU) to compile tod-kvm.c.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
---
 hw/s390x/Makefile.objs     |   3 +
 hw/s390x/s390-virtio-ccw.c |  57 +-----------------
 hw/s390x/tod-kvm.c         |  64 ++++++++++++++++++++
 hw/s390x/tod-qemu.c        |  49 +++++++++++++++
 hw/s390x/tod.c             | 119 +++++++++++++++++++++++++++++++++++++
 include/hw/s390x/tod.h     |  46 ++++++++++++++
 target/s390x/cpu.c         |  32 ----------
 target/s390x/cpu.h         |   2 -
 target/s390x/kvm_s390x.h   |   2 +
 9 files changed, 286 insertions(+), 88 deletions(-)
 create mode 100644 hw/s390x/tod-kvm.c
 create mode 100644 hw/s390x/tod-qemu.c
 create mode 100644 hw/s390x/tod.c
 create mode 100644 include/hw/s390x/tod.h

diff --git a/hw/s390x/Makefile.objs b/hw/s390x/Makefile.objs
index dc704b57d6..93282f7c59 100644
--- a/hw/s390x/Makefile.objs
+++ b/hw/s390x/Makefile.objs
@@ -14,6 +14,9 @@ obj-$(CONFIG_PCI) += s390-pci-bus.o s390-pci-inst.o
 obj-$(call lnot,$(CONFIG_PCI)) += s390-pci-stub.o
 obj-y += s390-skeys.o
 obj-y += s390-stattrib.o
+obj-y += tod.o
+obj-$(CONFIG_KVM) += tod-kvm.o
+obj-$(CONFIG_TCG) += tod-qemu.o
 obj-$(CONFIG_KVM) += s390-skeys-kvm.o
 obj-$(CONFIG_KVM) += s390-stattrib-kvm.o
 obj-y += s390-ccw.o
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 7ae5fb38dd..dea8dfa47b 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -35,6 +35,7 @@
 #include "migration/register.h"
 #include "cpu_models.h"
 #include "hw/nmi.h"
+#include "hw/s390x/tod.h"
 
 S390CPU *s390_cpu_addr2state(uint16_t cpu_addr)
 {
@@ -187,58 +188,6 @@ static void s390_memory_init(ram_addr_t mem_size)
     s390_stattrib_init();
 }
 
-#define S390_TOD_CLOCK_VALUE_MISSING    0x00
-#define S390_TOD_CLOCK_VALUE_PRESENT    0x01
-
-static void gtod_save(QEMUFile *f, void *opaque)
-{
-    uint64_t tod_low;
-    uint8_t tod_high;
-    int r;
-
-    r = s390_get_clock(&tod_high, &tod_low);
-    if (r) {
-        warn_report("Unable to get guest clock for migration: %s",
-                    strerror(-r));
-        error_printf("Guest clock will not be migrated "
-                     "which could cause the guest to hang.");
-        qemu_put_byte(f, S390_TOD_CLOCK_VALUE_MISSING);
-        return;
-    }
-
-    qemu_put_byte(f, S390_TOD_CLOCK_VALUE_PRESENT);
-    qemu_put_byte(f, tod_high);
-    qemu_put_be64(f, tod_low);
-}
-
-static int gtod_load(QEMUFile *f, void *opaque, int version_id)
-{
-    uint64_t tod_low;
-    uint8_t tod_high;
-    int r;
-
-    if (qemu_get_byte(f) == S390_TOD_CLOCK_VALUE_MISSING) {
-        warn_report("Guest clock was not migrated. This could "
-                    "cause the guest to hang.");
-        return 0;
-    }
-
-    tod_high = qemu_get_byte(f);
-    tod_low = qemu_get_be64(f);
-
-    r = s390_set_clock(&tod_high, &tod_low);
-    if (r) {
-        error_report("Unable to set KVM guest TOD clock: %s", strerror(-r));
-    }
-
-    return r;
-}
-
-static SaveVMHandlers savevm_gtod = {
-    .save_state = gtod_save,
-    .load_state = gtod_load,
-};
-
 static void s390_init_ipl_dev(const char *kernel_filename,
                               const char *kernel_cmdline,
                               const char *initrd_filename, const char *firmware,
@@ -363,8 +312,8 @@ static void ccw_init(MachineState *machine)
         s390_create_sclpconsole("sclplmconsole", serial_hd(1));
     }
 
-    /* Register savevm handler for guest TOD clock */
-    register_savevm_live(NULL, "todclock", 0, 1, &savevm_gtod, NULL);
+    /* init the TOD clock */
+    s390_init_tod();
 }
 
 static void s390_cpu_plug(HotplugHandler *hotplug_dev,
diff --git a/hw/s390x/tod-kvm.c b/hw/s390x/tod-kvm.c
new file mode 100644
index 0000000000..df564ab89c
--- /dev/null
+++ b/hw/s390x/tod-kvm.c
@@ -0,0 +1,64 @@
+/*
+ * TOD (Time Of Day) clock - KVM implementation
+ *
+ * Copyright 2018 Red Hat, Inc.
+ * Author(s): David Hildenbrand <david@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/s390x/tod.h"
+#include "kvm_s390x.h"
+
+static void kvm_s390_tod_get(const S390TODState *td, S390TOD *tod, Error **errp)
+{
+    int r;
+
+    r = kvm_s390_get_clock_ext(&tod->high, &tod->low);
+    if (r == -ENXIO) {
+        r = kvm_s390_get_clock(&tod->high, &tod->low);
+    }
+    if (r) {
+        error_setg(errp, "Unable to get KVM guest TOD clock: %s",
+                   strerror(-r));
+    }
+}
+
+static void kvm_s390_tod_set(S390TODState *td, const S390TOD *tod, Error **errp)
+{
+    int r;
+
+    r = kvm_s390_set_clock_ext(tod->high, tod->low);
+    if (r == -ENXIO) {
+        r = kvm_s390_set_clock(tod->high, tod->low);
+    }
+    if (r) {
+        error_setg(errp, "Unable to set KVM guest TOD clock: %s",
+                   strerror(-r));
+    }
+}
+
+static void kvm_s390_tod_class_init(ObjectClass *oc, void *data)
+{
+    S390TODClass *tdc = S390_TOD_CLASS(oc);
+
+    tdc->get = kvm_s390_tod_get;
+    tdc->set = kvm_s390_tod_set;
+}
+
+static TypeInfo kvm_s390_tod_info = {
+    .name = TYPE_KVM_S390_TOD,
+    .parent = TYPE_S390_TOD,
+    .instance_size = sizeof(S390TODState),
+    .class_init = kvm_s390_tod_class_init,
+    .class_size = sizeof(S390TODClass),
+};
+
+static void register_types(void)
+{
+    type_register_static(&kvm_s390_tod_info);
+}
+type_init(register_types);
diff --git a/hw/s390x/tod-qemu.c b/hw/s390x/tod-qemu.c
new file mode 100644
index 0000000000..03ea1ce4e8
--- /dev/null
+++ b/hw/s390x/tod-qemu.c
@@ -0,0 +1,49 @@
+/*
+ * TOD (Time Of Day) clock - QEMU implementation
+ *
+ * Copyright 2018 Red Hat, Inc.
+ * Author(s): David Hildenbrand <david@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/s390x/tod.h"
+
+static void qemu_s390_tod_get(const S390TODState *td, S390TOD *tod,
+                              Error **errp)
+{
+    /* FIXME */
+    tod->high = 0;
+    tod->low = 0;
+}
+
+static void qemu_s390_tod_set(S390TODState *td, const S390TOD *tod,
+                              Error **errp)
+{
+    /* FIXME */
+}
+
+static void qemu_s390_tod_class_init(ObjectClass *oc, void *data)
+{
+    S390TODClass *tdc = S390_TOD_CLASS(oc);
+
+    tdc->get = qemu_s390_tod_get;
+    tdc->set = qemu_s390_tod_set;
+}
+
+static TypeInfo qemu_s390_tod_info = {
+    .name = TYPE_QEMU_S390_TOD,
+    .parent = TYPE_S390_TOD,
+    .instance_size = sizeof(S390TODState),
+    .class_init = qemu_s390_tod_class_init,
+    .class_size = sizeof(S390TODClass),
+};
+
+static void register_types(void)
+{
+    type_register_static(&qemu_s390_tod_info);
+}
+type_init(register_types);
diff --git a/hw/s390x/tod.c b/hw/s390x/tod.c
new file mode 100644
index 0000000000..0501affa7f
--- /dev/null
+++ b/hw/s390x/tod.c
@@ -0,0 +1,119 @@
+/*
+ * TOD (Time Of Day) clock
+ *
+ * Copyright 2018 Red Hat, Inc.
+ * Author(s): David Hildenbrand <david@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "hw/s390x/tod.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+#include "sysemu/kvm.h"
+#include "migration/register.h"
+
+void s390_init_tod(void)
+{
+    Object *obj;
+
+    if (kvm_enabled()) {
+        obj = object_new(TYPE_KVM_S390_TOD);
+    } else {
+        obj = object_new(TYPE_QEMU_S390_TOD);
+    }
+    object_property_add_child(qdev_get_machine(), TYPE_S390_TOD, obj, NULL);
+    object_unref(obj);
+
+    qdev_init_nofail(DEVICE(obj));
+}
+
+#define S390_TOD_CLOCK_VALUE_MISSING    0x00
+#define S390_TOD_CLOCK_VALUE_PRESENT    0x01
+
+static void s390_tod_save(QEMUFile *f, void *opaque)
+{
+    S390TODState *td = opaque;
+    S390TODClass *tdc = S390_TOD_GET_CLASS(td);
+    Error *err = NULL;
+    S390TOD tod;
+
+    tdc->get(td, &tod, &err);
+    if (err) {
+        warn_report_err(err);
+        error_printf("Guest clock will not be migrated "
+                     "which could cause the guest to hang.");
+        qemu_put_byte(f, S390_TOD_CLOCK_VALUE_MISSING);
+        return;
+    }
+
+    qemu_put_byte(f, S390_TOD_CLOCK_VALUE_PRESENT);
+    qemu_put_byte(f, tod.high);
+    qemu_put_be64(f, tod.low);
+}
+
+static int s390_tod_load(QEMUFile *f, void *opaque, int version_id)
+{
+    S390TODState *td = opaque;
+    S390TODClass *tdc = S390_TOD_GET_CLASS(td);
+    Error *err = NULL;
+    S390TOD tod;
+
+    if (qemu_get_byte(f) == S390_TOD_CLOCK_VALUE_MISSING) {
+        warn_report("Guest clock was not migrated. This could "
+                    "cause the guest to hang.");
+        return 0;
+    }
+
+    tod.high = qemu_get_byte(f);
+    tod.low = qemu_get_be64(f);
+
+    tdc->set(td, &tod, &err);
+    if (err) {
+        error_report_err(err);
+        return -1;
+    }
+    return 0;
+}
+
+static SaveVMHandlers savevm_tod = {
+    .save_state = s390_tod_save,
+    .load_state = s390_tod_load,
+};
+
+static void s390_tod_realize(DeviceState *dev, Error **errp)
+{
+    S390TODState *td = S390_TOD(dev);
+
+    /* Legacy migration interface */
+    register_savevm_live(NULL, "todclock", 0, 1, &savevm_tod, td);
+}
+
+static void s390_tod_class_init(ObjectClass *oc, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(oc);
+
+    dc->desc = "TOD (Time Of Day) Clock";
+    dc->realize = s390_tod_realize;
+    set_bit(DEVICE_CATEGORY_MISC, dc->categories);
+
+    /* We only have one TOD clock in the system attached to the machine */
+    dc->user_creatable = false;
+}
+
+static TypeInfo s390_tod_info = {
+    .name = TYPE_S390_TOD,
+    .parent = TYPE_DEVICE,
+    .instance_size = sizeof(S390TODState),
+    .class_init = s390_tod_class_init,
+    .class_size = sizeof(S390TODClass),
+    .abstract = true,
+};
+
+static void register_types(void)
+{
+    type_register_static(&s390_tod_info);
+}
+type_init(register_types);
diff --git a/include/hw/s390x/tod.h b/include/hw/s390x/tod.h
new file mode 100644
index 0000000000..7096b574ae
--- /dev/null
+++ b/include/hw/s390x/tod.h
@@ -0,0 +1,46 @@
+/*
+ * TOD (Time Of Day) clock
+ *
+ * Copyright 2018 Red Hat, Inc.
+ * Author(s): David Hildenbrand <david@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef HW_S390_TOD_H
+#define HW_S390_TOD_H
+
+#include "hw/qdev.h"
+
+typedef struct S390TOD {
+    uint8_t high;
+    uint64_t low;
+} S390TOD;
+
+#define TYPE_S390_TOD "s390-tod"
+#define S390_TOD(obj) OBJECT_CHECK(S390TODState, (obj), TYPE_S390_TOD)
+#define S390_TOD_CLASS(oc) OBJECT_CLASS_CHECK(S390TODClass, (oc), \
+                                              TYPE_S390_TOD)
+#define S390_TOD_GET_CLASS(obj) OBJECT_GET_CLASS(S390TODClass, (obj), \
+                                                 TYPE_S390_TOD)
+#define TYPE_KVM_S390_TOD TYPE_S390_TOD "-kvm"
+#define TYPE_QEMU_S390_TOD TYPE_S390_TOD "-qemu"
+
+typedef struct S390TODState {
+    /* private */
+    DeviceState parent_obj;
+} S390TODState;
+
+typedef struct S390TODClass {
+    /* private */
+    DeviceClass parent_class;
+
+    /* public */
+    void (*get)(const S390TODState *td, S390TOD *tod, Error **errp);
+    void (*set)(S390TODState *td, const S390TOD *tod, Error **errp);
+} S390TODClass;
+
+void s390_init_tod(void);
+
+#endif
diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
index 68512e3e54..03ea6eafa7 100644
--- a/target/s390x/cpu.c
+++ b/target/s390x/cpu.c
@@ -390,38 +390,6 @@ unsigned int s390_cpu_set_state(uint8_t cpu_state, S390CPU *cpu)
     return s390_count_running_cpus();
 }
 
-int s390_get_clock(uint8_t *tod_high, uint64_t *tod_low)
-{
-    int r = 0;
-
-    if (kvm_enabled()) {
-        r = kvm_s390_get_clock_ext(tod_high, tod_low);
-        if (r == -ENXIO) {
-            return kvm_s390_get_clock(tod_high, tod_low);
-        }
-    } else {
-        /* Fixme TCG */
-        *tod_high = 0;
-        *tod_low = 0;
-    }
-
-    return r;
-}
-
-int s390_set_clock(uint8_t *tod_high, uint64_t *tod_low)
-{
-    int r = 0;
-
-    if (kvm_enabled()) {
-        r = kvm_s390_set_clock_ext(*tod_high, *tod_low);
-        if (r == -ENXIO) {
-            return kvm_s390_set_clock(*tod_high, *tod_low);
-        }
-    }
-    /* Fixme TCG */
-    return r;
-}
-
 int s390_set_memory_limit(uint64_t new_limit, uint64_t *hw_limit)
 {
     if (kvm_enabled()) {
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index 6629a533f3..ac51c17fb4 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -714,8 +714,6 @@ static inline void s390_do_cpu_load_normal(CPUState *cs, run_on_cpu_data arg)
 
 
 /* cpu.c */
-int s390_get_clock(uint8_t *tod_high, uint64_t *tod_low);
-int s390_set_clock(uint8_t *tod_high, uint64_t *tod_low);
 void s390_crypto_reset(void);
 bool s390_get_squash_mcss(void);
 int s390_set_memory_limit(uint64_t new_limit, uint64_t *hw_limit);
diff --git a/target/s390x/kvm_s390x.h b/target/s390x/kvm_s390x.h
index 36eb34bf99..6e52287da3 100644
--- a/target/s390x/kvm_s390x.h
+++ b/target/s390x/kvm_s390x.h
@@ -10,6 +10,8 @@
 #ifndef KVM_S390X_H
 #define KVM_S390X_H
 
+#include "cpu-qom.h"
+
 struct kvm_s390_irq;
 
 void kvm_s390_floating_interrupt_legacy(struct kvm_s390_irq *irq);
-- 
2.17.1

[Qemu-devel] [PATCH v3 4/9] s390x/tcg: drop tod_basetime

[David Hildenbrand]

Never set to anything but 0.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
---
 target/s390x/cpu.c         | 1 -
 target/s390x/cpu.h         | 1 -
 target/s390x/misc_helper.c | 4 ++--
 3 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
index 03ea6eafa7..a41b3f34ab 100644
--- a/target/s390x/cpu.c
+++ b/target/s390x/cpu.c
@@ -289,7 +289,6 @@ static void s390_cpu_initfn(Object *obj)
     qemu_get_timedate(&tm, 0);
     env->tod_offset = TOD_UNIX_EPOCH +
                       (time2tod(mktimegm(&tm)) * 1000000000ULL);
-    env->tod_basetime = 0;
     env->tod_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, s390x_tod_timer, cpu);
     env->cpu_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, s390x_cpu_timer, cpu);
     s390_cpu_set_state(S390_CPU_STATE_STOPPED, cpu);
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index ac51c17fb4..4abfe88a3d 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -131,7 +131,6 @@ struct CPUS390XState {
 #endif
 
     uint64_t tod_offset;
-    uint64_t tod_basetime;
     QEMUTimer *tod_timer;
 
     QEMUTimer *cpu_timer;
diff --git a/target/s390x/misc_helper.c b/target/s390x/misc_helper.c
index de1ced2082..dd5273949b 100644
--- a/target/s390x/misc_helper.c
+++ b/target/s390x/misc_helper.c
@@ -141,7 +141,7 @@ uint64_t HELPER(stck)(CPUS390XState *env)
     uint64_t time;
 
     time = env->tod_offset +
-        time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) - env->tod_basetime);
+        time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
 
     return time;
 }
@@ -161,7 +161,7 @@ void HELPER(sckc)(CPUS390XState *env, uint64_t time)
     /* nanoseconds */
     time = tod2time(time);
 
-    timer_mod(env->tod_timer, env->tod_basetime + time);
+    timer_mod(env->tod_timer, time);
 }
 
 /* Set Tod Programmable Field */
-- 
2.17.1

[Qemu-devel] [PATCH v3 5/9] s390x/tcg: properly implement the TOD

[David Hildenbrand]

Right now, each CPU has its own TOD. Especially, the TOD will differ
based on creation time of a CPU - e.g. when hotplugging a CPU the times
will differ quite a lot, resulting in stall warnings in the guest.

Let's use a single TOD by implementing our new TOD device. Prepare it
for TOD-clock epoch extension.

Most importantly, whenever we set the TOD, we have to update the CKC
timer.

Introduce "tcg_s390x.h" just like "kvm_s390x.h" for tcg specific
function declarations that should not go into cpu.h.

Signed-off-by: David Hildenbrand <david@redhat.com>
---
 hw/s390x/tod-qemu.c        | 46 ++++++++++++++++++++++++++++++++++----
 hw/s390x/tod.c             | 11 +++++++++
 include/hw/s390x/tod.h     | 19 ++++++++++++++++
 target/s390x/cpu.c         |  7 ------
 target/s390x/cpu.h         |  1 -
 target/s390x/internal.h    | 16 -------------
 target/s390x/misc_helper.c | 30 ++++++++++++++++++++-----
 target/s390x/tcg_s390x.h   | 18 +++++++++++++++
 8 files changed, 114 insertions(+), 34 deletions(-)
 create mode 100644 target/s390x/tcg_s390x.h

diff --git a/hw/s390x/tod-qemu.c b/hw/s390x/tod-qemu.c
index 03ea1ce4e8..941e84693e 100644
--- a/hw/s390x/tod-qemu.c
+++ b/hw/s390x/tod-qemu.c
@@ -11,19 +11,43 @@
 #include "qemu/osdep.h"
 #include "qapi/error.h"
 #include "hw/s390x/tod.h"
+#include "qemu/timer.h"
+#include "qemu/cutils.h"
+#include "cpu.h"
+#include "tcg_s390x.h"
 
 static void qemu_s390_tod_get(const S390TODState *td, S390TOD *tod,
                               Error **errp)
 {
-    /* FIXME */
-    tod->high = 0;
-    tod->low = 0;
+    *tod = td->base;
+
+    tod->low += time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
+    if (tod->low < td->base.low) {
+        tod->high++;
+    }
 }
 
 static void qemu_s390_tod_set(S390TODState *td, const S390TOD *tod,
                               Error **errp)
 {
-    /* FIXME */
+    CPUState *cpu;
+
+    td->base = *tod;
+
+    td->base.low -= time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
+    if (tod->low < td->base.low) {
+        td->base.high--;
+    }
+
+    /*
+     * The TOD has been changed and we have to recalculate the CKC values
+     * for all CPUs. We do this asynchronously, as "SET CLOCK should be
+     * issued only while all other activity on all CPUs .. has been
+     * suspended".
+     */
+    CPU_FOREACH(cpu) {
+        async_run_on_cpu(cpu, tcg_s390_tod_updated, RUN_ON_CPU_NULL);
+    }
 }
 
 static void qemu_s390_tod_class_init(ObjectClass *oc, void *data)
@@ -34,10 +58,24 @@ static void qemu_s390_tod_class_init(ObjectClass *oc, void *data)
     tdc->set = qemu_s390_tod_set;
 }
 
+static void qemu_s390_tod_init(Object *obj)
+{
+    S390TODState *td = S390_TOD(obj);
+    struct tm tm;
+
+    qemu_get_timedate(&tm, 0);
+    td->base.high = 0;
+    td->base.low = TOD_UNIX_EPOCH + (time2tod(mktimegm(&tm)) * 1000000000ULL);
+    if (td->base.low < TOD_UNIX_EPOCH) {
+        td->base.high += 1;
+    }
+}
+
 static TypeInfo qemu_s390_tod_info = {
     .name = TYPE_QEMU_S390_TOD,
     .parent = TYPE_S390_TOD,
     .instance_size = sizeof(S390TODState),
+    .instance_init = qemu_s390_tod_init,
     .class_init = qemu_s390_tod_class_init,
     .class_size = sizeof(S390TODClass),
 };
diff --git a/hw/s390x/tod.c b/hw/s390x/tod.c
index 0501affa7f..1c63f411e6 100644
--- a/hw/s390x/tod.c
+++ b/hw/s390x/tod.c
@@ -30,6 +30,17 @@ void s390_init_tod(void)
     qdev_init_nofail(DEVICE(obj));
 }
 
+S390TODState *s390_get_todstate(void)
+{
+    static S390TODState *ts;
+
+    if (!ts) {
+        ts = S390_TOD(object_resolve_path_type("", TYPE_S390_TOD, NULL));
+    }
+
+    return ts;
+}
+
 #define S390_TOD_CLOCK_VALUE_MISSING    0x00
 #define S390_TOD_CLOCK_VALUE_PRESENT    0x01
 
diff --git a/include/hw/s390x/tod.h b/include/hw/s390x/tod.h
index 7096b574ae..413c0d7c02 100644
--- a/include/hw/s390x/tod.h
+++ b/include/hw/s390x/tod.h
@@ -30,6 +30,9 @@ typedef struct S390TOD {
 typedef struct S390TODState {
     /* private */
     DeviceState parent_obj;
+
+    /* unused by KVM implementation */
+    S390TOD base;
 } S390TODState;
 
 typedef struct S390TODClass {
@@ -41,6 +44,22 @@ typedef struct S390TODClass {
     void (*set)(S390TODState *td, const S390TOD *tod, Error **errp);
 } S390TODClass;
 
+/* The value of the TOD clock for 1.1.1970. */
+#define TOD_UNIX_EPOCH 0x7d91048bca000000ULL
+
+/* Converts ns to s390's clock format */
+static inline uint64_t time2tod(uint64_t ns)
+{
+    return (ns << 9) / 125 + (((ns & 0xff10000000000000ull) / 125) << 9);
+}
+
+/* Converts s390's clock format to ns */
+static inline uint64_t tod2time(uint64_t t)
+{
+    return ((t >> 9) * 125) + (((t & 0x1ff) * 125) >> 9);
+}
+
 void s390_init_tod(void);
+S390TODState *s390_get_todstate(void);
 
 #endif
diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
index a41b3f34ab..40d6980229 100644
--- a/target/s390x/cpu.c
+++ b/target/s390x/cpu.c
@@ -30,7 +30,6 @@
 #include "kvm_s390x.h"
 #include "sysemu/kvm.h"
 #include "qemu-common.h"
-#include "qemu/cutils.h"
 #include "qemu/timer.h"
 #include "qemu/error-report.h"
 #include "trace.h"
@@ -275,9 +274,6 @@ static void s390_cpu_initfn(Object *obj)
     CPUState *cs = CPU(obj);
     S390CPU *cpu = S390_CPU(obj);
     CPUS390XState *env = &cpu->env;
-#if !defined(CONFIG_USER_ONLY)
-    struct tm tm;
-#endif
 
     cs->env_ptr = env;
     cs->halted = 1;
@@ -286,9 +282,6 @@ static void s390_cpu_initfn(Object *obj)
                         s390_cpu_get_crash_info_qom, NULL, NULL, NULL, NULL);
     s390_cpu_model_register_props(obj);
 #if !defined(CONFIG_USER_ONLY)
-    qemu_get_timedate(&tm, 0);
-    env->tod_offset = TOD_UNIX_EPOCH +
-                      (time2tod(mktimegm(&tm)) * 1000000000ULL);
     env->tod_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, s390x_tod_timer, cpu);
     env->cpu_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, s390x_cpu_timer, cpu);
     s390_cpu_set_state(S390_CPU_STATE_STOPPED, cpu);
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index 4abfe88a3d..2c3dd2d189 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -130,7 +130,6 @@ struct CPUS390XState {
     uint64_t cpuid;
 #endif
 
-    uint64_t tod_offset;
     QEMUTimer *tod_timer;
 
     QEMUTimer *cpu_timer;
diff --git a/target/s390x/internal.h b/target/s390x/internal.h
index 6cf63340bf..f2a771e2b4 100644
--- a/target/s390x/internal.h
+++ b/target/s390x/internal.h
@@ -237,22 +237,6 @@ enum cc_op {
     CC_OP_MAX
 };
 
-/* The value of the TOD clock for 1.1.1970. */
-#define TOD_UNIX_EPOCH 0x7d91048bca000000ULL
-
-/* Converts ns to s390's clock format */
-static inline uint64_t time2tod(uint64_t ns)
-{
-    return (ns << 9) / 125 + (((ns & 0xff10000000000000ull) / 125) << 9);
-
-}
-
-/* Converts s390's clock format to ns */
-static inline uint64_t tod2time(uint64_t t)
-{
-    return ((t >> 9) * 125) + (((t & 0x1ff) * 125) >> 9);
-}
-
 static inline hwaddr decode_basedisp_s(CPUS390XState *env, uint32_t ipb,
                                        uint8_t *ar)
 {
diff --git a/target/s390x/misc_helper.c b/target/s390x/misc_helper.c
index dd5273949b..be341b5295 100644
--- a/target/s390x/misc_helper.c
+++ b/target/s390x/misc_helper.c
@@ -28,6 +28,8 @@
 #include "qemu/timer.h"
 #include "exec/exec-all.h"
 #include "exec/cpu_ldst.h"
+#include "qapi/error.h"
+#include "tcg_s390x.h"
 
 #if !defined(CONFIG_USER_ONLY)
 #include "sysemu/cpus.h"
@@ -39,6 +41,7 @@
 #include "hw/s390x/ioinst.h"
 #include "hw/s390x/s390-pci-inst.h"
 #include "hw/boards.h"
+#include "hw/s390x/tod.h"
 #endif
 
 /* #define DEBUG_HELPER */
@@ -138,25 +141,32 @@ void HELPER(spx)(CPUS390XState *env, uint64_t a1)
 /* Store Clock */
 uint64_t HELPER(stck)(CPUS390XState *env)
 {
-    uint64_t time;
+    S390TODState *td = s390_get_todstate();
+    S390TODClass *tdc = S390_TOD_GET_CLASS(td);
+    S390TOD tod;
 
-    time = env->tod_offset +
-        time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
-
-    return time;
+    tdc->get(td, &tod, &error_abort);
+    return tod.low;
 }
 
 /* Set Clock Comparator */
 void HELPER(sckc)(CPUS390XState *env, uint64_t time)
 {
+    S390TODState *td = s390_get_todstate();
+    S390TODClass *tdc = S390_TOD_GET_CLASS(td);
+    S390TOD tod_base;
+
     if (time == -1ULL) {
         return;
     }
 
     env->ckc = time;
 
+    tdc->get(td, &tod_base, &error_abort);
+    tod_base.low -= time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
+
     /* difference between origins */
-    time -= env->tod_offset;
+    time -= tod_base.low;
 
     /* nanoseconds */
     time = tod2time(time);
@@ -164,6 +174,14 @@ void HELPER(sckc)(CPUS390XState *env, uint64_t time)
     timer_mod(env->tod_timer, time);
 }
 
+void tcg_s390_tod_updated(CPUState *cs, run_on_cpu_data opaque)
+{
+    S390CPU *cpu = S390_CPU(cs);
+    CPUS390XState *env = &cpu->env;
+
+    helper_sckc(env, env->ckc);
+}
+
 /* Set Tod Programmable Field */
 void HELPER(sckpf)(CPUS390XState *env, uint64_t r0)
 {
diff --git a/target/s390x/tcg_s390x.h b/target/s390x/tcg_s390x.h
new file mode 100644
index 0000000000..4e308aa0ce
--- /dev/null
+++ b/target/s390x/tcg_s390x.h
@@ -0,0 +1,18 @@
+/*
+ * QEMU TCG support -- s390x specific functions.
+ *
+ * Copyright 2018 Red Hat, Inc.
+ *
+ * Authors:
+ *   David Hildenbrand <david@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef TCG_S390X_H
+#define TCG_S390X_H
+
+void tcg_s390_tod_updated(CPUState *cs, run_on_cpu_data opaque);
+
+#endif /* TCG_S390X_H */
-- 
2.17.1

Re: [Qemu-devel] [qemu-s390x] [PATCH v3 5/9] s390x/tcg: properly implement the TOD

[Thomas Huth]

On 25.06.2018 13:53, David Hildenbrand wrote:
> Right now, each CPU has its own TOD. Especially, the TOD will differ
> based on creation time of a CPU - e.g. when hotplugging a CPU the times
> will differ quite a lot, resulting in stall warnings in the guest.
> 
> Let's use a single TOD by implementing our new TOD device. Prepare it
> for TOD-clock epoch extension.
> 
> Most importantly, whenever we set the TOD, we have to update the CKC
> timer.
> 
> Introduce "tcg_s390x.h" just like "kvm_s390x.h" for tcg specific
> function declarations that should not go into cpu.h.
> 
> Signed-off-by: David Hildenbrand <david@redhat.com>
> ---
>  hw/s390x/tod-qemu.c        | 46 ++++++++++++++++++++++++++++++++++----
>  hw/s390x/tod.c             | 11 +++++++++
>  include/hw/s390x/tod.h     | 19 ++++++++++++++++
>  target/s390x/cpu.c         |  7 ------
>  target/s390x/cpu.h         |  1 -
>  target/s390x/internal.h    | 16 -------------
>  target/s390x/misc_helper.c | 30 ++++++++++++++++++++-----
>  target/s390x/tcg_s390x.h   | 18 +++++++++++++++
>  8 files changed, 114 insertions(+), 34 deletions(-)
>  create mode 100644 target/s390x/tcg_s390x.h
> 
> diff --git a/hw/s390x/tod-qemu.c b/hw/s390x/tod-qemu.c
> index 03ea1ce4e8..941e84693e 100644
> --- a/hw/s390x/tod-qemu.c
> +++ b/hw/s390x/tod-qemu.c
> @@ -11,19 +11,43 @@
>  #include "qemu/osdep.h"
>  #include "qapi/error.h"
>  #include "hw/s390x/tod.h"
> +#include "qemu/timer.h"
> +#include "qemu/cutils.h"
> +#include "cpu.h"
> +#include "tcg_s390x.h"
>  
>  static void qemu_s390_tod_get(const S390TODState *td, S390TOD *tod,
>                                Error **errp)
>  {
> -    /* FIXME */
> -    tod->high = 0;
> -    tod->low = 0;
> +    *tod = td->base;
> +
> +    tod->low += time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
> +    if (tod->low < td->base.low) {
> +        tod->high++;
> +    }
>  }
>  
>  static void qemu_s390_tod_set(S390TODState *td, const S390TOD *tod,
>                                Error **errp)
>  {
> -    /* FIXME */
> +    CPUState *cpu;
> +
> +    td->base = *tod;
> +
> +    td->base.low -= time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
> +    if (tod->low < td->base.low) {

Just a matter of taste, but I'd rather use "td->base.low > tod->low"
here instead (since the operations before and after the if-statement are
related to td->base).

> +        td->base.high--;
> +    }
> +
> +    /*
> +     * The TOD has been changed and we have to recalculate the CKC values
> +     * for all CPUs. We do this asynchronously, as "SET CLOCK should be
> +     * issued only while all other activity on all CPUs .. has been
> +     * suspended".
> +     */
> +    CPU_FOREACH(cpu) {
> +        async_run_on_cpu(cpu, tcg_s390_tod_updated, RUN_ON_CPU_NULL);
> +    }
>  }
>  
>  static void qemu_s390_tod_class_init(ObjectClass *oc, void *data)
> @@ -34,10 +58,24 @@ static void qemu_s390_tod_class_init(ObjectClass *oc, void *data)
>      tdc->set = qemu_s390_tod_set;
>  }
>  
> +static void qemu_s390_tod_init(Object *obj)
> +{
> +    S390TODState *td = S390_TOD(obj);
> +    struct tm tm;
> +
> +    qemu_get_timedate(&tm, 0);
> +    td->base.high = 0;
> +    td->base.low = TOD_UNIX_EPOCH + (time2tod(mktimegm(&tm)) * 1000000000ULL);
> +    if (td->base.low < TOD_UNIX_EPOCH) {
> +        td->base.high += 1;
> +    }
> +}

Nit: It would be sufficient to do this in the realize() function instead.

>  static TypeInfo qemu_s390_tod_info = {
>      .name = TYPE_QEMU_S390_TOD,
>      .parent = TYPE_S390_TOD,
>      .instance_size = sizeof(S390TODState),
> +    .instance_init = qemu_s390_tod_init,
>      .class_init = qemu_s390_tod_class_init,
>      .class_size = sizeof(S390TODClass),
>  };
[...]
> diff --git a/target/s390x/misc_helper.c b/target/s390x/misc_helper.c
> index dd5273949b..be341b5295 100644
> --- a/target/s390x/misc_helper.c
> +++ b/target/s390x/misc_helper.c
> @@ -28,6 +28,8 @@
>  #include "qemu/timer.h"
>  #include "exec/exec-all.h"
>  #include "exec/cpu_ldst.h"
> +#include "qapi/error.h"
> +#include "tcg_s390x.h"
>  
>  #if !defined(CONFIG_USER_ONLY)
>  #include "sysemu/cpus.h"
> @@ -39,6 +41,7 @@
>  #include "hw/s390x/ioinst.h"
>  #include "hw/s390x/s390-pci-inst.h"
>  #include "hw/boards.h"
> +#include "hw/s390x/tod.h"
>  #endif
>  
>  /* #define DEBUG_HELPER */
> @@ -138,25 +141,32 @@ void HELPER(spx)(CPUS390XState *env, uint64_t a1)
>  /* Store Clock */
>  uint64_t HELPER(stck)(CPUS390XState *env)
>  {
> -    uint64_t time;
> +    S390TODState *td = s390_get_todstate();
> +    S390TODClass *tdc = S390_TOD_GET_CLASS(td);
> +    S390TOD tod;
>  
> -    time = env->tod_offset +
> -        time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
> -
> -    return time;
> +    tdc->get(td, &tod, &error_abort);
> +    return tod.low;
>  }
>  
>  /* Set Clock Comparator */
>  void HELPER(sckc)(CPUS390XState *env, uint64_t time)
>  {
> +    S390TODState *td = s390_get_todstate();
> +    S390TODClass *tdc = S390_TOD_GET_CLASS(td);
> +    S390TOD tod_base;
> +
>      if (time == -1ULL) {
>          return;
>      }
>  
>      env->ckc = time;
>  
> +    tdc->get(td, &tod_base, &error_abort);
> +    tod_base.low -= time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));

So the tdc->get first adds the time2tod, and then you subtract it here
again? Can't you simply use td->base.low directly instead?

>      /* difference between origins */
> -    time -= env->tod_offset;
> +    time -= tod_base.low;
>  
>      /* nanoseconds */
>      time = tod2time(time);
> @@ -164,6 +174,14 @@ void HELPER(sckc)(CPUS390XState *env, uint64_t time)
>      timer_mod(env->tod_timer, time);
>  }

... I found only nits, so with or without my suggested modifications:

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [Qemu-devel] [qemu-s390x] [PATCH v3 5/9] s390x/tcg: properly implement the TOD

[David Hildenbrand]

>> +    td->base = *tod;
>> +
>> +    td->base.low -= time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
>> +    if (tod->low < td->base.low) {
> 
> Just a matter of taste, but I'd rather use "td->base.low > tod->low"
> here instead (since the operations before and after the if-statement are
> related to td->base).

Changed.

> 
>> +        td->base.high--;
>> +    }
>> +
>> +    /*
>> +     * The TOD has been changed and we have to recalculate the CKC values
>> +     * for all CPUs. We do this asynchronously, as "SET CLOCK should be
>> +     * issued only while all other activity on all CPUs .. has been
>> +     * suspended".
>> +     */
>> +    CPU_FOREACH(cpu) {
>> +        async_run_on_cpu(cpu, tcg_s390_tod_updated, RUN_ON_CPU_NULL);
>> +    }
>>  }
>>  
>>  static void qemu_s390_tod_class_init(ObjectClass *oc, void *data)
>> @@ -34,10 +58,24 @@ static void qemu_s390_tod_class_init(ObjectClass *oc, void *data)
>>      tdc->set = qemu_s390_tod_set;
>>  }
>>  
>> +static void qemu_s390_tod_init(Object *obj)
>> +{
>> +    S390TODState *td = S390_TOD(obj);
>> +    struct tm tm;
>> +
>> +    qemu_get_timedate(&tm, 0);
>> +    td->base.high = 0;
>> +    td->base.low = TOD_UNIX_EPOCH + (time2tod(mktimegm(&tm)) * 1000000000ULL);
>> +    if (td->base.low < TOD_UNIX_EPOCH) {
>> +        td->base.high += 1;
>> +    }
>> +}
> 
> Nit: It would be sufficient to do this in the realize() function instead.

Then I'll have to overwrite the realize function and store the
parent_realize function - something that I want to avoid if not really
necessary.

(for now it was also done in the cpu initfn, so that should be fine)

> 
>>  static TypeInfo qemu_s390_tod_info = {
>>      .name = TYPE_QEMU_S390_TOD,
>>      .parent = TYPE_S390_TOD,
>>      .instance_size = sizeof(S390TODState),
>> +    .instance_init = qemu_s390_tod_init,
>>      .class_init = qemu_s390_tod_class_init,
>>      .class_size = sizeof(S390TODClass),
>>  };
> [...]
>> diff --git a/target/s390x/misc_helper.c b/target/s390x/misc_helper.c
>> index dd5273949b..be341b5295 100644
>> --- a/target/s390x/misc_helper.c
>> +++ b/target/s390x/misc_helper.c
>> @@ -28,6 +28,8 @@
>>  #include "qemu/timer.h"
>>  #include "exec/exec-all.h"
>>  #include "exec/cpu_ldst.h"
>> +#include "qapi/error.h"
>> +#include "tcg_s390x.h"
>>  
>>  #if !defined(CONFIG_USER_ONLY)
>>  #include "sysemu/cpus.h"
>> @@ -39,6 +41,7 @@
>>  #include "hw/s390x/ioinst.h"
>>  #include "hw/s390x/s390-pci-inst.h"
>>  #include "hw/boards.h"
>> +#include "hw/s390x/tod.h"
>>  #endif
>>  
>>  /* #define DEBUG_HELPER */
>> @@ -138,25 +141,32 @@ void HELPER(spx)(CPUS390XState *env, uint64_t a1)
>>  /* Store Clock */
>>  uint64_t HELPER(stck)(CPUS390XState *env)
>>  {
>> -    uint64_t time;
>> +    S390TODState *td = s390_get_todstate();
>> +    S390TODClass *tdc = S390_TOD_GET_CLASS(td);
>> +    S390TOD tod;
>>  
>> -    time = env->tod_offset +
>> -        time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
>> -
>> -    return time;
>> +    tdc->get(td, &tod, &error_abort);
>> +    return tod.low;
>>  }
>>  
>>  /* Set Clock Comparator */
>>  void HELPER(sckc)(CPUS390XState *env, uint64_t time)
>>  {
>> +    S390TODState *td = s390_get_todstate();
>> +    S390TODClass *tdc = S390_TOD_GET_CLASS(td);
>> +    S390TOD tod_base;
>> +
>>      if (time == -1ULL) {
>>          return;
>>      }
>>  
>>      env->ckc = time;
>>  
>> +    tdc->get(td, &tod_base, &error_abort);
>> +    tod_base.low -= time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
> 
> So the tdc->get first adds the time2tod, and then you subtract it here
> again? Can't you simply use td->base.low directly instead?

That might be a good idea, previously I had tdc->get_base(), but dropped
it because it cannot be implemented for KVM.

Simply accessing the member here should be fine. Thanks!

> 
>>      /* difference between origins */
>> -    time -= env->tod_offset;
>> +    time -= tod_base.low;
>>  
>>      /* nanoseconds */
>>      time = tod2time(time);
>> @@ -164,6 +174,14 @@ void HELPER(sckc)(CPUS390XState *env, uint64_t time)
>>      timer_mod(env->tod_timer, time);
>>  }
> 
> ... I found only nits, so with or without my suggested modifications:
> 
> Reviewed-by: Thomas Huth <thuth@redhat.com>
> 


-- 

Thanks,

David / dhildenb

Re: [Qemu-devel] [qemu-s390x] [PATCH v3 5/9] s390x/tcg: properly implement the TOD

[Cornelia Huck]

On Tue, 26 Jun 2018 14:06:00 +0200
David Hildenbrand <david@redhat.com> wrote:

> >> @@ -34,10 +58,24 @@ static void qemu_s390_tod_class_init(ObjectClass *oc, void *data)
> >>      tdc->set = qemu_s390_tod_set;
> >>  }
> >>  
> >> +static void qemu_s390_tod_init(Object *obj)
> >> +{
> >> +    S390TODState *td = S390_TOD(obj);
> >> +    struct tm tm;
> >> +
> >> +    qemu_get_timedate(&tm, 0);
> >> +    td->base.high = 0;
> >> +    td->base.low = TOD_UNIX_EPOCH + (time2tod(mktimegm(&tm)) * 1000000000ULL);
> >> +    if (td->base.low < TOD_UNIX_EPOCH) {
> >> +        td->base.high += 1;
> >> +    }
> >> +}  
> > 
> > Nit: It would be sufficient to do this in the realize() function instead.  
> 
> Then I'll have to overwrite the realize function and store the
> parent_realize function - something that I want to avoid if not really
> necessary.

Agreed. I'd just leave it as it is now.

> 
> (for now it was also done in the cpu initfn, so that should be fine)
> 
> >   
> >>  static TypeInfo qemu_s390_tod_info = {
> >>      .name = TYPE_QEMU_S390_TOD,
> >>      .parent = TYPE_S390_TOD,
> >>      .instance_size = sizeof(S390TODState),
> >> +    .instance_init = qemu_s390_tod_init,
> >>      .class_init = qemu_s390_tod_class_init,
> >>      .class_size = sizeof(S390TODClass),
> >>  };  
> > [...]  
> >> diff --git a/target/s390x/misc_helper.c b/target/s390x/misc_helper.c
> >> index dd5273949b..be341b5295 100644
> >> --- a/target/s390x/misc_helper.c
> >> +++ b/target/s390x/misc_helper.c
> >> @@ -28,6 +28,8 @@
> >>  #include "qemu/timer.h"
> >>  #include "exec/exec-all.h"
> >>  #include "exec/cpu_ldst.h"
> >> +#include "qapi/error.h"
> >> +#include "tcg_s390x.h"
> >>  
> >>  #if !defined(CONFIG_USER_ONLY)
> >>  #include "sysemu/cpus.h"
> >> @@ -39,6 +41,7 @@
> >>  #include "hw/s390x/ioinst.h"
> >>  #include "hw/s390x/s390-pci-inst.h"
> >>  #include "hw/boards.h"
> >> +#include "hw/s390x/tod.h"
> >>  #endif
> >>  
> >>  /* #define DEBUG_HELPER */
> >> @@ -138,25 +141,32 @@ void HELPER(spx)(CPUS390XState *env, uint64_t a1)
> >>  /* Store Clock */
> >>  uint64_t HELPER(stck)(CPUS390XState *env)
> >>  {
> >> -    uint64_t time;
> >> +    S390TODState *td = s390_get_todstate();
> >> +    S390TODClass *tdc = S390_TOD_GET_CLASS(td);
> >> +    S390TOD tod;
> >>  
> >> -    time = env->tod_offset +
> >> -        time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
> >> -
> >> -    return time;
> >> +    tdc->get(td, &tod, &error_abort);
> >> +    return tod.low;
> >>  }
> >>  
> >>  /* Set Clock Comparator */
> >>  void HELPER(sckc)(CPUS390XState *env, uint64_t time)
> >>  {
> >> +    S390TODState *td = s390_get_todstate();
> >> +    S390TODClass *tdc = S390_TOD_GET_CLASS(td);
> >> +    S390TOD tod_base;
> >> +
> >>      if (time == -1ULL) {
> >>          return;
> >>      }
> >>  
> >>      env->ckc = time;
> >>  
> >> +    tdc->get(td, &tod_base, &error_abort);
> >> +    tod_base.low -= time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));  
> > 
> > So the tdc->get first adds the time2tod, and then you subtract it here
> > again? Can't you simply use td->base.low directly instead?  
> 
> That might be a good idea, previously I had tdc->get_base(), but dropped
> it because it cannot be implemented for KVM.
> 
> Simply accessing the member here should be fine. Thanks!

So, we're guaranteed to have td->base.low at the correct value? (Sorry,
having a bit of a hard time following around here.)

> 
> >   
> >>      /* difference between origins */
> >> -    time -= env->tod_offset;
> >> +    time -= tod_base.low;
> >>  
> >>      /* nanoseconds */
> >>      time = tod2time(time);
> >> @@ -164,6 +174,14 @@ void HELPER(sckc)(CPUS390XState *env, uint64_t time)
> >>      timer_mod(env->tod_timer, time);
> >>  }  
> > 
> > ... I found only nits, so with or without my suggested modifications:
> > 
> > Reviewed-by: Thomas Huth <thuth@redhat.com>
> >   
> 
> 

Re: [Qemu-devel] [qemu-s390x] [PATCH v3 5/9] s390x/tcg: properly implement the TOD

[David Hildenbrand]

On 26.06.2018 14:27, Cornelia Huck wrote:
> On Tue, 26 Jun 2018 14:06:00 +0200
> David Hildenbrand <david@redhat.com> wrote:
> 
>>>> @@ -34,10 +58,24 @@ static void qemu_s390_tod_class_init(ObjectClass *oc, void *data)
>>>>      tdc->set = qemu_s390_tod_set;
>>>>  }
>>>>  
>>>> +static void qemu_s390_tod_init(Object *obj)
>>>> +{
>>>> +    S390TODState *td = S390_TOD(obj);
>>>> +    struct tm tm;
>>>> +
>>>> +    qemu_get_timedate(&tm, 0);
>>>> +    td->base.high = 0;
>>>> +    td->base.low = TOD_UNIX_EPOCH + (time2tod(mktimegm(&tm)) * 1000000000ULL);
>>>> +    if (td->base.low < TOD_UNIX_EPOCH) {
>>>> +        td->base.high += 1;
>>>> +    }
>>>> +}  
>>>
>>> Nit: It would be sufficient to do this in the realize() function instead.  
>>
>> Then I'll have to overwrite the realize function and store the
>> parent_realize function - something that I want to avoid if not really
>> necessary.
> 
> Agreed. I'd just leave it as it is now.
> 
>>
>> (for now it was also done in the cpu initfn, so that should be fine)
>>
>>>   
>>>>  static TypeInfo qemu_s390_tod_info = {
>>>>      .name = TYPE_QEMU_S390_TOD,
>>>>      .parent = TYPE_S390_TOD,
>>>>      .instance_size = sizeof(S390TODState),
>>>> +    .instance_init = qemu_s390_tod_init,
>>>>      .class_init = qemu_s390_tod_class_init,
>>>>      .class_size = sizeof(S390TODClass),
>>>>  };  
>>> [...]  
>>>> diff --git a/target/s390x/misc_helper.c b/target/s390x/misc_helper.c
>>>> index dd5273949b..be341b5295 100644
>>>> --- a/target/s390x/misc_helper.c
>>>> +++ b/target/s390x/misc_helper.c
>>>> @@ -28,6 +28,8 @@
>>>>  #include "qemu/timer.h"
>>>>  #include "exec/exec-all.h"
>>>>  #include "exec/cpu_ldst.h"
>>>> +#include "qapi/error.h"
>>>> +#include "tcg_s390x.h"
>>>>  
>>>>  #if !defined(CONFIG_USER_ONLY)
>>>>  #include "sysemu/cpus.h"
>>>> @@ -39,6 +41,7 @@
>>>>  #include "hw/s390x/ioinst.h"
>>>>  #include "hw/s390x/s390-pci-inst.h"
>>>>  #include "hw/boards.h"
>>>> +#include "hw/s390x/tod.h"
>>>>  #endif
>>>>  
>>>>  /* #define DEBUG_HELPER */
>>>> @@ -138,25 +141,32 @@ void HELPER(spx)(CPUS390XState *env, uint64_t a1)
>>>>  /* Store Clock */
>>>>  uint64_t HELPER(stck)(CPUS390XState *env)
>>>>  {
>>>> -    uint64_t time;
>>>> +    S390TODState *td = s390_get_todstate();
>>>> +    S390TODClass *tdc = S390_TOD_GET_CLASS(td);
>>>> +    S390TOD tod;
>>>>  
>>>> -    time = env->tod_offset +
>>>> -        time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));
>>>> -
>>>> -    return time;
>>>> +    tdc->get(td, &tod, &error_abort);
>>>> +    return tod.low;
>>>>  }
>>>>  
>>>>  /* Set Clock Comparator */
>>>>  void HELPER(sckc)(CPUS390XState *env, uint64_t time)
>>>>  {
>>>> +    S390TODState *td = s390_get_todstate();
>>>> +    S390TODClass *tdc = S390_TOD_GET_CLASS(td);
>>>> +    S390TOD tod_base;
>>>> +
>>>>      if (time == -1ULL) {
>>>>          return;
>>>>      }
>>>>  
>>>>      env->ckc = time;
>>>>  
>>>> +    tdc->get(td, &tod_base, &error_abort);
>>>> +    tod_base.low -= time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));  
>>>
>>> So the tdc->get first adds the time2tod, and then you subtract it here
>>> again? Can't you simply use td->base.low directly instead?  
>>
>> That might be a good idea, previously I had tdc->get_base(), but dropped
>> it because it cannot be implemented for KVM.
>>
>> Simply accessing the member here should be fine. Thanks!
> 
> So, we're guaranteed to have td->base.low at the correct value? (Sorry,
> having a bit of a hard time following around here.)
> 

Yes, just verified, changed and tested. We will have the value at that
place.

I'll give some more time to review the other parts, then I can resend.


-- 

Thanks,

David / dhildenb

Re: [Qemu-devel] [qemu-s390x] [PATCH v3 5/9] s390x/tcg: properly implement the TOD

[Cornelia Huck]

On Tue, 26 Jun 2018 14:28:18 +0200
David Hildenbrand <david@redhat.com> wrote:

> On 26.06.2018 14:27, Cornelia Huck wrote:
> > On Tue, 26 Jun 2018 14:06:00 +0200
> > David Hildenbrand <david@redhat.com> wrote:

> >>>>  /* Set Clock Comparator */
> >>>>  void HELPER(sckc)(CPUS390XState *env, uint64_t time)
> >>>>  {
> >>>> +    S390TODState *td = s390_get_todstate();
> >>>> +    S390TODClass *tdc = S390_TOD_GET_CLASS(td);
> >>>> +    S390TOD tod_base;
> >>>> +
> >>>>      if (time == -1ULL) {
> >>>>          return;
> >>>>      }
> >>>>  
> >>>>      env->ckc = time;
> >>>>  
> >>>> +    tdc->get(td, &tod_base, &error_abort);
> >>>> +    tod_base.low -= time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));    
> >>>
> >>> So the tdc->get first adds the time2tod, and then you subtract it here
> >>> again? Can't you simply use td->base.low directly instead?    
> >>
> >> That might be a good idea, previously I had tdc->get_base(), but dropped
> >> it because it cannot be implemented for KVM.
> >>
> >> Simply accessing the member here should be fine. Thanks!  
> > 
> > So, we're guaranteed to have td->base.low at the correct value? (Sorry,
> > having a bit of a hard time following around here.)
> >   
> 
> Yes, just verified, changed and tested. We will have the value at that
> place.
> 
> I'll give some more time to review the other parts, then I can resend.

Ok, thx.

[Qemu-devel] [PATCH v3 6/9] s390x/tcg: SET CLOCK COMPARATOR can clear CKC interrupts

[David Hildenbrand]

Let's stop the timer and delete any pending CKC IRQ before doing
anything else.

While at it, add a comment while the check for ckc == -1ULL is needed.

Signed-off-by: David Hildenbrand <david@redhat.com>
---
 target/s390x/misc_helper.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/target/s390x/misc_helper.c b/target/s390x/misc_helper.c
index be341b5295..4872bdd774 100644
--- a/target/s390x/misc_helper.c
+++ b/target/s390x/misc_helper.c
@@ -156,6 +156,13 @@ void HELPER(sckc)(CPUS390XState *env, uint64_t time)
     S390TODClass *tdc = S390_TOD_GET_CLASS(td);
     S390TOD tod_base;
 
+    /* stop the timer and remove pending CKC IRQs */
+    timer_del(env->tod_timer);
+    qemu_mutex_lock_iothread();
+    env->pending_int &= ~INTERRUPT_EXT_CLOCK_COMPARATOR;
+    qemu_mutex_unlock_iothread();
+
+    /* the tod has to exceed the ckc, this can never happen if ckc is all 1's */
     if (time == -1ULL) {
         return;
     }
-- 
2.17.1

Re: [Qemu-devel] [PATCH v3 6/9] s390x/tcg: SET CLOCK COMPARATOR can clear CKC interrupts

[Cornelia Huck]

On Mon, 25 Jun 2018 13:53:49 +0200
David Hildenbrand <david@redhat.com> wrote:

> Let's stop the timer and delete any pending CKC IRQ before doing
> anything else.
> 
> While at it, add a comment while the check for ckc == -1ULL is needed.

s/while/why/

> 
> Signed-off-by: David Hildenbrand <david@redhat.com>
> ---
>  target/s390x/misc_helper.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/target/s390x/misc_helper.c b/target/s390x/misc_helper.c
> index be341b5295..4872bdd774 100644
> --- a/target/s390x/misc_helper.c
> +++ b/target/s390x/misc_helper.c
> @@ -156,6 +156,13 @@ void HELPER(sckc)(CPUS390XState *env, uint64_t time)
>      S390TODClass *tdc = S390_TOD_GET_CLASS(td);
>      S390TOD tod_base;
>  
> +    /* stop the timer and remove pending CKC IRQs */
> +    timer_del(env->tod_timer);
> +    qemu_mutex_lock_iothread();
> +    env->pending_int &= ~INTERRUPT_EXT_CLOCK_COMPARATOR;
> +    qemu_mutex_unlock_iothread();
> +
> +    /* the tod has to exceed the ckc, this can never happen if ckc is all 1's */
>      if (time == -1ULL) {
>          return;
>      }

Re: [Qemu-devel] [qemu-s390x] [PATCH v3 6/9] s390x/tcg: SET CLOCK COMPARATOR can clear CKC interrupts

[Thomas Huth]

On 25.06.2018 13:53, David Hildenbrand wrote:
> Let's stop the timer and delete any pending CKC IRQ before doing
> anything else.
> 
> While at it, add a comment while the check for ckc == -1ULL is needed.
> 
> Signed-off-by: David Hildenbrand <david@redhat.com>
> ---
>  target/s390x/misc_helper.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/target/s390x/misc_helper.c b/target/s390x/misc_helper.c
> index be341b5295..4872bdd774 100644
> --- a/target/s390x/misc_helper.c
> +++ b/target/s390x/misc_helper.c
> @@ -156,6 +156,13 @@ void HELPER(sckc)(CPUS390XState *env, uint64_t time)
>      S390TODClass *tdc = S390_TOD_GET_CLASS(td);
>      S390TOD tod_base;
>  
> +    /* stop the timer and remove pending CKC IRQs */
> +    timer_del(env->tod_timer);
> +    qemu_mutex_lock_iothread();
> +    env->pending_int &= ~INTERRUPT_EXT_CLOCK_COMPARATOR;
> +    qemu_mutex_unlock_iothread();
> +
> +    /* the tod has to exceed the ckc, this can never happen if ckc is all 1's */
>      if (time == -1ULL) {
>          return;
>      }
> 

With the commit message ("while" -> "why") fixed:

Reviewed-by: Thomas Huth <thuth@redhat.com>

[Qemu-devel] [PATCH v3 7/9] s390x/tcg: implement SET CLOCK

[David Hildenbrand]

This allows a guest to change its TOD. We already take care of updating
all CKC timers from within S390TODClass.

Use MO_ALIGN to load the operand manually - this will properly trigger a
SPECIFICATION exception.

Signed-off-by: David Hildenbrand <david@redhat.com>
---
 target/s390x/helper.h      |  1 +
 target/s390x/insn-data.def |  3 +--
 target/s390x/misc_helper.c | 16 ++++++++++++++++
 target/s390x/translate.c   |  9 +++++++++
 4 files changed, 27 insertions(+), 2 deletions(-)

diff --git a/target/s390x/helper.h b/target/s390x/helper.h
index 59cba86a27..97c60ca7bc 100644
--- a/target/s390x/helper.h
+++ b/target/s390x/helper.h
@@ -127,6 +127,7 @@ DEF_HELPER_4(diag, void, env, i32, i32, i32)
 DEF_HELPER_3(load_psw, noreturn, env, i64, i64)
 DEF_HELPER_FLAGS_2(spx, TCG_CALL_NO_RWG, void, env, i64)
 DEF_HELPER_FLAGS_1(stck, TCG_CALL_NO_RWG_SE, i64, env)
+DEF_HELPER_FLAGS_2(sck, TCG_CALL_NO_RWG, i32, env, i64)
 DEF_HELPER_FLAGS_2(sckc, TCG_CALL_NO_RWG, void, env, i64)
 DEF_HELPER_FLAGS_2(sckpf, TCG_CALL_NO_RWG, void, env, i64)
 DEF_HELPER_FLAGS_1(stckc, TCG_CALL_NO_RWG, i64, env)
diff --git a/target/s390x/insn-data.def b/target/s390x/insn-data.def
index 157619403d..5c6f33ed9c 100644
--- a/target/s390x/insn-data.def
+++ b/target/s390x/insn-data.def
@@ -997,8 +997,7 @@
 /* SET ADDRESS SPACE CONTROL FAST */
     C(0xb279, SACF,    S,     Z,   0, a2, 0, 0, sacf, 0)
 /* SET CLOCK */
-    /* ??? Not implemented - is it necessary? */
-    C(0xb204, SCK,     S,     Z,   0, 0, 0, 0, 0, 0)
+    C(0xb204, SCK,     S,     Z,   la2, 0, 0, 0, sck, 0)
 /* SET CLOCK COMPARATOR */
     C(0xb206, SCKC,    S,     Z,   0, m2_64a, 0, 0, sckc, 0)
 /* SET CLOCK PROGRAMMABLE FIELD */
diff --git a/target/s390x/misc_helper.c b/target/s390x/misc_helper.c
index 4872bdd774..a223970d1c 100644
--- a/target/s390x/misc_helper.c
+++ b/target/s390x/misc_helper.c
@@ -189,6 +189,22 @@ void tcg_s390_tod_updated(CPUState *cs, run_on_cpu_data opaque)
     helper_sckc(env, env->ckc);
 }
 
+/* Set Clock */
+uint32_t HELPER(sck)(CPUS390XState *env, uint64_t tod_low)
+{
+    S390TODState *td = s390_get_todstate();
+    S390TODClass *tdc = S390_TOD_GET_CLASS(td);
+    S390TOD tod = {
+        .high = 0,
+        .low = tod_low,
+    };
+
+    qemu_mutex_lock_iothread();
+    tdc->set(td, &tod, &error_abort);
+    qemu_mutex_unlock_iothread();
+    return 0;
+}
+
 /* Set Tod Programmable Field */
 void HELPER(sckpf)(CPUS390XState *env, uint64_t r0)
 {
diff --git a/target/s390x/translate.c b/target/s390x/translate.c
index fdfec7feba..57c03cbf58 100644
--- a/target/s390x/translate.c
+++ b/target/s390x/translate.c
@@ -4016,6 +4016,15 @@ static DisasJumpType op_stcke(DisasContext *s, DisasOps *o)
     return DISAS_NEXT;
 }
 
+static DisasJumpType op_sck(DisasContext *s, DisasOps *o)
+{
+    check_privileged(s);
+    tcg_gen_qemu_ld_i64(o->in1, o->addr1, get_mem_index(s), MO_TEQ | MO_ALIGN);
+    gen_helper_sck(cc_op, cpu_env, o->in1);
+    set_cc_static(s);
+    return DISAS_NEXT;
+}
+
 static DisasJumpType op_sckc(DisasContext *s, DisasOps *o)
 {
     check_privileged(s);
-- 
2.17.1

Re: [Qemu-devel] [qemu-s390x] [PATCH v3 7/9] s390x/tcg: implement SET CLOCK

[Thomas Huth]

On 25.06.2018 13:53, David Hildenbrand wrote:
> This allows a guest to change its TOD. We already take care of updating
> all CKC timers from within S390TODClass.
> 
> Use MO_ALIGN to load the operand manually - this will properly trigger a
> SPECIFICATION exception.
> 
> Signed-off-by: David Hildenbrand <david@redhat.com>
> ---
>  target/s390x/helper.h      |  1 +
>  target/s390x/insn-data.def |  3 +--
>  target/s390x/misc_helper.c | 16 ++++++++++++++++
>  target/s390x/translate.c   |  9 +++++++++
>  4 files changed, 27 insertions(+), 2 deletions(-)
> 
> diff --git a/target/s390x/helper.h b/target/s390x/helper.h
> index 59cba86a27..97c60ca7bc 100644
> --- a/target/s390x/helper.h
> +++ b/target/s390x/helper.h
> @@ -127,6 +127,7 @@ DEF_HELPER_4(diag, void, env, i32, i32, i32)
>  DEF_HELPER_3(load_psw, noreturn, env, i64, i64)
>  DEF_HELPER_FLAGS_2(spx, TCG_CALL_NO_RWG, void, env, i64)
>  DEF_HELPER_FLAGS_1(stck, TCG_CALL_NO_RWG_SE, i64, env)
> +DEF_HELPER_FLAGS_2(sck, TCG_CALL_NO_RWG, i32, env, i64)
>  DEF_HELPER_FLAGS_2(sckc, TCG_CALL_NO_RWG, void, env, i64)
>  DEF_HELPER_FLAGS_2(sckpf, TCG_CALL_NO_RWG, void, env, i64)
>  DEF_HELPER_FLAGS_1(stckc, TCG_CALL_NO_RWG, i64, env)
> diff --git a/target/s390x/insn-data.def b/target/s390x/insn-data.def
> index 157619403d..5c6f33ed9c 100644
> --- a/target/s390x/insn-data.def
> +++ b/target/s390x/insn-data.def
> @@ -997,8 +997,7 @@
>  /* SET ADDRESS SPACE CONTROL FAST */
>      C(0xb279, SACF,    S,     Z,   0, a2, 0, 0, sacf, 0)
>  /* SET CLOCK */
> -    /* ??? Not implemented - is it necessary? */
> -    C(0xb204, SCK,     S,     Z,   0, 0, 0, 0, 0, 0)
> +    C(0xb204, SCK,     S,     Z,   la2, 0, 0, 0, sck, 0)
>  /* SET CLOCK COMPARATOR */
>      C(0xb206, SCKC,    S,     Z,   0, m2_64a, 0, 0, sckc, 0)
>  /* SET CLOCK PROGRAMMABLE FIELD */
> diff --git a/target/s390x/misc_helper.c b/target/s390x/misc_helper.c
> index 4872bdd774..a223970d1c 100644
> --- a/target/s390x/misc_helper.c
> +++ b/target/s390x/misc_helper.c
> @@ -189,6 +189,22 @@ void tcg_s390_tod_updated(CPUState *cs, run_on_cpu_data opaque)
>      helper_sckc(env, env->ckc);
>  }
>  
> +/* Set Clock */
> +uint32_t HELPER(sck)(CPUS390XState *env, uint64_t tod_low)
> +{
> +    S390TODState *td = s390_get_todstate();
> +    S390TODClass *tdc = S390_TOD_GET_CLASS(td);
> +    S390TOD tod = {
> +        .high = 0,
> +        .low = tod_low,
> +    };
> +
> +    qemu_mutex_lock_iothread();
> +    tdc->set(td, &tod, &error_abort);
> +    qemu_mutex_unlock_iothread();
> +    return 0;
> +}
> +
>  /* Set Tod Programmable Field */
>  void HELPER(sckpf)(CPUS390XState *env, uint64_t r0)
>  {
> diff --git a/target/s390x/translate.c b/target/s390x/translate.c
> index fdfec7feba..57c03cbf58 100644
> --- a/target/s390x/translate.c
> +++ b/target/s390x/translate.c
> @@ -4016,6 +4016,15 @@ static DisasJumpType op_stcke(DisasContext *s, DisasOps *o)
>      return DISAS_NEXT;
>  }
>  
> +static DisasJumpType op_sck(DisasContext *s, DisasOps *o)
> +{
> +    check_privileged(s);
> +    tcg_gen_qemu_ld_i64(o->in1, o->addr1, get_mem_index(s), MO_TEQ | MO_ALIGN);
> +    gen_helper_sck(cc_op, cpu_env, o->in1);
> +    set_cc_static(s);
> +    return DISAS_NEXT;
> +}
> +
>  static DisasJumpType op_sckc(DisasContext *s, DisasOps *o)
>  {
>      check_privileged(s);
> 

Acked-by: Thomas Huth <thuth@redhat.com>

[Qemu-devel] [PATCH v3 8/9] s390x/tcg: rearm the CKC timer during migration

[David Hildenbrand]

If the CPU data is migrated after the TOD clock, the CKC timer of a CPU
is not rearmed. Let's rearm it when loading the CPU state.

Introduce tcg-stub.c just like kvm-stub.c for tcg specific stubs.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: David Hildenbrand <david@redhat.com>
---
 target/s390x/Makefile.objs |  1 +
 target/s390x/machine.c     |  6 ++++++
 target/s390x/tcg-stub.c    | 20 ++++++++++++++++++++
 3 files changed, 27 insertions(+)
 create mode 100644 target/s390x/tcg-stub.c

diff --git a/target/s390x/Makefile.objs b/target/s390x/Makefile.objs
index 31932de9cf..22a9a9927a 100644
--- a/target/s390x/Makefile.objs
+++ b/target/s390x/Makefile.objs
@@ -5,6 +5,7 @@ obj-$(CONFIG_SOFTMMU) += machine.o ioinst.o arch_dump.o mmu_helper.o diag.o
 obj-$(CONFIG_SOFTMMU) += sigp.o
 obj-$(CONFIG_KVM) += kvm.o
 obj-$(call lnot,$(CONFIG_KVM)) += kvm-stub.o
+obj-$(call lnot,$(CONFIG_TCG)) += tcg-stub.o
 
 # build and run feature list generator
 feat-src = $(SRC_PATH)/target/$(TARGET_BASE_ARCH)/
diff --git a/target/s390x/machine.c b/target/s390x/machine.c
index 84b4928755..bd3230d027 100644
--- a/target/s390x/machine.c
+++ b/target/s390x/machine.c
@@ -19,6 +19,7 @@
 #include "cpu.h"
 #include "internal.h"
 #include "kvm_s390x.h"
+#include "tcg_s390x.h"
 #include "sysemu/kvm.h"
 
 static int cpu_post_load(void *opaque, int version_id)
@@ -34,6 +35,11 @@ static int cpu_post_load(void *opaque, int version_id)
         return kvm_s390_vcpu_interrupt_post_load(cpu);
     }
 
+    if (tcg_enabled()) {
+        /* Rearm the CKC timer if necessary */
+        tcg_s390_tod_updated(CPU(cpu), RUN_ON_CPU_NULL);
+    }
+
     return 0;
 }
 
diff --git a/target/s390x/tcg-stub.c b/target/s390x/tcg-stub.c
new file mode 100644
index 0000000000..c93501db0b
--- /dev/null
+++ b/target/s390x/tcg-stub.c
@@ -0,0 +1,20 @@
+/*
+ * QEMU TCG support -- s390x specific function stubs.
+ *
+ * Copyright (C) 2018 Red Hat Inc
+ *
+ * Authors:
+ *   David Hildenbrand <david@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu-common.h"
+#include "cpu.h"
+#include "tcg_s390x.h"
+
+void tcg_s390_tod_updated(CPUState *cs, run_on_cpu_data opaque)
+{
+}
-- 
2.17.1

[Qemu-devel] [PATCH v3 9/9] s390x/tcg: fix CPU hotplug with single-threaded TCG

[David Hildenbrand]

run_on_cpu() doesn't seem to work reliably until the CPU has been fully
created if the single-threaded TCG main loop is already running.

Therefore, hotplugging a CPU under single-threaded TCG does currently
not work. We should use the direct call instead of going via
run_on_cpu().

So let's use run_on_cpu() for KVM only - KVM requires it due to the initial
CPU reset ioctl. As a nice side effect, we get rif of the ifdef.

Signed-off-by: David Hildenbrand <david@redhat.com>
---
 target/s390x/cpu.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
index 40d6980229..271c5ce652 100644
--- a/target/s390x/cpu.c
+++ b/target/s390x/cpu.c
@@ -218,11 +218,18 @@ static void s390_cpu_realizefn(DeviceState *dev, Error **errp)
 #endif
     s390_cpu_gdb_init(cs);
     qemu_init_vcpu(cs);
-#if !defined(CONFIG_USER_ONLY)
-    run_on_cpu(cs, s390_do_cpu_full_reset, RUN_ON_CPU_NULL);
-#else
-    cpu_reset(cs);
-#endif
+
+    /*
+     * KVM requires the initial CPU reset ioctl to be executed on the target
+     * CPU thread. CPU hotplug under single-threaded TCG will not work with
+     * run_on_cpu(), as run_on_cpu() will not work properly if called while
+     * the main thread is already running but the CPU hasn't been realized.
+     */
+    if (kvm_enabled()) {
+        run_on_cpu(cs, s390_do_cpu_full_reset, RUN_ON_CPU_NULL);
+    } else {
+        cpu_reset(cs);
+    }
 
     scc->parent_realize(dev, &err);
 out:
-- 
2.17.1

Re: [Qemu-devel] [PATCH v3 9/9] s390x/tcg: fix CPU hotplug with single-threaded TCG

[Cornelia Huck]

On Mon, 25 Jun 2018 13:53:52 +0200
David Hildenbrand <david@redhat.com> wrote:

> run_on_cpu() doesn't seem to work reliably until the CPU has been fully
> created if the single-threaded TCG main loop is already running.
> 
> Therefore, hotplugging a CPU under single-threaded TCG does currently
> not work. We should use the direct call instead of going via
> run_on_cpu().
> 
> So let's use run_on_cpu() for KVM only - KVM requires it due to the initial
> CPU reset ioctl. As a nice side effect, we get rif of the ifdef.

s/rif/rid/

{Although "we get rif the thing" would be a nice contraction of "rid of" :}

> 
> Signed-off-by: David Hildenbrand <david@redhat.com>
> ---
>  target/s390x/cpu.c | 17 ++++++++++++-----
>  1 file changed, 12 insertions(+), 5 deletions(-)
> 
> diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
> index 40d6980229..271c5ce652 100644
> --- a/target/s390x/cpu.c
> +++ b/target/s390x/cpu.c
> @@ -218,11 +218,18 @@ static void s390_cpu_realizefn(DeviceState *dev, Error **errp)
>  #endif
>      s390_cpu_gdb_init(cs);
>      qemu_init_vcpu(cs);
> -#if !defined(CONFIG_USER_ONLY)
> -    run_on_cpu(cs, s390_do_cpu_full_reset, RUN_ON_CPU_NULL);
> -#else
> -    cpu_reset(cs);
> -#endif
> +
> +    /*
> +     * KVM requires the initial CPU reset ioctl to be executed on the target
> +     * CPU thread. CPU hotplug under single-threaded TCG will not work with
> +     * run_on_cpu(), as run_on_cpu() will not work properly if called while
> +     * the main thread is already running but the CPU hasn't been realized.
> +     */
> +    if (kvm_enabled()) {
> +        run_on_cpu(cs, s390_do_cpu_full_reset, RUN_ON_CPU_NULL);
> +    } else {
> +        cpu_reset(cs);
> +    }
>  
>      scc->parent_realize(dev, &err);
>  out:

Re: [Qemu-devel] [qemu-s390x] [PATCH v3 9/9] s390x/tcg: fix CPU hotplug with single-threaded TCG

[Thomas Huth]

On 25.06.2018 13:53, David Hildenbrand wrote:
> run_on_cpu() doesn't seem to work reliably until the CPU has been fully
> created if the single-threaded TCG main loop is already running.
> 
> Therefore, hotplugging a CPU under single-threaded TCG does currently
> not work. We should use the direct call instead of going via
> run_on_cpu().
> 
> So let's use run_on_cpu() for KVM only - KVM requires it due to the initial
> CPU reset ioctl. As a nice side effect, we get rif of the ifdef.

With the typo fixed:

Reviewed-by: Thomas Huth <thuth@redhat.com>