On Mon, Apr 29, 2019 at 08:06:00AM -0700, Paul E. McKenney wrote:
> On Mon, Apr 29, 2019 at 07:49:24AM -0700, Paul E. McKenney wrote:
> > On Mon, Apr 29, 2019 at 10:19:44AM +0200, Sebastian Andrzej Siewior wrote:
> > > On 2019-04-26 06:50:58 [-0700], Paul E. McKenney wrote:
> > > > One place to look is in the summary output:
> > > > 
> > > > TREE01 ------- 17540 GPs (58.4667/s) [rcu: g130629 f0x0 ]
> > > > 
> > > > The "58.4667/s" is the number of grace periods per second.  I would be
> > > > surprised if CONFIG_PARAVIRT_SPINLOCKS made a noticeable difference in
> > > > grace-period rate (given the natural variation), but you never know.
> > > 
> > > I did four runs of the different parts of the patch:
> > > - 5.1-rc6
> > > - #1 + kvm64 CPU + some config options
> > > - #2 + tsc-deadline=on and so on (the whole line)
> > > - #3 + CONFIG_PARAVIRT_SPINLOCKS (now everything)
> > > 
> > > the test command was
> > > 	tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 112 --duration 60 --configs "16*TREE08" --memory 4G
> > > 
> > > and the results:
> > > | HEAD is now at 085b7755808a... Linux 5.1-rc6
> > > | (28.5942 +27.4658 +28.0203 +27.2061 +28.0731 +26.9078 +27.8494 + 27.3392 +26.4339 +28.025 +27.4797 +27.6775 +28.0653 +28.0742 +27.9581 +28.6508)/ 16
> > > | 27.738775
> > > | 
> > > | HEAD is now at 36a12aa9761a... tune #1
> > > | (28.5761 +26.6514 +26.6989 +27.4375 +27.3442 +28.3228 +26.6353 +27.5461+28.5531 +27.7006 +27.8078 +27.9753 +27.4269 +28.0464 +27.6314 +27.8356) / 16
> > > | 27.6368375
> > > | 
> > > | HEAD is now at af5cd7196436... tune #2
> > > | (28.4867 +26.3675 +27.6364 +28.3344 +27.4153 +27.9306 +27.1703 +26.8461+27.3194 +28.5486 +27.8975 +27.4356 +28.12 +28.4397 +29.0186 +26.9328 )/ 16
> > > | 27.74371875
> > > | 
> > > | HEAD is now at 3701f64943f5... tune #3
> > > | (28.2431 +27.7831 +28.39 +28.2586 +27.7408 +27.9258 +26.6236 +26.7817+29.1178 +26.9564 +29.0525 +27.4258 +27.4931 +27.8928 +26.9308 +28.4833)/ 16
> > > | 27.8187
> > > 
> > > This 28.… is the number of GP/s. Based on the results in looks like
> > > noise to me. Also I have no idea why you have more than twice as many
> > > GP/s as I do.
> > 
> > My guess is that because you have more CPUs, the for_each_online_cpu()
> > loop takes longer on your system.
> 
> OK, that is rather oversimplified, to say the least.  A better way to
> put this is that the probability of some CPU holding things up is larger
> the more CPUs you have.  RCU doe take explicit steps to slow down grace
> periods, but that doesn't start kicking in until 256 CPUs.

And I ended up with the following variant of your patch.  If I don't
hear otherwise from you, I will assume that you are OK with it.  So if
something bothers you about it, please don't suffer in silence!

							Thanx, Paul

------------------------------------------------------------------------

commit 4a04229cf73ac9bc1ae15357beb32a0b37be1480
Author: Sebastian Andrzej Siewior <bigeasy at linutronix.de>
Date:   Wed Apr 24 09:34:46 2019 +0200

    rcutorture: Tweak kvm options
    
    In one of my rcutorture tests the TSC clocksource got marked unstable
    due to a large difference in the TSC value. I'm not sure if the guest
    run for a long time with disabled interrupts or if the host was very
    busy and didn't schedule the guest for some time.
    
    I took a look on the qemu/KVM options and decided to update the options:
    
    - Use kvm{32|64} as CPU. We could probably use `host' (like ARM does)
      for maximum available features but since we don't run any userland I'm
      not sure if it makes any difference.
    
    - Drop the "noapic" option. There is no history why the APIC was disabled,
      I see no reason for it.  Once old qemu versions fade away, we can add
      "x2apic=on,tsc-deadline=on,hypervisor=on,tsc_adjust=on".
    
    - Additional config options. It ensures that the kernel knowns that it
      runs as a kvm guest and can use virt devices like the kvm-clock as
      clocksource. The kvm-clock was the main motivation here.
    
    - I didn't add a random HW device. It would make the random device ready
      earlier (not it doesn't complete the initialisation at all) but I
      doubt that there is any need for this.
    
    Signed-off-by: Sebastian Andrzej Siewior <bigeasy at linutronix.de>
    [ paulmck: The world is not quite ready for CONFIG_PARAVIRT_SPINLOCKS=y
      and x2apic, so they are omitted for the time being. ]
    Signed-off-by: Paul E. McKenney <paulmck at linux.ibm.com>

diff --git a/tools/testing/selftests/rcutorture/bin/functions.sh b/tools/testing/selftests/rcutorture/bin/functions.sh
index 6bcb8b5b2ff2..c3a49fb4d6f6 100644
--- a/tools/testing/selftests/rcutorture/bin/functions.sh
+++ b/tools/testing/selftests/rcutorture/bin/functions.sh
@@ -172,7 +172,7 @@ identify_qemu_append () {
 	local console=ttyS0
 	case "$1" in
 	qemu-system-x86_64|qemu-system-i386)
-		echo noapic selinux=0 initcall_debug debug
+		echo selinux=0 initcall_debug debug
 		;;
 	qemu-system-aarch64)
 		console=ttyAMA0
@@ -191,8 +191,19 @@ identify_qemu_append () {
 # Output arguments for qemu arguments based on the TORTURE_QEMU_MAC
 # and TORTURE_QEMU_INTERACTIVE environment variables.
 identify_qemu_args () {
+	local KVM_CPU=""
+	case "$1" in
+	qemu-system-x86_64)
+		KVM_CPU=kvm64
+		;;
+	qemu-system-i386)
+		KVM_CPU=kvm32
+		;;
+	esac
 	case "$1" in
 	qemu-system-x86_64|qemu-system-i386)
+		echo -machine q35,accel=kvm
+		echo -cpu ${KVM_CPU}
 		;;
 	qemu-system-aarch64)
 		echo -machine virt,gic-version=host -cpu host
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/CFcommon b/tools/testing/selftests/rcutorture/configs/rcu/CFcommon
index d2d2a86139db..e19a444a0684 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/CFcommon
+++ b/tools/testing/selftests/rcutorture/configs/rcu/CFcommon
@@ -1,2 +1,5 @@
 CONFIG_RCU_TORTURE_TEST=y
 CONFIG_PRINTK_TIME=y
+CONFIG_HYPERVISOR_GUEST=y
+CONFIG_PARAVIRT=y
+CONFIG_KVM_GUEST=y