From: paulmck at linux.ibm.com (Paul E. McKenney)
Subject: [PATCH] rcutorture: Tweak kvm options
Date: Thu, 2 May 2019 11:41:16 -0700 [thread overview]
Message-ID: <20190502184116.GA8811@linux.ibm.com> (raw)
In-Reply-To: <20190429150600.GA14389@linux.ibm.com>
On Mon, Apr 29, 2019 at 08:06:00AM -0700, Paul E. McKenney wrote:
> On Mon, Apr 29, 2019 at 07:49:24AM -0700, Paul E. McKenney wrote:
> > On Mon, Apr 29, 2019 at 10:19:44AM +0200, Sebastian Andrzej Siewior wrote:
> > > On 2019-04-26 06:50:58 [-0700], Paul E. McKenney wrote:
> > > > One place to look is in the summary output:
> > > >
> > > > TREE01 ------- 17540 GPs (58.4667/s) [rcu: g130629 f0x0 ]
> > > >
> > > > The "58.4667/s" is the number of grace periods per second. I would be
> > > > surprised if CONFIG_PARAVIRT_SPINLOCKS made a noticeable difference in
> > > > grace-period rate (given the natural variation), but you never know.
> > >
> > > I did four runs of the different parts of the patch:
> > > - 5.1-rc6
> > > - #1 + kvm64 CPU + some config options
> > > - #2 + tsc-deadline=on and so on (the whole line)
> > > - #3 + CONFIG_PARAVIRT_SPINLOCKS (now everything)
> > >
> > > the test command was
> > > tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 112 --duration 60 --configs "16*TREE08" --memory 4G
> > >
> > > and the results:
> > > | HEAD is now at 085b7755808a... Linux 5.1-rc6
> > > | (28.5942 +27.4658 +28.0203 +27.2061 +28.0731 +26.9078 +27.8494 + 27.3392 +26.4339 +28.025 +27.4797 +27.6775 +28.0653 +28.0742 +27.9581 +28.6508)/ 16
> > > | 27.738775
> > > |
> > > | HEAD is now at 36a12aa9761a... tune #1
> > > | (28.5761 +26.6514 +26.6989 +27.4375 +27.3442 +28.3228 +26.6353 +27.5461+28.5531 +27.7006 +27.8078 +27.9753 +27.4269 +28.0464 +27.6314 +27.8356) / 16
> > > | 27.6368375
> > > |
> > > | HEAD is now at af5cd7196436... tune #2
> > > | (28.4867 +26.3675 +27.6364 +28.3344 +27.4153 +27.9306 +27.1703 +26.8461+27.3194 +28.5486 +27.8975 +27.4356 +28.12 +28.4397 +29.0186 +26.9328 )/ 16
> > > | 27.74371875
> > > |
> > > | HEAD is now at 3701f64943f5... tune #3
> > > | (28.2431 +27.7831 +28.39 +28.2586 +27.7408 +27.9258 +26.6236 +26.7817+29.1178 +26.9564 +29.0525 +27.4258 +27.4931 +27.8928 +26.9308 +28.4833)/ 16
> > > | 27.8187
> > >
> > > This 28.… is the number of GP/s. Based on the results in looks like
> > > noise to me. Also I have no idea why you have more than twice as many
> > > GP/s as I do.
> >
> > My guess is that because you have more CPUs, the for_each_online_cpu()
> > loop takes longer on your system.
>
> OK, that is rather oversimplified, to say the least. A better way to
> put this is that the probability of some CPU holding things up is larger
> the more CPUs you have. RCU doe take explicit steps to slow down grace
> periods, but that doesn't start kicking in until 256 CPUs.
And I ended up with the following variant of your patch. If I don't
hear otherwise from you, I will assume that you are OK with it. So if
something bothers you about it, please don't suffer in silence!
Thanx, Paul
------------------------------------------------------------------------
commit 4a04229cf73ac9bc1ae15357beb32a0b37be1480
Author: Sebastian Andrzej Siewior <bigeasy at linutronix.de>
Date: Wed Apr 24 09:34:46 2019 +0200
rcutorture: Tweak kvm options
In one of my rcutorture tests the TSC clocksource got marked unstable
due to a large difference in the TSC value. I'm not sure if the guest
run for a long time with disabled interrupts or if the host was very
busy and didn't schedule the guest for some time.
I took a look on the qemu/KVM options and decided to update the options:
- Use kvm{32|64} as CPU. We could probably use `host' (like ARM does)
for maximum available features but since we don't run any userland I'm
not sure if it makes any difference.
- Drop the "noapic" option. There is no history why the APIC was disabled,
I see no reason for it. Once old qemu versions fade away, we can add
"x2apic=on,tsc-deadline=on,hypervisor=on,tsc_adjust=on".
- Additional config options. It ensures that the kernel knowns that it
runs as a kvm guest and can use virt devices like the kvm-clock as
clocksource. The kvm-clock was the main motivation here.
- I didn't add a random HW device. It would make the random device ready
earlier (not it doesn't complete the initialisation at all) but I
doubt that there is any need for this.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy at linutronix.de>
[ paulmck: The world is not quite ready for CONFIG_PARAVIRT_SPINLOCKS=y
and x2apic, so they are omitted for the time being. ]
Signed-off-by: Paul E. McKenney <paulmck at linux.ibm.com>
diff --git a/tools/testing/selftests/rcutorture/bin/functions.sh b/tools/testing/selftests/rcutorture/bin/functions.sh
index 6bcb8b5b2ff2..c3a49fb4d6f6 100644
--- a/tools/testing/selftests/rcutorture/bin/functions.sh
+++ b/tools/testing/selftests/rcutorture/bin/functions.sh
@@ -172,7 +172,7 @@ identify_qemu_append () {
local console=ttyS0
case "$1" in
qemu-system-x86_64|qemu-system-i386)
- echo noapic selinux=0 initcall_debug debug
+ echo selinux=0 initcall_debug debug
;;
qemu-system-aarch64)
console=ttyAMA0
@@ -191,8 +191,19 @@ identify_qemu_append () {
# Output arguments for qemu arguments based on the TORTURE_QEMU_MAC
# and TORTURE_QEMU_INTERACTIVE environment variables.
identify_qemu_args () {
+ local KVM_CPU=""
+ case "$1" in
+ qemu-system-x86_64)
+ KVM_CPU=kvm64
+ ;;
+ qemu-system-i386)
+ KVM_CPU=kvm32
+ ;;
+ esac
case "$1" in
qemu-system-x86_64|qemu-system-i386)
+ echo -machine q35,accel=kvm
+ echo -cpu ${KVM_CPU}
;;
qemu-system-aarch64)
echo -machine virt,gic-version=host -cpu host
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/CFcommon b/tools/testing/selftests/rcutorture/configs/rcu/CFcommon
index d2d2a86139db..e19a444a0684 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/CFcommon
+++ b/tools/testing/selftests/rcutorture/configs/rcu/CFcommon
@@ -1,2 +1,5 @@
CONFIG_RCU_TORTURE_TEST=y
CONFIG_PRINTK_TIME=y
+CONFIG_HYPERVISOR_GUEST=y
+CONFIG_PARAVIRT=y
+CONFIG_KVM_GUEST=y
WARNING: multiple messages have this Message-ID (diff)
From: paulmck@linux.ibm.com (Paul E. McKenney)
Subject: [PATCH] rcutorture: Tweak kvm options
Date: Thu, 2 May 2019 11:41:16 -0700 [thread overview]
Message-ID: <20190502184116.GA8811@linux.ibm.com> (raw)
Message-ID: <20190502184116.I8vj6bKWGr66xCopYeUVq48WDDE6pg74GWVgL6uV4m0@z> (raw)
In-Reply-To: <20190429150600.GA14389@linux.ibm.com>
On Mon, Apr 29, 2019@08:06:00AM -0700, Paul E. McKenney wrote:
> On Mon, Apr 29, 2019@07:49:24AM -0700, Paul E. McKenney wrote:
> > On Mon, Apr 29, 2019@10:19:44AM +0200, Sebastian Andrzej Siewior wrote:
> > > On 2019-04-26 06:50:58 [-0700], Paul E. McKenney wrote:
> > > > One place to look is in the summary output:
> > > >
> > > > TREE01 ------- 17540 GPs (58.4667/s) [rcu: g130629 f0x0 ]
> > > >
> > > > The "58.4667/s" is the number of grace periods per second. I would be
> > > > surprised if CONFIG_PARAVIRT_SPINLOCKS made a noticeable difference in
> > > > grace-period rate (given the natural variation), but you never know.
> > >
> > > I did four runs of the different parts of the patch:
> > > - 5.1-rc6
> > > - #1 + kvm64 CPU + some config options
> > > - #2 + tsc-deadline=on and so on (the whole line)
> > > - #3 + CONFIG_PARAVIRT_SPINLOCKS (now everything)
> > >
> > > the test command was
> > > tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 112 --duration 60 --configs "16*TREE08" --memory 4G
> > >
> > > and the results:
> > > | HEAD is now at 085b7755808a... Linux 5.1-rc6
> > > | (28.5942 +27.4658 +28.0203 +27.2061 +28.0731 +26.9078 +27.8494 + 27.3392 +26.4339 +28.025 +27.4797 +27.6775 +28.0653 +28.0742 +27.9581 +28.6508)/ 16
> > > | 27.738775
> > > |
> > > | HEAD is now at 36a12aa9761a... tune #1
> > > | (28.5761 +26.6514 +26.6989 +27.4375 +27.3442 +28.3228 +26.6353 +27.5461+28.5531 +27.7006 +27.8078 +27.9753 +27.4269 +28.0464 +27.6314 +27.8356) / 16
> > > | 27.6368375
> > > |
> > > | HEAD is now at af5cd7196436... tune #2
> > > | (28.4867 +26.3675 +27.6364 +28.3344 +27.4153 +27.9306 +27.1703 +26.8461+27.3194 +28.5486 +27.8975 +27.4356 +28.12 +28.4397 +29.0186 +26.9328 )/ 16
> > > | 27.74371875
> > > |
> > > | HEAD is now at 3701f64943f5... tune #3
> > > | (28.2431 +27.7831 +28.39 +28.2586 +27.7408 +27.9258 +26.6236 +26.7817+29.1178 +26.9564 +29.0525 +27.4258 +27.4931 +27.8928 +26.9308 +28.4833)/ 16
> > > | 27.8187
> > >
> > > This 28.… is the number of GP/s. Based on the results in looks like
> > > noise to me. Also I have no idea why you have more than twice as many
> > > GP/s as I do.
> >
> > My guess is that because you have more CPUs, the for_each_online_cpu()
> > loop takes longer on your system.
>
> OK, that is rather oversimplified, to say the least. A better way to
> put this is that the probability of some CPU holding things up is larger
> the more CPUs you have. RCU doe take explicit steps to slow down grace
> periods, but that doesn't start kicking in until 256 CPUs.
And I ended up with the following variant of your patch. If I don't
hear otherwise from you, I will assume that you are OK with it. So if
something bothers you about it, please don't suffer in silence!
Thanx, Paul
------------------------------------------------------------------------
commit 4a04229cf73ac9bc1ae15357beb32a0b37be1480
Author: Sebastian Andrzej Siewior <bigeasy at linutronix.de>
Date: Wed Apr 24 09:34:46 2019 +0200
rcutorture: Tweak kvm options
In one of my rcutorture tests the TSC clocksource got marked unstable
due to a large difference in the TSC value. I'm not sure if the guest
run for a long time with disabled interrupts or if the host was very
busy and didn't schedule the guest for some time.
I took a look on the qemu/KVM options and decided to update the options:
- Use kvm{32|64} as CPU. We could probably use `host' (like ARM does)
for maximum available features but since we don't run any userland I'm
not sure if it makes any difference.
- Drop the "noapic" option. There is no history why the APIC was disabled,
I see no reason for it. Once old qemu versions fade away, we can add
"x2apic=on,tsc-deadline=on,hypervisor=on,tsc_adjust=on".
- Additional config options. It ensures that the kernel knowns that it
runs as a kvm guest and can use virt devices like the kvm-clock as
clocksource. The kvm-clock was the main motivation here.
- I didn't add a random HW device. It would make the random device ready
earlier (not it doesn't complete the initialisation at all) but I
doubt that there is any need for this.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy at linutronix.de>
[ paulmck: The world is not quite ready for CONFIG_PARAVIRT_SPINLOCKS=y
and x2apic, so they are omitted for the time being. ]
Signed-off-by: Paul E. McKenney <paulmck at linux.ibm.com>
diff --git a/tools/testing/selftests/rcutorture/bin/functions.sh b/tools/testing/selftests/rcutorture/bin/functions.sh
index 6bcb8b5b2ff2..c3a49fb4d6f6 100644
--- a/tools/testing/selftests/rcutorture/bin/functions.sh
+++ b/tools/testing/selftests/rcutorture/bin/functions.sh
@@ -172,7 +172,7 @@ identify_qemu_append () {
local console=ttyS0
case "$1" in
qemu-system-x86_64|qemu-system-i386)
- echo noapic selinux=0 initcall_debug debug
+ echo selinux=0 initcall_debug debug
;;
qemu-system-aarch64)
console=ttyAMA0
@@ -191,8 +191,19 @@ identify_qemu_append () {
# Output arguments for qemu arguments based on the TORTURE_QEMU_MAC
# and TORTURE_QEMU_INTERACTIVE environment variables.
identify_qemu_args () {
+ local KVM_CPU=""
+ case "$1" in
+ qemu-system-x86_64)
+ KVM_CPU=kvm64
+ ;;
+ qemu-system-i386)
+ KVM_CPU=kvm32
+ ;;
+ esac
case "$1" in
qemu-system-x86_64|qemu-system-i386)
+ echo -machine q35,accel=kvm
+ echo -cpu ${KVM_CPU}
;;
qemu-system-aarch64)
echo -machine virt,gic-version=host -cpu host
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/CFcommon b/tools/testing/selftests/rcutorture/configs/rcu/CFcommon
index d2d2a86139db..e19a444a0684 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/CFcommon
+++ b/tools/testing/selftests/rcutorture/configs/rcu/CFcommon
@@ -1,2 +1,5 @@
CONFIG_RCU_TORTURE_TEST=y
CONFIG_PRINTK_TIME=y
+CONFIG_HYPERVISOR_GUEST=y
+CONFIG_PARAVIRT=y
+CONFIG_KVM_GUEST=y
WARNING: multiple messages have this Message-ID (diff)
From: "Paul E. McKenney" <paulmck@linux.ibm.com>
To: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: linux-kselftest@vger.kernel.org, rcu@vger.kernel.org,
Josh Triplett <josh@joshtriplett.org>,
Steven Rostedt <rostedt@goodmis.org>,
Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
Lai Jiangshan <jiangshanlai@gmail.com>,
Shuah Khan <shuah@kernel.org>
Subject: Re: [PATCH] rcutorture: Tweak kvm options
Date: Thu, 2 May 2019 11:41:16 -0700 [thread overview]
Message-ID: <20190502184116.GA8811@linux.ibm.com> (raw)
In-Reply-To: <20190429150600.GA14389@linux.ibm.com>
On Mon, Apr 29, 2019 at 08:06:00AM -0700, Paul E. McKenney wrote:
> On Mon, Apr 29, 2019 at 07:49:24AM -0700, Paul E. McKenney wrote:
> > On Mon, Apr 29, 2019 at 10:19:44AM +0200, Sebastian Andrzej Siewior wrote:
> > > On 2019-04-26 06:50:58 [-0700], Paul E. McKenney wrote:
> > > > One place to look is in the summary output:
> > > >
> > > > TREE01 ------- 17540 GPs (58.4667/s) [rcu: g130629 f0x0 ]
> > > >
> > > > The "58.4667/s" is the number of grace periods per second. I would be
> > > > surprised if CONFIG_PARAVIRT_SPINLOCKS made a noticeable difference in
> > > > grace-period rate (given the natural variation), but you never know.
> > >
> > > I did four runs of the different parts of the patch:
> > > - 5.1-rc6
> > > - #1 + kvm64 CPU + some config options
> > > - #2 + tsc-deadline=on and so on (the whole line)
> > > - #3 + CONFIG_PARAVIRT_SPINLOCKS (now everything)
> > >
> > > the test command was
> > > tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 112 --duration 60 --configs "16*TREE08" --memory 4G
> > >
> > > and the results:
> > > | HEAD is now at 085b7755808a... Linux 5.1-rc6
> > > | (28.5942 +27.4658 +28.0203 +27.2061 +28.0731 +26.9078 +27.8494 + 27.3392 +26.4339 +28.025 +27.4797 +27.6775 +28.0653 +28.0742 +27.9581 +28.6508)/ 16
> > > | 27.738775
> > > |
> > > | HEAD is now at 36a12aa9761a... tune #1
> > > | (28.5761 +26.6514 +26.6989 +27.4375 +27.3442 +28.3228 +26.6353 +27.5461+28.5531 +27.7006 +27.8078 +27.9753 +27.4269 +28.0464 +27.6314 +27.8356) / 16
> > > | 27.6368375
> > > |
> > > | HEAD is now at af5cd7196436... tune #2
> > > | (28.4867 +26.3675 +27.6364 +28.3344 +27.4153 +27.9306 +27.1703 +26.8461+27.3194 +28.5486 +27.8975 +27.4356 +28.12 +28.4397 +29.0186 +26.9328 )/ 16
> > > | 27.74371875
> > > |
> > > | HEAD is now at 3701f64943f5... tune #3
> > > | (28.2431 +27.7831 +28.39 +28.2586 +27.7408 +27.9258 +26.6236 +26.7817+29.1178 +26.9564 +29.0525 +27.4258 +27.4931 +27.8928 +26.9308 +28.4833)/ 16
> > > | 27.8187
> > >
> > > This 28.… is the number of GP/s. Based on the results in looks like
> > > noise to me. Also I have no idea why you have more than twice as many
> > > GP/s as I do.
> >
> > My guess is that because you have more CPUs, the for_each_online_cpu()
> > loop takes longer on your system.
>
> OK, that is rather oversimplified, to say the least. A better way to
> put this is that the probability of some CPU holding things up is larger
> the more CPUs you have. RCU doe take explicit steps to slow down grace
> periods, but that doesn't start kicking in until 256 CPUs.
And I ended up with the following variant of your patch. If I don't
hear otherwise from you, I will assume that you are OK with it. So if
something bothers you about it, please don't suffer in silence!
Thanx, Paul
------------------------------------------------------------------------
commit 4a04229cf73ac9bc1ae15357beb32a0b37be1480
Author: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Date: Wed Apr 24 09:34:46 2019 +0200
rcutorture: Tweak kvm options
In one of my rcutorture tests the TSC clocksource got marked unstable
due to a large difference in the TSC value. I'm not sure if the guest
run for a long time with disabled interrupts or if the host was very
busy and didn't schedule the guest for some time.
I took a look on the qemu/KVM options and decided to update the options:
- Use kvm{32|64} as CPU. We could probably use `host' (like ARM does)
for maximum available features but since we don't run any userland I'm
not sure if it makes any difference.
- Drop the "noapic" option. There is no history why the APIC was disabled,
I see no reason for it. Once old qemu versions fade away, we can add
"x2apic=on,tsc-deadline=on,hypervisor=on,tsc_adjust=on".
- Additional config options. It ensures that the kernel knowns that it
runs as a kvm guest and can use virt devices like the kvm-clock as
clocksource. The kvm-clock was the main motivation here.
- I didn't add a random HW device. It would make the random device ready
earlier (not it doesn't complete the initialisation at all) but I
doubt that there is any need for this.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
[ paulmck: The world is not quite ready for CONFIG_PARAVIRT_SPINLOCKS=y
and x2apic, so they are omitted for the time being. ]
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
diff --git a/tools/testing/selftests/rcutorture/bin/functions.sh b/tools/testing/selftests/rcutorture/bin/functions.sh
index 6bcb8b5b2ff2..c3a49fb4d6f6 100644
--- a/tools/testing/selftests/rcutorture/bin/functions.sh
+++ b/tools/testing/selftests/rcutorture/bin/functions.sh
@@ -172,7 +172,7 @@ identify_qemu_append () {
local console=ttyS0
case "$1" in
qemu-system-x86_64|qemu-system-i386)
- echo noapic selinux=0 initcall_debug debug
+ echo selinux=0 initcall_debug debug
;;
qemu-system-aarch64)
console=ttyAMA0
@@ -191,8 +191,19 @@ identify_qemu_append () {
# Output arguments for qemu arguments based on the TORTURE_QEMU_MAC
# and TORTURE_QEMU_INTERACTIVE environment variables.
identify_qemu_args () {
+ local KVM_CPU=""
+ case "$1" in
+ qemu-system-x86_64)
+ KVM_CPU=kvm64
+ ;;
+ qemu-system-i386)
+ KVM_CPU=kvm32
+ ;;
+ esac
case "$1" in
qemu-system-x86_64|qemu-system-i386)
+ echo -machine q35,accel=kvm
+ echo -cpu ${KVM_CPU}
;;
qemu-system-aarch64)
echo -machine virt,gic-version=host -cpu host
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/CFcommon b/tools/testing/selftests/rcutorture/configs/rcu/CFcommon
index d2d2a86139db..e19a444a0684 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/CFcommon
+++ b/tools/testing/selftests/rcutorture/configs/rcu/CFcommon
@@ -1,2 +1,5 @@
CONFIG_RCU_TORTURE_TEST=y
CONFIG_PRINTK_TIME=y
+CONFIG_HYPERVISOR_GUEST=y
+CONFIG_PARAVIRT=y
+CONFIG_KVM_GUEST=y
next prev parent reply other threads:[~2019-05-02 18:41 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-04-24 7:34 [PATCH] rcutorture: Tweak kvm options bigeasy
2019-04-24 7:34 ` Sebastian Andrzej Siewior
2019-04-24 7:34 ` Sebastian Andrzej Siewior
2019-04-24 10:38 ` paulmck
2019-04-24 10:38 ` Paul E. McKenney
2019-04-24 10:38 ` Paul E. McKenney
2019-04-24 18:30 ` paulmck
2019-04-24 18:30 ` Paul E. McKenney
2019-04-24 18:30 ` Paul E. McKenney
2019-04-25 19:46 ` paulmck
2019-04-25 19:46 ` Paul E. McKenney
2019-04-25 19:46 ` Paul E. McKenney
2019-04-26 10:54 ` bigeasy
2019-04-26 10:54 ` Sebastian Andrzej Siewior
2019-04-26 10:54 ` Sebastian Andrzej Siewior
2019-04-26 13:50 ` paulmck
2019-04-26 13:50 ` Paul E. McKenney
2019-04-26 13:50 ` Paul E. McKenney
2019-04-29 8:19 ` bigeasy
2019-04-29 8:19 ` Sebastian Andrzej Siewior
2019-04-29 8:19 ` Sebastian Andrzej Siewior
2019-04-29 14:49 ` paulmck
2019-04-29 14:49 ` Paul E. McKenney
2019-04-29 14:49 ` Paul E. McKenney
2019-04-29 15:06 ` paulmck
2019-04-29 15:06 ` Paul E. McKenney
2019-04-29 15:06 ` Paul E. McKenney
2019-05-02 18:41 ` paulmck [this message]
2019-05-02 18:41 ` Paul E. McKenney
2019-05-02 18:41 ` Paul E. McKenney
2019-05-03 7:26 ` bigeasy
2019-05-03 7:26 ` Sebastian Andrzej Siewior
2019-05-03 7:26 ` Sebastian Andrzej Siewior
2019-04-25 16:45 ` joel
2019-04-25 16:45 ` Joel Fernandes
2019-04-25 16:45 ` Joel Fernandes
2019-04-25 17:13 ` bigeasy
2019-04-25 17:13 ` Sebastian Andrzej Siewior
2019-04-25 17:13 ` Sebastian Andrzej Siewior
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190502184116.GA8811@linux.ibm.com \
--to=unknown@example.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.