Re: [PATCH resend] x86,kvm: Add a kernel parameter to disable PV spinlock

From: Juergen Gross <jgross@suse.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Davidlohr Bueso <dave@stgolabs.net>,
	Oscar Salvador <osalvador@suse.de>,
	Ingo Molnar <mingo@kernel.org>,
	Paolo Bonzini <pbonzini@redhat.com>,
	"H . Peter Anvin" <hpa@zytor.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
	linux-doc@vger.kernel.org, Waiman Long <longman@redhat.com>
Subject: Re: [PATCH resend] x86,kvm: Add a kernel parameter to disable PV spinlock
Date: Tue, 5 Sep 2017 09:35:40 +0200	[thread overview]
Message-ID: <924fec17-548a-083d-edce-7adcb662c513@suse.com> (raw)
In-Reply-To: <20170905065837.rs767a4os2aumg7h@hirez.programming.kicks-ass.net>

On 05/09/17 08:58, Peter Zijlstra wrote:
> On Tue, Sep 05, 2017 at 08:28:10AM +0200, Juergen Gross wrote:
>> On 05/09/17 00:21, Davidlohr Bueso wrote:
>>> On Mon, 04 Sep 2017, Peter Zijlstra wrote:
>>>
>>>> For testing its trivial to hack your kernel and I don't feel this is
>>>> something an Admin can make reasonable decisions about.
>>>>
>>>> So why? In general less knobs is better.
>>>
>>> +1.
>>>
>>> Also, note how b8fa70b51aa (xen, pvticketlocks: Add xen_nopvspin parameter
>>> to disable xen pv ticketlocks) has no justification as to why its wanted
>>> in the first place. The only thing I could find was from 15a3eac0784
>>> (xen/spinlock: Document the xen_nopvspin parameter):
>>>
>>> "Useful for diagnosing issues and comparing benchmarks in over-commit
>>> CPU scenarios."
>>
>> Hmm, I think I should clarify the Xen knob, as I was the one requesting
>> it:
>>
>> In my previous employment we had a configuration where dom0 ran
>> exclusively on a dedicated set of physical cpus. We experienced
>> scalability problems when doing I/O performance tests: with a decent
>> number of dom0 cpus we achieved throughput of 700 MB/s with only 20%
>> cpu load in dom0. A higher dom0 cpu count let the throughput drop to
>> about 150 MB/s and cpu load was up to 100%. Reason was the additional
>> load due to hypervisor interactions on a high frequency lock.
>>
>> So in special configurations at least for Xen the knob is useful for
>> production environment.
> 
> So the problem with qspinlock is that it will revert to a classic
> test-and-set spinlock if you don't do paravirt but are running a HV.

In the Xen case we just use the bare metal settings when xen_nopvspin
has been specified. So paravirt, but without modifying any pv_lock_ops
functions.

Juergen

> 
> And test-and-set is unfair and has all kinds of ugly starvation cases,
> esp on slightly bigger hardware.
> 
> So if we'd want to cater to the 1:1 virt case, we'll need to come up
> with something else. _IF_ it is an issue of course.
> 