Re: [PATCH v12 3/3] ipc: Do cyclic id allocation with ipcmni_extend mode

From: Waiman Long <longman@redhat.com>
To: Manfred Spraul <manfred@colorfullife.com>,
	"Luis R. Rodriguez" <mcgrof@kernel.org>,
	Kees Cook <keescook@chromium.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Jonathan Corbet <corbet@lwn.net>
Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
	linux-doc@vger.kernel.org, Al Viro <viro@zeniv.linux.org.uk>,
	Matthew Wilcox <willy@infradead.org>,
	"Eric W. Biederman" <ebiederm@xmission.com>,
	Takashi Iwai <tiwai@suse.de>, Davidlohr Bueso <dbueso@suse.de>
Subject: Re: [PATCH v12 3/3] ipc: Do cyclic id allocation with ipcmni_extend mode
Date: Mon, 18 Mar 2019 14:53:11 -0400	[thread overview]
Message-ID: <35482058-12e3-b528-fc97-50c7f011dd15@redhat.com> (raw)
In-Reply-To: <eaee3fad-937b-9776-5ff4-874475acc162@redhat.com>

On 03/18/2019 02:37 PM, Waiman Long wrote:
> On 03/17/2019 02:27 PM, Manfred Spraul wrote:
>> Hi Waiman,
>>
>> On 2/28/19 7:47 PM, Waiman Long wrote:
>>> For ipcmni_extend mode, the sequence number space is only 7 bits. So
>>> the chance of id reuse is relatively high compared with the non-extended
>>> mode.
>>>
>>> To alleviate this id reuse problem, the id allocation will be done
>>> cyclically to cycle through all the 24-bit id space before wrapping
>>> around when in ipcmni_extend mode. This may cause the use of more memory
>>> in term of the number of xa_nodes allocated as well as potentially more
>>> cachelines used as the xa_nodes may be spread more sparsely in this
>>> case.
>>>
>>> There is probably a slight memory and performance cost in doing cyclic
>>> id allocation. For applications that really need more than 32k unique
>>> IPC
>>> identifiers, this is a small price to pay to avoid the id reuse problem.
>> Have you measured it?
>>
>> I have observed -3% for semop() for a 4 level radix tree compared to a
>> 1-level radix tree, and I'm a bit reluctant to accept that.
>> Especially as the percentage will increase if the syscall overhead
>> goes down again (-> less spectre impact).
>>
> It is both Spectre (retpoline) and Meltdown (PTI). PTI is not needed in
> AMD CPU and so you may see a bit higher slowdown.

The use of idr_replace() in your previous patch may also slow the code
path a bit to reduce the performance difference that you saw. This is
actually my main concern with using idr_replace() as suggested by
Matthew, but I am OK to use it if people think absolute correctness is
more important.

Cheers,
Longman