Re: seccomp ptrace selftest failures with 4.4-stable [Was: Re: LTS testing with latest kselftests - some failures]

From: Andy Lutomirski <luto@kernel.org>
To: Shuah Khan <shuah@kernel.org>
Cc: Kees Cook <keescook@chromium.org>,
	Sumit Semwal <sumit.semwal@linaro.org>,
	Brian Norris <computersforpeace@gmail.com>,
	"Luis R. Rodriguez" <mcgrof@kernel.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	LKML <linux-kernel@vger.kernel.org>,
	"# 3.4.x" <stable@vger.kernel.org>,
	"open list:KERNEL SELFTEST FRAMEWORK" 
	<linux-kselftest@vger.kernel.org>,
	Shuah Khan <shuahkh@osg.samsung.com>
Subject: Re: seccomp ptrace selftest failures with 4.4-stable [Was: Re: LTS testing with latest kselftests - some failures]
Date: Thu, 22 Jun 2017 10:49:07 -0700	[thread overview]
Message-ID: <CALCETrVa0RgBcsb1_GmobjG1prFNFs0F8krHdbmiKEyhua7xkw@mail.gmail.com> (raw)
In-Reply-To: <334e6a92-2d41-c9e1-c807-19e493f1af83@kernel.org>

On Thu, Jun 22, 2017 at 10:09 AM, Shuah Khan <shuah@kernel.org> wrote:
> On 06/22/2017 10:53 AM, Kees Cook wrote:
>> On Thu, Jun 22, 2017 at 9:18 AM, Sumit Semwal <sumit.semwal@linaro.org> wrote:
>>> Hi Kees, Andy,
>>>
>>> On 15 June 2017 at 23:26, Sumit Semwal <sumit.semwal@linaro.org> wrote:
>>>> 3. 'seccomp ptrace hole closure' patches got added in 4.7 [3] -
>>>> feature and test together.
>>>> - This one also seems like a security hole being closed, and the
>>>> 'feature' could be a candidate for stable backports, but Arnd tried
>>>> that, and it was quite non-trivial. So perhaps  we'll need some help
>>>> from the subsystem developers here.
>>>
>>> Could you please help us sort this out? Our goal is to help Greg with
>>> testing stable kernels, and currently the seccomp tests fail due to
>>> missing feature (seccomp ptrace hole closure) getting tested via
>>> latest kselftest.
>>>
>>> If you feel the feature isn't a stable candidate, then could you
>>> please help make the test degrade gracefully in its absence?
>>
>> I don't really want to have that change be a backport -- it's quite
>> invasive across multiple architectures.
>>
>> I would say just add a kernel version check to the test. This is
>> probably not the only selftest that will need such things. :)
>
> Adding release checks to selftests is going to problematic for maintenance.
> Tests should fail gracefully if feature isn't supported in older kernels.
>
> Several tests do that now and please find a way to check for dependencies
> and feature availability and fail the test gracefully. If there is a test
> that can't do that for some reason, we can discuss it, but as a general
> rule, I don't want to see kselftest patches that check release.

If a future kernel inadvertently loses the new feature and degrades to
the behavior of old kernels, that would be a serious bug and should be
caught.

--Andy