Re: 4.19: udpgso_bench_tx: setsockopt zerocopy: Unknown error 524

From: Willem de Bruijn <willemdebruijn.kernel@gmail.com>
To: Greg KH <gregkh@linuxfoundation.org>
Cc: David Miller <davem@davemloft.net>,
	Naresh Kamboju <naresh.kamboju@linaro.org>,
	Network Development <netdev@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	"open list:KERNEL SELFTEST FRAMEWORK" 
	<linux-kselftest@vger.kernel.org>,
	Fred Klassen <fklassen@appneta.com>
Subject: Re: 4.19: udpgso_bench_tx: setsockopt zerocopy: Unknown error 524
Date: Tue, 18 Jun 2019 16:04:42 -0400	[thread overview]
Message-ID: <CAF=yD-L8ZRwVJmp4WJcNW-B_1JdSAM9QmMHOQJ=x_nd24v5Qnw@mail.gmail.com> (raw)
In-Reply-To: <CA+FuTSdrphico4044QTD_-8VbanFFJx0FJuH+vVMfuHqbphkjw@mail.gmail.com>

On Tue, Jun 18, 2019 at 2:59 PM Willem de Bruijn
<willemdebruijn.kernel@gmail.com> wrote:
>
> On Tue, Jun 18, 2019 at 1:39 PM Greg KH <gregkh@linuxfoundation.org> wrote:
> >
> > On Tue, Jun 18, 2019 at 01:27:14PM -0400, Willem de Bruijn wrote:
> > > On Tue, Jun 18, 2019 at 1:15 PM Greg KH <gregkh@linuxfoundation.org> wrote:
> > > >
> > > > On Tue, Jun 18, 2019 at 09:47:59AM -0700, David Miller wrote:
> > > > > From: Willem de Bruijn <willemdebruijn.kernel@gmail.com>
> > > > > Date: Tue, 18 Jun 2019 12:37:33 -0400
> > > > >
> > > > > > Specific to the above test, I can add a check command testing
> > > > > > setsockopt SO_ZEROCOPY  return value. AFAIK kselftest has no explicit
> > > > > > way to denote "skipped", so this would just return "pass". Sounds a
> > > > > > bit fragile, passing success when a feature is absent.
> > > > >
> > > > > Especially since the feature might be absent because the 'config'
> > > > > template forgot to include a necessary Kconfig option.
> > > >
> > > > That is what the "skip" response is for, don't return "pass" if the
> > > > feature just isn't present.  That lets people run tests on systems
> > > > without the config option enabled as you say, or on systems without the
> > > > needed userspace tools present.
> > >
> > > I was not aware that kselftest had this feature.
> > >
> > > But it appears that exit code KSFT_SKIP (4) will achieve this. Okay,
> > > I'll send a patch and will keep that in mind for future tests.
> >
> > Wonderful, thanks for doing that!
>
> One complication: an exit code works for a single test, but here
> multiple test variants are run from a single shell script.
>
> I see that in similar such cases that use the test harness
> (ksft_test_result_skip) the overall test returns success as long as
> all individual cases return either success or skip.
>
> I think it's preferable to return KSFT_SKIP if any of the cases did so
> (and none returned an error). I'll do that unless anyone objects.

http://patchwork.ozlabs.org/patch/1118309/

The shell script scaffolding can perhaps be reused for other similar tests.