Re: [Autotest] [AUTOTEST] [PATCH 1/2] Add latest LTP test in autotest

From: sudhir kumar <smalikphy@gmail.com>
To: Martin Bligh <mbligh@google.com>
Cc: Lucas Meneghel Rodrigues <lmr@redhat.com>,
	Autotest mailing list <autotest@test.kernel.org>,
	Lucas Meneghel Rodrigues <mrodrigu@redhat.com>,
	Uri Lublin <uril@redhat.com>, kvm-devel <kvm@vger.kernel.org>
Subject: Re: [Autotest] [AUTOTEST] [PATCH 1/2] Add latest LTP test in autotest
Date: Wed, 8 Jul 2009 09:47:52 +0530	[thread overview]
Message-ID: <a50cf5ab0907072117t7773ed87nde2b1811f95cb33e@mail.gmail.com> (raw)
In-Reply-To: <33307c790907071045v72e19614i571c36ad8af8062c@mail.gmail.com>

Ok Then. So my idea is to include the patch in autotest and let the
people report failures(in compilation or execution), and we can patch
autotest to apply the fix patch and build and run ltp. I do not think
we can find all cases untill and unless we start execution.

However I will start the discussion on the ltp list and see the
response from people. At least we can get the new testcases to be
aware of virtualization.

On Tue, Jul 7, 2009 at 11:15 PM, Martin Bligh<mbligh@google.com> wrote:
> On Tue, Jul 7, 2009 at 12:24 AM, sudhir kumar<smalikphy@gmail.com> wrote:
>> On Tue, Jul 7, 2009 at 12:07 AM, Martin Bligh<mbligh@google.com> wrote:
>>>>> Issues: LTP has a history of some of the testcases getting broken.
>>>
>>> Right, that's always the concern with doing this.
>>>
>>>>> Anyways
>>>>> that has nothing to worry about with respect to autotest. One of the known issue
>>>>> is broken memory controller issue with latest kernels(cgroups and memory
>>>>> resource controller enabled kernels). The workaround for them I use is to
>>>>> disable or delete those tests from ltp source and tar it again with the same
>>>>> name. Though people might use different workarounds for it.
>>>
>>> OK, Can we encapsulate this into the wrapper though, rather than making
>>> people do it manually? in the existing ltp.patch or something?
>>>
>> definitely we can do that, but that needs to know about all the corner
>> cases of failure. So may be we can continue enhancing the patch as per
>> the failure reports on different OSes.
>>
>> 1 more thing I wanted to start a discussion on LTP mailing list is to
>> make aware the testcase if it is running on a physical host or on a
>> guest(say KVM guest). Testcases like power management, group
>> scheduling fairness etc do not make much sense to run on a guest(as
>> they will fail or break). So It is better for the test to recognise
>> the environment and not execute if it is under virtualization and it
>> is supposed to fail or break under that environment. Does that make
>> sense to you also ?
>
> Yup, we can pass an excluded test list. I really wish they'd fix their
> tests, but I've been saying that for 6 years now, and it hasn't happened
> yet ;-(
>

-- 
Sudhir Kumar