Re: Pull request: lttng-ust tests clean-up

Re: Pull request: lttng-ust tests clean-up

[Mathieu Desnoyers]

* Jérémie Galarneau (jeremie.galarneau@efficios.com) wrote:
> Hi all,
> 
> I'd like to propose a set of patches to clean-up the lttng-ust tests.
> Since the clean-up now weighs in at a fairly hefty 21 patches, I think
> a link to my personal repository might be more appropriate than
> posting the patch-set on this list.
> 
> The relevant commits are 8ae5122...0060eb2
> 
> I will also be posting a set of patches that fixes and moves some of
> the lttng-ust tests to lttng-tools since they depend on it.
> 
> Git access:
> git://github.com/jgalar/lttng-ust-tests-cleanup.git  -b test-cleanup
> 
> Web interface:
> https://github.com/jgalar/lttng-ust-tests-cleanup

I tried running the various batch files under tests/ in your tree, and
they pretty much all complain about:

compudj@thinkos:~/git/jgalar/lttng-ust-tests-cleanup/tests$
(git:test-cleanup)> ./runtests 
./runtests: line 31: .//snprintf/run: No such file or directory

Is that expected ?

Also, make check only runs a single test:

snprintf/test_snprintf .. ok   
All tests successful.
Files=1, Tests=1,  0 wallclock secs ( 0.01 usr +  0.00 sys =  0.01 CPU)
Result: PASS

Is that expected too ?

Thanks,

Mathieu

> 
> Comments are welcome, as always!
> 
> Regards,
> Jérémie
> 
> -- 
> Jérémie Galarneau
> EfficiOS Inc.
> http://www.efficios.com
> 
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Re: Pull request: lttng-ust tests clean-up

[Jérémie Galarneau]

On Tue, Mar 26, 2013 at 9:48 AM, Mathieu Desnoyers
<mathieu.desnoyers@efficios.com> wrote:
> * Jérémie Galarneau (jeremie.galarneau@efficios.com) wrote:
>> Hi all,
>>
>> I'd like to propose a set of patches to clean-up the lttng-ust tests.
>> Since the clean-up now weighs in at a fairly hefty 21 patches, I think
>> a link to my personal repository might be more appropriate than
>> posting the patch-set on this list.
>>
>> The relevant commits are 8ae5122...0060eb2
>>
>> I will also be posting a set of patches that fixes and moves some of
>> the lttng-ust tests to lttng-tools since they depend on it.
>>
>> Git access:
>> git://github.com/jgalar/lttng-ust-tests-cleanup.git  -b test-cleanup
>>
>> Web interface:
>> https://github.com/jgalar/lttng-ust-tests-cleanup
>
> I tried running the various batch files under tests/ in your tree, and
> they pretty much all complain about:
>
> compudj@thinkos:~/git/jgalar/lttng-ust-tests-cleanup/tests$
> (git:test-cleanup)> ./runtests
> ./runtests: line 31: .//snprintf/run: No such file or directory
>
> Is that expected ?
>

I meant to remove that file since, with Christian's recent lttng-tools
patches, we moved away from using runner scripts to using prove + test
lists. I'll correct the last commit (Tests: Use Perl prove as the
testsuite runner).

> Also, make check only runs a single test:
>
> snprintf/test_snprintf .. ok
> All tests successful.
> Files=1, Tests=1,  0 wallclock secs ( 0.01 usr +  0.00 sys =  0.01 CPU)
> Result: PASS
>
> Is that expected too ?

Yes, most of the tests were moved to lttng-tools since they depend on
lttng-sessiond. Hopefully, now that the infrastructure is in place,
It'll be easier to add self-contained UST tests.

I'll take the opportunity to ask if you, or anyone else really, have
suggestions for UST unit tests that don't depend on lttng-tools?

I can think of a few, such as validating the header files outputted by
lttng-gen-tp, testing the filter bytecode interpretation mechanisms,
etc. But I wonder if unit testing the control interface, for instance,
is realistic since it tends to change fairly often. Of course, lttng
contributors are welcome to submit new tests!

As for my lttng-tools patches, some of the new tests depend on the
lttng-tools python bindings (not included in the default
configuration). I'm wondering what woud be the best way to get tests
to run conditionally depending on the "configure" options now that we
are using Christian's "prove + test lists" scheme.

I have discussed the issue with him privately and we both agreed that
we could have separate test lists that would be used depending on the
current project configuration. While this would certainly be good
enough for now, manually maintaining test lists based on configuration
dependancies may grow tedious as we add configuration options and
tests.

Perhaps we could dynamically generate test lists depending on the
configuration options and each tests' requirements... but that
certainly sounds like overkill right now. Thoughts?

Jérémie

>
> Thanks,
>
> Mathieu
>
>>
>> Comments are welcome, as always!
>>
>> Regards,
>> Jérémie
>>
>> --
>> Jérémie Galarneau
>> EfficiOS Inc.
>> http://www.efficios.com
>>
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev@lists.lttng.org
>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>
> --
> Mathieu Desnoyers
> EfficiOS Inc.
> http://www.efficios.com



-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com

Re: Pull request: lttng-ust tests clean-up

[Christian Babeux]

Hi Jérémie,

> I'm wondering what would be the best way to get tests
> to run conditionally depending on the "configure" options now that we
> are using Christian's "prove + test lists" scheme.

What I have in mind is something along this way:

In the lttng-tools/tests/Makefile.am:

if USE_PYTHON
check-am:
        ./run.sh unit_tests
        ./run.sh fast_regression
        ./run.sh with_bindings_regression
else
check-am:
        ./run.sh unit_tests
        ./run.sh fast_regression
endif

The testlist with_bindings_regression would contain the appropriate
tests that have a dependency on the lttng-tools Python bindings.

> While this would certainly be good
> enough for now, manually maintaining test lists based on configuration
> dependancies may grow tedious as we add configuration options and
> tests.

 Agreed. This would only apply of course if we add those tests to the
default "make check" target.

> Perhaps we could dynamically generate test lists depending on the
> configuration options and each tests' requirements... but that
> certainly sounds like overkill right now.

Let's keep it simple for now :).

Christian

On Tue, Mar 26, 2013 at 10:48 AM, Jérémie Galarneau
<jeremie.galarneau@efficios.com> wrote:
> On Tue, Mar 26, 2013 at 9:48 AM, Mathieu Desnoyers
> <mathieu.desnoyers@efficios.com> wrote:
>> * Jérémie Galarneau (jeremie.galarneau@efficios.com) wrote:
>>> Hi all,
>>>
>>> I'd like to propose a set of patches to clean-up the lttng-ust tests.
>>> Since the clean-up now weighs in at a fairly hefty 21 patches, I think
>>> a link to my personal repository might be more appropriate than
>>> posting the patch-set on this list.
>>>
>>> The relevant commits are 8ae5122...0060eb2
>>>
>>> I will also be posting a set of patches that fixes and moves some of
>>> the lttng-ust tests to lttng-tools since they depend on it.
>>>
>>> Git access:
>>> git://github.com/jgalar/lttng-ust-tests-cleanup.git  -b test-cleanup
>>>
>>> Web interface:
>>> https://github.com/jgalar/lttng-ust-tests-cleanup
>>
>> I tried running the various batch files under tests/ in your tree, and
>> they pretty much all complain about:
>>
>> compudj@thinkos:~/git/jgalar/lttng-ust-tests-cleanup/tests$
>> (git:test-cleanup)> ./runtests
>> ./runtests: line 31: .//snprintf/run: No such file or directory
>>
>> Is that expected ?
>>
>
> I meant to remove that file since, with Christian's recent lttng-tools
> patches, we moved away from using runner scripts to using prove + test
> lists. I'll correct the last commit (Tests: Use Perl prove as the
> testsuite runner).
>
>> Also, make check only runs a single test:
>>
>> snprintf/test_snprintf .. ok
>> All tests successful.
>> Files=1, Tests=1,  0 wallclock secs ( 0.01 usr +  0.00 sys =  0.01 CPU)
>> Result: PASS
>>
>> Is that expected too ?
>
> Yes, most of the tests were moved to lttng-tools since they depend on
> lttng-sessiond. Hopefully, now that the infrastructure is in place,
> It'll be easier to add self-contained UST tests.
>
> I'll take the opportunity to ask if you, or anyone else really, have
> suggestions for UST unit tests that don't depend on lttng-tools?
>
> I can think of a few, such as validating the header files outputted by
> lttng-gen-tp, testing the filter bytecode interpretation mechanisms,
> etc. But I wonder if unit testing the control interface, for instance,
> is realistic since it tends to change fairly often. Of course, lttng
> contributors are welcome to submit new tests!
>
> As for my lttng-tools patches, some of the new tests depend on the
> lttng-tools python bindings (not included in the default
> configuration). I'm wondering what woud be the best way to get tests
> to run conditionally depending on the "configure" options now that we
> are using Christian's "prove + test lists" scheme.
>
> I have discussed the issue with him privately and we both agreed that
> we could have separate test lists that would be used depending on the
> current project configuration. While this would certainly be good
> enough for now, manually maintaining test lists based on configuration
> dependancies may grow tedious as we add configuration options and
> tests.
>
> Perhaps we could dynamically generate test lists depending on the
> configuration options and each tests' requirements... but that
> certainly sounds like overkill right now. Thoughts?
>
> Jérémie
>
>>
>> Thanks,
>>
>> Mathieu
>>
>>>
>>> Comments are welcome, as always!
>>>
>>> Regards,
>>> Jérémie
>>>
>>> --
>>> Jérémie Galarneau
>>> EfficiOS Inc.
>>> http://www.efficios.com
>>>
>>> _______________________________________________
>>> lttng-dev mailing list
>>> lttng-dev@lists.lttng.org
>>> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>>
>> --
>> Mathieu Desnoyers
>> EfficiOS Inc.
>> http://www.efficios.com
>
>
>
> --
> Jérémie Galarneau
> EfficiOS Inc.
> http://www.efficios.com

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: Pull request: lttng-ust tests clean-up

[Mathieu Desnoyers]

* Jérémie Galarneau (jeremie.galarneau@efficios.com) wrote:
> Hi all,
> 
> I'd like to propose a set of patches to clean-up the lttng-ust tests.
> Since the clean-up now weighs in at a fairly hefty 21 patches, I think
> a link to my personal repository might be more appropriate than
> posting the patch-set on this list.

merged, thanks!

I rebased it on master.

Mathieu

> 
> The relevant commits are 8ae5122...0060eb2
> 
> I will also be posting a set of patches that fixes and moves some of
> the lttng-ust tests to lttng-tools since they depend on it.
> 
> Git access:
> git://github.com/jgalar/lttng-ust-tests-cleanup.git  -b test-cleanup
> 
> Web interface:
> https://github.com/jgalar/lttng-ust-tests-cleanup
> 
> Comments are welcome, as always!
> 
> Regards,
> Jérémie
> 
> -- 
> Jérémie Galarneau
> EfficiOS Inc.
> http://www.efficios.com
> 
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

[PATCH lttng-ust] Fix: Remove test runner script

[Jérémie Galarneau]

Deprecated by the use of Perl prove and test lists.

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
---
 tests/Makefile.am |  2 +-
 tests/README      |  4 ++--
 tests/runtests    | 36 ------------------------------------
 tests/test_loop   |  2 +-
 4 files changed, 4 insertions(+), 40 deletions(-)
 delete mode 100755 tests/runtests

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 6f9b969..425440a 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -4,7 +4,7 @@ if CXX_WORKS
 SUBDIRS += hello.cxx
 endif
 
-dist_noinst_SCRIPTS = test_loop runtests trace_matches
+dist_noinst_SCRIPTS = test_loop run.sh trace_matches unit_tests
 
 noinst_LIBRARIES = libtap.a
 
diff --git a/tests/README b/tests/README
index 1c920eb..5f1da32 100644
--- a/tests/README
+++ b/tests/README
@@ -1,8 +1,8 @@
 Using the ust test suite.
 
 Running
-	./runtests
-runs all the tests once. The output is in TAP format.
+	./run.sh unit_tests
+runs all the tests once.
 
 Running
 	./test_loop
diff --git a/tests/runtests b/tests/runtests
deleted file mode 100755
index 8f5c874..0000000
--- a/tests/runtests
+++ /dev/null
@@ -1,36 +0,0 @@
-#!/bin/bash
-#
-# Copyright 2010 Ericsson AB
-#
-#    This file is part of LTTng-UST.
-#
-#    LTTng-UST is free software: you can redistribute it and/or modify
-#    it under the terms of the GNU General Public License as published by
-#    the Free Software Foundation, either version 2 of the License, or
-#    (at your option) any later version.
-#
-#    LTTng-UST is distributed in the hope that it will be useful,
-#    but WITHOUT ANY WARRANTY; without even the implied warranty of
-#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-#    GNU General Public License for more details.
-#
-#    You should have received a copy of the GNU General Public License
-#    along with LTTng-UST.  If not, see <http://www.gnu.org/licenses/>.
-
-
-tests_failed=0
-
-TESTDIR=$(dirname $0)
-
-function simple_harness_run() {
-    if ! "$TESTDIR/$@"; then
-	let tests_failed=$tests_failed+1
-    fi
-}
-
-./${TEST_DIR}/snprintf/run
-
-# This testcase is currently broken, do not run.
-# simple_harness_run tracepoint/run
-
-exit 0
diff --git a/tests/test_loop b/tests/test_loop
index 619b1e3..be3e16c 100755
--- a/tests/test_loop
+++ b/tests/test_loop
@@ -3,7 +3,7 @@
 DIR=$(dirname $0)
 
 while [ 0 ]; do
-	$DIR/runtests
+	$DIR/run.sh $DIR/unit_tests
 	if [ "$?" != "0" ]; then
 		break;
 	fi
-- 
1.8.2


_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [PATCH lttng-ust] Fix: Remove test runner script

[Mathieu Desnoyers]

* Jérémie Galarneau (jeremie.galarneau@efficios.com) wrote:
> Deprecated by the use of Perl prove and test lists.

merged, thanks!

Mathieu

> 
> Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
> ---
>  tests/Makefile.am |  2 +-
>  tests/README      |  4 ++--
>  tests/runtests    | 36 ------------------------------------
>  tests/test_loop   |  2 +-
>  4 files changed, 4 insertions(+), 40 deletions(-)
>  delete mode 100755 tests/runtests
> 
> diff --git a/tests/Makefile.am b/tests/Makefile.am
> index 6f9b969..425440a 100644
> --- a/tests/Makefile.am
> +++ b/tests/Makefile.am
> @@ -4,7 +4,7 @@ if CXX_WORKS
>  SUBDIRS += hello.cxx
>  endif
>  
> -dist_noinst_SCRIPTS = test_loop runtests trace_matches
> +dist_noinst_SCRIPTS = test_loop run.sh trace_matches unit_tests
>  
>  noinst_LIBRARIES = libtap.a
>  
> diff --git a/tests/README b/tests/README
> index 1c920eb..5f1da32 100644
> --- a/tests/README
> +++ b/tests/README
> @@ -1,8 +1,8 @@
>  Using the ust test suite.
>  
>  Running
> -	./runtests
> -runs all the tests once. The output is in TAP format.
> +	./run.sh unit_tests
> +runs all the tests once.
>  
>  Running
>  	./test_loop
> diff --git a/tests/runtests b/tests/runtests
> deleted file mode 100755
> index 8f5c874..0000000
> --- a/tests/runtests
> +++ /dev/null
> @@ -1,36 +0,0 @@
> -#!/bin/bash
> -#
> -# Copyright 2010 Ericsson AB
> -#
> -#    This file is part of LTTng-UST.
> -#
> -#    LTTng-UST is free software: you can redistribute it and/or modify
> -#    it under the terms of the GNU General Public License as published by
> -#    the Free Software Foundation, either version 2 of the License, or
> -#    (at your option) any later version.
> -#
> -#    LTTng-UST is distributed in the hope that it will be useful,
> -#    but WITHOUT ANY WARRANTY; without even the implied warranty of
> -#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> -#    GNU General Public License for more details.
> -#
> -#    You should have received a copy of the GNU General Public License
> -#    along with LTTng-UST.  If not, see <http://www.gnu.org/licenses/>.
> -
> -
> -tests_failed=0
> -
> -TESTDIR=$(dirname $0)
> -
> -function simple_harness_run() {
> -    if ! "$TESTDIR/$@"; then
> -	let tests_failed=$tests_failed+1
> -    fi
> -}
> -
> -./${TEST_DIR}/snprintf/run
> -
> -# This testcase is currently broken, do not run.
> -# simple_harness_run tracepoint/run
> -
> -exit 0
> diff --git a/tests/test_loop b/tests/test_loop
> index 619b1e3..be3e16c 100755
> --- a/tests/test_loop
> +++ b/tests/test_loop
> @@ -3,7 +3,7 @@
>  DIR=$(dirname $0)
>  
>  while [ 0 ]; do
> -	$DIR/runtests
> +	$DIR/run.sh $DIR/unit_tests
>  	if [ "$?" != "0" ]; then
>  		break;
>  	fi
> -- 
> 1.8.2
> 
> 
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Pull request: lttng-ust tests clean-up

[Jérémie Galarneau]

Hi all,

I'd like to propose a set of patches to clean-up the lttng-ust tests.
Since the clean-up now weighs in at a fairly hefty 21 patches, I think
a link to my personal repository might be more appropriate than
posting the patch-set on this list.

The relevant commits are 8ae5122...0060eb2

I will also be posting a set of patches that fixes and moves some of
the lttng-ust tests to lttng-tools since they depend on it.

Git access:
git://github.com/jgalar/lttng-ust-tests-cleanup.git  -b test-cleanup

Web interface:
https://github.com/jgalar/lttng-ust-tests-cleanup

Comments are welcome, as always!

Regards,
Jérémie

-- 
Jérémie Galarneau
EfficiOS Inc.
http://www.efficios.com