From: Jan Kiszka <jan.kiszka@siemens.com>
To: Florian Bezdeka <florian.bezdeka@siemens.com>,
Aaron Marcher <aaron@sigma-star.at>,
xenomai@lists.linux.dev
Cc: richard@nod.at
Subject: Re: [PATCH 01/10] testsuite: Fully integrate alchemytests into smokey
Date: Fri, 11 Aug 2023 19:32:56 +0200 [thread overview]
Message-ID: <1befb87a-cc47-460b-9615-10a04f752062@siemens.com> (raw)
In-Reply-To: <c8b63b8bf2c3d4f803d6049d874e9a37f5dac284.camel@siemens.com>
On 10.08.23 09:07, Florian Bezdeka wrote:
> On Mon, 2023-07-24 at 15:52 +0200, Aaron Marcher wrote:
>> This is the very first step to have the alchemy tests embedded
>> into our testsuite.
>>
>> Build them using Xenomai's build system and hook them up using
>> Smokey. Every test binary has it's own respective testcase in the
>> testsuite.
>>
>> Signed-off-by: Richard Weinberger <richard@nod.at>
>> Signed-off-by: Aaron Marcher <aaron@sigma-star.at>
>> ---
>
> [snip]
>
>>
>> diff --git a/testsuite/smokey/alchemytests/alchemytests.c b/testsuite/smokey/alchemytests/alchemytests.c
>> new file mode 100644
>> index 000000000..ee8d1703b
>> --- /dev/null
>> +++ b/testsuite/smokey/alchemytests/alchemytests.c
>> @@ -0,0 +1,56 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +#include <error.h>
>> +#include <libgen.h>
>> +#include <smokey/smokey.h>
>> +#include <stdbool.h>
>> +#include <stdio.h>
>> +#include <stdlib.h>
>> +#include <string.h>
>> +#include <unistd.h>
>
> IMHO includes can be cleaned up. Can't spot a usage of error.h,
> libgen.h, stdbool.h and unistd.h (at least).
>
> Applies to more locations within this series.
>
>> +
>> +#define TEST(name) \
>> + smokey_test_plugin(name, SMOKEY_NOARGS, "Run external test"); \
>> + static int run_##name(struct smokey_test *t, int argc, char *const argv[]) \
>> + { \
>> + return __run_extprog(t, argc, argv); \
>> + }
>> +
>> +static int __run_extprog(struct smokey_test *t, int argc, char *const argv[])
>> +{
>> + int ret;
>> + char *tst_path;
>> +
>> + ret = asprintf(&tst_path, "%s/%s --cpu-affinity=0", XENO_TEST_DIR, t->name);
>> + if (ret == -1)
>> + return -ENOMEM;
>> +
>> + ret = system(tst_path);
>> + free(tst_path);
>> +
>> + return ret;
>> +}
>> +
>> +TEST(alchemytests)
>> +TEST(alchemytests_alarm1)
>> +TEST(alchemytests_buffer1)
>> +TEST(alchemytests_event1)
>> +TEST(alchemytests_heap1)
>> +TEST(alchemytests_heap2)
>> +TEST(alchemytests_mq1)
>> +TEST(alchemytests_mq2)
>> +TEST(alchemytests_mq3)
>> +TEST(alchemytests_mutex1)
>> +TEST(alchemytests_pipe1)
>> +TEST(alchemytests_sem1)
>> +TEST(alchemytests_sem2)
>> +TEST(alchemytests_task1)
>> +TEST(alchemytests_task2)
>> +TEST(alchemytests_task3)
>> +TEST(alchemytests_task4)
>> +TEST(alchemytests_task5)
>> +TEST(alchemytests_task6)
>> +TEST(alchemytests_task7)
>> +TEST(alchemytests_task8)
>> +TEST(alchemytests_task9)
>> +TEST(alchemytests_task10)
>> +
>
> The following applies here and on several other places of this series:
>
> Do we really want to have a test plugin for each sub-test? I would have
> expected one plugin for each "test class", so basically one for each
> sub folder in testsuite/smokey.
>
> Each module could then call all the sub-tests and report back which one
> actually failed - if any.
>
Looking at smokey --list, I agree with this: too many test cases to
browse now. Please rework this on top of 'next'.
> Btw: I think Jan already reported that: I have to disable the
> vxworkstests_task2 test as it fails CI runs here.
>
Fix is in next.
Jan
--
Siemens AG, Technology
Linux Expert Center
next prev parent reply other threads:[~2023-08-11 17:33 UTC|newest]
Thread overview: 112+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-13 21:58 [PATCH v2 0/9] Revive alchemy, pSOS and VxWorks tests Richard Weinberger
2022-04-13 21:58 ` [PATCH 1/9] testsuite: Move alchemy tests into testsuite/ Richard Weinberger
2022-04-13 21:58 ` [PATCH 2/9] testsuite: Hook up alchemytests Richard Weinberger
2022-04-13 21:58 ` [PATCH 3/9] testsuite: Add a simple test driver for alchemytests Richard Weinberger
2022-04-14 11:28 ` Jan Kiszka
2022-04-14 11:42 ` Richard Weinberger
2022-04-13 21:58 ` [PATCH 4/9] Remove old alchemy tests Makefile Richard Weinberger
2022-04-13 21:58 ` [PATCH 5/9] alchemytests: Fix gcc warning in buffer-1 Richard Weinberger
2022-04-13 21:58 ` [PATCH 6/9] alchemytests: Fix gcc warning in task-9 Richard Weinberger
2022-04-13 21:58 ` [PATCH 7/9] testsuite: Move pSOS tests into testsuite/ Richard Weinberger
2022-04-13 21:58 ` [PATCH 8/9] testsuite: Add a simple test driver for psostests Richard Weinberger
2022-04-13 21:58 ` [PATCH 9/9] testsuite: Integrate vwworks tests into testsuite/ Richard Weinberger
2022-04-14 11:18 ` [PATCH v2 0/9] Revive alchemy, pSOS and VxWorks tests Jan Kiszka
2022-04-14 11:29 ` Richard Weinberger
2022-04-14 12:58 ` Richard Weinberger
2022-04-14 14:15 ` Bezdeka, Florian
2022-04-14 14:19 ` Richard Weinberger
2022-04-14 15:13 ` Richard Weinberger
2022-04-14 15:34 ` Jan Kiszka
2022-04-14 15:41 ` Richard Weinberger
2022-04-14 15:43 ` Jan Kiszka
2023-04-06 10:33 ` [PATCH v3 0/11] " Aaron Marcher
2023-04-06 10:33 ` [PATCH 01/11] testsuite: Move alchemy tests into testsuite/ Aaron Marcher
2023-04-06 10:33 ` [PATCH 02/11] testsuite: Hook up alchemytests Aaron Marcher
2023-04-06 10:33 ` [PATCH 03/11] testsuite: Add a simple test driver for alchemytests Aaron Marcher
2023-04-06 10:33 ` [PATCH 04/11] Remove old alchemy tests Makefile Aaron Marcher
2023-04-06 10:33 ` [PATCH 05/11] alchemytests: Fix gcc warning in buffer-1 Aaron Marcher
2023-04-06 10:33 ` [PATCH 06/11] alchemytests: Fix gcc warning in task-9 Aaron Marcher
2023-04-06 10:33 ` [PATCH 07/11] testsuite: Move pSOS tests into testsuite/ Aaron Marcher
2023-04-06 10:33 ` [PATCH 08/11] testsuite: Add a simple test driver for psostests Aaron Marcher
2023-04-06 10:33 ` [PATCH 09/11] testsuite: Integrate vwworks tests into testsuite/ Aaron Marcher
2023-04-06 10:33 ` [PATCH 10/11] alchemytests: check pipe-1 asprintf ret value Aaron Marcher
2023-04-06 10:33 ` [PATCH 11/11] vxworkstests: fix Makefile.am reference Aaron Marcher
2023-04-16 7:33 ` Jan Kiszka
2023-04-16 7:32 ` [PATCH v3 0/11] Revive alchemy, pSOS and VxWorks tests Jan Kiszka
2023-04-16 7:41 ` Jan Kiszka
2023-04-16 9:03 ` Richard Weinberger
2023-05-17 7:30 ` Aaron Marcher
2023-05-17 9:35 ` Jan Kiszka
2023-05-20 14:57 ` Florian Bezdeka
2023-07-20 14:51 ` [PATCH v4 00/10] " Aaron Marcher
2023-07-20 14:51 ` [PATCH 01/10] testsuite: Fully integrate alchemytests into smokey Aaron Marcher
2023-07-20 16:50 ` Jan Kiszka
2023-07-20 14:51 ` [PATCH 02/10] testsuite: Fully integrate psostests " Aaron Marcher
2023-07-20 14:51 ` [PATCH 03/10] testsuite: Fully integrate vwworkstests " Aaron Marcher
2023-07-20 14:51 ` [PATCH 04/10] testsuite: Remove old alchemytests Makefile Aaron Marcher
2023-07-20 14:51 ` [PATCH 05/10] alchemytests: Fix gcc warning in buffer-1 Aaron Marcher
2023-07-20 14:51 ` [PATCH 06/10] alchemytests: Fix gcc warning in task-9 Aaron Marcher
2023-07-20 14:51 ` [PATCH 07/10] alchemytests: Check pipe-1 asprintf ret value Aaron Marcher
2023-07-20 14:51 ` [PATCH 08/10] alchemytests: Fix task-2 Aaron Marcher
2023-07-20 14:51 ` [PATCH 09/10] alchemytests: Fix task-5 logic Aaron Marcher
2023-07-20 14:51 ` [PATCH 10/10] testsuite: disable broken tests Aaron Marcher
2023-07-20 14:54 ` [PATCH v4 00/10] Revive alchemy, pSOS and VxWorks tests Jan Kiszka
2023-07-20 15:01 ` Aaron Marcher
2023-07-24 13:52 ` [PATCH v5 " Aaron Marcher
2023-07-24 13:52 ` [PATCH 01/10] testsuite: Fully integrate alchemytests into smokey Aaron Marcher
2023-08-10 7:07 ` Florian Bezdeka
2023-08-11 17:32 ` Jan Kiszka [this message]
2023-08-16 12:43 ` Aaron Marcher
2023-09-14 7:41 ` Jan Kiszka
2023-09-20 13:52 ` [PATCH 0/5] Improvements for smokey skintests Aaron Marcher
2023-09-20 13:52 ` [PATCH 1/5] lib: Add smokey_run_extprog as helper for testsuite Aaron Marcher
2023-09-22 15:49 ` Jan Kiszka
2023-09-22 21:03 ` Richard Weinberger
2023-09-25 13:03 ` [PATCH] lib: Drop cpu-affinity=0 from skin-tests helper Aaron Marcher
2023-09-25 15:04 ` Philippe Gerum
2023-09-25 15:12 ` [PATCH 0/1 V2] " Aaron Marcher
2023-09-25 15:12 ` [PATCH 1/1] " Aaron Marcher
2023-09-25 15:42 ` Jan Kiszka
2023-09-28 17:04 ` Jan Kiszka
2023-09-28 17:30 ` Jan Kiszka
2023-09-29 12:12 ` Aaron Marcher
2023-09-29 12:34 ` Jan Kiszka
2023-09-20 13:52 ` [PATCH 2/5] testsuite: Merge alchemytests in single smokey case Aaron Marcher
2023-09-20 13:52 ` [PATCH 3/5] testsuite: Merge psostests in a " Aaron Marcher
2023-09-20 13:52 ` [PATCH 4/5] testsuite: Merge vxworkstests " Aaron Marcher
2023-09-20 13:52 ` [PATCH 5/5] testsuite: Drop skin test "drivers" Aaron Marcher
2023-09-22 15:52 ` [PATCH 0/5] Improvements for smokey skintests Jan Kiszka
2023-09-20 14:00 ` [PATCH 01/10] testsuite: Fully integrate alchemytests into smokey Aaron Marcher
2023-08-16 12:41 ` Aaron Marcher
2023-08-16 14:19 ` Jan Kiszka
2023-07-24 13:52 ` [PATCH 02/10] testsuite: Fully integrate psostests " Aaron Marcher
2023-07-24 13:52 ` [PATCH 03/10] testsuite: Fully integrate vwworkstests " Aaron Marcher
2023-07-24 13:52 ` [PATCH 04/10] testsuite: Remove old alchemytests Makefile Aaron Marcher
2023-07-24 13:52 ` [PATCH 05/10] alchemytests: Fix gcc warning in buffer-1 Aaron Marcher
2023-07-24 13:52 ` [PATCH 06/10] alchemytests: Fix gcc warning in task-9 Aaron Marcher
2023-07-24 13:52 ` [PATCH 07/10] alchemytests: Check pipe-1 asprintf ret value Aaron Marcher
2023-07-24 13:52 ` [PATCH 08/10] alchemytests: Fix task-2 Aaron Marcher
2023-07-24 14:39 ` Richard Weinberger
2023-07-25 8:29 ` [PATCH v6 00/10] Revive alchemy, pSOS and VxWorks tests Aaron Marcher
2023-07-25 8:29 ` [PATCH 01/10] testsuite: Fully integrate alchemytests into smokey Aaron Marcher
2023-08-09 7:28 ` Jan Kiszka
2023-08-09 11:32 ` [PATCH v7 " Jan Kiszka
2023-07-25 8:29 ` [PATCH 02/10] testsuite: Fully integrate psostests " Aaron Marcher
2023-08-09 11:32 ` [PATCH v7 " Jan Kiszka
2023-07-25 8:29 ` [PATCH 03/10] testsuite: Fully integrate vwworkstests " Aaron Marcher
2023-08-09 11:32 ` [PATCH v7 " Jan Kiszka
2023-08-09 15:02 ` Jan Kiszka
2023-07-25 8:29 ` [PATCH 04/10] testsuite: Remove old alchemytests Makefile Aaron Marcher
2023-07-25 8:29 ` [PATCH 05/10] alchemytests: Fix gcc warning in buffer-1 Aaron Marcher
2023-07-25 8:29 ` [PATCH 06/10] alchemytests: Fix gcc warning in task-9 Aaron Marcher
2023-07-25 8:29 ` [PATCH 07/10] alchemytests: Check pipe-1 asprintf ret value Aaron Marcher
2023-07-25 8:29 ` [PATCH 08/10] alchemytests: Fix task-2 Aaron Marcher
2023-07-25 8:29 ` [PATCH 09/10] alchemytests: Fix task-5 logic Aaron Marcher
2023-07-25 8:29 ` [PATCH 10/10] testsuite: disable broken tests Aaron Marcher
2023-08-03 15:35 ` [PATCH v6 00/10] Revive alchemy, pSOS and VxWorks tests Florian Bezdeka
2023-08-09 7:36 ` Jan Kiszka
2023-08-09 11:33 ` Jan Kiszka
2023-08-16 12:37 ` Aaron Marcher
2023-08-09 11:32 ` [PATCH v7 11/10] testsuite/smokey: Hook up new tests Jan Kiszka
2023-07-24 13:52 ` [PATCH 09/10] alchemytests: Fix task-5 logic Aaron Marcher
2023-07-24 13:52 ` [PATCH 10/10] testsuite: disable broken tests Aaron Marcher
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1befb87a-cc47-460b-9615-10a04f752062@siemens.com \
--to=jan.kiszka@siemens.com \
--cc=aaron@sigma-star.at \
--cc=florian.bezdeka@siemens.com \
--cc=richard@nod.at \
--cc=xenomai@lists.linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).