* [LTP] [PATCH] controllers: detect previous test failure on cgroup mounts
@ 2021-07-20 15:05 Krzysztof Kozlowski
2021-08-11 10:04 ` Krzysztof Kozlowski
2021-09-08 15:13 ` Cyril Hrubis
0 siblings, 2 replies; 4+ messages in thread
From: Krzysztof Kozlowski @ 2021-07-20 15:05 UTC (permalink / raw)
To: ltp
The failure of memcg_stress_test.sh cleanup went unnoticed (except
echo message) and caused further cgroup_fj and memcg_control_test
failures because of unclean cgroups. Flow is usually like:
1. memcg_stress_test succeeds but actually the /dev/memcg was not
cleaned up. Notice lack of any error message due to 2>/dev/null.
memcg_stress_test 1 TINFO: Testing 150 cgroups, using 1273 MB, interval 5
memcg_stress_test 1 TINFO: Starting cgroups
memcg_stress_test 1 TINFO: Testing cgroups for 900s
memcg_stress_test 1 TINFO: Killing groups
tag=memcg_stress stime=1626770787 dur=903 exit=exited stat=2 core=no cu=19 cs=12
2. memcg_control_test has false-positive. It succeeds but actually no
test was done due to /dev/memcg pollution from previous test:
mkdir: cannot create directory ?/tmp/ltp-q8DjShPJeB/mnt/1?: File exists
memcg_control 0 TINFO : Test #1: Checking if the memory usage limit imposed by the topmost group is enforced
sh: echo: I/O error
/opt/ltp/testcases/bin/memcg_control_test.sh: 86: /opt/ltp/testcases/bin/memcg_control_test.sh: cannot create /tmp/ltp-q8DjShPJeB/mnt/1/memory.memsw.limit_in_bytes:
Permission denied
rmdir: failed to remove 'sub': Device or resource busy
rmdir: failed to remove '/tmp/ltp-q8DjShPJeB/mnt/1': Device or resource busy
memcg_control 1 TPASS : memcg_control: passed
tag=memcg_control stime=1626771695 dur=6 exit=exited stat=0 core=no cu=2 cs=1
3. cgroup_fj_function2_memory fails with a cryptic message of mounting a
path with new line (because /dev/memcg was not cleaned up before):
cgroup_fj_function2_memory 1 TINFO: Subsystem memory is mounted at /sys/fs/cgroup/memory
/dev/memcg
mkdir: cannot create directory ?/sys/fs/cgroup/memory?: File exists
cgroup_fj_function2_memory 1 TBROK: mkdir /sys/fs/cgroup/memory
/dev/memcg/ltp failed
cgroup_fj_function2_memory 1 TINFO: Removing all ltp subgroups...
find: ?/sys/fs/cgroup/memory\n/dev/memcg/ltp/?: No such file or directory
The actual failure was in memcg_stress_test executed before other tests,
however it went unnoticed. Debugging such failures is difficult as
result of failing test depends on running another which did not fail.
Instead, detect unclean cgroups mounts and explicitly test it.
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
---
.../kernel/controllers/cgroup_fj/cgroup_fj_common.sh | 4 ++--
.../controllers/memcg/control/memcg_control_test.sh | 10 ++++++----
.../controllers/memcg/stress/memcg_stress_test.sh | 8 ++++----
3 files changed, 12 insertions(+), 10 deletions(-)
diff --git a/testcases/kernel/controllers/cgroup_fj/cgroup_fj_common.sh b/testcases/kernel/controllers/cgroup_fj/cgroup_fj_common.sh
index 5594fe9de426..53ab637e8910 100755
--- a/testcases/kernel/controllers/cgroup_fj/cgroup_fj_common.sh
+++ b/testcases/kernel/controllers/cgroup_fj/cgroup_fj_common.sh
@@ -123,10 +123,10 @@ cleanup()
fi
if grep -q "$mount_point" /proc/mounts; then
- umount "$mount_point"
+ EXPECT_PASS umount "$mount_point"
fi
if [ -e "$mount_point" ]; then
- rmdir "$mount_point"
+ EXPECT_PASS rmdir "$mount_point"
fi
}
diff --git a/testcases/kernel/controllers/memcg/control/memcg_control_test.sh b/testcases/kernel/controllers/memcg/control/memcg_control_test.sh
index 4d9f1bb5d586..626f5e676831 100644
--- a/testcases/kernel/controllers/memcg/control/memcg_control_test.sh
+++ b/testcases/kernel/controllers/memcg/control/memcg_control_test.sh
@@ -53,6 +53,8 @@ STATUS_PIPE="$TMP/status_pipe"
PASS=0
FAIL=1
+. test.sh
+
# Check if the test process is killed on crossing boundary
test_proc_kill()
{
@@ -118,8 +120,8 @@ result()
cleanup()
{
if [ -e $TST_PATH/mnt ]; then
- umount $TST_PATH/mnt 2> /dev/null
- rm -rf $TST_PATH/mnt
+ EXPECT_PASS umount $TST_PATH/mnt
+ EXPECT_PASS rm -rf $TST_PATH/mnt
fi
}
@@ -127,8 +129,8 @@ do_mount()
{
cleanup
- mkdir $TST_PATH/mnt
- mount -t cgroup -o memory cgroup $TST_PATH/mnt 2> /dev/null
+ EXPECT_PASS mkdir $TST_PATH/mnt
+ EXPECT_PASS mount -t cgroup -o memory cgroup $TST_PATH/mnt 2> /dev/null
if [ $? -ne 0 ]; then
tst_brkm TBROK NULL "Mounting cgroup to temp dir failed"
rmdir $TST_PATH/mnt
diff --git a/testcases/kernel/controllers/memcg/stress/memcg_stress_test.sh b/testcases/kernel/controllers/memcg/stress/memcg_stress_test.sh
index c2501e164018..8f7a0eb9dc37 100755
--- a/testcases/kernel/controllers/memcg/stress/memcg_stress_test.sh
+++ b/testcases/kernel/controllers/memcg/stress/memcg_stress_test.sh
@@ -44,8 +44,8 @@ setup()
cleanup()
{
if [ -e /dev/memcg ]; then
- umount /dev/memcg 2> /dev/null
- rmdir /dev/memcg 2> /dev/null
+ EXPECT_PASS umount /dev/memcg
+ EXPECT_PASS rmdir /dev/memcg
fi
}
@@ -53,8 +53,8 @@ do_mount()
{
cleanup
- mkdir /dev/memcg 2> /dev/null
- mount -t cgroup -omemory memcg /dev/memcg
+ EXPECT_PASS mkdir /dev/memcg
+ EXPECT_PASS mount -t cgroup -omemory memcg /dev/memcg
}
# $1 Number of cgroups
--
2.27.0
^ permalink raw reply related [flat|nested] 4+ messages in thread
* [LTP] [PATCH] controllers: detect previous test failure on cgroup mounts
2021-07-20 15:05 [LTP] [PATCH] controllers: detect previous test failure on cgroup mounts Krzysztof Kozlowski
@ 2021-08-11 10:04 ` Krzysztof Kozlowski
2021-09-08 15:13 ` Cyril Hrubis
1 sibling, 0 replies; 4+ messages in thread
From: Krzysztof Kozlowski @ 2021-08-11 10:04 UTC (permalink / raw)
To: ltp
On 20/07/2021 17:05, Krzysztof Kozlowski wrote:
> The failure of memcg_stress_test.sh cleanup went unnoticed (except
> echo message) and caused further cgroup_fj and memcg_control_test
> failures because of unclean cgroups. Flow is usually like:
>
> 1. memcg_stress_test succeeds but actually the /dev/memcg was not
> cleaned up. Notice lack of any error message due to 2>/dev/null.
>
> memcg_stress_test 1 TINFO: Testing 150 cgroups, using 1273 MB, interval 5
> memcg_stress_test 1 TINFO: Starting cgroups
> memcg_stress_test 1 TINFO: Testing cgroups for 900s
> memcg_stress_test 1 TINFO: Killing groups
> tag=memcg_stress stime=1626770787 dur=903 exit=exited stat=2 core=no cu=19 cs=12
>
> 2. memcg_control_test has false-positive. It succeeds but actually no
> test was done due to /dev/memcg pollution from previous test:
>
> mkdir: cannot create directory ?/tmp/ltp-q8DjShPJeB/mnt/1?: File exists
> memcg_control 0 TINFO : Test #1: Checking if the memory usage limit imposed by the topmost group is enforced
> sh: echo: I/O error
> /opt/ltp/testcases/bin/memcg_control_test.sh: 86: /opt/ltp/testcases/bin/memcg_control_test.sh: cannot create /tmp/ltp-q8DjShPJeB/mnt/1/memory.memsw.limit_in_bytes:
> Permission denied
> rmdir: failed to remove 'sub': Device or resource busy
> rmdir: failed to remove '/tmp/ltp-q8DjShPJeB/mnt/1': Device or resource busy
> memcg_control 1 TPASS : memcg_control: passed
> tag=memcg_control stime=1626771695 dur=6 exit=exited stat=0 core=no cu=2 cs=1
>
> 3. cgroup_fj_function2_memory fails with a cryptic message of mounting a
> path with new line (because /dev/memcg was not cleaned up before):
>
> cgroup_fj_function2_memory 1 TINFO: Subsystem memory is mounted at /sys/fs/cgroup/memory
> /dev/memcg
> mkdir: cannot create directory ?/sys/fs/cgroup/memory?: File exists
> cgroup_fj_function2_memory 1 TBROK: mkdir /sys/fs/cgroup/memory
> /dev/memcg/ltp failed
> cgroup_fj_function2_memory 1 TINFO: Removing all ltp subgroups...
> find: ?/sys/fs/cgroup/memory\n/dev/memcg/ltp/?: No such file or directory
>
> The actual failure was in memcg_stress_test executed before other tests,
> however it went unnoticed. Debugging such failures is difficult as
> result of failing test depends on running another which did not fail.
> Instead, detect unclean cgroups mounts and explicitly test it.
>
> Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
> ---
> .../kernel/controllers/cgroup_fj/cgroup_fj_common.sh | 4 ++--
> .../controllers/memcg/control/memcg_control_test.sh | 10 ++++++----
> .../controllers/memcg/stress/memcg_stress_test.sh | 8 ++++----
> 3 files changed, 12 insertions(+), 10 deletions(-)
>
Hi all,
Any comments here?
Best regards,
Krzysztof
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [LTP] [PATCH] controllers: detect previous test failure on cgroup mounts
@ 2021-09-08 15:13 ` Cyril Hrubis
2021-09-09 10:19 ` Krzysztof Kozlowski
0 siblings, 1 reply; 4+ messages in thread
From: Cyril Hrubis @ 2021-09-08 15:13 UTC (permalink / raw)
To: Krzysztof Kozlowski; +Cc: ltp
Hi!
> Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
> ---
> .../kernel/controllers/cgroup_fj/cgroup_fj_common.sh | 4 ++--
> .../controllers/memcg/control/memcg_control_test.sh | 10 ++++++----
> .../controllers/memcg/stress/memcg_stress_test.sh | 8 ++++----
> 3 files changed, 12 insertions(+), 10 deletions(-)
>
> diff --git a/testcases/kernel/controllers/cgroup_fj/cgroup_fj_common.sh b/testcases/kernel/controllers/cgroup_fj/cgroup_fj_common.sh
> index 5594fe9de426..53ab637e8910 100755
> --- a/testcases/kernel/controllers/cgroup_fj/cgroup_fj_common.sh
> +++ b/testcases/kernel/controllers/cgroup_fj/cgroup_fj_common.sh
> @@ -123,10 +123,10 @@ cleanup()
> fi
>
> if grep -q "$mount_point" /proc/mounts; then
> - umount "$mount_point"
> + EXPECT_PASS umount "$mount_point"
> fi
>
> if [ -e "$mount_point" ]; then
> - rmdir "$mount_point"
> + EXPECT_PASS rmdir "$mount_point"
> fi
> }
> diff --git a/testcases/kernel/controllers/memcg/control/memcg_control_test.sh b/testcases/kernel/controllers/memcg/control/memcg_control_test.sh
> index 4d9f1bb5d586..626f5e676831 100644
> --- a/testcases/kernel/controllers/memcg/control/memcg_control_test.sh
> +++ b/testcases/kernel/controllers/memcg/control/memcg_control_test.sh
> @@ -53,6 +53,8 @@ STATUS_PIPE="$TMP/status_pipe"
> PASS=0
> FAIL=1
>
> +. test.sh
We cannot casually include the test.sh in testcases that does not use
the shell library as that will change the behavior of the tst_ commands.
Due to historicall reason we have two sets of tst_resm/tst_brkm/..., one
set are actuall binaries with the tst_foo name and the second set is
provided by the test.sh library. It's unfortunate but the binary
commands are still used in a few places and it's hard to even find out
where, since we have to look for shell scripts that does not source
(even indirectly test.sh) but use the tst_resm interfaces.
So unless you have reviewed the test carefully and made sure that it
works well with the test.sh library this will possibly introduce subtle
breakage.
The changes in the other two shell scripts looks good.
--
Cyril Hrubis
chrubis@suse.cz
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [LTP] [PATCH] controllers: detect previous test failure on cgroup mounts
@ 2021-09-09 10:19 ` Krzysztof Kozlowski
0 siblings, 0 replies; 4+ messages in thread
From: Krzysztof Kozlowski @ 2021-09-09 10:19 UTC (permalink / raw)
To: Cyril Hrubis; +Cc: ltp
On 08/09/2021 17:13, Cyril Hrubis wrote:
> Hi!
>> Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
>> ---
>> .../kernel/controllers/cgroup_fj/cgroup_fj_common.sh | 4 ++--
>> .../controllers/memcg/control/memcg_control_test.sh | 10 ++++++----
>> .../controllers/memcg/stress/memcg_stress_test.sh | 8 ++++----
>> 3 files changed, 12 insertions(+), 10 deletions(-)
>>
>> diff --git a/testcases/kernel/controllers/cgroup_fj/cgroup_fj_common.sh b/testcases/kernel/controllers/cgroup_fj/cgroup_fj_common.sh
>> index 5594fe9de426..53ab637e8910 100755
>> --- a/testcases/kernel/controllers/cgroup_fj/cgroup_fj_common.sh
>> +++ b/testcases/kernel/controllers/cgroup_fj/cgroup_fj_common.sh
>> @@ -123,10 +123,10 @@ cleanup()
>> fi
>>
>> if grep -q "$mount_point" /proc/mounts; then
>> - umount "$mount_point"
>> + EXPECT_PASS umount "$mount_point"
>> fi
>>
>> if [ -e "$mount_point" ]; then
>> - rmdir "$mount_point"
>> + EXPECT_PASS rmdir "$mount_point"
>> fi
>> }
>> diff --git a/testcases/kernel/controllers/memcg/control/memcg_control_test.sh b/testcases/kernel/controllers/memcg/control/memcg_control_test.sh
>> index 4d9f1bb5d586..626f5e676831 100644
>> --- a/testcases/kernel/controllers/memcg/control/memcg_control_test.sh
>> +++ b/testcases/kernel/controllers/memcg/control/memcg_control_test.sh
>> @@ -53,6 +53,8 @@ STATUS_PIPE="$TMP/status_pipe"
>> PASS=0
>> FAIL=1
>>
>> +. test.sh
>
> We cannot casually include the test.sh in testcases that does not use
> the shell library as that will change the behavior of the tst_ commands.
>
> Due to historicall reason we have two sets of tst_resm/tst_brkm/..., one
> set are actuall binaries with the tst_foo name and the second set is
> provided by the test.sh library. It's unfortunate but the binary
> commands are still used in a few places and it's hard to even find out
> where, since we have to look for shell scripts that does not source
> (even indirectly test.sh) but use the tst_resm interfaces.
>
> So unless you have reviewed the test carefully and made sure that it
> works well with the test.sh library this will possibly introduce subtle
> breakage.
I tested the change but I think I did not review it in mentioned aspect.
>
> The changes in the other two shell scripts looks good.
>
I'll send a v2 with a reduced scope of changes.
Best regards,
Krzysztof
--
Mailing list info: https://lists.linux.it/listinfo/ltp
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-09-09 10:20 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-07-20 15:05 [LTP] [PATCH] controllers: detect previous test failure on cgroup mounts Krzysztof Kozlowski
2021-08-11 10:04 ` Krzysztof Kozlowski
2021-09-08 15:13 ` Cyril Hrubis
2021-09-08 15:13 ` Cyril Hrubis
2021-09-09 10:19 ` Krzysztof Kozlowski
2021-09-09 10:19 ` Krzysztof Kozlowski
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.