[LTP] [PATCH 00/10] Max_runtime and other minor fixes

[LTP] [PATCH 00/10] Max_runtime and other minor fixes

[Martin Doucha]

The introduction of max_runtime system shortly before the latest LTP release
has caused some random timeout issues, particularly for KSM tests,
sendfile09 and ioctl_sg01. This patchset fixes those issues, along with
some minor improvements found during during test review.

All of these patches are independent and can be merged out of order.

Martin Doucha (10):
  syscalls/pty06: Add CVE reference
  cve-2017-2671: Increase max_runtime
  ksm: Add max_runtime to tests
  ioctl_sg01: Set unlimited max_runtime
  csf_bandwidth01: Add max_runtime
  copy_file_range01: Add max_runtime
  perf_event_open02: Add max_runtime
  readahead02: Add max_runtime
  request_key03: Add max_runtime
  sendfile09: Add max_runtime

 testcases/cve/cve-2017-2671.c                                 | 2 +-
 testcases/kernel/mem/ksm/ksm02.c                              | 1 +
 testcases/kernel/mem/ksm/ksm04.c                              | 1 +
 testcases/kernel/mem/ksm/ksm05.c                              | 1 +
 testcases/kernel/pty/pty06.c                                  | 3 ++-
 testcases/kernel/sched/cfs-scheduler/cfs_bandwidth01.c        | 1 +
 testcases/kernel/syscalls/copy_file_range/copy_file_range01.c | 1 +
 testcases/kernel/syscalls/ioctl/ioctl_sg01.c                  | 1 +
 testcases/kernel/syscalls/perf_event_open/perf_event_open02.c | 1 +
 testcases/kernel/syscalls/readahead/readahead02.c             | 1 +
 testcases/kernel/syscalls/request_key/request_key03.c         | 1 +
 testcases/kernel/syscalls/sendfile/sendfile09.c               | 1 +
 12 files changed, 13 insertions(+), 2 deletions(-)

-- 
2.37.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH 01/10] syscalls/pty06: Add CVE reference

[Martin Doucha]

Signed-off-by: Martin Doucha <mdoucha@suse.cz>
---
 testcases/kernel/pty/pty06.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/testcases/kernel/pty/pty06.c b/testcases/kernel/pty/pty06.c
index 7e0f4233c..cc95eb1a8 100644
--- a/testcases/kernel/pty/pty06.c
+++ b/testcases/kernel/pty/pty06.c
@@ -99,7 +99,8 @@ static struct tst_test test = {
 	.taint_check = TST_TAINT_W | TST_TAINT_D,
 	.max_runtime = 150,
 	.tags = (const struct tst_tag[]) {
-	    { "linux-git", "ca4463bf8438"},
+	    {"CVE", "2020-36557"},
+	    {"linux-git", "ca4463bf8438"},
 	    {}
 	}
 };
-- 
2.37.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH 02/10] cve-2017-2671: Increase max_runtime

[Martin Doucha]

The current max_runtime is insufficient to finish the race. Increase it.

Signed-off-by: Martin Doucha <mdoucha@suse.cz>
---
 testcases/cve/cve-2017-2671.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/testcases/cve/cve-2017-2671.c b/testcases/cve/cve-2017-2671.c
index e17cd9765..9092481de 100644
--- a/testcases/cve/cve-2017-2671.c
+++ b/testcases/cve/cve-2017-2671.c
@@ -109,7 +109,7 @@ static struct tst_test test = {
 	.test_all = run,
 	.cleanup = cleanup,
 	.needs_root = 1,
-	.max_runtime = 20,
+	.max_runtime = 40,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "43a6684519ab"},
 		{"CVE", "2017-2671"},
-- 
2.37.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH 03/10] ksm: Add max_runtime to tests

[Martin Doucha]

ksm02, ksm04 and ksm05 take 10+ seconds to finish. Set max_runtime to avoid
random timeout issues.

Signed-off-by: Martin Doucha <mdoucha@suse.cz>
---
 testcases/kernel/mem/ksm/ksm02.c | 1 +
 testcases/kernel/mem/ksm/ksm04.c | 1 +
 testcases/kernel/mem/ksm/ksm05.c | 1 +
 3 files changed, 3 insertions(+)

diff --git a/testcases/kernel/mem/ksm/ksm02.c b/testcases/kernel/mem/ksm/ksm02.c
index 1cb7d8e73..1f5677425 100644
--- a/testcases/kernel/mem/ksm/ksm02.c
+++ b/testcases/kernel/mem/ksm/ksm02.c
@@ -110,6 +110,7 @@ static struct tst_test test = {
 	},
 	.test_all = verify_ksm,
 	.min_kver = "2.6.32",
+	.max_runtime = 20,
 	.needs_cgroup_ctrls = (const char *const []){ "cpuset", NULL },
 };
 
diff --git a/testcases/kernel/mem/ksm/ksm04.c b/testcases/kernel/mem/ksm/ksm04.c
index 39c741876..f7dc5befc 100644
--- a/testcases/kernel/mem/ksm/ksm04.c
+++ b/testcases/kernel/mem/ksm/ksm04.c
@@ -112,6 +112,7 @@ static struct tst_test test = {
 	},
 	.test_all = verify_ksm,
 	.min_kver = "2.6.32",
+	.max_runtime = 20,
 	.needs_cgroup_ctrls = (const char *const []){
 		"memory", "cpuset", NULL
 	},
diff --git a/testcases/kernel/mem/ksm/ksm05.c b/testcases/kernel/mem/ksm/ksm05.c
index 146a9a3b7..6f94c4a9c 100644
--- a/testcases/kernel/mem/ksm/ksm05.c
+++ b/testcases/kernel/mem/ksm/ksm05.c
@@ -88,6 +88,7 @@ static struct tst_test test = {
 	.forks_child = 1,
 	.test_all = test_ksm,
 	.min_kver = "2.6.32",
+	.max_runtime = 10,
 	.save_restore = (const struct tst_path_val[]) {
 		{"!/sys/kernel/mm/ksm/run", "1"},
 		{}
-- 
2.37.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH 04/10] ioctl_sg01: Set unlimited max_runtime

[Martin Doucha]

Test setup can take very long time depending on the amount of free RAM.
Set unlimited max_runtime to avoid timeout issues.

Signed-off-by: Martin Doucha <mdoucha@suse.cz>
---
 testcases/kernel/syscalls/ioctl/ioctl_sg01.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/testcases/kernel/syscalls/ioctl/ioctl_sg01.c b/testcases/kernel/syscalls/ioctl/ioctl_sg01.c
index 94b30dc6b..4a5b0f35e 100644
--- a/testcases/kernel/syscalls/ioctl/ioctl_sg01.c
+++ b/testcases/kernel/syscalls/ioctl/ioctl_sg01.c
@@ -126,6 +126,7 @@ static struct tst_test test = {
 	.test_all = run,
 	.setup = setup,
 	.cleanup = cleanup,
+	.max_runtime = TST_UNLIMITED_RUNTIME,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "a45b599ad808"},
 		{"CVE", "2018-1000204"},
-- 
2.37.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH 05/10] csf_bandwidth01: Add max_runtime

[Martin Doucha]

Signed-off-by: Martin Doucha <mdoucha@suse.cz>
---
 testcases/kernel/sched/cfs-scheduler/cfs_bandwidth01.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/testcases/kernel/sched/cfs-scheduler/cfs_bandwidth01.c b/testcases/kernel/sched/cfs-scheduler/cfs_bandwidth01.c
index a24dccc4e..8f2da4493 100644
--- a/testcases/kernel/sched/cfs-scheduler/cfs_bandwidth01.c
+++ b/testcases/kernel/sched/cfs-scheduler/cfs_bandwidth01.c
@@ -170,6 +170,7 @@ static struct tst_test test = {
 	.cleanup = cleanup,
 	.forks_child = 1,
 	.needs_checkpoints = 1,
+	.max_runtime = 20,
 	.taint_check = TST_TAINT_W | TST_TAINT_D,
 	.needs_kconfigs = (const char *[]) {
 		"CONFIG_CFS_BANDWIDTH",
-- 
2.37.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH 06/10] copy_file_range01: Add max_runtime

[Martin Doucha]

Signed-off-by: Martin Doucha <mdoucha@suse.cz>
---
 testcases/kernel/syscalls/copy_file_range/copy_file_range01.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/testcases/kernel/syscalls/copy_file_range/copy_file_range01.c b/testcases/kernel/syscalls/copy_file_range/copy_file_range01.c
index 7d27007a3..23a5ec501 100644
--- a/testcases/kernel/syscalls/copy_file_range/copy_file_range01.c
+++ b/testcases/kernel/syscalls/copy_file_range/copy_file_range01.c
@@ -232,4 +232,5 @@ static struct tst_test test = {
 	.all_filesystems = 1,
 	.test = copy_file_range_verify,
 	.test_variants = TEST_VARIANTS,
+	.max_runtime = 20
 };
-- 
2.37.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH 07/10] perf_event_open02: Add max_runtime

[Martin Doucha]

Signed-off-by: Martin Doucha <mdoucha@suse.cz>
---
 testcases/kernel/syscalls/perf_event_open/perf_event_open02.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/testcases/kernel/syscalls/perf_event_open/perf_event_open02.c b/testcases/kernel/syscalls/perf_event_open/perf_event_open02.c
index 618a27016..b96578369 100644
--- a/testcases/kernel/syscalls/perf_event_open/perf_event_open02.c
+++ b/testcases/kernel/syscalls/perf_event_open/perf_event_open02.c
@@ -334,4 +334,5 @@ static struct tst_test test = {
 	},
 	.test_all = verify,
 	.needs_root = 1,
+	.max_runtime = 15
 };
-- 
2.37.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH 08/10] readahead02: Add max_runtime

[Martin Doucha]

Signed-off-by: Martin Doucha <mdoucha@suse.cz>
---
 testcases/kernel/syscalls/readahead/readahead02.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/testcases/kernel/syscalls/readahead/readahead02.c b/testcases/kernel/syscalls/readahead/readahead02.c
index 4fa8cfaf8..dc96eaef4 100644
--- a/testcases/kernel/syscalls/readahead/readahead02.c
+++ b/testcases/kernel/syscalls/readahead/readahead02.c
@@ -406,6 +406,7 @@ static struct tst_test test = {
 	},
 	.test = test_readahead,
 	.tcnt = ARRAY_SIZE(tcases),
+	.max_runtime = 30,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "b833a3660394"},
 		{"linux-git", "5b910bd615ba"},
-- 
2.37.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH 09/10] request_key03: Add max_runtime

[Martin Doucha]

Signed-off-by: Martin Doucha <mdoucha@suse.cz>
---
 testcases/kernel/syscalls/request_key/request_key03.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/testcases/kernel/syscalls/request_key/request_key03.c b/testcases/kernel/syscalls/request_key/request_key03.c
index 3f0c093f3..5cd9b319d 100644
--- a/testcases/kernel/syscalls/request_key/request_key03.c
+++ b/testcases/kernel/syscalls/request_key/request_key03.c
@@ -182,6 +182,7 @@ static void do_test(void)
 static struct tst_test test = {
 	.test_all = do_test,
 	.forks_child = 1,
+	.max_runtime = 10,
 	.options = (struct tst_option[]) {
 		{"b:", &opt_bug,  "Bug to test for (cve-2017-15299 or cve-2017-15951; default is both)"},
 		{}
-- 
2.37.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

[LTP] [PATCH 10/10] sendfile09: Add max_runtime

[Martin Doucha]

Signed-off-by: Martin Doucha <mdoucha@suse.cz>
---
 testcases/kernel/syscalls/sendfile/sendfile09.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/testcases/kernel/syscalls/sendfile/sendfile09.c b/testcases/kernel/syscalls/sendfile/sendfile09.c
index 320649dcd..353ef07d0 100644
--- a/testcases/kernel/syscalls/sendfile/sendfile09.c
+++ b/testcases/kernel/syscalls/sendfile/sendfile09.c
@@ -97,6 +97,7 @@ static struct tst_test test = {
 	.test = run,
 	.tcnt = ARRAY_SIZE(tc),
 	.min_kver = "2.6.33",
+	.max_runtime = 20,
 	.tags = (const struct tst_tag[]) {
 		{"linux-git", "5d73320a96fcc"},
 		{}
-- 
2.37.2


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 01/10] syscalls/pty06: Add CVE reference

[Cyril Hrubis]

Hi!
Applied, thanks.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 02/10] cve-2017-2671: Increase max_runtime

[Cyril Hrubis]

Hi!
Applied, thanks.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 03/10] ksm: Add max_runtime to tests

[Cyril Hrubis]

Hi!
> ksm02, ksm04 and ksm05 take 10+ seconds to finish. Set max_runtime to avoid
> random timeout issues.

I wonder if we can do better.

I guess that the actual runtime does depends on the size of the RAM
because we wait for at least two finished full scans for ksmd. I guess
that for large enough machines we would end up with minutes of runtime.

So I guess that it would make more sense to treat the max_runtime as a
upper bound and set it to large enough number as we do for AIO testcases
(30 minutes) and then make the wait_ksmd_full_scan() runtime avare so
that it exits when the runtime is exhausted. With that we would get a
clear message that we timed-out in the loop that waited for the ksmd
scan.

> Signed-off-by: Martin Doucha <mdoucha@suse.cz>
> ---
>  testcases/kernel/mem/ksm/ksm02.c | 1 +
>  testcases/kernel/mem/ksm/ksm04.c | 1 +
>  testcases/kernel/mem/ksm/ksm05.c | 1 +
>  3 files changed, 3 insertions(+)
> 
> diff --git a/testcases/kernel/mem/ksm/ksm02.c b/testcases/kernel/mem/ksm/ksm02.c
> index 1cb7d8e73..1f5677425 100644
> --- a/testcases/kernel/mem/ksm/ksm02.c
> +++ b/testcases/kernel/mem/ksm/ksm02.c
> @@ -110,6 +110,7 @@ static struct tst_test test = {
>  	},
>  	.test_all = verify_ksm,
>  	.min_kver = "2.6.32",
> +	.max_runtime = 20,
>  	.needs_cgroup_ctrls = (const char *const []){ "cpuset", NULL },
>  };
>  
> diff --git a/testcases/kernel/mem/ksm/ksm04.c b/testcases/kernel/mem/ksm/ksm04.c
> index 39c741876..f7dc5befc 100644
> --- a/testcases/kernel/mem/ksm/ksm04.c
> +++ b/testcases/kernel/mem/ksm/ksm04.c
> @@ -112,6 +112,7 @@ static struct tst_test test = {
>  	},
>  	.test_all = verify_ksm,
>  	.min_kver = "2.6.32",
> +	.max_runtime = 20,
>  	.needs_cgroup_ctrls = (const char *const []){
>  		"memory", "cpuset", NULL
>  	},
> diff --git a/testcases/kernel/mem/ksm/ksm05.c b/testcases/kernel/mem/ksm/ksm05.c
> index 146a9a3b7..6f94c4a9c 100644
> --- a/testcases/kernel/mem/ksm/ksm05.c
> +++ b/testcases/kernel/mem/ksm/ksm05.c
> @@ -88,6 +88,7 @@ static struct tst_test test = {
>  	.forks_child = 1,
>  	.test_all = test_ksm,
>  	.min_kver = "2.6.32",
> +	.max_runtime = 10,
>  	.save_restore = (const struct tst_path_val[]) {
>  		{"!/sys/kernel/mm/ksm/run", "1"},
>  		{}
> -- 
> 2.37.2
> 
> 
> -- 
> Mailing list info: https://lists.linux.it/listinfo/ltp

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 04/10] ioctl_sg01: Set unlimited max_runtime

[Cyril Hrubis]

Hi!
> Test setup can take very long time depending on the amount of free RAM.
> Set unlimited max_runtime to avoid timeout issues.

I would rather go for large enough number e.g. 30 minutes than
unlimited as the unlimited runtime turns off the timeouts and such test
can run undefinitely.

The unlimited runtime is really meant for unpredictable tests, such as
OOM testcases.

> Signed-off-by: Martin Doucha <mdoucha@suse.cz>
> ---
>  testcases/kernel/syscalls/ioctl/ioctl_sg01.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/testcases/kernel/syscalls/ioctl/ioctl_sg01.c b/testcases/kernel/syscalls/ioctl/ioctl_sg01.c
> index 94b30dc6b..4a5b0f35e 100644
> --- a/testcases/kernel/syscalls/ioctl/ioctl_sg01.c
> +++ b/testcases/kernel/syscalls/ioctl/ioctl_sg01.c
> @@ -126,6 +126,7 @@ static struct tst_test test = {
>  	.test_all = run,
>  	.setup = setup,
>  	.cleanup = cleanup,
> +	.max_runtime = TST_UNLIMITED_RUNTIME,
>  	.tags = (const struct tst_tag[]) {
>  		{"linux-git", "a45b599ad808"},
>  		{"CVE", "2018-1000204"},
> -- 
> 2.37.2
> 
> 
> -- 
> Mailing list info: https://lists.linux.it/listinfo/ltp

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 04/10] ioctl_sg01: Set unlimited max_runtime

[Martin Doucha]

On 30. 08. 22 17:16, Cyril Hrubis wrote:
> Hi!
>> Test setup can take very long time depending on the amount of free RAM.
>> Set unlimited max_runtime to avoid timeout issues.
> 
> I would rather go for large enough number e.g. 30 minutes than
> unlimited as the unlimited runtime turns off the timeouts and such test
> can run undefinitely.

Even setting max_runtime is an ugly hack here because the test itself is 
nearly instant. The setup() function is what actually times out. But 
there's currently no way to change the default 30 second timeout 
allocated for setup() and cleanup(). It'd be much better if I could set 
that setup() has unlimited timeout and the test itself is limited to 5 
seconds.

-- 
Martin Doucha   mdoucha@suse.cz
QA Engineer for Software Maintenance
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 05/10] csf_bandwidth01: Add max_runtime

[Cyril Hrubis]

Hi!
Applied, thanks.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 04/10] ioctl_sg01: Set unlimited max_runtime

[Cyril Hrubis]

Hi!
> >> Test setup can take very long time depending on the amount of free RAM.
> >> Set unlimited max_runtime to avoid timeout issues.
> > 
> > I would rather go for large enough number e.g. 30 minutes than
> > unlimited as the unlimited runtime turns off the timeouts and such test
> > can run undefinitely.
> 
> Even setting max_runtime is an ugly hack here because the test itself is 
> nearly instant. The setup() function is what actually times out. But 
> there's currently no way to change the default 30 second timeout 
> allocated for setup() and cleanup(). It'd be much better if I could set 
> that setup() has unlimited timeout and the test itself is limited to 5 
> seconds.

I guess that we can add setup_max_runtime and cleanup_max_runtime
however I would still set the setup_max_runtime to a large but finite
value here.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 06/10] copy_file_range01: Add max_runtime

[Petr Vorel]

Hi Martin,

LGTM (test uses all_filesystems, but 20 for all possible filesystems should be OK).

Reviewed-by: Petr Vorel <pvorel@suse.cz>

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 07/10] perf_event_open02: Add max_runtime

[Petr Vorel]

Hi,

Reviewed-by: Petr Vorel <pvorel@suse.cz>

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 06/10] copy_file_range01: Add max_runtime

[Cyril Hrubis]

Hi!
> LGTM (test uses all_filesystems, but 20 for all possible filesystems should be OK).

The runtime applies per single call of the run() function.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 06/10] copy_file_range01: Add max_runtime

[Martin Doucha]

On 31. 08. 22 10:21, Petr Vorel wrote:
> Hi Martin,
> 
> LGTM (test uses all_filesystems, but 20 for all possible filesystems should be OK).
Hmm, I've missed the .all_filesystems setting here. In that case the 
test should take ~2 seconds per filesystems and the patch is not needed. 
Feel free to discard it then.

-- 
Martin Doucha   mdoucha@suse.cz
QA Engineer for Software Maintenance
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 06/10] copy_file_range01: Add max_runtime

[Cyril Hrubis]

Hi!
> Signed-off-by: Martin Doucha <mdoucha@suse.cz>
> ---
>  testcases/kernel/syscalls/copy_file_range/copy_file_range01.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/testcases/kernel/syscalls/copy_file_range/copy_file_range01.c b/testcases/kernel/syscalls/copy_file_range/copy_file_range01.c
> index 7d27007a3..23a5ec501 100644
> --- a/testcases/kernel/syscalls/copy_file_range/copy_file_range01.c
> +++ b/testcases/kernel/syscalls/copy_file_range/copy_file_range01.c
> @@ -232,4 +232,5 @@ static struct tst_test test = {
>  	.all_filesystems = 1,
>  	.test = copy_file_range_verify,
>  	.test_variants = TEST_VARIANTS,
> +	.max_runtime = 20
>  };

I would still make the loops that copy and read the data runtime aware
to make the error messages more obvious. In the case that you want to
keep TBROK return value it would be:

diff --git a/testcases/kernel/syscalls/copy_file_range/copy_file_range01.c b/testcases/kernel/syscalls/copy_file_range/copy_file_range01.c
index 7d27007a3..9b39554d6 100644
--- a/testcases/kernel/syscalls/copy_file_range/copy_file_range01.c
+++ b/testcases/kernel/syscalls/copy_file_range/copy_file_range01.c
@@ -52,6 +52,10 @@ static int check_file_content(const char *fname1, const char *fname2,
                ch1 = getc(fp1);
                ch2 = getc(fp2);
                count++;
+
+               if (!(count%100) && !tst_remaining_runtime())
+                       tst_brk(TBROK, "Out of runtime, exiting");
+
        } while ((count < len) && (ch1 == ch2));

        SAFE_FCLOSE(fp1);
@@ -135,6 +139,9 @@ static void test_one(size_t len, loff_t *off_in, loff_t *off_out, char *path)
                        return;
                }

+               if (!tst_remaining_runtime())
+                       tst_brk(TBROK, "Out of runtime, exiting");
+
                to_copy -= TST_RET;
        } while (to_copy > 0);

@@ -230,6 +237,7 @@ static struct tst_test test = {
        .mount_device = 1,
        .mntpoint = MNTPOINT,
        .all_filesystems = 1,
+       .max_runtime = 20,
        .test = copy_file_range_verify,
        .test_variants = TEST_VARIANTS,
 };

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 06/10] copy_file_range01: Add max_runtime

[Cyril Hrubis]

Hi!
> > LGTM (test uses all_filesystems, but 20 for all possible filesystems should be OK).
> Hmm, I've missed the .all_filesystems setting here. In that case the 
> test should take ~2 seconds per filesystems and the patch is not needed. 
> Feel free to discard it then.

I would still vote for adding something as .max_runtime = 5 or something
like this so that it's obvious that the test can run for a few seconds
per run() call especially on slower disks.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 08/10] readahead02: Add max_runtime

[Petr Vorel]

Hi Martin,

Reviewed-by: Petr Vorel <pvorel@suse.cz>

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 09/10] request_key03: Add max_runtime

[Petr Vorel]

Hi Martin,

Reviewed-by: Petr Vorel <pvorel@suse.cz>

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 09/10] request_key03: Add max_runtime

[Petr Vorel]

Hi Martin,

Reviewed-by: Petr Vorel <pvorel@suse.cz>

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 10/10] sendfile09: Add max_runtime

[Petr Vorel]

Hi Martin,

Reviewed-by: Petr Vorel <pvorel@suse.cz>

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 07/10] perf_event_open02: Add max_runtime

[Cyril Hrubis]

Hi!
> Signed-off-by: Martin Doucha <mdoucha@suse.cz>
> ---
>  testcases/kernel/syscalls/perf_event_open/perf_event_open02.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/testcases/kernel/syscalls/perf_event_open/perf_event_open02.c b/testcases/kernel/syscalls/perf_event_open/perf_event_open02.c
> index 618a27016..b96578369 100644
> --- a/testcases/kernel/syscalls/perf_event_open/perf_event_open02.c
> +++ b/testcases/kernel/syscalls/perf_event_open/perf_event_open02.c
> @@ -334,4 +334,5 @@ static struct tst_test test = {
>  	},
>  	.test_all = verify,
>  	.needs_root = 1,
> +	.max_runtime = 15
>  };

We do spend significant part of the runtime in the setup while we count
hardware counters so if we are going to add .setup_max_runtime this
should something as:

..
	.setup_max_runtime = 64,
	.max_runtime = 8,
...


Technically the maximal time the setup can run is MAX_CTRS * 2 but I do
not think that we will see hardware with more than 32 hardware counters
anytime soon if ever.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 08/10] readahead02: Add max_runtime

[Cyril Hrubis]

Hi!
> Signed-off-by: Martin Doucha <mdoucha@suse.cz>
> ---
>  testcases/kernel/syscalls/readahead/readahead02.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/testcases/kernel/syscalls/readahead/readahead02.c b/testcases/kernel/syscalls/readahead/readahead02.c
> index 4fa8cfaf8..dc96eaef4 100644
> --- a/testcases/kernel/syscalls/readahead/readahead02.c
> +++ b/testcases/kernel/syscalls/readahead/readahead02.c
> @@ -406,6 +406,7 @@ static struct tst_test test = {
>  	},
>  	.test = test_readahead,
>  	.tcnt = ARRAY_SIZE(tcases),
> +	.max_runtime = 30,
>  	.tags = (const struct tst_tag[]) {
>  		{"linux-git", "b833a3660394"},
>  		{"linux-git", "5b910bd615ba"},

There are actually couple of things to consider here. First of all the
testfile_size defaults to 64MB but it can be changed by a command line
parameter, so we can easily overrun the 30 seconds by asking for large
enough file. The best solution would be to add runtime 30 seconds as
this patch does but if user passes -s parameter adjust the runtime in
the test setup as well. Something as (30 * size / 64_MB) should be
better than nothing.

However as we run the test with a loop device it will fail with ENOSPC
if we pass -s bigger than the loop device, we should at least check if
the device is large enough in the test setup too.

Also as we run the test inside a loop device I guess that we can also
sync and drop caches just for the device, which should be faster than
syncing and dropping the whole system. Possibly we just need to umount
it and mount it again.

And lastly we should as well use tst_parse_filesize() so that we can
pass -s 128M but that is very minor.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 06/10] copy_file_range01: Add max_runtime

[Petr Vorel]

> Hi!
> > LGTM (test uses all_filesystems, but 20 for all possible filesystems should be OK).

> The runtime applies per single call of the run() function.

Thanks for a hint Cyril. I didn't read the sources correctly before. Now I see
alarm(results->timeout) in fork_testrun() which is called for each filesystem.

Kind regards,
Petr


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 06/10] copy_file_range01: Add max_runtime

[Petr Vorel]

> Hi!
> > > LGTM (test uses all_filesystems, but 20 for all possible filesystems should be OK).
> > Hmm, I've missed the .all_filesystems setting here. In that case the 
> > test should take ~2 seconds per filesystems and the patch is not needed. 
> > Feel free to discard it then.

> I would still vote for adding something as .max_runtime = 5 or something
> like this so that it's obvious that the test can run for a few seconds
> per run() call especially on slower disks.

+1

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 09/10] request_key03: Add max_runtime

[Cyril Hrubis]

Hi!
> Signed-off-by: Martin Doucha <mdoucha@suse.cz>
> ---
>  testcases/kernel/syscalls/request_key/request_key03.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/testcases/kernel/syscalls/request_key/request_key03.c b/testcases/kernel/syscalls/request_key/request_key03.c
> index 3f0c093f3..5cd9b319d 100644
> --- a/testcases/kernel/syscalls/request_key/request_key03.c
> +++ b/testcases/kernel/syscalls/request_key/request_key03.c
> @@ -182,6 +182,7 @@ static void do_test(void)
>  static struct tst_test test = {
>  	.test_all = do_test,
>  	.forks_child = 1,
> +	.max_runtime = 10,
>  	.options = (struct tst_option[]) {
>  		{"b:", &opt_bug,  "Bug to test for (cve-2017-15299 or cve-2017-15951; default is both)"},
>  		{}

First of all I would split the do_test() into three subtest by setting
.test and .tcnt = 3 so that we can apply the max_runtime per one call of
the test_with_key_type(). Then we can make the test_with_key_type()
runtime aware. I guess that we can just spin both of the children until
they consume a defined portion of runtime.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 03/10] ksm: Add max_runtime to tests

[Martin Doucha]

On 30. 08. 22 16:49, Cyril Hrubis wrote:
> Hi!
>> ksm02, ksm04 and ksm05 take 10+ seconds to finish. Set max_runtime to avoid
>> random timeout issues.
> 
> I wonder if we can do better.
> 
> I guess that the actual runtime does depends on the size of the RAM
> because we wait for at least two finished full scans for ksmd. I guess
> that for large enough machines we would end up with minutes of runtime.
> 
> So I guess that it would make more sense to treat the max_runtime as a
> upper bound and set it to large enough number as we do for AIO testcases
> (30 minutes) and then make the wait_ksmd_full_scan() runtime avare so
> that it exits when the runtime is exhausted. With that we would get a
> clear message that we timed-out in the loop that waited for the ksmd
> scan.

Alternatively, we could measure 1 full ksmd scan in setup() and then set 
max_runtime dynamically. Each call of create_same_memory() would need 
roughly 16 scan times. Time spent in ksm_child_memset() is included in 
that estimate.

-- 
Martin Doucha   mdoucha@suse.cz
QA Engineer for Software Maintenance
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 03/10] ksm: Add max_runtime to tests

[Cyril Hrubis]

Hi!
> > I wonder if we can do better.
> > 
> > I guess that the actual runtime does depends on the size of the RAM
> > because we wait for at least two finished full scans for ksmd. I guess
> > that for large enough machines we would end up with minutes of runtime.
> > 
> > So I guess that it would make more sense to treat the max_runtime as a
> > upper bound and set it to large enough number as we do for AIO testcases
> > (30 minutes) and then make the wait_ksmd_full_scan() runtime avare so
> > that it exits when the runtime is exhausted. With that we would get a
> > clear message that we timed-out in the loop that waited for the ksmd
> > scan.
> 
> Alternatively, we could measure 1 full ksmd scan in setup() and then set 
> max_runtime dynamically. Each call of create_same_memory() would need 
> roughly 16 scan times. Time spent in ksm_child_memset() is included in 
> that estimate.

That sounds good as well, but I would still set the .max_runtime to
rough guess in tst_test structure and then adjusted it in the test
setup().

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 03/10] ksm: Add max_runtime to tests

[Martin Doucha]

On 31. 08. 22 14:50, Cyril Hrubis wrote:
> Hi!
>> Alternatively, we could measure 1 full ksmd scan in setup() and then set
>> max_runtime dynamically. Each call of create_same_memory() would need
>> roughly 16 scan times. Time spent in ksm_child_memset() is included in
>> that estimate.
> 
> That sounds good as well, but I would still set the .max_runtime to
> rough guess in tst_test structure and then adjusted it in the test
> setup().

The current patch is a good enough guess for ~2-4GB machines. Or do you 
want to target bigger machines by default?

-- 
Martin Doucha   mdoucha@suse.cz
QA Engineer for Software Maintenance
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 10/10] sendfile09: Add max_runtime

[Jan Stancek]

On Tue, Aug 30, 2022 at 3:51 PM Martin Doucha <mdoucha@suse.cz> wrote:
>
> Signed-off-by: Martin Doucha <mdoucha@suse.cz>
> ---
>  testcases/kernel/syscalls/sendfile/sendfile09.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/testcases/kernel/syscalls/sendfile/sendfile09.c b/testcases/kernel/syscalls/sendfile/sendfile09.c
> index 320649dcd..353ef07d0 100644
> --- a/testcases/kernel/syscalls/sendfile/sendfile09.c
> +++ b/testcases/kernel/syscalls/sendfile/sendfile09.c
> @@ -97,6 +97,7 @@ static struct tst_test test = {
>         .test = run,
>         .tcnt = ARRAY_SIZE(tc),
>         .min_kver = "2.6.33",
> +       .max_runtime = 20,

I already saw reports of this timing out with 60 second timeout on
bare metal, so I'd go with higher value:

tst_test.c:1526: TINFO: Timeout per run is 0h 01m 00s
Test timeouted, sending SIGKILL!
tst_test.c:1577: TINFO: If you are running on slow machine, try
exporting LTP_TIMEOUT_MUL > 1
tst_test.c:1579: TBROK: Test killed! (timeout?)


>         .tags = (const struct tst_tag[]) {
>                 {"linux-git", "5d73320a96fcc"},
>                 {}
> --
> 2.37.2
>
>
> --
> Mailing list info: https://lists.linux.it/listinfo/ltp
>


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 03/10] ksm: Add max_runtime to tests

[Petr Vorel]

> On 31. 08. 22 14:50, Cyril Hrubis wrote:
> > Hi!
> > > Alternatively, we could measure 1 full ksmd scan in setup() and then set
> > > max_runtime dynamically. Each call of create_same_memory() would need
> > > roughly 16 scan times. Time spent in ksm_child_memset() is included in
> > > that estimate.

> > That sounds good as well, but I would still set the .max_runtime to
> > rough guess in tst_test structure and then adjusted it in the test
> > setup().

> The current patch is a good enough guess for ~2-4GB machines. Or do you want
> to target bigger machines by default?

I guess it'd be safer to expect machines with bigger memory.

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 03/10] ksm: Add max_runtime to tests

[Cyril Hrubis]

Hi!
> > > > Alternatively, we could measure 1 full ksmd scan in setup() and then set
> > > > max_runtime dynamically. Each call of create_same_memory() would need
> > > > roughly 16 scan times. Time spent in ksm_child_memset() is included in
> > > > that estimate.
> 
> > > That sounds good as well, but I would still set the .max_runtime to
> > > rough guess in tst_test structure and then adjusted it in the test
> > > setup().
> 
> > The current patch is a good enough guess for ~2-4GB machines. Or do you want
> > to target bigger machines by default?
> 
> I guess it'd be safer to expect machines with bigger memory.

I would just multiply the value you proposed by 10, which should be
large enough default.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp