Possible bug - LTP failure for memcg

Possible bug - LTP failure for memcg

[Nikolay Borisov]

Hello,

I'm running the ltp version 20150420 and stock kernel 4.0 and I've
observed that the memcg_function test is failing. Here is a relevant
snipped from the log:


memcg_function_test   14  TFAIL  :  ltpapicmd.c:190: process 5827 is not
killed
/opt/ltp/testcases/bin/memcg_lib.sh: line 210:  5843 Killed
     $TEST_PATH/memcg_process $2 -s $3
memcg_function_test   15  TPASS  :  process 5843 is killed
/opt/ltp/testcases/bin/memcg_lib.sh: line 210:  5859 Killed
     $TEST_PATH/memcg_process $2 -s $3
memcg_function_test   16  TPASS  :  process 5859 is killed
/opt/ltp/testcases/bin/memcg_lib.sh: line 210:  5877 Killed
     $TEST_PATH/memcg_process $2 -s $3
memcg_function_test   17  TPASS  :  process 5877 is killed
/opt/ltp/testcases/bin/memcg_lib.sh: line 210:  5894 Killed
     $TEST_PATH/memcg_process $2 -s $3
memcg_function_test   18  TPASS  :  process 5894 is killed
/opt/ltp/testcases/bin/memcg_lib.sh: line 210:  5911 Killed
     $TEST_PATH/memcg_process $2 -s $3
memcg_function_test   19  TPASS  :  process 5911 is killed
/opt/ltp/testcases/bin/memcg_lib.sh: line 210:  5927 Killed
     $TEST_PATH/memcg_process $2 -s $3
memcg_function_test   20  TPASS  :  process 5927 is killed
/opt/ltp/testcases/bin/memcg_lib.sh: line 210:  5942 Killed
     $TEST_PATH/memcg_process $2 -s $3
memcg_function_test   21  TPASS  :  process 5942 is killed
memcg_function_test   22  TFAIL  :  ltpapicmd.c:190: input=4095,
limit_in_bytes=0
memcg_function_test   23  TFAIL  :  ltpapicmd.c:190: input=4097,
limit_in_bytes=4096
memcg_function_test   24  TFAIL  :  ltpapicmd.c:190: input=1,
limit_in_bytes=0
memcg_function_test   25  TPASS  :  return value is 0
memcg_function_test   26  TPASS  :  return value is 1
memcg_function_test   27  TPASS  :  return value is 1
memcg_function_test   28  TPASS  :  return value is 1
memcg_function_test   29  TPASS  :  force memory succeeded
memcg_function_test   30  TFAIL  :  ltpapicmd.c:190: force memory should
fail
memcg_function_test   31  TPASS  :  return value is 0
memcg_function_test   32  TPASS  :  return value is 0
memcg_function_test   33  TPASS  :  return value is 0
memcg_function_test   34  TPASS  :  return value is 0
memcg_function_test   35  TPASS  :  return value is 1
Running /opt/ltp/testcases/bin/memcg_process --mmap-anon -s 4096
Warming up for test: 36, pid: 6128
Process is still here after warm up: 6128
memcg_function_test   36  TPASS  :  rss=4096/4096
memcg_function_test   36  TPASS  :  rss=0/0
Running /opt/ltp/testcases/bin/memcg_process --mmap-anon -s 4096
Warming up for test: 37, pid: 6155
Process is still here after warm up: 6155
memcg_function_test   37  TPASS  :  rss=4096/4096
memcg_function_test   37  TPASS  :  rss=0/0
Running /opt/ltp/testcases/bin/memcg_process --mmap-anon -s 4096
Warming up for test: 38, pid: 6182
Process is still here after warm up: 6182
memcg_function_test   38  TPASS  :  rss=4096/4096
memcg_function_test   38  TPASS  :  rss=0/0
<<<execution_status>>>
initiation_status="ok"
duration=135 termination_type=exited termination_id=5 corefile=no
cutime=8 cstime=15
<<<test_end>>>
INFO: ltp-pan reported some tests FAIL
LTP Version: 20150420

According to the file at :
https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh


The failing test cases 14, 22, 23, 24 and 30 test respectively:

14: Hogging memory like so: mmap(NULL, memsize, PROT_WRITE | PROT_READ,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_LOCKED, 0, 0);

# Case 22 - 24: Test limit_in_bytes will be aligned to PAGESIZE - The
output clearly indicates that the limits in bytes is not being page
aligned? Is this desired behavior, in which case ltp is broken or is it
a genuine bug?

30: Again, it locks memory with mmap and then tries to see if
force_empty would succeed. Expecting it to fail, but in this particular
case it succeeds?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Possible bug - LTP failure for memcg

[Michal Hocko]

On Wed 13-05-15 18:29:13, Nikolay Borisov wrote:
> Hello,
> 
> I'm running the ltp version 20150420 and stock kernel 4.0 and I've
> observed that the memcg_function test is failing. Here is a relevant
> snipped from the log:
> 
> 
> memcg_function_test   14  TFAIL  :  ltpapicmd.c:190: process 5827 is not
> killed
> /opt/ltp/testcases/bin/memcg_lib.sh: line 210:  5843 Killed
>      $TEST_PATH/memcg_process $2 -s $3
> memcg_function_test   15  TPASS  :  process 5843 is killed
> /opt/ltp/testcases/bin/memcg_lib.sh: line 210:  5859 Killed
>      $TEST_PATH/memcg_process $2 -s $3
> memcg_function_test   16  TPASS  :  process 5859 is killed
> /opt/ltp/testcases/bin/memcg_lib.sh: line 210:  5877 Killed
>      $TEST_PATH/memcg_process $2 -s $3
> memcg_function_test   17  TPASS  :  process 5877 is killed
> /opt/ltp/testcases/bin/memcg_lib.sh: line 210:  5894 Killed
>      $TEST_PATH/memcg_process $2 -s $3
> memcg_function_test   18  TPASS  :  process 5894 is killed
> /opt/ltp/testcases/bin/memcg_lib.sh: line 210:  5911 Killed
>      $TEST_PATH/memcg_process $2 -s $3
> memcg_function_test   19  TPASS  :  process 5911 is killed
> /opt/ltp/testcases/bin/memcg_lib.sh: line 210:  5927 Killed
>      $TEST_PATH/memcg_process $2 -s $3
> memcg_function_test   20  TPASS  :  process 5927 is killed
> /opt/ltp/testcases/bin/memcg_lib.sh: line 210:  5942 Killed
>      $TEST_PATH/memcg_process $2 -s $3
> memcg_function_test   21  TPASS  :  process 5942 is killed
> memcg_function_test   22  TFAIL  :  ltpapicmd.c:190: input=4095,
> limit_in_bytes=0
> memcg_function_test   23  TFAIL  :  ltpapicmd.c:190: input=4097,
> limit_in_bytes=4096
> memcg_function_test   24  TFAIL  :  ltpapicmd.c:190: input=1,
> limit_in_bytes=0
> memcg_function_test   25  TPASS  :  return value is 0
> memcg_function_test   26  TPASS  :  return value is 1
> memcg_function_test   27  TPASS  :  return value is 1
> memcg_function_test   28  TPASS  :  return value is 1
> memcg_function_test   29  TPASS  :  force memory succeeded
> memcg_function_test   30  TFAIL  :  ltpapicmd.c:190: force memory should
> fail
> memcg_function_test   31  TPASS  :  return value is 0
> memcg_function_test   32  TPASS  :  return value is 0
> memcg_function_test   33  TPASS  :  return value is 0
> memcg_function_test   34  TPASS  :  return value is 0
> memcg_function_test   35  TPASS  :  return value is 1
> Running /opt/ltp/testcases/bin/memcg_process --mmap-anon -s 4096
> Warming up for test: 36, pid: 6128
> Process is still here after warm up: 6128
> memcg_function_test   36  TPASS  :  rss=4096/4096
> memcg_function_test   36  TPASS  :  rss=0/0
> Running /opt/ltp/testcases/bin/memcg_process --mmap-anon -s 4096
> Warming up for test: 37, pid: 6155
> Process is still here after warm up: 6155
> memcg_function_test   37  TPASS  :  rss=4096/4096
> memcg_function_test   37  TPASS  :  rss=0/0
> Running /opt/ltp/testcases/bin/memcg_process --mmap-anon -s 4096
> Warming up for test: 38, pid: 6182
> Process is still here after warm up: 6182
> memcg_function_test   38  TPASS  :  rss=4096/4096
> memcg_function_test   38  TPASS  :  rss=0/0
> <<<execution_status>>>
> initiation_status="ok"
> duration=135 termination_type=exited termination_id=5 corefile=no
> cutime=8 cstime=15
> <<<test_end>>>
> INFO: ltp-pan reported some tests FAIL
> LTP Version: 20150420
> 
> According to the file at :
> https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
> 
> 
> The failing test cases 14, 22, 23, 24 and 30 test respectively:
> 
> 14: Hogging memory like so: mmap(NULL, memsize, PROT_WRITE | PROT_READ,
> MAP_PRIVATE | MAP_ANONYMOUS | MAP_LOCKED, 0, 0);

MAP_LOCKED will not trigger the OOM killer as explained
http://marc.info/?l=linux-mm&m=142122902313315&w=2. So this is expected
and Cyril is working on fixing the test case.

> # Case 22 - 24: Test limit_in_bytes will be aligned to PAGESIZE - The
> output clearly indicates that the limits in bytes is not being page
> aligned?

I can see
> memcg_function_test   22  TFAIL  :  ltpapicmd.c:190: input=4095,
> limit_in_bytes=0
> memcg_function_test   23  TFAIL  :  ltpapicmd.c:190: input=4097,
> limit_in_bytes=4096
> memcg_function_test   24  TFAIL  :  ltpapicmd.c:190: input=1,
> limit_in_bytes=0

So limit_in_bytes _is_ page aligned but we round down rather than up.

> Is this desired behavior, in which case ltp is broken or is it
> a genuine bug?

This behavior has changed by 3e32cb2e0a12 ("mm: memcontrol: lockless
page counters") introduced in 3.19. The change in rounding has been
pointed out during the review
http://marc.info/?l=linux-mm&m=141207518827336&w=2 but the conclusion
was that the original round up wasn't really much better
http://marc.info/?l=linux-mm&m=141226210316376&w=2 resp.
http://marc.info/?l=linux-mm&m=141234785111200&w=2

I will post fix for ltp in the reply

> 30: Again, it locks memory with mmap and then tries to see if
> force_empty would succeed. Expecting it to fail, but in this particular
> case it succeeds?

I am not sure I understand this testcase. It does:
	TEST_PATH/memcg_process --mmap-anon -s $PAGESIZE
	[...]
        echo 1 > memory.force_empty 2> /dev/null
        if [ $? -ne 0 ]; then
                result $PASS "force memory failed as expected"
        else    
                result $FAIL "force memory should fail"
        fi

and that means:
void mmap_anon()
{               
        static char *p; 
        
        if (!flag_allocated) {
                p = mmap(NULL, memsize, PROT_WRITE | PROT_READ,
                         MAP_PRIVATE | MAP_ANONYMOUS, 0, 0);
                if (p == MAP_FAILED)
                        err(1, "mmap(anonymous) failed");
                touch_memory(p, memsize);
        } else {
                if (munmap(p, memsize) == -1)
                        err(1, "munmap(anonymous) failed");
        }
}

so there is no mlock there. Why should the force reclaim fail then?

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Possible bug - LTP failure for memcg

[Michal Hocko]

Cyril, what about this to fix the rounding issue?
---

Re: Possible bug - LTP failure for memcg

[Michal Hocko]

Cyril, what about this to fix the rounding issue?
---
From b83d740193490d78547197f47eee918732ed1f60 Mon Sep 17 00:00:00 2001
From: Michal Hocko <mhocko-AlSwsSmVLrQ@public.gmane.org>
Date: Thu, 14 May 2015 11:08:20 +0200
Subject: [PATCH] controllers/memcg: Fix limit alignment expectations

since 3e32cb2e0a12 ("mm: memcontrol: lockless page counters") kernel
commit the limit_in_bytes is round down rather than up. This behavior
change has been discussed during the review and it was considered a
reasonable so fix the test accordingly.

While we are at it remove the test case 24 as it doesn't really test
anything more than test case 22.

Signed-off-by: Michal Hocko <mhocko-AlSwsSmVLrQ@public.gmane.org>
---
 .../kernel/controllers/memcg/functional/memcg_function_test.sh   | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)

diff --git a/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh b/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
index cfc75fa730df..e3baf9d032f6 100755
--- a/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
+++ b/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
@@ -158,17 +158,12 @@ testcase_21()
 # Case 22 - 24: Test limit_in_bytes will be aligned to PAGESIZE
 testcase_22()
 {
-	test_limit_in_bytes $((PAGESIZE-1)) $PAGESIZE 0
+	test_limit_in_bytes $((PAGESIZE-1)) 0 0
 }
 
 testcase_23()
 {
-	test_limit_in_bytes $((PAGESIZE+1)) $((PAGESIZE*2)) 0
-}
-
-testcase_24()
-{
-	test_limit_in_bytes 1 $PAGESIZE 0
+	test_limit_in_bytes $((PAGESIZE+1)) $((PAGESIZE)) 0
 }
 
 # Case 25 - 28: Test invaild memory.limit_in_bytes
-- 
2.1.4

-- 
Michal Hocko
SUSE Labs

Re: Possible bug - LTP failure for memcg

[Cyril Hrubis]

Hi!
> --- a/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
> +++ b/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
> @@ -158,17 +158,12 @@ testcase_21()
>  # Case 22 - 24: Test limit_in_bytes will be aligned to PAGESIZE
>  testcase_22()
>  {
> -	test_limit_in_bytes $((PAGESIZE-1)) $PAGESIZE 0
> +	test_limit_in_bytes $((PAGESIZE-1)) 0 0
>  }
>  
>  testcase_23()
>  {
> -	test_limit_in_bytes $((PAGESIZE+1)) $((PAGESIZE*2)) 0
> -}
> -
> -testcase_24()
> -{
> -	test_limit_in_bytes 1 $PAGESIZE 0
> +	test_limit_in_bytes $((PAGESIZE+1)) $((PAGESIZE)) 0
>  }

That would fail on older kernels without the patch, woudln't it?

If we are going to fix it, we should do that in backward compatible
fashion. So either we modify the testcases to accept both rounding up
and rounding down or choose what we expect based on kernel version.

-- 
Cyril Hrubis
chrubis@suse.cz

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Possible bug - LTP failure for memcg

[Michal Hocko]

On Thu 14-05-15 12:35:43, Cyril Hrubis wrote:
> Hi!
> > --- a/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
> > +++ b/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
> > @@ -158,17 +158,12 @@ testcase_21()
> >  # Case 22 - 24: Test limit_in_bytes will be aligned to PAGESIZE
> >  testcase_22()
> >  {
> > -	test_limit_in_bytes $((PAGESIZE-1)) $PAGESIZE 0
> > +	test_limit_in_bytes $((PAGESIZE-1)) 0 0
> >  }
> >  
> >  testcase_23()
> >  {
> > -	test_limit_in_bytes $((PAGESIZE+1)) $((PAGESIZE*2)) 0
> > -}
> > -
> > -testcase_24()
> > -{
> > -	test_limit_in_bytes 1 $PAGESIZE 0
> > +	test_limit_in_bytes $((PAGESIZE+1)) $((PAGESIZE)) 0
> >  }
> 
> That would fail on older kernels without the patch, woudln't it?

Yes it will. I thought those would be using some stable release (I do
not have much idea about the release process of ltp...). You are
definitely right that a backward compatible way is better. I will cook
up a patch later today.

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Possible bug - LTP failure for memcg

[Cyril Hrubis]

Hi!
> > That would fail on older kernels without the patch, woudln't it?
> 
> Yes it will. I thought those would be using some stable release (I do
> not have much idea about the release process of ltp...). You are
> definitely right that a backward compatible way is better. I will cook
> up a patch later today.

The thing is that we do not have manpower to backport fixes to stable
releases. So the latest stable release is always recomended for testing
and because of that we have to fix testcases in backward compatible way.

-- 
Cyril Hrubis
chrubis@suse.cz

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Possible bug - LTP failure for memcg

[Cyril Hrubis]

Hi!
> > That would fail on older kernels without the patch, woudln't it?
> 
> Yes it will. I thought those would be using some stable release (I do
> not have much idea about the release process of ltp...). You are
> definitely right that a backward compatible way is better. I will cook
> up a patch later today.

The thing is that we do not have manpower to backport fixes to stable
releases. So the latest stable release is always recomended for testing
and because of that we have to fix testcases in backward compatible way.

-- 
Cyril Hrubis
chrubis-AlSwsSmVLrQ@public.gmane.org

Re: Possible bug - LTP failure for memcg

[Cyril Hrubis]

Hi!
> > The failing test cases 14, 22, 23, 24 and 30 test respectively:
> > 
> > 14: Hogging memory like so: mmap(NULL, memsize, PROT_WRITE | PROT_READ,
> > MAP_PRIVATE | MAP_ANONYMOUS | MAP_LOCKED, 0, 0);
> 
> MAP_LOCKED will not trigger the OOM killer as explained
> http://marc.info/?l=linux-mm&m=142122902313315&w=2. So this is expected
> and Cyril is working on fixing the test case.

It's on my TODO, I should get to this i a few weeks. Till then just
ignore the failure.

> > # Case 22 - 24: Test limit_in_bytes will be aligned to PAGESIZE - The
> > output clearly indicates that the limits in bytes is not being page
> > aligned?
> 
> I can see
> > memcg_function_test   22  TFAIL  :  ltpapicmd.c:190: input=4095,
> > limit_in_bytes=0
> > memcg_function_test   23  TFAIL  :  ltpapicmd.c:190: input=4097,
> > limit_in_bytes=4096
> > memcg_function_test   24  TFAIL  :  ltpapicmd.c:190: input=1,
> > limit_in_bytes=0
> 
> So limit_in_bytes _is_ page aligned but we round down rather than up.
> 
> > Is this desired behavior, in which case ltp is broken or is it
> > a genuine bug?
> 
> This behavior has changed by 3e32cb2e0a12 ("mm: memcontrol: lockless
> page counters") introduced in 3.19. The change in rounding has been
> pointed out during the review
> http://marc.info/?l=linux-mm&m=141207518827336&w=2 but the conclusion
> was that the original round up wasn't really much better
> http://marc.info/?l=linux-mm&m=141226210316376&w=2 resp.
> http://marc.info/?l=linux-mm&m=141234785111200&w=2
> 
> I will post fix for ltp in the reply
> 
> > 30: Again, it locks memory with mmap and then tries to see if
> > force_empty would succeed. Expecting it to fail, but in this particular
> > case it succeeds?
> 
> I am not sure I understand this testcase. It does:
> 	TEST_PATH/memcg_process --mmap-anon -s $PAGESIZE
> 	[...]

Looking at the code, it does two more things here:

echo $pid > tasks
kill -s USR1 $pid 2> /dev/null

Sending SIGUSR1 to the memcg_process causes it to repeat the allocation
that has been passed to it by the command line parameters.

So my guess is that it expect the force_empty to fail if the process has
allocated some memory after it has been moved to the current cgroup.

The previous testcases does exactly this but moves the process to the
parent with:

echo $pid > ../tasks

Before it tries the force_empty and expects it to succeed.

Was this some old implementation limitation that has been lifted
meanwhile?

-- 
Cyril Hrubis
chrubis@suse.cz

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Possible bug - LTP failure for memcg

[Cyril Hrubis]

Hi!
> > The failing test cases 14, 22, 23, 24 and 30 test respectively:
> > 
> > 14: Hogging memory like so: mmap(NULL, memsize, PROT_WRITE | PROT_READ,
> > MAP_PRIVATE | MAP_ANONYMOUS | MAP_LOCKED, 0, 0);
> 
> MAP_LOCKED will not trigger the OOM killer as explained
> http://marc.info/?l=linux-mm&m=142122902313315&w=2. So this is expected
> and Cyril is working on fixing the test case.

It's on my TODO, I should get to this i a few weeks. Till then just
ignore the failure.

> > # Case 22 - 24: Test limit_in_bytes will be aligned to PAGESIZE - The
> > output clearly indicates that the limits in bytes is not being page
> > aligned?
> 
> I can see
> > memcg_function_test   22  TFAIL  :  ltpapicmd.c:190: input=4095,
> > limit_in_bytes=0
> > memcg_function_test   23  TFAIL  :  ltpapicmd.c:190: input=4097,
> > limit_in_bytes=4096
> > memcg_function_test   24  TFAIL  :  ltpapicmd.c:190: input=1,
> > limit_in_bytes=0
> 
> So limit_in_bytes _is_ page aligned but we round down rather than up.
> 
> > Is this desired behavior, in which case ltp is broken or is it
> > a genuine bug?
> 
> This behavior has changed by 3e32cb2e0a12 ("mm: memcontrol: lockless
> page counters") introduced in 3.19. The change in rounding has been
> pointed out during the review
> http://marc.info/?l=linux-mm&m=141207518827336&w=2 but the conclusion
> was that the original round up wasn't really much better
> http://marc.info/?l=linux-mm&m=141226210316376&w=2 resp.
> http://marc.info/?l=linux-mm&m=141234785111200&w=2
> 
> I will post fix for ltp in the reply
> 
> > 30: Again, it locks memory with mmap and then tries to see if
> > force_empty would succeed. Expecting it to fail, but in this particular
> > case it succeeds?
> 
> I am not sure I understand this testcase. It does:
> 	TEST_PATH/memcg_process --mmap-anon -s $PAGESIZE
> 	[...]

Looking at the code, it does two more things here:

echo $pid > tasks
kill -s USR1 $pid 2> /dev/null

Sending SIGUSR1 to the memcg_process causes it to repeat the allocation
that has been passed to it by the command line parameters.

So my guess is that it expect the force_empty to fail if the process has
allocated some memory after it has been moved to the current cgroup.

The previous testcases does exactly this but moves the process to the
parent with:

echo $pid > ../tasks

Before it tries the force_empty and expects it to succeed.

Was this some old implementation limitation that has been lifted
meanwhile?

-- 
Cyril Hrubis
chrubis-AlSwsSmVLrQ@public.gmane.org

Re: Possible bug - LTP failure for memcg

[Michal Hocko]

On Thu 14-05-15 12:31:49, Cyril Hrubis wrote:
[...]
> Looking at the code, it does two more things here:
> 
> echo $pid > tasks
> kill -s USR1 $pid 2> /dev/null
> 
> Sending SIGUSR1 to the memcg_process causes it to repeat the allocation
> that has been passed to it by the command line parameters.
> 
> So my guess is that it expect the force_empty to fail if the process has
> allocated some memory after it has been moved to the current cgroup.

force_empty fails if the memcg cannot be reclaimed down to 0. The memory
charged after the task has moved to the group is easily reclaimable so
I do not see any reason why we would fail here.

> The previous testcases does exactly this but moves the process to the
> parent with:
> 
> echo $pid > ../tasks
> 
> Before it tries the force_empty and expects it to succeed.
> 
> Was this some old implementation limitation that has been lifted
> meanwhile?

OK, now I remember... f61c42a7d911 ("memcg: remove tasks/children test
from mem_cgroup_force_empty()") which goes back to 3.16. So the test
case is invalid.

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Possible bug - LTP failure for memcg

[Michal Hocko]

On Thu 14-05-15 12:31:49, Cyril Hrubis wrote:
[...]
> Looking at the code, it does two more things here:
> 
> echo $pid > tasks
> kill -s USR1 $pid 2> /dev/null
> 
> Sending SIGUSR1 to the memcg_process causes it to repeat the allocation
> that has been passed to it by the command line parameters.
> 
> So my guess is that it expect the force_empty to fail if the process has
> allocated some memory after it has been moved to the current cgroup.

force_empty fails if the memcg cannot be reclaimed down to 0. The memory
charged after the task has moved to the group is easily reclaimable so
I do not see any reason why we would fail here.

> The previous testcases does exactly this but moves the process to the
> parent with:
> 
> echo $pid > ../tasks
> 
> Before it tries the force_empty and expects it to succeed.
> 
> Was this some old implementation limitation that has been lifted
> meanwhile?

OK, now I remember... f61c42a7d911 ("memcg: remove tasks/children test
from mem_cgroup_force_empty()") which goes back to 3.16. So the test
case is invalid.

-- 
Michal Hocko
SUSE Labs

Re: Possible bug - LTP failure for memcg

[Cyril Hrubis]

Hi!
> > The previous testcases does exactly this but moves the process to the
> > parent with:
> > 
> > echo $pid > ../tasks
> > 
> > Before it tries the force_empty and expects it to succeed.
> > 
> > Was this some old implementation limitation that has been lifted
> > meanwhile?
> 
> OK, now I remember... f61c42a7d911 ("memcg: remove tasks/children test
> from mem_cgroup_force_empty()") which goes back to 3.16. So the test
> case is invalid.

Then please send a patch to remove the test.

-- 
Cyril Hrubis
chrubis@suse.cz

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Possible bug - LTP failure for memcg

[Cyril Hrubis]

Hi!
> > The previous testcases does exactly this but moves the process to the
> > parent with:
> > 
> > echo $pid > ../tasks
> > 
> > Before it tries the force_empty and expects it to succeed.
> > 
> > Was this some old implementation limitation that has been lifted
> > meanwhile?
> 
> OK, now I remember... f61c42a7d911 ("memcg: remove tasks/children test
> from mem_cgroup_force_empty()") which goes back to 3.16. So the test
> case is invalid.

Then please send a patch to remove the test.

-- 
Cyril Hrubis
chrubis-AlSwsSmVLrQ@public.gmane.org

Re: Possible bug - LTP failure for memcg

[Michal Hocko]

On Thu 14-05-15 14:01:42, Cyril Hrubis wrote:
> Hi!
> > > The previous testcases does exactly this but moves the process to the
> > > parent with:
> > > 
> > > echo $pid > ../tasks
> > > 
> > > Before it tries the force_empty and expects it to succeed.
> > > 
> > > Was this some old implementation limitation that has been lifted
> > > meanwhile?
> > 
> > OK, now I remember... f61c42a7d911 ("memcg: remove tasks/children test
> > from mem_cgroup_force_empty()") which goes back to 3.16. So the test
> > case is invalid.
> 
> Then please send a patch to remove the test.

I think we can still fix both tescases and expect not to fail with
regular mmap but fail it with unreclaimable memory (e.g. disallow
swapout or use mlock).
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Possible bug - LTP failure for memcg

[Michal Hocko]

On Thu 14-05-15 14:01:42, Cyril Hrubis wrote:
> Hi!
> > > The previous testcases does exactly this but moves the process to the
> > > parent with:
> > > 
> > > echo $pid > ../tasks
> > > 
> > > Before it tries the force_empty and expects it to succeed.
> > > 
> > > Was this some old implementation limitation that has been lifted
> > > meanwhile?
> > 
> > OK, now I remember... f61c42a7d911 ("memcg: remove tasks/children test
> > from mem_cgroup_force_empty()") which goes back to 3.16. So the test
> > case is invalid.
> 
> Then please send a patch to remove the test.

I think we can still fix both tescases and expect not to fail with
regular mmap but fail it with unreclaimable memory (e.g. disallow
swapout or use mlock).
-- 
Michal Hocko
SUSE Labs

Re: Possible bug - LTP failure for memcg

[Cyril Hrubis]

Hi!
> > Then please send a patch to remove the test.
> 
> I think we can still fix both tescases and expect not to fail with
> regular mmap but fail it with unreclaimable memory (e.g. disallow
> swapout or use mlock).

That sounds even better.

-- 
Cyril Hrubis
chrubis@suse.cz

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Possible bug - LTP failure for memcg

[Michal Hocko]

On Thu 14-05-15 14:38:16, Cyril Hrubis wrote:
> Hi!
> > > Then please send a patch to remove the test.
> > 
> > I think we can still fix both tescases and expect not to fail with
> > regular mmap but fail it with unreclaimable memory (e.g. disallow
> > swapout or use mlock).
> 
> That sounds even better.

untested patch below:
---

Re: Possible bug - LTP failure for memcg

[Michal Hocko]

On Thu 14-05-15 14:38:16, Cyril Hrubis wrote:
> Hi!
> > > Then please send a patch to remove the test.
> > 
> > I think we can still fix both tescases and expect not to fail with
> > regular mmap but fail it with unreclaimable memory (e.g. disallow
> > swapout or use mlock).
> 
> That sounds even better.

untested patch below:
---
From 4a53f301804bfc9df33ca226f941b46c4c9aadbb Mon Sep 17 00:00:00 2001
From: Michal Hocko <mhocko@suse.cz>
Date: Thu, 14 May 2015 16:24:38 +0200
Subject: [PATCH] controllers/memcg: fix force_empty

testcase_29 and testcase_30 are no longer testing anything because
the kernel allows to use force_empty even for memcgs with active
tasks since f61c42a7d911 ("memcg: remove tasks/children test from
mem_cgroup_force_empty()) kernel commit.

If we really want to test this functionality then just expect the
success for regular mmap and expect the failure when the charged
memory is mlocked.

Signed-off-by: Michal Hocko <miso@dhcp22.suse.cz>
---
 testcases/kernel/controllers/memcg/functional/memcg_function_test.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh b/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
index cfc75fa730df..399c5614468a 100755
--- a/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
+++ b/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
@@ -211,8 +211,8 @@ testcase_29()
 	echo $pid > tasks
 	kill -s USR1 $pid 2> /dev/null
 	sleep 1
-	echo $pid > ../tasks
 
+	# This expects that there is swap configured
 	echo 1 > memory.force_empty
 	if [ $? -eq 0 ]; then
 		result $PASS "force memory succeeded"
@@ -225,7 +225,7 @@ testcase_29()
 
 testcase_30()
 {
-	$TEST_PATH/memcg_process --mmap-anon -s $PAGESIZE &
+	$TEST_PATH/memcg_process --mmap-lock2 -s $PAGESIZE &
 	pid=$!
 	sleep 1
 	echo $pid > tasks
-- 
2.1.4

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Possible bug - LTP failure for memcg

[Cyril Hrubis]

Hi!
> Signed-off-by: Michal Hocko <miso@dhcp22.suse.cz>

                                 ^
			       forgotten git config user.email?
> ---
>  testcases/kernel/controllers/memcg/functional/memcg_function_test.sh | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh b/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
> index cfc75fa730df..399c5614468a 100755
> --- a/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
> +++ b/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
> @@ -211,8 +211,8 @@ testcase_29()
>  	echo $pid > tasks
>  	kill -s USR1 $pid 2> /dev/null
>  	sleep 1
> -	echo $pid > ../tasks

This change breaks the testcase on older kernels:

./memcg_function_test.sh: line 215: echo: write error: Device or resource busy
memcg_function_test   29  TFAIL  :  ltpapicmd.c:190: force memory failed

$ uname -r
3.0.101-0.35-default

> +	# This expects that there is swap configured
>  	echo 1 > memory.force_empty
>  	if [ $? -eq 0 ]; then
>  		result $PASS "force memory succeeded"
> @@ -225,7 +225,7 @@ testcase_29()
>  
>  testcase_30()
>  {
> -	$TEST_PATH/memcg_process --mmap-anon -s $PAGESIZE &
> +	$TEST_PATH/memcg_process --mmap-lock2 -s $PAGESIZE &
>  	pid=$!
>  	sleep 1
>  	echo $pid > tasks

This part is OK.

-- 
Cyril Hrubis
chrubis@suse.cz

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Possible bug - LTP failure for memcg

[Cyril Hrubis]

Hi!
> Signed-off-by: Michal Hocko <miso-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>

                                 ^
			       forgotten git config user.email?
> ---
>  testcases/kernel/controllers/memcg/functional/memcg_function_test.sh | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh b/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
> index cfc75fa730df..399c5614468a 100755
> --- a/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
> +++ b/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
> @@ -211,8 +211,8 @@ testcase_29()
>  	echo $pid > tasks
>  	kill -s USR1 $pid 2> /dev/null
>  	sleep 1
> -	echo $pid > ../tasks

This change breaks the testcase on older kernels:

./memcg_function_test.sh: line 215: echo: write error: Device or resource busy
memcg_function_test   29  TFAIL  :  ltpapicmd.c:190: force memory failed

$ uname -r
3.0.101-0.35-default

> +	# This expects that there is swap configured
>  	echo 1 > memory.force_empty
>  	if [ $? -eq 0 ]; then
>  		result $PASS "force memory succeeded"
> @@ -225,7 +225,7 @@ testcase_29()
>  
>  testcase_30()
>  {
> -	$TEST_PATH/memcg_process --mmap-anon -s $PAGESIZE &
> +	$TEST_PATH/memcg_process --mmap-lock2 -s $PAGESIZE &
>  	pid=$!
>  	sleep 1
>  	echo $pid > tasks

This part is OK.

-- 
Cyril Hrubis
chrubis-AlSwsSmVLrQ@public.gmane.org

Re: Possible bug - LTP failure for memcg

[Michal Hocko]

On Thu 14-05-15 16:44:20, Cyril Hrubis wrote:
> Hi!
> > Signed-off-by: Michal Hocko <miso@dhcp22.suse.cz>
> 
>                                  ^
> 			       forgotten git config user.email?

Dohh...

> > ---
> >  testcases/kernel/controllers/memcg/functional/memcg_function_test.sh | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh b/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
> > index cfc75fa730df..399c5614468a 100755
> > --- a/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
> > +++ b/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
> > @@ -211,8 +211,8 @@ testcase_29()
> >  	echo $pid > tasks
> >  	kill -s USR1 $pid 2> /dev/null
> >  	sleep 1
> > -	echo $pid > ../tasks
> 
> This change breaks the testcase on older kernels:

Yeah, my bad, I've started with this then changed it back but forgot to
add it to the commit. Sorry about that. Hopefully the correct one:
---

Re: Possible bug - LTP failure for memcg

[Michal Hocko]

On Thu 14-05-15 16:44:20, Cyril Hrubis wrote:
> Hi!
> > Signed-off-by: Michal Hocko <miso-2MMpYkNvuYDjFM9bn6wA6Q@public.gmane.org>
> 
>                                  ^
> 			       forgotten git config user.email?

Dohh...

> > ---
> >  testcases/kernel/controllers/memcg/functional/memcg_function_test.sh | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh b/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
> > index cfc75fa730df..399c5614468a 100755
> > --- a/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
> > +++ b/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
> > @@ -211,8 +211,8 @@ testcase_29()
> >  	echo $pid > tasks
> >  	kill -s USR1 $pid 2> /dev/null
> >  	sleep 1
> > -	echo $pid > ../tasks
> 
> This change breaks the testcase on older kernels:

Yeah, my bad, I've started with this then changed it back but forgot to
add it to the commit. Sorry about that. Hopefully the correct one:
---
From 90a487d96fe09763b4517314797e2db32ce232b4 Mon Sep 17 00:00:00 2001
From: Michal Hocko <mhocko-AlSwsSmVLrQ@public.gmane.org>
Date: Thu, 14 May 2015 16:24:38 +0200
Subject: [PATCH] controllers/memcg: fix force_empty

testcase_29 and testcase_30 are no longer testing anything because
the kernel allows to use force_empty even for memcgs with active
tasks since f61c42a7d911 ("memcg: remove tasks/children test from
mem_cgroup_force_empty()) kernel commit.

If we really want to test this functionality then just expect the
success for regular mmap and expect the failure when the charged
memory is mlocked.

Signed-off-by: Michal Hocko <mhocko-AlSwsSmVLrQ@public.gmane.org>
---
 testcases/kernel/controllers/memcg/functional/memcg_function_test.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh b/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
index cfc75fa730df..f55593996b86 100755
--- a/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
+++ b/testcases/kernel/controllers/memcg/functional/memcg_function_test.sh
@@ -213,6 +213,7 @@ testcase_29()
 	sleep 1
 	echo $pid > ../tasks
 
+	# This expects that there is swap configured
 	echo 1 > memory.force_empty
 	if [ $? -eq 0 ]; then
 		result $PASS "force memory succeeded"
@@ -225,7 +226,7 @@ testcase_29()
 
 testcase_30()
 {
-	$TEST_PATH/memcg_process --mmap-anon -s $PAGESIZE &
+	$TEST_PATH/memcg_process --mmap-lock2 -s $PAGESIZE &
 	pid=$!
 	sleep 1
 	echo $pid > tasks
-- 
2.1.4

-- 
Michal Hocko
SUSE Labs

Re: Possible bug - LTP failure for memcg

[Cyril Hrubis]

Hi!
> testcase_29 and testcase_30 are no longer testing anything because
> the kernel allows to use force_empty even for memcgs with active
> tasks since f61c42a7d911 ("memcg: remove tasks/children test from
> mem_cgroup_force_empty()) kernel commit.
> 
> If we really want to test this functionality then just expect the
> success for regular mmap and expect the failure when the charged
> memory is mlocked.

Applied, thanks.

-- 
Cyril Hrubis
chrubis@suse.cz

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Possible bug - LTP failure for memcg

[Cyril Hrubis]

Hi!
> testcase_29 and testcase_30 are no longer testing anything because
> the kernel allows to use force_empty even for memcgs with active
> tasks since f61c42a7d911 ("memcg: remove tasks/children test from
> mem_cgroup_force_empty()) kernel commit.
> 
> If we really want to test this functionality then just expect the
> success for regular mmap and expect the failure when the charged
> memory is mlocked.

Applied, thanks.

-- 
Cyril Hrubis
chrubis-AlSwsSmVLrQ@public.gmane.org

Re: Possible bug - LTP failure for memcg

[Michal Hocko]

On Wed 13-05-15 18:29:13, Nikolay Borisov wrote:
[...]
> memcg_function_test   22  TFAIL  :  ltpapicmd.c:190: input=4095,
> limit_in_bytes=0
> memcg_function_test   23  TFAIL  :  ltpapicmd.c:190: input=4097,
> limit_in_bytes=4096
> memcg_function_test   24  TFAIL  :  ltpapicmd.c:190: input=1,
> limit_in_bytes=0

Before we go and fix these test cases. Do they make any sense at all?
Why should anybody even care that the limit is in page units? I do not
see anything like that mentioned in the documentation. Sure having
the limit in page size units makes a lot of sense from the
implementation POV but should userspace care? Would something break if
we change internals and allow also !page_aligned values? I have hard
time to imagine that.
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Possible bug - LTP failure for memcg

[Michal Hocko]

On Wed 13-05-15 18:29:13, Nikolay Borisov wrote:
[...]
> memcg_function_test   22  TFAIL  :  ltpapicmd.c:190: input=4095,
> limit_in_bytes=0
> memcg_function_test   23  TFAIL  :  ltpapicmd.c:190: input=4097,
> limit_in_bytes=4096
> memcg_function_test   24  TFAIL  :  ltpapicmd.c:190: input=1,
> limit_in_bytes=0

Before we go and fix these test cases. Do they make any sense at all?
Why should anybody even care that the limit is in page units? I do not
see anything like that mentioned in the documentation. Sure having
the limit in page size units makes a lot of sense from the
implementation POV but should userspace care? Would something break if
we change internals and allow also !page_aligned values? I have hard
time to imagine that.
-- 
Michal Hocko
SUSE Labs