* Re: [Resend] Question: kselftests: bpf/test_maps failed
[not found] <747824f6-4877-69fe-9f1d-f3c18fed6a01@intel.com>
2018-02-09 14:01 ` daniel
@ 2018-02-09 14:01 ` daniel
0 siblings, 0 replies; 6+ messages in thread
From: Daniel Borkmann @ 2018-02-09 14:01 UTC (permalink / raw)
To: Li Zhijian, ast, shuah, linux-kselftest; +Cc: linux-kernel, Li, Philip
On 02/09/2018 06:14 AM, Li Zhijian wrote:
> Hi
>
> INTEL 0-Day noticed that bpf/test_maps has different results at different platforms.
> when it fails, the details are like
Sorry for the late reply and thanks for reporting! More below:
> ------------------
> 880 Failed to create hashmap key=16 value=131072 'Cannot allocate memory'
> 881 Failed to create hashmap key=8 value=32768 'Cannot allocate memory'
> 882 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> 883 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> 884 Failed to create hashmap key=8 value=16384 'Cannot allocate memory'
> 885 Failed to create hashmap key=16 value=16384 'Cannot allocate memory'
> 886 Failed to create hashmap key=8 value=65536 'Cannot allocate memory'
> 887 Failed to create hashmap key=16 value=131072 'Cannot allocate memory'
> 888 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> 889 Failed to create hashmap key=16 value=65536 'Cannot allocate memory'
> 890 Failed to create hashmap key=8 value=65536 'Cannot allocate memory'
> 891 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> 892 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> 893 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> 894 Failed to create hashmap key=8 value=16384 'Cannot allocate memory'
> 895 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> 896 Failed to create hashmap key=16 value=8192 'Cannot allocate memory'
> 897 Failed to create hashmap key=8 value=32768 'Cannot allocate memory'
> 898 Failed to create hashmap key=16 value=8192 'Cannot allocate memory'
> 899 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> 900 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> 901 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> 902 Failed to create hashmap key=16 value=262144 'Cannot allocate memory'
> 903 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> 904 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> 905 test_maps: test_maps.c:955: run_parallel: Assertion `status == 0' failed.
> 906 Aborted
> 907 not ok 1..3 selftests: test_maps [FAIL]
> ------------------
>
> After a simply looking at the code, looks it's related to the cpu number and system memory.
>
> below are the result under different platform
> 1. Good
> model: Sandy Bridge
> nr_node: 1
> nr_cpu: 4
> memory: 6G
>
> 2. Good
> model: qemu-system-x86_64 -enable-kvm
> nr_cpu: 2
> memory: 4G
>
> 3. Bad
> model: Ivytown Ivy Bridge-EP
> nr_cpu: 48
> memory: 64G
>
> 4. Bad
> model: Skylake
> nr_cpu: 104
> memory: 64G
>
> I try to change the process number to 10 from 100, so it can pass at above Skylake(4) machine.
> ------------
> lizhijian@haswell-OptiPlex-9020:~/lkp/linux/tools/testing/selftests/bpf$ git diff
> diff --git a/tools/testing/selftests/bpf/test_maps.c b/tools/testing/selftests/bpf/test_maps.c
> index 040356e..b788ca1 100644
> --- a/tools/testing/selftests/bpf/test_maps.c
> +++ b/tools/testing/selftests/bpf/test_maps.c
> @@ -960,7 +960,7 @@ static void test_map_stress(void)
> {
> run_parallel(100, test_hashmap, NULL);
> run_parallel(100, test_hashmap_percpu, NULL);
> - run_parallel(100, test_hashmap_sizes, NULL);
> + run_parallel(10, test_hashmap_sizes, NULL);
> run_parallel(100, test_hashmap_walk, NULL);
>
> run_parallel(100, test_arraymap, NULL);
Unless Alexei has some better idea, I think if the bpf_create_map() error in
the stress test is about ENOMEM, then we shouldn't fail hard via exit(), for
all other cases we should however. So probably makes sense to just check for
errno == ENOMEM in case of fd < 0 in test_hashmap_sizes() and then continue
to keep trying under stress. Feel free to send a patch, Li.
Thanks again,
Daniel
^ permalink raw reply [flat|nested] 6+ messages in thread
* [Linux-kselftest-mirror] [Resend] Question: kselftests: bpf/test_maps failed
@ 2018-02-09 14:01 ` daniel
0 siblings, 0 replies; 6+ messages in thread
From: daniel @ 2018-02-09 14:01 UTC (permalink / raw)
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 4245 bytes --]
On 02/09/2018 06:14 AM, Li Zhijian wrote:
> Hi
>
> INTEL 0-Day noticed that bpf/test_maps has different results at different platforms.
> when it fails, the details are like
Sorry for the late reply and thanks for reporting! More below:
> ------------------
> 880 Failed to create hashmap key=16 value=131072 'Cannot allocate memory'
> 881 Failed to create hashmap key=8 value=32768 'Cannot allocate memory'
> 882 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> 883 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> 884 Failed to create hashmap key=8 value=16384 'Cannot allocate memory'
> 885 Failed to create hashmap key=16 value=16384 'Cannot allocate memory'
> 886 Failed to create hashmap key=8 value=65536 'Cannot allocate memory'
> 887 Failed to create hashmap key=16 value=131072 'Cannot allocate memory'
> 888 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> 889 Failed to create hashmap key=16 value=65536 'Cannot allocate memory'
> 890 Failed to create hashmap key=8 value=65536 'Cannot allocate memory'
> 891 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> 892 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> 893 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> 894 Failed to create hashmap key=8 value=16384 'Cannot allocate memory'
> 895 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> 896 Failed to create hashmap key=16 value=8192 'Cannot allocate memory'
> 897 Failed to create hashmap key=8 value=32768 'Cannot allocate memory'
> 898 Failed to create hashmap key=16 value=8192 'Cannot allocate memory'
> 899 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> 900 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> 901 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> 902 Failed to create hashmap key=16 value=262144 'Cannot allocate memory'
> 903 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> 904 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> 905 test_maps: test_maps.c:955: run_parallel: Assertion `status == 0' failed.
> 906 Aborted
> 907 not ok 1..3 selftests: test_maps [FAIL]
> ------------------
>
> After a simply looking at the code, looks it's related to the cpu number and system memory.
>
> below are the result under different platform
> 1. Good
> model: Sandy Bridge
> nr_node: 1
> nr_cpu: 4
> memory: 6G
>
> 2. Good
> model: qemu-system-x86_64 -enable-kvm
> nr_cpu: 2
> memory: 4G
>
> 3. Bad
> model: Ivytown Ivy Bridge-EP
> nr_cpu: 48
> memory: 64G
>
> 4. Bad
> model: Skylake
> nr_cpu: 104
> memory: 64G
>
> I try to change the process number to 10 from 100, so it can pass at above Skylake(4) machine.
> ------------
> lizhijian at haswell-OptiPlex-9020:~/lkp/linux/tools/testing/selftests/bpf$ git diff
> diff --git a/tools/testing/selftests/bpf/test_maps.c b/tools/testing/selftests/bpf/test_maps.c
> index 040356e..b788ca1 100644
> --- a/tools/testing/selftests/bpf/test_maps.c
> +++ b/tools/testing/selftests/bpf/test_maps.c
> @@ -960,7 +960,7 @@ static void test_map_stress(void)
> {
> run_parallel(100, test_hashmap, NULL);
> run_parallel(100, test_hashmap_percpu, NULL);
> - run_parallel(100, test_hashmap_sizes, NULL);
> + run_parallel(10, test_hashmap_sizes, NULL);
> run_parallel(100, test_hashmap_walk, NULL);
>
> run_parallel(100, test_arraymap, NULL);
Unless Alexei has some better idea, I think if the bpf_create_map() error in
the stress test is about ENOMEM, then we shouldn't fail hard via exit(), for
all other cases we should however. So probably makes sense to just check for
errno == ENOMEM in case of fd < 0 in test_hashmap_sizes() and then continue
to keep trying under stress. Feel free to send a patch, Li.
Thanks again,
Daniel
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread
* [Linux-kselftest-mirror] [Resend] Question: kselftests: bpf/test_maps failed
@ 2018-02-09 14:01 ` daniel
0 siblings, 0 replies; 6+ messages in thread
From: Daniel Borkmann @ 2018-02-09 14:01 UTC (permalink / raw)
On 02/09/2018 06:14 AM, Li Zhijian wrote:
> Hi
>
> INTEL 0-Day noticed that bpf/test_maps has different results at different platforms.
> when it fails, the details are like
Sorry for the late reply and thanks for reporting! More below:
> ------------------
> 880 Failed to create hashmap key=16 value=131072 'Cannot allocate memory'
> 881 Failed to create hashmap key=8 value=32768 'Cannot allocate memory'
> 882 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> 883 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> 884 Failed to create hashmap key=8 value=16384 'Cannot allocate memory'
> 885 Failed to create hashmap key=16 value=16384 'Cannot allocate memory'
> 886 Failed to create hashmap key=8 value=65536 'Cannot allocate memory'
> 887 Failed to create hashmap key=16 value=131072 'Cannot allocate memory'
> 888 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> 889 Failed to create hashmap key=16 value=65536 'Cannot allocate memory'
> 890 Failed to create hashmap key=8 value=65536 'Cannot allocate memory'
> 891 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> 892 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> 893 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> 894 Failed to create hashmap key=8 value=16384 'Cannot allocate memory'
> 895 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> 896 Failed to create hashmap key=16 value=8192 'Cannot allocate memory'
> 897 Failed to create hashmap key=8 value=32768 'Cannot allocate memory'
> 898 Failed to create hashmap key=16 value=8192 'Cannot allocate memory'
> 899 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> 900 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> 901 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> 902 Failed to create hashmap key=16 value=262144 'Cannot allocate memory'
> 903 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> 904 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> 905 test_maps: test_maps.c:955: run_parallel: Assertion `status == 0' failed.
> 906 Aborted
> 907 not ok 1..3 selftests: test_maps [FAIL]
> ------------------
>
> After a simply looking at the code, looks it's related to the cpu number and system memory.
>
> below are the result under different platform
> 1. Good
> model: Sandy Bridge
> nr_node: 1
> nr_cpu: 4
> memory: 6G
>
> 2. Good
> model: qemu-system-x86_64 -enable-kvm
> nr_cpu: 2
> memory: 4G
>
> 3. Bad
> model: Ivytown Ivy Bridge-EP
> nr_cpu: 48
> memory: 64G
>
> 4. Bad
> model: Skylake
> nr_cpu: 104
> memory: 64G
>
> I try to change the process number to 10 from 100, so it can pass at above Skylake(4) machine.
> ------------
> lizhijian at haswell-OptiPlex-9020:~/lkp/linux/tools/testing/selftests/bpf$ git diff
> diff --git a/tools/testing/selftests/bpf/test_maps.c b/tools/testing/selftests/bpf/test_maps.c
> index 040356e..b788ca1 100644
> --- a/tools/testing/selftests/bpf/test_maps.c
> +++ b/tools/testing/selftests/bpf/test_maps.c
> @@ -960,7 +960,7 @@ static void test_map_stress(void)
> {
> run_parallel(100, test_hashmap, NULL);
> run_parallel(100, test_hashmap_percpu, NULL);
> - run_parallel(100, test_hashmap_sizes, NULL);
> + run_parallel(10, test_hashmap_sizes, NULL);
> run_parallel(100, test_hashmap_walk, NULL);
>
> run_parallel(100, test_arraymap, NULL);
Unless Alexei has some better idea, I think if the bpf_create_map() error in
the stress test is about ENOMEM, then we shouldn't fail hard via exit(), for
all other cases we should however. So probably makes sense to just check for
errno == ENOMEM in case of fd < 0 in test_hashmap_sizes() and then continue
to keep trying under stress. Feel free to send a patch, Li.
Thanks again,
Daniel
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Resend] Question: kselftests: bpf/test_maps failed
2018-02-09 14:01 ` daniel
(?)
@ 2018-02-10 1:36 ` alexei.starovoitov
-1 siblings, 0 replies; 6+ messages in thread
From: Alexei Starovoitov @ 2018-02-10 1:36 UTC (permalink / raw)
To: Daniel Borkmann
Cc: Li Zhijian, ast, shuah, linux-kselftest, linux-kernel, Li,
Philip, netdev
On Fri, Feb 09, 2018 at 03:01:57PM +0100, Daniel Borkmann wrote:
> On 02/09/2018 06:14 AM, Li Zhijian wrote:
> > Hi
> >
> > INTEL 0-Day noticed that bpf/test_maps has different results at different platforms.
> > when it fails, the details are like
>
> Sorry for the late reply and thanks for reporting! More below:
>
> > ------------------
> > 880 Failed to create hashmap key=16 value=131072 'Cannot allocate memory'
> > 881 Failed to create hashmap key=8 value=32768 'Cannot allocate memory'
> > 882 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> > 883 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> > 884 Failed to create hashmap key=8 value=16384 'Cannot allocate memory'
> > 885 Failed to create hashmap key=16 value=16384 'Cannot allocate memory'
> > 886 Failed to create hashmap key=8 value=65536 'Cannot allocate memory'
> > 887 Failed to create hashmap key=16 value=131072 'Cannot allocate memory'
> > 888 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> > 889 Failed to create hashmap key=16 value=65536 'Cannot allocate memory'
> > 890 Failed to create hashmap key=8 value=65536 'Cannot allocate memory'
> > 891 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> > 892 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> > 893 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> > 894 Failed to create hashmap key=8 value=16384 'Cannot allocate memory'
> > 895 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> > 896 Failed to create hashmap key=16 value=8192 'Cannot allocate memory'
> > 897 Failed to create hashmap key=8 value=32768 'Cannot allocate memory'
> > 898 Failed to create hashmap key=16 value=8192 'Cannot allocate memory'
> > 899 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 900 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 901 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 902 Failed to create hashmap key=16 value=262144 'Cannot allocate memory'
> > 903 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 904 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 905 test_maps: test_maps.c:955: run_parallel: Assertion `status == 0' failed.
> > 906 Aborted
> > 907 not ok 1..3 selftests: test_maps [FAIL]
> > ------------------
> >
> > After a simply looking at the code, looks it's related to the cpu number and system memory.
> >
> > below are the result under different platform
> > 1. Good
> > model: Sandy Bridge
> > nr_node: 1
> > nr_cpu: 4
> > memory: 6G
> >
> > 2. Good
> > model: qemu-system-x86_64 -enable-kvm
> > nr_cpu: 2
> > memory: 4G
> >
> > 3. Bad
> > model: Ivytown Ivy Bridge-EP
> > nr_cpu: 48
> > memory: 64G
> >
> > 4. Bad
> > model: Skylake
> > nr_cpu: 104
> > memory: 64G
> >
> > I try to change the process number to 10 from 100, so it can pass at above Skylake(4) machine.
> > ------------
> > lizhijian@haswell-OptiPlex-9020:~/lkp/linux/tools/testing/selftests/bpf$ git diff
> > diff --git a/tools/testing/selftests/bpf/test_maps.c b/tools/testing/selftests/bpf/test_maps.c
> > index 040356e..b788ca1 100644
> > --- a/tools/testing/selftests/bpf/test_maps.c
> > +++ b/tools/testing/selftests/bpf/test_maps.c
> > @@ -960,7 +960,7 @@ static void test_map_stress(void)
> > {
> > run_parallel(100, test_hashmap, NULL);
> > run_parallel(100, test_hashmap_percpu, NULL);
> > - run_parallel(100, test_hashmap_sizes, NULL);
> > + run_parallel(10, test_hashmap_sizes, NULL);
> > run_parallel(100, test_hashmap_walk, NULL);
> >
> > run_parallel(100, test_arraymap, NULL);
>
> Unless Alexei has some better idea, I think if the bpf_create_map() error in
> the stress test is about ENOMEM, then we shouldn't fail hard via exit(), for
> all other cases we should however. So probably makes sense to just check for
> errno == ENOMEM in case of fd < 0 in test_hashmap_sizes() and then continue
> to keep trying under stress. Feel free to send a patch, Li.
that's probably good path for now.
I also see that test_maps fails on freshly booted kernel with such assert,
but then restarting test_maps again works and repeated runs succeed too.
I suspect there is a deeper issue here related to memory allocation.
Either slab or percpu allocator are behaving funky.
It needs to be further debugged.
^ permalink raw reply [flat|nested] 6+ messages in thread
* [Linux-kselftest-mirror] [Resend] Question: kselftests: bpf/test_maps failed
@ 2018-02-10 1:36 ` alexei.starovoitov
0 siblings, 0 replies; 6+ messages in thread
From: alexei.starovoitov @ 2018-02-10 1:36 UTC (permalink / raw)
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 4797 bytes --]
On Fri, Feb 09, 2018 at 03:01:57PM +0100, Daniel Borkmann wrote:
> On 02/09/2018 06:14 AM, Li Zhijian wrote:
> > Hi
> >
> > INTEL 0-Day noticed that bpf/test_maps has different results at different platforms.
> > when it fails, the details are like
>
> Sorry for the late reply and thanks for reporting! More below:
>
> > ------------------
> > 880 Failed to create hashmap key=16 value=131072 'Cannot allocate memory'
> > 881 Failed to create hashmap key=8 value=32768 'Cannot allocate memory'
> > 882 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> > 883 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> > 884 Failed to create hashmap key=8 value=16384 'Cannot allocate memory'
> > 885 Failed to create hashmap key=16 value=16384 'Cannot allocate memory'
> > 886 Failed to create hashmap key=8 value=65536 'Cannot allocate memory'
> > 887 Failed to create hashmap key=16 value=131072 'Cannot allocate memory'
> > 888 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> > 889 Failed to create hashmap key=16 value=65536 'Cannot allocate memory'
> > 890 Failed to create hashmap key=8 value=65536 'Cannot allocate memory'
> > 891 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> > 892 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> > 893 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> > 894 Failed to create hashmap key=8 value=16384 'Cannot allocate memory'
> > 895 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> > 896 Failed to create hashmap key=16 value=8192 'Cannot allocate memory'
> > 897 Failed to create hashmap key=8 value=32768 'Cannot allocate memory'
> > 898 Failed to create hashmap key=16 value=8192 'Cannot allocate memory'
> > 899 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 900 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 901 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 902 Failed to create hashmap key=16 value=262144 'Cannot allocate memory'
> > 903 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 904 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 905 test_maps: test_maps.c:955: run_parallel: Assertion `status == 0' failed.
> > 906 Aborted
> > 907 not ok 1..3 selftests: test_maps [FAIL]
> > ------------------
> >
> > After a simply looking at the code, looks it's related to the cpu number and system memory.
> >
> > below are the result under different platform
> > 1. Good
> > model: Sandy Bridge
> > nr_node: 1
> > nr_cpu: 4
> > memory: 6G
> >
> > 2. Good
> > model: qemu-system-x86_64 -enable-kvm
> > nr_cpu: 2
> > memory: 4G
> >
> > 3. Bad
> > model: Ivytown Ivy Bridge-EP
> > nr_cpu: 48
> > memory: 64G
> >
> > 4. Bad
> > model: Skylake
> > nr_cpu: 104
> > memory: 64G
> >
> > I try to change the process number to 10 from 100, so it can pass at above Skylake(4) machine.
> > ------------
> > lizhijian at haswell-OptiPlex-9020:~/lkp/linux/tools/testing/selftests/bpf$ git diff
> > diff --git a/tools/testing/selftests/bpf/test_maps.c b/tools/testing/selftests/bpf/test_maps.c
> > index 040356e..b788ca1 100644
> > --- a/tools/testing/selftests/bpf/test_maps.c
> > +++ b/tools/testing/selftests/bpf/test_maps.c
> > @@ -960,7 +960,7 @@ static void test_map_stress(void)
> > {
> > run_parallel(100, test_hashmap, NULL);
> > run_parallel(100, test_hashmap_percpu, NULL);
> > - run_parallel(100, test_hashmap_sizes, NULL);
> > + run_parallel(10, test_hashmap_sizes, NULL);
> > run_parallel(100, test_hashmap_walk, NULL);
> >
> > run_parallel(100, test_arraymap, NULL);
>
> Unless Alexei has some better idea, I think if the bpf_create_map() error in
> the stress test is about ENOMEM, then we shouldn't fail hard via exit(), for
> all other cases we should however. So probably makes sense to just check for
> errno == ENOMEM in case of fd < 0 in test_hashmap_sizes() and then continue
> to keep trying under stress. Feel free to send a patch, Li.
that's probably good path for now.
I also see that test_maps fails on freshly booted kernel with such assert,
but then restarting test_maps again works and repeated runs succeed too.
I suspect there is a deeper issue here related to memory allocation.
Either slab or percpu allocator are behaving funky.
It needs to be further debugged.
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread
* [Linux-kselftest-mirror] [Resend] Question: kselftests: bpf/test_maps failed
@ 2018-02-10 1:36 ` alexei.starovoitov
0 siblings, 0 replies; 6+ messages in thread
From: Alexei Starovoitov @ 2018-02-10 1:36 UTC (permalink / raw)
On Fri, Feb 09, 2018@03:01:57PM +0100, Daniel Borkmann wrote:
> On 02/09/2018 06:14 AM, Li Zhijian wrote:
> > Hi
> >
> > INTEL 0-Day noticed that bpf/test_maps has different results at different platforms.
> > when it fails, the details are like
>
> Sorry for the late reply and thanks for reporting! More below:
>
> > ------------------
> > 880 Failed to create hashmap key=16 value=131072 'Cannot allocate memory'
> > 881 Failed to create hashmap key=8 value=32768 'Cannot allocate memory'
> > 882 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> > 883 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> > 884 Failed to create hashmap key=8 value=16384 'Cannot allocate memory'
> > 885 Failed to create hashmap key=16 value=16384 'Cannot allocate memory'
> > 886 Failed to create hashmap key=8 value=65536 'Cannot allocate memory'
> > 887 Failed to create hashmap key=16 value=131072 'Cannot allocate memory'
> > 888 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> > 889 Failed to create hashmap key=16 value=65536 'Cannot allocate memory'
> > 890 Failed to create hashmap key=8 value=65536 'Cannot allocate memory'
> > 891 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> > 892 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> > 893 Failed to create hashmap key=16 value=32768 'Cannot allocate memory'
> > 894 Failed to create hashmap key=8 value=16384 'Cannot allocate memory'
> > 895 Failed to create hashmap key=8 value=131072 'Cannot allocate memory'
> > 896 Failed to create hashmap key=16 value=8192 'Cannot allocate memory'
> > 897 Failed to create hashmap key=8 value=32768 'Cannot allocate memory'
> > 898 Failed to create hashmap key=16 value=8192 'Cannot allocate memory'
> > 899 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 900 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 901 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 902 Failed to create hashmap key=16 value=262144 'Cannot allocate memory'
> > 903 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 904 Failed to create hashmap key=8 value=262144 'Cannot allocate memory'
> > 905 test_maps: test_maps.c:955: run_parallel: Assertion `status == 0' failed.
> > 906 Aborted
> > 907 not ok 1..3 selftests: test_maps [FAIL]
> > ------------------
> >
> > After a simply looking at the code, looks it's related to the cpu number and system memory.
> >
> > below are the result under different platform
> > 1. Good
> > model: Sandy Bridge
> > nr_node: 1
> > nr_cpu: 4
> > memory: 6G
> >
> > 2. Good
> > model: qemu-system-x86_64 -enable-kvm
> > nr_cpu: 2
> > memory: 4G
> >
> > 3. Bad
> > model: Ivytown Ivy Bridge-EP
> > nr_cpu: 48
> > memory: 64G
> >
> > 4. Bad
> > model: Skylake
> > nr_cpu: 104
> > memory: 64G
> >
> > I try to change the process number to 10 from 100, so it can pass at above Skylake(4) machine.
> > ------------
> > lizhijian at haswell-OptiPlex-9020:~/lkp/linux/tools/testing/selftests/bpf$ git diff
> > diff --git a/tools/testing/selftests/bpf/test_maps.c b/tools/testing/selftests/bpf/test_maps.c
> > index 040356e..b788ca1 100644
> > --- a/tools/testing/selftests/bpf/test_maps.c
> > +++ b/tools/testing/selftests/bpf/test_maps.c
> > @@ -960,7 +960,7 @@ static void test_map_stress(void)
> > {
> > run_parallel(100, test_hashmap, NULL);
> > run_parallel(100, test_hashmap_percpu, NULL);
> > - run_parallel(100, test_hashmap_sizes, NULL);
> > + run_parallel(10, test_hashmap_sizes, NULL);
> > run_parallel(100, test_hashmap_walk, NULL);
> >
> > run_parallel(100, test_arraymap, NULL);
>
> Unless Alexei has some better idea, I think if the bpf_create_map() error in
> the stress test is about ENOMEM, then we shouldn't fail hard via exit(), for
> all other cases we should however. So probably makes sense to just check for
> errno == ENOMEM in case of fd < 0 in test_hashmap_sizes() and then continue
> to keep trying under stress. Feel free to send a patch, Li.
that's probably good path for now.
I also see that test_maps fails on freshly booted kernel with such assert,
but then restarting test_maps again works and repeated runs succeed too.
I suspect there is a deeper issue here related to memory allocation.
Either slab or percpu allocator are behaving funky.
It needs to be further debugged.
--
To unsubscribe from this list: send the line "unsubscribe linux-kselftest" in
the body of a message to majordomo at vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2018-02-10 1:36 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <747824f6-4877-69fe-9f1d-f3c18fed6a01@intel.com>
2018-02-09 14:01 ` [Resend] Question: kselftests: bpf/test_maps failed Daniel Borkmann
2018-02-09 14:01 ` [Linux-kselftest-mirror] " Daniel Borkmann
2018-02-09 14:01 ` daniel
2018-02-10 1:36 ` Alexei Starovoitov
2018-02-10 1:36 ` [Linux-kselftest-mirror] " Alexei Starovoitov
2018-02-10 1:36 ` alexei.starovoitov
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.