[PATCH v2] selftests: net: Correct case name

[PATCH v2] selftests: net: Correct case name

[Li Zhijian]

ipv6_addr_bind/ipv4_addr_bind are function names. Previously, bind test
would not be run by default due to the wrong case names

Fixes: 34d0302ab861 ("selftests: Add ipv6 address bind tests to fcnal-test")
Fixes: 75b2b2b3db4c ("selftests: Add ipv4 address bind tests to fcnal-test")
Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
---
 tools/testing/selftests/net/fcnal-test.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/net/fcnal-test.sh b/tools/testing/selftests/net/fcnal-test.sh
index 3313566ce906..7f5b265fcb90 100755
--- a/tools/testing/selftests/net/fcnal-test.sh
+++ b/tools/testing/selftests/net/fcnal-test.sh
@@ -4002,8 +4002,8 @@ EOF
 ################################################################################
 # main
 
-TESTS_IPV4="ipv4_ping ipv4_tcp ipv4_udp ipv4_addr_bind ipv4_runtime ipv4_netfilter"
-TESTS_IPV6="ipv6_ping ipv6_tcp ipv6_udp ipv6_addr_bind ipv6_runtime ipv6_netfilter"
+TESTS_IPV4="ipv4_ping ipv4_tcp ipv4_udp ipv4_bind ipv4_runtime ipv4_netfilter"
+TESTS_IPV6="ipv6_ping ipv6_tcp ipv6_udp ipv6_bind ipv6_runtime ipv6_netfilter"
 TESTS_OTHER="use_cases"
 
 PAUSE_ON_FAIL=no
-- 
2.33.0

Re: [PATCH v2] selftests: net: Correct case name

[patchwork-bot+netdevbpf]

Hello:

This patch was applied to netdev/net.git (master)
by David S. Miller <davem@davemloft.net>:

On Thu, 2 Dec 2021 10:28:41 +0800 you wrote:
> ipv6_addr_bind/ipv4_addr_bind are function names. Previously, bind test
> would not be run by default due to the wrong case names
> 
> Fixes: 34d0302ab861 ("selftests: Add ipv6 address bind tests to fcnal-test")
> Fixes: 75b2b2b3db4c ("selftests: Add ipv4 address bind tests to fcnal-test")
> Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
> 
> [...]

Here is the summary with links:
  - [v2] selftests: net: Correct case name
    https://git.kernel.org/netdev/net/c/a05431b22be8

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html

Re: [PATCH v2] selftests: net: Correct case name

[lizhijian]

Hi folks

Currently,  LKP/0Day will cover bind tests, but LKP/0DAY observered 3 failures.

> # TESTS=bind ./fcnal-test.sh
> ###########################################################################
> IPv4 address binds
> ###########################################################################
>
>
> #################################################################
> No VRF
>
> TEST: Raw socket bind to local address - ns-A IP                              [ OK ]
> TEST: Raw socket bind to local address after device bind - ns-A IP            [ OK ]
> TEST: Raw socket bind to local address - ns-A loopback IP                     [ OK ]
> TEST: Raw socket bind to local address after device bind - ns-A loopback IP   [ OK ]
> TEST: TCP socket bind to local address - ns-A IP                              [ OK ]
> TEST: TCP socket bind to local address after device bind - ns-A IP            [ OK ]
>
> #################################################################
> With VRF
>
> TEST: Raw socket bind to local address - ns-A IP                              [ OK ]
> TEST: Raw socket bind to local address after device bind - ns-A IP            [ OK ]
> TEST: Raw socket bind to local address after VRF bind - ns-A IP               [ OK ]
> TEST: Raw socket bind to local address - VRF IP                               [FAIL]
>

i found that above case failed with "server: error binding socket: 99: Cannot assign requested address"
i have manually check it with below command after setup(), same errors:

# ip netns exec ns-A nettest -s -R -P icmp -l 172.16.3.1 -b
05:55:11 server: error binding socket: 99: Cannot assign requested address

But when i specified specific network interface, it works
# ip netns exec ns-A nettest -s -R -P icmp -l 172.16.3.1 -b -I red
# echo $?
0
# ip netns exec ns-A nettest -s -R -P icmp -l 172.16.3.1 -b
06:01:55 server: error binding socket: 99: Cannot assign requested address
# echo $?
1


So i wonder if i missed something ?




>
> TEST: Raw socket bind to local address after device bind - VRF IP             [ OK ]
> TEST: Raw socket bind to local address after VRF bind - VRF IP                [ OK ]
> TEST: Raw socket bind to out of scope address after VRF bind - ns-A loopback IP  [ OK ]
> TEST: TCP socket bind to local address - ns-A IP                              [ OK ]
> TEST: TCP socket bind to local address after device bind - ns-A IP            [ OK ]
> TEST: TCP socket bind to local address - VRF IP                               [ OK ]
> TEST: TCP socket bind to local address after device bind - VRF IP             [ OK ]
> TEST: TCP socket bind to invalid local address for VRF - ns-A loopback IP     [ OK ]
> TEST: TCP socket bind to invalid local address for device bind - ns-A loopback IP  [ OK ]
>
>
>
> # TESTS=bind6 ./fcnal-test.sh
>
> ###########################################################################
> IPv6 address binds
> ###########################################################################
>
>
> #################################################################
> No VRF
>
> TEST: Raw socket bind to local address - ns-A IPv6                            [FAIL]
> TEST: Raw socket bind to local address after device bind - ns-A IPv6          [ OK ]
> TEST: Raw socket bind to local address - ns-A loopback IPv6                   [ OK ]
> TEST: Raw socket bind to local address after device bind - ns-A loopback IPv6  [ OK ]
> TEST: TCP socket bind to local address - ns-A IPv6                            [ OK ]
> TEST: TCP socket bind to local address after device bind - ns-A IPv6          [ OK ]
> TEST: TCP socket bind to out of scope local address - ns-A loopback IPv6      [FAIL]
>
> #################################################################
> With VRF
>
> TEST: Raw socket bind to local address after vrf bind - ns-A IPv6             [ OK ]
> TEST: Raw socket bind to local address after device bind - ns-A IPv6          [ OK ]
> TEST: Raw socket bind to local address after vrf bind - VRF IPv6              [ OK ]
> TEST: Raw socket bind to local address after device bind - VRF IPv6           [ OK ]
> TEST: Raw socket bind to invalid local address after vrf bind - ns-A loopback IPv6  [ OK ]
> TEST: TCP socket bind to local address with VRF bind - ns-A IPv6              [ OK ]
> TEST: TCP socket bind to local address with VRF bind - VRF IPv6               [ OK ]
> TEST: TCP socket bind to local address with device bind - ns-A IPv6           [ OK ]
> TEST: TCP socket bind to VRF address with device bind - VRF IPv6              [FAIL]
> TEST: TCP socket bind to invalid local address for VRF - ns-A loopback IPv6   [ OK ]
> TEST: TCP socket bind to invalid local address for device bind - ns-A loopback IPv6  [ OK ]


Thanks
Zhijian


On 02/12/2021 10:28, Li Zhijian wrote:
> ipv6_addr_bind/ipv4_addr_bind are function names. Previously, bind test
> would not be run by default due to the wrong case names
>
> Fixes: 34d0302ab861 ("selftests: Add ipv6 address bind tests to fcnal-test")
> Fixes: 75b2b2b3db4c ("selftests: Add ipv4 address bind tests to fcnal-test")
> Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
> ---
>   tools/testing/selftests/net/fcnal-test.sh | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/tools/testing/selftests/net/fcnal-test.sh b/tools/testing/selftests/net/fcnal-test.sh
> index 3313566ce906..7f5b265fcb90 100755
> --- a/tools/testing/selftests/net/fcnal-test.sh
> +++ b/tools/testing/selftests/net/fcnal-test.sh
> @@ -4002,8 +4002,8 @@ EOF
>   ################################################################################
>   # main
>   
> -TESTS_IPV4="ipv4_ping ipv4_tcp ipv4_udp ipv4_addr_bind ipv4_runtime ipv4_netfilter"
> -TESTS_IPV6="ipv6_ping ipv6_tcp ipv6_udp ipv6_addr_bind ipv6_runtime ipv6_netfilter"
> +TESTS_IPV4="ipv4_ping ipv4_tcp ipv4_udp ipv4_bind ipv4_runtime ipv4_netfilter"
> +TESTS_IPV6="ipv6_ping ipv6_tcp ipv6_udp ipv6_bind ipv6_runtime ipv6_netfilter"
>   TESTS_OTHER="use_cases"
>   
>   PAUSE_ON_FAIL=no

Re: [PATCH v2] selftests: net: Correct case name

[David Ahern]

On 12/6/21 11:05 PM, lizhijian@fujitsu.com wrote:
>> #################################################################
>> With VRF
>>
>> TEST: Raw socket bind to local address - ns-A IP                              [ OK ]
>> TEST: Raw socket bind to local address after device bind - ns-A IP            [ OK ]
>> TEST: Raw socket bind to local address after VRF bind - ns-A IP               [ OK ]
>> TEST: Raw socket bind to local address - VRF IP                               [FAIL]
>>
> 
> i found that above case failed with "server: error binding socket: 99: Cannot assign requested address"
> i have manually check it with below command after setup(), same errors:
> 
> # ip netns exec ns-A nettest -s -R -P icmp -l 172.16.3.1 -b
> 05:55:11 server: error binding socket: 99: Cannot assign requested address
> 
> But when i specified specific network interface, it works
> # ip netns exec ns-A nettest -s -R -P icmp -l 172.16.3.1 -b -I red
> # echo $?
> 0
> # ip netns exec ns-A nettest -s -R -P icmp -l 172.16.3.1 -b
> 06:01:55 server: error binding socket: 99: Cannot assign requested address
> # echo $?
> 1
> 
> 
> So i wonder if i missed something ?
> 

That test should be a negative test as is the first one in that group -
in both cases the address bind should fail since the socket is not in
the VRF but the address is. The first on currently shows "OK" but that
is because of 5cad8bce26e01 that made changes to the config to validate
MD5 changes. Will send a patch to fix.

Re: [PATCH v2] selftests: net: Correct case name

[David Ahern]

On 12/6/21 11:05 PM, lizhijian@fujitsu.com wrote:
>> # TESTS=bind6 ./fcnal-test.sh
>>
>> ###########################################################################
>> IPv6 address binds
>> ###########################################################################
>>
>>
>> #################################################################
>> No VRF
>>
>> TEST: Raw socket bind to local address - ns-A IPv6                            [FAIL]

This one passes for me.

Can you run the test with '-v -p'? -v will give you the command line
that is failing. -p will pause the tests at the failure. From there you
can do:

ip netns exec ns-A bash

Look at the routing - no VRF is involved so the address should be local
to the device and the loopback. Run the test manually to see if it
really is failing.


>> TEST: Raw socket bind to local address after device bind - ns-A IPv6          [ OK ]
>> TEST: Raw socket bind to local address - ns-A loopback IPv6                   [ OK ]
>> TEST: Raw socket bind to local address after device bind - ns-A loopback IPv6  [ OK ]
>> TEST: TCP socket bind to local address - ns-A IPv6                            [ OK ]
>> TEST: TCP socket bind to local address after device bind - ns-A IPv6          [ OK ]
>> TEST: TCP socket bind to out of scope local address - ns-A loopback IPv6      [FAIL]

This one seems to be a new problem. The socket is bound to eth1 and the
address bind is to an address on loopback. That should not be working.

>>
>> #################################################################
>> With VRF
>>
>> TEST: Raw socket bind to local address after vrf bind - ns-A IPv6             [ OK ]
>> TEST: Raw socket bind to local address after device bind - ns-A IPv6          [ OK ]
>> TEST: Raw socket bind to local address after vrf bind - VRF IPv6              [ OK ]
>> TEST: Raw socket bind to local address after device bind - VRF IPv6           [ OK ]
>> TEST: Raw socket bind to invalid local address after vrf bind - ns-A loopback IPv6  [ OK ]
>> TEST: TCP socket bind to local address with VRF bind - ns-A IPv6              [ OK ]
>> TEST: TCP socket bind to local address with VRF bind - VRF IPv6               [ OK ]
>> TEST: TCP socket bind to local address with device bind - ns-A IPv6           [ OK ]
>> TEST: TCP socket bind to VRF address with device bind - VRF IPv6              [FAIL]

This failure is similar to the last one. Need to see if a recent commit
changed something.


>> TEST: TCP socket bind to invalid local address for VRF - ns-A loopback IPv6   [ OK ]
>> TEST: TCP socket bind to invalid local address for device bind - ns-A loopback IPv6  [ OK ]
> 
> 
> Thanks
> Zhijian
> 
> 

Re: [PATCH v2] selftests: net: Correct case name

[lizhijian]

On 08/12/2021 11:01, David Ahern wrote:
> On 12/6/21 11:05 PM, lizhijian@fujitsu.com wrote:
>>> #################################################################
>>> With VRF
>>>
>>> TEST: Raw socket bind to local address - ns-A IP                              [ OK ]
>>> TEST: Raw socket bind to local address after device bind - ns-A IP            [ OK ]
>>> TEST: Raw socket bind to local address after VRF bind - ns-A IP               [ OK ]
>>> TEST: Raw socket bind to local address - VRF IP                               [FAIL]
>>>
>> i found that above case failed with "server: error binding socket: 99: Cannot assign requested address"
>> i have manually check it with below command after setup(), same errors:
>>
>> # ip netns exec ns-A nettest -s -R -P icmp -l 172.16.3.1 -b
>> 05:55:11 server: error binding socket: 99: Cannot assign requested address
>>
>> But when i specified specific network interface, it works
>> # ip netns exec ns-A nettest -s -R -P icmp -l 172.16.3.1 -b -I red
>> # echo $?
>> 0
>> # ip netns exec ns-A nettest -s -R -P icmp -l 172.16.3.1 -b
>> 06:01:55 server: error binding socket: 99: Cannot assign requested address
>> # echo $?
>> 1
>>
>>
>> So i wonder if i missed something ?
>>
> That test should be a negative test as is the first one in that group -
> in both cases the address bind should fail since the socket is not in
> the VRF but the address is. The first on currently shows "OK" but that
> is because of 5cad8bce26e01 that made changes to the config to validate
> MD5 changes. Will send a patch to fix.
>
Got it, thanks a lot for your help.
I'm looking forward to it :)

Thanks
Zhijian

Re: [PATCH v2] selftests: net: Correct case name

[lizhijian]

On 08/12/2021 11:14, David Ahern wrote:
> On 12/6/21 11:05 PM, lizhijian@fujitsu.com wrote:
>>> # TESTS=bind6 ./fcnal-test.sh
>>>
>>> ###########################################################################
>>> IPv6 address binds
>>> ###########################################################################
>>>
>>>
>>> #################################################################
>>> No VRF
>>>
>>> TEST: Raw socket bind to local address - ns-A IPv6                            [FAIL]
> This one passes for me.
Err, i didn't notice this one when i sent this mail. Since it was passed too in my
previous multiple runs.




>
> Can you run the test with '-v -p'? -v will give you the command line
> that is failing. -p will pause the tests at the failure. From there you
> can do:
>
> ip netns exec ns-A bash
>
> Look at the routing - no VRF is involved so the address should be local
> to the device and the loopback. Run the test manually to see if it
> really is failing.

thanks for your advice, i will take a look if it appears again.



>
>
>>> TEST: Raw socket bind to local address after device bind - ns-A IPv6          [ OK ]
>>> TEST: Raw socket bind to local address - ns-A loopback IPv6                   [ OK ]
>>> TEST: Raw socket bind to local address after device bind - ns-A loopback IPv6  [ OK ]
>>> TEST: TCP socket bind to local address - ns-A IPv6                            [ OK ]
>>> TEST: TCP socket bind to local address after device bind - ns-A IPv6          [ OK ]
>>> TEST: TCP socket bind to out of scope local address - ns-A loopback IPv6      [FAIL]
> This one seems to be a new problem. The socket is bound to eth1 and the
> address bind is to an address on loopback. That should not be working.

My colleague had another thread with the verbose detailed message
https://lore.kernel.org/netdev/PH0PR11MB4792DC680F7E383D72C2E8C5C56E9@PH0PR11MB4792.namprd11.prod.outlook.com/



>
>>> #################################################################
>>> With VRF
>>>
>>> TEST: Raw socket bind to local address after vrf bind - ns-A IPv6             [ OK ]
>>> TEST: Raw socket bind to local address after device bind - ns-A IPv6          [ OK ]
>>> TEST: Raw socket bind to local address after vrf bind - VRF IPv6              [ OK ]
>>> TEST: Raw socket bind to local address after device bind - VRF IPv6           [ OK ]
>>> TEST: Raw socket bind to invalid local address after vrf bind - ns-A loopback IPv6  [ OK ]
>>> TEST: TCP socket bind to local address with VRF bind - ns-A IPv6              [ OK ]
>>> TEST: TCP socket bind to local address with VRF bind - VRF IPv6               [ OK ]
>>> TEST: TCP socket bind to local address with device bind - ns-A IPv6           [ OK ]
>>> TEST: TCP socket bind to VRF address with device bind - VRF IPv6              [FAIL]
> This failure is similar to the last one. Need to see if a recent commit
> changed something.

Good to know this


Thanks
Zhijian

>
>
>>> TEST: TCP socket bind to invalid local address for VRF - ns-A loopback IPv6   [ OK ]
>>> TEST: TCP socket bind to invalid local address for device bind - ns-A loopback IPv6  [ OK ]
>>
>> Thanks
>> Zhijian
>>
>>

Re: [PATCH v2] selftests: net: Correct case name

[David Ahern]

On 12/7/21 8:38 PM, lizhijian@fujitsu.com wrote:
> 
> 
> On 08/12/2021 11:14, David Ahern wrote:
>> On 12/6/21 11:05 PM, lizhijian@fujitsu.com wrote:
>>>> # TESTS=bind6 ./fcnal-test.sh
>>>>
>>>> ###########################################################################
>>>> IPv6 address binds
>>>> ###########################################################################
>>>>
>>>>
>>>> #################################################################
>>>> No VRF
>>>>
>>>> TEST: Raw socket bind to local address - ns-A IPv6                            [FAIL]
>> This one passes for me.
> Err, i didn't notice this one when i sent this mail. Since it was passed too in my
> previous multiple runs.
> 
> 
> 
> 
>>
>> Can you run the test with '-v -p'? -v will give you the command line
>> that is failing. -p will pause the tests at the failure. From there you
>> can do:
>>
>> ip netns exec ns-A bash
>>
>> Look at the routing - no VRF is involved so the address should be local
>> to the device and the loopback. Run the test manually to see if it
>> really is failing.
> 
> thanks for your advice, i will take a look if it appears again.
> 
> 
> 
>>
>>
>>>> TEST: Raw socket bind to local address after device bind - ns-A IPv6          [ OK ]
>>>> TEST: Raw socket bind to local address - ns-A loopback IPv6                   [ OK ]
>>>> TEST: Raw socket bind to local address after device bind - ns-A loopback IPv6  [ OK ]
>>>> TEST: TCP socket bind to local address - ns-A IPv6                            [ OK ]
>>>> TEST: TCP socket bind to local address after device bind - ns-A IPv6          [ OK ]
>>>> TEST: TCP socket bind to out of scope local address - ns-A loopback IPv6      [FAIL]
>> This one seems to be a new problem. The socket is bound to eth1 and the
>> address bind is to an address on loopback. That should not be working.

actually that one should be commented out similar to the test at the end
of ipv4_addr_bind_novrf. It documents unexpected behavior - binding to a
device should limit the addresses it can bind to but the kernel does
not. Legacy behavior.

> 
> My colleague had another thread with the verbose detailed message
> https://lore.kernel.org/netdev/PH0PR11MB4792DC680F7E383D72C2E8C5C56E9@PH0PR11MB4792.namprd11.prod.outlook.com/
> 
> 
> 
>>
>>>> #################################################################
>>>> With VRF
>>>>
>>>> TEST: Raw socket bind to local address after vrf bind - ns-A IPv6             [ OK ]
>>>> TEST: Raw socket bind to local address after device bind - ns-A IPv6          [ OK ]
>>>> TEST: Raw socket bind to local address after vrf bind - VRF IPv6              [ OK ]
>>>> TEST: Raw socket bind to local address after device bind - VRF IPv6           [ OK ]
>>>> TEST: Raw socket bind to invalid local address after vrf bind - ns-A loopback IPv6  [ OK ]
>>>> TEST: TCP socket bind to local address with VRF bind - ns-A IPv6              [ OK ]
>>>> TEST: TCP socket bind to local address with VRF bind - VRF IPv6               [ OK ]
>>>> TEST: TCP socket bind to local address with device bind - ns-A IPv6           [ OK ]
>>>> TEST: TCP socket bind to VRF address with device bind - VRF IPv6              [FAIL]
>> This failure is similar to the last one. Need to see if a recent commit
>> changed something.
> 

similarly here. Want to send a patch that comments them out with the
same explanation as in ipv4_addr_bind_novrf?

Both fail on v5.8 so I do not believe a recent change affected either
test. I guess these bind tests slipped through the cracks with the
misname in the TESTS variable. Thanks for the patch to fix that.

Also, make sure you always cc the author of the Fixes tag when sending
patches.

Re: [PATCH v2] selftests: net: Correct case name

[Zhou, Jie2X]

hi,

I try to apply the "selftests: Fix raw socket bind tests with VRF" patch.
https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=0f108ae44520

And found that following changes.
TEST: Raw socket bind to local address - ns-A IP                              [ OK ]
=> TEST: Raw socket bind to local address - ns-A IP                              [FAIL]
TEST: Raw socket bind to local address - VRF IP                               [FAIL]
=> TEST: Raw socket bind to local address - VRF IP                               [ OK ]

Use -v to see the failed details.
#######################################################
COMMAND: ip netns exec ns-A nettest -s -R -P icmp -l 172.16.1.1 -b

TEST: Raw socket bind to local address - ns-A IP                              [FAIL]

best regards,

________________________________________
From: David Ahern <dsahern@gmail.com>
Sent: Wednesday, December 8, 2021 12:07 PM
To: lizhijian@fujitsu.com; davem@davemloft.net; kuba@kernel.org; shuah@kernel.org
Cc: netdev@vger.kernel.org; linux-kselftest@vger.kernel.org; linux-kernel@vger.kernel.org; Zhou, Jie2X; Li, ZhijianX
Subject: Re: [PATCH v2] selftests: net: Correct case name

On 12/7/21 8:38 PM, lizhijian@fujitsu.com wrote:
>
>
> On 08/12/2021 11:14, David Ahern wrote:
>> On 12/6/21 11:05 PM, lizhijian@fujitsu.com wrote:
>>>> # TESTS=bind6 ./fcnal-test.sh
>>>>
>>>> ###########################################################################
>>>> IPv6 address binds
>>>> ###########################################################################
>>>>
>>>>
>>>> #################################################################
>>>> No VRF
>>>>
>>>> TEST: Raw socket bind to local address - ns-A IPv6                            [FAIL]
>> This one passes for me.
> Err, i didn't notice this one when i sent this mail. Since it was passed too in my
> previous multiple runs.
>
>
>
>
>>
>> Can you run the test with '-v -p'? -v will give you the command line
>> that is failing. -p will pause the tests at the failure. From there you
>> can do:
>>
>> ip netns exec ns-A bash
>>
>> Look at the routing - no VRF is involved so the address should be local
>> to the device and the loopback. Run the test manually to see if it
>> really is failing.
>
> thanks for your advice, i will take a look if it appears again.
>
>
>
>>
>>
>>>> TEST: Raw socket bind to local address after device bind - ns-A IPv6          [ OK ]
>>>> TEST: Raw socket bind to local address - ns-A loopback IPv6                   [ OK ]
>>>> TEST: Raw socket bind to local address after device bind - ns-A loopback IPv6  [ OK ]
>>>> TEST: TCP socket bind to local address - ns-A IPv6                            [ OK ]
>>>> TEST: TCP socket bind to local address after device bind - ns-A IPv6          [ OK ]
>>>> TEST: TCP socket bind to out of scope local address - ns-A loopback IPv6      [FAIL]
>> This one seems to be a new problem. The socket is bound to eth1 and the
>> address bind is to an address on loopback. That should not be working.

actually that one should be commented out similar to the test at the end
of ipv4_addr_bind_novrf. It documents unexpected behavior - binding to a
device should limit the addresses it can bind to but the kernel does
not. Legacy behavior.

>
> My colleague had another thread with the verbose detailed message
> https://lore.kernel.org/netdev/PH0PR11MB4792DC680F7E383D72C2E8C5C56E9@PH0PR11MB4792.namprd11.prod.outlook.com/
>
>
>
>>
>>>> #################################################################
>>>> With VRF
>>>>
>>>> TEST: Raw socket bind to local address after vrf bind - ns-A IPv6             [ OK ]
>>>> TEST: Raw socket bind to local address after device bind - ns-A IPv6          [ OK ]
>>>> TEST: Raw socket bind to local address after vrf bind - VRF IPv6              [ OK ]
>>>> TEST: Raw socket bind to local address after device bind - VRF IPv6           [ OK ]
>>>> TEST: Raw socket bind to invalid local address after vrf bind - ns-A loopback IPv6  [ OK ]
>>>> TEST: TCP socket bind to local address with VRF bind - ns-A IPv6              [ OK ]
>>>> TEST: TCP socket bind to local address with VRF bind - VRF IPv6               [ OK ]
>>>> TEST: TCP socket bind to local address with device bind - ns-A IPv6           [ OK ]
>>>> TEST: TCP socket bind to VRF address with device bind - VRF IPv6              [FAIL]
>> This failure is similar to the last one. Need to see if a recent commit
>> changed something.
>

similarly here. Want to send a patch that comments them out with the
same explanation as in ipv4_addr_bind_novrf?

Both fail on v5.8 so I do not believe a recent change affected either
test. I guess these bind tests slipped through the cracks with the
misname in the TESTS variable. Thanks for the patch to fix that.

Also, make sure you always cc the author of the Fixes tag when sending
patches.

Re: [PATCH v2] selftests: net: Correct case name

[David Ahern]

On 12/12/21 8:08 PM, Zhou, Jie2X wrote:
> hi,
> 
> I try to apply the "selftests: Fix raw socket bind tests with VRF" patch.
> https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=0f108ae44520
> 
> And found that following changes.
> TEST: Raw socket bind to local address - ns-A IP                              [ OK ]
> => TEST: Raw socket bind to local address - ns-A IP                              [FAIL]
> TEST: Raw socket bind to local address - VRF IP                               [FAIL]
> => TEST: Raw socket bind to local address - VRF IP                               [ OK ]
> 

After the last round of patches all tests but 2 pass with the 5.16.0-rc3
kernel (net-next based) and ubuntu 20.04 OS.

The 2 failures are due local pings and to bugs in 'ping' - it removes
the device bind by calling setsockopt with an "" arg.

Re: [PATCH v2] selftests: net: Correct case name

[Zhou, Jie2X]

hi,

>After the last round of patches all tests but 2 pass with the 5.16.0-rc3
>kernel (net-next based) and ubuntu 20.04 OS.
>The 2 failures are due local pings and to bugs in 'ping' - it removes
>the device bind by calling setsockopt with an "" arg.

The failed testcase command is nettest not ping.
COMMAND: ip netns exec ns-A nettest -s -R -P icmp -l 172.16.1.1 -b
TEST: Raw socket bind to local address - ns-A IP                              [FAIL]

It failed because it return 0.
But the patch expected return 1.

May be the patch should expected 0 return value for  ${NSA_IP}.
And expected 1 return value for  ${VRF_IP}.

diff --git a/tools/testing/selftests/net/fcnal-test.sh b/tools/testing/selftests/net/fcnal-test.sh
index dd7437dd2680b..4340477863d36 100755
--- a/tools/testing/selftests/net/fcnal-test.sh
+++ b/tools/testing/selftests/net/fcnal-test.sh
@@ -1810,8 +1810,9 @@ ipv4_addr_bind_vrf()
        for a in ${NSA_IP} ${VRF_IP}
        do
                log_start
+               show_hint "Socket not bound to VRF, but address is in VRF"
                run_cmd nettest -s -R -P icmp -l ${a} -b
-               log_test_addr ${a} $? 0 "Raw socket bind to local address"
+               log_test_addr ${a} $? 1 "Raw socket bind to local address"

                log_start
                run_cmd nettest -s -R -P icmp -l ${a} -I ${NSA_DEV} -b

best regards,

________________________________________
From: David Ahern <dsahern@gmail.com>
Sent: Monday, December 13, 2021 12:09 PM
To: Zhou, Jie2X; lizhijian@fujitsu.com; davem@davemloft.net; kuba@kernel.org; shuah@kernel.org
Cc: netdev@vger.kernel.org; linux-kselftest@vger.kernel.org; linux-kernel@vger.kernel.org; Li, ZhijianX; Li, Philip; Ma, XinjianX
Subject: Re: [PATCH v2] selftests: net: Correct case name

On 12/12/21 8:08 PM, Zhou, Jie2X wrote:
> hi,
>
> I try to apply the "selftests: Fix raw socket bind tests with VRF" patch.
> https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=0f108ae44520
>
> And found that following changes.
> TEST: Raw socket bind to local address - ns-A IP                              [ OK ]
> => TEST: Raw socket bind to local address - ns-A IP                              [FAIL]
> TEST: Raw socket bind to local address - VRF IP                               [FAIL]
> => TEST: Raw socket bind to local address - VRF IP                               [ OK ]
>

After the last round of patches all tests but 2 pass with the 5.16.0-rc3
kernel (net-next based) and ubuntu 20.04 OS.

The 2 failures are due local pings and to bugs in 'ping' - it removes
the device bind by calling setsockopt with an "" arg.

Re: [PATCH v2] selftests: net: Correct case name

[David Ahern]

On 12/13/21 2:44 AM, Zhou, Jie2X wrote:
> hi,
> 
>> After the last round of patches all tests but 2 pass with the 5.16.0-rc3
>> kernel (net-next based) and ubuntu 20.04 OS.
>> The 2 failures are due local pings and to bugs in 'ping' - it removes
>> the device bind by calling setsockopt with an "" arg.
> 
> The failed testcase command is nettest not ping.
> COMMAND: ip netns exec ns-A nettest -s -R -P icmp -l 172.16.1.1 -b
> TEST: Raw socket bind to local address - ns-A IP                              [FAIL]
> 
> It failed because it return 0.
> But the patch expected return 1.
> 
> May be the patch should expected 0 return value for  ${NSA_IP}.
> And expected 1 return value for  ${VRF_IP}.
> 
> diff --git a/tools/testing/selftests/net/fcnal-test.sh b/tools/testing/selftests/net/fcnal-test.sh
> index dd7437dd2680b..4340477863d36 100755
> --- a/tools/testing/selftests/net/fcnal-test.sh
> +++ b/tools/testing/selftests/net/fcnal-test.sh
> @@ -1810,8 +1810,9 @@ ipv4_addr_bind_vrf()
>         for a in ${NSA_IP} ${VRF_IP}
>         do
>                 log_start
> +               show_hint "Socket not bound to VRF, but address is in VRF"
>                 run_cmd nettest -s -R -P icmp -l ${a} -b
> -               log_test_addr ${a} $? 0 "Raw socket bind to local address"
> +               log_test_addr ${a} $? 1 "Raw socket bind to local address"
> 
>                 log_start
>                 run_cmd nettest -s -R -P icmp -l ${a} -I ${NSA_DEV} -b
> 

apply *all* patches.

https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=0f108ae44520

Re: [PATCH v2] selftests: net: Correct case name

[Zhou, Jie2X]

hi,

  I try to test the "selftests: Fix raw socket bind tests with VRF" patch.
  https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=0f108ae44520

  But the TEST: Raw socket bind to local address - ns-A IP  which used to be passed became failed.

best regards,

________________________________________
From: David Ahern <dsahern@gmail.com>
Sent: Monday, December 13, 2021 11:58 PM
To: Zhou, Jie2X; lizhijian@fujitsu.com; davem@davemloft.net; kuba@kernel.org; shuah@kernel.org
Cc: netdev@vger.kernel.org; linux-kselftest@vger.kernel.org; linux-kernel@vger.kernel.org; Li, ZhijianX; Li, Philip; Ma, XinjianX
Subject: Re: [PATCH v2] selftests: net: Correct case name

On 12/13/21 2:44 AM, Zhou, Jie2X wrote:
> hi,
>
>> After the last round of patches all tests but 2 pass with the 5.16.0-rc3
>> kernel (net-next based) and ubuntu 20.04 OS.
>> The 2 failures are due local pings and to bugs in 'ping' - it removes
>> the device bind by calling setsockopt with an "" arg.
>
> The failed testcase command is nettest not ping.
> COMMAND: ip netns exec ns-A nettest -s -R -P icmp -l 172.16.1.1 -b
> TEST: Raw socket bind to local address - ns-A IP                              [FAIL]
>
> It failed because it return 0.
> But the patch expected return 1.
>
> May be the patch should expected 0 return value for  ${NSA_IP}.
> And expected 1 return value for  ${VRF_IP}.
>
> diff --git a/tools/testing/selftests/net/fcnal-test.sh b/tools/testing/selftests/net/fcnal-test.sh
> index dd7437dd2680b..4340477863d36 100755
> --- a/tools/testing/selftests/net/fcnal-test.sh
> +++ b/tools/testing/selftests/net/fcnal-test.sh
> @@ -1810,8 +1810,9 @@ ipv4_addr_bind_vrf()
>         for a in ${NSA_IP} ${VRF_IP}
>         do
>                 log_start
> +               show_hint "Socket not bound to VRF, but address is in VRF"
>                 run_cmd nettest -s -R -P icmp -l ${a} -b
> -               log_test_addr ${a} $? 0 "Raw socket bind to local address"
> +               log_test_addr ${a} $? 1 "Raw socket bind to local address"
>
>                 log_start
>                 run_cmd nettest -s -R -P icmp -l ${a} -I ${NSA_DEV} -b
>

apply *all* patches.

https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=0f108ae44520