Re: Ping frame drop

Re: Ping frame drop

[Jakub Kicinski]

On Fri, 18 Jun 2021 15:03:59 +0000 Arijit De wrote:
> Hi,
> 
> In the latest linux kernel (i.e. 5.12.x) I am observing that for my
> Ethernet driver ping test has stopped working, it was working in
> 5.4.x and all the older kernels. I have debugged the issue and root
> caused that it's because of the recent commit
> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=8f9a69a92fc63917c9bd921b28e3b2912980becf
> 
> In my Network card HW it supports checksum offload for IPv4 frame,
> but it can't verify checksum for the ICMP frames, so I use
> CHECKSUM_PARTIAL in the skb->ip_summed for this kind of scenario.

Do you mean that your drivers sets up CHECKSUM_PARTIAL on Rx? If HW
hasn't validated the checksum just leave the skb with CHECKSUM_NONE,
the stack will validate.

> But now because of this new logic what you have added ping frames are
> getting dropped.
> 
> My Ping packets skb dump:
> [112241.545219] skb len=88 headroom=78 headlen=0 tailroom=0
> [112241.545219] mac=(64,-64) net=(0,-1) trans=-1
> [112241.545219] shinfo(txflags=0 nr_frags=1 gso(size=0 type=0 segs=0))
> [112241.545219] csum(0x0 ip_summed=3 complete_sw=0 valid=0 level=0)
> [112241.545219] hash(0x0 sw=0 l4=0) proto=0x0800 pkttype=0 iif=0
> [112241.572837] dev name=enp137s0 feat=0x0x0000010000004813
> [112241.578141] skb headroom: 00000000: 4c 00 70 35 09 de b4 e4 00 11
> 22 33 44 01 08 06 [112241.585876] skb headroom: 00000010: 00 01 08 00
> 06 04 00 02 00 11 22 33 44 01 0a 1c [112241.593611] skb headroom:
> 00000020: 28 13 70 35 09 de b4 e4 0a 1c 28 01 00 00 00 00
> [112241.601345] skb headroom: 00000030: 50 04 00 00 55 50 00 00 14 00
> 03 00 00 00 00 00 [112241.609080] skb headroom: 00000040: 00 11 22 33
> 44 01 ac 1f 6b d2 c0 e5 08 00 [112241.616293] skb frag:     00000000:
> 45 00 00 54 87 a2 40 00 40 01 4d e3 0a 1c 28 d9 [112241.624027] skb
> frag:     00000010: 0a 1c 28 13 08 00 fd 50 0c a8 00 03 6a 94 cc 60
> [112241.631762] skb frag:     00000020: 00 00 00 00 f5 3b 03 00 00 00
> 00 00 10 11 12 13 [112241.639496] skb frag:     00000030: 14 15 16 17
> 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 [112241.647230] skb frag:
> 00000040: 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33
> [112241.654965] skb frag:     00000050: 34 35 36 37 9c b0 19 b9
> 
> Its hitting the check for CHECKSUM_PARTIAL in pskb_trim_rcsum_slow()
> and getting dropped there. Can you please let me know how can I
> satisfy the requirement such that I can keep supporting the
> CHECKSUM_PARTIAL cases for my network card ? I have checked
> include/linux/skbuff.h For the documentation of CHECKSUM_PARTIAL, but
> could not understand what change I have to do to make it working
> again. My network driver is not up streamed yet in the linux kernel.
> For any more information please do let me know.

RE: [EXT] Re: Ping frame drop

[Arijit De]

Hi Jakub,

In my network card HW it can verify the received frame's checksum of IPv4 header, but it can't verify the checksum of ICMP header.
So for ICMP kind of received frames driver sets the checksum state to CHECKSUM_PARTIAL in skb->ip_summed. Which is as per the linux kernel documentation also.
Now before the commit  https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=8f9a69a92fc63917c9bd921b28e3b2912980becf this use case was working.
But now after introducing this logic for CHECKSUM_PARTIAL case, my received ICMP ping frames are getting dropped in the linux kernel pskb_trim_rcsum_slow(). 
I do understand that to bypass this scenario I can use CHECKSUM_NONE, but in that case HW's capability where checksum is already verified for the IPv4 header will be unutilized. 

So please do share if any documentation update has happened for the CHECKSUM_PARTIAL scenario or please do let me know what need to be updated in the skb for the receive frame in this scenario where only Networking layer (i.e. IPv4 in this case) checksum is verified but the ICMP(ping) header checksum is not verified  ?

Thanks
Arijit

-----Original Message-----
From: Jakub Kicinski <kuba@kernel.org> 
Sent: Saturday, June 19, 2021 3:50 AM
To: Arijit De <arijitde@marvell.com>
Cc: netdev@vger.kernel.org
Subject: [EXT] Re: Ping frame drop

External Email

----------------------------------------------------------------------
On Fri, 18 Jun 2021 15:03:59 +0000 Arijit De wrote:
> Hi,
> 
> In the latest linux kernel (i.e. 5.12.x) I am observing that for my 
> Ethernet driver ping test has stopped working, it was working in 5.4.x 
> and all the older kernels. I have debugged the issue and root caused 
> that it's because of the recent commit 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__git.kernel.org_pu
> b_scm_linux_kernel_git_stable_linux.git_commit_-3Fid-3D8f9a69a92fc6391
> 7c9bd921b28e3b2912980becf&d=DwICAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=TaxnDgE1
> 8KxGhqsUtny5eMiBbXC81IdotAx6pqBUXWE&m=llYaO7Eowy-EL2vj6Lr3tV6AWHL1zodg
> flS7funQd3I&s=fkqonnyQCEHc929sQLcwua-9L5utQkXp_kAr73NhQiA&e=
> 
> In my Network card HW it supports checksum offload for IPv4 frame, but 
> it can't verify checksum for the ICMP frames, so I use 
> CHECKSUM_PARTIAL in the skb->ip_summed for this kind of scenario.

Do you mean that your drivers sets up CHECKSUM_PARTIAL on Rx? If HW hasn't validated the checksum just leave the skb with CHECKSUM_NONE, the stack will validate.

> But now because of this new logic what you have added ping frames are 
> getting dropped.
> 
> My Ping packets skb dump:
> [112241.545219] skb len=88 headroom=78 headlen=0 tailroom=0 
> [112241.545219] mac=(64,-64) net=(0,-1) trans=-1 [112241.545219] 
> shinfo(txflags=0 nr_frags=1 gso(size=0 type=0 segs=0)) [112241.545219] 
> csum(0x0 ip_summed=3 complete_sw=0 valid=0 level=0) [112241.545219] 
> hash(0x0 sw=0 l4=0) proto=0x0800 pkttype=0 iif=0 [112241.572837] dev 
> name=enp137s0 feat=0x0x0000010000004813 [112241.578141] skb headroom: 
> 00000000: 4c 00 70 35 09 de b4 e4 00 11
> 22 33 44 01 08 06 [112241.585876] skb headroom: 00000010: 00 01 08 00
> 06 04 00 02 00 11 22 33 44 01 0a 1c [112241.593611] skb headroom:
> 00000020: 28 13 70 35 09 de b4 e4 0a 1c 28 01 00 00 00 00 
> [112241.601345] skb headroom: 00000030: 50 04 00 00 55 50 00 00 14 00
> 03 00 00 00 00 00 [112241.609080] skb headroom: 00000040: 00 11 22 33
> 44 01 ac 1f 6b d2 c0 e5 08 00 [112241.616293] skb frag:     00000000:
> 45 00 00 54 87 a2 40 00 40 01 4d e3 0a 1c 28 d9 [112241.624027] skb
> frag:     00000010: 0a 1c 28 13 08 00 fd 50 0c a8 00 03 6a 94 cc 60
> [112241.631762] skb frag:     00000020: 00 00 00 00 f5 3b 03 00 00 00
> 00 00 10 11 12 13 [112241.639496] skb frag:     00000030: 14 15 16 17
> 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 [112241.647230] skb frag:
> 00000040: 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33
> [112241.654965] skb frag:     00000050: 34 35 36 37 9c b0 19 b9
> 
> Its hitting the check for CHECKSUM_PARTIAL in pskb_trim_rcsum_slow() 
> and getting dropped there. Can you please let me know how can I 
> satisfy the requirement such that I can keep supporting the 
> CHECKSUM_PARTIAL cases for my network card ? I have checked 
> include/linux/skbuff.h For the documentation of CHECKSUM_PARTIAL, but 
> could not understand what change I have to do to make it working 
> again. My network driver is not up streamed yet in the linux kernel.
> For any more information please do let me know.

Re: [EXT] Re: Ping frame drop

[Jakub Kicinski]

Please don't top post.

On Sat, 19 Jun 2021 15:14:35 +0000 Arijit De wrote:
> In my network card HW it can verify the received frame's checksum of
> IPv4 header, but it can't verify the checksum of ICMP header. 

Linux does not offload IPv4 header checksums.

> So for
> ICMP kind of received frames driver sets the checksum state to
> CHECKSUM_PARTIAL in skb->ip_summed. Which is as per the linux kernel
> documentation also. 

What documentation are you reading? Setting ip_summed to PARTIAL on
receive is only valid for software/virtual devices, never real HW.

> Now before the commit
> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=8f9a69a92fc63917c9bd921b28e3b2912980becf
> this use case was working. But now after introducing this logic for
> CHECKSUM_PARTIAL case, my received ICMP ping frames are getting
> dropped in the linux kernel pskb_trim_rcsum_slow(). I do understand
> that to bypass this scenario I can use CHECKSUM_NONE, but in that
> case HW's capability where checksum is already verified for the IPv4
> header will be unutilized. 
> 
> So please do share if any documentation update has happened for the
> CHECKSUM_PARTIAL scenario or please do let me know what need to be
> updated in the skb for the receive frame in this scenario where only
> Networking layer (i.e. IPv4 in this case) checksum is verified but
> the ICMP(ping) header checksum is not verified  ?