Re: [PATCH net-next v4 11/11] net: dsa: realtek: rtl8365mb: multiple cpu ports, non cpu extint

[Florian Fainelli <f.fainelli@gmail.com>]

On 1/29/2022 8:42 PM, Luiz Angelo Daros de Luca wrote:
>>> I suggested it might be checksum problem because I'm also affected. In
>>> my case, I have an mt7620a SoC connected to the rtl8367s switch. The
>>> OS offloads checksum to HW but the mt7620a cannot calculate the
>>> checksum with the (EtherType) Realtek CPU Tag in place. I'll try to
>>> move the CPU tag to test if the mt7620a will then digest the frame
>>> correctly.
>>
>> I implemented a new DSA tag (rtl8_4t, with "t" as in trailing) that
>> puts the DSA tag before the Ethernet CRC (the switch supports both).
>> With no tag in the mac layer, mediatek correctly calculated the ip
>> checksum. However, mediatek SoC included the extra bytes from the DSA
>> tag in the TCP checksum, even if they are after the ip length.
>>
>> This is the packet leaving the OS:
>>
>> 0000   04 0e 3c fc 4f aa 50 d4 f7 33 15 8a 08 00 45 10
>> 0010   00 3c 00 00 40 00 40 06 b7 58 c0 a8 01 01 c0 a8
>> 0020   01 02 00 16 a1 50 80 da 39 e9 b2 2a 23 cf a0 12
>> 0030   fe 88 83 82 00 00 02 04 05 b4 04 02 08 0a 01 64
>> 0040   fb 28 66 42 e0 79 01 03 03 03 88 99 04 00 00 20
>> 0050   00 08
>>
>> TCP checksum is at 0x0032 with 0x8382 is the tcp checksum
>> DSA Tag is at 0x4a with 8899040000200008
>>
>> This is what arrived at the other end:
>>
>> 0000   04 0e 3c fc 4f aa 50 d4 f7 33 15 8a 08 00 45 10
>> 0010   00 3c 00 00 40 00 40 06 b7 58 c0 a8 01 01 c0 a8
>> 0020   01 02 00 16 a1 50 80 da 39 e9 b2 2a 23 cf a0 12
>> 0030   fe 88 c3 e8 00 00 02 04 05 b4 04 02 08 0a 01 64
>> 0040   fb 28 66 42 e0 79 01 03 03 03
>>
>> TCP checksum is 0xc3e8, but the correct one should be 0x50aa
>> If you calculate tcp checksum including 8899040000200008, you'll get exactly
>> 0xc3e8 (I did the math).
>>
>> So, If we use a trailing DSA tag, we can leave the IP checksum offloading on
>> and just turn off the TCP checksum offload. Is it worth it?
> 
> No, IP checksum is always done in SW.
> 
>> Is it still interesting to have the rtl8_4t merged?
> 
> Maybe it is. It has uncovered a problem. The case of trailing tags
> seems to be unsolvable even with csum_start. AFAIK, the driver must
> cksum from "skb->csum_start up to the end". When the switch is using
> an incompatible tag, we have:
> 
> slave(): my features copied from master tells me I can offload
> checksum. Do nothing
> tagger(): add tag to the end of skb
> master(): Offloading HW, chksum from csum_start until the end,
> including the added tag
> switch(): remove the tag, forward to the network
> remove_client(): I got a packet with a broken checksum.

This is unfortunately expected here, because you program the hardware 
with the full Ethernet frame length which does include the trailer tag, 
and it then uses that length to calculate the transport header checksum 
over the enter payload, thinking the trailer tag is the UDP/TCP payload.

The checksum is calculated "on the fly" as part of the DMA operation to 
send the packet on the wire, so you cannot decouple the checksum 
calculation from the DMA operation, other than by not asking the HW *not 
to* checksum the packet, and instead having software provide that.

Now looking at the datasheet you quoted, there is this:

241. FE_GLO_CFG: Frame Engine Global Configuration (offset: 0x0000)

7:4 RW L2_SPACE L2 Space
(unit: 8 bytes)
0xB

Can you play with this and see if you can account for the extra 4 bytes 
added by the Realtek tag?

> 
> ndo_features_check() will not help because, either in HW or SW, it is
> expected to calculate the checksum up to the end. However, there is no
> csum_end or csum_len. I don't know if HW offloading will support some
> kind of csum_end but it would not be a problem in SW (considering
> skb_checksum_help() is adapted to something like skb_checksum_trimmed
> without the clone).
> 
> That amount of bytes to ignore at the end is a complex question: the
> driver either needs some hint (like it happens with skb->csum_offset)
> to know where transport payload ends or the taggers (or the dsa) must
> save the amount of extra bytes (or which tags were added) in the
> sbk_buff. With that info, the driver can check if HW will work with a
> different csum_start / csum_end or if only a supported tag is in use.
> 
> In my case, using an incompatible tailing tag, I just made it work
> hacking dsa and forcing slave interfaces to disable offloading. This
> way, checksum is calculated before any tag is added and offloading is
> skipped. But it is not a real solution.

Not sure which one is not a "real solution", but for this specific 
combination of DSA conduit driver and switch tag, you have to disable 
checksum offload in the conduit driver and provide it in software. The 
other way would be to configure the realtek switch to work with 
DSA_TAG_8021Q and see if you can continue to offload the data path since 
tagging would use regular 802.1Q vlans, but that means you are going to 
lose a whole lot of management functionality offered by the native 
Realtek tag.
-- 
Florian