Re: [PATCH v4 3/3] mbuf:replace the inner_l2_len and the inner_l3_len fields

[Olivier MATZ <olivier.matz-pdR9zngts4EAvxtiuMwx3w@public.gmane.org>]

Hi Didier, Konstantin, Jijiang,

On 12/02/2014 04:36 PM, Ananyev, Konstantin wrote:
> Hi Didier
>
>> -----Original Message-----
>> From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of didier.pallard
>> Sent: Tuesday, December 02, 2014 2:53 PM
>> To: Liu, Jijiang; dev-VfR2kkLFssw@public.gmane.org
>> Subject: Re: [dpdk-dev] [PATCH v4 3/3] mbuf:replace the inner_l2_len and the inner_l3_len fields
>>
>> Hello,
>>
>> On 12/02/2014 07:52 AM, Jijiang Liu wrote:
>>> Replace the inner_l2_len and the inner_l3_len field with the outer_l2_len and outer_l3_len field, and rework csum forward engine
>> and i40e PMD due to  these changes.
>>>
>>> Signed-off-by: Jijiang Liu <jijiang.liu-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>> [...]
>>> --- a/lib/librte_mbuf/rte_mbuf.h
>>> +++ b/lib/librte_mbuf/rte_mbuf.h
>>> @@ -276,8 +276,8 @@ struct rte_mbuf {
>>>    			uint64_t tso_segsz:16; /**< TCP TSO segment size */
>>>
>>>    			/* fields for TX offloading of tunnels */
>>> -			uint64_t inner_l3_len:9; /**< inner L3 (IP) Hdr Length. */
>>> -			uint64_t inner_l2_len:7; /**< inner L2 (MAC) Hdr Length. */
>>> +			uint64_t outer_l3_len:9; /**< Outer L3 (IP) Hdr Length. */
>>> +			uint64_t outer_l2_len:7; /**< Outer L2 (MAC) Hdr Length. */
>>>
>>>    			/* uint64_t unused:8; */
>>>    		};
>>
>> Sorry for entering lately this discussion, but i'm not convinced by the
>> choice of outer_lx_len rather than inner_lx_len for new fields.
>> I agree with Olivier that new flags should only be related to the use of
>> new fields, to maintain coherency with oldest implementations.
>> But from a stack point of view, i think it is better to have lx_len
>> fields that target the outer layers, and to name new fields inner_lx_len.
>>
>> Let's discuss the two possibilities.
>>
>> 1) outer_lx_len fields are introduced.
>> In this case, the stack should have knowledge that it is processing
>> tunneled packets to use outer_lx_len rather than lx_len,
>> or stack should always use outer_lx_len packet and move those fields to
>> lx_len packets if no tunneling occurs...
>> I think it will induce extra processing that does not seem to be really
>> needed.
>>
>> 2) inner_lx_len fields are introduced.
>> In this case, the stack first uses lx_len fields. When packet should be
>> tunneled, lx_len fields are moved to inner_lx_len fields.
>> Then the stack can process the outer layer and still use the lx_len fields.
>
> Not sure, that I understood why 2) is better than 1).
> Let say,  you have a 'normal' (non-tunnelling) packet: ether/IP/TCP
> In that case you still use mbuf's l2_len/l3_len/l4_len fields and setup ol_flags as usual.
> Then later, you decided to 'tunnel' that packet.
> So you just fill mbuf's outer_l2_len/outer_l3_len, setup TX_OUTER_* and TX_TUNNEL_* bits in ol_flags and probably update l2_len.
> l3_len/l4_len and ol_flags bits set for them remain intact.
> That's with 1)
>
> With 2) - you'll have to move l3_len/l4_len to inner_lx_len.
> And I suppose ol_flags values too:
> ol_flags &= ~PKT_ IP_CKSUM;
> ol_flgas  |=  PKT_INNER_IP_CKSUM
> ?
> And same for L4_CKSUM flags too?

After reading Didier's mail, I start to be convinced that keeping inner
may be better than outer. From a network stack architecture point of
view, 2) looks better:

- the functions in the network stack that write the Ether/IP header
   always deal with l2_len/l3_len, whatever it's a tunnel or not.

- the function that adds the tunnel header moves the l2_len/l3_len and
   the flags to inner_l2_len/inner_l3_len and inner_flags.

Althought it was my first idea, now I cannot find a better argument in
favor of outer_lX_len. The initial argument was that the correspondance
between a flag and a lX_len should always remain the same, but it is
still possible with Didier's approach:
   - PKT_IP_CKSUM uses l2_len and l3_len
   - PKT_INNER_CKSUM uses inner_l2_len and inner_l3_len

Jijiang, I'm sorry to change my mind about this. If you want (and if
Konstantin is also ok with that), I can try to rebase your patches to
match this. Or do you prefer to do it by yourself?

Regards,
Olivier