Re: [RFC PATCH] sctp: Don't lookup dst if transport dst is still valid

[Vlad Yasevich <vyasevich@gmail.com>]

On 07/05/2013 10:09 AM, Vlad Yasevich wrote:
> On 07/03/2013 10:33 PM, Fan Du wrote:
>>
>>
>> On 2013年07月03日 21:23, Vlad Yasevich wrote:
>>> On 07/02/2013 10:18 PM, Fan Du wrote:
>>>>
>>>>
>>>> On 2013年07月02日 22:29, Vlad Yasevich wrote:
>>>>> On 07/02/2013 02:39 AM, Fan Du wrote:
>>>>>> When sctp sits on IPv6, sctp_transport_dst_check pass cookie as ZERO,
>>>>>> as a result ip6_dst_check always fail out. This behaviour makes
>>>>>> transport->dst useless, because every sctp_packet_transmit must look
>>>>>> for valid dst(Is this what supposed to be?)
>>>>>>
>>>>>> One aggressive way is to call rt_genid_bump which invalid all dst to
>>>>>> make new dst for transport, apparently it also hurts others.
>>>>>> I'm sure this may not be the best for all, so any commnets?
>>>>>>
>>>>>> Signed-off-by: Fan Du <fan.du@windriver.com>
>>>>>> ---
>>>>>> include/net/sctp/sctp.h | 18 ++++++++++++------
>>>>>> net/sctp/ipv6.c | 2 ++
>>>>>> 2 files changed, 14 insertions(+), 6 deletions(-)
>>>>>>
>>>>>> diff --git a/include/net/sctp/sctp.h b/include/net/sctp/sctp.h
>>>>>> index cd89510..f05af01 100644
>>>>>> --- a/include/net/sctp/sctp.h
>>>>>> +++ b/include/net/sctp/sctp.h
>>>>>> @@ -719,14 +719,20 @@ static inline void sctp_v4_map_v6(union
>>>>>> sctp_addr *addr)
>>>>>> addr->v6.sin6_addr.s6_addr32[2] = htonl(0x0000ffff);
>>>>>> }
>>>>>>
>>>>>> -/* The cookie is always 0 since this is how it's used in the
>>>>>> - * pmtu code.
>>>>>> - */
>>>>>> +/* Set cookie with the right one for IPv6 and zero for others */
>>>>>> static inline struct dst_entry *sctp_transport_dst_check(struct
>>>>>> sctp_transport *t)
>>>>>> {
>>>>>> - if (t->dst && !dst_check(t->dst, 0)) {
>>>>>> - dst_release(t->dst);
>>>>>> - t->dst = NULL;
>>>>>> +
>>>>>> + if (t->dst) {
>>>>>> + struct rt6_info *rt = (struct rt6_info *)t->dst;
>>>>>> + u32 cookie = 0;
>>>>>> +
>>>>>> + if ((t->af_specific->sa_family == AF_INET6) && rt->rt6i_node)
>>>>>> + cookie = rt->rt6i_node->fn_sernum;
>>>>>> + if (!dst_check(t->dst, cookie)) {
>>>>>> + dst_release(t->dst);
>>>>>> + t->dst = NULL;
>>>>>> + }
>>>>>> }
>>>>>
>>>>> I think it would be better if we stored the dst_cookie in the
>>>>> transport structure and initialized it at lookup time. If you do that,
>>>>> then if the route table changes, we'd correctly detect it without
>>>>> artificially bumping rt_genid (and hurting ipv4).
>>>>
>>>> Hi Vlad/Neil
>>>>
>>>> Is this what you mean?
>>>
>>> Yes, exactly.
>>>
>>
>> Hi Vlad
>>
>> I thinks twice about below patch, this is actually a chicken-egg issue.
>> Look below scenario:
>>   (1) The first time we push packet through a transport, dst_cookie is 0,
>>       so sctp_transport_dst_check also pass cookie as 0, then return dst
>> as NULL.
>>       Then we lookup dst by sctp_transport_route, and in there we
>> initiate dst_cookie
>>       with rt->rt6i_node->fn_sernum
>>
>>   (2) Then the next time we push packet through this transport again,
>>       we pass dst_cookie(rt->rt6i_node->fn_sernum) to ip6_dst_check, and
>>       return valid dst without bothering to lookup dst again.
>
> No, if the route was removed rt6i_node will be NULL, and NULL will be
> returned from ip6_dst_check().  If the route still exists then we'll
> compare the serial number with a cookie.
>
>>
>> BUT, suppose when deleting the source address of this dst after
>> transport->dst_cookie
>> has been well initialized. transport->dst_cookie still holds
>> rt->rt6i_node->fn_sernum,
>> meaning ip6_dst_check will return valid dst, which it shouldn't in this
>> case, the
>> result will be association ABORT.
>
> No, removing the address cause the route for that prefix to be removed
> as well.  This will set rt6i_node to NULL.
>
>>
>> Other way is invalid all transport->dst which using the deleting address
>> as source address
>> without bumping gen_id, problem is the traverse times depends heavily on
>> transport number,
>> and also need to take account locking issue it will introduce.
>>
>>  >
>>  > No, you are not missing anything. IPv4 doesn't use the cookie and
>> always seems to pass it as 0.
>>  >
>>  > Yes, ipv4 will bump the gen_id thus invalidating all routes (there
>> has been disagreement about it).
>>  > IPv6 doesn't do that. In ipv6, when the addresses are added or
>> removed, routes are also added or removed and
>>  > any time the route is added it will have a new serial number. So, you
>> don't have to disturb ipv4 cache when ipv6 routing info changes.
>>
>> Thank you very much for your explanation!
>>
>> IPv6 don't bump gen_id, when adding/deleting address, and tag an serial
>> number with each route.
>> Doing this way loose the semantic of dst_check, because SCTP depends no
>> dst_check fulfill its
>> duty to actually check whether the holding dst is still valid, well most
>> other Layer 4 protocol
>> simply rely on ip6_route_output/ip6_dst_lookup_flow to grab dst every
>> time sending data out.
>
> Look at how other protocols (tcp, dccp) do this.  It is sufficient to
> cache the route serial number into the dst_cookie at the time the route
> was lookup-up and cached.  Then the cookie is passed to dst_check to
> validate the route.


Hi Fan

Have you tried the updated patch you sent?  Based on what the tcp/udp 
code is doing, the updated patch should work correctly.  If it does, can 
you re-post with attribution/sign-off

Thanks
-vlad

>
> -vlad
>>
>> So please pronounce a final judgment.
>>
>>> -vlad
>>>
>>>>
>>>> diff --git a/include/net/sctp/sctp.h b/include/net/sctp/sctp.h
>>>> index cd89510..0a646a5 100644
>>>> --- a/include/net/sctp/sctp.h
>>>> +++ b/include/net/sctp/sctp.h
>>>> @@ -724,7 +724,7 @@ static inline void sctp_v4_map_v6(union sctp_addr
>>>> *addr)
>>>> */
>>>> static inline struct dst_entry *sctp_transport_dst_check(struct
>>>> sctp_transport *t)
>>>> {
>>>> - if (t->dst && !dst_check(t->dst, 0)) {
>>>> + if (t->dst && !dst_check(t->dst, t->dst_cookie)) {
>>>> dst_release(t->dst);
>>>> t->dst = NULL;
>>>> }
>>>> diff --git a/include/net/sctp/structs.h b/include/net/sctp/structs.h
>>>> index 1bd4c41..cafdd19 100644
>>>> --- a/include/net/sctp/structs.h
>>>> +++ b/include/net/sctp/structs.h
>>>> @@ -946,6 +946,8 @@ struct sctp_transport {
>>>> __u64 hb_nonce;
>>>>
>>>> struct rcu_head rcu;
>>>> +
>>>> + u32 dst_cookie;
>>>> };
>>>>
>>>> struct sctp_transport *sctp_transport_new(struct net *, const union
>>>> sctp_addr *,
>>>> diff --git a/net/sctp/ipv6.c b/net/sctp/ipv6.c
>>>> index 8ee553b..82a420f 100644
>>>> --- a/net/sctp/ipv6.c
>>>> +++ b/net/sctp/ipv6.c
>>>> @@ -350,6 +350,7 @@ out:
>>>> struct rt6_info *rt;
>>>> rt = (struct rt6_info *)dst;
>>>> t->dst = dst;
>>>> + t->dst_cookie = rt->rt6i_node ? rt->rt6i_node->fn_sernum
>>>> : 0;
>>>> SCTP_DEBUG_PRINTK("rt6_dst:%pI6 rt6_src:%pI6\n",
>>>> &rt->rt6i_dst.addr, &fl6->saddr);
>>>> } else {
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> -vlad
>>>>>
>>>>>>
>>>>>> return t->dst;
>>>>>> diff --git a/net/sctp/ipv6.c b/net/sctp/ipv6.c
>>>>>> index 8ee553b..cfae77e 100644
>>>>>> --- a/net/sctp/ipv6.c
>>>>>> +++ b/net/sctp/ipv6.c
>>>>>> @@ -137,6 +137,8 @@ static int sctp_inet6addr_event(struct
>>>>>> notifier_block *this, unsigned long ev,
>>>>>> break;
>>>>>> }
>>>>>>
>>>>>> + /* invalid all transport dst forcing to look up new dst */
>>>>>> + rt_genid_bump(net);
>>>>>> return NOTIFY_DONE;
>>>>>> }
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>