[PATCH ipsec-next] xfrm: Use VRF master index if output device is enslaved

[PATCH ipsec-next] xfrm: Use VRF master index if output device is enslaved

[David Ahern]

Directs route lookups to VRF table. Compiles out if NET_VRF is not
enabled. With this patch able to successfully bring up ipsec tunnels
in VRFs, even with duplicate network configuration (IPv4 tested).

Signed-off-by: David Ahern <dsa@cumulusnetworks.com>
---
 net/ipv4/xfrm4_policy.c | 7 +++++--
 net/ipv6/xfrm6_policy.c | 7 +++++--
 2 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/net/ipv4/xfrm4_policy.c b/net/ipv4/xfrm4_policy.c
index 55b3c0f4dde5..35757f6af2d5 100644
--- a/net/ipv4/xfrm4_policy.c
+++ b/net/ipv4/xfrm4_policy.c
@@ -15,6 +15,7 @@
 #include <net/dst.h>
 #include <net/xfrm.h>
 #include <net/ip.h>
+#include <net/vrf.h>
 
 static struct xfrm_policy_afinfo xfrm4_policy_afinfo;
 
@@ -107,8 +108,10 @@ _decode_session4(struct sk_buff *skb, struct flowi *fl, int reverse)
 	struct flowi4 *fl4 = &fl->u.ip4;
 	int oif = 0;
 
-	if (skb_dst(skb))
-		oif = skb_dst(skb)->dev->ifindex;
+	if (skb_dst(skb)) {
+		oif = vrf_master_ifindex_rcu(skb_dst(skb)->dev) ?
+			: skb_dst(skb)->dev->ifindex;
+	}
 
 	memset(fl4, 0, sizeof(struct flowi4));
 	fl4->flowi4_mark = skb->mark;
diff --git a/net/ipv6/xfrm6_policy.c b/net/ipv6/xfrm6_policy.c
index a74013d3eceb..4a88b89becf5 100644
--- a/net/ipv6/xfrm6_policy.c
+++ b/net/ipv6/xfrm6_policy.c
@@ -20,6 +20,7 @@
 #include <net/ip.h>
 #include <net/ipv6.h>
 #include <net/ip6_route.h>
+#include <net/vrf.h>
 #if IS_ENABLED(CONFIG_IPV6_MIP6)
 #include <net/mip6.h>
 #endif
@@ -131,8 +132,10 @@ _decode_session6(struct sk_buff *skb, struct flowi *fl, int reverse)
 
 	nexthdr = nh[nhoff];
 
-	if (skb_dst(skb))
-		oif = skb_dst(skb)->dev->ifindex;
+	if (skb_dst(skb)) {
+		oif = vrf_master_ifindex_rcu(skb_dst(skb)->dev) ?
+			: skb_dst(skb)->dev->ifindex;
+	}
 
 	memset(fl6, 0, sizeof(struct flowi6));
 	fl6->flowi6_mark = skb->mark;
-- 
2.3.2 (Apple Git-55)

Re: [PATCH ipsec-next] xfrm: Use VRF master index if output device is enslaved

[Nikolay Aleksandrov]

> On Aug 18, 2015, at 6:54 PM, David Ahern <dsa@cumulusnetworks.com> wrote:
> 
> Directs route lookups to VRF table. Compiles out if NET_VRF is not
> enabled. With this patch able to successfully bring up ipsec tunnels
> in VRFs, even with duplicate network configuration (IPv4 tested).
> 
> Signed-off-by: David Ahern <dsa@cumulusnetworks.com>
> ---
> net/ipv4/xfrm4_policy.c | 7 +++++--
> net/ipv6/xfrm6_policy.c | 7 +++++--
> 2 files changed, 10 insertions(+), 4 deletions(-)

I think you should use the new vrf_master_index() helper that acquires rcu because
it looks possible to call ->decode_session() without rcu read lock, e.g. in the hold_timer
function xfrm_policy_queue_process(), though I haven’t tested it and might be missing
something. :-)

Cheers,
 Nik

Re: [PATCH ipsec-next] xfrm: Use VRF master index if output device is enslaved

[David Ahern]

On 8/19/15 5:47 AM, Nikolay Aleksandrov wrote:
>
>> On Aug 18, 2015, at 6:54 PM, David Ahern <dsa@cumulusnetworks.com> wrote:
>>
>> Directs route lookups to VRF table. Compiles out if NET_VRF is not
>> enabled. With this patch able to successfully bring up ipsec tunnels
>> in VRFs, even with duplicate network configuration (IPv4 tested).
>>
>> Signed-off-by: David Ahern <dsa@cumulusnetworks.com>
>> ---
>> net/ipv4/xfrm4_policy.c | 7 +++++--
>> net/ipv6/xfrm6_policy.c | 7 +++++--
>> 2 files changed, 10 insertions(+), 4 deletions(-)
>
> I think you should use the new vrf_master_index() helper that acquires rcu because
> it looks possible to call ->decode_session() without rcu read lock, e.g. in the hold_timer
> function xfrm_policy_queue_process(), though I haven’t tested it and might be missing
> something. :-)

I was digging into code paths yesterday. Today I added WARN_ON and seems 
like the rcu_read_lock is held:

         if (skb_dst(skb)) {
WARN_ON(!rcu_read_lock_held() && !rcu_read_lock_bh_held());
                 oif = vrf_master_ifindex_rcu(skb_dst(skb)->dev) ?
                         : skb_dst(skb)->dev->ifindex;
pr_info("_decode_session: oif %d skb_dst(skb)->dev->ifindex %d\n", oif, 
skb_dst(skb)->dev->ifindex);
         }

I get the printk, but not the WARN_ON splat.

Re: [PATCH ipsec-next] xfrm: Use VRF master index if output device is enslaved

[Steffen Klassert]

On Wed, Aug 19, 2015 at 11:35:55AM -0700, David Ahern wrote:
> >
> >I think you should use the new vrf_master_index() helper that acquires rcu because
> >it looks possible to call ->decode_session() without rcu read lock, e.g. in the hold_timer
> >function xfrm_policy_queue_process(), though I haven’t tested it and might be missing
> >something. :-)
> 
> I was digging into code paths yesterday. Today I added WARN_ON and
> seems like the rcu_read_lock is held:
> 
>         if (skb_dst(skb)) {
> WARN_ON(!rcu_read_lock_held() && !rcu_read_lock_bh_held());
>                 oif = vrf_master_ifindex_rcu(skb_dst(skb)->dev) ?
>                         : skb_dst(skb)->dev->ifindex;
> pr_info("_decode_session: oif %d skb_dst(skb)->dev->ifindex %d\n",
> oif, skb_dst(skb)->dev->ifindex);
>         }
> 
> I get the printk, but not the WARN_ON splat.

Well, this depends on the codepath that called xfrm_decode_session().
It really think it was not called through xfrm_policy_queue_process()
because this codepath is just used if the sysctl xfrm_larval_drop is switched
off (on by default) and the required xfrm_state is not yet established.