linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	stable@vger.kernel.org, Daniel Borkmann <daniel@iogearbox.net>,
	Mahesh Bandewar <maheshb@google.com>,
	David Ahern <dsa@cumulusnetworks.com>,
	Florian Westphal <fw@strlen.de>, Martynas Pumputis <m@lambda.lt>,
	"David S. Miller" <davem@davemloft.net>
Subject: [PATCH 4.19 05/74] ipvlan, l3mdev: fix broken l3s mode wrt local routes
Date: Mon,  4 Feb 2019 11:36:18 +0100	[thread overview]
Message-ID: <20190204103620.282240117@linuxfoundation.org> (raw)
In-Reply-To: <20190204103619.714714157@linuxfoundation.org>

4.19-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Daniel Borkmann <daniel@iogearbox.net>

[ Upstream commit d5256083f62e2720f75bb3c5a928a0afe47d6bc3 ]

While implementing ipvlan l3 and l3s mode for kubernetes CNI plugin,
I ran into the issue that while l3 mode is working fine, l3s mode
does not have any connectivity to kube-apiserver and hence all pods
end up in Error state as well. The ipvlan master device sits on
top of a bond device and hostns traffic to kube-apiserver (also running
in hostns) is DNATed from 10.152.183.1:443 to 139.178.29.207:37573
where the latter is the address of the bond0. While in l3 mode, a
curl to https://10.152.183.1:443 or to https://139.178.29.207:37573
works fine from hostns, neither of them do in case of l3s. In the
latter only a curl to https://127.0.0.1:37573 appeared to work where
for local addresses of bond0 I saw kernel suddenly starting to emit
ARP requests to query HW address of bond0 which remained unanswered
and neighbor entries in INCOMPLETE state. These ARP requests only
happen while in l3s.

Debugging this further, I found the issue is that l3s mode is piggy-
backing on l3 master device, and in this case local routes are using
l3mdev_master_dev_rcu(dev) instead of net->loopback_dev as per commit
f5a0aab84b74 ("net: ipv4: dst for local input routes should use l3mdev
if relevant") and 5f02ce24c269 ("net: l3mdev: Allow the l3mdev to be
a loopback"). I found that reverting them back into using the
net->loopback_dev fixed ipvlan l3s connectivity and got everything
working for the CNI.

Now judging from 4fbae7d83c98 ("ipvlan: Introduce l3s mode") and the
l3mdev paper in [0] the only sole reason why ipvlan l3s is relying
on l3 master device is to get the l3mdev_ip_rcv() receive hook for
setting the dst entry of the input route without adding its own
ipvlan specific hacks into the receive path, however, any l3 domain
semantics beyond just that are breaking l3s operation. Note that
ipvlan also has the ability to dynamically switch its internal
operation from l3 to l3s for all ports via ipvlan_set_port_mode()
at runtime. In any case, l3 vs l3s soley distinguishes itself by
'de-confusing' netfilter through switching skb->dev to ipvlan slave
device late in NF_INET_LOCAL_IN before handing the skb to L4.

Minimal fix taken here is to add a IFF_L3MDEV_RX_HANDLER flag which,
if set from ipvlan setup, gets us only the wanted l3mdev_l3_rcv() hook
without any additional l3mdev semantics on top. This should also have
minimal impact since dev->priv_flags is already hot in cache. With
this set, l3s mode is working fine and I also get things like
masquerading pod traffic on the ipvlan master properly working.

  [0] https://netdevconf.org/1.2/papers/ahern-what-is-l3mdev-paper.pdf

Fixes: f5a0aab84b74 ("net: ipv4: dst for local input routes should use l3mdev if relevant")
Fixes: 5f02ce24c269 ("net: l3mdev: Allow the l3mdev to be a loopback")
Fixes: 4fbae7d83c98 ("ipvlan: Introduce l3s mode")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Cc: Mahesh Bandewar <maheshb@google.com>
Cc: David Ahern <dsa@cumulusnetworks.com>
Cc: Florian Westphal <fw@strlen.de>
Cc: Martynas Pumputis <m@lambda.lt>
Acked-by: David Ahern <dsa@cumulusnetworks.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
 drivers/net/ipvlan/ipvlan_main.c |    6 +++---
 include/linux/netdevice.h        |    8 ++++++++
 include/net/l3mdev.h             |    3 ++-
 3 files changed, 13 insertions(+), 4 deletions(-)

--- a/drivers/net/ipvlan/ipvlan_main.c
+++ b/drivers/net/ipvlan/ipvlan_main.c
@@ -97,12 +97,12 @@ static int ipvlan_set_port_mode(struct i
 			err = ipvlan_register_nf_hook(read_pnet(&port->pnet));
 			if (!err) {
 				mdev->l3mdev_ops = &ipvl_l3mdev_ops;
-				mdev->priv_flags |= IFF_L3MDEV_MASTER;
+				mdev->priv_flags |= IFF_L3MDEV_RX_HANDLER;
 			} else
 				goto fail;
 		} else if (port->mode == IPVLAN_MODE_L3S) {
 			/* Old mode was L3S */
-			mdev->priv_flags &= ~IFF_L3MDEV_MASTER;
+			mdev->priv_flags &= ~IFF_L3MDEV_RX_HANDLER;
 			ipvlan_unregister_nf_hook(read_pnet(&port->pnet));
 			mdev->l3mdev_ops = NULL;
 		}
@@ -162,7 +162,7 @@ static void ipvlan_port_destroy(struct n
 	struct sk_buff *skb;
 
 	if (port->mode == IPVLAN_MODE_L3S) {
-		dev->priv_flags &= ~IFF_L3MDEV_MASTER;
+		dev->priv_flags &= ~IFF_L3MDEV_RX_HANDLER;
 		ipvlan_unregister_nf_hook(dev_net(dev));
 		dev->l3mdev_ops = NULL;
 	}
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -1455,6 +1455,7 @@ struct net_device_ops {
  * @IFF_NO_RX_HANDLER: device doesn't support the rx_handler hook
  * @IFF_FAILOVER: device is a failover master device
  * @IFF_FAILOVER_SLAVE: device is lower dev of a failover master device
+ * @IFF_L3MDEV_RX_HANDLER: only invoke the rx handler of L3 master device
  */
 enum netdev_priv_flags {
 	IFF_802_1Q_VLAN			= 1<<0,
@@ -1486,6 +1487,7 @@ enum netdev_priv_flags {
 	IFF_NO_RX_HANDLER		= 1<<26,
 	IFF_FAILOVER			= 1<<27,
 	IFF_FAILOVER_SLAVE		= 1<<28,
+	IFF_L3MDEV_RX_HANDLER		= 1<<29,
 };
 
 #define IFF_802_1Q_VLAN			IFF_802_1Q_VLAN
@@ -1516,6 +1518,7 @@ enum netdev_priv_flags {
 #define IFF_NO_RX_HANDLER		IFF_NO_RX_HANDLER
 #define IFF_FAILOVER			IFF_FAILOVER
 #define IFF_FAILOVER_SLAVE		IFF_FAILOVER_SLAVE
+#define IFF_L3MDEV_RX_HANDLER		IFF_L3MDEV_RX_HANDLER
 
 /**
  *	struct net_device - The DEVICE structure.
@@ -4464,6 +4467,11 @@ static inline bool netif_supports_nofcs(
 	return dev->priv_flags & IFF_SUPP_NOFCS;
 }
 
+static inline bool netif_has_l3_rx_handler(const struct net_device *dev)
+{
+	return dev->priv_flags & IFF_L3MDEV_RX_HANDLER;
+}
+
 static inline bool netif_is_l3_master(const struct net_device *dev)
 {
 	return dev->priv_flags & IFF_L3MDEV_MASTER;
--- a/include/net/l3mdev.h
+++ b/include/net/l3mdev.h
@@ -142,7 +142,8 @@ struct sk_buff *l3mdev_l3_rcv(struct sk_
 
 	if (netif_is_l3_slave(skb->dev))
 		master = netdev_master_upper_dev_get_rcu(skb->dev);
-	else if (netif_is_l3_master(skb->dev))
+	else if (netif_is_l3_master(skb->dev) ||
+		 netif_has_l3_rx_handler(skb->dev))
 		master = skb->dev;
 
 	if (master && master->l3mdev_ops->l3mdev_l3_rcv)



  parent reply	other threads:[~2019-02-04 10:47 UTC|newest]

Thread overview: 77+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-02-04 10:36 [PATCH 4.19 00/74] 4.19.20-stable review Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 01/74] Fix "net: ipv4: do not handle duplicate fragments as overlapping" Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 02/74] drm/msm/gpu: fix building without debugfs Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 03/74] ipv6: Consider sk_bound_dev_if when binding a socket to an address Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 04/74] ipv6: sr: clear IP6CB(skb) on SRH ip4ip6 encapsulation Greg Kroah-Hartman
2019-02-04 10:36 ` Greg Kroah-Hartman [this message]
2019-02-04 10:36 ` [PATCH 4.19 06/74] l2tp: copy 4 more bytes to linear part if necessary Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 07/74] l2tp: fix reading optional fields of L2TPv3 Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 08/74] net: ip_gre: always reports o_key to userspace Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 09/74] net: ip_gre: use erspan key field for tunnel lookup Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 10/74] net/mlx4_core: Add masking for a few queries on HCA caps Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 11/74] netrom: switch to sock timer API Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 12/74] net/rose: fix NULL ax25_cb kernel panic Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 13/74] net: set default network namespace in init_dummy_netdev() Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 14/74] ravb: expand rx descriptor data to accommodate hw checksum Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 15/74] sctp: improve the events for sctp stream reset Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 16/74] tun: move the call to tun_set_real_num_queues Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 17/74] ucc_geth: Reset BQL queue when stopping device Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 18/74] vhost: fix OOB in get_rx_bufs() Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 19/74] net: ip6_gre: always reports o_key to userspace Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 20/74] sctp: improve the events for sctp stream adding Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 21/74] net/mlx5e: Allow MAC invalidation while spoofchk is ON Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 22/74] ip6mr: Fix notifiers call on mroute_clean_tables() Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 23/74] Revert "net/mlx5e: E-Switch, Initialize eswitch only if eswitch manager" Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 24/74] sctp: set chunk transport correctly when its a new asoc Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 25/74] sctp: set flow sport from saddr only when its 0 Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 26/74] virtio_net: Dont enable NAPI when interface is down Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 27/74] virtio_net: Dont call free_old_xmit_skbs for xdp_frames Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 28/74] virtio_net: Fix not restoring real_num_rx_queues Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 29/74] virtio_net: Fix out of bounds access of sq Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 30/74] virtio_net: Dont process redirected XDP frames when XDP is disabled Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 31/74] virtio_net: Use xdp_return_frame to free xdp_frames on destroying vqs Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 32/74] virtio_net: Differentiate sk_buff and xdp_frame on freeing Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 33/74] CIFS: Do not count -ENODATA as failure for query directory Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 34/74] CIFS: Fix trace command logging for SMB2 reads and writes Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 35/74] CIFS: Do not consider -ENODATA as stat failure for reads Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 36/74] fs/dcache: Fix incorrect nr_dentry_unused accounting in shrink_dcache_sb() Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 37/74] iommu/vt-d: Fix memory leak in intel_iommu_put_resv_regions() Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 38/74] selftests/seccomp: Enhance per-arch ptrace syscall skip tests Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 39/74] NFS: Fix up return value on fatal errors in nfs_page_async_flush() Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 40/74] ARM: cns3xxx: Fix writing to wrong PCI config registers after alignment Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 41/74] arm64: kaslr: ensure randomized quantities are clean also when kaslr is off Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 42/74] arm64: Do not issue IPIs for user executable ptes Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 43/74] arm64: hyp-stub: Forbid kprobing of the hyp-stub Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 44/74] arm64: hibernate: Clean the __hyp_text to PoC after resume Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 45/74] gpio: altera-a10sr: Set proper output level for direction_output Greg Kroah-Hartman
2019-02-04 10:36 ` [PATCH 4.19 46/74] gpiolib: fix line event timestamps for nested irqs Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 47/74] gpio: pcf857x: Fix interrupts on multiple instances Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 48/74] gpio: sprd: Fix the incorrect data register Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 49/74] gpio: sprd: Fix incorrect irq type setting for the async EIC Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 50/74] gfs2: Revert "Fix loop in gfs2_rbm_find" Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 51/74] mmc: bcm2835: Fix DMA channel leak on probe error Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 52/74] mmc: mediatek: fix incorrect register setting of hs400_cmd_int_delay Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 53/74] ALSA: usb-audio: Add Opus #3 to quirks for native DSD support Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 54/74] ALSA: hda/realtek - Fixed hp_pin no value Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 55/74] IB/hfi1: Remove overly conservative VM_EXEC flag check Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 56/74] platform/x86: asus-nb-wmi: Map 0x35 to KEY_SCREENLOCK Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 57/74] platform/x86: asus-nb-wmi: Drop mapping of 0x33 and 0x34 scan codes Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 58/74] mmc: sdhci-iproc: handle mmc_of_parse() errors during probe Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 59/74] Btrfs: fix deadlock when allocating tree block during leaf/node split Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 60/74] btrfs: On error always free subvol_name in btrfs_mount Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 61/74] kernel/exit.c: release ptraced tasks before zap_pid_ns_processes Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 62/74] mm/hugetlb.c: teach follow_hugetlb_page() to handle FOLL_NOWAIT Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 63/74] oom, oom_reaper: do not enqueue same task twice Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 64/74] mm,memory_hotplug: fix scan_movable_pages() for gigantic hugepages Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 65/74] mm, oom: fix use-after-free in oom_kill_process Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 66/74] mm: hwpoison: use do_send_sig_info() instead of force_sig() Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 67/74] mm: migrate: dont rely on __PageMovable() of newpage after unlocking it Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 68/74] of: Convert to using %pOFn instead of device_node.name Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 69/74] of: overlay: add tests to validate kfrees from overlay removal Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 70/74] of: overlay: add missing of_node_get() in __of_attach_node_sysfs Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 71/74] of: overlay: use prop add changeset entry for property in new nodes Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 72/74] of: overlay: do not duplicate properties from overlay for " Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 73/74] md/raid5: fix out of memory during raid cache recovery Greg Kroah-Hartman
2019-02-04 10:37 ` [PATCH 4.19 74/74] cifs: Always resolve hostname before reconnecting Greg Kroah-Hartman
2019-02-04 21:49 ` [PATCH 4.19 00/74] 4.19.20-stable review Guenter Roeck
2019-02-05  6:14 ` Naresh Kamboju

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190204103620.282240117@linuxfoundation.org \
    --to=gregkh@linuxfoundation.org \
    --cc=daniel@iogearbox.net \
    --cc=davem@davemloft.net \
    --cc=dsa@cumulusnetworks.com \
    --cc=fw@strlen.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=m@lambda.lt \
    --cc=maheshb@google.com \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).