[PATCH net-next 0/1] veth: Support bonding events

[PATCH net-next 0/1] veth: Support bonding events

[Alexandra Winter]

In case virtual instances are attached to an external network via veth
and a bridge, the interface to the external network can be a bond
interface. Bonding drivers generate specific events during failover
that trigger switch updates.  When a veth device is attached to a
bridge with a bond interface, we want external switches to learn about
the veth devices as well.

Without this patch we have seen cases where recovery after bond
failover took an unacceptable amount of time (depending on timeout
settings in the network).

Due to the symmetric nature of veth special care is required to avoid
endless notification loops. Therefore we only notify from a veth
bridgeport to a peer that is not a bridgeport.

References:
Same handling as for macvlan:
4c9912556867 ("macvlan: Support bonding events"
and vlan:
4aa5dee4d999 ("net: convert resend IGMP to notifier event")

Alternatives:
Propagate notifier events to all ports of a bridge. IIUC, this was
rejected in https://www.spinics.net/lists/netdev/msg717292.html
It also seems difficult to avoid re-bouncing the notifier.

Alexandra Winter (1):
  veth: Support bonding events

 drivers/net/veth.c | 53 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

-- 
2.32.0

[PATCH net-next 1/1] veth: Support bonding events

[Alexandra Winter]

Bonding drivers generate specific events during failover that trigger
switch updates.  When a veth device is attached to a bridge with a
bond interface, we want external switches to learn about the veth
devices as well.

Signed-off-by: Alexandra Winter <wintera@linux.ibm.com>
---
 drivers/net/veth.c | 53 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index d29fb9759cc9..9019c9852daf 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c
@@ -1579,6 +1579,57 @@ static void veth_setup(struct net_device *dev)
 	dev->mpls_features = NETIF_F_HW_CSUM | NETIF_F_GSO_SOFTWARE;
 }
 
+static bool _is_veth(const struct net_device *dev)
+{
+	return (dev->netdev_ops->ndo_open == veth_open);
+}
+
+static void veth_notify_peer(unsigned long event, const struct net_device *dev)
+{
+	struct net_device *peer;
+	struct veth_priv *priv;
+
+	priv = netdev_priv(dev);
+	peer = rtnl_dereference(priv->peer);
+	/* avoid re-bounce between 2 bridges */
+	if (!netif_is_bridge_port(peer))
+		call_netdevice_notifiers(event, peer);
+}
+
+/* Called under rtnl_lock */
+static int veth_device_event(struct notifier_block *unused,
+			     unsigned long event, void *ptr)
+{
+	struct net_device *dev, *lower;
+	struct list_head *iter;
+
+	dev = netdev_notifier_info_to_dev(ptr);
+
+	switch (event) {
+	case NETDEV_NOTIFY_PEERS:
+	case NETDEV_BONDING_FAILOVER:
+	case NETDEV_RESEND_IGMP:
+		/* propagate to peer of a bridge attached veth */
+		if (netif_is_bridge_master(dev)) {
+			iter = &dev->adj_list.lower;
+			lower = netdev_next_lower_dev_rcu(dev, &iter);
+			while (lower) {
+				if (_is_veth(lower))
+					veth_notify_peer(event, lower);
+				lower = netdev_next_lower_dev_rcu(dev, &iter);
+			}
+		}
+		break;
+	default:
+		break;
+	}
+	return NOTIFY_DONE;
+}
+
+static struct notifier_block veth_notifier_block __read_mostly = {
+		.notifier_call  = veth_device_event,
+};
+
 /*
  * netlink interface
  */
@@ -1824,12 +1875,14 @@ static struct rtnl_link_ops veth_link_ops = {
 
 static __init int veth_init(void)
 {
+	register_netdevice_notifier(&veth_notifier_block);
 	return rtnl_link_register(&veth_link_ops);
 }
 
 static __exit void veth_exit(void)
 {
 	rtnl_link_unregister(&veth_link_ops);
+	unregister_netdevice_notifier(&veth_notifier_block);
 }
 
 module_init(veth_init);
-- 
2.32.0

Re: [PATCH net-next 0/1] veth: Support bonding events

[Jakub Kicinski]

On Mon, 28 Mar 2022 10:14:16 +0200 Alexandra Winter wrote:
> In case virtual instances are attached to an external network via veth
> and a bridge, the interface to the external network can be a bond
> interface. Bonding drivers generate specific events during failover
> that trigger switch updates.  When a veth device is attached to a
> bridge with a bond interface, we want external switches to learn about
> the veth devices as well.

Can you please add an ASCII diagram of a setup your trying to describe?

> Without this patch we have seen cases where recovery after bond
> failover took an unacceptable amount of time (depending on timeout
> settings in the network).
> 
> Due to the symmetric nature of veth special care is required to avoid
> endless notification loops. Therefore we only notify from a veth
> bridgeport to a peer that is not a bridgeport.
> 
> References:
> Same handling as for macvlan:
> 4c9912556867 ("macvlan: Support bonding events"
> and vlan:
> 4aa5dee4d999 ("net: convert resend IGMP to notifier event")

When sending a single patch change you can put all the information 
in the commit message of the patch, the cover letter is only necessary
for series of multiple patches.

Re: [PATCH net-next 1/1] veth: Support bonding events

[Jakub Kicinski]

On Mon, 28 Mar 2022 10:14:17 +0200 Alexandra Winter wrote:
> +static bool _is_veth(const struct net_device *dev)

netif_is_veth()?

> +{
> +	return (dev->netdev_ops->ndo_open == veth_open);

Why compare ndo_open and not entire netdev_ops or rtnl_link_ops?

> +}

Re: [PATCH net-next 0/1] veth: Support bonding events

[Alexandra Winter]

On 29.03.22 04:14, Jakub Kicinski wrote:
> On Mon, 28 Mar 2022 10:14:16 +0200 Alexandra Winter wrote:
>> In case virtual instances are attached to an external network via veth
>> and a bridge, the interface to the external network can be a bond
>> interface. Bonding drivers generate specific events during failover
>> that trigger switch updates.  When a veth device is attached to a
>> bridge with a bond interface, we want external switches to learn about
>> the veth devices as well.
> 
> Can you please add an ASCII diagram of a setup your trying to describe?

	
	| veth_a2   |  veth_b2  |  veth_c2 |
	------o-----------o----------o------
	       \	  |	    /
		o	  o	   o
	      veth_a1  veth_b1  veth_c1
	      -------------------------
	      |        bridge         |
	      -------------------------
			bond0
			/  \
		     eth0  eth1

In case of failover from eth0 to eth1, the netdev_notifier needs to
be propagated, so e.g. veth_a2 can re-announce its MAC address to the
external hardware attached to eth1.

Does that help?
> 
>> Without this patch we have seen cases where recovery after bond
>> failover took an unacceptable amount of time (depending on timeout
>> settings in the network).
>>
>> Due to the symmetric nature of veth special care is required to avoid
>> endless notification loops. Therefore we only notify from a veth
>> bridgeport to a peer that is not a bridgeport.
>>
>> References:
>> Same handling as for macvlan:
>> 4c9912556867 ("macvlan: Support bonding events"
>> and vlan:
>> 4aa5dee4d999 ("net: convert resend IGMP to notifier event")
> 
> When sending a single patch change you can put all the information 
> in the commit message of the patch, the cover letter is only necessary
> for series of multiple patches.
Thank you, I will do so in v2.