[PATCH next v5 2/6] bonding: implement bond_poll_controller()

[PATCH next v5 2/6] bonding: implement bond_poll_controller()

[Mahesh Bandewar]

This patches implements the poll_controller support for all
bonding driver. If the slaves have poll_controller net_op defined,
this implementation calls them. This is mode agnostic implementation
and iterates through all slaves (based on mode) and calls respective
handler.

Signed-off-by: Mahesh Bandewar <maheshb@google.com>
---
v1:
   Initial version
v2:
   Eliminate bool variable.
v3:
   Rebase
v4:
   Removed 3AD port_operational check
v5:
   Added rtnl protection for bond_for_each_slave()

 drivers/net/bonding/bond_main.c | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
index b979c265fc51..63e6c0dbe7b3 100644
--- a/drivers/net/bonding/bond_main.c
+++ b/drivers/net/bonding/bond_main.c
@@ -928,6 +928,39 @@ static inline void slave_disable_netpoll(struct slave *slave)
 
 static void bond_poll_controller(struct net_device *bond_dev)
 {
+	struct bonding *bond = netdev_priv(bond_dev);
+	struct slave *slave = NULL;
+	struct list_head *iter;
+	struct ad_info ad_info;
+	struct netpoll_info *ni;
+	const struct net_device_ops *ops;
+
+	if (BOND_MODE(bond) == BOND_MODE_8023AD)
+		if (bond_3ad_get_active_agg_info(bond, &ad_info))
+			return;
+
+	rtnl_lock();
+	bond_for_each_slave(bond, slave, iter) {
+		ops = slave->dev->netdev_ops;
+		if (!bond_slave_is_up(slave) || !ops->ndo_poll_controller)
+			continue;
+
+		if (BOND_MODE(bond) == BOND_MODE_8023AD) {
+			struct aggregator *agg =
+			    SLAVE_AD_INFO(slave)->port.aggregator;
+
+			if (agg &&
+			    agg->aggregator_identifier != ad_info.aggregator_id)
+				continue;
+		}
+
+		ni = rcu_dereference_bh(slave->dev->npinfo);
+		if (down_trylock(&ni->dev_lock))
+			continue;
+		ops->ndo_poll_controller(slave->dev);
+		up(&ni->dev_lock);
+	}
+	rtnl_unlock();
 }
 
 static void bond_netpoll_cleanup(struct net_device *bond_dev)
-- 
2.2.0.rc0.207.ga3a616c

Re: [PATCH next v5 2/6] bonding: implement bond_poll_controller()

[Nikolay Aleksandrov]

On 02/18/2015 11:31 PM, Mahesh Bandewar wrote:
> This patches implements the poll_controller support for all
> bonding driver. If the slaves have poll_controller net_op defined,
> this implementation calls them. This is mode agnostic implementation
> and iterates through all slaves (based on mode) and calls respective
> handler.
> 
> Signed-off-by: Mahesh Bandewar <maheshb@google.com>
> ---
> v1:
>    Initial version
> v2:
>    Eliminate bool variable.
> v3:
>    Rebase
> v4:
>    Removed 3AD port_operational check
> v5:
>    Added rtnl protection for bond_for_each_slave()
> 
>  drivers/net/bonding/bond_main.c | 33 +++++++++++++++++++++++++++++++++
>  1 file changed, 33 insertions(+)
> 

Hi Mahesh,
I should've explained more in my review, you cannot sleep in
bond_poll_controller() so you cannot acquire rtnl like that. I was thinking
more about using rcu and switching to the _rcu version of
bond_for_each_slave instead.

Cheers,
 Nik

> diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
> index b979c265fc51..63e6c0dbe7b3 100644
> --- a/drivers/net/bonding/bond_main.c
> +++ b/drivers/net/bonding/bond_main.c
> @@ -928,6 +928,39 @@ static inline void slave_disable_netpoll(struct slave *slave)
>  
>  static void bond_poll_controller(struct net_device *bond_dev)
>  {
> +	struct bonding *bond = netdev_priv(bond_dev);
> +	struct slave *slave = NULL;
> +	struct list_head *iter;
> +	struct ad_info ad_info;
> +	struct netpoll_info *ni;
> +	const struct net_device_ops *ops;
> +
> +	if (BOND_MODE(bond) == BOND_MODE_8023AD)
> +		if (bond_3ad_get_active_agg_info(bond, &ad_info))
> +			return;
> +
> +	rtnl_lock();
> +	bond_for_each_slave(bond, slave, iter) {
> +		ops = slave->dev->netdev_ops;
> +		if (!bond_slave_is_up(slave) || !ops->ndo_poll_controller)
> +			continue;
> +
> +		if (BOND_MODE(bond) == BOND_MODE_8023AD) {
> +			struct aggregator *agg =
> +			    SLAVE_AD_INFO(slave)->port.aggregator;
> +
> +			if (agg &&
> +			    agg->aggregator_identifier != ad_info.aggregator_id)
> +				continue;
> +		}
> +
> +		ni = rcu_dereference_bh(slave->dev->npinfo);
> +		if (down_trylock(&ni->dev_lock))
> +			continue;
> +		ops->ndo_poll_controller(slave->dev);
> +		up(&ni->dev_lock);
> +	}
> +	rtnl_unlock();
>  }
>  
>  static void bond_netpoll_cleanup(struct net_device *bond_dev)
> 

Re: [PATCH next v5 2/6] bonding: implement bond_poll_controller()

[Mahesh Bandewar]

On Wed, Feb 18, 2015 at 4:10 PM, Nikolay Aleksandrov <nikolay@redhat.com> wrote:
> On 02/18/2015 11:31 PM, Mahesh Bandewar wrote:
>> This patches implements the poll_controller support for all
>> bonding driver. If the slaves have poll_controller net_op defined,
>> this implementation calls them. This is mode agnostic implementation
>> and iterates through all slaves (based on mode) and calls respective
>> handler.
>>
>> Signed-off-by: Mahesh Bandewar <maheshb@google.com>
>> ---
>> v1:
>>    Initial version
>> v2:
>>    Eliminate bool variable.
>> v3:
>>    Rebase
>> v4:
>>    Removed 3AD port_operational check
>> v5:
>>    Added rtnl protection for bond_for_each_slave()
>>
>>  drivers/net/bonding/bond_main.c | 33 +++++++++++++++++++++++++++++++++
>>  1 file changed, 33 insertions(+)
>>
>
> Hi Mahesh,
> I should've explained more in my review, you cannot sleep in
> bond_poll_controller() so you cannot acquire rtnl like that. I was thinking
> more about using rcu and switching to the _rcu version of
> bond_for_each_slave instead.
>
That makes sense. The path that triggered this netpoll() could have
been holding the rtnl  itself and this would be a problem. I think
using the _rcu variant of the slave iterator is a good idea, my bad!

> Cheers,
>  Nik
>
>> diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
>> index b979c265fc51..63e6c0dbe7b3 100644
>> --- a/drivers/net/bonding/bond_main.c
>> +++ b/drivers/net/bonding/bond_main.c
>> @@ -928,6 +928,39 @@ static inline void slave_disable_netpoll(struct slave *slave)
>>
>>  static void bond_poll_controller(struct net_device *bond_dev)
>>  {
>> +     struct bonding *bond = netdev_priv(bond_dev);
>> +     struct slave *slave = NULL;
>> +     struct list_head *iter;
>> +     struct ad_info ad_info;
>> +     struct netpoll_info *ni;
>> +     const struct net_device_ops *ops;
>> +
>> +     if (BOND_MODE(bond) == BOND_MODE_8023AD)
>> +             if (bond_3ad_get_active_agg_info(bond, &ad_info))
>> +                     return;
>> +
>> +     rtnl_lock();
>> +     bond_for_each_slave(bond, slave, iter) {
>> +             ops = slave->dev->netdev_ops;
>> +             if (!bond_slave_is_up(slave) || !ops->ndo_poll_controller)
>> +                     continue;
>> +
>> +             if (BOND_MODE(bond) == BOND_MODE_8023AD) {
>> +                     struct aggregator *agg =
>> +                         SLAVE_AD_INFO(slave)->port.aggregator;
>> +
>> +                     if (agg &&
>> +                         agg->aggregator_identifier != ad_info.aggregator_id)
>> +                             continue;
>> +             }
>> +
>> +             ni = rcu_dereference_bh(slave->dev->npinfo);
>> +             if (down_trylock(&ni->dev_lock))
>> +                     continue;
>> +             ops->ndo_poll_controller(slave->dev);
>> +             up(&ni->dev_lock);
>> +     }
>> +     rtnl_unlock();
>>  }
>>
>>  static void bond_netpoll_cleanup(struct net_device *bond_dev)
>>
>

Re: [PATCH next v5 2/6] bonding: implement bond_poll_controller()

[Mahesh Bandewar]

On Wed, Feb 18, 2015 at 5:19 PM, Mahesh Bandewar <maheshb@google.com> wrote:
> On Wed, Feb 18, 2015 at 4:10 PM, Nikolay Aleksandrov <nikolay@redhat.com> wrote:
>> On 02/18/2015 11:31 PM, Mahesh Bandewar wrote:
>>> This patches implements the poll_controller support for all
>>> bonding driver. If the slaves have poll_controller net_op defined,
>>> this implementation calls them. This is mode agnostic implementation
>>> and iterates through all slaves (based on mode) and calls respective
>>> handler.
>>>
>>> Signed-off-by: Mahesh Bandewar <maheshb@google.com>
>>> ---
>>> v1:
>>>    Initial version
>>> v2:
>>>    Eliminate bool variable.
>>> v3:
>>>    Rebase
>>> v4:
>>>    Removed 3AD port_operational check
>>> v5:
>>>    Added rtnl protection for bond_for_each_slave()
>>>
>>>  drivers/net/bonding/bond_main.c | 33 +++++++++++++++++++++++++++++++++
>>>  1 file changed, 33 insertions(+)
>>>
>>
>> Hi Mahesh,
>> I should've explained more in my review, you cannot sleep in
>> bond_poll_controller() so you cannot acquire rtnl like that. I was thinking
>> more about using rcu and switching to the _rcu version of
>> bond_for_each_slave instead.
>>
> That makes sense. The path that triggered this netpoll() could have
> been holding the rtnl  itself and this would be a problem. I think
> using the _rcu variant of the slave iterator is a good idea, my bad!
>
... however we cannot use the _rcu variant either since there is the
netpoll mutex (ni->dev_lock)!
The fact that we are here itself means that something bad had happened
and trying to take additional lock(s) would complicate the situation
further.


>> Cheers,
>>  Nik
>>
>>> diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
>>> index b979c265fc51..63e6c0dbe7b3 100644
>>> --- a/drivers/net/bonding/bond_main.c
>>> +++ b/drivers/net/bonding/bond_main.c
>>> @@ -928,6 +928,39 @@ static inline void slave_disable_netpoll(struct slave *slave)
>>>
>>>  static void bond_poll_controller(struct net_device *bond_dev)
>>>  {
>>> +     struct bonding *bond = netdev_priv(bond_dev);
>>> +     struct slave *slave = NULL;
>>> +     struct list_head *iter;
>>> +     struct ad_info ad_info;
>>> +     struct netpoll_info *ni;
>>> +     const struct net_device_ops *ops;
>>> +
>>> +     if (BOND_MODE(bond) == BOND_MODE_8023AD)
>>> +             if (bond_3ad_get_active_agg_info(bond, &ad_info))
>>> +                     return;
>>> +
>>> +     rtnl_lock();
>>> +     bond_for_each_slave(bond, slave, iter) {
>>> +             ops = slave->dev->netdev_ops;
>>> +             if (!bond_slave_is_up(slave) || !ops->ndo_poll_controller)
>>> +                     continue;
>>> +
>>> +             if (BOND_MODE(bond) == BOND_MODE_8023AD) {
>>> +                     struct aggregator *agg =
>>> +                         SLAVE_AD_INFO(slave)->port.aggregator;
>>> +
>>> +                     if (agg &&
>>> +                         agg->aggregator_identifier != ad_info.aggregator_id)
>>> +                             continue;
>>> +             }
>>> +
>>> +             ni = rcu_dereference_bh(slave->dev->npinfo);
>>> +             if (down_trylock(&ni->dev_lock))
>>> +                     continue;
>>> +             ops->ndo_poll_controller(slave->dev);
>>> +             up(&ni->dev_lock);
>>> +     }
>>> +     rtnl_unlock();
>>>  }
>>>
>>>  static void bond_netpoll_cleanup(struct net_device *bond_dev)
>>>
>>

Re: [PATCH next v5 2/6] bonding: implement bond_poll_controller()

[Maciej Żenczykowski]

>>> Hi Mahesh,
>>> I should've explained more in my review, you cannot sleep in
>>> bond_poll_controller() so you cannot acquire rtnl like that. I was thinking
>>> more about using rcu and switching to the _rcu version of
>>> bond_for_each_slave instead.
>>>
>> That makes sense. The path that triggered this netpoll() could have
>> been holding the rtnl  itself and this would be a problem. I think
>> using the _rcu variant of the slave iterator is a good idea, my bad!
>>
> ... however we cannot use the _rcu variant either since there is the
> netpoll mutex (ni->dev_lock)!
> The fact that we are here itself means that something bad had happened
> and trying to take additional lock(s) would complicate the situation
> further.

I think you might be incorrectly assuming that we only get here on
kernel crashes,
upstream (netconsole) [may] violate[s] this assumption.

(background: internally we don't consider netconsole production
worthy, but we do
'abuse' the netpoll support framework to generate a minimal network
dump on kernel crash,
in which case if we crash or otherwise violate assumptions while
crashing, we're not any worse off...)

- Maciej

Re: [PATCH next v5 2/6] bonding: implement bond_poll_controller()

[Mahesh Bandewar]

On Wed, Feb 18, 2015 at 6:26 PM, Maciej Żenczykowski <maze@google.com> wrote:
>>>> Hi Mahesh,
>>>> I should've explained more in my review, you cannot sleep in
>>>> bond_poll_controller() so you cannot acquire rtnl like that. I was thinking
>>>> more about using rcu and switching to the _rcu version of
>>>> bond_for_each_slave instead.
>>>>
>>> That makes sense. The path that triggered this netpoll() could have
>>> been holding the rtnl  itself and this would be a problem. I think
>>> using the _rcu variant of the slave iterator is a good idea, my bad!
>>>
>> ... however we cannot use the _rcu variant either since there is the
>> netpoll mutex (ni->dev_lock)!
>> The fact that we are here itself means that something bad had happened
>> and trying to take additional lock(s) would complicate the situation
>> further.
>
> I think you might be incorrectly assuming that we only get here on
> kernel crashes,
> upstream (netconsole) [may] violate[s] this assumption.
>
> (background: internally we don't consider netconsole production
> worthy, but we do
> 'abuse' the netpoll support framework to generate a minimal network
> dump on kernel crash,
> in which case if we crash or otherwise violate assumptions while
> crashing, we're not any worse off...)
>
I think irrespective of how we use it, there are few contradictory things here -
(a) If it can't sleep in netpoll() (which I agree with) then that
netpoll mutex does not make sense. But if it can then rtnl isn't a
wrong thing.
(b) If the netpoll mutex is right then slaves can not be iterated with
rcu protection.
(c) If we takeout the netpoll-mutex, then everything seems right but
can we really do that?

I'll take this patch out of this series for the time being and let
other patches proceed while I think about this patch some more.

--mahesh..

> - Maciej