Re: [PATCH net-next v7 5/6] flow_offload: support get multi-subsystem block

From: Vlad Buslov <vladbu@mellanox.com>
To: Jakub Kicinski <jakub.kicinski@netronome.com>
Cc: Vlad Buslov <vladbu@mellanox.com>, wenxu <wenxu@ucloud.cn>,
	David Miller <davem@davemloft.net>, Jiri Pirko <jiri@resnulli.us>,
	"pablo@netfilter.org" <pablo@netfilter.org>,
	"netfilter-devel@vger.kernel.org"
	<netfilter-devel@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>
Subject: Re: [PATCH net-next v7 5/6] flow_offload: support get multi-subsystem block
Date: Mon, 19 Aug 2019 07:26:07 +0000	[thread overview]
Message-ID: <vbftvady5tg.fsf@mellanox.com> (raw)
In-Reply-To: <20190816105627.57c1c2aa@cakuba.netronome.com>

On Fri 16 Aug 2019 at 20:56, Jakub Kicinski <jakub.kicinski@netronome.com> wrote:
> On Fri, 16 Aug 2019 15:04:44 +0000, Vlad Buslov wrote:
>> >> [  401.511871] RSP: 002b:00007ffca2a9fad8 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
>> >> [  401.511875] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007fad892d30f8
>> >> [  401.511878] RDX: 0000000000000002 RSI: 000055afeb072a90 RDI: 0000000000000001
>> >> [  401.511881] RBP: 000055afeb072a90 R08: 00000000ffffffff R09: 000000000000000a
>> >> [  401.511884] R10: 000055afeb058710 R11: 0000000000000246 R12: 0000000000000002
>> >> [  401.511887] R13: 00007fad893a8780 R14: 0000000000000002 R15: 00007fad893a3740
>> >>
>> >> I don't think it is correct approach to try to call these callbacks with
>> >> rcu protection because:
>> >>
>> >> - Cls API uses sleeping locks that cannot be used in rcu read section
>> >>   (hence the included trace).
>> >>
>> >> - It assumes that all implementation of classifier ops reoffload() don't
>> >>   sleep.
>> >>
>> >> - And that all driver offload callbacks (both block and classifier
>> >>   setup) don't sleep, which is not the case.
>> >>
>> >> I don't see any straightforward way to fix this, besides using some
>> >> other locking mechanism to protect block_ing_cb_list.
>> >>
>> >> Regards,
>> >> Vlad
>> >
>> > Maybe get the  mutex flow_indr_block_ing_cb_lock for both lookup, add, delete?
>> >
>> > the callbacks_lists. the add and delete is work only on modules init case. So the
>> >
>> > lookup is also not frequently(ony [un]register) and can protect with the locks.
>>
>> That should do the job. I'll send the patch.
>
> Hi Vlad!
>
> While looking into this, would you mind also add the missing
> flow_block_cb_is_busy() calls in the indirect handlers in the drivers?
>
> LMK if you're too busy, I don't want this to get forgotten :)

Hi Jakub,

I've checked the code and it looks like only nfp driver is affected:

- I added check in nfp to lookup cb_priv with
  nfp_flower_indr_block_cb_priv_lookup() and call
  flow_block_cb_is_busy() if cb_priv exists.

- In mlx5 en_rep.c there is already a check that indr_priv exists, so
  trying to lookup block_cb->cb_indent==indr_priv is redundant.

- Switch drivers (mlxsw and ocelot) take reference to block_cb on
  FLOW_BLOCK_BIND, so they should not require any modifications.

Tell me if I missed anything. Sending the patch for nfp.

Regards,
Vlad