* [PATCH rdma-next] IB/core: Add ordered workqueue for RoCE GID management
@ 2017-05-30 6:58 Leon Romanovsky
[not found] ` <20170530065806.6245-1-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
0 siblings, 1 reply; 2+ messages in thread
From: Leon Romanovsky @ 2017-05-30 6:58 UTC (permalink / raw)
To: Doug Ledford; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, Majd Dibbiny
From: Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Currently the RoCE GID management uses the ib_wq to do add and delete new GIDs
according to the netdev events.
The ib_wq isn't an ordered workqueue and thus two work elements can be executed
concurrently which will result in unexpected behavior and inconsistency of the
GIDs cache content.
Example:
ifconfig eth1 11.11.11.11/16 up
This command will invoke the following netdev events in the following order:
1. NETDEV_UP
2. NETDEV_DOWN
3. NETDEV_UP
If (2) and (3) will be executed concurrently or in reverse order, instead of
having a new GID with 11.11.11.11 IP, we will end up without any new GIDs.
Signed-off-by: Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
---
drivers/infiniband/core/roce_gid_mgmt.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/core/roce_gid_mgmt.c b/drivers/infiniband/core/roce_gid_mgmt.c
index db958d3207ef..94a9eefb3cfc 100644
--- a/drivers/infiniband/core/roce_gid_mgmt.c
+++ b/drivers/infiniband/core/roce_gid_mgmt.c
@@ -42,6 +42,8 @@
#include <rdma/ib_cache.h>
#include <rdma/ib_addr.h>
+static struct workqueue_struct *gid_cache_wq;
+
enum gid_op_type {
GID_DEL = 0,
GID_ADD
@@ -560,7 +562,7 @@ static int netdevice_queue_work(struct netdev_event_work_cmd *cmds,
}
INIT_WORK(&ndev_work->work, netdevice_event_work_handler);
- queue_work(ib_wq, &ndev_work->work);
+ queue_work(gid_cache_wq, &ndev_work->work);
return NOTIFY_DONE;
}
@@ -693,7 +695,7 @@ static int addr_event(struct notifier_block *this, unsigned long event,
dev_hold(ndev);
work->gid_attr.ndev = ndev;
- queue_work(ib_wq, &work->work);
+ queue_work(gid_cache_wq, &work->work);
return NOTIFY_DONE;
}
@@ -740,6 +742,10 @@ static struct notifier_block nb_inet6addr = {
int __init roce_gid_mgmt_init(void)
{
+ gid_cache_wq = alloc_ordered_workqueue("gid-cache-wq", 0);
+ if (!gid_cache_wq)
+ return -ENOMEM;
+
register_inetaddr_notifier(&nb_inetaddr);
if (IS_ENABLED(CONFIG_IPV6))
register_inet6addr_notifier(&nb_inet6addr);
@@ -764,4 +770,5 @@ void __exit roce_gid_mgmt_cleanup(void)
* ib-core is removed, all physical devices have been removed,
* so no issue with remaining hardware contexts.
*/
+ destroy_workqueue(gid_cache_wq);
}
--
2.12.2
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 2+ messages in thread
* Re: [PATCH rdma-next] IB/core: Add ordered workqueue for RoCE GID management
[not found] ` <20170530065806.6245-1-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
@ 2017-06-06 7:12 ` Yuval Shaia
0 siblings, 0 replies; 2+ messages in thread
From: Yuval Shaia @ 2017-06-06 7:12 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Doug Ledford, linux-rdma-u79uwXL29TY76Z2rM5mHXA, Majd Dibbiny
On Tue, May 30, 2017 at 09:58:06AM +0300, Leon Romanovsky wrote:
> From: Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
>
> Currently the RoCE GID management uses the ib_wq to do add and delete new GIDs
> according to the netdev events.
>
> The ib_wq isn't an ordered workqueue and thus two work elements can be executed
> concurrently which will result in unexpected behavior and inconsistency of the
> GIDs cache content.
>
> Example:
> ifconfig eth1 11.11.11.11/16 up
>
> This command will invoke the following netdev events in the following order:
> 1. NETDEV_UP
> 2. NETDEV_DOWN
> 3. NETDEV_UP
>
> If (2) and (3) will be executed concurrently or in reverse order, instead of
> having a new GID with 11.11.11.11 IP, we will end up without any new GIDs.
>
> Signed-off-by: Majd Dibbiny <majd-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
> ---
> drivers/infiniband/core/roce_gid_mgmt.c | 11 +++++++++--
> 1 file changed, 9 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/infiniband/core/roce_gid_mgmt.c b/drivers/infiniband/core/roce_gid_mgmt.c
> index db958d3207ef..94a9eefb3cfc 100644
> --- a/drivers/infiniband/core/roce_gid_mgmt.c
> +++ b/drivers/infiniband/core/roce_gid_mgmt.c
> @@ -42,6 +42,8 @@
> #include <rdma/ib_cache.h>
> #include <rdma/ib_addr.h>
>
> +static struct workqueue_struct *gid_cache_wq;
> +
> enum gid_op_type {
> GID_DEL = 0,
> GID_ADD
> @@ -560,7 +562,7 @@ static int netdevice_queue_work(struct netdev_event_work_cmd *cmds,
> }
> INIT_WORK(&ndev_work->work, netdevice_event_work_handler);
>
> - queue_work(ib_wq, &ndev_work->work);
> + queue_work(gid_cache_wq, &ndev_work->work);
>
> return NOTIFY_DONE;
> }
> @@ -693,7 +695,7 @@ static int addr_event(struct notifier_block *this, unsigned long event,
> dev_hold(ndev);
> work->gid_attr.ndev = ndev;
>
> - queue_work(ib_wq, &work->work);
> + queue_work(gid_cache_wq, &work->work);
>
> return NOTIFY_DONE;
> }
> @@ -740,6 +742,10 @@ static struct notifier_block nb_inet6addr = {
>
> int __init roce_gid_mgmt_init(void)
> {
> + gid_cache_wq = alloc_ordered_workqueue("gid-cache-wq", 0);
> + if (!gid_cache_wq)
> + return -ENOMEM;
> +
> register_inetaddr_notifier(&nb_inetaddr);
> if (IS_ENABLED(CONFIG_IPV6))
> register_inet6addr_notifier(&nb_inet6addr);
> @@ -764,4 +770,5 @@ void __exit roce_gid_mgmt_cleanup(void)
> * ib-core is removed, all physical devices have been removed,
> * so no issue with remaining hardware contexts.
> */
> + destroy_workqueue(gid_cache_wq);
> }
Reviewed-by: Yuval Shaia <yuval.shaia-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
> --
> 2.12.2
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2017-06-06 7:12 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-05-30 6:58 [PATCH rdma-next] IB/core: Add ordered workqueue for RoCE GID management Leon Romanovsky
[not found] ` <20170530065806.6245-1-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
2017-06-06 7:12 ` Yuval Shaia
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.