From: Jason Gunthorpe <jgg@nvidia.com>
To: Leon Romanovsky <leon@kernel.org>
Cc: Patrisious Haddad <phaddad@nvidia.com>,
linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org,
Mark Zhang <markzhang@nvidia.com>
Subject: Re: [PATCH rdma-next 2/2] RDMA/core: Add a netevent notifier to cma
Date: Tue, 10 May 2022 21:04:11 -0300 [thread overview]
Message-ID: <20220511000411.GA1164300@nvidia.com> (raw)
In-Reply-To: <8c85028f89a877e9b4e6bb58bdd8a7f2cb4567a9.1649075034.git.leonro@nvidia.com>
On Mon, Apr 04, 2022 at 03:27:27PM +0300, Leon Romanovsky wrote:
> @@ -5054,10 +5061,95 @@ static int cma_netdev_callback(struct notifier_block *self, unsigned long event,
> return ret;
> }
>
> +static void cma_netevent_work_handler(struct work_struct *_work)
> +{
> + struct cma_netevent_work *network =
> + container_of(_work, struct cma_netevent_work, work);
> + struct rdma_cm_event event = {};
> +
> + mutex_lock(&network->id_priv->handler_mutex);
> +
> + if (READ_ONCE(network->id_priv->state) == RDMA_CM_DESTROYING ||
> + READ_ONCE(network->id_priv->state) == RDMA_CM_DEVICE_REMOVAL)
> + goto out_unlock;
> +
> + event.event = RDMA_CM_EVENT_UNREACHABLE;
> + event.status = -ETIMEDOUT;
> +
> + if (cma_cm_event_handler(network->id_priv, &event)) {
> + __acquire(&network->id_priv->handler_mutex);
??
> + network->id_priv->cm_id.ib = NULL;
> + cma_id_put(network->id_priv);
> + destroy_id_handler_unlock(network->id_priv);
> + kfree(network);
> + return;
> + }
> +
> +out_unlock:
> + mutex_unlock(&network->id_priv->handler_mutex);
> + cma_id_put(network->id_priv);
> + kfree(network);
> +}
> +
> +static int cma_netevent_callback(struct notifier_block *self,
> + unsigned long event, void *ctx)
> +{
> + struct id_table_entry *ips_node = NULL;
> + struct rdma_id_private *current_id;
> + struct cma_netevent_work *network;
> + struct neighbour *neigh = ctx;
> + unsigned long flags;
> +
> + if (event != NETEVENT_NEIGH_UPDATE)
> + return NOTIFY_DONE;
> +
> + spin_lock_irqsave(&id_table_lock, flags);
> + if (neigh->tbl->family == AF_INET6) {
> + struct sockaddr_in6 neigh_sock_6;
> +
> + neigh_sock_6.sin6_family = AF_INET6;
> + neigh_sock_6.sin6_addr = *(struct in6_addr *)neigh->primary_key;
> + ips_node = node_from_ndev_ip(&id_table, neigh->dev->ifindex,
> + (struct sockaddr *)&neigh_sock_6);
> + } else if (neigh->tbl->family == AF_INET) {
> + struct sockaddr_in neigh_sock_4;
> +
> + neigh_sock_4.sin_family = AF_INET;
> + neigh_sock_4.sin_addr.s_addr = *(__be32 *)(neigh->primary_key);
> + ips_node = node_from_ndev_ip(&id_table, neigh->dev->ifindex,
> + (struct sockaddr *)&neigh_sock_4);
> + } else
> + goto out;
> +
> + if (!ips_node)
> + goto out;
> +
> + list_for_each_entry(current_id, &ips_node->id_list, id_list_entry) {
> + if (!memcmp(current_id->id.route.addr.dev_addr.dst_dev_addr,
> + neigh->ha, ETH_ALEN))
> + continue;
> + network = kzalloc(sizeof(*network), GFP_ATOMIC);
> + if (!network)
> + goto out;
> +
> + INIT_WORK(&network->work, cma_netevent_work_handler);
> + network->id_priv = current_id;
> + cma_id_get(current_id);
> + queue_work(cma_netevent_wq, &network->work);
This is pretty ugly that we need to do atomic allocations for every
matching id.
It would be better to add the work directly to the rdma_cm_id and just
waste that memory.
> + cma_netevent_wq = alloc_ordered_workqueue("rdma_cm_netevent", 0);
> + if (!cma_netevent_wq) {
> + ret = -ENOMEM;
> + goto err_netevent_wq;
> + }
Why do we need another WQ? Why does it need to be ordered?
Jason
next prev parent reply other threads:[~2022-05-11 0:04 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-04 12:27 [PATCH rdma-next 0/2] Add gratuitous ARP support to RDMA-CM Leon Romanovsky
2022-04-04 12:27 ` [PATCH rdma-next 1/2] RDMA/core: Add an rb_tree that stores cm_ids sorted by ifindex and remote IP Leon Romanovsky
2022-05-10 23:56 ` Jason Gunthorpe
2022-04-04 12:27 ` [PATCH rdma-next 2/2] RDMA/core: Add a netevent notifier to cma Leon Romanovsky
2022-05-11 0:04 ` Jason Gunthorpe [this message]
2022-04-29 16:07 ` [PATCH rdma-next 0/2] Add gratuitous ARP support to RDMA-CM Olga Kornievskaia
2022-04-29 16:23 ` Leon Romanovsky
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220511000411.GA1164300@nvidia.com \
--to=jgg@nvidia.com \
--cc=leon@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=markzhang@nvidia.com \
--cc=phaddad@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).