All of lore.kernel.org
 help / color / mirror / Atom feed
From: Sagi Grimberg <sagi@grimberg.me>
To: linux-rdma@vger.kernel.org, linux-nvme@lists.infradead.org,
	linux-block@vger.kernel.org
Cc: netdev@vger.kernel.org, Saeed Mahameed <saeedm@mellanox.com>,
	Or Gerlitz <ogerlitz@mellanox.com>,
	Christoph Hellwig <hch@lst.de>
Subject: [PATCH rfc 3/6] RDMA/core: expose affinity mappings per completion vector
Date: Sun,  2 Apr 2017 16:41:29 +0300	[thread overview]
Message-ID: <1491140492-25703-4-git-send-email-sagi@grimberg.me> (raw)
In-Reply-To: <1491140492-25703-1-git-send-email-sagi@grimberg.me>

This will allow ULPs to intelligently locate threads based
on completion vector cpu affinity mappings. In case the
driver does not expose a get_vector_affinity callout, return
NULL so the caller can maintain a fallback logic.

Signed-off-by: Sagi Grimberg <sagi@grimberg.me>
---
 include/rdma/ib_verbs.h | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 0f1813c13687..d44b62791c64 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -2150,6 +2150,8 @@ struct ib_device {
 	 */
 	int (*get_port_immutable)(struct ib_device *, u8, struct ib_port_immutable *);
 	void (*get_dev_fw_str)(struct ib_device *, char *str, size_t str_len);
+	const struct cpumask *(*get_vector_affinity)(struct ib_device *ibdev,
+						     int comp_vector);
 };
 
 struct ib_client {
@@ -3377,4 +3379,26 @@ void ib_drain_qp(struct ib_qp *qp);
 
 int ib_resolve_eth_dmac(struct ib_device *device,
 			struct ib_ah_attr *ah_attr);
+
+/**
+ * ib_get_vector_affinity - Get the affinity mappings of a given completion
+ *   vector
+ * @device:         the rdma device
+ * @comp_vector:    index of completion vector
+ *
+ * Returns NULL on failure, otherwise a corresponding cpu map of the
+ * completion vector (returns all-cpus map if the device driver doesn't
+ * implement get_vector_affinity).
+ */
+static inline const struct cpumask *
+ib_get_vector_affinity(struct ib_device *device, int comp_vector)
+{
+	if (comp_vector > device->num_comp_vectors ||
+	    !device->get_vector_affinity)
+		return NULL;
+
+	return device->get_vector_affinity(device, comp_vector);
+
+}
+
 #endif /* IB_VERBS_H */
-- 
2.7.4

WARNING: multiple messages have this Message-ID (diff)
From: sagi@grimberg.me (Sagi Grimberg)
Subject: [PATCH rfc 3/6] RDMA/core: expose affinity mappings per completion vector
Date: Sun,  2 Apr 2017 16:41:29 +0300	[thread overview]
Message-ID: <1491140492-25703-4-git-send-email-sagi@grimberg.me> (raw)
In-Reply-To: <1491140492-25703-1-git-send-email-sagi@grimberg.me>

This will allow ULPs to intelligently locate threads based
on completion vector cpu affinity mappings. In case the
driver does not expose a get_vector_affinity callout, return
NULL so the caller can maintain a fallback logic.

Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
 include/rdma/ib_verbs.h | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 0f1813c13687..d44b62791c64 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -2150,6 +2150,8 @@ struct ib_device {
 	 */
 	int (*get_port_immutable)(struct ib_device *, u8, struct ib_port_immutable *);
 	void (*get_dev_fw_str)(struct ib_device *, char *str, size_t str_len);
+	const struct cpumask *(*get_vector_affinity)(struct ib_device *ibdev,
+						     int comp_vector);
 };
 
 struct ib_client {
@@ -3377,4 +3379,26 @@ void ib_drain_qp(struct ib_qp *qp);
 
 int ib_resolve_eth_dmac(struct ib_device *device,
 			struct ib_ah_attr *ah_attr);
+
+/**
+ * ib_get_vector_affinity - Get the affinity mappings of a given completion
+ *   vector
+ * @device:         the rdma device
+ * @comp_vector:    index of completion vector
+ *
+ * Returns NULL on failure, otherwise a corresponding cpu map of the
+ * completion vector (returns all-cpus map if the device driver doesn't
+ * implement get_vector_affinity).
+ */
+static inline const struct cpumask *
+ib_get_vector_affinity(struct ib_device *device, int comp_vector)
+{
+	if (comp_vector > device->num_comp_vectors ||
+	    !device->get_vector_affinity)
+		return NULL;
+
+	return device->get_vector_affinity(device, comp_vector);
+
+}
+
 #endif /* IB_VERBS_H */
-- 
2.7.4

  parent reply	other threads:[~2017-04-02 13:42 UTC|newest]

Thread overview: 52+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-04-02 13:41 [PATCH rfc 0/6] Automatic affinity settings for nvme over rdma Sagi Grimberg
2017-04-02 13:41 ` Sagi Grimberg
2017-04-02 13:41 ` [PATCH rfc 1/6] mlx5: convert to generic pci_alloc_irq_vectors Sagi Grimberg
2017-04-02 13:41   ` Sagi Grimberg
2017-04-04  6:27   ` Christoph Hellwig
2017-04-04  6:27     ` Christoph Hellwig
2017-04-02 13:41 ` [PATCH rfc 2/6] mlx5: move affinity hints assignments to generic code Sagi Grimberg
2017-04-02 13:41   ` Sagi Grimberg
2017-04-04  6:32   ` Christoph Hellwig
2017-04-04  6:32     ` Christoph Hellwig
2017-04-06  8:29     ` Sagi Grimberg
2017-04-06  8:29       ` Sagi Grimberg
2017-04-02 13:41 ` Sagi Grimberg [this message]
2017-04-02 13:41   ` [PATCH rfc 3/6] RDMA/core: expose affinity mappings per completion vector Sagi Grimberg
2017-04-04  6:32   ` Christoph Hellwig
2017-04-04  6:32     ` Christoph Hellwig
2017-04-02 13:41 ` [PATCH rfc 4/6] mlx5: support ->get_vector_affinity Sagi Grimberg
2017-04-02 13:41   ` Sagi Grimberg
2017-04-04  6:33   ` Christoph Hellwig
2017-04-04  6:33     ` Christoph Hellwig
2017-04-02 13:41 ` [PATCH rfc 5/6] block: Add rdma affinity based queue mapping helper Sagi Grimberg
2017-04-02 13:41   ` Sagi Grimberg
2017-04-04  6:33   ` Christoph Hellwig
2017-04-04  6:33     ` Christoph Hellwig
2017-04-04  7:46   ` Max Gurtovoy
2017-04-04  7:46     ` Max Gurtovoy
2017-04-04  7:46     ` Max Gurtovoy
2017-04-04  7:46     ` Max Gurtovoy
2017-04-04 13:09     ` Christoph Hellwig
2017-04-04 13:09       ` Christoph Hellwig
2017-04-06  9:23     ` Sagi Grimberg
2017-04-06  9:23       ` Sagi Grimberg
2017-04-06  9:23       ` Sagi Grimberg
2017-04-05 14:17   ` Jens Axboe
2017-04-05 14:17     ` Jens Axboe
2017-04-02 13:41 ` [PATCH rfc 6/6] nvme-rdma: use intelligent affinity based queue mappings Sagi Grimberg
2017-04-02 13:41   ` Sagi Grimberg
2017-04-04  6:34   ` Christoph Hellwig
2017-04-04  6:34     ` Christoph Hellwig
2017-04-06  8:30     ` Sagi Grimberg
2017-04-06  8:30       ` Sagi Grimberg
2017-04-04  7:51 ` [PATCH rfc 0/6] Automatic affinity settings for nvme over rdma Max Gurtovoy
2017-04-04  7:51   ` Max Gurtovoy
2017-04-04  7:51   ` Max Gurtovoy
2017-04-04  7:51   ` Max Gurtovoy
2017-04-06  8:34   ` Sagi Grimberg
2017-04-06  8:34     ` Sagi Grimberg
2017-04-06  8:34     ` Sagi Grimberg
2017-04-10 18:05 ` Steve Wise
2017-04-10 18:05   ` Steve Wise
2017-04-12  6:34   ` Christoph Hellwig
2017-04-12  6:34     ` Christoph Hellwig

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1491140492-25703-4-git-send-email-sagi@grimberg.me \
    --to=sagi@grimberg.me \
    --cc=hch@lst.de \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=ogerlitz@mellanox.com \
    --cc=saeedm@mellanox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.