* [PATCH 0/2] qla2xxx - add nvme map_queues support
@ 2021-08-23 12:56 Nilesh Javali
2021-08-23 12:56 ` [PATCH 1/2] nvme-fc: Add support for map_queues Nilesh Javali
` (2 more replies)
0 siblings, 3 replies; 14+ messages in thread
From: Nilesh Javali @ 2021-08-23 12:56 UTC (permalink / raw)
To: martin.petersen, linux-nvme; +Cc: linux-scsi, GR-QLogic-Storage-Upstream
Currently nvme fc doesn't support map queue functionality. This patch
set adds map_queue functionality to nvme_fc_mq_ops and
nvme_fc_port_template, providing an option to LLDs to map queues
similar to SCSI. For qla2xxx, minimum 10% improvement is noticed
with this change as it helps in reducing cpu thrashing.
Saurav Kashyap (2):
nvme-fc: Add support for map_queues.
qla2xxx: Add map_queues support to nvme.
drivers/nvme/host/fc.c | 25 +++++++++++++++++++++++++
drivers/scsi/qla2xxx/qla_nvme.c | 14 ++++++++++++++
include/linux/nvme-fc-driver.h | 7 +++++++
3 files changed, 46 insertions(+)
base-commit: 92cc94adfce4683d0b421cbf59013703368aaeb9
--
2.23.1
^ permalink raw reply [flat|nested] 14+ messages in thread* [PATCH 1/2] nvme-fc: Add support for map_queues. 2021-08-23 12:56 [PATCH 0/2] qla2xxx - add nvme map_queues support Nilesh Javali @ 2021-08-23 12:56 ` Nilesh Javali 2021-08-23 12:56 ` [PATCH 2/2] qla2xxx: Add map_queues support to nvme Nilesh Javali 2021-08-23 17:21 ` [PATCH 0/2] qla2xxx - add nvme map_queues support Sagi Grimberg 2 siblings, 0 replies; 14+ messages in thread From: Nilesh Javali @ 2021-08-23 12:56 UTC (permalink / raw) To: martin.petersen, linux-nvme; +Cc: linux-scsi, GR-QLogic-Storage-Upstream From: Saurav Kashyap <skashyap@marvell.com> NVMe FC don't have support for map queues as compared to pci, rdma and tcp. It doesn't provide a provision to LLDs to change the queue mapping like scsi layer. This patch adds an option for LLDs to change the queue mapping. Signed-off-by: Saurav Kashyap <skashyap@marvell.com> Signed-off-by: Nilesh Javali <njavali@marvell.com> --- drivers/nvme/host/fc.c | 25 +++++++++++++++++++++++++ include/linux/nvme-fc-driver.h | 7 +++++++ 2 files changed, 32 insertions(+) diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c index b08a61ca283f..078895809098 100644 --- a/drivers/nvme/host/fc.c +++ b/drivers/nvme/host/fc.c @@ -16,6 +16,7 @@ #include <linux/nvme-fc.h> #include "fc.h" #include <scsi/scsi_transport_fc.h> +#include <linux/blk-mq-pci.h> /* *************************** Data Structures/Defines ****************** */ @@ -2839,6 +2840,29 @@ nvme_fc_complete_rq(struct request *rq) nvme_fc_ctrl_put(ctrl); } +static int nvme_fc_map_queues(struct blk_mq_tag_set *set) +{ + int i; + struct nvme_fc_ctrl *ctrl; + struct blk_mq_queue_map *map = NULL; + + ctrl = set->driver_data; + for (i = 0; i < set->nr_maps; i++) { + map = &set->map[i]; + + if (!map->nr_queues) { + WARN_ON(i == HCTX_TYPE_DEFAULT); + continue; + } + + /* Call LLDD map queue functionality if defined */ + if (ctrl->lport->ops->map_queues) + ctrl->lport->ops->map_queues(&ctrl->lport->localport, map); + else + blk_mq_map_queues(map); + } + return 0; +} static const struct blk_mq_ops nvme_fc_mq_ops = { .queue_rq = nvme_fc_queue_rq, @@ -2847,6 +2871,7 @@ static const struct blk_mq_ops nvme_fc_mq_ops = { .exit_request = nvme_fc_exit_request, .init_hctx = nvme_fc_init_hctx, .timeout = nvme_fc_timeout, + .map_queues = nvme_fc_map_queues, }; static int diff --git a/include/linux/nvme-fc-driver.h b/include/linux/nvme-fc-driver.h index 2a38f2b477a5..cb909edb76c4 100644 --- a/include/linux/nvme-fc-driver.h +++ b/include/linux/nvme-fc-driver.h @@ -7,6 +7,7 @@ #define _NVME_FC_DRIVER_H 1 #include <linux/scatterlist.h> +#include <linux/blk-mq.h> /* @@ -497,6 +498,8 @@ struct nvme_fc_port_template { int (*xmt_ls_rsp)(struct nvme_fc_local_port *localport, struct nvme_fc_remote_port *rport, struct nvmefc_ls_rsp *ls_rsp); + void (*map_queues)(struct nvme_fc_local_port *localport, + struct blk_mq_queue_map *map); u32 max_hw_queues; u16 max_sgl_segments; @@ -779,6 +782,10 @@ struct nvmet_fc_target_port { * LS received. * Entrypoint is Mandatory. * + * @map_queues: This functions lets the driver expose the queue mapping + * to the block layer. + * Entrypoint is Optional. + * * @fcp_op: Called to perform a data transfer or transmit a response. * The nvmefc_tgt_fcp_req structure is the same LLDD-supplied * exchange structure specified in the nvmet_fc_rcv_fcp_req() call -- 2.23.1 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 2/2] qla2xxx: Add map_queues support to nvme. 2021-08-23 12:56 [PATCH 0/2] qla2xxx - add nvme map_queues support Nilesh Javali 2021-08-23 12:56 ` [PATCH 1/2] nvme-fc: Add support for map_queues Nilesh Javali @ 2021-08-23 12:56 ` Nilesh Javali 2021-08-23 17:21 ` [PATCH 0/2] qla2xxx - add nvme map_queues support Sagi Grimberg 2 siblings, 0 replies; 14+ messages in thread From: Nilesh Javali @ 2021-08-23 12:56 UTC (permalink / raw) To: martin.petersen, linux-nvme; +Cc: linux-scsi, GR-QLogic-Storage-Upstream From: Saurav Kashyap <skashyap@marvell.com> Add callback for map queues and use block layer blk_mq_pci_map_queues for mapping. With this mapping minimum 10% increase in performance is noticed. Signed-off-by: Saurav Kashyap <skashyap@marvell.com> Signed-off-by: Nilesh Javali <njavali@marvell.com> --- drivers/scsi/qla2xxx/qla_nvme.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/drivers/scsi/qla2xxx/qla_nvme.c b/drivers/scsi/qla2xxx/qla_nvme.c index 05cad06ff165..7f9a7abc615b 100644 --- a/drivers/scsi/qla2xxx/qla_nvme.c +++ b/drivers/scsi/qla2xxx/qla_nvme.c @@ -8,6 +8,8 @@ #include <linux/delay.h> #include <linux/nvme.h> #include <linux/nvme-fc.h> +#include <linux/blk-mq-pci.h> +#include <linux/blk-mq.h> static struct nvme_fc_port_template qla_nvme_fc_transport; @@ -634,6 +636,17 @@ static int qla_nvme_post_cmd(struct nvme_fc_local_port *lport, return rval; } +static void qla_nvme_map_queues(struct nvme_fc_local_port *lport, struct blk_mq_queue_map *map) +{ + int rc; + struct scsi_qla_host *vha = lport->private; + + rc = blk_mq_pci_map_queues(map, vha->hw->pdev, vha->irq_offset); + if (rc) + ql_log(ql_log_warn, vha, 0x21de, + "pci map queue failed 0x%x", rc); +} + static void qla_nvme_localport_delete(struct nvme_fc_local_port *lport) { struct scsi_qla_host *vha = lport->private; @@ -668,6 +681,7 @@ static struct nvme_fc_port_template qla_nvme_fc_transport = { .ls_abort = qla_nvme_ls_abort, .fcp_io = qla_nvme_post_cmd, .fcp_abort = qla_nvme_fcp_abort, + .map_queues = qla_nvme_map_queues, .max_hw_queues = 8, .max_sgl_segments = 1024, .max_dif_sgl_segments = 64, -- 2.23.1 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH 0/2] qla2xxx - add nvme map_queues support 2021-08-23 12:56 [PATCH 0/2] qla2xxx - add nvme map_queues support Nilesh Javali 2021-08-23 12:56 ` [PATCH 1/2] nvme-fc: Add support for map_queues Nilesh Javali 2021-08-23 12:56 ` [PATCH 2/2] qla2xxx: Add map_queues support to nvme Nilesh Javali @ 2021-08-23 17:21 ` Sagi Grimberg 2021-08-24 3:38 ` Saurav Kashyap 2 siblings, 1 reply; 14+ messages in thread From: Sagi Grimberg @ 2021-08-23 17:21 UTC (permalink / raw) To: Nilesh Javali, martin.petersen, linux-nvme, Ming Lei Cc: linux-scsi, GR-QLogic-Storage-Upstream On 8/23/21 5:56 AM, Nilesh Javali wrote: > Currently nvme fc doesn't support map queue functionality. This patch > set adds map_queue functionality to nvme_fc_mq_ops and > nvme_fc_port_template, providing an option to LLDs to map queues > similar to SCSI. For qla2xxx, minimum 10% improvement is noticed > with this change as it helps in reducing cpu thrashing. Does this make nvme-fc use managed irq? CCing Ming to see if this affects his patchset. ^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [PATCH 0/2] qla2xxx - add nvme map_queues support 2021-08-23 17:21 ` [PATCH 0/2] qla2xxx - add nvme map_queues support Sagi Grimberg @ 2021-08-24 3:38 ` Saurav Kashyap 2021-08-24 3:46 ` Ming Lei 0 siblings, 1 reply; 14+ messages in thread From: Saurav Kashyap @ 2021-08-24 3:38 UTC (permalink / raw) To: Sagi Grimberg, Nilesh Javali, martin.petersen, linux-nvme, Ming Lei Cc: linux-scsi, GR-QLogic-Storage-Upstream Hi Sagi, Comments inline > -----Original Message----- > From: Sagi Grimberg <sagi@grimberg.me> > Sent: Monday, August 23, 2021 10:51 PM > To: Nilesh Javali <njavali@marvell.com>; martin.petersen@oracle.com; linux- > nvme@lists.infradead.org; Ming Lei <ming.lei@redhat.com> > Cc: linux-scsi@vger.kernel.org; GR-QLogic-Storage-Upstream <GR-QLogic- > Storage-Upstream@marvell.com> > Subject: Re: [PATCH 0/2] qla2xxx - add nvme map_queues support > > > On 8/23/21 5:56 AM, Nilesh Javali wrote: > > Currently nvme fc doesn't support map queue functionality. This patch > > set adds map_queue functionality to nvme_fc_mq_ops and > > nvme_fc_port_template, providing an option to LLDs to map queues > > similar to SCSI. For qla2xxx, minimum 10% improvement is noticed > > with this change as it helps in reducing cpu thrashing. > > Does this make nvme-fc use managed irq? qla2xxx driver uses pci_alloc_irq_vectors_affinity to have affinity with each MSI-X vector. Currently nvme queue are not mapped based on affinity and irq offset. The change is to use blk_mq_pci_map_queues for mapping, this function consider irq affinity as well as irq offset. Thanks, ~Saurav > > CCing Ming to see if this affects his patchset. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 0/2] qla2xxx - add nvme map_queues support 2021-08-24 3:38 ` Saurav Kashyap @ 2021-08-24 3:46 ` Ming Lei 2021-08-25 4:58 ` [EXT] " Saurav Kashyap 2021-09-14 9:31 ` Daniel Wagner 0 siblings, 2 replies; 14+ messages in thread From: Ming Lei @ 2021-08-24 3:46 UTC (permalink / raw) To: Saurav Kashyap Cc: Sagi Grimberg, Nilesh Javali, martin.petersen, linux-nvme, linux-scsi, GR-QLogic-Storage-Upstream On Tue, Aug 24, 2021 at 03:38:24AM +0000, Saurav Kashyap wrote: > Hi Sagi, > Comments inline > > > -----Original Message----- > > From: Sagi Grimberg <sagi@grimberg.me> > > Sent: Monday, August 23, 2021 10:51 PM > > To: Nilesh Javali <njavali@marvell.com>; martin.petersen@oracle.com; linux- > > nvme@lists.infradead.org; Ming Lei <ming.lei@redhat.com> > > Cc: linux-scsi@vger.kernel.org; GR-QLogic-Storage-Upstream <GR-QLogic- > > Storage-Upstream@marvell.com> > > Subject: Re: [PATCH 0/2] qla2xxx - add nvme map_queues support > > > > > > On 8/23/21 5:56 AM, Nilesh Javali wrote: > > > Currently nvme fc doesn't support map queue functionality. This patch > > > set adds map_queue functionality to nvme_fc_mq_ops and > > > nvme_fc_port_template, providing an option to LLDs to map queues > > > similar to SCSI. For qla2xxx, minimum 10% improvement is noticed > > > with this change as it helps in reducing cpu thrashing. > > > > Does this make nvme-fc use managed irq? > > qla2xxx driver uses pci_alloc_irq_vectors_affinity to have affinity with each MSI-X vector. Currently nvme queue are not mapped based on affinity and irq offset. The change is to use blk_mq_pci_map_queues for mapping, this function consider irq affinity as well as irq offset. > OK, got it. Even though without this patchset, nvme-fc actually relies on managed irq since qla2xxx driver uses pci_alloc_irq_vectors_affinity. Now the patchset[1] isn't good for addressing the issue in blk_mq_alloc_request_hctx(). [1] https://lore.kernel.org/linux-block/YR7demOSG6MKFVAF@T590/T/#t Thanks, Ming ^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [EXT] Re: [PATCH 0/2] qla2xxx - add nvme map_queues support 2021-08-24 3:46 ` Ming Lei @ 2021-08-25 4:58 ` Saurav Kashyap 2021-09-07 9:18 ` Saurav Kashyap 2021-09-14 9:31 ` Daniel Wagner 1 sibling, 1 reply; 14+ messages in thread From: Saurav Kashyap @ 2021-08-25 4:58 UTC (permalink / raw) To: Ming Lei Cc: Sagi Grimberg, Nilesh Javali, martin.petersen, linux-nvme, linux-scsi, GR-QLogic-Storage-Upstream Hi Ming, Comments inline > -----Original Message----- > From: Ming Lei <ming.lei@redhat.com> > Sent: Tuesday, August 24, 2021 9:17 AM > To: Saurav Kashyap <skashyap@marvell.com> > Cc: Sagi Grimberg <sagi@grimberg.me>; Nilesh Javali <njavali@marvell.com>; > martin.petersen@oracle.com; linux-nvme@lists.infradead.org; linux- > scsi@vger.kernel.org; GR-QLogic-Storage-Upstream <GR-QLogic-Storage- > Upstream@marvell.com> > Subject: [EXT] Re: [PATCH 0/2] qla2xxx - add nvme map_queues support > > External Email > > ---------------------------------------------------------------------- > On Tue, Aug 24, 2021 at 03:38:24AM +0000, Saurav Kashyap wrote: > > Hi Sagi, > > Comments inline > > > > > -----Original Message----- > > > From: Sagi Grimberg <sagi@grimberg.me> > > > Sent: Monday, August 23, 2021 10:51 PM > > > To: Nilesh Javali <njavali@marvell.com>; martin.petersen@oracle.com; > linux- > > > nvme@lists.infradead.org; Ming Lei <ming.lei@redhat.com> > > > Cc: linux-scsi@vger.kernel.org; GR-QLogic-Storage-Upstream <GR-QLogic- > > > Storage-Upstream@marvell.com> > > > Subject: Re: [PATCH 0/2] qla2xxx - add nvme map_queues support > > > > > > > > > On 8/23/21 5:56 AM, Nilesh Javali wrote: > > > > Currently nvme fc doesn't support map queue functionality. This patch > > > > set adds map_queue functionality to nvme_fc_mq_ops and > > > > nvme_fc_port_template, providing an option to LLDs to map queues > > > > similar to SCSI. For qla2xxx, minimum 10% improvement is noticed > > > > with this change as it helps in reducing cpu thrashing. > > > > > > Does this make nvme-fc use managed irq? > > > > qla2xxx driver uses pci_alloc_irq_vectors_affinity to have affinity with each > MSI-X vector. Currently nvme queue are not mapped based on affinity and irq > offset. The change is to use blk_mq_pci_map_queues for mapping, this function > consider irq affinity as well as irq offset. > > > > OK, got it. Even though without this patchset, nvme-fc actually relies > on managed irq since qla2xxx driver uses pci_alloc_irq_vectors_affinity. > > Now the patchset[1] isn't good for addressing the issue in > blk_mq_alloc_request_hctx(). Can you please elaborate on this? Is there something needs to be done from my side? Thanks, ~Saurav > > [1] https://urldefense.proofpoint.com/v2/url?u=https- > 3A__lore.kernel.org_linux-2Dblock_YR7demOSG6MKFVAF-40T590_T_- > 23t&d=DwIBAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=ZHZbmY_LbM3DUZK_BDO1 > OITP3ot_Vkb_5w- > gas5TBMQ&m=CqFDnfAsZphubKXkUx5gsRF6RZ2Qe6sxWkYq4pBfFD0&s=2Nba > EUI5eB6_R6PxW8ld1Xn2OU3_UdD6D30uvFAWhow&e= > > > Thanks, > Ming ^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [EXT] Re: [PATCH 0/2] qla2xxx - add nvme map_queues support 2021-08-25 4:58 ` [EXT] " Saurav Kashyap @ 2021-09-07 9:18 ` Saurav Kashyap 2021-09-14 8:07 ` Saurav Kashyap 0 siblings, 1 reply; 14+ messages in thread From: Saurav Kashyap @ 2021-09-07 9:18 UTC (permalink / raw) To: Ming Lei, Sagi Grimberg, linux-nvme Cc: Nilesh Javali, martin.petersen, linux-scsi, GR-QLogic-Storage-Upstream Hi, Can I get a review for this patch set? Thanks, ~Saurav > -----Original Message----- > From: Saurav Kashyap <skashyap@marvell.com> > Sent: Wednesday, August 25, 2021 10:29 AM > To: Ming Lei <ming.lei@redhat.com> > Cc: Sagi Grimberg <sagi@grimberg.me>; Nilesh Javali <njavali@marvell.com>; > martin.petersen@oracle.com; linux-nvme@lists.infradead.org; linux- > scsi@vger.kernel.org; GR-QLogic-Storage-Upstream <GR-QLogic-Storage- > Upstream@marvell.com> > Subject: RE: [EXT] Re: [PATCH 0/2] qla2xxx - add nvme map_queues support > > Hi Ming, > Comments inline > > > -----Original Message----- > > From: Ming Lei <ming.lei@redhat.com> > > Sent: Tuesday, August 24, 2021 9:17 AM > > To: Saurav Kashyap <skashyap@marvell.com> > > Cc: Sagi Grimberg <sagi@grimberg.me>; Nilesh Javali <njavali@marvell.com>; > > martin.petersen@oracle.com; linux-nvme@lists.infradead.org; linux- > > scsi@vger.kernel.org; GR-QLogic-Storage-Upstream <GR-QLogic-Storage- > > Upstream@marvell.com> > > Subject: [EXT] Re: [PATCH 0/2] qla2xxx - add nvme map_queues support > > > > External Email > > > > ---------------------------------------------------------------------- > > On Tue, Aug 24, 2021 at 03:38:24AM +0000, Saurav Kashyap wrote: > > > Hi Sagi, > > > Comments inline > > > > > > > -----Original Message----- > > > > From: Sagi Grimberg <sagi@grimberg.me> > > > > Sent: Monday, August 23, 2021 10:51 PM > > > > To: Nilesh Javali <njavali@marvell.com>; martin.petersen@oracle.com; > > linux- > > > > nvme@lists.infradead.org; Ming Lei <ming.lei@redhat.com> > > > > Cc: linux-scsi@vger.kernel.org; GR-QLogic-Storage-Upstream <GR-QLogic- > > > > Storage-Upstream@marvell.com> > > > > Subject: Re: [PATCH 0/2] qla2xxx - add nvme map_queues support > > > > > > > > > > > > On 8/23/21 5:56 AM, Nilesh Javali wrote: > > > > > Currently nvme fc doesn't support map queue functionality. This patch > > > > > set adds map_queue functionality to nvme_fc_mq_ops and > > > > > nvme_fc_port_template, providing an option to LLDs to map queues > > > > > similar to SCSI. For qla2xxx, minimum 10% improvement is noticed > > > > > with this change as it helps in reducing cpu thrashing. > > > > > > > > Does this make nvme-fc use managed irq? > > > > > > qla2xxx driver uses pci_alloc_irq_vectors_affinity to have affinity with each > > MSI-X vector. Currently nvme queue are not mapped based on affinity and irq > > offset. The change is to use blk_mq_pci_map_queues for mapping, this > function > > consider irq affinity as well as irq offset. > > > > > > > OK, got it. Even though without this patchset, nvme-fc actually relies > > on managed irq since qla2xxx driver uses pci_alloc_irq_vectors_affinity. > > > > Now the patchset[1] isn't good for addressing the issue in > > blk_mq_alloc_request_hctx(). > > Can you please elaborate on this? Is there something needs to be done from my > side? > > Thanks, > ~Saurav > > > > [1] https://urldefense.proofpoint.com/v2/url?u=https- > > 3A__lore.kernel.org_linux-2Dblock_YR7demOSG6MKFVAF-40T590_T_- > > > 23t&d=DwIBAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=ZHZbmY_LbM3DUZK_BDO1 > > OITP3ot_Vkb_5w- > > > gas5TBMQ&m=CqFDnfAsZphubKXkUx5gsRF6RZ2Qe6sxWkYq4pBfFD0&s=2Nba > > EUI5eB6_R6PxW8ld1Xn2OU3_UdD6D30uvFAWhow&e= > > > > > > Thanks, > > Ming ^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [EXT] Re: [PATCH 0/2] qla2xxx - add nvme map_queues support 2021-09-07 9:18 ` Saurav Kashyap @ 2021-09-14 8:07 ` Saurav Kashyap 2021-09-14 8:32 ` hch 0 siblings, 1 reply; 14+ messages in thread From: Saurav Kashyap @ 2021-09-14 8:07 UTC (permalink / raw) To: Ming Lei, Sagi Grimberg, linux-nvme, hch Cc: Nilesh Javali, martin.petersen, linux-scsi, GR-QLogic-Storage-Upstream, james.smart, axboe, kbusch Hi Sagi/Christoph, I haven't heard anything on this and there are no review comments on this patch set, kindly include this in nvme tree. Thanks, ~Saurav > -----Original Message----- > From: Saurav Kashyap > Sent: Tuesday, September 7, 2021 2:48 PM > To: Ming Lei <ming.lei@redhat.com>; Sagi Grimberg <sagi@grimberg.me>; > linux-nvme@lists.infradead.org > Cc: Nilesh Javali <njavali@marvell.com>; martin.petersen@oracle.com; linux- > scsi@vger.kernel.org; GR-QLogic-Storage-Upstream <GR-QLogic-Storage- > Upstream@marvell.com> > Subject: RE: [EXT] Re: [PATCH 0/2] qla2xxx - add nvme map_queues support > > Hi, > > Can I get a review for this patch set? > > Thanks, > ~Saurav > > > -----Original Message----- > > From: Saurav Kashyap <skashyap@marvell.com> > > Sent: Wednesday, August 25, 2021 10:29 AM > > To: Ming Lei <ming.lei@redhat.com> > > Cc: Sagi Grimberg <sagi@grimberg.me>; Nilesh Javali <njavali@marvell.com>; > > martin.petersen@oracle.com; linux-nvme@lists.infradead.org; linux- > > scsi@vger.kernel.org; GR-QLogic-Storage-Upstream <GR-QLogic-Storage- > > Upstream@marvell.com> > > Subject: RE: [EXT] Re: [PATCH 0/2] qla2xxx - add nvme map_queues support > > > > Hi Ming, > > Comments inline > > > > > -----Original Message----- > > > From: Ming Lei <ming.lei@redhat.com> > > > Sent: Tuesday, August 24, 2021 9:17 AM > > > To: Saurav Kashyap <skashyap@marvell.com> > > > Cc: Sagi Grimberg <sagi@grimberg.me>; Nilesh Javali > <njavali@marvell.com>; > > > martin.petersen@oracle.com; linux-nvme@lists.infradead.org; linux- > > > scsi@vger.kernel.org; GR-QLogic-Storage-Upstream <GR-QLogic-Storage- > > > Upstream@marvell.com> > > > Subject: [EXT] Re: [PATCH 0/2] qla2xxx - add nvme map_queues support > > > > > > External Email > > > > > > ---------------------------------------------------------------------- > > > On Tue, Aug 24, 2021 at 03:38:24AM +0000, Saurav Kashyap wrote: > > > > Hi Sagi, > > > > Comments inline > > > > > > > > > -----Original Message----- > > > > > From: Sagi Grimberg <sagi@grimberg.me> > > > > > Sent: Monday, August 23, 2021 10:51 PM > > > > > To: Nilesh Javali <njavali@marvell.com>; martin.petersen@oracle.com; > > > linux- > > > > > nvme@lists.infradead.org; Ming Lei <ming.lei@redhat.com> > > > > > Cc: linux-scsi@vger.kernel.org; GR-QLogic-Storage-Upstream <GR- > QLogic- > > > > > Storage-Upstream@marvell.com> > > > > > Subject: Re: [PATCH 0/2] qla2xxx - add nvme map_queues support > > > > > > > > > > > > > > > On 8/23/21 5:56 AM, Nilesh Javali wrote: > > > > > > Currently nvme fc doesn't support map queue functionality. This patch > > > > > > set adds map_queue functionality to nvme_fc_mq_ops and > > > > > > nvme_fc_port_template, providing an option to LLDs to map queues > > > > > > similar to SCSI. For qla2xxx, minimum 10% improvement is noticed > > > > > > with this change as it helps in reducing cpu thrashing. > > > > > > > > > > Does this make nvme-fc use managed irq? > > > > > > > > qla2xxx driver uses pci_alloc_irq_vectors_affinity to have affinity with > each > > > MSI-X vector. Currently nvme queue are not mapped based on affinity and > irq > > > offset. The change is to use blk_mq_pci_map_queues for mapping, this > > function > > > consider irq affinity as well as irq offset. > > > > > > > > > > OK, got it. Even though without this patchset, nvme-fc actually relies > > > on managed irq since qla2xxx driver uses pci_alloc_irq_vectors_affinity. > > > > > > Now the patchset[1] isn't good for addressing the issue in > > > blk_mq_alloc_request_hctx(). > > > > Can you please elaborate on this? Is there something needs to be done from > my > > side? > > > > Thanks, > > ~Saurav > > > > > > [1] https://urldefense.proofpoint.com/v2/url?u=https- > > > 3A__lore.kernel.org_linux-2Dblock_YR7demOSG6MKFVAF-40T590_T_- > > > > > > 23t&d=DwIBAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=ZHZbmY_LbM3DUZK_BDO1 > > > OITP3ot_Vkb_5w- > > > > > > gas5TBMQ&m=CqFDnfAsZphubKXkUx5gsRF6RZ2Qe6sxWkYq4pBfFD0&s=2Nba > > > EUI5eB6_R6PxW8ld1Xn2OU3_UdD6D30uvFAWhow&e= > > > > > > > > > Thanks, > > > Ming ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [EXT] Re: [PATCH 0/2] qla2xxx - add nvme map_queues support 2021-09-14 8:07 ` Saurav Kashyap @ 2021-09-14 8:32 ` hch 2021-09-22 4:28 ` Saurav Kashyap 2021-10-13 5:09 ` Saurav Kashyap 0 siblings, 2 replies; 14+ messages in thread From: hch @ 2021-09-14 8:32 UTC (permalink / raw) To: Saurav Kashyap Cc: Ming Lei, Sagi Grimberg, linux-nvme, hch, Nilesh Javali, martin.petersen, linux-scsi, GR-QLogic-Storage-Upstream, james.smart, axboe, kbusch On Tue, Sep 14, 2021 at 08:07:02AM +0000, Saurav Kashyap wrote: > Hi Sagi/Christoph, > > I haven't heard anything on this and there are no review comments on this patch set, kindly include this in nvme tree. > I'll queue this up once the 5.16 tree opens unless James voices any objections. ^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [EXT] Re: [PATCH 0/2] qla2xxx - add nvme map_queues support 2021-09-14 8:32 ` hch @ 2021-09-22 4:28 ` Saurav Kashyap 2021-10-13 5:09 ` Saurav Kashyap 1 sibling, 0 replies; 14+ messages in thread From: Saurav Kashyap @ 2021-09-22 4:28 UTC (permalink / raw) To: hch Cc: Ming Lei, Sagi Grimberg, linux-nvme, Nilesh Javali, martin.petersen, linux-scsi, GR-QLogic-Storage-Upstream, james.smart, axboe, kbusch > On Tue, Sep 14, 2021 at 08:07:02AM +0000, Saurav Kashyap wrote: > > Hi Sagi/Christoph, > > > > I haven't heard anything on this and there are no review comments on this > patch set, kindly include this in nvme tree. > > > > I'll queue this up once the 5.16 tree opens unless James voices any > objections. Sounds good. Thanks, ~Saurav ^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: [EXT] Re: [PATCH 0/2] qla2xxx - add nvme map_queues support 2021-09-14 8:32 ` hch 2021-09-22 4:28 ` Saurav Kashyap @ 2021-10-13 5:09 ` Saurav Kashyap 2021-10-13 5:11 ` hch 1 sibling, 1 reply; 14+ messages in thread From: Saurav Kashyap @ 2021-10-13 5:09 UTC (permalink / raw) To: hch Cc: Ming Lei, Sagi Grimberg, linux-nvme, Nilesh Javali, martin.petersen, linux-scsi, GR-QLogic-Storage-Upstream, james.smart, axboe, kbusch Hi Christoph, > > Hi Sagi/Christoph, > > > > I haven't heard anything on this and there are no review comments on this > patch set, kindly include this in nvme tree. > > > > I'll queue this up once the 5.16 tree opens unless James voices any > objections. Any updates on this one? Thanks, ~Saurav ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [EXT] Re: [PATCH 0/2] qla2xxx - add nvme map_queues support 2021-10-13 5:09 ` Saurav Kashyap @ 2021-10-13 5:11 ` hch 0 siblings, 0 replies; 14+ messages in thread From: hch @ 2021-10-13 5:11 UTC (permalink / raw) To: Saurav Kashyap Cc: hch, Ming Lei, Sagi Grimberg, linux-nvme, Nilesh Javali, martin.petersen, linux-scsi, GR-QLogic-Storage-Upstream, james.smart, axboe, kbusch On Wed, Oct 13, 2021 at 05:09:04AM +0000, Saurav Kashyap wrote: > Hi Christoph, > > > > Hi Sagi/Christoph, > > > > > > I haven't heard anything on this and there are no review comments on this > > patch set, kindly include this in nvme tree. > > > > > > > I'll queue this up once the 5.16 tree opens unless James voices any > > objections. > > Any updates on this one? http://git.infradead.org/nvme.git/commitdiff/062b5291ef352d6198c6c8414c71ae2debd06cf7 ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 0/2] qla2xxx - add nvme map_queues support 2021-08-24 3:46 ` Ming Lei 2021-08-25 4:58 ` [EXT] " Saurav Kashyap @ 2021-09-14 9:31 ` Daniel Wagner 1 sibling, 0 replies; 14+ messages in thread From: Daniel Wagner @ 2021-09-14 9:31 UTC (permalink / raw) To: Ming Lei Cc: Saurav Kashyap, Sagi Grimberg, Nilesh Javali, martin.petersen, linux-nvme, linux-scsi, GR-QLogic-Storage-Upstream On Tue, Aug 24, 2021 at 11:46:32AM +0800, Ming Lei wrote: > OK, got it. Even though without this patchset, nvme-fc actually relies > on managed irq since qla2xxx driver uses pci_alloc_irq_vectors_affinity. > > Now the patchset[1] isn't good for addressing the issue in > blk_mq_alloc_request_hctx(). > > [1] https://lore.kernel.org/linux-block/YR7demOSG6MKFVAF@T590/T/#t Ming, do you have any ideas/suggestions how to address it? I can work on it. ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2021-10-13 5:11 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-08-23 12:56 [PATCH 0/2] qla2xxx - add nvme map_queues support Nilesh Javali 2021-08-23 12:56 ` [PATCH 1/2] nvme-fc: Add support for map_queues Nilesh Javali 2021-08-23 12:56 ` [PATCH 2/2] qla2xxx: Add map_queues support to nvme Nilesh Javali 2021-08-23 17:21 ` [PATCH 0/2] qla2xxx - add nvme map_queues support Sagi Grimberg 2021-08-24 3:38 ` Saurav Kashyap 2021-08-24 3:46 ` Ming Lei 2021-08-25 4:58 ` [EXT] " Saurav Kashyap 2021-09-07 9:18 ` Saurav Kashyap 2021-09-14 8:07 ` Saurav Kashyap 2021-09-14 8:32 ` hch 2021-09-22 4:28 ` Saurav Kashyap 2021-10-13 5:09 ` Saurav Kashyap 2021-10-13 5:11 ` hch 2021-09-14 9:31 ` Daniel Wagner
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).