* use PCI layer IRQ affinity in lpfc
@ 2016-11-17 15:14 Christoph Hellwig
2016-11-17 15:14 ` [PATCH 1/2] lpfc: use pci_irq_alloc_vectors and pci_irq_free_vectors Christoph Hellwig
` (3 more replies)
0 siblings, 4 replies; 11+ messages in thread
From: Christoph Hellwig @ 2016-11-17 15:14 UTC (permalink / raw)
To: james.smart; +Cc: hare, linux-scsi
This series has two patches: the first is a simple conversion of
lpfc to use pci_alloc_irq_vectors. The second is more interesting,
and makes use of the PCI_IRQ_AFFINITY option to use the core interrupt
affinity assignment that takes nodes into account and can be easily
queried. It also ensures we propagate this information to blk-mq
to make sure the block layer queues are properly aligned to the
interrupt vectors.
Note that these patches require core IRQ changes from a stable
branch in the tip tree to be pulled in first:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/for-block
Also be aware that I don't have any lpfc hardware to actually test these.
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 1/2] lpfc: use pci_irq_alloc_vectors and pci_irq_free_vectors
2016-11-17 15:14 use PCI layer IRQ affinity in lpfc Christoph Hellwig
@ 2016-11-17 15:14 ` Christoph Hellwig
2016-11-17 15:14 ` [PATCH 2/2] lpfc: use PCI-layer irq affinity handling Christoph Hellwig
` (2 subsequent siblings)
3 siblings, 0 replies; 11+ messages in thread
From: Christoph Hellwig @ 2016-11-17 15:14 UTC (permalink / raw)
To: james.smart; +Cc: hare, linux-scsi
This avoids having to store the msix_entries array and simpliefies the
shutdown and cleanup path a lot.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
drivers/scsi/lpfc/lpfc.h | 2 -
drivers/scsi/lpfc/lpfc_init.c | 240 ++++++++++--------------------------------
drivers/scsi/lpfc/lpfc_sli4.h | 1 -
3 files changed, 54 insertions(+), 189 deletions(-)
diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h
index 8a20b4e..e4a57cf 100644
--- a/drivers/scsi/lpfc/lpfc.h
+++ b/drivers/scsi/lpfc/lpfc.h
@@ -879,8 +879,6 @@ struct lpfc_hba {
enum intr_type_t intr_type;
uint32_t intr_mode;
#define LPFC_INTR_ERROR 0xFFFFFFFF
- struct msix_entry msix_entries[LPFC_MSIX_VECTORS];
-
struct list_head port_list;
struct lpfc_vport *pport; /* physical lpfc_vport pointer */
uint16_t max_vpi; /* Maximum virtual nports */
diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index 4776fd8..c4e8fad 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -5510,17 +5510,6 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba)
goto out_free_fcf_rr_bmask;
}
- phba->sli4_hba.msix_entries = kzalloc((sizeof(struct msix_entry) *
- (fof_vectors +
- phba->cfg_fcp_io_channel)), GFP_KERNEL);
- if (!phba->sli4_hba.msix_entries) {
- lpfc_printf_log(phba, KERN_ERR, LOG_INIT,
- "2573 Failed allocate memory for msi-x "
- "interrupt vector entries\n");
- rc = -ENOMEM;
- goto out_free_fcp_eq_hdl;
- }
-
phba->sli4_hba.cpu_map = kzalloc((sizeof(struct lpfc_vector_map_info) *
phba->sli4_hba.num_present_cpu),
GFP_KERNEL);
@@ -5529,7 +5518,7 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba)
"3327 Failed allocate memory for msi-x "
"interrupt vector mapping\n");
rc = -ENOMEM;
- goto out_free_msix;
+ goto out_free_fcp_eq_hdl;
}
if (lpfc_used_cpu == NULL) {
lpfc_used_cpu = kzalloc((sizeof(uint16_t) * lpfc_present_cpu),
@@ -5540,7 +5529,7 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba)
"interrupt vector mapping\n");
kfree(phba->sli4_hba.cpu_map);
rc = -ENOMEM;
- goto out_free_msix;
+ goto out_free_fcp_eq_hdl;
}
for (i = 0; i < lpfc_present_cpu; i++)
lpfc_used_cpu[i] = LPFC_VECTOR_MAP_EMPTY;
@@ -5575,8 +5564,6 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba)
return 0;
-out_free_msix:
- kfree(phba->sli4_hba.msix_entries);
out_free_fcp_eq_hdl:
kfree(phba->sli4_hba.fcp_eq_hdl);
out_free_fcf_rr_bmask:
@@ -5612,9 +5599,6 @@ lpfc_sli4_driver_resource_unset(struct lpfc_hba *phba)
phba->sli4_hba.num_online_cpu = 0;
phba->sli4_hba.curr_disp_cpu = 0;
- /* Free memory allocated for msi-x interrupt vector entries */
- kfree(phba->sli4_hba.msix_entries);
-
/* Free memory allocated for fast-path work queue handles */
kfree(phba->sli4_hba.fcp_eq_hdl);
@@ -8485,16 +8469,7 @@ lpfc_sli4_pci_mem_unset(struct lpfc_hba *phba)
* @phba: pointer to lpfc hba data structure.
*
* This routine is invoked to enable the MSI-X interrupt vectors to device
- * with SLI-3 interface specs. The kernel function pci_enable_msix_exact()
- * is called to enable the MSI-X vectors. Note that pci_enable_msix_exact(),
- * once invoked, enables either all or nothing, depending on the current
- * availability of PCI vector resources. The device driver is responsible
- * for calling the individual request_irq() to register each MSI-X vector
- * with a interrupt handler, which is done in this function. Note that
- * later when device is unloading, the driver should always call free_irq()
- * on all MSI-X vectors it has done request_irq() on before calling
- * pci_disable_msix(). Failure to do so results in a BUG_ON() and a device
- * will be left with MSI-X enabled and leaks its vectors.
+ * with SLI-3 interface specs.
*
* Return codes
* 0 - successful
@@ -8503,33 +8478,24 @@ lpfc_sli4_pci_mem_unset(struct lpfc_hba *phba)
static int
lpfc_sli_enable_msix(struct lpfc_hba *phba)
{
- int rc, i;
+ int rc;
LPFC_MBOXQ_t *pmb;
/* Set up MSI-X multi-message vectors */
- for (i = 0; i < LPFC_MSIX_VECTORS; i++)
- phba->msix_entries[i].entry = i;
-
- /* Configure MSI-X capability structure */
- rc = pci_enable_msix_exact(phba->pcidev, phba->msix_entries,
- LPFC_MSIX_VECTORS);
- if (rc) {
+ rc = pci_alloc_irq_vectors(phba->pcidev,
+ LPFC_MSIX_VECTORS, LPFC_MSIX_VECTORS, PCI_IRQ_MSIX);
+ if (rc < 0) {
lpfc_printf_log(phba, KERN_INFO, LOG_INIT,
"0420 PCI enable MSI-X failed (%d)\n", rc);
goto vec_fail_out;
}
- for (i = 0; i < LPFC_MSIX_VECTORS; i++)
- lpfc_printf_log(phba, KERN_INFO, LOG_INIT,
- "0477 MSI-X entry[%d]: vector=x%x "
- "message=%d\n", i,
- phba->msix_entries[i].vector,
- phba->msix_entries[i].entry);
+
/*
* Assign MSI-X vectors to interrupt handlers
*/
/* vector-0 is associated to slow-path handler */
- rc = request_irq(phba->msix_entries[0].vector,
+ rc = request_irq(pci_irq_vector(phba->pcidev, 0),
&lpfc_sli_sp_intr_handler, 0,
LPFC_SP_DRIVER_HANDLER_NAME, phba);
if (rc) {
@@ -8540,7 +8506,7 @@ lpfc_sli_enable_msix(struct lpfc_hba *phba)
}
/* vector-1 is associated to fast-path handler */
- rc = request_irq(phba->msix_entries[1].vector,
+ rc = request_irq(pci_irq_vector(phba->pcidev, 1),
&lpfc_sli_fp_intr_handler, 0,
LPFC_FP_DRIVER_HANDLER_NAME, phba);
@@ -8585,42 +8551,21 @@ lpfc_sli_enable_msix(struct lpfc_hba *phba)
mem_fail_out:
/* free the irq already requested */
- free_irq(phba->msix_entries[1].vector, phba);
+ free_irq(pci_irq_vector(phba->pcidev, 1), phba);
irq_fail_out:
/* free the irq already requested */
- free_irq(phba->msix_entries[0].vector, phba);
+ free_irq(pci_irq_vector(phba->pcidev, 0), phba);
msi_fail_out:
/* Unconfigure MSI-X capability structure */
- pci_disable_msix(phba->pcidev);
+ pci_free_irq_vectors(phba->pcidev);
vec_fail_out:
return rc;
}
/**
- * lpfc_sli_disable_msix - Disable MSI-X interrupt mode on SLI-3 device.
- * @phba: pointer to lpfc hba data structure.
- *
- * This routine is invoked to release the MSI-X vectors and then disable the
- * MSI-X interrupt mode to device with SLI-3 interface spec.
- **/
-static void
-lpfc_sli_disable_msix(struct lpfc_hba *phba)
-{
- int i;
-
- /* Free up MSI-X multi-message vectors */
- for (i = 0; i < LPFC_MSIX_VECTORS; i++)
- free_irq(phba->msix_entries[i].vector, phba);
- /* Disable MSI-X */
- pci_disable_msix(phba->pcidev);
-
- return;
-}
-
-/**
* lpfc_sli_enable_msi - Enable MSI interrupt mode on SLI-3 device.
* @phba: pointer to lpfc hba data structure.
*
@@ -8660,24 +8605,6 @@ lpfc_sli_enable_msi(struct lpfc_hba *phba)
}
/**
- * lpfc_sli_disable_msi - Disable MSI interrupt mode to SLI-3 device.
- * @phba: pointer to lpfc hba data structure.
- *
- * This routine is invoked to disable the MSI interrupt mode to device with
- * SLI-3 interface spec. The driver calls free_irq() on MSI vector it has
- * done request_irq() on before calling pci_disable_msi(). Failure to do so
- * results in a BUG_ON() and a device will be left with MSI enabled and leaks
- * its vector.
- */
-static void
-lpfc_sli_disable_msi(struct lpfc_hba *phba)
-{
- free_irq(phba->pcidev->irq, phba);
- pci_disable_msi(phba->pcidev);
- return;
-}
-
-/**
* lpfc_sli_enable_intr - Enable device interrupt to SLI-3 device.
* @phba: pointer to lpfc hba data structure.
*
@@ -8748,19 +8675,20 @@ lpfc_sli_enable_intr(struct lpfc_hba *phba, uint32_t cfg_mode)
static void
lpfc_sli_disable_intr(struct lpfc_hba *phba)
{
- /* Disable the currently initialized interrupt mode */
+ int nr_irqs, i;
+
if (phba->intr_type == MSIX)
- lpfc_sli_disable_msix(phba);
- else if (phba->intr_type == MSI)
- lpfc_sli_disable_msi(phba);
- else if (phba->intr_type == INTx)
- free_irq(phba->pcidev->irq, phba);
+ nr_irqs = LPFC_MSIX_VECTORS;
+ else
+ nr_irqs = 1;
+
+ for (i = 0; i < nr_irqs; i++)
+ free_irq(pci_irq_vector(phba->pcidev, i), phba);
+ pci_free_irq_vectors(phba->pcidev);
/* Reset interrupt management states */
phba->intr_type = NONE;
phba->sli.slistat.sli_intr = 0;
-
- return;
}
/**
@@ -8916,7 +8844,7 @@ lpfc_sli4_set_affinity(struct lpfc_hba *phba, int vectors)
lpfc_used_cpu[cpu] = phys_id;
/* Associate vector with selected CPU */
- cpup->irq = phba->sli4_hba.msix_entries[idx].vector;
+ cpup->irq = pci_irq_vector(phba->pcidev, idx);
/* Associate IO channel with selected CPU */
cpup->channel_id = idx;
@@ -8926,14 +8854,14 @@ lpfc_sli4_set_affinity(struct lpfc_hba *phba, int vectors)
first_cpu = cpu;
/* Now affinitize to the selected CPU */
- i = irq_set_affinity_hint(phba->sli4_hba.msix_entries[idx].
- vector, get_cpu_mask(cpu));
+ i = irq_set_affinity_hint(pci_irq_vector(phba->pcidev, idx),
+ get_cpu_mask(cpu));
lpfc_printf_log(phba, KERN_INFO, LOG_INIT,
"3330 Set Affinity: CPU %d channel %d "
"irq %d (%x)\n",
cpu, cpup->channel_id,
- phba->sli4_hba.msix_entries[idx].vector, i);
+ pci_irq_vector(phba->pcidev, idx), i);
/* Spread vector mapping across multple physical CPU nodes */
phys_id++;
@@ -9048,14 +8976,7 @@ lpfc_sli4_set_affinity(struct lpfc_hba *phba, int vectors)
* @phba: pointer to lpfc hba data structure.
*
* This routine is invoked to enable the MSI-X interrupt vectors to device
- * with SLI-4 interface spec. The kernel function pci_enable_msix_range()
- * is called to enable the MSI-X vectors. The device driver is responsible
- * for calling the individual request_irq() to register each MSI-X vector
- * with a interrupt handler, which is done in this function. Note that
- * later when device is unloading, the driver should always call free_irq()
- * on all MSI-X vectors it has done request_irq() on before calling
- * pci_disable_msix(). Failure to do so results in a BUG_ON() and a device
- * will be left with MSI-X enabled and leaks its vectors.
+ * with SLI-4 interface spec.
*
* Return codes
* 0 - successful
@@ -9067,17 +8988,11 @@ lpfc_sli4_enable_msix(struct lpfc_hba *phba)
int vectors, rc, index;
/* Set up MSI-X multi-message vectors */
- for (index = 0; index < phba->cfg_fcp_io_channel; index++)
- phba->sli4_hba.msix_entries[index].entry = index;
-
- /* Configure MSI-X capability structure */
vectors = phba->cfg_fcp_io_channel;
- if (phba->cfg_fof) {
- phba->sli4_hba.msix_entries[index].entry = index;
+ if (phba->cfg_fof)
vectors++;
- }
- rc = pci_enable_msix_range(phba->pcidev, phba->sli4_hba.msix_entries,
- 2, vectors);
+
+ rc = pci_alloc_irq_vectors(phba->pcidev, 2, vectors, PCI_IRQ_MSIX);
if (rc < 0) {
lpfc_printf_log(phba, KERN_INFO, LOG_INIT,
"0484 PCI enable MSI-X failed (%d)\n", rc);
@@ -9085,14 +9000,6 @@ lpfc_sli4_enable_msix(struct lpfc_hba *phba)
}
vectors = rc;
- /* Log MSI-X vector assignment */
- for (index = 0; index < vectors; index++)
- lpfc_printf_log(phba, KERN_INFO, LOG_INIT,
- "0489 MSI-X entry[%d]: vector=x%x "
- "message=%d\n", index,
- phba->sli4_hba.msix_entries[index].vector,
- phba->sli4_hba.msix_entries[index].entry);
-
/* Assign MSI-X vectors to interrupt handlers */
for (index = 0; index < vectors; index++) {
memset(&phba->sli4_hba.handler_name[index], 0, 16);
@@ -9104,14 +9011,12 @@ lpfc_sli4_enable_msix(struct lpfc_hba *phba)
phba->sli4_hba.fcp_eq_hdl[index].phba = phba;
atomic_set(&phba->sli4_hba.fcp_eq_hdl[index].fcp_eq_in_use, 1);
if (phba->cfg_fof && (index == (vectors - 1)))
- rc = request_irq(
- phba->sli4_hba.msix_entries[index].vector,
+ rc = request_irq(pci_irq_vector(phba->pcidev, index),
&lpfc_sli4_fof_intr_handler, 0,
(char *)&phba->sli4_hba.handler_name[index],
&phba->sli4_hba.fcp_eq_hdl[index]);
else
- rc = request_irq(
- phba->sli4_hba.msix_entries[index].vector,
+ rc = request_irq(pci_irq_vector(phba->pcidev, index),
&lpfc_sli4_hba_intr_handler, 0,
(char *)&phba->sli4_hba.handler_name[index],
&phba->sli4_hba.fcp_eq_hdl[index]);
@@ -9141,49 +9046,20 @@ lpfc_sli4_enable_msix(struct lpfc_hba *phba)
cfg_fail_out:
/* free the irq already requested */
for (--index; index >= 0; index--) {
- irq_set_affinity_hint(phba->sli4_hba.msix_entries[index].
- vector, NULL);
- free_irq(phba->sli4_hba.msix_entries[index].vector,
- &phba->sli4_hba.fcp_eq_hdl[index]);
+ int irq = pci_irq_vector(phba->pcidev, index);
+
+ irq_set_affinity_hint(irq, NULL);
+ free_irq(irq, &phba->sli4_hba.fcp_eq_hdl[index]);
}
/* Unconfigure MSI-X capability structure */
- pci_disable_msix(phba->pcidev);
+ pci_free_irq_vectors(phba->pcidev);
vec_fail_out:
return rc;
}
/**
- * lpfc_sli4_disable_msix - Disable MSI-X interrupt mode to SLI-4 device
- * @phba: pointer to lpfc hba data structure.
- *
- * This routine is invoked to release the MSI-X vectors and then disable the
- * MSI-X interrupt mode to device with SLI-4 interface spec.
- **/
-static void
-lpfc_sli4_disable_msix(struct lpfc_hba *phba)
-{
- int index;
-
- /* Free up MSI-X multi-message vectors */
- for (index = 0; index < phba->cfg_fcp_io_channel; index++) {
- irq_set_affinity_hint(phba->sli4_hba.msix_entries[index].
- vector, NULL);
- free_irq(phba->sli4_hba.msix_entries[index].vector,
- &phba->sli4_hba.fcp_eq_hdl[index]);
- }
- if (phba->cfg_fof) {
- free_irq(phba->sli4_hba.msix_entries[index].vector,
- &phba->sli4_hba.fcp_eq_hdl[index]);
- }
- /* Disable MSI-X */
- pci_disable_msix(phba->pcidev);
-
- return;
-}
-
-/**
* lpfc_sli4_enable_msi - Enable MSI interrupt mode to SLI-4 device
* @phba: pointer to lpfc hba data structure.
*
@@ -9234,24 +9110,6 @@ lpfc_sli4_enable_msi(struct lpfc_hba *phba)
}
/**
- * lpfc_sli4_disable_msi - Disable MSI interrupt mode to SLI-4 device
- * @phba: pointer to lpfc hba data structure.
- *
- * This routine is invoked to disable the MSI interrupt mode to device with
- * SLI-4 interface spec. The driver calls free_irq() on MSI vector it has
- * done request_irq() on before calling pci_disable_msi(). Failure to do so
- * results in a BUG_ON() and a device will be left with MSI enabled and leaks
- * its vector.
- **/
-static void
-lpfc_sli4_disable_msi(struct lpfc_hba *phba)
-{
- free_irq(phba->pcidev->irq, phba);
- pci_disable_msi(phba->pcidev);
- return;
-}
-
-/**
* lpfc_sli4_enable_intr - Enable device interrupt to SLI-4 device
* @phba: pointer to lpfc hba data structure.
*
@@ -9336,18 +9194,28 @@ static void
lpfc_sli4_disable_intr(struct lpfc_hba *phba)
{
/* Disable the currently initialized interrupt mode */
- if (phba->intr_type == MSIX)
- lpfc_sli4_disable_msix(phba);
- else if (phba->intr_type == MSI)
- lpfc_sli4_disable_msi(phba);
- else if (phba->intr_type == INTx)
+ if (phba->intr_type == MSIX) {
+ int index;
+
+ /* Free up MSI-X multi-message vectors */
+ for (index = 0; index < phba->cfg_fcp_io_channel; index++) {
+ int irq = pci_irq_vector(phba->pcidev, index);
+
+ irq_set_affinity_hint(irq, NULL);
+ free_irq(irq, &phba->sli4_hba.fcp_eq_hdl[index]);
+ }
+
+ if (phba->cfg_fof)
+ free_irq(pci_irq_vector(phba->pcidev, index), phba);
+ } else {
free_irq(phba->pcidev->irq, phba);
+ }
+
+ pci_free_irq_vectors(phba->pcidev);
/* Reset interrupt management states */
phba->intr_type = NONE;
phba->sli.slistat.sli_intr = 0;
-
- return;
}
/**
diff --git a/drivers/scsi/lpfc/lpfc_sli4.h b/drivers/scsi/lpfc/lpfc_sli4.h
index 0b88b570..dfbb25e 100644
--- a/drivers/scsi/lpfc/lpfc_sli4.h
+++ b/drivers/scsi/lpfc/lpfc_sli4.h
@@ -515,7 +515,6 @@ struct lpfc_sli4_hba {
uint32_t ue_to_rp;
struct lpfc_register sli_intf;
struct lpfc_pc_sli4_params pc_sli4_params;
- struct msix_entry *msix_entries;
uint8_t handler_name[LPFC_SLI4_HANDLER_CNT][LPFC_SLI4_HANDLER_NAME_SZ];
struct lpfc_fcp_eq_hdl *fcp_eq_hdl; /* FCP per-WQ handle */
--
2.1.4
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH 2/2] lpfc: use PCI-layer irq affinity handling
2016-11-17 15:14 use PCI layer IRQ affinity in lpfc Christoph Hellwig
2016-11-17 15:14 ` [PATCH 1/2] lpfc: use pci_irq_alloc_vectors and pci_irq_free_vectors Christoph Hellwig
@ 2016-11-17 15:14 ` Christoph Hellwig
2016-11-18 13:13 ` use PCI layer IRQ affinity in lpfc Johannes Thumshirn
2016-11-18 16:21 ` James Smart
3 siblings, 0 replies; 11+ messages in thread
From: Christoph Hellwig @ 2016-11-17 15:14 UTC (permalink / raw)
To: james.smart; +Cc: hare, linux-scsi
Use the PCI_IRQ_AFFINITY flags to pci_alloc_irq_vectors to get automatic
assignment of irq affinity from the core PCI and interrupt handling code.
For blk-mq we just have to wire it up to the default map_queues handler,
and for the non blk-mq case we keep a local copy of the map_queues helper,
which operates on the lpfc-internal lpfc_hba structure.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
drivers/scsi/lpfc/lpfc.h | 3 -
drivers/scsi/lpfc/lpfc_attr.c | 168 --------------------
drivers/scsi/lpfc/lpfc_init.c | 346 +++---------------------------------------
drivers/scsi/lpfc/lpfc_scsi.c | 35 ++---
drivers/scsi/lpfc/lpfc_sli4.h | 16 +-
5 files changed, 38 insertions(+), 530 deletions(-)
diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h
index e4a57cf..8b205a2 100644
--- a/drivers/scsi/lpfc/lpfc.h
+++ b/drivers/scsi/lpfc/lpfc.h
@@ -711,7 +711,6 @@ struct lpfc_hba {
#define LPFC_FCF_FOV 1 /* Fast fcf failover */
#define LPFC_FCF_PRIORITY 2 /* Priority fcf failover */
uint32_t cfg_fcf_failover_policy;
- uint32_t cfg_fcp_io_sched;
uint32_t cfg_fcp2_no_tgt_reset;
uint32_t cfg_cr_delay;
uint32_t cfg_cr_count;
@@ -789,8 +788,6 @@ struct lpfc_hba {
uint32_t hbq_count; /* Count of configured HBQs */
struct hbq_s hbqs[LPFC_MAX_HBQS]; /* local copy of hbq indicies */
- atomic_t fcp_qidx; /* next work queue to post work to */
-
phys_addr_t pci_bar0_map; /* Physical address for PCI BAR0 */
phys_addr_t pci_bar1_map; /* Physical address for PCI BAR1 */
phys_addr_t pci_bar2_map; /* Physical address for PCI BAR2 */
diff --git a/drivers/scsi/lpfc/lpfc_attr.c b/drivers/scsi/lpfc/lpfc_attr.c
index c847755..b4825bd 100644
--- a/drivers/scsi/lpfc/lpfc_attr.c
+++ b/drivers/scsi/lpfc/lpfc_attr.c
@@ -4189,157 +4189,6 @@ lpfc_fcp_imax_init(struct lpfc_hba *phba, int val)
static DEVICE_ATTR(lpfc_fcp_imax, S_IRUGO | S_IWUSR,
lpfc_fcp_imax_show, lpfc_fcp_imax_store);
-/**
- * lpfc_state_show - Display current driver CPU affinity
- * @dev: class converted to a Scsi_host structure.
- * @attr: device attribute, not used.
- * @buf: on return contains text describing the state of the link.
- *
- * Returns: size of formatted string.
- **/
-static ssize_t
-lpfc_fcp_cpu_map_show(struct device *dev, struct device_attribute *attr,
- char *buf)
-{
- struct Scsi_Host *shost = class_to_shost(dev);
- struct lpfc_vport *vport = (struct lpfc_vport *)shost->hostdata;
- struct lpfc_hba *phba = vport->phba;
- struct lpfc_vector_map_info *cpup;
- int len = 0;
-
- if ((phba->sli_rev != LPFC_SLI_REV4) ||
- (phba->intr_type != MSIX))
- return len;
-
- switch (phba->cfg_fcp_cpu_map) {
- case 0:
- len += snprintf(buf + len, PAGE_SIZE-len,
- "fcp_cpu_map: No mapping (%d)\n",
- phba->cfg_fcp_cpu_map);
- return len;
- case 1:
- len += snprintf(buf + len, PAGE_SIZE-len,
- "fcp_cpu_map: HBA centric mapping (%d): "
- "%d online CPUs\n",
- phba->cfg_fcp_cpu_map,
- phba->sli4_hba.num_online_cpu);
- break;
- case 2:
- len += snprintf(buf + len, PAGE_SIZE-len,
- "fcp_cpu_map: Driver centric mapping (%d): "
- "%d online CPUs\n",
- phba->cfg_fcp_cpu_map,
- phba->sli4_hba.num_online_cpu);
- break;
- }
-
- while (phba->sli4_hba.curr_disp_cpu < phba->sli4_hba.num_present_cpu) {
- cpup = &phba->sli4_hba.cpu_map[phba->sli4_hba.curr_disp_cpu];
-
- /* margin should fit in this and the truncated message */
- if (cpup->irq == LPFC_VECTOR_MAP_EMPTY)
- len += snprintf(buf + len, PAGE_SIZE-len,
- "CPU %02d io_chan %02d "
- "physid %d coreid %d\n",
- phba->sli4_hba.curr_disp_cpu,
- cpup->channel_id, cpup->phys_id,
- cpup->core_id);
- else
- len += snprintf(buf + len, PAGE_SIZE-len,
- "CPU %02d io_chan %02d "
- "physid %d coreid %d IRQ %d\n",
- phba->sli4_hba.curr_disp_cpu,
- cpup->channel_id, cpup->phys_id,
- cpup->core_id, cpup->irq);
-
- phba->sli4_hba.curr_disp_cpu++;
-
- /* display max number of CPUs keeping some margin */
- if (phba->sli4_hba.curr_disp_cpu <
- phba->sli4_hba.num_present_cpu &&
- (len >= (PAGE_SIZE - 64))) {
- len += snprintf(buf + len, PAGE_SIZE-len, "more...\n");
- break;
- }
- }
-
- if (phba->sli4_hba.curr_disp_cpu == phba->sli4_hba.num_present_cpu)
- phba->sli4_hba.curr_disp_cpu = 0;
-
- return len;
-}
-
-/**
- * lpfc_fcp_cpu_map_store - Change CPU affinity of driver vectors
- * @dev: class device that is converted into a Scsi_host.
- * @attr: device attribute, not used.
- * @buf: one or more lpfc_polling_flags values.
- * @count: not used.
- *
- * Returns:
- * -EINVAL - Not implemented yet.
- **/
-static ssize_t
-lpfc_fcp_cpu_map_store(struct device *dev, struct device_attribute *attr,
- const char *buf, size_t count)
-{
- int status = -EINVAL;
- return status;
-}
-
-/*
-# lpfc_fcp_cpu_map: Defines how to map CPUs to IRQ vectors
-# for the HBA.
-#
-# Value range is [0 to 2]. Default value is LPFC_DRIVER_CPU_MAP (2).
-# 0 - Do not affinitze IRQ vectors
-# 1 - Affintize HBA vectors with respect to each HBA
-# (start with CPU0 for each HBA)
-# 2 - Affintize HBA vectors with respect to the entire driver
-# (round robin thru all CPUs across all HBAs)
-*/
-static int lpfc_fcp_cpu_map = LPFC_DRIVER_CPU_MAP;
-module_param(lpfc_fcp_cpu_map, int, S_IRUGO|S_IWUSR);
-MODULE_PARM_DESC(lpfc_fcp_cpu_map,
- "Defines how to map CPUs to IRQ vectors per HBA");
-
-/**
- * lpfc_fcp_cpu_map_init - Set the initial sr-iov virtual function enable
- * @phba: lpfc_hba pointer.
- * @val: link speed value.
- *
- * Description:
- * If val is in a valid range [0-2], then affinitze the adapter's
- * MSIX vectors.
- *
- * Returns:
- * zero if val saved.
- * -EINVAL val out of range
- **/
-static int
-lpfc_fcp_cpu_map_init(struct lpfc_hba *phba, int val)
-{
- if (phba->sli_rev != LPFC_SLI_REV4) {
- phba->cfg_fcp_cpu_map = 0;
- return 0;
- }
-
- if (val >= LPFC_MIN_CPU_MAP && val <= LPFC_MAX_CPU_MAP) {
- phba->cfg_fcp_cpu_map = val;
- return 0;
- }
-
- lpfc_printf_log(phba, KERN_ERR, LOG_INIT,
- "3326 lpfc_fcp_cpu_map: %d out of range, using "
- "default\n", val);
- phba->cfg_fcp_cpu_map = LPFC_DRIVER_CPU_MAP;
-
- return 0;
-}
-
-static DEVICE_ATTR(lpfc_fcp_cpu_map, S_IRUGO | S_IWUSR,
- lpfc_fcp_cpu_map_show, lpfc_fcp_cpu_map_store);
-
/*
# lpfc_fcp_class: Determines FC class to use for the FCP protocol.
# Value range is [2,3]. Default value is 3.
@@ -4409,19 +4258,6 @@ static DEVICE_ATTR(lpfc_max_scsicmpl_time, S_IRUGO | S_IWUSR,
LPFC_ATTR_R(ack0, 0, 0, 1, "Enable ACK0 support");
/*
-# lpfc_fcp_io_sched: Determine scheduling algrithmn for issuing FCP cmds
-# range is [0,1]. Default value is 0.
-# For [0], FCP commands are issued to Work Queues ina round robin fashion.
-# For [1], FCP commands are issued to a Work Queue associated with the
-# current CPU.
-# It would be set to 1 by the driver if it's able to set up cpu affinity
-# for FCP I/Os through Work Queue associated with the current CPU. Otherwise,
-# roundrobin scheduling of FCP I/Os through WQs will be used.
-*/
-LPFC_ATTR_RW(fcp_io_sched, 0, 0, 1, "Determine scheduling algorithm for "
- "issuing commands [0] - Round Robin, [1] - Current CPU");
-
-/*
# lpfc_fcp2_no_tgt_reset: Determine bus reset behavior
# range is [0,1]. Default value is 0.
# For [0], bus reset issues target reset to ALL devices
@@ -4719,7 +4555,6 @@ struct device_attribute *lpfc_hba_attrs[] = {
&dev_attr_lpfc_topology,
&dev_attr_lpfc_scan_down,
&dev_attr_lpfc_link_speed,
- &dev_attr_lpfc_fcp_io_sched,
&dev_attr_lpfc_fcp2_no_tgt_reset,
&dev_attr_lpfc_cr_delay,
&dev_attr_lpfc_cr_count,
@@ -4747,7 +4582,6 @@ struct device_attribute *lpfc_hba_attrs[] = {
&dev_attr_lpfc_task_mgmt_tmo,
&dev_attr_lpfc_use_msi,
&dev_attr_lpfc_fcp_imax,
- &dev_attr_lpfc_fcp_cpu_map,
&dev_attr_lpfc_fcp_io_channel,
&dev_attr_lpfc_enable_bg,
&dev_attr_lpfc_soft_wwnn,
@@ -5724,7 +5558,6 @@ struct fc_function_template lpfc_vport_transport_functions = {
void
lpfc_get_cfgparam(struct lpfc_hba *phba)
{
- lpfc_fcp_io_sched_init(phba, lpfc_fcp_io_sched);
lpfc_fcp2_no_tgt_reset_init(phba, lpfc_fcp2_no_tgt_reset);
lpfc_cr_delay_init(phba, lpfc_cr_delay);
lpfc_cr_count_init(phba, lpfc_cr_count);
@@ -5743,7 +5576,6 @@ lpfc_get_cfgparam(struct lpfc_hba *phba)
lpfc_enable_SmartSAN_init(phba, lpfc_enable_SmartSAN);
lpfc_use_msi_init(phba, lpfc_use_msi);
lpfc_fcp_imax_init(phba, lpfc_fcp_imax);
- lpfc_fcp_cpu_map_init(phba, lpfc_fcp_cpu_map);
lpfc_fcp_io_channel_init(phba, lpfc_fcp_io_channel);
lpfc_enable_hba_reset_init(phba, lpfc_enable_hba_reset);
lpfc_enable_hba_heartbeat_init(phba, lpfc_enable_hba_heartbeat);
diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index c4e8fad..0202d79 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -60,10 +60,6 @@ char *_dump_buf_dif;
unsigned long _dump_buf_dif_order;
spinlock_t _dump_buf_lock;
-/* Used when mapping IRQ vectors in a driver centric manner */
-uint16_t *lpfc_used_cpu;
-uint32_t lpfc_present_cpu;
-
static void lpfc_get_hba_model_desc(struct lpfc_hba *, uint8_t *, uint8_t *);
static int lpfc_post_rcv_buf(struct lpfc_hba *);
static int lpfc_sli4_queue_verify(struct lpfc_hba *);
@@ -5175,7 +5171,6 @@ lpfc_sli_driver_resource_unset(struct lpfc_hba *phba)
static int
lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba)
{
- struct lpfc_vector_map_info *cpup;
struct lpfc_sli *psli;
LPFC_MBOXQ_t *mboxq;
int rc, i, hbq_count, max_buf_size;
@@ -5510,40 +5505,15 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba)
goto out_free_fcf_rr_bmask;
}
- phba->sli4_hba.cpu_map = kzalloc((sizeof(struct lpfc_vector_map_info) *
- phba->sli4_hba.num_present_cpu),
- GFP_KERNEL);
- if (!phba->sli4_hba.cpu_map) {
+ phba->sli4_hba.channel_map = kcalloc(nr_cpu_ids,
+ sizeof(*phba->sli4_hba.channel_map), GFP_KERNEL);
+ if (!phba->sli4_hba.channel_map) {
lpfc_printf_log(phba, KERN_ERR, LOG_INIT,
"3327 Failed allocate memory for msi-x "
"interrupt vector mapping\n");
rc = -ENOMEM;
goto out_free_fcp_eq_hdl;
}
- if (lpfc_used_cpu == NULL) {
- lpfc_used_cpu = kzalloc((sizeof(uint16_t) * lpfc_present_cpu),
- GFP_KERNEL);
- if (!lpfc_used_cpu) {
- lpfc_printf_log(phba, KERN_ERR, LOG_INIT,
- "3335 Failed allocate memory for msi-x "
- "interrupt vector mapping\n");
- kfree(phba->sli4_hba.cpu_map);
- rc = -ENOMEM;
- goto out_free_fcp_eq_hdl;
- }
- for (i = 0; i < lpfc_present_cpu; i++)
- lpfc_used_cpu[i] = LPFC_VECTOR_MAP_EMPTY;
- }
-
- /* Initialize io channels for round robin */
- cpup = phba->sli4_hba.cpu_map;
- rc = 0;
- for (i = 0; i < phba->sli4_hba.num_present_cpu; i++) {
- cpup->channel_id = rc;
- rc++;
- if (rc >= phba->cfg_fcp_io_channel)
- rc = 0;
- }
/*
* Enable sr-iov virtual functions if supported and configured
@@ -5594,10 +5564,7 @@ lpfc_sli4_driver_resource_unset(struct lpfc_hba *phba)
struct lpfc_fcf_conn_entry *conn_entry, *next_conn_entry;
/* Free memory allocated for msi-x interrupt vector to CPU mapping */
- kfree(phba->sli4_hba.cpu_map);
- phba->sli4_hba.num_present_cpu = 0;
- phba->sli4_hba.num_online_cpu = 0;
- phba->sli4_hba.curr_disp_cpu = 0;
+ kfree(phba->sli4_hba.channel_map);
/* Free memory allocated for fast-path work queue handles */
kfree(phba->sli4_hba.fcp_eq_hdl);
@@ -7228,9 +7195,6 @@ lpfc_sli4_queue_verify(struct lpfc_hba *phba)
if (cpu_online(cpu))
i++;
}
- phba->sli4_hba.num_online_cpu = i;
- phba->sli4_hba.num_present_cpu = lpfc_present_cpu;
- phba->sli4_hba.curr_disp_cpu = 0;
if (i < cfg_fcp_io_channel) {
lpfc_printf_log(phba,
@@ -8691,286 +8655,28 @@ lpfc_sli_disable_intr(struct lpfc_hba *phba)
phba->sli.slistat.sli_intr = 0;
}
-/**
- * lpfc_find_next_cpu - Find next available CPU that matches the phys_id
- * @phba: pointer to lpfc hba data structure.
- *
- * Find next available CPU to use for IRQ to CPU affinity.
- */
-static int
-lpfc_find_next_cpu(struct lpfc_hba *phba, uint32_t phys_id)
-{
- struct lpfc_vector_map_info *cpup;
- int cpu;
-
- cpup = phba->sli4_hba.cpu_map;
- for (cpu = 0; cpu < phba->sli4_hba.num_present_cpu; cpu++) {
- /* CPU must be online */
- if (cpu_online(cpu)) {
- if ((cpup->irq == LPFC_VECTOR_MAP_EMPTY) &&
- (lpfc_used_cpu[cpu] == LPFC_VECTOR_MAP_EMPTY) &&
- (cpup->phys_id == phys_id)) {
- return cpu;
- }
- }
- cpup++;
- }
-
- /*
- * If we get here, we have used ALL CPUs for the specific
- * phys_id. Now we need to clear out lpfc_used_cpu and start
- * reusing CPUs.
- */
-
- for (cpu = 0; cpu < phba->sli4_hba.num_present_cpu; cpu++) {
- if (lpfc_used_cpu[cpu] == phys_id)
- lpfc_used_cpu[cpu] = LPFC_VECTOR_MAP_EMPTY;
- }
-
- cpup = phba->sli4_hba.cpu_map;
- for (cpu = 0; cpu < phba->sli4_hba.num_present_cpu; cpu++) {
- /* CPU must be online */
- if (cpu_online(cpu)) {
- if ((cpup->irq == LPFC_VECTOR_MAP_EMPTY) &&
- (cpup->phys_id == phys_id)) {
- return cpu;
- }
- }
- cpup++;
- }
- return LPFC_VECTOR_MAP_EMPTY;
-}
-
-/**
- * lpfc_sli4_set_affinity - Set affinity for HBA IRQ vectors
- * @phba: pointer to lpfc hba data structure.
- * @vectors: number of HBA vectors
- *
- * Affinitize MSIX IRQ vectors to CPUs. Try to equally spread vector
- * affinization across multple physical CPUs (numa nodes).
- * In addition, this routine will assign an IO channel for each CPU
- * to use when issuing I/Os.
- */
static int
lpfc_sli4_set_affinity(struct lpfc_hba *phba, int vectors)
{
- int i, idx, saved_chann, used_chann, cpu, phys_id;
- int max_phys_id, min_phys_id;
- int num_io_channel, first_cpu, chan;
- struct lpfc_vector_map_info *cpup;
-#ifdef CONFIG_X86
- struct cpuinfo_x86 *cpuinfo;
-#endif
- uint8_t chann[LPFC_FCP_IO_CHAN_MAX+1];
+ const struct cpumask *mask;
+ int cpu, i;
/* If there is no mapping, just return */
if (!phba->cfg_fcp_cpu_map)
return 1;
- /* Init cpu_map array */
- memset(phba->sli4_hba.cpu_map, 0xff,
- (sizeof(struct lpfc_vector_map_info) *
- phba->sli4_hba.num_present_cpu));
-
- max_phys_id = 0;
- min_phys_id = 0xff;
- phys_id = 0;
- num_io_channel = 0;
- first_cpu = LPFC_VECTOR_MAP_EMPTY;
-
- /* Update CPU map with physical id and core id of each CPU */
- cpup = phba->sli4_hba.cpu_map;
- for (cpu = 0; cpu < phba->sli4_hba.num_present_cpu; cpu++) {
-#ifdef CONFIG_X86
- cpuinfo = &cpu_data(cpu);
- cpup->phys_id = cpuinfo->phys_proc_id;
- cpup->core_id = cpuinfo->cpu_core_id;
-#else
- /* No distinction between CPUs for other platforms */
- cpup->phys_id = 0;
- cpup->core_id = 0;
-#endif
-
- lpfc_printf_log(phba, KERN_INFO, LOG_INIT,
- "3328 CPU physid %d coreid %d\n",
- cpup->phys_id, cpup->core_id);
-
- if (cpup->phys_id > max_phys_id)
- max_phys_id = cpup->phys_id;
- if (cpup->phys_id < min_phys_id)
- min_phys_id = cpup->phys_id;
- cpup++;
- }
-
- phys_id = min_phys_id;
- /* Now associate the HBA vectors with specific CPUs */
- for (idx = 0; idx < vectors; idx++) {
- cpup = phba->sli4_hba.cpu_map;
- cpu = lpfc_find_next_cpu(phba, phys_id);
- if (cpu == LPFC_VECTOR_MAP_EMPTY) {
-
- /* Try for all phys_id's */
- for (i = 1; i < max_phys_id; i++) {
- phys_id++;
- if (phys_id > max_phys_id)
- phys_id = min_phys_id;
- cpu = lpfc_find_next_cpu(phba, phys_id);
- if (cpu == LPFC_VECTOR_MAP_EMPTY)
- continue;
- goto found;
- }
-
- /* Use round robin for scheduling */
- phba->cfg_fcp_io_sched = LPFC_FCP_SCHED_ROUND_ROBIN;
- chan = 0;
- cpup = phba->sli4_hba.cpu_map;
- for (i = 0; i < phba->sli4_hba.num_present_cpu; i++) {
- cpup->channel_id = chan;
- cpup++;
- chan++;
- if (chan >= phba->cfg_fcp_io_channel)
- chan = 0;
- }
-
- lpfc_printf_log(phba, KERN_ERR, LOG_INIT,
- "3329 Cannot set affinity:"
- "Error mapping vector %d (%d)\n",
- idx, vectors);
- return 0;
- }
-found:
- cpup += cpu;
- if (phba->cfg_fcp_cpu_map == LPFC_DRIVER_CPU_MAP)
- lpfc_used_cpu[cpu] = phys_id;
-
- /* Associate vector with selected CPU */
- cpup->irq = pci_irq_vector(phba->pcidev, idx);
-
- /* Associate IO channel with selected CPU */
- cpup->channel_id = idx;
- num_io_channel++;
-
- if (first_cpu == LPFC_VECTOR_MAP_EMPTY)
- first_cpu = cpu;
-
- /* Now affinitize to the selected CPU */
- i = irq_set_affinity_hint(pci_irq_vector(phba->pcidev, idx),
- get_cpu_mask(cpu));
-
- lpfc_printf_log(phba, KERN_INFO, LOG_INIT,
- "3330 Set Affinity: CPU %d channel %d "
- "irq %d (%x)\n",
- cpu, cpup->channel_id,
- pci_irq_vector(phba->pcidev, idx), i);
-
- /* Spread vector mapping across multple physical CPU nodes */
- phys_id++;
- if (phys_id > max_phys_id)
- phys_id = min_phys_id;
- }
-
- /*
- * Finally fill in the IO channel for any remaining CPUs.
- * At this point, all IO channels have been assigned to a specific
- * MSIx vector, mapped to a specific CPU.
- * Base the remaining IO channel assigned, to IO channels already
- * assigned to other CPUs on the same phys_id.
- */
- for (i = min_phys_id; i <= max_phys_id; i++) {
- /*
- * If there are no io channels already mapped to
- * this phys_id, just round robin thru the io_channels.
- * Setup chann[] for round robin.
- */
- for (idx = 0; idx < phba->cfg_fcp_io_channel; idx++)
- chann[idx] = idx;
-
- saved_chann = 0;
- used_chann = 0;
-
- /*
- * First build a list of IO channels already assigned
- * to this phys_id before reassigning the same IO
- * channels to the remaining CPUs.
- */
- cpup = phba->sli4_hba.cpu_map;
- cpu = first_cpu;
- cpup += cpu;
- for (idx = 0; idx < phba->sli4_hba.num_present_cpu;
- idx++) {
- if (cpup->phys_id == i) {
- /*
- * Save any IO channels that are
- * already mapped to this phys_id.
- */
- if (cpup->irq != LPFC_VECTOR_MAP_EMPTY) {
- if (saved_chann <=
- LPFC_FCP_IO_CHAN_MAX) {
- chann[saved_chann] =
- cpup->channel_id;
- saved_chann++;
- }
- goto out;
- }
-
- /* See if we are using round-robin */
- if (saved_chann == 0)
- saved_chann =
- phba->cfg_fcp_io_channel;
+ for (i = 0; i < vectors; i++) {
+ mask = pci_irq_get_affinity(phba->pcidev, i);
+ if (!mask)
+ return -EINVAL;
- /* Associate next IO channel with CPU */
- cpup->channel_id = chann[used_chann];
- num_io_channel++;
- used_chann++;
- if (used_chann == saved_chann)
- used_chann = 0;
-
- lpfc_printf_log(phba, KERN_INFO, LOG_INIT,
- "3331 Set IO_CHANN "
- "CPU %d channel %d\n",
- idx, cpup->channel_id);
- }
-out:
- cpu++;
- if (cpu >= phba->sli4_hba.num_present_cpu) {
- cpup = phba->sli4_hba.cpu_map;
- cpu = 0;
- } else {
- cpup++;
- }
- }
+ for_each_cpu(cpu, mask)
+ phba->sli4_hba.channel_map[cpu] = i;
}
- if (phba->sli4_hba.num_online_cpu != phba->sli4_hba.num_present_cpu) {
- cpup = phba->sli4_hba.cpu_map;
- for (idx = 0; idx < phba->sli4_hba.num_present_cpu; idx++) {
- if (cpup->channel_id == LPFC_VECTOR_MAP_EMPTY) {
- cpup->channel_id = 0;
- num_io_channel++;
-
- lpfc_printf_log(phba, KERN_INFO, LOG_INIT,
- "3332 Assign IO_CHANN "
- "CPU %d channel %d\n",
- idx, cpup->channel_id);
- }
- cpup++;
- }
- }
-
- /* Sanity check */
- if (num_io_channel != phba->sli4_hba.num_present_cpu)
- lpfc_printf_log(phba, KERN_ERR, LOG_INIT,
- "3333 Set affinity mismatch:"
- "%d chann != %d cpus: %d vectors\n",
- num_io_channel, phba->sli4_hba.num_present_cpu,
- vectors);
-
- /* Enable using cpu affinity for scheduling */
- phba->cfg_fcp_io_sched = LPFC_FCP_SCHED_BY_CPU;
return 1;
}
-
/**
* lpfc_sli4_enable_msix - Enable MSI-X interrupt mode to SLI-4 device
* @phba: pointer to lpfc hba data structure.
@@ -8986,13 +8692,17 @@ static int
lpfc_sli4_enable_msix(struct lpfc_hba *phba)
{
int vectors, rc, index;
+ struct irq_affinity desc = { 0, };
/* Set up MSI-X multi-message vectors */
vectors = phba->cfg_fcp_io_channel;
- if (phba->cfg_fof)
+ if (phba->cfg_fof) {
vectors++;
+ desc.post_vectors++;
+ }
- rc = pci_alloc_irq_vectors(phba->pcidev, 2, vectors, PCI_IRQ_MSIX);
+ rc = pci_alloc_irq_vectors_affinity(phba->pcidev, 2, vectors,
+ PCI_IRQ_MSIX | PCI_IRQ_AFFINITY, &desc);
if (rc < 0) {
lpfc_printf_log(phba, KERN_INFO, LOG_INIT,
"0484 PCI enable MSI-X failed (%d)\n", rc);
@@ -9046,10 +8756,8 @@ lpfc_sli4_enable_msix(struct lpfc_hba *phba)
cfg_fail_out:
/* free the irq already requested */
for (--index; index >= 0; index--) {
- int irq = pci_irq_vector(phba->pcidev, index);
-
- irq_set_affinity_hint(irq, NULL);
- free_irq(irq, &phba->sli4_hba.fcp_eq_hdl[index]);
+ free_irq(pci_irq_vector(phba->pcidev, index),
+ &phba->sli4_hba.fcp_eq_hdl[index]);
}
/* Unconfigure MSI-X capability structure */
@@ -9199,10 +8907,8 @@ lpfc_sli4_disable_intr(struct lpfc_hba *phba)
/* Free up MSI-X multi-message vectors */
for (index = 0; index < phba->cfg_fcp_io_channel; index++) {
- int irq = pci_irq_vector(phba->pcidev, index);
-
- irq_set_affinity_hint(irq, NULL);
- free_irq(irq, &phba->sli4_hba.fcp_eq_hdl[index]);
+ free_irq(pci_irq_vector(phba->pcidev, index),
+ &phba->sli4_hba.fcp_eq_hdl[index]);
}
if (phba->cfg_fof)
@@ -11345,7 +11051,6 @@ static struct miscdevice lpfc_mgmt_dev = {
static int __init
lpfc_init(void)
{
- int cpu;
int error = 0;
printk(LPFC_MODULE_DESC "\n");
@@ -11369,12 +11074,6 @@ lpfc_init(void)
return -ENOMEM;
}
- /* Initialize in case vector mapping is needed */
- lpfc_used_cpu = NULL;
- lpfc_present_cpu = 0;
- for_each_present_cpu(cpu)
- lpfc_present_cpu++;
-
error = pci_register_driver(&lpfc_driver);
if (error) {
fc_release_transport(lpfc_transport_template);
@@ -11411,7 +11110,6 @@ lpfc_exit(void)
(1L << _dump_buf_dif_order), _dump_buf_dif);
free_pages((unsigned long)_dump_buf_dif, _dump_buf_dif_order);
}
- kfree(lpfc_used_cpu);
idr_destroy(&lpfc_hba_index);
}
diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c
index 1b0ef79..f28b4e6 100644
--- a/drivers/scsi/lpfc/lpfc_scsi.c
+++ b/drivers/scsi/lpfc/lpfc_scsi.c
@@ -25,6 +25,7 @@
#include <linux/delay.h>
#include <asm/unaligned.h>
#include <linux/crc-t10dif.h>
+#include <linux/blk-mq-pci.h>
#include <net/checksum.h>
#include <scsi/scsi.h>
@@ -3875,33 +3876,16 @@ int lpfc_sli4_scmd_to_wqidx_distr(struct lpfc_hba *phba,
struct lpfc_scsi_buf *lpfc_cmd)
{
struct scsi_cmnd *cmnd = lpfc_cmd->pCmd;
- struct lpfc_vector_map_info *cpup;
- int chann, cpu;
- uint32_t tag;
- uint16_t hwq;
if (cmnd && shost_use_blk_mq(cmnd->device->host)) {
- tag = blk_mq_unique_tag(cmnd->request);
- hwq = blk_mq_unique_tag_to_hwq(tag);
+ u32 tag = blk_mq_unique_tag(cmnd->request);
- return hwq;
- }
-
- if (phba->cfg_fcp_io_sched == LPFC_FCP_SCHED_BY_CPU
- && phba->cfg_fcp_io_channel > 1) {
- cpu = smp_processor_id();
- if (cpu < phba->sli4_hba.num_present_cpu) {
- cpup = phba->sli4_hba.cpu_map;
- cpup += cpu;
- return cpup->channel_id;
- }
+ return blk_mq_unique_tag_to_hwq(tag);
+ } else {
+ return phba->sli4_hba.channel_map[raw_smp_processor_id()];
}
- chann = atomic_add_return(1, &phba->fcp_qidx);
- chann = (chann % phba->cfg_fcp_io_channel);
- return chann;
}
-
/**
* lpfc_scsi_cmd_iocb_cmpl - Scsi cmnd IOCB completion routine
* @phba: The Hba for which this call is being executed.
@@ -5569,6 +5553,13 @@ lpfc_slave_destroy(struct scsi_device *sdev)
return;
}
+static int lpfc_map_queues(struct Scsi_Host *shost)
+{
+ struct lpfc_vport *vport = (struct lpfc_vport *)shost->hostdata;
+
+ return blk_mq_pci_map_queues(&shost->tag_set, vport->phba->pcidev);
+}
+
/**
* lpfc_create_device_data - creates and initializes device data structure for OAS
* @pha: Pointer to host bus adapter structure.
@@ -5935,6 +5926,7 @@ struct scsi_host_template lpfc_template = {
.slave_configure = lpfc_slave_configure,
.slave_destroy = lpfc_slave_destroy,
.scan_finished = lpfc_scan_finished,
+ .map_queues = lpfc_map_queues,
.this_id = -1,
.sg_tablesize = LPFC_DEFAULT_SG_SEG_CNT,
.cmd_per_lun = LPFC_CMD_PER_LUN,
@@ -5960,6 +5952,7 @@ struct scsi_host_template lpfc_vport_template = {
.slave_configure = lpfc_slave_configure,
.slave_destroy = lpfc_slave_destroy,
.scan_finished = lpfc_scan_finished,
+ .map_queues = lpfc_map_queues,
.this_id = -1,
.sg_tablesize = LPFC_DEFAULT_SG_SEG_CNT,
.cmd_per_lun = LPFC_CMD_PER_LUN,
diff --git a/drivers/scsi/lpfc/lpfc_sli4.h b/drivers/scsi/lpfc/lpfc_sli4.h
index dfbb25e..a5bbce8 100644
--- a/drivers/scsi/lpfc/lpfc_sli4.h
+++ b/drivers/scsi/lpfc/lpfc_sli4.h
@@ -449,15 +449,6 @@ struct lpfc_sli4_lnk_info {
LPFC_FOF_IO_CHAN_NUM)
#define LPFC_SLI4_HANDLER_NAME_SZ 16
-/* Used for IRQ vector to CPU mapping */
-struct lpfc_vector_map_info {
- uint16_t phys_id;
- uint16_t core_id;
- uint16_t irq;
- uint16_t channel_id;
-};
-#define LPFC_VECTOR_MAP_EMPTY 0xffff
-
/* SLI4 HBA data structure entries */
struct lpfc_sli4_hba {
void __iomem *conf_regs_memmap_p; /* Kernel memory mapped address for
@@ -605,11 +596,8 @@ struct lpfc_sli4_hba {
spinlock_t abts_sgl_list_lock; /* list of aborted els IOs */
uint32_t physical_port;
- /* CPU to vector mapping information */
- struct lpfc_vector_map_info *cpu_map;
- uint16_t num_online_cpu;
- uint16_t num_present_cpu;
- uint16_t curr_disp_cpu;
+ /* CPU to vector mapping information (for the non blk-mq case) */
+ unsigned int *channel_map;
};
enum lpfc_sge_type {
--
2.1.4
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: use PCI layer IRQ affinity in lpfc
2016-11-17 15:14 use PCI layer IRQ affinity in lpfc Christoph Hellwig
2016-11-17 15:14 ` [PATCH 1/2] lpfc: use pci_irq_alloc_vectors and pci_irq_free_vectors Christoph Hellwig
2016-11-17 15:14 ` [PATCH 2/2] lpfc: use PCI-layer irq affinity handling Christoph Hellwig
@ 2016-11-18 13:13 ` Johannes Thumshirn
2016-11-18 13:22 ` Hannes Reinecke
2016-11-18 13:22 ` Christoph Hellwig
2016-11-18 16:21 ` James Smart
3 siblings, 2 replies; 11+ messages in thread
From: Johannes Thumshirn @ 2016-11-18 13:13 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: james.smart, hare, linux-scsi
Hi Christoph,
On Thu, Nov 17, 2016 at 04:14:51PM +0100, Christoph Hellwig wrote:
> This series has two patches: the first is a simple conversion of
> lpfc to use pci_alloc_irq_vectors. The second is more interesting,
> and makes use of the PCI_IRQ_AFFINITY option to use the core interrupt
> affinity assignment that takes nodes into account and can be easily
> queried. It also ensures we propagate this information to blk-mq
> to make sure the block layer queues are properly aligned to the
> interrupt vectors.
>
> Note that these patches require core IRQ changes from a stable
> branch in the tip tree to be pulled in first:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/for-block
>
> Also be aware that I don't have any lpfc hardware to actually test these.
This is what /proc/interrupts looks like after booting from the lpfc HBA,
with your patches:
ettrick:~ # grep lpfc /proc/interrupts
44: 2056 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 5242880-edge lpfc
46: 2186 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 5244928-edge lpfc
48: 69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6815744-edge lpfc:sp
49: 2060 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6815745-edge lpfc:fp
51: 64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6817792-edge lpfc:sp
52: 1074 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6817793-edge lpfc:fp
ettrick:~ # for irq in 44 46 48 49 51 52; do echo -n "$irq: "; \
> cat /proc/irq/$irq/smp_affinity; done
44: 55555555
46: 55555555
48: 55555555
49: 55555555
51: 55555555
52: 55555555
ettrick:~ #
Anything else you want me to look at?
Apart from that, for the whole series:
Tested-by: Johannes Thumshirn <jthumshirn@suse.de>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
--
Johannes Thumshirn Storage
jthumshirn@suse.de +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: use PCI layer IRQ affinity in lpfc
2016-11-18 13:13 ` use PCI layer IRQ affinity in lpfc Johannes Thumshirn
@ 2016-11-18 13:22 ` Hannes Reinecke
2016-11-18 13:22 ` Christoph Hellwig
1 sibling, 0 replies; 11+ messages in thread
From: Hannes Reinecke @ 2016-11-18 13:22 UTC (permalink / raw)
To: Johannes Thumshirn, Christoph Hellwig; +Cc: james.smart, linux-scsi
On 11/18/2016 02:13 PM, Johannes Thumshirn wrote:
> Hi Christoph,
>
> On Thu, Nov 17, 2016 at 04:14:51PM +0100, Christoph Hellwig wrote:
>> This series has two patches: the first is a simple conversion of
>> lpfc to use pci_alloc_irq_vectors. The second is more interesting,
>> and makes use of the PCI_IRQ_AFFINITY option to use the core interrupt
>> affinity assignment that takes nodes into account and can be easily
>> queried. It also ensures we propagate this information to blk-mq
>> to make sure the block layer queues are properly aligned to the
>> interrupt vectors.
>>
>> Note that these patches require core IRQ changes from a stable
>> branch in the tip tree to be pulled in first:
>>
>> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/for-block
>>
>> Also be aware that I don't have any lpfc hardware to actually test these.
>
> This is what /proc/interrupts looks like after booting from the lpfc HBA,
> with your patches:
>
> ettrick:~ # grep lpfc /proc/interrupts
> 44: 2056 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 5242880-edge lpfc
> 46: 2186 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 5244928-edge lpfc
> 48: 69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6815744-edge lpfc:sp
> 49: 2060 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6815745-edge lpfc:fp
> 51: 64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6817792-edge lpfc:sp
> 52: 1074 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6817793-edge lpfc:fp
To clarify: this machine has 4 lpfc HBAs, two Zephyr-X (having just one
MSI interrupt each) and two Saturn-X (having one fastpath and one
slowpath each).
So it's not surprising all ending up on the same CPU.
Cheers,
Hannes
--
Dr. Hannes Reinecke Teamlead Storage & Networking
hare@suse.de +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: use PCI layer IRQ affinity in lpfc
2016-11-18 13:13 ` use PCI layer IRQ affinity in lpfc Johannes Thumshirn
2016-11-18 13:22 ` Hannes Reinecke
@ 2016-11-18 13:22 ` Christoph Hellwig
2016-11-18 13:25 ` Johannes Thumshirn
2016-11-18 14:20 ` Johannes Thumshirn
1 sibling, 2 replies; 11+ messages in thread
From: Christoph Hellwig @ 2016-11-18 13:22 UTC (permalink / raw)
To: Johannes Thumshirn; +Cc: Christoph Hellwig, james.smart, hare, linux-scsi
On Fri, Nov 18, 2016 at 02:13:12PM +0100, Johannes Thumshirn wrote:
> This is what /proc/interrupts looks like after booting from the lpfc HBA,
> with your patches:
>
> ettrick:~ # grep lpfc /proc/interrupts
> 44: 2056 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 5242880-edge lpfc
> 46: 2186 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 5244928-edge lpfc
> 48: 69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6815744-edge lpfc:sp
> 49: 2060 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6815745-edge lpfc:fp
> 51: 64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6817792-edge lpfc:sp
> 52: 1074 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6817793-edge lpfc:fp
> ettrick:~ # for irq in 44 46 48 49 51 52; do echo -n "$irq: "; \
> > cat /proc/irq/$irq/smp_affinity; done
> 44: 55555555
> 46: 55555555
> 48: 55555555
> 49: 55555555
> 51: 55555555
> 52: 55555555
> ettrick:~ #
>
> Anything else you want me to look at?
Looks like you have non SLI-4 devices, which doesn't support
multiple queues, so patch 2 shouldn't have made a difference anyway.
But even with an SLI-4 device we'd need some actual I/O from different
CPUs to it to see how the interrupts were spread.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: use PCI layer IRQ affinity in lpfc
2016-11-18 13:22 ` Christoph Hellwig
@ 2016-11-18 13:25 ` Johannes Thumshirn
2016-11-18 14:20 ` Johannes Thumshirn
1 sibling, 0 replies; 11+ messages in thread
From: Johannes Thumshirn @ 2016-11-18 13:25 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Christoph Hellwig, james.smart, hare, linux-scsi
On Fri, Nov 18, 2016 at 05:22:11AM -0800, Christoph Hellwig wrote:
> On Fri, Nov 18, 2016 at 02:13:12PM +0100, Johannes Thumshirn wrote:
> > This is what /proc/interrupts looks like after booting from the lpfc HBA,
> > with your patches:
> >
> > ettrick:~ # grep lpfc /proc/interrupts
> > 44: 2056 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 5242880-edge lpfc
> > 46: 2186 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 5244928-edge lpfc
> > 48: 69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6815744-edge lpfc:sp
> > 49: 2060 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6815745-edge lpfc:fp
> > 51: 64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6817792-edge lpfc:sp
> > 52: 1074 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6817793-edge lpfc:fp
> > ettrick:~ # for irq in 44 46 48 49 51 52; do echo -n "$irq: "; \
> > > cat /proc/irq/$irq/smp_affinity; done
> > 44: 55555555
> > 46: 55555555
> > 48: 55555555
> > 49: 55555555
> > 51: 55555555
> > 52: 55555555
> > ettrick:~ #
> >
> > Anything else you want me to look at?
>
> Looks like you have non SLI-4 devices, which doesn't support
> multiple queues, so patch 2 shouldn't have made a difference anyway.
We've found a host with SLI-4, I'll check this one as well.
>
> But even with an SLI-4 device we'd need some actual I/O from different
> CPUs to it to see how the interrupts were spread.
--
Johannes Thumshirn Storage
jthumshirn@suse.de +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: use PCI layer IRQ affinity in lpfc
2016-11-18 13:22 ` Christoph Hellwig
2016-11-18 13:25 ` Johannes Thumshirn
@ 2016-11-18 14:20 ` Johannes Thumshirn
1 sibling, 0 replies; 11+ messages in thread
From: Johannes Thumshirn @ 2016-11-18 14:20 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Christoph Hellwig, james.smart, hare, linux-scsi
On Fri, Nov 18, 2016 at 05:22:11AM -0800, Christoph Hellwig wrote:
> On Fri, Nov 18, 2016 at 02:13:12PM +0100, Johannes Thumshirn wrote:
> > This is what /proc/interrupts looks like after booting from the lpfc HBA,
> > with your patches:
> >
> > ettrick:~ # grep lpfc /proc/interrupts
> > 44: 2056 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 5242880-edge lpfc
> > 46: 2186 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 5244928-edge lpfc
> > 48: 69 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6815744-edge lpfc:sp
> > 49: 2060 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6815745-edge lpfc:fp
> > 51: 64 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6817792-edge lpfc:sp
> > 52: 1074 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI 6817793-edge lpfc:fp
> > ettrick:~ # for irq in 44 46 48 49 51 52; do echo -n "$irq: "; \
> > > cat /proc/irq/$irq/smp_affinity; done
> > 44: 55555555
> > 46: 55555555
> > 48: 55555555
> > 49: 55555555
> > 51: 55555555
> > 52: 55555555
> > ettrick:~ #
> >
> > Anything else you want me to look at?
>
> Looks like you have non SLI-4 devices, which doesn't support
> multiple queues, so patch 2 shouldn't have made a difference anyway.
>
> But even with an SLI-4 device we'd need some actual I/O from different
> CPUs to it to see how the interrupts were spread.
Hmmm
c79:/suse/jthumshirn # fio --rw=randread --name=test --direct=1 \
> --size=1G --time_based --runtime=30s --filename=/dev/sdk \
> --group_reporting --max-jobs=24 --ioengine=libaio --numjobs=24 \
> --iodepth=128
[...]
c79:/suse/jthumshirn # grep lpfc /proc/interrupts
51: 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSI 3674112-edge lpfc:0
52: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSI 3674113-edge lpfc:1
53: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSI 3674114-edge lpfc:2
54: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSI 3674115-edge lpfc:3
56: 907298 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSI 3676160-edge lpfc:0
57: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSI 3676161-edge lpfc:1
58: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSI 3676162-edge lpfc:2
59: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSI 3676163-edge lpfc:3
c79:/suse/jthumshirn # for irq in 51 52 53 54 56 57 58 59; do echo -n "$irq: ";\
> cat /proc/irq/$irq/smp_affinity; done
51: 00000000,00007007
52: 00000000,00038038
53: 00000000,001c01c0
54: 00000000,00e00e00
56: 00000000,00007007
57: 00000000,00038038
58: 00000000,001c01c0
59: 00000000,00e00e00
c79:/suse/jthumshirn # lspci -s 07:00.2 -vvv | grep "Product Name"
Product Name: Emulex OneConnect OCe14102-UM 10GbE 2-Port SFP+ PCIe 3.0 Universal CNA, FCoE PF
c79:/suse/jthumshirn # lspci -s 07:00.2 -vvv | grep "MSI-X"
Capabilities: [48] MSI-X: Enable+ Count=8 Masked-
c79:/suse/jthumshirn # systemctl status irqbalance.service
● irqbalance.service - irqbalance daemon
Loaded: loaded (/usr/lib/systemd/system/irqbalance.service; disabled; vendor preset: enabled)
Active: inactive (dead)
Where did I mess up?
--
Johannes Thumshirn Storage
jthumshirn@suse.de +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: use PCI layer IRQ affinity in lpfc
2016-11-17 15:14 use PCI layer IRQ affinity in lpfc Christoph Hellwig
` (2 preceding siblings ...)
2016-11-18 13:13 ` use PCI layer IRQ affinity in lpfc Johannes Thumshirn
@ 2016-11-18 16:21 ` James Smart
2016-11-21 14:06 ` Christoph Hellwig
3 siblings, 1 reply; 11+ messages in thread
From: James Smart @ 2016-11-18 16:21 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: hare, linux-scsi
Hi Christoph,
We had already done this work in the larger lpfc patch that had been
posted for nvme work.
I'll look at separating out just the irq part and post a reply.
-- james
On 11/17/2016 7:14 AM, Christoph Hellwig wrote:
> This series has two patches: the first is a simple conversion of
> lpfc to use pci_alloc_irq_vectors. The second is more interesting,
> and makes use of the PCI_IRQ_AFFINITY option to use the core interrupt
> affinity assignment that takes nodes into account and can be easily
> queried. It also ensures we propagate this information to blk-mq
> to make sure the block layer queues are properly aligned to the
> interrupt vectors.
>
> Note that these patches require core IRQ changes from a stable
> branch in the tip tree to be pulled in first:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/for-block
>
> Also be aware that I don't have any lpfc hardware to actually test these.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: use PCI layer IRQ affinity in lpfc
2016-11-18 16:21 ` James Smart
@ 2016-11-21 14:06 ` Christoph Hellwig
2017-01-08 15:02 ` Christoph Hellwig
0 siblings, 1 reply; 11+ messages in thread
From: Christoph Hellwig @ 2016-11-21 14:06 UTC (permalink / raw)
To: James Smart; +Cc: Christoph Hellwig, hare, linux-scsi
On Fri, Nov 18, 2016 at 08:21:35AM -0800, James Smart wrote:
> Hi Christoph,
>
> We had already done this work in the larger lpfc patch that had been posted
> for nvme work.
Which one is that?
>
> I'll look at separating out just the irq part and post a reply.
Thanks.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: use PCI layer IRQ affinity in lpfc
2016-11-21 14:06 ` Christoph Hellwig
@ 2017-01-08 15:02 ` Christoph Hellwig
0 siblings, 0 replies; 11+ messages in thread
From: Christoph Hellwig @ 2017-01-08 15:02 UTC (permalink / raw)
To: James Smart; +Cc: Christoph Hellwig, hare, linux-scsi
On Mon, Nov 21, 2016 at 03:06:37PM +0100, Christoph Hellwig wrote:
> On Fri, Nov 18, 2016 at 08:21:35AM -0800, James Smart wrote:
> > Hi Christoph,
> >
> > We had already done this work in the larger lpfc patch that had been posted
> > for nvme work.
>
> Which one is that?
>
> >
> > I'll look at separating out just the irq part and post a reply.
>
> Thanks.
Any progress on this? Now that 4.10-rc has NVMe-FC core support it
would also be good to get the lpfc code out for review.
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2017-01-08 15:02 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-11-17 15:14 use PCI layer IRQ affinity in lpfc Christoph Hellwig
2016-11-17 15:14 ` [PATCH 1/2] lpfc: use pci_irq_alloc_vectors and pci_irq_free_vectors Christoph Hellwig
2016-11-17 15:14 ` [PATCH 2/2] lpfc: use PCI-layer irq affinity handling Christoph Hellwig
2016-11-18 13:13 ` use PCI layer IRQ affinity in lpfc Johannes Thumshirn
2016-11-18 13:22 ` Hannes Reinecke
2016-11-18 13:22 ` Christoph Hellwig
2016-11-18 13:25 ` Johannes Thumshirn
2016-11-18 14:20 ` Johannes Thumshirn
2016-11-18 16:21 ` James Smart
2016-11-21 14:06 ` Christoph Hellwig
2017-01-08 15:02 ` Christoph Hellwig
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.